stella-ops.org/git.stella-ops.org
1
0
Fork 0
Code Issues Pull Requests Actions 2 Releases Wiki Activity
Files
cf5b72974f923126944f4dcd47e85864ebb31a3b
git.stella-ops.org/docs/features/unchecked/scanner/tiered-scanner-precision.md

1.6 KiB
Raw Blame History

Tiered Scanner Precision (Imported/Executed/Tainted-Sink Tiers with PR-AUC Metrics)

Module

Scanner

Status

IMPLEMENTED

Description

Fidelity-aware analysis with tiered precision is implemented including benchmark corpus management, metrics calculation, fidelity endpoints, and reproducibility verification.

Implementation Details

  • Fidelity-Aware Analyzer:
    • src/Scanner/__Libraries/StellaOps.Scanner.Orchestration/Fidelity/FidelityAwareAnalyzer.cs - FidelityAwareAnalyzer adjusting analysis precision based on configured fidelity tier (Imported, Executed, Tainted-Sink) with corresponding precision/recall tradeoffs
  • Fidelity API:
    • src/Scanner/StellaOps.Scanner.WebService/Endpoints/FidelityEndpoints.cs - FidelityEndpoints REST API for querying and configuring fidelity tiers and viewing precision metrics
  • Tests:
    • src/Scanner/__Tests/StellaOps.Scanner.Core.Tests/Fidelity/FidelityAwareAnalyzerTests.cs - Unit tests for fidelity-aware analysis including benchmark corpus and PR-AUC metric validation

E2E Test Plan

  • Configure the scanner at "Imported" fidelity tier and verify it produces results with high recall but lower precision
  • Configure "Executed" fidelity tier and verify results include only dependencies confirmed as loaded/executed
  • Configure "Tainted-Sink" fidelity tier and verify results are restricted to vulnerabilities with confirmed data flow to sinks
  • Verify fidelity endpoints return precision metrics (PR-AUC scores) for each tier against benchmark corpus
  • Verify fidelity tier selection is deterministic and reproducible across identical scan inputs