# Vendor comparison / scanner parity tracking ## Module Bench ## Status VERIFIED ## Description Scanner analyzer benchmark parity tracking capabilities are present through benchmark reports and metric exports. Fresh behavioral verification confirmed parity-report fields are emitted in benchmark JSON output and CLI error semantics are enforced for invalid configuration. ## What's Implemented - **Scanner Analyzers Benchmark**: `src/Bench/StellaOps.Bench/Scanner.Analyzers/StellaOps.Bench.ScannerAnalyzers/` -- benchmark harness evaluating analyzer scenarios and recording metrics. - **Baseline Loader**: `src/Bench/StellaOps.Bench/Scanner.Analyzers/StellaOps.Bench.ScannerAnalyzers/Baseline/BaselineLoader.cs` -- loads baseline data for benchmark comparisons. - **Baseline Entry**: `src/Bench/StellaOps.Bench/Scanner.Analyzers/StellaOps.Bench.ScannerAnalyzers/Baseline/BaselineEntry.cs` -- baseline model. - **Benchmark Scenario Report**: `src/Bench/StellaOps.Bench/Scanner.Analyzers/StellaOps.Bench.ScannerAnalyzers/Reporting/BenchmarkScenarioReport.cs` -- per-scenario report model including regression and parity fields. - **Benchmark JSON Writer**: `src/Bench/StellaOps.Bench/Scanner.Analyzers/StellaOps.Bench.ScannerAnalyzers/Reporting/BenchmarkJsonWriter.cs` -- JSON report writer. - **Prometheus Writer**: `src/Bench/StellaOps.Bench/Scanner.Analyzers/StellaOps.Bench.ScannerAnalyzers/Reporting/PrometheusWriter.cs` -- Prometheus metrics exporter. - **Vendor Parity Analyzer**: `src/Bench/StellaOps.Bench/Scanner.Analyzers/StellaOps.Bench.ScannerAnalyzers/Reporting/VendorParityAnalyzer.cs` -- computes vendor parity projections where vendor fixtures are available. ## E2E Test Plan - [x] Run scanner-analyzers benchmark harness and verify JSON/Prometheus/CSV outputs are generated. - [x] Validate benchmark JSON output contains `vendorParity` fields in scenario reports. - [x] Verify baseline/regression metadata is emitted in benchmark JSON. - [x] Verify negative-path behavior with missing config returns non-zero exit code. ## Verification - Verified on 2026-02-11 via FLOW Tier 0/1/2 replay in `run-001`. - Tier 0: `docs/qa/feature-checks/runs/bench/vendor-comparison-scanner-parity-tracking/run-001/tier0-source-check.json` - Tier 1: `docs/qa/feature-checks/runs/bench/vendor-comparison-scanner-parity-tracking/run-001/tier1-build-check.json` - Tier 2: `docs/qa/feature-checks/runs/bench/vendor-comparison-scanner-parity-tracking/run-001/tier2-integration-check.json`