31 lines
2.1 KiB
Markdown
31 lines
2.1 KiB
Markdown
# Expanded Reachability Benchmark Fixtures
|
|
|
|
## Module
|
|
__Tests
|
|
|
|
## Status
|
|
VERIFIED
|
|
|
|
## Description
|
|
Expanded benchmark corpus with real CVE cases (WordPress, Rust/Axum, runc, Redis) and cross-platform test runners.
|
|
|
|
## Implementation Details
|
|
- **Reachability Test Corpus**: `src/__Tests/reachability/` -- multi-language reachability test corpus with labeled samples for PHP (WordPress), Rust (Axum), Go (runc), C (Redis), and other ecosystems.
|
|
- **Benchmark Datasets**: `src/__Tests/__Datasets/` -- ground-truth datasets for reachability benchmarks with labeled reachable/unreachable code paths.
|
|
- **Scanner Analyzers Benchmark**: `src/Bench/StellaOps.Bench/Scanner.Analyzers/StellaOps.Bench.ScannerAnalyzers/` -- benchmark runner that executes scanner analyzers against the reachability corpus and measures precision/recall.
|
|
- **Baseline Loader**: `src/Bench/StellaOps.Bench/Scanner.Analyzers/StellaOps.Bench.ScannerAnalyzers/Baseline/BaselineLoader.cs` -- loads ground-truth baseline data for benchmark comparison.
|
|
|
|
## E2E Test Plan
|
|
- [ ] Run the reachability benchmark against the WordPress (PHP) corpus and verify precision and recall metrics are computed against the ground truth labels
|
|
- [ ] Run the benchmark against the Rust/Axum corpus and verify cross-language reachability analysis produces correct results
|
|
- [ ] Run the benchmark against the runc (Go) corpus and verify native code reachability paths are correctly identified
|
|
- [ ] Run the benchmark against the Redis (C) corpus and verify native memory access patterns are correctly analyzed
|
|
- [ ] Verify cross-platform compatibility: run the benchmark on both Linux and Windows and confirm results are identical
|
|
- [ ] Verify new fixture addition: add a new labeled sample to the corpus and confirm the benchmark runner includes it in the next evaluation
|
|
|
|
## Verification
|
|
- Verified on 2026-02-13 via `run-001`.
|
|
- Tier 0: Source files confirmed present on disk.
|
|
- Tier 1: `dotnet build` passed (0 errors); 266/266 tests passed across Chaos.Tests, Evidence.Tests, Replay.Tests, FixtureTests.
|
|
- Tier 2d: `docs/qa/feature-checks/runs/tests/expanded-reachability-benchmark-fixtures/run-001/tier2-integration-check.json`
|