1.8 KiB
1.8 KiB
Expanded Reachability Benchmark Fixtures
Module
__Tests
Status
IMPLEMENTED
Description
Expanded benchmark corpus with real CVE cases (WordPress, Rust/Axum, runc, Redis) and cross-platform test runners.
Implementation Details
- Reachability Test Corpus:
src/__Tests/reachability/-- multi-language reachability test corpus with labeled samples for PHP (WordPress), Rust (Axum), Go (runc), C (Redis), and other ecosystems. - Benchmark Datasets:
src/__Tests/__Datasets/-- ground-truth datasets for reachability benchmarks with labeled reachable/unreachable code paths. - Scanner Analyzers Benchmark:
src/Bench/StellaOps.Bench/Scanner.Analyzers/StellaOps.Bench.ScannerAnalyzers/-- benchmark runner that executes scanner analyzers against the reachability corpus and measures precision/recall. - Baseline Loader:
src/Bench/StellaOps.Bench/Scanner.Analyzers/StellaOps.Bench.ScannerAnalyzers/Baseline/BaselineLoader.cs-- loads ground-truth baseline data for benchmark comparison.
E2E Test Plan
- Run the reachability benchmark against the WordPress (PHP) corpus and verify precision and recall metrics are computed against the ground truth labels
- Run the benchmark against the Rust/Axum corpus and verify cross-language reachability analysis produces correct results
- Run the benchmark against the runc (Go) corpus and verify native code reachability paths are correctly identified
- Run the benchmark against the Redis (C) corpus and verify native memory access patterns are correctly analyzed
- Verify cross-platform compatibility: run the benchmark on both Linux and Windows and confirm results are identical
- Verify new fixture addition: add a new labeled sample to the corpus and confirm the benchmark runner includes it in the next evaluation