Files
git.stella-ops.org/docs/features/unchecked/tests/expanded-reachability-benchmark-fixtures.md

1.8 KiB

Expanded Reachability Benchmark Fixtures

Module

__Tests

Status

IMPLEMENTED

Description

Expanded benchmark corpus with real CVE cases (WordPress, Rust/Axum, runc, Redis) and cross-platform test runners.

Implementation Details

  • Reachability Test Corpus: src/__Tests/reachability/ -- multi-language reachability test corpus with labeled samples for PHP (WordPress), Rust (Axum), Go (runc), C (Redis), and other ecosystems.
  • Benchmark Datasets: src/__Tests/__Datasets/ -- ground-truth datasets for reachability benchmarks with labeled reachable/unreachable code paths.
  • Scanner Analyzers Benchmark: src/Bench/StellaOps.Bench/Scanner.Analyzers/StellaOps.Bench.ScannerAnalyzers/ -- benchmark runner that executes scanner analyzers against the reachability corpus and measures precision/recall.
  • Baseline Loader: src/Bench/StellaOps.Bench/Scanner.Analyzers/StellaOps.Bench.ScannerAnalyzers/Baseline/BaselineLoader.cs -- loads ground-truth baseline data for benchmark comparison.

E2E Test Plan

  • Run the reachability benchmark against the WordPress (PHP) corpus and verify precision and recall metrics are computed against the ground truth labels
  • Run the benchmark against the Rust/Axum corpus and verify cross-language reachability analysis produces correct results
  • Run the benchmark against the runc (Go) corpus and verify native code reachability paths are correctly identified
  • Run the benchmark against the Redis (C) corpus and verify native memory access patterns are correctly analyzed
  • Verify cross-platform compatibility: run the benchmark on both Linux and Windows and confirm results are identical
  • Verify new fixture addition: add a new labeled sample to the corpus and confirm the benchmark runner includes it in the next evaluation