35 lines
1.7 KiB
Markdown
35 lines
1.7 KiB
Markdown
# Golden Benchmark Fixtures (Core-10)
|
|
|
|
## Status
|
|
PARTIALLY_IMPLEMENTED
|
|
|
|
## Description
|
|
The advisory describes 10 golden reachability benchmark fixtures (C, Java, .NET, Python, container), but no pre-built fixture datasets were found in the source tree. The ReachGraph service infrastructure exists but the specific Core-10 fixture data files are not present.
|
|
|
|
## Module
|
|
Bench
|
|
|
|
## What's Implemented
|
|
- **Multi-runtime corpus**: `src/__Tests/reachability/corpus/` (5 runtimes: dotnet, go, java, python, rust)
|
|
- **Additional VEX corpus**: `src/tests/reachability/corpus/` (4 runtimes with OpenVEX files)
|
|
- **Expanded benchmarks**: `src/__Tests/reachability/fixtures/reachbench-2025-expanded/`
|
|
- **Patch oracles**: `src/__Tests/reachability/fixtures/patch-oracles/`
|
|
- **PoE fixtures**: `src/__Tests/reachability/PoE/Fixtures/`
|
|
- **Scoring golden corpus**: `src/__Tests/__Benchmarks/golden-corpus/` (VEX scenarios and severity levels)
|
|
- **Fixture harvester tool**: `src/__Tests/Tools/FixtureHarvester/SbomGoldenCommand.cs`
|
|
- **Corpus management scripts**: `src/__Tests/reachability/scripts/update_corpus_manifest.py`
|
|
- **Fixture tests**: `src/__Tests/reachability/StellaOps.Reachability.FixtureTests/`
|
|
|
|
## What's Missing
|
|
- The exact advisory-specified "Core-10" named fixture set (10 specific golden reachability benchmark fixtures covering C, Java, .NET, Python, container)
|
|
- Formal Core-10 naming convention and documentation
|
|
- C runtime reachability corpus (only dotnet, go, java, python, rust present)
|
|
|
|
## Implementation Plan
|
|
- Audit existing fixtures against the Core-10 specification from the advisory
|
|
- Add C runtime corpus if required
|
|
- Formalize Core-10 naming and documentation
|
|
|
|
## Source
|
|
- Feature matrix scan
|