up
Some checks failed
Docs CI / lint-and-preview (push) Has been cancelled
AOC Guard CI / aoc-verify (push) Has been cancelled
AOC Guard CI / aoc-guard (push) Has been cancelled
Policy Lint & Smoke / policy-lint (push) Has been cancelled
SDK Publish & Sign / sdk-publish (push) Has been cancelled
sdk-generator-smoke / sdk-smoke (push) Has been cancelled

This commit is contained in:
StellaOps Bot
2025-11-27 08:51:10 +02:00
parent ea970ead2a
commit c34fb7256d
126 changed files with 18553 additions and 693 deletions

View File

@@ -64,15 +64,15 @@ python run_bench.py --sboms inputs/sboms/*.json --vex inputs/vex/*.json \
--config configs/scanners.json --shuffle --output results
# Reachability dataset (optional)
python run_reachability.py --graphs ../reachability/graphs/*.json \
--runtime ../reachability/runtime/*.ndjson.gz --output results-reach.csv
python run_reachability.py --graphs inputs/graphs/*.json \
--runtime inputs/runtime/*.ndjson --output results
```
Outputs are written to `results.csv` (determinism) and `results-reach.csv` (reachability stability) plus SHA manifests.
Outputs are written to `results.csv` (determinism), `results-reach.csv`/`results-reach.json` (reachability hashes), and manifests `inputs.sha256` + `dataset.sha256`.
## How to run (CI)
- Workflow `.gitea/workflows/bench-determinism.yml` calls `scripts/bench/determinism-run.sh`, which runs the harness with the bundled mock scanner and uploads `out/bench-determinism/**` (results, manifests, summary). Set `DET_EXTRA_INPUTS` to include frozen feed bundles in `inputs.sha256`.
- Workflow `.gitea/workflows/bench-determinism.yml` calls `scripts/bench/determinism-run.sh`, which runs the harness with the bundled mock scanner and uploads `out/bench-determinism/**` (results, manifests, summary). Set `DET_EXTRA_INPUTS` to include frozen feed bundles in `inputs.sha256`; optional `DET_REACH_GRAPHS`/`DET_REACH_RUNTIME` adds reachability hashes + `dataset.sha256`.
- Optional `bench:reachability` target (future) will replay reachability corpus, recompute graph hashes, and compare against expected `dataset.sha256`.
- CI fails when `determinism_rate` < `BENCH_DETERMINISM_THRESHOLD` (defaults to 0.95; set via env in the workflow).