stella-ops.org/git.stella-ops.org
1
0
Fork 0
Code Issues Pull Requests Actions 16 Releases Wiki Activity

up

Browse Source
This commit is contained in:
StellaOps Bot
2025-11-27 21:09:47 +02:00
parent e901d31acf
commit cfa2274d31
15 changed files with 123 additions and 47 deletions
Download Patch File Download Diff File Expand all files Collapse all files

7
docs/implplan/SPRINT_0512_0001_0001_bench.md
View File

@@ -32,7 +32,7 @@
| 5 | BENCH-POLICY-20-002 | BLOCKED | PREP-BENCH-POLICY-20-002-POLICY-DELTA-SAMPLE | Bench Guild · Policy Guild · Scheduler Guild | Add incremental run benchmark measuring delta evaluation vs full; capture SLA compliance. |
| 6 | BENCH-SIG-26-001 | BLOCKED | PREP-BENCH-SIG-26-001-REACHABILITY-SCHEMA-FIX | Bench Guild · Signals Guild | Develop benchmark for reachability scoring pipeline (facts/sec, latency, memory) using synthetic callgraphs/runtime batches. |
| 7 | BENCH-SIG-26-002 | BLOCKED | PREP-BENCH-SIG-26-002-BLOCKED-ON-26-001-OUTPU | Bench Guild · Policy Guild | Measure policy evaluation overhead with reachability cache hot/cold; ensure ≤8 ms p95 added latency. |
| 8 | BENCH-DETERMINISM-401-057 | DONE (2025-11-26) | Feed-freeze hash + SBOM/VEX bundle list from Sprint 0401. | Bench Guild · Signals Guild · Policy Guild (`bench/determinism`, `docs/benchmarks/signals/bench-determinism.md`) | Run cross-scanner determinism bench from 23-Nov advisory; publish determinism% and CVSS delta σ; CI workflow `bench-determinism` runs harness and uploads manifests/results. |
| 8 | BENCH-DETERMINISM-401-057 | DONE (2025-11-27) | Feed-freeze hash + SBOM/VEX bundle list from Sprint 0401. | Bench Guild · Signals Guild · Policy Guild (`bench/determinism`, `docs/benchmarks/signals/bench-determinism.md`) | Run cross-scanner determinism bench from 23-Nov advisory; publish determinism% and CVSS delta σ; CI workflow `bench-determinism` runs harness and uploads manifests/results; offline runner added. |
## Wave Coordination
- Single wave; benches sequenced by dataset availability. No parallel wave gating beyond Delivery Tracker dependencies.
@@ -76,6 +76,11 @@
## Execution Log
| Date (UTC) | Update | Owner |
| --- | --- | --- |
| 2025-11-27 | Added offline runner `Determinism/offline_run.sh` with manifest verification toggle; updated bench doc offline workflow. | Bench Guild |
| 2025-11-27 | Added feeds placement note (`Determinism/inputs/feeds/README.md`) and linked in bench offline workflow. | Bench Guild |
| 2025-11-27 | Added sample manifest `inputs/inputs.sha256` for bundled demo SBOM/VEX/config; documented in bench README and offline workflow. | Bench Guild |
| 2025-11-27 | Synced BENCH-DETERMINISM-401-057 status date to 2025-11-27 after offline runner/docs completion. | Bench Guild |
| 2025-11-27 | Added offline runner `src/Bench/StellaOps.Bench/Determinism/offline_run.sh` (defaults runs=10, threshold=0.95) for air-gapped determinism/reachability runs; mirrored in bench doc offline workflow. | Bench Guild |
| 2025-11-26 | Added optional reachability hashing path (DET_REACH_GRAPHS/DET_REACH_RUNTIME) to determinism run script; reachability helper `run_reachability.py` with sample graph/runtime fixtures and unit tests added. | Bench Guild |
| 2025-11-26 | Default runs raised to 10 per scanner/SBOM pair in harness and determinism-run wrapper to match 10x2 matrix requirement. | Bench Guild |
| 2025-11-26 | Added DET_EXTRA_INPUTS/DET_RUN_EXTRA_ARGS support to determinism run script to include frozen feeds in manifests; documented in scripts/bench/README.md. | Bench Guild |