stella-ops.org/git.stella-ops.org
1
0
Fork 0
Code Issues Pull Requests Actions 16 Releases Wiki Activity

up

Browse Source
This commit is contained in:
StellaOps Bot
2025-11-27 21:09:47 +02:00
parent e901d31acf
commit cfa2274d31
15 changed files with 123 additions and 47 deletions
Download Patch File Download Diff File Expand all files Collapse all files

9
docs/benchmarks/signals/bench-determinism.md
View File

@@ -68,7 +68,7 @@ python run_reachability.py --graphs inputs/graphs/*.json \
--runtime inputs/runtime/*.ndjson --output results
```
Outputs are written to `results.csv` (determinism), `results-reach.csv`/`results-reach.json` (reachability hashes), and manifests `inputs.sha256` + `dataset.sha256`.
Outputs are written to `results.csv` (determinism), `results-reach.csv`/`results-reach.json` (reachability hashes), and manifests `inputs.sha256` + `dataset.sha256` (if reachability). Feed bundle hashes live in the same manifest when provided via `DET_EXTRA_INPUTS`.
## How to run (CI)
@@ -78,10 +78,9 @@ Outputs are written to `results.csv` (determinism), `results-reach.csv`/`results
## Offline/air-gap workflow
1. Place feeds bundle, SBOMs, VEX, and reachability corpus under `offline/inputs/` with matching `inputs.sha256` and `dataset.sha256`.
2. Run `./offline_run.sh --inputs offline/inputs --outputs offline/results` to execute both benches without network.
3. Verify hashes: `sha256sum -c offline/inputs/inputs.sha256` and `sha256sum -c offline/inputs/dataset.sha256`.
4. Store outputs plus manifests in Offline Kit; include DSSE envelope if signing is enabled (`./sign_results.sh`).
1. Place feeds bundle (see `src/Bench/StellaOps.Bench/Determinism/inputs/feeds/README.md`), SBOMs, VEX, and optional reachability corpus under `offline/inputs/` with matching `inputs.sha256` and (if reachability) `dataset.sha256`. A sample `inputs/inputs.sha256` is provided for the bundled demo SBOM/VEX/config.
2. Run `./offline_run.sh --inputs offline/inputs --output offline/results` (script lives under `src/Bench/StellaOps.Bench/Determinism`) to execute benches without network (defaults: runs=10, threshold=0.95; manifest verification on). Use `--no-verify` to skip hash checks if manifests are absent.
3. Store outputs plus manifests in Offline Kit; include DSSE envelope if signing is enabled (`./sign_results.sh`).
## Notes
- Keep file ordering deterministic (lexicographic) when generating manifests.