stella-ops.org/git.stella-ops.org
1
0
Fork 0
Code Issues Pull Requests Actions 16 Releases Wiki Activity

up
Some checks failed
Signals CI & Image / signals-ci (push) Has been cancelled
Details
Signals Reachability Scoring & Events / reachability-smoke (push) Has been cancelled
Details
Signals Reachability Scoring & Events / sign-and-upload (push) Has been cancelled
Details
Manifest Integrity / Validate Schema Integrity (push) Has been cancelled
Details
Manifest Integrity / Validate Contract Documents (push) Has been cancelled
Details
Manifest Integrity / Validate Pack Fixtures (push) Has been cancelled
Details
Manifest Integrity / Audit SHA256SUMS Files (push) Has been cancelled
Details
Manifest Integrity / Verify Merkle Roots (push) Has been cancelled
Details
Docs CI / lint-and-preview (push) Has been cancelled
Details

Browse Source
This commit is contained in:
StellaOps Bot
2025-12-12 09:35:37 +02:00
parent ce5ec9c158
commit efaf3cb789
238 changed files with 146274 additions and 5767 deletions
Download Patch File Download Diff File Expand all files Collapse all files

12
docs/benchmarks/policy/bench-policy-20-002-prep.md
View File

@@ -1,6 +1,6 @@
# Bench Prep — PREP-BENCH-POLICY-20-002 (Policy delta benchmark)
Status: **Ready for implementation** (2025-11-20)
Status: **Ready for execution** (2025-12-11)
Owners: Bench Guild · Policy Guild · Scheduler Guild
Scope: Provide deterministic inputs and harness expectations to measure delta policy evaluation vs full runs.
@@ -11,12 +11,12 @@ Scope: Provide deterministic inputs and harness expectations to measure delta po
## Dataset
- Baseline snapshot: `docs/samples/policy/policy-delta-baseline.ndjson`
- 5,000 records of `{ "tenant": "bench", "policyId": "pol-<0001..5000>", "package": "bench.pkg.<n>", "version": "1.0.<n>", "decision": "allow|deny", "factors": { ... } }`
- Deterministic ordering; SHA256 file saved as `policy-delta-baseline.ndjson.sha256`.
- Deterministic ordering; SHA256 `40ca9ee15065a9e16f51a259d3feec778203ab461db2af3bf196f5fcd9f0d590` (`policy-delta-baseline.ndjson.sha256`).
- Delta patch: `docs/samples/policy/policy-delta-changes.ndjson`
- 500 changes mixing updates/inserts/deletes (encoded with `op`: "upsert"|"delete").
- Sorted by `policyId` then `op` for deterministic replay.
- Sorted by `policyId` then `op` for deterministic replay; SHA256 `7f9d7f124830b9fe4d3f232b4cc7e2e728be2ef725e8a66606b9e95682bf6318` (`policy-delta-changes.ndjson.sha256`).
## Harness plan (to be built under `src/Bench/StellaOps.Bench.Policy`)
## Harness plan (implemented under `src/Bench/StellaOps.Bench/PolicyDelta/policy_delta_bench.py`)
- Run 1 (Full): load baseline snapshot, evaluate full policy set; record metrics.
- Run 2 (Delta): apply delta patch to in-memory store, run incremental evaluation; record metrics.
- Metrics captured to NDJSON per run:
@@ -31,5 +31,5 @@ Scope: Provide deterministic inputs and harness expectations to measure delta po
- Delta run shows reduced duration vs full run; metrics captured for both p95/p99 and throughput.
## Next steps
- Add sample files + hashes to `docs/samples/policy/` (can be generated with fixed seed).
- Implement harness CLI wrapper `dotnet run -- policy-delta --baseline <path> --delta <path> [--threads 1]` writing outputs to `out/bench/policy/` with `.sha256`.
- Harness CLI: `python src/Bench/StellaOps.Bench/PolicyDelta/policy_delta_bench.py --baseline docs/samples/policy/policy-delta-baseline.ndjson --delta docs/samples/policy/policy-delta-changes.ndjson --output src/Bench/StellaOps.Bench/PolicyDelta/results/policy-delta.ndjson --threads 1 --seed 20250101`.
- Results hashed at `src/Bench/StellaOps.Bench/PolicyDelta/results/policy-delta.ndjson.sha256`.