# Sprint 0512 · Ops & Offline · Bench (190.G)

## Topic & Scope
- Build and capture performance benchmarks for graph, UI interactions, impact index, policy deltas, and reachability scoring to support offline/ops readiness.
- Target harnesses under `src/Bench/StellaOps.Bench` with reproducible datasets.
- **Working directory:** `src/Bench/StellaOps.Bench`.

## Dependencies & Concurrency
- Upstream data: graph fixtures (SAMPLES-GRAPH-24-003), reachability schema (Sprint 0400/0401), policy delta inputs.
- UI bench depends on BENCH-GRAPH-21-001/002 harness foundation.

## Documentation Prerequisites
- docs/07_HIGH_LEVEL_ARCHITECTURE.md
- docs/modules/platform/architecture-overview.md
- docs/modules/graph/architecture.md (for graph bench scenarios)
- docs/modules/signals/architecture.md (for reachability benches)
- docs/modules/policy/architecture.md

## Delivery Tracker
| # | Task ID | Status | Key dependency / next step | Owners | Task Definition |
| --- | --- | --- | --- | --- | --- |
| P1 | PREP-BENCH-GRAPH-21-001-NEED-GRAPH-BENCH-HARN | DONE (2025-11-20) | Prep doc at `docs/benchmarks/graph/bench-graph-21-001-prep.md`; awaits fixtures (SAMPLES-GRAPH-24-003). | Bench Guild · Graph Platform Guild | Need graph bench harness scaffolding (50k/100k nodes). <br><br> Document artefact/deliverable for BENCH-GRAPH-21-001 and publish location so downstream tasks can proceed. |
| P2 | PREP-BENCH-GRAPH-21-002-BLOCKED-ON-21-001-HAR | DONE (2025-11-20) | Due 2025-11-26 · Accountable: Bench Guild · UI Guild | Bench Guild · UI Guild | Prep artefact published at `docs/benchmarks/graph/bench-graph-21-002-prep.md` (Playwright UI bench plan leveraging 50k/100k fixtures; scenarios, metrics, determinism). |
| P3 | PREP-BENCH-IMPACT-16-001-IMPACT-INDEX-DATASET | DONE (2025-11-20) | Due 2025-11-26 · Accountable: Bench Guild · Scheduler Team | Bench Guild · Scheduler Team | Prep artefact published at `docs/benchmarks/impact/bench-impact-16-001-prep.md` (dataset shape, replay plan, deterministic metrics). |
| P4 | PREP-BENCH-POLICY-20-002-POLICY-DELTA-SAMPLE | DONE (2025-11-20) | Due 2025-11-26 · Accountable: Bench Guild · Policy Guild · Scheduler Guild | Bench Guild · Policy Guild · Scheduler Guild | Prep artefact published at `docs/benchmarks/policy/bench-policy-20-002-prep.md` (baseline + delta datasets, deterministic harness plan, metrics). |
| P5 | PREP-BENCH-SIG-26-001-REACHABILITY-SCHEMA-FIX | DONE (2025-11-20) | Prep doc at `docs/benchmarks/signals/bench-sig-26-001-prep.md`; awaits reachability schema hash. | Bench Guild · Signals Guild | Reachability schema/fixtures pending Sprint 0400/0401. <br><br> Document artefact/deliverable for BENCH-SIG-26-001 and publish location so downstream tasks can proceed. |
| P6 | PREP-BENCH-SIG-26-002-BLOCKED-ON-26-001-OUTPU | DONE (2025-11-20) | Prep doc at `docs/benchmarks/signals/bench-sig-26-002-prep.md`; depends on 26-001 datasets. | Bench Guild · Policy Guild | Blocked on 26-001 outputs. <br><br> Document artefact/deliverable for BENCH-SIG-26-002 and publish location so downstream tasks can proceed. |
| 1 | BENCH-GRAPH-21-001 | BLOCKED | PREP-BENCH-GRAPH-21-001-NEED-GRAPH-BENCH-HARN | Bench Guild · Graph Platform Guild | Build graph viewport/path benchmark harness (50k/100k nodes) measuring Graph API/Indexer latency, memory, and tile cache hit rates. |
| 2 | BENCH-GRAPH-21-002 | BLOCKED | PREP-BENCH-GRAPH-21-002-BLOCKED-ON-21-001-HAR | Bench Guild · UI Guild | Add headless UI load benchmark (Playwright) for graph canvas interactions to track render times and FPS budgets. |
| 3 | BENCH-GRAPH-24-002 | BLOCKED | Waiting for 50k/100k graph fixture (SAMPLES-GRAPH-24-003) | Bench Guild · UI Guild | Implement UI interaction benchmarks (filter/zoom/table operations) citing p95 latency; integrate with perf dashboards. |
| 4 | BENCH-IMPACT-16-001 | BLOCKED | PREP-BENCH-IMPACT-16-001-IMPACT-INDEX-DATASET | Bench Guild · Scheduler Team | ImpactIndex throughput bench (resolve 10k productKeys) + RAM profile. |
| 5 | BENCH-POLICY-20-002 | BLOCKED | PREP-BENCH-POLICY-20-002-POLICY-DELTA-SAMPLE | Bench Guild · Policy Guild · Scheduler Guild | Add incremental run benchmark measuring delta evaluation vs full; capture SLA compliance. |
| 6 | BENCH-SIG-26-001 | BLOCKED | PREP-BENCH-SIG-26-001-REACHABILITY-SCHEMA-FIX | Bench Guild · Signals Guild | Develop benchmark for reachability scoring pipeline (facts/sec, latency, memory) using synthetic callgraphs/runtime batches. |
| 7 | BENCH-SIG-26-002 | BLOCKED | PREP-BENCH-SIG-26-002-BLOCKED-ON-26-001-OUTPU | Bench Guild · Policy Guild | Measure policy evaluation overhead with reachability cache hot/cold; ensure ≤8 ms p95 added latency. |
| 8 | BENCH-DETERMINISM-401-057 | DONE (2025-11-26) | Feed-freeze hash + SBOM/VEX bundle list from Sprint 0401. | Bench Guild · Signals Guild · Policy Guild (`bench/determinism`, `docs/benchmarks/signals/bench-determinism.md`) | Run cross-scanner determinism bench from 23-Nov advisory; publish determinism% and CVSS delta σ; CI workflow `bench-determinism` runs harness and uploads manifests/results. |

## Wave Coordination
- Single wave; benches sequenced by dataset availability. No parallel wave gating beyond Delivery Tracker dependencies.

## Wave Detail Snapshots
- N/A (single wave). Add per-wave snapshots if additional waves are introduced.

## Interlocks
- Graph fixtures SAMPLES-GRAPH-24-003 delivery (Bench Guild ↔ Graph Platform Guild).
- Reachability schema alignment from Sprints 0400/0401 (Signals Guild ↔ Policy Guild).
- Policy delta dataset delivery (Policy Guild ↔ Scheduler Guild).

## Upcoming Checkpoints
- 2025-11-22 · Confirm availability of graph fixtures for BENCH-GRAPH-21-001/002/24-002. Owner: Bench Guild.
- 2025-11-23 · Escalate to Graph Platform Guild if SAMPLES-GRAPH-24-003 location still missing; confirm interim synthetic path (ACT-0512-04). Owner: Bench Guild.
- 2025-11-24 · Reachability schema alignment outcome to unblock BENCH-SIG-26-001. Owner: Signals Guild.
- 2025-11-26 · Decide impact index dataset for BENCH-IMPACT-16-001. Owner: Scheduler Team.

## Action Tracker
| Action ID | Status | Owner | Due (UTC) | Details |
| --- | --- | --- | --- | --- |
| ACT-0512-01 | PENDING | Bench Guild | 2025-11-22 | Confirm SAMPLES-GRAPH-24-003 fixtures availability and publish location for BENCH-GRAPH-21-001/002/24-002. |
| ACT-0512-02 | PENDING | Signals Guild | 2025-11-24 | Provide reachability schema hash/output to unblock BENCH-SIG-26-001/002. |
| ACT-0512-03 | PENDING | Scheduler Team | 2025-11-26 | Finalize impact index dataset selection and share deterministic replay bundle. |
| ACT-0512-04 | PENDING | Bench Guild | 2025-11-24 | Prepare interim synthetic 50k/100k graph fixture (documented in `samples/graph/fixtures-plan.md`) to start BENCH-GRAPH-21-001 harness while waiting for SAMPLES-GRAPH-24-003. |
| ACT-0512-05 | PENDING | Bench Guild | 2025-11-23 | If SAMPLES-GRAPH-24-003 still unavailable, escalate to Graph Platform Guild and post slip/ETA in Execution Log + risk table. |
| ACT-0512-06 | PENDING | Signals Guild | 2025-11-24 | If reachability schema hash slips past 2025-11-24, publish synthetic schema + sample batches in `docs/benchmarks/signals/bench-sig-26-001-prep.md` to unblock BENCH-SIG-26-001/002 harness scaffolding. |
| ACT-0512-07 | PENDING | Bench Guild · UI Guild | 2025-11-25 | Draft Playwright bench harness skeleton (headless, deterministic seeds, no network) reusing `bench-graph-21-002-prep` scenarios; commit once fixture source (real or synthetic) is bound. |

## Decisions & Risks
| Risk | Impact | Mitigation | Status | Owner | Due (UTC) |
| --- | --- | --- | --- | --- | --- |
| Graph fixtures SAMPLES-GRAPH-24-003 not delivered | Blocks BENCH-GRAPH-21-001/002/24-002; benches unstartable | Track via ACT-0512-01; ACT-0512-05 escalation if missed | At risk | Bench Guild | 2025-11-22 |
| Reachability schema hash pending from Sprint 0400/0401 | BENCH-SIG-26-001/002 remain blocked | ACT-0512-02 to deliver hash; ACT-0512-06 fallback synthetic set if delayed | Open | Signals Guild | 2025-11-24 |
| Impact index dataset undecided | BENCH-IMPACT-16-001 stalled; no reproducibility | ACT-0512-03 to finalize dataset; require deterministic replay bundle | Open | Scheduler Team | 2025-11-26 |
| UI harness blocked waiting for fixture binding | BENCH-GRAPH-21-002/24-002 cannot start scripting | ACT-0512-07 to draft harness skeleton with deterministic seeds; binds once fixture path set | Open | Bench Guild · UI Guild | 2025-11-25 |

- Graph fixture still blocked per `docs/implplan/SPRINT_0509_0001_0001_samples.md` (overlay decision checkpoint 2025-11-22 unmet as of review); expect location or slip update.
- Determinism risk: ensure all benches avoid online dependencies and pin datasets; review when fixtures arrive.

## Execution Log
| Date (UTC) | Update | Owner |
| --- | --- | --- |
| 2025-11-26 | Default runs raised to 10 per scanner/SBOM pair in harness and determinism-run wrapper to match 10x2 matrix requirement. | Bench Guild |
| 2025-11-26 | Added DET_EXTRA_INPUTS/DET_RUN_EXTRA_ARGS support to determinism run script to include frozen feeds in manifests; documented in scripts/bench/README.md. | Bench Guild |
| 2025-11-26 | Added scripts/bench/README.md documenting determinism-run wrapper and threshold env. | Bench Guild |
| 2025-11-26 | Bench CI workflow added (`.gitea/workflows/bench-determinism.yml`) with threshold gating via `BENCH_DETERMINISM_THRESHOLD`; run wrapper `scripts/bench/determinism-run.sh` uploads artifacts. | Bench Guild |
| 2025-11-26 | Added `scripts/bench/determinism-run.sh` and CI workflow `.gitea/workflows/bench-determinism.yml` to run/upload determinism artifacts. | Bench Guild |
| 2025-11-26 | Built determinism bench harness with mock scanner at `src/Bench/StellaOps.Bench/Determinism`, added sample SBOM/VEX inputs, generated `results/inputs.sha256` + `results.csv`, updated bench doc, and marked BENCH-DETERMINISM-401-057 DONE. Tests: `python -m unittest discover -s src/Bench/StellaOps.Bench/Determinism/tests -t src/Bench/StellaOps.Bench/Determinism`. | Bench Guild |
| 2025-11-22 | Added ACT-0512-07 and corresponding risk entry to have UI bench harness skeleton ready once fixtures bind; no status changes. | Project Mgmt |
| 2025-11-22 | Added ACT-0512-04 to build interim synthetic graph fixture so BENCH-GRAPH-21-001 can start while awaiting SAMPLES-GRAPH-24-003; no status changes. | Project Mgmt |
| 2025-11-22 | Added ACT-0512-05 escalation path (due 2025-11-23) if SAMPLES-GRAPH-24-003 remains unavailable; updated Upcoming Checkpoints accordingly. | Project Mgmt |
| 2025-11-22 | Added ACT-0512-06 fallback synthetic reachability schema/fixtures (due 2025-11-24) in case hash delivery slips; no status changes. | Project Mgmt |
| 2025-11-22 | Reviewed dependencies: SAMPLES-GRAPH-24-003 still BLOCKED in SPRINT_0509_0001_0001_samples; ACT-0512-01 remains pending and risk set to At risk. | Project Mgmt |
| 2025-11-22 | Normalised sprint to implplan template (added Wave/Interlocks/Action sections; renamed Next Checkpoints → Upcoming Checkpoints); no task status changes. | Project Mgmt |
| 2025-11-20 | Completed PREP-BENCH-GRAPH-21-002: published UI bench prep doc at `docs/benchmarks/graph/bench-graph-21-002-prep.md`; status set to DONE. | Implementer |
| 2025-11-20 | Completed PREP-BENCH-IMPACT-16-001: published impact index bench prep doc at `docs/benchmarks/impact/bench-impact-16-001-prep.md`; status set to DONE. | Implementer |
| 2025-11-20 | Completed PREP-BENCH-POLICY-20-002: published policy delta bench prep doc at `docs/benchmarks/policy/bench-policy-20-002-prep.md`; status set to DONE. | Implementer |
| 2025-11-20 | Published prep artefacts for PREP-BENCH-GRAPH-21-001, PREP-BENCH-SIG-26-001, and PREP-BENCH-SIG-26-002 under `docs/benchmarks/`; marked P1, P5, P6 DONE. | Implementer |
| 2025-11-19 | Trimmed trailing hyphen from PREP-BENCH-POLICY-20-002 Task ID to keep BENCH-POLICY-20-002 blocker resolvable. | Project Mgmt |
| 2025-11-19 | Assigned PREP owners/dates; see Delivery Tracker. | Planning |
| 2025-11-18 | Marked BENCH-GRAPH-24-002, BENCH-IMPACT-16-001, BENCH-POLICY-20-002, BENCH-SIG-26-001/002 as BLOCKED pending fixtures/datasets and reachability schema. | Bench |
| 2025-11-18 | Normalised sprint to standard template; renamed from SPRINT_512_bench.md. | Ops/Docs |