efaf3cb789
Some checks failed
Signals CI & Image / signals-ci (push) Has been cancelled
Signals Reachability Scoring & Events / reachability-smoke (push) Has been cancelled
Signals Reachability Scoring & Events / sign-and-upload (push) Has been cancelled
Manifest Integrity / Validate Schema Integrity (push) Has been cancelled
Manifest Integrity / Validate Contract Documents (push) Has been cancelled
Manifest Integrity / Validate Pack Fixtures (push) Has been cancelled
Manifest Integrity / Audit SHA256SUMS Files (push) Has been cancelled
Manifest Integrity / Verify Merkle Roots (push) Has been cancelled
Docs CI / lint-and-preview (push) Has been cancelled
2.0 KiB
2.0 KiB
Reachability Scoring Bench Prep — PREP-BENCH-SIG-26-001-REACHABILITY-SCHEMA-FIX
Status: Ready for execution (2025-12-11) Owners: Bench Guild Aú Signals Guild Scope: Define inputs/fixtures and schema for reachability scoring benchmarks (10k/50k functions) to unblock BENCH-SIG-26-001.
Dependencies
- Reachability schema hash captured locally for synthetic fixtures.
- Sample callgraph/runtime traces sized for 10k/50k functions.
Harness
- Project:
src/Bench/StellaOps.Bench/Signals/reachability_bench.py. - Inputs:
- Callgraph:
docs/samples/signals/reachability/callgraph-10k.ndjson(callgraph-10k.ndjson.sha256). - Runtime traces:
docs/samples/signals/reachability/runtime-10k.ndjson(runtime-10k.ndjson.sha256). - 50k variants under the same directory (
callgraph-50k.ndjson,runtime-50k.ndjson+.sha256).
- Callgraph:
- Schema:
docs/benchmarks/signals/reachability-schema.json(sha256aaa5c8ab5cc2fe91e50976fafd8c73597387ab9a881af6d5d9818d202beba24e). - Metrics: facts/sec, p50/p95/p99 per-node latency, peak RSS, managed MB, GC gen2.
- Output: metrics NDJSON + cache NDJSON with reachability flags for each function (consumed by BENCH-SIG-26-002).
Acceptance
- Schema hash recorded and referenced. ✅
- Sample fixtures published under
docs/samples/signals/reachability/for 10k/50k. ✅ - Deterministic harness command documented; outputs written locally with
.sha256hashes. ✅
Commands
- 10k:
python src/Bench/StellaOps.Bench/Signals/reachability_bench.py --callgraph docs/samples/signals/reachability/callgraph-10k.ndjson --runtime docs/samples/signals/reachability/runtime-10k.ndjson --output src/Bench/StellaOps.Bench/Signals/results/reachability-metrics-10k.ndjson --cache-output src/Bench/StellaOps.Bench/Signals/results/reachability-cache-10k.ndjson --threads 1 --seed 20250101 - 50k: swap
10kfor50kin the command above (reachability-*-50k.ndjson).
Handoff
Use these fixtures + commands to run BENCH-SIG-26-001. Cache outputs (reachability-cache-*.ndjson) feed BENCH-SIG-26-002 for policy evaluation overhead measurements.