Bench Prep — PREP-BENCH-GRAPH-21-001 (Graph API/Indexer harness)

Status: Ready for implementation (2025-11-20) Owners: Bench Guild · Graph Platform Guild Scope: Build deterministic Graph benchmark harness for 50k/100k node fixtures measuring API/Indexer latency, memory, and tile cache hit rates.

Fixtures

Use SAMPLES-GRAPH-24-003 (40–50k) and extend to 100k via duplication with new ids; store under docs/samples/graph/50k.ndjson and 100k.ndjson with .sha256 hashes.
Node ordering deterministic; timestamps fixed to 2025-01-01T00:00:00Z.

Harness plan (project: `src/Bench/StellaOps.Bench.GraphApi`)

Scenarios (repeat 5x; report median/p95):
1. Viewport fetch: /v1/graph/tiles?bbox=<seed> — measure server latency + tile count.
2. Path query: /v1/graph/path?from=...&to=... — latency + hops + cache hits.
3. Overlay apply: apply policy overlay to 1k nodes; measure apply time and index rebuild cost.
4. Cold vs warm cache: run viewport + path with cache cold then warm; capture hit rate.
Metrics captured as NDJSON per run: { scenario, fixture, pass: cold|warm, medianMs, p95Ms, maxMs, rssMb, managedMb, cacheHitRate } plus start/end UTC timestamps.
Determinism: fixed seed (GRAPH_BENCH_SEED=2025-01-01T00:00:00Z); single-thread option --threads 1 for reproducibility; clear caches between cold/warm phases.

Outputs

Store under out/bench/graph/api/{runId}/results.ndjson with .sha256.
Summary CSV optional derived from NDJSON; no dynamic wall-clock in filenames beyond runId.

Acceptance criteria

Harness runs offline against local fixtures; no external calls.
Median/p95 for each scenario produced for both 50k and 100k fixtures; cache hit rate recorded where applicable.
Re-running with same seed/fixtures yields identical NDJSON (apart from RSS variance).

Next steps

Generate fixtures + hashes; wire CLI entry dotnet run -- graph-api --fixture docs/samples/graph/50k.ndjson --seed 20250101.
Add perf dashboard hook if available; otherwise publish artifacts under out/bench/graph/api/latest/.

2.1 KiB Raw Blame History Unescape Escape