prep docs and service updates
Some checks failed
Docs CI / lint-and-preview (push) Has been cancelled
Some checks failed
Docs CI / lint-and-preview (push) Has been cancelled
This commit is contained in:
31
docs/benchmarks/graph/bench-graph-21-001-prep.md
Normal file
31
docs/benchmarks/graph/bench-graph-21-001-prep.md
Normal file
@@ -0,0 +1,31 @@
|
||||
# Bench Prep — PREP-BENCH-GRAPH-21-001 (Graph API/Indexer harness)
|
||||
|
||||
Status: **Ready for implementation** (2025-11-20)
|
||||
Owners: Bench Guild · Graph Platform Guild
|
||||
Scope: Build deterministic Graph benchmark harness for 50k/100k node fixtures measuring API/Indexer latency, memory, and tile cache hit rates.
|
||||
|
||||
## Fixtures
|
||||
- Use SAMPLES-GRAPH-24-003 (40–50k) and extend to 100k via duplication with new ids; store under `docs/samples/graph/50k.ndjson` and `100k.ndjson` with `.sha256` hashes.
|
||||
- Node ordering deterministic; timestamps fixed to `2025-01-01T00:00:00Z`.
|
||||
|
||||
## Harness plan (project: `src/Bench/StellaOps.Bench.GraphApi`)
|
||||
- Scenarios (repeat 5x; report median/p95):
|
||||
1. **Viewport fetch**: `/v1/graph/tiles?bbox=<seed>` — measure server latency + tile count.
|
||||
2. **Path query**: `/v1/graph/path?from=...&to=...` — latency + hops + cache hits.
|
||||
3. **Overlay apply**: apply policy overlay to 1k nodes; measure apply time and index rebuild cost.
|
||||
4. **Cold vs warm cache**: run viewport + path with cache cold then warm; capture hit rate.
|
||||
- Metrics captured as NDJSON per run: `{ scenario, fixture, pass: cold|warm, medianMs, p95Ms, maxMs, rssMb, managedMb, cacheHitRate }` plus start/end UTC timestamps.
|
||||
- Determinism: fixed seed (`GRAPH_BENCH_SEED=2025-01-01T00:00:00Z`); single-thread option `--threads 1` for reproducibility; clear caches between cold/warm phases.
|
||||
|
||||
## Outputs
|
||||
- Store under `out/bench/graph/api/{runId}/results.ndjson` with `.sha256`.
|
||||
- Summary CSV optional derived from NDJSON; no dynamic wall-clock in filenames beyond runId.
|
||||
|
||||
## Acceptance criteria
|
||||
- Harness runs offline against local fixtures; no external calls.
|
||||
- Median/p95 for each scenario produced for both 50k and 100k fixtures; cache hit rate recorded where applicable.
|
||||
- Re-running with same seed/fixtures yields identical NDJSON (apart from RSS variance).
|
||||
|
||||
## Next steps
|
||||
- Generate fixtures + hashes; wire CLI entry `dotnet run -- graph-api --fixture docs/samples/graph/50k.ndjson --seed 20250101`.
|
||||
- Add perf dashboard hook if available; otherwise publish artifacts under `out/bench/graph/api/latest/`.
|
||||
38
docs/benchmarks/graph/bench-graph-21-002-prep.md
Normal file
38
docs/benchmarks/graph/bench-graph-21-002-prep.md
Normal file
@@ -0,0 +1,38 @@
|
||||
# Bench Prep — PREP-BENCH-GRAPH-21-002 (UI headless graph benchmarks)
|
||||
|
||||
Status: **Ready for implementation** (2025-11-20)
|
||||
Owners: Bench Guild · UI Guild
|
||||
Scope: Define the Playwright-based UI benchmark that rides on the graph harness from BENCH-GRAPH-21-001 (50k/100k node fixtures) and produces deterministic latency/FPS metrics.
|
||||
|
||||
## Dependencies
|
||||
- Harness + fixtures from BENCH-GRAPH-21-001 (must expose HTTP endpoints and data seeds for 50k/100k graphs).
|
||||
- Graph API/Indexer stable query contract (per `docs/modules/graph/architecture.md`).
|
||||
|
||||
## Benchmark plan
|
||||
- Runner: Playwright (Chromium, headless) driven via `src/Bench/StellaOps.Bench.GraphUi`.
|
||||
- Environment:
|
||||
- Viewport: 1920x1080, device scale 1.0, throttling disabled; CPU pinned via `--disable-features=CPUThrottling`.
|
||||
- Fixed session seed `GRAPH_BENCH_SEED=2025-01-01T00:00:00Z` for RNG use in camera jitter.
|
||||
- Scenarios (each repeated 5x, median + p95 recorded):
|
||||
1. **Canvas load**: open `/graph/bench?fixture=50k` → measure TTI, first contentful paint, tiles loaded count.
|
||||
2. **Pan/zoom loop**: pan 500px x 20 iterations + zoom in/out (2x each) → record average FPS and frame jank percentage.
|
||||
3. **Path query**: submit shortest-path query between two seeded nodes → measure query latency (client + API) and render latency.
|
||||
4. **Filter drill-down**: apply two filters (severity=high, product=“core”) → measure time to filtered render + memory delta.
|
||||
- Metrics captured to NDJSON per run:
|
||||
- `timestampUtc`, `scenario`, `fixture`, `p95_ms`, `median_ms`, `avg_fps`, `jank_pct`, `mem_mb`, `api_latency_ms` (where applicable).
|
||||
- Determinism:
|
||||
- All timestamps recorded in UTC ISO-8601; RNG seeded; cache cleared before each scenario; `--disable-features=UseAFH` disabled to avoid adaptive throttling.
|
||||
|
||||
## Outputs
|
||||
- NDJSON benchmark results stored under `out/bench/graph/ui/{runId}.ndjson` with a `.sha256` alongside.
|
||||
- Summary CSV optional, derived from NDJSON for reporting only.
|
||||
- CI step publishes artifacts to `out/bench/graph/ui/latest/` with write-once semantics per runId.
|
||||
|
||||
## Acceptance criteria
|
||||
- Playwright suite reproducibly exercises the four scenarios on 50k and 100k fixtures with seeded inputs.
|
||||
- Metrics include p95 and median for each scenario and fixture size; FPS ≥ 30 on 50k fixture baseline.
|
||||
- Archive outputs are deterministic for given fixture and seed (excluding wall-clock timestamps in filenames; embed timestamps only in content).
|
||||
|
||||
## Next steps
|
||||
- Wire Playwright harness into `BENCH-GRAPH-21-001` pipeline once fixtures ready.
|
||||
- Hook results into perf dashboard if available; otherwise store NDJSON + hashes.
|
||||
Reference in New Issue
Block a user