up
Some checks failed
AOC Guard CI / aoc-guard (push) Has been cancelled
AOC Guard CI / aoc-verify (push) Has been cancelled
Concelier Attestation Tests / attestation-tests (push) Has been cancelled
Docs CI / lint-and-preview (push) Has been cancelled
Export Center CI / export-ci (push) Has been cancelled
devportal-offline / build-offline (push) Has been cancelled
Some checks failed
AOC Guard CI / aoc-guard (push) Has been cancelled
AOC Guard CI / aoc-verify (push) Has been cancelled
Concelier Attestation Tests / attestation-tests (push) Has been cancelled
Docs CI / lint-and-preview (push) Has been cancelled
Export Center CI / export-ci (push) Has been cancelled
devportal-offline / build-offline (push) Has been cancelled
This commit is contained in:
@@ -3,17 +3,18 @@
|
||||
Purpose: measure basic graph load/adjacency build and shallow path exploration over deterministic fixtures.
|
||||
|
||||
## Fixtures
|
||||
- Use interim synthetic fixtures under `samples/graph/interim/graph-50k` or `graph-100k`.
|
||||
- Canonical: `samples/graph/graph-40k` (SAMPLES-GRAPH-24-003) with overlay + manifest hashes.
|
||||
- Legacy interim (still usable for comparisons): `samples/graph/interim/graph-50k` and `graph-100k`.
|
||||
- Each fixture includes `nodes.ndjson`, `edges.ndjson`, and `manifest.json` with hashes/counts.
|
||||
- Optional overlay: drop `overlay.ndjson` next to the fixture (or set `overlay.path` in `manifest.json`) to apply extra edges/layers; hashes are captured in results.
|
||||
|
||||
## Usage
|
||||
```bash
|
||||
python graph_bench.py \
|
||||
--fixture ../../../samples/graph/interim/graph-50k \
|
||||
--output results/graph-50k.json \
|
||||
--fixture ../../../../samples/graph/graph-40k \
|
||||
--output results/graph-40k.json \
|
||||
--samples 100 \
|
||||
--overlay ../../../samples/graph/interim/graph-50k/overlay.ndjson # optional
|
||||
--overlay ../../../../samples/graph/graph-40k/overlay.ndjson # optional
|
||||
```
|
||||
|
||||
Outputs a JSON summary with:
|
||||
@@ -28,6 +29,6 @@ Determinism:
|
||||
- Sorted node ids, fixed sample size, stable ordering, no randomness beyond fixture content.
|
||||
- No network access; pure local file reads.
|
||||
|
||||
Next steps (after overlay schema lands):
|
||||
- Extend to load overlay snapshots and measure overlay-join overhead.
|
||||
Next steps:
|
||||
- Keep results in sync with canonical fixture hashes; if overlay schema changes regenerate fixture + manifests.
|
||||
- Add p95/median latency over multiple runs and optional concurrency knobs.
|
||||
|
||||
44
src/Bench/StellaOps.Bench/Graph/results/graph-40k.json
Normal file
44
src/Bench/StellaOps.Bench/Graph/results/graph-40k.json
Normal file
@@ -0,0 +1,44 @@
|
||||
{
|
||||
"avg_reach_3": 14.32,
|
||||
"bfs_ms": 0.8,
|
||||
"bfs_samples": 100,
|
||||
"build_ms": 5563.14,
|
||||
"edges": 100171,
|
||||
"fixture": "graph-40k",
|
||||
"manifest": {
|
||||
"counts": {
|
||||
"edges": 100071,
|
||||
"nodes": 40000,
|
||||
"overlays": {
|
||||
"policy.overlay.v1": 100
|
||||
}
|
||||
},
|
||||
"generated_at": "2025-11-22T00:00:00Z",
|
||||
"hashes": {
|
||||
"edges_ndjson_sha256": "143a294446f46ffa273846e821f83fd5e5023aea2cf74947ba7ccaeeab7ceba4",
|
||||
"nodes_ndjson_sha256": "d14e8c642d1b4450d8779971da79cecc190af22fe237dee56ec0dd583f0442f5",
|
||||
"overlay_ndjson_sha256": "627a0d8c273f55b2426c8c005037ef01d88324a75084ad44bd620b1330a539cc"
|
||||
},
|
||||
"inputs": {
|
||||
"sbom_source": "mock-sbom-v1"
|
||||
},
|
||||
"overlay": {
|
||||
"id_scheme": "sha256(tenant|nodeId|overlayKind)",
|
||||
"kind": "policy.overlay.v1",
|
||||
"path": "overlay.ndjson"
|
||||
},
|
||||
"seed": 424242,
|
||||
"snapshot_id": "graph-40k-policy-overlay-20251122",
|
||||
"tenant": "demo-tenant"
|
||||
},
|
||||
"max_reach_3": 36,
|
||||
"nodes": 40100,
|
||||
"overlay": {
|
||||
"added_edges": 100,
|
||||
"applied": true,
|
||||
"introduced_nodes": 100,
|
||||
"path": "/mnt/e/dev/git.stella-ops.org/samples/graph/graph-40k/overlay.ndjson",
|
||||
"sha256": "627a0d8c273f55b2426c8c005037ef01d88324a75084ad44bd620b1330a539cc"
|
||||
},
|
||||
"overlay_ms": 52.24
|
||||
}
|
||||
@@ -4,7 +4,8 @@ set -euo pipefail
|
||||
ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
# Repo root is four levels up from Graph/
|
||||
REPO_ROOT="$(cd "${ROOT}/../../../.." && pwd)"
|
||||
FIXTURES_ROOT="${FIXTURES_ROOT:-${REPO_ROOT}/samples/graph/interim}"
|
||||
# Default to canonical graph-40k fixture; allow override or fallback to interim.
|
||||
FIXTURES_ROOT="${FIXTURES_ROOT:-${REPO_ROOT}/samples/graph}"
|
||||
OUT_DIR="${OUT_DIR:-$ROOT/results}"
|
||||
OVERLAY_ROOT="${OVERLAY_ROOT:-${FIXTURES_ROOT}}"
|
||||
SAMPLES="${SAMPLES:-100}"
|
||||
@@ -26,7 +27,16 @@ run_one() {
|
||||
python "${ROOT}/graph_bench.py" "${args[@]}"
|
||||
}
|
||||
|
||||
run_one "${FIXTURES_ROOT}/graph-50k"
|
||||
run_one "${FIXTURES_ROOT}/graph-100k"
|
||||
if [[ -d "${FIXTURES_ROOT}/graph-40k" ]]; then
|
||||
run_one "${FIXTURES_ROOT}/graph-40k"
|
||||
fi
|
||||
|
||||
# legacy/interim comparisons
|
||||
if [[ -d "${FIXTURES_ROOT}/interim/graph-50k" ]]; then
|
||||
run_one "${FIXTURES_ROOT}/interim/graph-50k"
|
||||
fi
|
||||
if [[ -d "${FIXTURES_ROOT}/interim/graph-100k" ]]; then
|
||||
run_one "${FIXTURES_ROOT}/interim/graph-100k"
|
||||
fi
|
||||
|
||||
echo "Graph bench complete. Results in ${OUT_DIR}"
|
||||
|
||||
@@ -3,7 +3,8 @@
|
||||
Purpose: provide a deterministic, headless flow for measuring graph UI interactions over large fixtures (50k/100k nodes).
|
||||
|
||||
## Scope
|
||||
- Use synthetic fixtures under `samples/graph/interim/` until canonical SAMPLES-GRAPH-24-003 lands.
|
||||
- Default fixture: `samples/graph/graph-40k` (SAMPLES-GRAPH-24-003) with policy overlay hashes.
|
||||
- Legacy comparison fixtures remain under `samples/graph/interim/`.
|
||||
- Optional overlay layer (`overlay.ndjson`) is loaded when present and toggled during the run to capture render/merge overhead.
|
||||
- Drive a deterministic sequence of interactions:
|
||||
1) Load graph canvas with specified fixture.
|
||||
|
||||
Reference in New Issue
Block a user