up
This commit is contained in:
@@ -35,11 +35,26 @@ Graph Indexer + Graph API build the tenant-scoped knowledge graph that powers bl
|
||||
- Logs/traces: structured ETL logs, query planner traces, WebGL interaction telemetry (once UI lands).
|
||||
- Offline bundles: deterministic `nodes.jsonl`, `edges.jsonl`, overlay manifests + DSSE signatures, consumable by Export Center and CLI mirroring.
|
||||
|
||||
## Operations & runbook (Sprint 030)
|
||||
- Dashboards: import `Observability/graph-api-grafana.json` (panels for latency, budget denials, overlay cache ratio, export latency). Apply tenant filter in every panel.
|
||||
- Health checks: `/healthz` should be 200; search/query/paths/diff/export endpoints require `X-Stella-Tenant`, `Authorization`, and scopes (`graph:read/query/export`).
|
||||
- Key metrics (new):
|
||||
- `graph_tile_latency_seconds` histogram (label `route`); alert when p95 > 1.5s for 5m.
|
||||
- `graph_query_budget_denied_total` counter (label `reason`); investigate spikes (>50 in 5m).
|
||||
- `graph_overlay_cache_hits_total` / `graph_overlay_cache_misses_total`; watch miss ratio > 0.4 for 10m.
|
||||
- `graph_export_latency_seconds` histogram (label `format`); alert when p95 > 2s for ndjson/graphml.
|
||||
- Triage playbook:
|
||||
- Budget denials: lower default edges/nodes budget or guide callers to request smaller scopes; verify overlay includes are truly required.
|
||||
- Overlay cache misses: ensure cache TTL is ≥5m; check overlay service connectivity to Policy Engine; warm cache by replaying recent hot nodes.
|
||||
- Export slowness: reduce export `Limit`, offload PNG/SVG to worker, and confirm disk I/O headroom.
|
||||
- If alerts fire, capture tenant, route, cursor/budget values, and recent deploy SHA in incident note.
|
||||
|
||||
## Key docs & updates
|
||||
- [`architecture.md`](architecture.md) — inputs, pipelines, APIs, storage choices, observability, offline handling.
|
||||
- [`implementation_plan.md`](implementation_plan.md) — phased delivery roadmap, work breakdown, risks, test strategy.
|
||||
- [`schema.md`](schema.md) — canonical node/edge schema and attribute dictionary (keep in sync with indexer code).
|
||||
- Updates: `docs/updates/2025-10-26-scheduler-graph-jobs.md`, `docs/updates/2025-10-26-authority-graph-scopes.md`, `docs/updates/2025-10-30-devops-governance.md` for the latest decisions/dependencies.
|
||||
- Index: see `architecture-index.md` for data model, ingestion pipeline, overlays/caches, events, and API/observability pointers.
|
||||
|
||||
## Epic alignment
|
||||
- **Epic 5 – SBOM Graph Explorer:** Graph Indexer, Graph API, saved queries, overlays, Console/CLI experiences, Offline Kit parity.
|
||||
|
||||
48
docs/modules/graph/architecture-index.md
Normal file
48
docs/modules/graph/architecture-index.md
Normal file
@@ -0,0 +1,48 @@
|
||||
# Graph Module · Architecture Index
|
||||
|
||||
Status: Draft (2025-11-26) — aligns with Sprint 0141 (Graph Indexer) and Sprint 0207 (Graph API).
|
||||
|
||||
## Data model (canonical)
|
||||
- Nodes/edges follow `docs/modules/graph/schema.md`.
|
||||
- Tenancy: every node/edge/snapshot/event carries `tenant`.
|
||||
- Snapshots: immutable bundles of nodes + edges; overlays stored separately (`graph_overlays_cache`) to avoid mutating historical snapshots.
|
||||
|
||||
## Ingestion pipeline
|
||||
1) **SBOM snapshots** from SBOM Service → Graph Indexer (`graph_snapshots`, `graph_nodes`, `graph_edges`).
|
||||
2) **Advisory/VEX** from Concelier/Excititor → edges and overlay seeds.
|
||||
3) **Policy overlays** from Policy Engine (POLICY-ENGINE-30-001..003) → cached per node in `graph_overlays_cache`.
|
||||
4) **Runtime/signals** (future) → additional edges/attributes; stored alongside snapshot edges with tenant guard.
|
||||
|
||||
## Overlays & caches
|
||||
- Overlay types: policy (`policy.overlay.v1`), vex (`openvex.v1`), analytics (clusters/centrality).
|
||||
- Cache TTL: 5–10 minutes recommended; deterministic IDs (SHA256 over tenant|node|kind).
|
||||
- Analytics overlays emitted as NDJSON (`overlays/clusters.ndjson`, `overlays/centrality.ndjson`) with stable ordering.
|
||||
|
||||
## Events & change streams
|
||||
- Inputs: `sbom.snapshot.created`, `advisory.observation.updated@1`, `sbom.version.created`, `sbom.asset.updated`.
|
||||
- Outputs: Graph Indexer change-stream/backfill emits node/edge upserts and overlay refresh notifications for Graph API caches.
|
||||
- Downstream consumers: Scheduler (recompute jobs), Graph API (query caches), Export Center (offline bundles).
|
||||
|
||||
## APIs (reader surfaces)
|
||||
- Graph API (Sprint 0207): `/graph/search`, `/graph/query`, `/graph/paths`, `/graph/diff`, `/graph/export`.
|
||||
- Saved queries and snapshots exposed through Graph API; RBAC scopes enforced (`graph:read/query/export`).
|
||||
- Health check: `/healthz` (200 when service ready).
|
||||
See also `docs/api/graph.md` (pending) for full endpoint parameters and error envelopes.
|
||||
|
||||
## Observability (current metrics)
|
||||
- `graph_tile_latency_seconds{route}`
|
||||
- `graph_query_budget_denied_total{reason}`
|
||||
- `graph_overlay_cache_hits_total`, `graph_overlay_cache_misses_total`
|
||||
- `graph_export_latency_seconds{format}`
|
||||
- Ingest/backfill metrics live in Indexer (Sprint 0141).
|
||||
|
||||
## Offline / bundles
|
||||
- Exports: deterministic NDJSON (`nodes.jsonl`, `edges.jsonl`, overlays) with SHA256 manifest and optional DSSE.
|
||||
- Air-gap friendly: no external calls; rely on cached schemas and local snapshots.
|
||||
|
||||
## References
|
||||
- `architecture.md` — deep dive.
|
||||
- `implementation_plan.md` — roadmap and risks.
|
||||
- `schema.md` — canonical node/edge shapes.
|
||||
- `docs/implplan/SPRINT_0141_0001_0001_graph_indexer.md` — Indexer sprint.
|
||||
- `docs/implplan/SPRINT_0207_0001_0001_graph.md` — Graph API sprint.
|
||||
Reference in New Issue
Block a user