Restructure solution layout by module
This commit is contained in:
33
src/Graph/StellaOps.Graph.Api/AGENTS.md
Normal file
33
src/Graph/StellaOps.Graph.Api/AGENTS.md
Normal file
@@ -0,0 +1,33 @@
|
||||
# Graph API Guild Charter (Epic 5)
|
||||
|
||||
## Mission
|
||||
Provide tenant-scoped Graph Explorer APIs for search, query, paths, diffs, overlays, and exports. Deliver cost-aware streaming endpoints that integrate with Policy Engine, Conseiller, Excitator, and the Graph Indexer while honoring AOC and RBAC.
|
||||
|
||||
## Scope
|
||||
- Service under `src/Graph/StellaOps.Graph.Api` (Minimal API + streaming pipeline + query planner).
|
||||
- Query validation/planning, cost estimation, tile streaming, overlay composition, export serializers.
|
||||
- Integration with Authority scopes, Web API Gateway, Policy Engine explain endpoints, Graph Indexer storage.
|
||||
- Saved query management and diff endpoints.
|
||||
|
||||
## Principles
|
||||
1. **Bounded interactivity** – Enforce budgets (nodes, edges, time) per tenant and surface truncation clearly.
|
||||
2. **Determinism** – Same query + seed yields same streamed content; maintain layout seeds for client.
|
||||
3. **Security first** – RBAC enforced server-side; input validation, tenant isolation, query sanitization.
|
||||
4. **AOC alignment** – API surfaces readonly data; overlays annotate Policy Engine outputs; never mutate facts.
|
||||
5. **Observability** – Every query logs cost, latency, truncation, caching; metrics + traces integrated.
|
||||
|
||||
## Collaboration
|
||||
- Maintain `src/Graph/StellaOps.Graph.Api/TASKS.md`, `../../docs/implplan/SPRINTS.md` alignment.
|
||||
- Coordinate with Graph Indexer (storage contracts), Web Gateway, Console, CLI, Policy Engine, DevOps, and Docs teams.
|
||||
- Publish OpenAPI + JSON schema for queries and streaming tiles.
|
||||
|
||||
## Tooling
|
||||
- .NET 10 preview Minimal API with async streaming; pipeline pattern for parsing/planning/fetching.
|
||||
- Mongo aggregation / adjacency store from Graph Indexer; optional caching layer.
|
||||
- SSE/WebSockets or chunked NDJSON responses for progressive loading.
|
||||
|
||||
## Definition of Done
|
||||
- APIs shipped with OpenAPI, unit/integration/load tests, budget enforcement.
|
||||
- Metrics/logs/traces wired; dashboards seeded.
|
||||
- Documentation updated (API doc, query schema, cost/limit guidance).
|
||||
- Offline kit instructions include CLI + API usage.
|
||||
16
src/Graph/StellaOps.Graph.Api/TASKS.md
Normal file
16
src/Graph/StellaOps.Graph.Api/TASKS.md
Normal file
@@ -0,0 +1,16 @@
|
||||
# Graph API Task Board — Epic 5: SBOM Graph Explorer
|
||||
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|
||||
|----|--------|----------|------------|-------------|---------------|
|
||||
| GRAPH-API-28-001 | TODO | Graph API Guild | GRAPH-INDEX-28-001, WEB-GRAPH-21-001 | Define OpenAPI + JSON schema for graph search/query/paths/diff/export endpoints, including cost metadata and streaming tile schema. | OpenAPI committed; schema validated via CI; clients regenerated; docs updated. |
|
||||
| GRAPH-API-28-002 | TODO | Graph API Guild | GRAPH-API-28-001, GRAPH-INDEX-28-002 | Implement `/graph/search` with multi-type index lookup, prefix/exact match, RBAC enforcement, and result ranking + caching. | Endpoint returns ranked results within budget; tests cover scope errors + caching; metrics logged. |
|
||||
| GRAPH-API-28-003 | TODO | Graph API Guild | GRAPH-API-28-001, GRAPH-INDEX-28-002..005 | Build query planner + cost estimator for `/graph/query`, stream tiles (nodes/edges/stats) progressively, enforce budgets, provide cursor tokens. | Query endpoint streams tiles deterministically, enforces budgets, surfaces truncation flags; integration tests cover large graphs. |
|
||||
| GRAPH-API-28-004 | TODO | Graph API Guild | GRAPH-API-28-003 | Implement `/graph/paths` with depth ≤6, constraint filters, heuristic shortest path search, and optional policy overlay rendering. | Paths API returns expected routes; policy overlay applied; guardrails enforced; tests cover over-budget errors. |
|
||||
| GRAPH-API-28-005 | TODO | Graph API Guild | GRAPH-INDEX-28-006, GRAPH-API-28-003 | Implement `/graph/diff` streaming added/removed/changed nodes/edges between SBOM snapshots; include overlay deltas and policy/VEX/advisory metadata. | Diff endpoint streams deterministic results; tests cover sample diffs; metrics record diff compute time. |
|
||||
| GRAPH-API-28-006 | TODO | Graph API Guild | GRAPH-INDEX-28-002..005, POLICY-ENGINE-27-001 | Consume Policy Engine overlay contract (`POLICY-ENGINE-30-001..003`) and surface advisory/VEX/policy overlays with caching, partial materialization, and explain trace sampling for focused nodes. | Overlay pipeline delivers heatmap stats + explain samples; caches invalidate on policy/VEX/advisory change; tests cover concurrency. |
|
||||
| GRAPH-API-28-007 | TODO | Graph API Guild | GRAPH-API-28-003..006 | Implement exports (`graphml`, `csv`, `ndjson`, `png`, `svg`) with async job management, checksum manifests, and streaming downloads. | Export job API returns manifest + download URLs; tests validate formats; docs updated. |
|
||||
| GRAPH-API-28-008 | TODO | Graph API Guild, Authority Guild | AUTH-GRAPH-26-001, AUTH-GRAPH-21-001 | Integrate RBAC scopes (`graph:read`, `graph:query`, `graph:export`), tenant headers, audit logging, and rate limiting. | Unauthorized access rejected; audit logs include query hash & scope; rate limits enforced; integration tests pass; scope checks use `StellaOpsScopes` constants (no string literals). |
|
||||
|
||||
> 2025-10-26 — Waiting on Graph API host scaffolding. When endpoints land, ensure all scope enforcement relies on `StellaOpsScopes` before closing GRAPH-API-28-008.
|
||||
| GRAPH-API-28-009 | TODO | Graph API Guild, Observability Guild | GRAPH-API-28-002..007 | Instrument metrics (`graph_tile_latency_seconds`, `graph_query_budget_denied_total`, `graph_overlay_cache_hit_ratio`), structured logs, and traces per query stage; publish dashboards. | Metrics exposed; dashboards live; alerts configured; docs updated. |
|
||||
| GRAPH-API-28-010 | TODO | Graph API Guild, QA Guild | GRAPH-API-28-002..007 | Build unit/integration/load tests with synthetic datasets (500k nodes/2M edges), fuzz query validation, verify determinism across runs. | Test suite green; load test report captured; determinism harness passes with fixed seed. |
|
||||
| GRAPH-API-28-011 | TODO | Graph API Guild, DevOps Guild | GRAPH-API-28-003..007 | Provide deployment manifests, offline kit support, API gateway integration docs, and smoke tests. | Deployment descriptors merged; gateway routes documented; offline kit instructions updated; smoke tests executed. |
|
||||
33
src/Graph/StellaOps.Graph.Indexer/AGENTS.md
Normal file
33
src/Graph/StellaOps.Graph.Indexer/AGENTS.md
Normal file
@@ -0,0 +1,33 @@
|
||||
# Graph Indexer Guild Charter (Epic 5)
|
||||
|
||||
## Mission
|
||||
Project SBOM, advisory, VEX, and policy overlay data into a tenant-scoped property graph powering the SBOM Graph Explorer. Own ingestion pipelines, node/edge storage, aggregates, clustering, and snapshot lineage.
|
||||
|
||||
## Scope
|
||||
- Service source under `src/Graph/StellaOps.Graph.Indexer` (workers, ingestion pipelines, schema builders).
|
||||
- Mongo collections/object storage for `graph_nodes`, `graph_edges`, `graph_snapshots`, clustering metadata.
|
||||
- Event consumers: SBOM ingest, Conseiller advisories, Excitator VEX, Policy overlay materials.
|
||||
- Incremental rebuild, diff, and cache warmers for graph overlays.
|
||||
|
||||
## Principles
|
||||
1. **Immutability** – Graph mirrors SBOM snapshots; new data creates new snapshots rather than mutating historical records.
|
||||
2. **Determinism** – Given identical inputs, node/edge ids, hashes, and aggregates remain stable across runs.
|
||||
3. **Tenant isolation** – Enforce isolation at ingestion, storage, and job levels; no cross-tenant leakage.
|
||||
4. **AOC alignment** – Indexer links facts; it never mutates advisories/VEX/policy outcomes. Conseiller/Excitator/Policy Engine remain authoritative.
|
||||
5. **Performance & telemetry** – Every job emits metrics (latency, node/edge counts, queue lag) and structured logs.
|
||||
|
||||
## Collaboration
|
||||
- Keep `src/Graph/StellaOps.Graph.Indexer/TASKS.md`, `../../docs/implplan/SPRINTS.md` synchronized.
|
||||
- Coordinate with SBOM Service, Policy Engine, Conseiller, Excitator, Scheduler, Web Gateway, and Console teams.
|
||||
- Publish schema docs and fixtures for clients; share cost/identity conventions across services.
|
||||
|
||||
## Tooling
|
||||
- .NET 10 preview workers (HostedService + channel pipelines).
|
||||
- MongoDB for node/edge storage; S3-compatible buckets for layout tiles/snapshots if needed.
|
||||
- Scheduler integration (jobs, change streams) to handle incremental updates.
|
||||
|
||||
## Definition of Done
|
||||
- Pipelines deterministic and tested; fixtures validated.
|
||||
- Metrics/logs/traces wired with tenant context.
|
||||
- Schema docs + OpenAPI (where applicable) updated; compliance checklist appended.
|
||||
- Offline kit includes seed data for air-gapped installs.
|
||||
13
src/Graph/StellaOps.Graph.Indexer/TASKS.md
Normal file
13
src/Graph/StellaOps.Graph.Indexer/TASKS.md
Normal file
@@ -0,0 +1,13 @@
|
||||
# Graph Indexer Task Board — Epic 5: SBOM Graph Explorer
|
||||
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|
||||
|----|--------|----------|------------|-------------|---------------|
|
||||
| GRAPH-INDEX-28-001 | TODO | Graph Indexer Guild | SBOM-SERVICE-21-001, CARTO-GRAPH-21-001 | Define canonical node/edge schemas, attribute dictionaries, identity rules, and seed fixtures; publish schema doc. | Schema doc merged; identity property tests pass; fixtures committed for CI usage. |
|
||||
| GRAPH-INDEX-28-002 | TODO | Graph Indexer Guild | GRAPH-INDEX-28-001, SBOM-SERVICE-21-002 | Implement SBOM ingest consumer producing artifact/package/file nodes and edges with `valid_from/valid_to`, scope metadata, and provenance links. | Ingest pipeline processes sample SBOMs deterministically; metrics recorded; unit tests cover identity stability. |
|
||||
| GRAPH-INDEX-28-003 | TODO | Graph Indexer Guild | GRAPH-INDEX-28-001, CONCELIER-CONSOLE-23-001 | Project Concelier linksets into overlay tiles (`affected_by` edges, evidence refs) without mutating source observations; keep advisory aggregates in overlay store only. | Overlay documents generated deterministically; raw node/edge collections remain immutable; tests cover overlay refresh and eviction. |
|
||||
| GRAPH-INDEX-28-004 | TODO | Graph Indexer Guild | GRAPH-INDEX-28-001, EXCITITOR-CONSOLE-23-001 | Integrate VEX statements (`vex_exempts` edges) with justification metadata and precedence markers for overlays. | VEX edges generated; conflicts resolved deterministically; tests cover status transitions. |
|
||||
| GRAPH-INDEX-28-005 | TODO | Graph Indexer Guild, Policy Guild | POLICY-ENGINE-27-001, POLICY-ENGINE-27-002 | Hydrate policy overlays into graph (`governs_with` nodes/edges) referencing effective findings and explain hashes for sampled nodes. | Overlay nodes stored with policy version id, severity, status; explain references captured; validation tests pass. |
|
||||
| GRAPH-INDEX-28-006 | TODO | Graph Indexer Guild | GRAPH-INDEX-28-002..005 | Generate graph snapshots per SBOM with lineage (`derived_from`), adjacency manifests, and metadata for diff jobs. | Snapshot documents produced; lineage recorded; tests assert diff readiness; metrics emitted. |
|
||||
| GRAPH-INDEX-28-007 | TODO | Graph Indexer Guild, Observability Guild | GRAPH-INDEX-28-002..006 | Implement clustering/centrality background jobs (Louvain/degree/betweenness approximations) with configurable schedules and store cluster ids on nodes. | Clustering jobs run on fixtures; metrics logged; cluster ids accessible via API; SLA documented. |
|
||||
| GRAPH-INDEX-28-008 | TODO | Graph Indexer Guild | GRAPH-INDEX-28-002..007 | Provide incremental update + backfill pipeline with change streams, retry/backoff, idempotent operations, and backlog metrics. | Incremental updates replay sample change logs; retries/backoff validated; backlog metrics exported. |
|
||||
| GRAPH-INDEX-28-009 | TODO | Graph Indexer Guild, QA Guild | GRAPH-INDEX-28-002..008 | Add unit/property/integration tests, synthetic large graph fixtures, chaos testing (missing overlays, cycles), and determinism checks across runs. | Test suite green; determinism harness passes across two runs; perf metrics recorded. |
|
||||
| GRAPH-INDEX-28-010 | TODO | Graph Indexer Guild, DevOps Guild | GRAPH-INDEX-28-008 | Package deployment artifacts (Helm/Compose), offline seed bundles, and configuration docs; integrate Offline Kit. | Deployment descriptors merged; offline seed bundle documented; smoke deploy tested. |
|
||||
Reference in New Issue
Block a user