Restructure solution layout by module
This commit is contained in:
33
src/Graph/StellaOps.Graph.Indexer/AGENTS.md
Normal file
33
src/Graph/StellaOps.Graph.Indexer/AGENTS.md
Normal file
@@ -0,0 +1,33 @@
|
||||
# Graph Indexer Guild Charter (Epic 5)
|
||||
|
||||
## Mission
|
||||
Project SBOM, advisory, VEX, and policy overlay data into a tenant-scoped property graph powering the SBOM Graph Explorer. Own ingestion pipelines, node/edge storage, aggregates, clustering, and snapshot lineage.
|
||||
|
||||
## Scope
|
||||
- Service source under `src/Graph/StellaOps.Graph.Indexer` (workers, ingestion pipelines, schema builders).
|
||||
- Mongo collections/object storage for `graph_nodes`, `graph_edges`, `graph_snapshots`, clustering metadata.
|
||||
- Event consumers: SBOM ingest, Conseiller advisories, Excitator VEX, Policy overlay materials.
|
||||
- Incremental rebuild, diff, and cache warmers for graph overlays.
|
||||
|
||||
## Principles
|
||||
1. **Immutability** – Graph mirrors SBOM snapshots; new data creates new snapshots rather than mutating historical records.
|
||||
2. **Determinism** – Given identical inputs, node/edge ids, hashes, and aggregates remain stable across runs.
|
||||
3. **Tenant isolation** – Enforce isolation at ingestion, storage, and job levels; no cross-tenant leakage.
|
||||
4. **AOC alignment** – Indexer links facts; it never mutates advisories/VEX/policy outcomes. Conseiller/Excitator/Policy Engine remain authoritative.
|
||||
5. **Performance & telemetry** – Every job emits metrics (latency, node/edge counts, queue lag) and structured logs.
|
||||
|
||||
## Collaboration
|
||||
- Keep `src/Graph/StellaOps.Graph.Indexer/TASKS.md`, `../../docs/implplan/SPRINTS.md` synchronized.
|
||||
- Coordinate with SBOM Service, Policy Engine, Conseiller, Excitator, Scheduler, Web Gateway, and Console teams.
|
||||
- Publish schema docs and fixtures for clients; share cost/identity conventions across services.
|
||||
|
||||
## Tooling
|
||||
- .NET 10 preview workers (HostedService + channel pipelines).
|
||||
- MongoDB for node/edge storage; S3-compatible buckets for layout tiles/snapshots if needed.
|
||||
- Scheduler integration (jobs, change streams) to handle incremental updates.
|
||||
|
||||
## Definition of Done
|
||||
- Pipelines deterministic and tested; fixtures validated.
|
||||
- Metrics/logs/traces wired with tenant context.
|
||||
- Schema docs + OpenAPI (where applicable) updated; compliance checklist appended.
|
||||
- Offline kit includes seed data for air-gapped installs.
|
||||
13
src/Graph/StellaOps.Graph.Indexer/TASKS.md
Normal file
13
src/Graph/StellaOps.Graph.Indexer/TASKS.md
Normal file
@@ -0,0 +1,13 @@
|
||||
# Graph Indexer Task Board — Epic 5: SBOM Graph Explorer
|
||||
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|
||||
|----|--------|----------|------------|-------------|---------------|
|
||||
| GRAPH-INDEX-28-001 | TODO | Graph Indexer Guild | SBOM-SERVICE-21-001, CARTO-GRAPH-21-001 | Define canonical node/edge schemas, attribute dictionaries, identity rules, and seed fixtures; publish schema doc. | Schema doc merged; identity property tests pass; fixtures committed for CI usage. |
|
||||
| GRAPH-INDEX-28-002 | TODO | Graph Indexer Guild | GRAPH-INDEX-28-001, SBOM-SERVICE-21-002 | Implement SBOM ingest consumer producing artifact/package/file nodes and edges with `valid_from/valid_to`, scope metadata, and provenance links. | Ingest pipeline processes sample SBOMs deterministically; metrics recorded; unit tests cover identity stability. |
|
||||
| GRAPH-INDEX-28-003 | TODO | Graph Indexer Guild | GRAPH-INDEX-28-001, CONCELIER-CONSOLE-23-001 | Project Concelier linksets into overlay tiles (`affected_by` edges, evidence refs) without mutating source observations; keep advisory aggregates in overlay store only. | Overlay documents generated deterministically; raw node/edge collections remain immutable; tests cover overlay refresh and eviction. |
|
||||
| GRAPH-INDEX-28-004 | TODO | Graph Indexer Guild | GRAPH-INDEX-28-001, EXCITITOR-CONSOLE-23-001 | Integrate VEX statements (`vex_exempts` edges) with justification metadata and precedence markers for overlays. | VEX edges generated; conflicts resolved deterministically; tests cover status transitions. |
|
||||
| GRAPH-INDEX-28-005 | TODO | Graph Indexer Guild, Policy Guild | POLICY-ENGINE-27-001, POLICY-ENGINE-27-002 | Hydrate policy overlays into graph (`governs_with` nodes/edges) referencing effective findings and explain hashes for sampled nodes. | Overlay nodes stored with policy version id, severity, status; explain references captured; validation tests pass. |
|
||||
| GRAPH-INDEX-28-006 | TODO | Graph Indexer Guild | GRAPH-INDEX-28-002..005 | Generate graph snapshots per SBOM with lineage (`derived_from`), adjacency manifests, and metadata for diff jobs. | Snapshot documents produced; lineage recorded; tests assert diff readiness; metrics emitted. |
|
||||
| GRAPH-INDEX-28-007 | TODO | Graph Indexer Guild, Observability Guild | GRAPH-INDEX-28-002..006 | Implement clustering/centrality background jobs (Louvain/degree/betweenness approximations) with configurable schedules and store cluster ids on nodes. | Clustering jobs run on fixtures; metrics logged; cluster ids accessible via API; SLA documented. |
|
||||
| GRAPH-INDEX-28-008 | TODO | Graph Indexer Guild | GRAPH-INDEX-28-002..007 | Provide incremental update + backfill pipeline with change streams, retry/backoff, idempotent operations, and backlog metrics. | Incremental updates replay sample change logs; retries/backoff validated; backlog metrics exported. |
|
||||
| GRAPH-INDEX-28-009 | TODO | Graph Indexer Guild, QA Guild | GRAPH-INDEX-28-002..008 | Add unit/property/integration tests, synthetic large graph fixtures, chaos testing (missing overlays, cycles), and determinism checks across runs. | Test suite green; determinism harness passes across two runs; perf metrics recorded. |
|
||||
| GRAPH-INDEX-28-010 | TODO | Graph Indexer Guild, DevOps Guild | GRAPH-INDEX-28-008 | Package deployment artifacts (Helm/Compose), offline seed bundles, and configuration docs; integrate Offline Kit. | Deployment descriptors merged; offline seed bundle documented; smoke deploy tested. |
|
||||
Reference in New Issue
Block a user