git.stella-ops.org/docs/implplan/EPIC_5.md

Here’s Epic 5 in the same paste‑into‑repo, implementation‑ready format as the prior epics. It’s exhaustive, formal, and designed to slot into AOC, Policy Engine, Conseiller/Excitator, and the Console.

---

Epic 5: SBOM Graph Explorer

Short name: Graph Explorer Services touched: SBOM Service, Graph Indexer (new), Graph API (new), Policy Engine, Conseiller (Feedser), Excitator (Vexer), Web API Gateway, Authority (authN/Z), Workers/Scheduler, Telemetry Surfaces: Console (Web UI) graph module, CLI, Exports Deliverables: Interactive graph UI with semantic zoom, saved queries, policy/VEX/advisory overlays, diff views, impact analysis, exports

---

1) What it is

SBOM Graph Explorer is the interactive, tenant‑scoped view of all supply‑chain relationships the platform knows about, rendered as a navigable graph. It connects:

• Artifacts (applications, images, libs), Packages/Versions, Files/Paths, Licenses, Advisories (from Conseiller), VEX statements (from Excitator), Provenance (builds, sources), and Policies (overlays of determinations)
• Edges like depends_on, contains, built_from, declared_in, affected_by, vex_exempts, governs_with
• Time/version dimension: multiple SBOM snapshots with diffs

It’s built for investigation and review: find where a vulnerable package enters; see which apps are impacted; understand why a finding exists; simulate a policy version and see the delta. The explorer observes AOC enforcement: it never mutates facts; it aggregates and visualizes them. Only the Policy Engine may classify, and classification is displayed as overlays.

---

2) Why

• SBOMs are graphs. Tables flatten what matters and hide transitive risk.
• Engineers, security, and auditors need impact answers quickly: “What pulls in log4j:2.17 and where is it at runtime?”
• Policy/VEX/advisory interactions are nuanced. A visual overlay makes precedence and outcomes obvious.
• Review is collaborative; you need saved queries, deep links, exports, and consistent evidence.

---

3) How it should work (maximum detail)

3.1 Domain model

Nodes (typed, versioned, tenant‑scoped):

• Artifact: application, service, container image, library, module
• Package: name + ecosystem (purl), PackageVersion with resolved version
• File: path within artifact or image layer
• License: SPDX id
• Advisory: normalized advisory id (GHSA, CVE, vendor), source = Conseiller
• VEX: statement with product context, status, justification, source = Excitator
• SBOM: ingestion unit; includes metadata (tool, sha, build info)
• PolicyDetermination: materialized view of Policy Engine results (read‑only overlay)
• Build: provenance, commit, workflow run
• Source: repo, tag, commit

Edges (directed):

• declared_in (PackageVersion → SBOM)
• contains (Artifact → PackageVersion | File)
• depends_on (PackageVersion → PackageVersion) with scope attr (prod|dev|test|optional)
• built_from (Artifact → Build), provenance_of (Build → Source)
• affected_by (PackageVersion → Advisory) with range semantics
• vex_exempts (Advisory ↔ VEX) scoped by product/component
• licensed_under (Artifact|PackageVersion → License)
• governs_with (Artifact|PackageVersion → PolicyDetermination)
• derived_from (SBOM → SBOM) for superseding snapshots

Identity & versioning

• Every node has a stable key: {tenant}:{type}:{natural_id} (e.g., purl for packages, digest for images).
• SBOM snapshots are immutable; edges carry valid_from/valid_to for time travel and diffing.

Imposed rule: Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.

3.2 User capabilities (end‑to‑end)

• Search & Navigate: global search (purls, CVEs, repos, licenses), keyboard nav, breadcrumbs, semantic zoom.

• Lenses: toggle views (Security, License, Provenance, Runtime vs Dev, Policy effect).

• Overlays:

  • Advisory overlay: show affected nodes/edges with source, severity, ranges.
  • VEX overlay: show suppressions/justifications; collapse exempted paths.
  • Policy overlay: choose a policy version; nodes/edges reflect determinations (severity, status) with explain sampling.

• Impact analysis: pick a vulnerable node; highlight upstream/downstream dependents, scope filters, shortest/all paths with constraints.

• Diff view: compare SBOM A vs B; show added/removed nodes/edges, changed versions, changed determinations.

• Saved queries: visual builder + JSON query; shareable permalinks scoped by tenant and environment.

• Exports: GraphML, CSV edge list, NDJSON of findings, PNG/SVG snapshot.

• Evidence details: side panel with raw facts, advisory links, VEX statements, policy explain trace, provenance.

• Accessibility: tab‑navigable, high‑contrast, screen‑reader labels for nodes and sidebars.

3.3 Query model

• Visual builder for common queries:

  • “Show all paths from Artifact X to Advisory Y up to depth 6.”
  • “All runtime dependencies with license = GPL‑3.0.”
  • “All artifacts affected by GHSA‑… with no applicable VEX.”
  • “Which SBOMs introduced/removed openssl between build 120 and 130?”

• JSON query (internal, POST body) with:

  • start: list of node selectors (type + id or attributes)
  • expand: edge types and depth, direction, scope filters
  • where: predicates on node/edge attributes
  • overlay: policy version id, advisory sources, VEX filters
  • limit: nodes, edges, timebox, cost budget

• Cost control: server estimates cost, denies or pages results; UI streams partial graph tiles.

3.4 UI architecture (Console)

• Canvas: WebGL renderer with level‑of‑detail, edge bundling, and label culling; deterministic layout when possible (seeded).

• Semantic zoom:

  • Far: clusters by artifact/repo/ecosystem, color by lens
  • Mid: package groups, advisory badges, license swatches
  • Near: concrete versions, direct edges, inline badges for policy determinations

• Panels:

  • Left: search, filters, lens selector, saved queries
  • Right: details, explain trace, evidence tabs (Advisory/VEX/Policy/Provenance)
  • Bottom: query expression, diagnostics, performance/stream status

• Diff mode: split or overlay, color legend (add/remove/changed), filter by node type.

• Deep links: URL encodes query + viewport; shareable respecting RBAC.

• Keyboard: space drag, +/- zoom, F to focus, G to expand neighbors, P to show paths.

3.5 Back‑end architecture

Graph Indexer (new)

• Consumes SBOM ingests, Conseiller advisories, Excitator VEX statements, Policy Engine determinations (read‑only).

• Projects facts into a property graph persisted in:

  • Primary: document store + adjacency sets (e.g., Mongo collections + compressed adjacency lists)
  • Optional driver for graph DB backends if needed (pluggable)

• Maintains materialized aggregates: degree, critical paths cache, affected artifact counts, license distribution.

• Emits graph snapshots per SBOM with lineage to original ingestion.

Graph API (new)

• Endpoints for search, neighbor expansion, path queries, diffs, overlays, exports.
• Streaming responses for large graphs (chunked NDJSON tiles).
• Cost accounting + quotas per tenant.

Workers

• Centrality & clustering precompute on idle: betweenness approximations, connected components, Louvain clusters.
• Diff compute on new SBOM ingestion pairs (previous vs current).
• Overlay materialization cache for popular policy versions.

Policy Engine integration

• Graph API requests can specify a policy version.
• For sampled nodes, the API fetches explain traces; for counts, uses precomputed overlay materializations where available.

AOC enforcement

• Graph Indexer never merges or edits advisories/VEX; it links them and exposes overlays that the Policy Engine evaluates.
• Conseiller and Excitator remain authoritative sources; severities come from Policy‑governed normalization.

3.6 APIs (representative)

• GET /graph/search?q=...&type=package|artifact|advisory|license
• POST /graph/query ⇒ stream tiles {nodes[], edges[], stats, cursor}
• POST /graph/paths body: {from, to, depth<=6, constraints{scope, runtime_only}}
• POST /graph/diff body: {sbom_a, sbom_b, filters}
• GET /graph/snapshots/{sbom_id} ⇒ graph metadata, counts, top advisories
• POST /graph/export body: {format: graphml|csv|ndjson|png|svg, query|snapshot}
• GET /graph/saved / POST /graph/saved save and list tenant queries
• GET /graph/overlays/policy/{version_id} ⇒ summary stats for caching

All endpoints tenant‑scoped, RBAC‑checked. Timeouts and pagination by server. Errors return structured diagnostics.

3.7 CLI

stella sbom graph search "purl:pkg:maven/org.apache.logging.log4j/log4j-core"
stella sbom graph query --file ./query.json --export graphml > graph.graphml
stella sbom graph impacted --advisory GHSA-xxxx --runtime-only --limit 100
stella sbom graph paths --from artifact:service-a --to advisory:GHSA-xxxx --depth 5 --policy 1.3.0
stella sbom graph diff --sbom-a 2025-03-15T10:00Z --sbom-b 2025-03-22T10:00Z --export csv > diff.csv
stella sbom graph save --name "openssl-runtime" --file ./query.json

Exit codes: 0 ok, 2 query validation error, 3 over‑budget, 4 not found, 5 RBAC denied.

Imposed rule: Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.

3.8 Performance & scale

• Progressive loading: server pages tiles by BFS frontier; client renders incrementally.
• Viewport culling: only visible nodes/edges in canvas; offscreen demoted to aggregates.
• Level‑of‑detail: simplified glyphs and collapsed clusters at distance.
• Query budgets: per‑tenant rate + node/edge caps; interactive paths limited to depth ≤ 6.
• Caching: hot queries memoized per tenant + overlay version; diffs precomputed for consecutive SBOMs.

3.9 Security

• Multi‑tenant isolation at storage and API layers.

• RBAC roles:

  • Viewer: browse graphs, saved queries
  • Investigator: run queries, export data
  • Operator: configure budgets, purge caches
  • Auditor: download evidence bundles

• Input validation for query JSON; deny disallowed edge traversals; strict CSP in web app.

3.10 Observability

• Metrics: tile latency, nodes/edges per tile, cache hit rate, query denials, memory pressure.
• Logs: structured, include query hash, cost, truncation flags.
• Traces: server spans per stage (parse, plan, fetch, overlay, stream).

3.11 Accessibility & UX guarantees

• Keyboard complete, ARIA roles for graph and panels, high‑contrast theme.
• Deterministic layout on reload for shareable investigations.

3.12 Data retention

• Graph nodes derived from SBOMs share retention with SBOM artifacts; overlays are ephemeral caches.
• Saved queries retained until deleted; references to missing objects show warnings.

---

4) Implementation plan

4.1 Services

• Graph Indexer (new microservice)

  • Subscribes to SBOM ingest events, Conseiller advisory updates, Excitator VEX updates, Policy overlay materializations.
  • Builds adjacency lists and node documents; computes aggregates and clusters.

• Graph API (new microservice)

  • Validates and executes queries; streams tiles; composes overlays; serves diffs and exports.
  • Integrates with Policy Engine for explain sample retrieval.

• SBOM Service (existing)

  • Emits ingestion events with SBOM ids and lineage; exposes SBOM metadata to Graph API.

• Web API Gateway

  • Routes /graph/*, injects tenant context, enforces RBAC.

4.2 Console (Web UI) feature module

• packages/features/graph-explorer

  • Canvas renderer (WebGL), panels, query builder, diff mode, overlays, exports.
  • Deep‑link router and viewport state serializer.

4.3 Workers

• Centrality/clustering worker, diff worker, overlay materialization worker.
• Schedules on low‑traffic windows; backpressure aware.

4.4 Data model (storage)

• Collections:

  • graph_nodes: {_id, tenant, type, natural_id, attrs, degree, created_at, updated_at}
  • graph_edges: {_id, tenant, from_id, to_id, type, attrs, valid_from, valid_to}
  • graph_snapshots: per‑SBOM node/edge references
  • graph_saved_queries: {_id, tenant, name, query_json, created_by}
  • graph_overlays_cache: keyed by {tenant, policy_version, hash(query)}

• Indexes: compound on {tenant, type, natural_id}, {tenant, from_id}, {tenant, to_id}, time bounds.

---

5) Documentation changes (create/update)

1. /docs/sbom/graph-explorer-overview.md

   • Concepts, node/edge taxonomy, lenses, overlays, roles, limitations.

2. /docs/sbom/graph-using-the-console.md

   • Walkthroughs: search, navigate, impact, diff, export; screenshots and keyboard cheatsheet.

3. /docs/sbom/graph-query-language.md

   • JSON schema, examples, constraints, cost/budget rules.

4. /docs/sbom/graph-api.md

   • REST endpoints, request/response examples, streaming and pagination.

5. /docs/sbom/graph-cli.md

   • CLI command reference and example pipelines.

6. /docs/policy/graph-overlays.md

   • How policy versions render in Graph; explain sampling; AOC guardrails.

7. /docs/vex/graph-integration.md

   • How VEX suppressions appear and how to validate product scoping.

8. /docs/advisories/graph-integration.md

   • Advisory linkage and severity normalization by policy.

9. /docs/architecture/graph-services.md

   • Graph Indexer, Graph API, storage choices, failure modes.

10. /docs/observability/graph-telemetry.md

    • Metrics, logs, tracing, dashboards.

11. /docs/runbooks/graph-incidents.md

    • Handling runaway queries, cache poisoning, degraded render.

12. /docs/security/graph-rbac.md

    • Permissions matrix, multi‑tenant boundaries.

Every doc should end with a “Compliance checklist.” Imposed rule: Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.

---

6) Tasks

6.1 Backend: Graph Indexer

• Define node/edge schemas and attribute dictionaries for each type.
• Implement event consumers for SBOM ingests, Conseiller updates, Excitator updates.
• Build ingestion pipeline that populates nodes/edges and maintains valid_from/valid_to.
• Implement aggregate counters and degree metrics.
• Implement clustering job and persist cluster ids per node.
• Implement snapshot materialization per SBOM and lineage tracking.
• Unit tests for each node/edge builder; property‑based tests for identity stability.

6.2 Backend: Graph API

• Implement /graph/search with prefix and exact match across node types.
• Implement /graph/query with validation, planning, cost estimation, and streaming tile results.
• Implement /graph/paths with constraints and depth limits; shortest path heuristic.
• Implement /graph/diff computing adds/removes/changed versions; stream results.
• Implement overlays: advisory join, VEX join, policy materialization and explain sampling.
• Implement exports: GraphML, CSV edge list, NDJSON findings, PNG/SVG snapshots.
• RBAC middleware integration; multi‑tenant scoping.
• Load tests with synthetic large SBOMs; define default budgets.

6.3 Policy Engine integration

• Add endpoint to fetch explain traces for specific node ids in batch.
• Add materialization export that Graph API can cache per policy version.

6.4 Console (Web UI)

• Create graph-explorer module with routes /graph, /graph/diff, /graph/q/:id.
• Implement WebGL canvas with LOD, culling, edge bundling, deterministic layout seed.
• Build search, filter, lens, and overlay toolbars.
• Side panels: details, evidence tabs, explain viewer.
• Diff mode: split/overlay toggles and color legend.
• Saved queries: create, update, run; deep links.
• Export UI: formats, server round‑trip, progress indicators.
• a11y audit and keyboard‑only flow.

6.5 CLI

• Implement stella sbom graph * subcommands with JSON IO and piping support.
• Document examples and stable output schemas for CI consumption.

6.6 Observability & Ops

• Dashboards for tile latency, query denials, cache hit rate, memory.
• Alerting on query error spikes, OOM risk, cache churn.
• Runbooks in /docs/runbooks/graph-incidents.md.

6.7 Docs

• Author all docs in section 5, link from Console contextual help.
• Add end‑to‑end tutorial: “Investigate GHSA‑XXXX across prod artifacts.”

Imposed rule: Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.

---

7) Acceptance criteria

• Console renders large SBOM graphs with semantic zoom, overlays, and responsive interactions.
• Users can run impact and path queries with bounded depth and get results within budget.
• VEX suppressions and advisory severities appear correctly and are consistent with policy.
• Diff view clearly shows added/removed/changed nodes/edges between two SBOMs.
• Saved queries and deep links reproduce the same view deterministically (given same data).
• Exports produce valid GraphML/CSV/NDJSON and image snapshots.
• CLI supports search, query, paths, impacted, diff, and export with stable schemas.
• AOC guardrails: explorer never mutates facts; overlays reflect Policy Engine decisions.
• RBAC enforced; all actions logged and observable.

---

8) Risks & mitigations

• Graph explosion on large monorepos → tiling, clustering, budgets, and strict depth limits.
• Inconsistent identities across tools → canonicalize purls/digests; property‑based tests for identity stability.
• Policy overlay latency → precompute materializations for hot policy versions; sample explains only on focus.
• User confusion → strong lens defaults, deterministic layouts, legends, in‑context help.

---

9) Test plan

• Unit: node/edge builders, identity normalization, cost estimator.
• Integration: ingest SBOM + advisories + VEX, verify overlays and counts.
• E2E: Playwright flows for search→impact→diff→export; deep link determinism.
• Performance: simulate 500k nodes/2M edges; measure tile latency and memory.
• Security: RBAC matrix; tenant isolation tests; query validation fuzzing.
• Determinism: snapshot round‑trip: same query and seed produce identical layout and stats.

---

10) Feature flags

• graph.explorer (UI feature module)
• graph.paths (advanced path queries)
• graph.diff (SBOM diff mode)
• graph.overlays.policy (policy overlay + explain sampling)
• graph.export (exports enabled)

Documented in /docs/observability/graph-telemetry.md.

---

11) Non‑goals (this epic)

• Real‑time process/runtime call graphs.
• Full substitution for text reports; Explorer complements Reports.
• Cross‑tenant graphs; all queries are tenant‑scoped.

---

12) Philosophy

• See the system: security and license risk are structural. If you cannot see structure, you will miss risk.
• Evidence over assertion: every colored node corresponds to raw facts and explainable determinations.
• Bounded interactivity: fast, partial answers beat slow “complete” ones.
• Immutability: graphs mirror SBOM snapshots and are never rewritten; we add context, not edits.

Final reminder: Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.