Here’s a compact, practical way to add an **explanation graph** that traces every vulnerability verdict back to raw evidence—so auditors can verify results without trusting an LLM. --- # What it is (in one line) A small, immutable graph that connects a **verdict** → to **reasoning steps** → to **raw evidence** (source scan records, binary symbol/build‑ID matches, external advisories/feeds), with cryptographic hashes so anyone can replay/verify it. --- # Minimal data model (vendor‑neutral) ```json { "explanationGraph": { "scanId": "uuid", "artifact": { "purl": "pkg:docker/redis@7.2.4", "digest": "sha256:…", "buildId": "elf:abcd…|pe:…|macho:…" }, "verdicts": [ { "verdictId": "uuid", "cve": "CVE-2024-XXXX", "status": "affected|not_affected|under_investigation", "policy": "vex/lattice:v1", "reasoning": [ {"stepId":"s1","type":"callgraph.reachable","evidenceRef":"e1"}, {"stepId":"s2","type":"version.match","evidenceRef":"e2"}, {"stepId":"s3","type":"vendor.vex.override","evidenceRef":"e3"} ], "provenance": { "scanner": "StellaOps.Scanner@1.3.0", "rulesHash": "sha256:…", "time": "2025-11-25T12:34:56Z", "attestation": "dsse:…" } } ], "evidence": [ { "evidenceId":"e1", "kind":"binary.callgraph", "hash":"sha256:…", "summary":"main -> libssl!EVP_* path present", "blobPointer":"ipfs://… | file://… | s3://…" }, { "evidenceId":"e2", "kind":"source.scan", "hash":"sha256:…", "summary":"Detected libssl 3.0.14 via SONAME + build‑id", "blobPointer":"…" }, { "evidenceId":"e3", "kind":"external.feed", "hash":"sha256:…", "summary":"Vendor VEX: CVE not reachable when FIPS mode enabled", "blobPointer":"…", "externalRef":{"type":"advisory","id":"VEX-ACME-2025-001","url":"…"} } ] } } ``` --- # How it works (flow) * **Collect** raw artifacts: scanner findings, binary symbol matches (Build‑ID / PDB / dSYM), SBOM components, external feeds (NVD, vendor VEX). * **Normalize** to evidence nodes (immutable blobs with content hash + pointer). * **Reason** via small, deterministic rules (your lattice/policy). Each rule emits a *reasoning step* that points to evidence. * **Emit a verdict** with status + full chain of steps. * **Seal** with DSSE/Sigstore (or your offline signer) so the whole graph is replayable. --- # Why this helps (auditable AI) * **No black box**: every “affected/not affected” claim links to verifiable bytes. * **Deterministic**: same inputs + rules = same verdict (hashes prove it). * **Reproducible for clients/regulators**: export graph + blobs, they replay locally. * **LLM‑optional**: you can add LLM explanations as *non‑authoritative* annotations; the verdict remains policy‑driven. --- # C# drop‑in (Stella Ops style) ```csharp public record EvidenceNode( string EvidenceId, string Kind, string Hash, string Summary, string BlobPointer, ExternalRef? ExternalRef = null); public record ReasoningStep(string StepId, string Type, string EvidenceRef); public record Verdict( string VerdictId, string Cve, string Status, string Policy, IReadOnlyList Reasoning, Provenance Provenance); public record Provenance(string Scanner, string RulesHash, DateTimeOffset Time, string Attestation); public record ExplanationGraph( Guid ScanId, Artifact Artifact, IReadOnlyList Verdicts, IReadOnlyList Evidence); public record Artifact(string Purl, string Digest, string BuildId); ``` * Persist as immutable documents (Mongo collection `explanations`). * Store large evidence blobs in object storage; keep `hash` + `blobPointer` in Mongo. * Sign the serialized graph (DSSE) and store the signature alongside. --- # UI (compact “trace” panel) * **Top line:** CVE → Status chip (Affected / Not affected / Needs review). * **Three tabs:** *Evidence*, *Reasoning*, *Provenance*. * **One‑click export:** “Download Replay Bundle (.zip)” → JSON graph + evidence blobs + verify script. * **Badge:** “Deterministic ✓” when rulesHash + inputs resolve to prior signature. --- # Ops & replay * Bundle a tiny CLI: `stellaops-explain verify graph.json --evidence ./blobs/`. * Verification checks: all hashes match, DSSE signature valid, rulesHash known, verdict derivable from steps. --- # Where to start (1‑week sprint) * Day 1–2: Model + Mongo collections + signer service. * Day 3: Scanner adapters emit `EvidenceNode` records; policy engine emits `ReasoningStep`. * Day 4: Verdict assembly + DSSE signing + export bundle. * Day 5: Minimal UI trace panel + CLI verifier. If you want, I can generate the Mongo schemas, a DSSE signing helper, and the React/Angular trace panel stub next. Here’s a concrete implementation plan you can hand to your developers so they’re not guessing what to build. I’ll break it down by **phases**, and inside each phase I’ll call out **owner**, **deliverables**, and **acceptance criteria**. --- ## Phase 0 – Scope & decisions (½ day) **Goal:** Lock in the “rules of the game” so nobody bikesheds later. **Decisions to confirm (write in a short ADR):** 1. **Canonical representation & hashing** * Format for hashing: **canonical JSON** (stable property ordering, UTF‑8, no whitespace). * Algorithm: **SHA‑256** for: * `ExplanationGraph` document * each `EvidenceNode` * Hash scope: * `evidence.hash` = hash of the raw evidence blob (or canonical subset if huge) * `graphHash` = hash of the entire explanation graph document (minus signature). 2. **Signing** * Format: **DSSE envelope** (`payloadType = "stellaops/explanation-graph@v1"`). * Key management: use existing **offline signing key** or Sigstore‑style keyless if already in org. * Signature attached as: * `provenance.attestation` field inside each verdict **and** * stored in a separate `explanation_signatures` collection or S3 path for replay. 3. **Storage** * Metadata: **MongoDB** collection `explanation_graphs`. * Evidence blobs: * S3 (or compatible) bucket `stella-explanations/` with layout: * `evidence/{evidenceId}` or `evidence/{hash}`. 4. **ID formats** * `scanId`: UUID (string). * `verdictId`, `evidenceId`, `stepId`: UUID (string). * `buildId`: reuse existing convention (`elf:`, `pe:`, `macho:`). **Deliverable:** 1–2 page ADR in repo (`/docs/adr/000-explanation-graph.md`). --- ## Phase 1 – Domain model & persistence (backend) **Owner:** Backend ### 1.1. Define core C# domain models Place in `StellaOps.Explanations` project or equivalent: ```csharp public record ArtifactRef( string Purl, string Digest, string BuildId); public record ExternalRef( string Type, // "advisory", "vex", "nvd", etc. string Id, string Url); public record EvidenceNode( string EvidenceId, string Kind, // "binary.callgraph", "source.scan", "external.feed", ... string Hash, // sha256 of blob string Summary, string BlobPointer, // s3://..., file://..., ipfs://... ExternalRef? ExternalRef = null); public record ReasoningStep( string StepId, string Type, // "callgraph.reachable", "version.match", ... string EvidenceRef); // EvidenceId public record Provenance( string Scanner, string RulesHash, // hash of rules/policy bundle used DateTimeOffset Time, string Attestation); // DSSE envelope (base64 or JSON) public record Verdict( string VerdictId, string Cve, string Status, // "affected", "not_affected", "under_investigation" string Policy, // e.g. "vex.lattice:v1" IReadOnlyList Reasoning, Provenance Provenance); public record ExplanationGraph( Guid ScanId, ArtifactRef Artifact, IReadOnlyList Verdicts, IReadOnlyList Evidence, string GraphHash); // sha256 of canonical JSON ``` ### 1.2. MongoDB schema Collection: `explanation_graphs` Document shape: ```jsonc { "_id": "scanId:artifactDigest", // composite key or just ObjectId + separate fields "scanId": "uuid", "artifact": { "purl": "pkg:docker/redis@7.2.4", "digest": "sha256:...", "buildId": "elf:abcd..." }, "verdicts": [ /* Verdict[] */ ], "evidence": [ /* EvidenceNode[] */ ], "graphHash": "sha256:..." } ``` **Indexes:** * `{ scanId: 1 }` * `{ "artifact.digest": 1 }` * `{ "verdicts.cve": 1, "artifact.digest": 1 }` (compound) * Optional: TTL or archiving mechanism if you don’t want to keep these forever. **Acceptance criteria:** * You can serialize/deserialize `ExplanationGraph` to Mongo without loss. * Indexes exist and queries by `scanId`, `artifact.digest`, and `(digest + CVE)` are efficient. --- ## Phase 2 – Evidence ingestion plumbing **Goal:** Make every relevant raw fact show up as an `EvidenceNode`. **Owner:** Backend scanner team ### 2.1. Evidence factory service Create `IEvidenceService`: ```csharp public interface IEvidenceService { Task StoreBinaryCallgraphAsync( Guid scanId, ArtifactRef artifact, byte[] callgraphBytes, string summary, ExternalRef? externalRef = null); Task StoreSourceScanAsync( Guid scanId, ArtifactRef artifact, byte[] scanResultJson, string summary); Task StoreExternalFeedAsync( Guid scanId, ExternalRef externalRef, byte[] rawPayload, string summary); } ``` Implementation tasks: 1. **Hash computation** * Compute SHA‑256 over raw bytes. * Prefer a helper: ```csharp public static string Sha256Hex(ReadOnlySpan data) { ... } ``` 2. **Blob storage** * S3 key format, e.g.: `explanations/{scanId}/{evidenceId}`. * `BlobPointer` string = `s3://stella-explanations/explanations/{scanId}/{evidenceId}`. 3. **EvidenceNode creation** * Generate `evidenceId = Guid.NewGuid().ToString("N")`. * Populate `kind`, `hash`, `summary`, `blobPointer`, `externalRef`. 4. **Graph assembly contract** * Evidence service **does not** write to Mongo. * It only uploads blobs and returns `EvidenceNode` objects. * The **ExplanationGraphBuilder** (next phase) collects them. **Acceptance criteria:** * Given a callgraph binary, a corresponding `EvidenceNode` is returned with: * hash matching the blob (verified in tests), * blob present in S3, * summary populated. --- ## Phase 3 – Reasoning & policy integration **Goal:** Instrument your existing VEX / lattice policy engine to emit deterministic **reasoning steps** instead of just a boolean status. **Owner:** Policy / rules engine team ### 3.1. Expose rule evaluation trace Assume you already have something like: ```csharp VulnerabilityStatus Evaluate(ArtifactRef artifact, string cve, Findings findings); ``` Extend it to: ```csharp public sealed class RuleEvaluationTrace { public string StepType { get; init; } // e.g. "version.match" public string RuleId { get; init; } // "rule:openssl:versionFromElf" public string Description { get; init; } // human-readable explanation public string EvidenceKind { get; init; } // to match with EvidenceService public object EvidencePayload { get; init; } // callgraph bytes, json, etc. } public sealed class EvaluationResult { public string Status { get; init; } // "affected", etc. public IReadOnlyList Trace { get; init; } } ``` New API: ```csharp EvaluationResult EvaluateWithTrace( ArtifactRef artifact, string cve, Findings findings); ``` ### 3.2. From trace to ReasoningStep + EvidenceNode Create `ExplanationGraphBuilder`: ```csharp public interface IExplanationGraphBuilder { Task BuildAsync( Guid scanId, ArtifactRef artifact, IReadOnlyList cveFindings, string scannerName); } ``` Internal algorithm for each `CveFinding`: 1. Call `EvaluateWithTrace(artifact, cve, finding)` to get `EvaluationResult`. 2. For each `RuleEvaluationTrace`: * Use `EvidenceService` with appropriate method based on `EvidenceKind`. * Get back an `EvidenceNode` with `evidenceId`. * Create `ReasoningStep`: * `StepId = Guid.NewGuid()` * `Type = trace.StepType` * `EvidenceRef = evidenceNode.EvidenceId` 3. Assemble `Verdict`: ```csharp var verdict = new Verdict( verdictId: Guid.NewGuid().ToString("N"), cve: finding.Cve, status: result.Status, policy: "vex.lattice:v1", reasoning: steps, provenance: new Provenance( scanner: scannerName, rulesHash: rulesBundleHash, time: DateTimeOffset.UtcNow, attestation: "" // set in Phase 4 ) ); ``` 4. Collect: * all `EvidenceNode`s (dedupe by `hash` to avoid duplicates). * all `Verdict`s. **Acceptance criteria:** * Given deterministic inputs (scan + rules bundle hash), repeated runs produce: * same sequence of `ReasoningStep` types, * same set of `EvidenceNode.hash` values, * same `status`. --- ## Phase 4 – Graph hashing & DSSE signing **Owner:** Security / platform ### 4.1. Canonical JSON for hash Implement: ```csharp public static class ExplanationGraphSerializer { public static string ToCanonicalJson(ExplanationGraph graph) { // no graphHash, no attestation in this step } } ``` Key requirements: * Consistent property ordering (e.g. alphabetical). * No extra whitespace. * UTF‑8 encoding. * Primitive formatting options fixed (e.g. date as ISO 8601 with `Z`). ### 4.2. Hash and sign Before persisting: ```csharp var graphWithoutHash = graph with { GraphHash = "" }; var canonicalJson = ExplanationGraphSerializer.ToCanonicalJson(graphWithoutHash); var graphHash = Sha256Hex(Encoding.UTF8.GetBytes(canonicalJson)); // sign DSSE envelope var envelope = dsseSigner.Sign( payloadType: "stellaops/explanation-graph@v1", payload: Encoding.UTF8.GetBytes(canonicalJson) ); // attach var signedVerdicts = graph.Verdicts .Select(v => v with { Provenance = v.Provenance with { Attestation = envelope.ToJson() } }) .ToList(); var finalGraph = graph with { GraphHash = $"sha256:{graphHash}", Verdicts = signedVerdicts }; ``` Then write `finalGraph` to Mongo. **Acceptance criteria:** * Recomputing `graphHash` from Mongo document (zeroing `graphHash` and `attestation`) matches stored value. * Verifying DSSE signature with the public key succeeds. --- ## Phase 5 – Backend APIs & export bundle **Owner:** Backend / API ### 5.1. Read APIs Add endpoints (REST-ish): 1. **Get graph for scan-artifact** `GET /explanations/scans/{scanId}/artifacts/{digest}` * Returns entire `ExplanationGraph` JSON. 2. **Get single verdict** `GET /explanations/scans/{scanId}/artifacts/{digest}/cves/{cve}` * Returns `Verdict` + its subset of `EvidenceNode`s. 3. **Search by CVE** `GET /explanations/search?cve=CVE-2024-XXXX&digest=sha256:...` * Returns list of `(scanId, artifact, verdictId)`. ### 5.2. Export replay bundle `POST /explanations/{scanId}/{digest}/export` Implementation: * Create a temporary directory. * Write: * `graph.json` → `ExplanationGraph` as stored. * `signature.json` → DSSE envelope alone (optional). * Evidence blobs: * For each `EvidenceNode`: * Download from S3 and store as `evidence/{evidenceId}`. * Zip the folder: `explanation-{scanId}-{shortDigest}.zip`. * Stream as download. ### 5.3. CLI verifier Small .NET / Go CLI: Commands: ```bash stellaops-explain verify graph.json --evidence ./evidence ``` Verification steps: 1. Load `graph.json`, parse to `ExplanationGraph`. 2. Strip `graphHash` & `attestation`, re‑serialize canonical JSON. 3. Recompute SHA‑256 and compare to `graphHash`. 4. Verify DSSE envelope with public key. 5. For each `EvidenceNode`: * Read file `./evidence/{evidenceId}`. * Recompute hash and compare with `evidence.hash`. Exit with non‑zero code if anything fails; print a short summary. **Acceptance criteria:** * Export bundle round‑trips: `verify` passes on an exported zip. * APIs documented in OpenAPI / Swagger. --- ## Phase 6 – UI: Explanation trace panel **Owner:** Frontend ### 6.1. API integration New calls in frontend client: * `GET /explanations/scans/{scanId}/artifacts/{digest}` * Optionally `GET /explanations/.../cves/{cve}` if you want lazy loading per CVE. ### 6.2. Component UX On the “vulnerability detail” view: * Add **“Explanation”** tab with three sections: 1. **Verdict summary** * Badge: `Affected` / `Not affected` / `Under investigation`. * Text: `Derived using policy {policy}, rules hash {rulesHash[..8]}.` 2. **Reasoning timeline** * Vertical list of `ReasoningStep`s: * Icon per type (e.g. “flow” icon for `callgraph.reachable`). * Title = `Type` (humanized). * Click to expand underlying `EvidenceNode.summary`. * Optional “View raw evidence” link (downloads blob via S3 signed URL). 3. **Provenance** * Show: * `scanner` * `rulesHash` * `time` * “Attested ✓” if DSSE verifies on the backend (or pre‑computed). 4. **Export** * Button: “Download replay bundle (.zip)” * Calls export endpoint and triggers browser download. **Acceptance criteria:** * For any CVE in UI, a user can: * See why it is (not) affected in at most 2 clicks. * Download a replay bundle via the UI. --- ## Phase 7 – Testing strategy **Owner:** QA + all devs ### 7.1. Unit tests * EvidenceService: * Hash matches blob contents. * BlobPointer formats are as expected. * ExplanationGraphBuilder: * Given fixed test input, the resulting graph JSON matches golden file. * Serializer: * Canonical JSON is stable under property reordering in the code. ### 7.2. Integration tests * End‑to‑end fake scan: * Simulate scanner output + rules. * Build graph → persist → fetch via API. * Run CLI verify on exported bundle in CI. ### 7.3. Security tests * Signature tampering: * Modify `graph.json` in exported bundle; `verify` must fail. * Evidence tampering: * Modify an evidence file; `verify` must fail. --- ## Phase 8 – Rollout **Owner:** PM / Tech lead 1. **Feature flag** * Start with explanation graph generation behind a flag for: * subset of scanners, * subset of tenants. 2. **Backfill (optional)** * If useful, run a one‑off job that: * Takes recent scans, * Rebuilds explanation graphs, * Stores them in Mongo. 3. **Docs** * Short doc page for customers: * “What is an Explanation Graph?” * “How to verify it with the CLI?” --- ## Developer checklist (TL;DR) You can literally drop this into Jira as epics/tasks: 1. **Backend** * [ ] Implement domain models (`ExplanationGraph`, `Verdict`, `EvidenceNode`, etc.). * [ ] Implement `IEvidenceService` + S3 integration. * [ ] Extend policy engine to `EvaluateWithTrace`. * [ ] Implement `ExplanationGraphBuilder`. * [ ] Implement canonical serializer, hashing, DSSE signing. * [ ] Implement Mongo persistence + indexes. * [ ] Implement REST APIs + export ZIP. 2. **Frontend** * [ ] Wire new APIs into the vulnerability detail view. * [ ] Build Explanation tab (Summary / Reasoning / Provenance). * [ ] Implement “Download replay bundle” button. 3. **Tools** * [ ] Implement `stellaops-explain verify` CLI. * [ ] Add CI test that runs verify against a sample bundle. 4. **QA** * [ ] Golden‑file tests for graphs. * [ ] Signature & evidence tampering tests. * [ ] UI functional tests on explanations. --- If you’d like, next step I can turn this into: * concrete **OpenAPI spec** for the new endpoints, and/or * a **sample `stellaops-explain verify` CLI skeleton** (C# or Go).