stella-ops.org/git.stella-ops.org
1
0
Fork 0
Code Issues Pull Requests Actions 12 Releases Wiki Activity
Files
01f4943ab93475ffa49581e8e4d1d3603151e465
git.stella-ops.org/docs/product-advisories/13-Dec-2025 - Smart‑Diff - Defining Meaningful Risk Change.md
master 01f4943ab9 up
2025-12-14 16:23:44 +02:00

24 KiB
Raw Blame History

Here’s a crisp, first‑time‑friendly blueprint for Smart‑Diff—a minimal‑noise way to highlight only changes that actually shift security risk, not every tiny SBOM/VEX delta.

What “Smart‑Diff” means (in plain terms)

Smart‑Diff is the smallest set of changes between two builds/releases that materially change risk. We only surface a change when it affects exploitability or policy—not when a dev-only transitive bumped a patch with no runtime path.

Count it as a Smart‑Diff only if at least one of these flips:

  • Reachability: new reachable vulnerable code appears, or previously reachable code becomes unreachable.
  • VEX status: a CVE’s status changes (e.g., to not_affected).
  • Version vs affected ranges: a dependency crosses into/out of a known vulnerable range.
  • KEV/EPSS/Policy: CISA KEV listing, EPSS spike, or your org policy gates change.

Ignore:

  • CVEs that are both unreachable and VEX = not_affected.
  • Pure patch‑level churn that doesn’t cross an affected range and isn’t KEV‑listed.
  • Dev/test‑only deps with no runtime path.

Minimal data model (practical)

  • DiffSet { added, removed, changed } for packages, symbols, CVEs, and policy gates.
  • AffectedGraph { package → symbol → call‑site }: reachability edges from entrypoints to vulnerable sinks.
  • EvidenceLink { attestation | VEX | KEV | scanner trace } per item, so every claim is traceable.

Core algorithms (what makes it “smart”)

  • Reachability‑aware set ops: run set diffs only on reachable vuln findings.
  • SemVer gates: treat “crossing an affected range” as a boolean boundary; patch bumps inside a safe range don’t alert.
  • VEX merge logic: vendor or internal VEX that says not_affected suppresses noise unless KEV contradicts.
  • EPSS‑weighted priority: rank surfaced diffs by latest EPSS; KEV always escalates to top.
  • Policy overlays: org rules (e.g., “block any KEV,” “warn if EPSS > 0.7”) applied last.

Example (why it’s quieter, but safer)

  • OpenSSL 3.0.10 → 3.0.11 with VEX not_affected for a CVE: Smart‑Diff marks risk down and closes the prior alert.
  • A transitive dev dependency changes with no runtime path: Smart‑Diff logs only, no red flag.

Implementation plan (Stella Ops‑ready)

1) Inputs

  • SBOM (CycloneDX/SPDX) old vs new
  • VEX (OpenVEX/CycloneDX VEX)
  • Vuln feeds (NVD, vendor), CISA KEV, EPSS
  • Reachability traces (per language analyzers)

2) Normalize

  • Map all deps to purl, normalize versions, index CVEs → affected ranges.
  • Ingest VEX and attach to CVE ↔ component with clear status precedence.

3) Build graphs

  • Generate/refresh AffectedGraph per build: entrypoints → call stacks → vulnerable symbols.
  • Tag each finding with {reachable?, vex_status, kev?, epss, policy_flags}.

4) Diff

  • Compute DiffSet between builds for:

    • Reachable findings
    • VEX statuses
    • Version/range crossings
    • Policy/KEV/EPSS gates

5) Prioritize & suppress

  • Drop items that are unreachable AND not_affected.
  • Collapse patch‑level churn unless KEV‑listed.
  • Sort remaining by KEV first, then EPSS, then runtime blast‑radius (fan‑in/fan‑out).

6) Evidence

  • Attach EvidenceLink to each surfaced change:

    • VEX doc (line/ID)
    • KEV entry
    • EPSS score + timestamp
    • Reachability call stack (top 1‑3 paths)

7) UX

  • Pipeline‑first: output a Smart‑Diff report JSON + concise CLI table:

    • risk ↑/↓, reason (reachability/VEX/KEV/EPSS), component@version, CVE, one example call‑stack.

  • UI is an explainer: expand to full stack, VEX note, KEV link, and “minimum safe change” suggestion.

Module sketch (your stack)

  • Services: Sbomer.Diff, Vexer.Merge, Scanner.Reachability, Feedser.KEV/EPSS, Policy.Engine, SmartDiff.Service

  • Store: PostgreSQL (SoR), Valkey cache (ephemeral). Tables: components, cves, vex_entries, reachability_edges, smartdiff_events, evidence_links.

  • APIs:

    • POST /smartdiff/compare → returns filtered diff + priorities
    • GET /smartdiff/:id/evidence → links to VEX/KEV/EPSS + trace

  • CI usage: stella smart-diff --old sbomA.json --new sbomB.json --vex vex.json --out smartdiff.json

Guardrails (to keep it deterministic)

  • Freeze feed snapshots per run (hash KEV/EPSS CSVs + VEX docs).
  • Version the merge rules (VEX precedence + policy) and emit in the report header.
  • Log the exact semver comparisons that triggered/exempted an alert.

If you want, I can draft the Postgres schema, the .NET 10 DTOs for DiffSet and AffectedGraph, and a CLI prototype (stella smart-diff) you can drop into your pipeline. Noted: the services are Concelier (feeds: KEV/EPSS/NVD/vendor snapshots) and Excititor (VEX merge + status resolution). I’ll use those names going forward.

Below is a product + business analysis implementation spec that a developer can follow to build the Smart‑Diff capability you described.

1) Product objective

Problem

Classic SBOM/VEX diffs are noisy: they surface all dependency/CVE churn, even when nothing changes in actual exploitable risk.

Goal

Produce a Smart‑Diff report between two builds/releases that highlights only changes that materially impact security risk, with evidence attached.

Success criteria

  • Noise reduction: >80% fewer diff items vs raw SBOM diff for typical builds (measured by count).
  • No missed “high-risk flips”: any change that creates or removes a reachable vulnerable path must appear.
  • Traceability: every surfaced Smart‑Diff item has at least one evidence link (VEX entry, reachability trace, KEV reference, feed snapshot hash, scanner output).

2) Scope

In scope (MVP)

  • Compare two “build snapshots”: {SBOM, VEX, reachability traces, vuln feed snapshot, policy snapshot}

  • Detect & report these change types:

    1. Reachability flips (reachable ↔ unreachable)
    2. VEX status changes (e.g., affected → not_affected)
    3. Version crosses vuln boundary (safe ↔ affected range)
    4. KEV/EPSS/policy gate flips (e.g., becomes KEV-listed)

  • Suppress noise using explicit rules (see section 6)

  • Output:

    • JSON report for CI
    • concise CLI output (table)
    • optional UI list view (later)

Out of scope (for now)

  • Full remediation planning / patch PR automation
  • Cross-repo portfolio aggregation (doable later)
  • Advanced exploit intelligence beyond KEV/EPSS

3) Key definitions (developers must implement these exactly)

3.1 Finding

A “finding” is a tuple:

FindingKey = (component_purl, component_version, cve_id)

…and includes computed fields:

  • reachable: bool | unknown
  • vex_status: enum (see 3.3)
  • in_affected_range: bool | unknown
  • kev: bool
  • epss_score: float | null
  • policy_flags: set<string>
  • evidence_links: list<EvidenceLink>

3.2 Material risk change (Smart‑Diff item)

A change is “material” if it changes the computed RiskState for any FindingKey or creates/removes a FindingKey that is in-scope after suppression rules.

3.3 VEX status vocabulary

Normalize all incoming VEX statuses into a fixed internal enum:

  • AFFECTED
  • NOT_AFFECTED
  • FIXED
  • UNDER_INVESTIGATION
  • UNKNOWN (no statement or unparseable)

Note: Use OpenVEX/CycloneDX VEX mappings, but internal logic must operate on the above set.

4) System context and responsibilities

You already have a modular setup. Developers should implement Smart‑Diff as a pipeline over these components:

Components (names aligned to your system)

  • Sbomer

    • Ingest SBOM(s), normalize to purl/version graph

  • Scanner.Reachability

    • Produce reachability traces: entrypoints → call paths → vulnerable symbol/sink

  • Concelier

    • Fetch + snapshot vulnerability intelligence (NVD/vendor/OSV as applicable), CISA KEV, EPSS
    • Provide feed snapshot identifiers (hashes) per run

  • Excititor

    • Ingest and merge VEX sources
    • Resolve a final vex_status per (component, cve)
    • Provide precedence + explanation

  • Policy.Engine

    • Evaluate org rules against a computed finding (e.g., “block if KEV”)

  • SmartDiff.Service

    • Compute risk states for “old” and “new”
    • Diff them
    • Suppress noise
    • Rank + output report with evidence

5) Developer deliverables

Deliverable A: Smart‑Diff computation library

A deterministic library that takes:

  • OldSnapshot and NewSnapshot (see section 7)
  • returns a SmartDiffReport

Deliverable B: Service endpoint

POST /smartdiff/compare returns report JSON.

Deliverable C: CLI command

stella smart-diff --old <dir|file> --new <dir|file> [--policy policy.json] --out smartdiff.json

6) Smart‑Diff rules

Developers must implement these as explicit, testable rule functions.

6.1 Suppression rules (noise filters)

A finding is suppressed if ALL apply:

  1. reachable == false (or unknown treated as false only if you explicitly decide; recommended: unknown is not suppressible)
  2. vex_status == NOT_AFFECTED
  3. kev == false
  4. no policy requires it (e.g., “report all vuln findings” override)

Patch churn suppression

  • If a component version changes but:

    • in_affected_range remains false in both versions, AND
    • no KEV/policy flag flips,
    • then suppress (don’t surface).

Dev/test dependency suppression (optional if you already tag scopes)

  • If SBOM scope indicates dev/test AND reachable == false, suppress.
  • If reachability is unknown, do not suppress by scope alone (avoid false negatives).

6.2 Material change detection rules

Surface a Smart‑Diff item when any of the following changes between old and new:

Rule R1: Reachability flip

  • reachable changes: false → true (risk ↑) or true → false (risk ↓)
  • Include at least one call path as evidence if reachable is true.

Rule R2: VEX status flip

  • vex_status changes meaningfully:

    • AFFECTED ↔ NOT_AFFECTED
    • UNDER_INVESTIGATION → NOT_AFFECTED etc.

  • Changes involving UNKNOWN should be shown but ranked lower unless KEV.

Rule R3: Affected range boundary

  • in_affected_range flips:

    • false → true (risk ↑)
    • true → false (risk ↓)

  • This is the main guard against patch churn noise.

Rule R4: Intelligence / policy flip

  • kev changes false → true or epss_score crosses a configured threshold
  • any policy_flag changes severity (warn → block)

7) Snapshot contract (what Smart‑Diff compares)

Define a stable internal format:

{
  "snapshot_id": "build-2025.12.14+sha.abc123",
  "created_at": "2025-12-14T12:34:56Z",
  "sbom": { "...": "CycloneDX or SPDX raw" },
  "vex_documents": [ { "...": "OpenVEX/CycloneDX VEX raw" } ],
  "reachability": {
    "analyzer": "java-callgraph@1.2.0",
    "entrypoints": ["com.app.Main#main"],
    "paths": [
      {
        "component_purl": "pkg:maven/org.example/foo@1.2.3",
        "cve": "CVE-2024-1234",
        "sink": "org.example.foo.VulnClass#vulnMethod",
        "callstack": ["...", "..."]
      }
    ]
  },
  "concelier_feed_snapshot": {
    "kev_hash": "sha256:...",
    "epss_hash": "sha256:...",
    "vuln_db_hash": "sha256:..."
  },
  "policy_snapshot": { "policy_hash": "sha256:...", "rules": [ ... ] }
}

Implementation note

  • SBOM/VEX can remain “raw”, but you must also build normalized indexes (in-memory or stored) for diffing.

8) Data normalization requirements

8.1 Component identity

  • Use purl as canonical component ID.
  • Normalize casing, qualifiers, and version string normalization per ecosystem.

8.2 Vulnerability identity

  • Use CVE-* as primary key where available.
  • If you ingest OSV IDs too, map them to CVE when possible but keep OSV ID in evidence.

8.3 Affected range evaluation

Implement: bool? IsVersionInAffectedRange(version, affectedRanges)

Return null (unknown) if version cannot be parsed or range semantics are unknown.

9) Excititor: VEX merge requirements

Developers should implement Excititor as a deterministic resolver:

9.1 Inputs

  • List of VEX documents, each with metadata:

    • source (vendor/internal/scanner)
    • issued_at
    • signature/attestation info (if present)

9.2 Output

For each (component_purl, cve_id):

  • final_status
  • winning_statement_id
  • precedence_reason
  • all_statements[] (for audit)

9.3 Precedence rules (recommendation)

Implement as ordered priority (highest wins), unless overridden by your org:

  1. Internal signed VEX (security team attested)
  2. Vendor signed VEX
  3. Internal unsigned VEX
  4. Scanner/VEX-like annotations
  5. None → UNKNOWN

Conflict handling:

  • If two same-priority statements disagree, pick newest by issued_at, but record conflict and surface it as a low-priority Smart‑Diff meta-item (optional).

10) Concelier: feed snapshot requirements

Concelier must provide deterministic inputs to Smart‑Diff.

10.1 What Concelier stores

  • KEV list snapshot
  • EPSS snapshot
  • Vulnerability database snapshot (your choice: NVD mirror, OSV, vendor advisories)

10.2 Required APIs (internal)

  • GET /concelier/snapshots/latest
  • GET /concelier/snapshots/{hash}
  • GET /concelier/kev/{snapshotHash}/is_listed?cve=CVE-...
  • GET /concelier/epss/{snapshotHash}/score?cve=CVE-...

10.3 Determinism

Smart‑Diff report must include the snapshot hashes used, so the result can be reproduced.

11) RiskState computation (core dev logic)

Implement a pure function:

RiskState ComputeRiskState(FindingKey key, Snapshot snapshot)

Inputs used

  • SBOM: to confirm component exists, scope, runtime path
  • Concelier feeds: KEV, EPSS, affected ranges
  • Excititor: VEX status
  • Reachability analyzer output
  • Policy engine: flags based on org rules

Output

{
  "finding_key": { "purl": "...", "version": "...", "cve": "..." },
  "reachable": true,
  "vex_status": "AFFECTED",
  "in_affected_range": true,
  "kev": false,
  "epss": 0.42,
  "policy": {
    "decision": "WARN|BLOCK|ALLOW",
    "flags": ["epss_over_0_4"]
  },
  "evidence": [
    { "type": "reachability_trace", "ref": "trace:abc", "detail": "short call stack..." },
    { "type": "vex", "ref": "openvex:doc123#stmt7" },
    { "type": "concelier_snapshot", "ref": "sha256:..." }
  ]
}

12) Diff engine specification

12.1 Inputs

  • OldRiskStates: map<FindingKey, RiskState>
  • NewRiskStates: map<FindingKey, RiskState>

You build these maps by:

  1. Enumerating candidate findings in each snapshot:

    • from vulnerability matching against SBOM components (affected ranges)
    • plus any VEX statements referencing components

  2. Joining with reachability traces

  3. Resolving status via Excititor

  4. Applying Concelier intelligence + policy

12.2 Diff output types

Return SmartDiffItem with:

  • change_type: ADDED|REMOVED|CHANGED
  • risk_direction: UP|DOWN|NEUTRAL
  • reason_codes: [REACHABILITY_FLIP, VEX_FLIP, RANGE_FLIP, KEV_FLIP, POLICY_FLIP, EPSS_THRESHOLD]
  • old_state / new_state
  • priority_score
  • evidence_links[]

12.3 Suppress AFTER diff, not before

Important: compute diff on full sets, then suppress items by rules, because:

  • suppression itself can flip (e.g., VEX becomes not_affected → item disappears, which is meaningful as “risk down”).

13) Priority scoring & ranking

Implement a deterministic score:

Hard ordering

  1. kev == true in new state → top tier
  2. Reachable in new state (reachable == true) → next tier

Numeric scoring (example)

score =
  + 1000 if new.kev
  + 500 if new.reachable
  + 200 if reason includes RANGE_FLIP to affected
  + 150 if VEX_FLIP to AFFECTED
  + 0..100 based on EPSS (epss * 100)
  + policy weight: +300 if decision BLOCK, +100 if WARN

Always include score_breakdown in report for explainability.

14) Evidence requirements (must implement)

Every Smart‑Diff item must include at least one evidence link, and ideally 2–4:

EvidenceLink schema:

{
  "type": "vex|reachability|kev|epss|scanner|sbom|policy",
  "ref": "stable identifier",
  "summary": "one-line human readable",
  "blob_hash": "sha256 of raw evidence payload (optional)"
}

Examples:

  • type=kev: ref is concelier:kev@{snapshotHash}#CVE-2024-1234
  • type=reachability: ref is reach:{snapshotId}:{traceId}
  • type=vex: ref is openvex:{docHash}#statement:{id}

15) API specification

15.1 Compare endpoint

POST /smartdiff/compare

Request:

{
  "old_snapshot_id": "buildA",
  "new_snapshot_id": "buildB",
  "options": {
    "include_suppressed": false,
    "max_items": 200,
    "epss_threshold": 0.7
  }
}

Response:

{
  "report_id": "smartdiff:2025-12-14:xyz",
  "old": { "snapshot_id": "buildA", "feed_hashes": { ... } },
  "new": { "snapshot_id": "buildB", "feed_hashes": { ... } },
  "summary": {
    "risk_up": 3,
    "risk_down": 8,
    "reachable_new": 2,
    "kev_new": 1,
    "suppressed": 143
  },
  "items": [
    {
      "change_type": "CHANGED",
      "risk_direction": "UP",
      "priority_score": 1680,
      "reason_codes": ["REACHABILITY_FLIP","RANGE_FLIP"],
      "finding_key": {
        "purl": "pkg:maven/org.example/foo",
        "version_old": "1.2.3",
        "version_new": "1.2.4",
        "cve": "CVE-2024-1234"
      },
      "old_state": { "...": "RiskState" },
      "new_state": { "...": "RiskState" },
      "evidence": [ ... ]
    }
  ]
}

15.2 Evidence endpoint

GET /smartdiff/{report_id}/evidence/{evidence_ref}

Returns raw stored evidence (or a signed URL if you store blobs elsewhere).

16) CLI behavior

Command:

stella smart-diff \
  --old ./snapshots/buildA \
  --new ./snapshots/buildB \
  --policy ./policy.json \
  --out ./smartdiff.json

CLI output (human):

  • Summary line: risk ↑ 3 | risk ↓ 8 | new reachable 2 | new KEV 1

  • Then top N items sorted by priority, each one line:

    • ↑ REACHABILITY_FLIP foo@1.2.4 CVE-2024-1234 (EPSS 0.42) path: Main→...→vulnMethod

Exit code:

  • 0 if policy decision overall is ALLOW/WARN
  • 2 if any item triggers policy BLOCK in new snapshot (configurable)

17) Storage schema (Postgres) — implementation-ready

You can implement in a single schema to start; split later.

Core tables

snapshots

  • snapshot_id (pk)
  • created_at
  • sbom_hash
  • policy_hash
  • kev_hash
  • epss_hash
  • vuln_db_hash
  • metadata jsonb

components

  • component_id (pk) (internal UUID)
  • snapshot_id (fk)
  • purl
  • version
  • scope (runtime/dev/test/unknown)
  • direct bool
  • indexes on (snapshot_id, purl) and (purl, version)

findings

  • finding_id (pk)
  • snapshot_id (fk)
  • purl
  • version
  • cve
  • reachable bool null
  • vex_status text
  • in_affected_range bool null
  • kev bool
  • epss real null
  • policy_decision text
  • policy_flags text[]
  • index (snapshot_id, purl, cve)

reachability_traces

  • trace_id (pk)
  • snapshot_id (fk)
  • purl
  • cve
  • sink
  • callstack jsonb
  • index (snapshot_id, purl, cve)

vex_statements

  • stmt_id (pk)
  • snapshot_id (fk)
  • purl
  • cve
  • source
  • issued_at
  • status
  • doc_hash
  • raw jsonb
  • index (snapshot_id, purl, cve)

smartdiff_reports

  • report_id (pk)
  • created_at
  • old_snapshot_id
  • new_snapshot_id
  • options jsonb
  • summary jsonb

smartdiff_items

  • item_id (pk)
  • report_id (fk)
  • change_type
  • risk_direction
  • priority_score
  • reason_codes text[]
  • purl
  • cve
  • old_version
  • new_version
  • old_state jsonb
  • new_state jsonb

evidence_links

  • evidence_id (pk)
  • report_id (fk)
  • item_id (fk)
  • type
  • ref
  • summary
  • blob_hash

18) Implementation plan (developer-focused)

Phase 1 — MVP (end-to-end working)

  1. Normalize SBOM

    • Parse CycloneDX/SPDX
    • Build components list with purl + version + scope

  2. Concelier integration

    • Load KEV + EPSS snapshots (even from local files initially)
    • Expose snapshot hashes

  3. Excititor integration

    • Parse OpenVEX/CycloneDX VEX
    • Implement precedence rules and output final_status

  4. Affected range matching

    • For each component, query vulnerability DB snapshot for affected ranges
    • Produce candidate findings (purl, version, cve)

  5. Reachability ingestion

    • Accept reachability JSON traces (even if generated elsewhere initially)
    • Mark reachable=true when trace exists for (purl,cve)

  6. Compute RiskState

    • For each finding compute kev, epss, policy_decision

  7. Diff + suppression + ranking

    • Generate SmartDiffReport

  8. Outputs

    • JSON report + CLI table
    • Store report + items in Postgres

Acceptance tests for Phase 1:

  • Given a known pair of snapshots, Smart‑Diff only includes:

    • reachable vulnerable changes
    • VEX flips
    • affected range boundary flips
    • KEV flips

  • Patch churn not crossing ranges is absent.

Phase 2 — Determinism & evidence hardening

  • Store raw evidence blobs (VEX doc hash, trace payload hash)
  • Ensure feed snapshots are immutable and referenced by hash
  • Add score_breakdown
  • Add conflict surfacing for VEX merge

Phase 3 — Performance & scale

  • Incremental computation (only recompute affected components changed)
  • Cache Concelier lookups by (snapshotHash, cve)
  • Batch range matching queries
  • Add pagination and max_items enforcement

19) Edge cases developers must handle

  1. Reachability unknown

    • If no analyzer output exists, set reachable = null
    • Do not suppress solely based on reachable=null

  2. Version parse failures

    • in_affected_range = null
    • Surface range-related changes only when one side is determinable

  3. Component renamed / purl drift

    • Consider purl normalization rules (namespace casing, qualifiers)
    • If purl changes but is same artifact, treat as new component (unless you implement alias mapping later)

  4. Multiple CVE sources / duplicates

    • Deduplicate by CVE ID per component+version

  5. Conflicting VEX statements

    • Pick winner deterministically, but log conflict evidence

  6. KEV listed but VEX says not affected

    • Still suppress? Recommended:

      • Do not suppress; surface as “KEV listed but VEX not_affected” and rank high (KEV tier)

  7. Policy config changes

    • Treat policy hash difference as a diff dimension; surface “policy flip” items even if underlying vuln unchanged

20) Testing strategy (must implement)

Unit tests

  • SemVer compare + affected range evaluation
  • Excititor precedence resolution
  • Suppression rules (table-driven tests)
  • Priority scoring determinism

Integration tests

  • Build synthetic snapshots:

    • A: vuln present, unreachable, VEX not_affected
    • B: same vuln reachable
    • Assert Smart‑Diff surfaces exactly one item with REACHABILITY_FLIP

  • KEV flip test:

    • Same findings, KEV list changes between Concelier snapshots
    • Assert item surfaces with KEV_FLIP

Regression suite

Keep a folder of snapshot pairs and expected outputs:

  • fixtures/snapA, fixtures/snapB, expected.smartdiff.json

21) What the developer should code first (practical order)

  1. DTOs:

    • Snapshot, Component, VexStatement, ReachTrace, FindingKey, RiskState, SmartDiffItem, SmartDiffReport

  2. Pure functions:

    • NormalizePurl
    • IsVersionInAffectedRange
    • ResolveVexStatus (Excititor)
    • ComputeRiskState
    • DiffRiskStates
    • ApplySuppression
    • ScoreAndRank

  3. Persistence:

    • store snapshots and computed findings

  4. API + CLI wrappers

If you want, I can also provide:

  • a concrete JSON Schema for SmartDiffReport
  • C# (.NET 10) interfaces + class skeletons for ConcelierClient, ExcititorResolver, and SmartDiffService
  • a fixture set (sample SBOM/VEX/reach traces) to bootstrap the test suite