git.stella-ops.org/docs/product-advisories/25-Dec-2025 - Building a Deterministic Verdict Engine.md

Here’s a tight, practical blueprint for evolving Stella Ops’s policy engine into a fully deterministic verdict engine—so the same SBOM + VEX + reachability subgraph ⇒ the exact same, replayable verdict every time, with auditor‑grade trails and signed “delta verdicts.”

Why this matters (quick)

• Reproducibility: auditors can replay any scan and get identical results.
• Trust & scale: cross‑agent consensus via content‑addressed inputs and signed outputs.
• Operational clarity: diffs between builds become crisp, machine‑verifiable artifacts.

Core principles

• Determinism-first: no wall‑clock time, no random iteration order, no network during evaluation.
• Content‑addressing: hash every input (SBOM, VEX docs, reachability subgraph, policy set, rule versions, feed snapshots).
• Declarative state: a compact Scan Manifest lists input hashes + policy bundle hash + engine version.
• Pure evaluation: the verdict function is referentially transparent: Verdict = f(Manifest).

Data artifacts

• Scan Manifest (manifest.jsonc)

  • sbom_sha256, vex_set_sha256[], reach_subgraph_sha256, feeds_snapshot_sha256, policy_bundle_sha256, engine_version, policy_semver, options_hash

• Verdict (verdict.json)

  • canonical JSON (stable key order); includes:

    • risk_score, status (pass/warn/fail), unknowns_count
    • evidence_refs: content IDs for cited VEX statements, nodes/edges from reachability, CVE records, feature‑flags, env‑guards
    • explanations: stable, template‑driven strings (+ machine reasons)

• Delta Verdict (delta.json)

  • computed between two manifests/verdicts:

    • added_findings[], removed_findings[], severity_shift[], unknowns_delta, policy_effects[]

  • signed (DSSE/COSE/JWS), time‑stamped, and linkable to both verdicts

Engine architecture (deterministic path)

1. Normalize inputs

   • SBOM: sort by packageUrl/name@version; resolve aliases; freeze semver comparison rules.
   • VEX: normalize provider → vex_id, product_ref, status (affected, not_affected, …), with source trust score precomputed from a trust registry (strict, versioned).
   • Reachability: store subgraph as adjacency lists sorted by node ID; hash after topological stable ordering.
   • Feeds: lock to a snapshot (timestamp + commit/hash); no live calls.

2. Policy bundle

   • Declarative rules (e.g., lattice/merge semantics), compiled to a canonical IR (e.g., OPA‑Rego → sorted DNF).
   • Merge precedence is explicit (e.g., vendor > distro > internal can be replaced by a lattice‑merge table).
   • Unknowns policy baked in: e.g., fail_if_unknowns > N in prod.

3. Evaluation

   • Build a finding set: (component, vuln, context) tuples with deterministic IDs.
   • Apply lattice‑based VEX merge (proof‑carrying): each suppression must carry an evidence pointer (feature flag off, code path unreachable, patched‑backport proof).
   • Compute final status and risk_score using fixed‑precision math; round rules are part of the bundle.

4. Emit

   • Canonicalize verdict JSON; attach evidence map (content IDs only).
   • Sign verdict; attach as OCI attestation to image/digest.

APIs (minimal but complete)

• POST /evaluate → returns verdict.json + attestation
• POST /delta with {base_verdict, head_verdict} → delta.json (signed)
• GET /replay?manifest_sha= → re‑executes using cached snapshot bundles, returns the same verdict_sha
• GET /evidence/:cid → fetches immutable evidence blobs (offline‑ready)

Storage & indexing

• CAS (content‑addressable store): /evidence/<sha256> for SBOM/VEX/graphs/feeds/policies.
• Verdict registry: keyed by (image_digest, manifest_sha, engine_version).
• Delta ledger: append‑only, signed; supports cross‑agent consensus (multiple engines can co‑sign identical deltas).

UI slices (where it lives)

• Run details → “Verdict” tab: status, risk score, unknowns, top evidence links.
• “Diff” tab: render Delta Verdict (added/removed/changed), with drill‑down to proofs.
• “Replay” button: shows the exact manifest & engine version; one‑click re‑evaluation (offline possible).
• Audit export: zip of manifest.jsonc, verdict.json, delta.json (if any), attestation, and referenced evidence.

Testing & QA (must‑have)

• Golden tests: fixtures of manifests → frozen verdict JSONs (byte‑for‑byte).
• Chaos determinism tests: vary thread counts, env vars, map iteration seeds; assert identical verdicts.
• Cross‑engine round‑trips: two independent builds of the engine produce the same verdict for the same manifest.
• Time‑travel tests: replay older feed snapshots to ensure stability.

Rollout plan

1. Phase 1: Introduce Manifest + canonical verdict format alongside existing policy engine (shadow mode).
2. Phase 2: Make verdicts the first‑class artifact (OCI‑attached); ship UI “Verdict/Diff”.
3. Phase 3: Enforce delta‑gates in CI/CD (risk budgets + exception packs referenceable by content ID).
4. Phase 4: Open consensus mode—accept externally signed identical delta verdicts to strengthen trust.

Notes for Stella modules

• scanner.webservice: keep lattice algorithms here (per your standing rule). Concelier/Excitors “preserve‑prune source.”
• Authority/Attestor: handle DSSE signing, key management, regional crypto profiles (eIDAS/FIPS/GOST/SM).
• Feedser/Vexer: produce immutable snapshot bundles; never query live during evaluation.
• Router/Scheduler: schedule replay jobs; cache manifests to speed up audits.
• Db: Postgres as SoR; Valkey only for ephemeral queues/caches (per your BSD‑only profile).

If you want, I can generate:

• a sample Manifest + Verdict + Delta trio,
• the canonical JSON schema,
• and a .NET 10 reference evaluator (deterministic LINQ pipeline + fixed‑precision math) you can drop into scanner.webservice.