docs consoliation work

2025-12-24 14:19:46 +02:00
parent 40362de568
commit 5540ce9430
58 changed files with 2671 additions and 1751 deletions
--- a/docs/product-advisories/24-Dec-2025
+++ b/docs/product-advisories/24-Dec-2025
@@ -0,0 +1,413 @@
+# Product Advisory: Deterministic Resolver Architecture
+
+**Date:** 2025-12-24
+**Author:** Architecture Guild
+**Status:** Approved for Implementation
+**Sprint Epoch:** 9100
+**Priority:** P0 (Critical Path for Audit Compliance)
+
+---
+
+## Executive Summary
+
+This advisory defines the architecture for a **unified deterministic resolver** that evaluates vulnerability graphs using only declared evidence edges and produces cryptographically verifiable, reproducible verdicts. The resolver guarantees: **same inputs → same traversal order → same per-node inputs → same verdicts → same output digest**.
+
+This is a foundational capability for:
+- Auditor-friendly verification ("same inputs → same verdicts")
+- Offline replay by vendors/distributors
+- Single-digest comparison across runs
+- CI/CD gate assertions
+
+---
+
+## Problem Statement
+
+Current implementation has strong building blocks but lacks unified orchestration:
+
+| Component | Location | Gap |
+|-----------|----------|-----|
+| Topological Sort | `DeterministicGraphOrderer` | Traversal only; no verdict output |
+| Lattice Evaluation | `TrustLatticeEngine`, `PolicyEvaluator` | No traversal sequence in output |
+| Content Addressing | `ContentAddressedIdGenerator` | Node IDs only; no edge IDs |
+| Purity Analysis | `ProhibitedPatternAnalyzer` | Static only; no runtime enforcement |
+| Digest Computation | Various | No unified "run digest" |
+
+**Result:** Cannot produce a single `ResolutionResult` that captures:
+1. The exact traversal sequence used
+2. Per-node verdicts with individual digests
+3. A composite `FinalDigest` for verification
+
+---
+
+## Solution Architecture
+
+### Core Data Model
+
+```csharp
+// New: src/__Libraries/StellaOps.Resolver/
+
+/// <summary>
+/// Content-addressed node identifier.
+/// Format: sha256(kind || ":" || normalized-key)
+/// </summary>
+public sealed record NodeId(string Value) : IComparable<NodeId>
+{
+    public int CompareTo(NodeId? other) =>
+        string.Compare(Value, other?.Value, StringComparison.Ordinal);
+
+    public static NodeId From(string kind, string normalizedKey) =>
+        new(HexSha256($"{kind}:{normalizedKey}"));
+}
+
+/// <summary>
+/// Content-addressed edge identifier.
+/// Format: sha256(srcID || "->" || edgeKind || "->" || dstID)
+/// </summary>
+public sealed record EdgeId(string Value) : IComparable<EdgeId>
+{
+    public static EdgeId From(NodeId src, string kind, NodeId dst) =>
+        new(HexSha256($"{src.Value}->{kind}->{dst.Value}"));
+}
+
+/// <summary>
+/// Graph edge with content-addressed identity and optional cycle-cut marker.
+/// </summary>
+public sealed record Edge(
+    EdgeId Id,
+    NodeId Src,
+    string Kind,
+    NodeId Dst,
+    JsonElement Attrs,
+    bool IsCycleCut = false);
+
+/// <summary>
+/// Graph node with content-addressed identity.
+/// </summary>
+public sealed record Node(
+    NodeId Id,
+    string Kind,
+    JsonElement Attrs);
+
+/// <summary>
+/// Immutable policy with content-addressed identity.
+/// </summary>
+public sealed record Policy(
+    string Version,
+    JsonElement Rules,
+    string ConstantsDigest);
+
+/// <summary>
+/// Individual verdict with its own content-addressed digest.
+/// </summary>
+public sealed record Verdict(
+    NodeId Node,
+    string Status,
+    JsonElement Evidence,
+    string VerdictDigest);
+
+/// <summary>
+/// Complete resolution result with all digests for verification.
+/// </summary>
+public sealed record ResolutionResult(
+    ImmutableArray<NodeId> TraversalSequence,
+    ImmutableArray<Verdict> Verdicts,
+    string GraphDigest,
+    string PolicyDigest,
+    string FinalDigest);
+```
+
+### Resolver Algorithm
+
+```text
+normalize(graph);           // IDs, fields, ordering
+validate(graph);            // No implicit data, cycles declared
+assert no-ambient-inputs(); // Runtime purity check
+
+order = topoSortWithLexTieBreak(graph); // Uses only evidence edges
+verdicts = []
+
+for node in order:
+  inbound = gatherInboundEvidence(node)  // Pure, no IO
+  verdict = Evaluate(node, inbound, policy)  // Pure, no IO
+  verdicts.append(canonicalize(verdict) with VerdictDigest)
+
+result = {
+  traversal: order,
+  verdicts: verdicts,
+  graphDigest: digest(canonical(graph)),
+  policyDigest: digest(canonical(policy))
+}
+result.finalDigest = digest(canonical(result))
+return result
+```
+
+### Key Invariants
+
+1. **Canonical IDs**: `NodeId = sha256(kind:normalized-key)`, `EdgeId = sha256(src->kind->dst)`
+2. **Canonical Serialization**: Alphabetical keys, sorted arrays, fixed numerics, ISO-8601 Z timestamps
+3. **Fixed Traversal**: Kahn's algorithm with lexicographic tie-breaker (`SortedSet<NodeId>`)
+4. **Explicit Cycles**: Cycles require `IsCycleCut = true` edge; unmarked cycles → invalid graph
+5. **Evidence-Only Evaluation**: Node verdict = f(node, inbound_edges, policy) — no ambient IO
+6. **Stable Outputs**: `(TraversalSequence, Verdicts[], GraphDigest, PolicyDigest, FinalDigest)`
+
+---
+
+## Gap Analysis vs Current Implementation
+
+### Already Implemented (No Work Needed)
+
+| Requirement | Implementation | Location |
+|-------------|---------------|----------|
+| Canonical JSON | RFC 8785 compliant | `CanonicalJsonSerializer`, `Rfc8785JsonCanonicalizer` |
+| Alphabetical key sorting | `StableDictionaryConverter` | `StellaOps.Canonicalization` |
+| ISO-8601 UTC timestamps | `Iso8601DateTimeConverter` | `StellaOps.Canonicalization` |
+| Topological sort with lex tie-break | `SortedSet<string>` | `DeterministicGraphOrderer:134` |
+| K4 Four-Valued Lattice | Complete | `K4Lattice.cs` |
+| VEX Lattice Merging | With traces | `OpenVexStatementMerger` |
+| Merkle Tree Proofs | SHA256-based | `DeterministicMerkleTreeBuilder` |
+| Static Purity Analysis | 16+ patterns | `ProhibitedPatternAnalyzer` |
+| Injected Timestamp | Context-based | `PolicyEvaluationContext.Now` |
+| Graph Content Hash | SHA256 | `DeterministicGraphOrderer.ComputeCanonicalHash()` |
+
+### Gaps to Implement
+
+| Gap | Priority | Sprint | Description |
+|-----|----------|--------|-------------|
+| Unified Resolver | P0 | 9100.0001.0001 | Single entry point producing `ResolutionResult` |
+| Cycle-Cut Edges | P1 | 9100.0001.0002 | `IsCycleCut` edge property; validation |
+| EdgeId | P2 | 9100.0001.0003 | Content-addressed edge identifiers |
+| FinalDigest | P1 | 9100.0002.0001 | Composite run-level digest |
+| Per-Node VerdictDigest | P2 | 9100.0002.0002 | Individual verdict digests |
+| Runtime Purity | P1 | 9100.0003.0001 | Runtime enforcement beyond static analysis |
+| Graph Validation + NFC | P3 | 9100.0003.0002 | Pre-traversal validation; NFC normalization |
+
+---
+
+## Implementation Phases
+
+### Phase 1: Core Resolver Package (Sprint 9100.0001.*)
+
+**Goal:** Create `StellaOps.Resolver` library with unified resolver pattern.
+
+1. **Sprint 9100.0001.0001** — Core Resolver
+   - `DeterministicResolver` class
+   - `ResolutionResult` record
+   - Integration with existing `DeterministicGraphOrderer`
+   - Integration with existing `TrustLatticeEngine`
+
+2. **Sprint 9100.0001.0002** — Cycle-Cut Edges
+   - `IsCycleCut` property on edges
+   - Cycle validation (unmarked cycles → error)
+   - Graph validation before traversal
+
+3. **Sprint 9100.0001.0003** — Content-Addressed EdgeId
+   - `EdgeId` record
+   - Edge content addressing
+   - Merkle tree inclusion of edges
+
+### Phase 2: Digest Infrastructure (Sprint 9100.0002.*)
+
+**Goal:** Implement comprehensive digest chain for verification.
+
+1. **Sprint 9100.0002.0001** — FinalDigest
+   - Composite digest computation
+   - `sha256(canonical({graphDigest, policyDigest, verdicts[]}))`
+   - Integration with attestation system
+
+2. **Sprint 9100.0002.0002** — Per-Node VerdictDigest
+   - Verdict-level content addressing
+   - Drill-down debugging ("which node changed?")
+   - Delta detection between runs
+
+### Phase 3: Purity & Validation (Sprint 9100.0003.*)
+
+**Goal:** Harden determinism guarantees with runtime enforcement.
+
+1. **Sprint 9100.0003.0001** — Runtime Purity Enforcement
+   - Runtime guards beyond static analysis
+   - Dependency injection shims for ambient services
+   - Test harness for purity verification
+
+2. **Sprint 9100.0003.0002** — Graph Validation & NFC
+   - Pre-traversal validation ("no implicit data")
+   - Unicode NFC normalization for string fields
+   - Evidence completeness assertions
+
+---
+
+## Testing Requirements
+
+### Mandatory Test Types
+
+1. **Replay Test**: Same input twice → identical `FinalDigest`
+2. **Permutation Test**: Shuffle input nodes/edges → identical outputs
+3. **Cycle Test**: Introduce cycle → fail unless `IsCycleCut` edge present
+4. **Ambience Test**: Forbid calls to time/env/network during evaluation
+5. **Serialization Test**: Canonicalization changes → predictable digest changes
+
+### Property-Based Tests
+
+```csharp
+// Idempotency
+[Property]
+public void Resolver_SameInput_SameOutput(Graph graph)
+{
+    var result1 = resolver.Run(graph);
+    var result2 = resolver.Run(graph);
+    Assert.Equal(result1.FinalDigest, result2.FinalDigest);
+}
+
+// Order Independence
+[Property]
+public void Resolver_ShuffledInputs_SameOutput(Graph graph, int seed)
+{
+    var shuffled = ShuffleNodesAndEdges(graph, seed);
+    var result1 = resolver.Run(graph);
+    var result2 = resolver.Run(shuffled);
+    Assert.Equal(result1.FinalDigest, result2.FinalDigest);
+}
+
+// Cycle Detection
+[Property]
+public void Resolver_UnmarkedCycle_Throws(Graph graphWithCycle)
+{
+    Assume.That(!graphWithCycle.Edges.Any(e => e.IsCycleCut));
+    Assert.Throws<InvalidGraphException>(() => resolver.Run(graphWithCycle));
+}
+```
+
+---
+
+## Integration Points
+
+### Existing Components to Integrate
+
+| Component | Integration |
+|-----------|-------------|
+| `DeterministicGraphOrderer` | Use for traversal order computation |
+| `TrustLatticeEngine` | Use for K4 lattice evaluation |
+| `CanonicalJsonSerializer` | Use for all serialization |
+| `ContentAddressedIdGenerator` | Extend for EdgeId and VerdictId |
+| `ProhibitedPatternAnalyzer` | Combine with runtime guards |
+| `PolicyEvaluationContext` | Use for injected inputs |
+
+### New Modules
+
+| Module | Purpose |
+|--------|---------|
+| `StellaOps.Resolver` | Core resolver library |
+| `StellaOps.Resolver.Testing` | Test fixtures and harnesses |
+| `StellaOps.Resolver.Attestation` | Integration with proof chain |
+
+---
+
+## Success Criteria
+
+1. **Unified API**: Single `resolver.Run(graph)` produces complete `ResolutionResult`
+2. **Deterministic**: 100% pass rate on replay/permutation/cycle tests
+3. **Auditable**: `FinalDigest` enables single-value verification
+4. **Performant**: No more than 10% overhead vs current fragmented approach
+5. **Documented**: Full API documentation and integration guide
+
+---
+
+## Appendix: C# Reference Implementation
+
+```csharp
+public sealed class DeterministicResolver
+{
+    private readonly Policy _policy;
+    private readonly string _policyDigest;
+    private readonly DeterministicGraphOrderer _orderer;
+    private readonly TrustLatticeEngine _lattice;
+    private readonly CanonicalJsonSerializer _serializer;
+
+    public DeterministicResolver(
+        Policy policy,
+        DeterministicGraphOrderer orderer,
+        TrustLatticeEngine lattice,
+        CanonicalJsonSerializer serializer)
+    {
+        _policy = Canon(policy);
+        _policyDigest = HexSha256(serializer.SerializeToBytes(_policy));
+        _orderer = orderer;
+        _lattice = lattice;
+        _serializer = serializer;
+    }
+
+    public ResolutionResult Run(EvidenceGraph graph)
+    {
+        // 1. Canonicalize and validate
+        var canonical = _orderer.Canonicalize(graph);
+        ValidateCycles(canonical);
+        EnsureNoAmbientInputs();
+
+        // 2. Traverse in deterministic order
+        var traversal = canonical.Nodes
+            .Select(n => NodeId.Parse(n.Id))
+            .ToImmutableArray();
+
+        // 3. Evaluate each node purely
+        var verdicts = new List<Verdict>(traversal.Length);
+        foreach (var nodeId in traversal)
+        {
+            var node = GetNode(canonical, nodeId);
+            var inbound = GatherInboundEvidence(canonical, nodeId);
+            var verdict = EvaluatePure(node, inbound, _policy);
+            var verdictBytes = _serializer.SerializeToBytes(verdict);
+            verdicts.Add(verdict with { VerdictDigest = HexSha256(verdictBytes) });
+        }
+
+        // 4. Compute digests
+        var verdictsArray = verdicts.ToImmutableArray();
+        var resultWithoutFinal = new ResolutionResult(
+            traversal,
+            verdictsArray,
+            canonical.ContentHash,
+            _policyDigest,
+            "");
+
+        var finalBytes = _serializer.SerializeToBytes(resultWithoutFinal);
+        return resultWithoutFinal with { FinalDigest = HexSha256(finalBytes) };
+    }
+
+    private void ValidateCycles(CanonicalGraph graph)
+    {
+        // Detect cycles; require IsCycleCut for each
+        var cycles = DetectCycles(graph);
+        var unmarked = cycles.Where(c => !c.CutEdge.IsCycleCut).ToList();
+        if (unmarked.Any())
+        {
+            throw new InvalidGraphException(
+                $"Graph contains {unmarked.Count} unmarked cycle(s). " +
+                "Add IsCycleCut=true to break cycles.");
+        }
+    }
+
+    private Verdict EvaluatePure(Node node, InboundEvidence evidence, Policy policy)
+    {
+        // Pure evaluation: no IO, no ambient state
+        return _lattice.Evaluate(node, evidence, policy);
+    }
+}
+```
+
+---
+
+## Related Documents
+
+- `docs/modules/attestor/proof-chain-specification.md`
+- `docs/modules/policy/architecture.md`
+- `docs/testing/testing-strategy-models.md`
+- `src/Policy/__Libraries/StellaOps.Policy/TrustLattice/K4Lattice.cs`
+- `src/Scanner/__Libraries/StellaOps.Scanner.Reachability/Ordering/DeterministicGraphOrderer.cs`
+
+---
+
+## Approval
+
+| Role | Name | Date | Decision |
+|------|------|------|----------|
+| Architecture Lead | — | 2025-12-24 | Approved |
+| Policy Guild Lead | — | 2025-12-24 | Approved |
+| Scanner Guild Lead | — | 2025-12-24 | Approved |