docs consolidation and others

2026-01-06 19:02:21 +02:00
parent d7bdca6d97
commit 4789027317
849 changed files with 16551 additions and 66770 deletions
--- a/docs/modules/reach-graph/guides/function-level-evidence.md
+++ b/docs/modules/reach-graph/guides/function-level-evidence.md
@@ -0,0 +1,535 @@
+# Function-Level Evidence Guide
+
+_Last updated: 2025-12-13. Owner: Docs Guild._
+
+This guide documents the cross-module function-level evidence chain that enables provable reachability claims. It covers the schema, identifiers, API usage, CLI commands, and integration patterns for Scanner, Signals, Policy, and Replay.
+
+---
+
+## 1. Overview
+
+StellaOps implements a **function-level evidence chain** that anchors every vulnerability finding to immutable identifiers (`code_id`, `symbol_id`, `graph_hash`) enabling:
+
+- **Provable reachability:** Deterministic call-path evidence from entry points to vulnerable functions.
+- **Stripped binary support:** `code_id` + `code_block_hash` provides identity when symbols are absent.
+- **Evidence replay:** Sealed artifacts with DSSE attestation allow offline verification.
+- **Cross-module linking:** Scanner -> Signals -> Policy -> VEX -> UI/CLI evidence chain.
+
+### 1.1 Core Identifiers
+
+| Identifier | Format | Purpose | Example |
+|------------|--------|---------|---------|
+| `symbol_id` | `sym:{lang}:{base64url}` | Canonical function identity | `sym:java:R3JlZXRpbmc...` |
+| `code_id` | `code:{lang}:{base64url}` | Identity for name-less code blocks | `code:binary:YWJjZGVm...` |
+| `graph_hash` | `blake3:{hex}` | Content-addressable graph identity | `blake3:a1b2c3d4e5f6...` |
+| `symbol_digest` | `sha256:{hex}` | Hash of symbol_id for edge linking | `sha256:e5f6a7b8c9d0...` |
+| `build_id` | `gnu-build-id:{hex}` | ELF/PE debug identifier | `gnu-build-id:5f0c7c3c...` |
+
+### 1.2 Evidence Chain Flow
+
+```
+Scanner -> richgraph-v1 -> Signals -> Scoring -> Policy -> VEX -> UI/CLI
+   |            |             |          |         |        |        |
+   |            |             |          |         |        |        +-- stella graph explain
+   |            |             |          |         |        +-- OpenVEX with call-path proofs
+   |            |             |          |         +-- Policy gates + reachability.state
+   |            |             |          +-- Lattice state + confidence + riskScore
+   |            |             +-- Runtime facts + static paths
+   |            +-- BLAKE3 graph_hash + DSSE attestation
+   +-- code_id, symbol_id, build_id per node
+```
+
+---
+
+## 2. Schema Reference
+
+### 2.1 SymbolID Construction
+
+Per-language canonical tuple format (NUL-separated, then SHA-256 -> base64url):
+
+| Language | Tuple Components | Example |
+|----------|------------------|---------|
+| Java | `{package}\0{class}\0{method}\0{descriptor}` | `com.example\0Foo\0bar\0(Ljava/lang/String;)V` |
+| .NET | `{assembly}\0{namespace}\0{type}\0{member_signature}` | `MyApp\0Controllers\0UserController\0GetById(int)` |
+| Go | `{module}\0{package}\0{receiver}\0{func}` | `github.com/user/repo\0handler\0*Server\0Handle` |
+| Node | `{pkg_or_path}\0{export_path}\0{kind}` | `lodash\0get\0function` |
+| Binary | `{file_hash}\0{section}\0{addr}\0{name}\0{linkage}\0{code_block_hash?}` | `sha256:abc...\0.text\00x401000\0ssl3_read\0global\0` |
+| Python | `{pkg_or_path}\0{module}\0{qualified_name}` | `requests\0api\0get` |
+| Ruby | `{gem_or_path}\0{module}\0{method}` | `rails\0ActionController::Base\0render` |
+| PHP | `{composer_pkg}\0{namespace}\0{qualified_name}` | `symfony/http-kernel\0Kernel\0handle` |
+
+### 2.2 CodeID Construction
+
+For stripped binaries or name-less code blocks:
+
+```
+code:{lang}:{base64url_sha256(format + file_hash + addr + length + section + code_block_hash)}
+```
+
+Example for stripped ELF:
+```
+code:binary:YWJjZGVmZ2hpamtsbW5vcHFyc3R1dnd4eXo
+```
+
+### 2.3 Graph Node Schema
+
+Each node in a richgraph-v1 document includes:
+
+```json
+{
+  "id": "sym:java:R3JlZXRpbmdTZXJ2aWNl...",
+  "symbol_id": "sym:java:R3JlZXRpbmdTZXJ2aWNl...",
+  "code_id": "code:java:...",
+  "lang": "java",
+  "kind": "method",
+  "display": "com.example.GreetingService.greet(String)",
+  "purl": "pkg:maven/com.example/greeting-service@1.0.0",
+  "build_id": "gnu-build-id:5f0c7c3c...",
+  "symbol_digest": "sha256:e5f6a7b8...",
+  "code_block_hash": "sha256:deadbeef...",
+  "symbol": {
+    "mangled": null,
+    "demangled": "com.example.GreetingService.greet(String)",
+    "source": "DWARF",
+    "confidence": 0.98
+  },
+  "evidence": ["import", "bytecode"],
+  "attributes": {}
+}
+```
+
+### 2.4 Graph Edge Schema
+
+Edges carry callee `purl` and `symbol_digest` for SBOM correlation:
+
+```json
+{
+  "from": "sym:java:caller...",
+  "to": "sym:java:callee...",
+  "kind": "call",
+  "purl": "pkg:maven/org.apache.logging.log4j/log4j-core@2.14.1",
+  "symbol_digest": "sha256:f1e2d3c4...",
+  "confidence": 0.92,
+  "evidence": ["bytecode", "import"],
+  "candidates": []
+}
+```
+
+### 2.5 Evidence Block Schema
+
+Evidence blocks in Policy/VEX responses cite all relevant identifiers:
+
+```json
+{
+  "evidence": {
+    "graph_hash": "blake3:a1b2c3d4e5f6...",
+    "graph_cas_uri": "cas://reachability/graphs/a1b2c3d4e5f6...",
+    "dsse_uri": "cas://reachability/graphs/a1b2c3d4e5f6....dsse",
+    "path": [
+      {"symbol_id": "sym:java:...", "display": "main()"},
+      {"symbol_id": "sym:java:...", "display": "processRequest()"},
+      {"symbol_id": "sym:java:...", "display": "log4j.error()"}
+    ],
+    "path_length": 3,
+    "confidence": 0.85,
+    "runtime_hits": ["probe:jfr:1234"],
+    "analyzer": {
+      "name": "scanner.java",
+      "version": "1.2.0",
+      "toolchain_digest": "sha256:..."
+    }
+  }
+}
+```
+
+---
+
+## 3. API Usage
+
+### 3.1 Signals Callgraph Ingestion
+
+Submit a callgraph and receive a deterministic `graph_hash`:
+
+```http
+POST /signals/callgraphs
+Authorization: Bearer <token>
+Content-Type: application/json
+
+{
+  "schema": "richgraph-v1",
+  "analyzer": {"name": "scanner.java", "version": "1.2.0"},
+  "nodes": [...],
+  "edges": [...],
+  "roots": [...]
+}
+```
+
+**Response:**
+
+```json
+{
+  "graphHash": "blake3:a1b2c3d4e5f6...",
+  "casUri": "cas://reachability/graphs/a1b2c3d4e5f6...",
+  "dsseUri": "cas://reachability/graphs/a1b2c3d4e5f6....dsse",
+  "nodeCount": 1247,
+  "edgeCount": 3891
+}
+```
+
+### 3.2 Signals Runtime Facts
+
+Submit runtime observations with `code_id` anchors:
+
+```http
+POST /signals/runtime-facts/ndjson?scanId=scan-123&imageDigest=sha256:abc123
+Authorization: Bearer <token>
+Content-Type: application/x-ndjson
+Content-Encoding: gzip
+
+{"symbolId":"sym:java:...","codeId":"code:java:...","hitCount":47,"loaderBase":"0x7f...","processId":1234,"observedAt":"2025-12-13T10:00:00Z"}
+{"symbolId":"sym:java:...","codeId":"code:java:...","hitCount":12,"loaderBase":"0x7f...","processId":1234,"observedAt":"2025-12-13T10:00:01Z"}
+```
+
+**Response:**
+
+```json
+{
+  "accepted": 128,
+  "duplicates": 2,
+  "evidenceUri": "cas://reachability/runtime/sha256:xyz789..."
+}
+```
+
+### 3.3 Fetch Reachability Facts
+
+Query reachability state for a subject:
+
+```http
+GET /signals/facts/{subjectKey}
+Authorization: Bearer <token>
+```
+
+**Response:**
+
+```json
+{
+  "subjectKey": "scan:123:pkg:maven/log4j:2.14.1:CVE-2021-44228",
+  "metadata": {
+    "fact": {
+      "digest": "sha256:abc123...",
+      "version": 3
+    }
+  },
+  "states": [
+    {
+      "symbol": "sym:java:...",
+      "latticeState": "CR",
+      "bucket": "runtime",
+      "confidence": 0.92,
+      "score": 0.78,
+      "path": ["sym:java:main...", "sym:java:process...", "sym:java:log4j..."],
+      "evidence": {
+        "static": {"graphHash": "blake3:...", "pathLength": 3, "confidence": 0.85},
+        "runtime": {"probeId": "probe:jfr:1234", "hitCount": 47, "observedAt": "2025-12-13T10:00:00Z"}
+      }
+    }
+  ],
+  "score": 0.78,
+  "aggregateTier": "T2",
+  "riskScore": 0.65
+}
+```
+
+### 3.4 Policy Findings with Reachability Evidence
+
+```http
+GET /api/policy/findings/{policyId}/{findingId}/explain?mode=verbose
+Authorization: Bearer <token>
+```
+
+**Response (excerpt):**
+
+```json
+{
+  "findingId": "P-7:S-42:pkg:maven/log4j@2.14.1:CVE-2021-44228",
+  "reachability": {
+    "state": "CR",
+    "confidence": 0.92,
+    "evidence": {
+      "graph_hash": "blake3:a1b2c3d4...",
+      "path": [
+        {"symbol_id": "sym:java:...", "display": "main()"},
+        {"symbol_id": "sym:java:...", "display": "Logger.error()"}
+      ],
+      "runtime_hits": 47,
+      "fact_digest": "sha256:abc123..."
+    }
+  },
+  "steps": [
+    {"rule": "reachability_gate", "state": "CR", "allowed": true},
+    {"rule": "severity_baseline", "severity": {"normalized": "Critical", "score": 10.0}}
+  ]
+}
+```
+
+---
+
+## 4. CLI Usage
+
+### 4.1 Graph Explain Command
+
+View the call path and evidence for a finding:
+
+```bash
+stella graph explain --finding "pkg:maven/log4j@2.14.1:CVE-2021-44228" --scan-id scan-123
+
+# Output:
+Finding: CVE-2021-44228 in pkg:maven/log4j@2.14.1
+Reachability: CONFIRMED_REACHABLE (CR)
+Confidence: 0.92
+Graph Hash: blake3:a1b2c3d4e5f6...
+
+Call Path (3 hops):
+  1. main() [sym:java:R3JlZXRpbmcuLi4=]
+     -> processRequest() [direct call]
+  2. processRequest() [sym:java:cHJvY2Vzcy4uLg==]
+     -> Logger.error() [virtual call]
+  3. Logger.error() [sym:java:bG9nNGouLi4=]
+     [VULNERABLE: CVE-2021-44228]
+
+Runtime Evidence:
+  - JFR probe hit: 47 times
+  - Last observed: 2025-12-13T10:00:00Z
+
+DSSE Attestation: cas://reachability/graphs/a1b2c3d4....dsse
+```
+
+### 4.2 Graph Export Command
+
+Export a reachability graph for offline analysis:
+
+```bash
+stella graph export --scan-id scan-123 --output ./evidence-bundle/
+
+# Creates:
+# ./evidence-bundle/richgraph-v1.json       # Canonical graph
+# ./evidence-bundle/richgraph-v1.json.dsse  # DSSE envelope
+# ./evidence-bundle/meta.json               # Metadata
+# ./evidence-bundle/runtime-facts.ndjson    # Runtime observations
+```
+
+### 4.3 Graph Verify Command
+
+Verify a graph's DSSE signature and Rekor inclusion:
+
+```bash
+stella graph verify --graph ./evidence-bundle/richgraph-v1.json \
+                    --dsse ./evidence-bundle/richgraph-v1.json.dsse \
+                    --rekor-log
+
+# Output:
+Graph Hash: blake3:a1b2c3d4e5f6...
+DSSE Signature: VALID (key: scanner-signing-2025)
+Rekor Entry: 12345678 (verified)
+Timestamp: 2025-12-13T09:30:00Z
+```
+
+---
+
+## 5. OpenVEX Integration
+
+### 5.1 OpenVEX with Reachability Evidence
+
+When Policy emits VEX decisions, reachability evidence is included:
+
+```json
+{
+  "@context": "https://openvex.dev/ns/v0.2.0",
+  "@id": "https://stellaops.example/vex/2025-12-13/001",
+  "author": "StellaOps Policy Engine",
+  "timestamp": "2025-12-13T10:00:00Z",
+  "version": 1,
+  "statements": [
+    {
+      "vulnerability": {"@id": "CVE-2021-44228"},
+      "products": [{"@id": "pkg:oci/myapp@sha256:abc123..."}],
+      "status": "affected",
+      "justification": "vulnerable_code_in_container",
+      "impact_statement": "Vulnerable Log4j method reachable from main entry point.",
+      "action_statement": "Upgrade to log4j 2.17.1 or later.",
+      "stellaops:reachability": {
+        "state": "CR",
+        "confidence": 0.92,
+        "graph_hash": "blake3:a1b2c3d4e5f6...",
+        "path_length": 3,
+        "evidence_uri": "cas://reachability/graphs/a1b2c3d4..."
+      }
+    }
+  ]
+}
+```
+
+### 5.2 VEX "not_affected" with Unreachability Evidence
+
+When code is provably unreachable:
+
+```json
+{
+  "statements": [
+    {
+      "vulnerability": {"@id": "CVE-2023-XXXXX"},
+      "products": [{"@id": "pkg:oci/myapp@sha256:abc123..."}],
+      "status": "not_affected",
+      "justification": "vulnerable_code_not_in_execute_path",
+      "impact_statement": "Vulnerable function not reachable from any entry point.",
+      "stellaops:reachability": {
+        "state": "CU",
+        "confidence": 0.88,
+        "graph_hash": "blake3:d4e5f6a7b8c9...",
+        "evidence_uri": "cas://reachability/graphs/d4e5f6a7b8c9...",
+        "runtime_observation_window": "72h",
+        "runtime_hits": 0
+      }
+    }
+  ]
+}
+```
+
+---
+
+## 6. Replay Manifest v2
+
+### 6.1 Manifest Structure
+
+Replay manifests now enforce BLAKE3 hashing and CAS registration:
+
+```json
+{
+  "schema": "stellaops.replay.manifest@v2",
+  "subject": "scan:123",
+  "generatedAt": "2025-12-13T10:00:00Z",
+  "hashAlg": "blake3",
+  "artifacts": [
+    {
+      "kind": "richgraph",
+      "uri": "cas://reachability/graphs/blake3:a1b2c3d4e5f6...",
+      "hash": "blake3:a1b2c3d4e5f6...",
+      "dsseUri": "cas://reachability/graphs/blake3:a1b2c3d4e5f6....dsse"
+    },
+    {
+      "kind": "runtime-facts",
+      "uri": "cas://reachability/runtime/sha256:xyz789...",
+      "hash": "sha256:xyz789..."
+    },
+    {
+      "kind": "sbom",
+      "uri": "cas://scanner-artifacts/sbom.cdx.json",
+      "hash": "sha256:def456..."
+    }
+  ],
+  "analyzer": {
+    "name": "scanner.java",
+    "version": "1.2.0",
+    "toolchain_digest": "sha256:..."
+  },
+  "code_id_coverage": {
+    "total_symbols": 1247,
+    "with_code_id": 1189,
+    "coverage_pct": 95.3
+  }
+}
+```
+
+### 6.2 Determinism Verification
+
+Replay a manifest to verify determinism:
+
+```bash
+stella replay verify --manifest ./manifest.json --sealed
+
+# Output:
+Manifest: stellaops.replay.manifest@v2
+Subject: scan:123
+Artifacts: 3
+
+Verifying richgraph...
+  Computed: blake3:a1b2c3d4e5f6...
+  Expected: blake3:a1b2c3d4e5f6...
+  Status: MATCH
+
+Verifying runtime-facts...
+  Computed: sha256:xyz789...
+  Expected: sha256:xyz789...
+  Status: MATCH
+
+Verifying sbom...
+  Computed: sha256:def456...
+  Expected: sha256:def456...
+  Status: MATCH
+
+All artifacts verified. Determinism check PASSED.
+```
+
+---
+
+## 7. Module Integration Guide
+
+### 7.1 Scanner -> Signals
+
+Scanner emits richgraph-v1 with `code_id` and `symbol_id`:
+
+1. Scanner analyzes container/artifact
+2. Callgraph generators emit nodes with `symbol_id`, `code_id`, `build_id`
+3. RichGraphWriter canonicalizes (sorted arrays/keys) and computes `graph_hash` (BLAKE3)
+4. DSSE signer wraps canonical JSON
+5. CAS store persists body + envelope
+6. Signals ingestion API receives URI reference
+
+### 7.2 Signals -> Policy
+
+Signals provides reachability facts to Policy:
+
+1. Policy queries `/signals/facts/{subjectKey}`
+2. Response includes `metadata.fact.digest`, `states[]`, `score`
+3. Policy gates check `latticeState` (U, SR, SU, RO, RU, CR, CU, X)
+4. Evidence blocks in findings reference `graph_hash`, `path[]`, `runtime_hits[]`
+
+### 7.3 Policy -> VEX/UI
+
+Policy emits OpenVEX with evidence:
+
+1. VexDecisionEmitter serializes OpenVEX with `stellaops:reachability` extension
+2. UI explain drawer fetches evidence via `/api/policy/findings/{id}/explain`
+3. CLI `stella graph explain` renders call path and attestation refs
+
+---
+
+## 8. CAS Layout Reference
+
+```
+cas://reachability/
+  graphs/
+    {blake3}/                    # Graph body (canonical JSON)
+    {blake3}.dsse                # DSSE envelope
+  edges/
+    {graph_hash}/{bundle_id}     # Edge bundle body (optional)
+    {graph_hash}/{bundle_id}.dsse
+  runtime/
+    {sha256}/                    # Runtime facts NDJSON
+```
+
+---
+
+## 9. Related Documentation
+
+- [Reachability Lattice Model](./lattice.md) - State definitions and join rules
+- [richgraph-v1 Contract](../contracts/richgraph-v1.md) - Schema specification
+- [Evidence Schema](./evidence-schema.md) - Detailed field definitions
+- [Signals API Contract](../api/signals/reachability-contract.md) - API reference
+- [Policy Gates](./policy-gate.md) - Gate configuration
+- [Hybrid Attestation](./hybrid-attestation.md) - Graph and edge-bundle DSSE
+- [Ground Truth Schema](./ground-truth-schema.md) - Test fixture format
+
+---
+
+_Last updated: 2025-12-13. See Sprint 0401 GAP-DOC-008 for change history._