Add signal contracts for reachability, exploitability, trust, and unknown symbols

- Introduced `ReachabilityState`, `RuntimeHit`, `ExploitabilitySignal`, `ReachabilitySignal`, `SignalEnvelope`, `SignalType`, `TrustSignal`, and `UnknownSymbolSignal` records to define various signal types and their properties. - Implemented JSON serialization attributes for proper data interchange. - Created project files for the new signal contracts library and corresponding test projects. - Added deterministic test fixtures for micro-interaction testing. - Included cryptographic keys for secure operations with cosign.
2025-12-05 00:27:00 +02:00
parent b018949a8d
commit 8768c27f30
192 changed files with 27569 additions and 2552 deletions
--- a/docs/modules/policy/contracts/feed-snapshot-thresholds.md
+++ b/docs/modules/policy/contracts/feed-snapshot-thresholds.md
@@ -0,0 +1,317 @@
+# Feed Snapshot Freeze/Staleness Thresholds (SP6)
+
+Status: Draft · Date: 2025-12-04
+Scope: Codify feed snapshot governance including freshness budgets, staleness thresholds, freeze procedures, and validation checks.
+
+## Objectives
+
+- Define maximum age for vulnerability feed snapshots.
+- Establish freeze/thaw procedures for reproducible scans.
+- Set staleness detection and alerting thresholds.
+- Enable deterministic feed state for replay scenarios.
+
+## Freshness Budgets
+
+### Feed Types
+
+| Feed Type | Source | Max Age | Staleness Threshold | Critical Threshold |
+|-----------|--------|---------|---------------------|-------------------|
+| NVD | NIST NVD | 24 hours | 48 hours | 7 days |
+| OSV | OSV.dev | 12 hours | 24 hours | 3 days |
+| GHSA | GitHub | 12 hours | 24 hours | 3 days |
+| Alpine | Alpine SecDB | 24 hours | 48 hours | 7 days |
+| Debian | Debian Security Tracker | 24 hours | 48 hours | 7 days |
+| RHEL | Red Hat OVAL | 24 hours | 72 hours | 7 days |
+| Ubuntu | Ubuntu CVE Tracker | 24 hours | 48 hours | 7 days |
+
+### Definitions
+
+- **Max Age**: Maximum time since last successful sync before warnings
+- **Staleness Threshold**: Time after which scans emit staleness warnings
+- **Critical Threshold**: Time after which scans are blocked without override
+
+## Snapshot States
+
+```
+┌─────────────┐
+│   SYNCING   │ ──────────────────────────────────┐
+└─────────────┘                                   │
+       │                                          │
+       ▼                                          ▼
+┌─────────────┐     ┌─────────────┐     ┌─────────────┐
+│    FRESH    │ ──► │    STALE    │ ──► │  CRITICAL   │
+└─────────────┘     └─────────────┘     └─────────────┘
+       │                   │                   │
+       ▼                   ▼                   ▼
+┌─────────────┐     ┌─────────────┐     ┌─────────────┐
+│   FROZEN    │     │   FROZEN    │     │   FROZEN    │
+│   (fresh)   │     │   (stale)   │     │  (critical) │
+└─────────────┘     └─────────────┘     └─────────────┘
+```
+
+### State Transitions
+
+| Current State | Trigger | New State | Action |
+|---------------|---------|-----------|--------|
+| SYNCING | Sync complete | FRESH | Record snapshot hash |
+| FRESH | Max age exceeded | STALE | Emit warning |
+| STALE | Staleness threshold exceeded | CRITICAL | Block scans |
+| CRITICAL | Manual override | CRITICAL (override) | Log override |
+| Any | Freeze command | FROZEN | Lock snapshot |
+| FROZEN | Thaw command | Previous state | Unlock snapshot |
+
+## Snapshot Schema
+
+```json
+{
+  "id": "nvd-2025-12-04T00-00-00Z",
+  "feed": "nvd",
+  "source": "https://nvd.nist.gov/feeds/json/cve/1.1/nvdcve-1.1-modified.json.gz",
+  "syncedAt": "2025-12-04T00:00:00Z",
+  "hash": {
+    "blake3": "...",
+    "sha256": "..."
+  },
+  "state": "fresh",
+  "advisoryCount": 250000,
+  "lastModified": "2025-12-03T23:45:00Z",
+  "frozen": false,
+  "frozenAt": null,
+  "frozenBy": null,
+  "thresholds": {
+    "maxAge": "PT24H",
+    "staleness": "PT48H",
+    "critical": "P7D"
+  }
+}
+```
+
+## Freeze Procedures
+
+### Manual Freeze
+
+```bash
+# Freeze current snapshot for reproducible scans
+stellaops feed freeze --feed nvd --reason "Audit baseline 2025-Q4"
+
+# Freeze with specific snapshot
+stellaops feed freeze --feed nvd --snapshot-id nvd-2025-12-04T00-00-00Z
+
+# List frozen snapshots
+stellaops feed list --frozen
+```
+
+### Automatic Freeze (Replay Mode)
+
+When recording a scan for replay:
+
+1. Scanner captures current snapshot IDs for all feeds
+2. Snapshot hashes recorded in replay manifest
+3. Replay execution uses frozen snapshots from manifest
+
+```json
+{
+  "replay": {
+    "feeds": {
+      "nvd": {
+        "snapshotId": "nvd-2025-12-04T00-00-00Z",
+        "hash": "b3:...",
+        "state": "frozen"
+      },
+      "osv": {
+        "snapshotId": "osv-2025-12-04T00-00-00Z",
+        "hash": "b3:...",
+        "state": "frozen"
+      }
+    }
+  }
+}
+```
+
+### Thaw Procedures
+
+```bash
+# Thaw a frozen snapshot (resume normal sync)
+stellaops feed thaw --feed nvd --snapshot-id nvd-2025-12-04T00-00-00Z
+
+# Force thaw all frozen snapshots
+stellaops feed thaw --all --force
+```
+
+## Validation Checks
+
+### Pre-Scan Validation
+
+```python
+# scripts/scanner/validate_feeds.py
+def validate_feed_freshness(feed_states: dict) -> ValidationResult:
+    errors = []
+    warnings = []
+
+    for feed, state in feed_states.items():
+        age = datetime.utcnow() - state['syncedAt']
+        thresholds = state['thresholds']
+
+        if age > parse_duration(thresholds['critical']):
+            if not state.get('override'):
+                errors.append(f"{feed}: Critical staleness ({age})")
+        elif age > parse_duration(thresholds['staleness']):
+            warnings.append(f"{feed}: Stale ({age})")
+        elif age > parse_duration(thresholds['maxAge']):
+            warnings.append(f"{feed}: Approaching staleness ({age})")
+
+    return ValidationResult(
+        valid=len(errors) == 0,
+        errors=errors,
+        warnings=warnings
+    )
+```
+
+### Scan Output Metadata
+
+```json
+{
+  "scan": {
+    "id": "scan-12345",
+    "feedState": {
+      "nvd": {
+        "snapshotId": "nvd-2025-12-04T00-00-00Z",
+        "state": "fresh",
+        "age": "PT12H",
+        "hash": "b3:..."
+      },
+      "osv": {
+        "snapshotId": "osv-2025-12-04T00-00-00Z",
+        "state": "stale",
+        "age": "PT36H",
+        "hash": "b3:...",
+        "warning": "Feed approaching staleness threshold"
+      }
+    }
+  }
+}
+```
+
+## API Endpoints
+
+### Feed Status
+
+```http
+GET /api/v1/feeds/status
+```
+
+Response:
+```json
+{
+  "feeds": {
+    "nvd": {
+      "state": "fresh",
+      "snapshotId": "nvd-2025-12-04T00-00-00Z",
+      "syncedAt": "2025-12-04T00:00:00Z",
+      "age": "PT12H",
+      "frozen": false,
+      "thresholds": {
+        "maxAge": "PT24H",
+        "staleness": "PT48H",
+        "critical": "P7D"
+      }
+    }
+  }
+}
+```
+
+### Freeze Feed
+
+```http
+POST /api/v1/feeds/{feed}/freeze
+Content-Type: application/json
+
+{
+  "snapshotId": "nvd-2025-12-04T00-00-00Z",
+  "reason": "Audit baseline"
+}
+```
+
+### Override Staleness
+
+```http
+POST /api/v1/feeds/{feed}/override
+Content-Type: application/json
+
+{
+  "snapshotId": "nvd-2025-12-04T00-00-00Z",
+  "reason": "Emergency scan required",
+  "approvedBy": "security-admin@example.com",
+  "expiresAt": "2025-12-05T00:00:00Z"
+}
+```
+
+## Alerting
+
+### Alert Conditions
+
+| Condition | Severity | Action |
+|-----------|----------|--------|
+| Feed sync failed | Warning | Retry with backoff |
+| Feed approaching staleness | Info | Log, notify ops |
+| Feed stale | Warning | Notify ops, scan warnings |
+| Feed critical | Critical | Block scans, page on-call |
+| Feed frozen > 30 days | Warning | Review freeze necessity |
+
+### Alert Payload
+
+```json
+{
+  "alert": {
+    "type": "feed_staleness",
+    "feed": "nvd",
+    "severity": "warning",
+    "snapshotId": "nvd-2025-12-04T00-00-00Z",
+    "age": "PT50H",
+    "threshold": "PT48H",
+    "message": "NVD feed exceeds staleness threshold (50h > 48h)",
+    "timestamp": "2025-12-06T02:00:00Z"
+  }
+}
+```
+
+## Determinism Integration
+
+### Replay Requirements
+
+For deterministic replay:
+
+1. All feeds MUST be frozen at recorded state
+2. Snapshot hashes MUST match manifest
+3. No network fetch during replay
+4. Staleness validation skipped (uses recorded state)
+
+### Validation Script
+
+```bash
+#!/bin/bash
+# scripts/scanner/verify_feed_hashes.sh
+
+MANIFEST="replay-manifest.json"
+
+for feed in $(jq -r '.replay.feeds | keys[]' "${MANIFEST}"); do
+    expected_hash=$(jq -r ".replay.feeds.${feed}.hash" "${MANIFEST}")
+    snapshot_id=$(jq -r ".replay.feeds.${feed}.snapshotId" "${MANIFEST}")
+
+    actual_hash=$(stellaops feed hash --snapshot-id "${snapshot_id}")
+
+    if [[ "${actual_hash}" != "${expected_hash}" ]]; then
+        echo "FAIL: ${feed} snapshot hash mismatch"
+        echo "  expected: ${expected_hash}"
+        echo "  actual:   ${actual_hash}"
+        exit 1
+    fi
+    echo "PASS: ${feed} snapshot ${snapshot_id}"
+done
+```
+
+## Links
+
+- Sprint: `docs/implplan/SPRINT_0186_0001_0001_record_deterministic_execution.md` (SP6)
+- Spine Versioning: `docs/modules/policy/contracts/spine-versioning-plan.md` (SP1)
+- Replay: `docs/replay/DETERMINISTIC_REPLAY.md`
--- a/docs/modules/policy/contracts/sbom-vex-diff-rules.md
+++ b/docs/modules/policy/contracts/sbom-vex-diff-rules.md
@@ -0,0 +1,331 @@
+# SBOM/VEX Deterministic Diff Rules (SP5)
+
+Status: Draft · Date: 2025-12-04
+Scope: Define deterministic diff rules and fixtures for SBOM/VEX deltas, ensuring reproducible comparison results and stable hash expectations.
+
+## Objectives
+
+- Enable deterministic diffing between SBOM/VEX versions.
+- Define canonical ordering for diff output.
+- Provide fixtures for validating diff implementations.
+- Ensure diff results are hash-stable.
+
+## Diff Operations
+
+### Supported Operations
+
+| Operation | Description | Output Format |
+|-----------|-------------|---------------|
+| `component-diff` | Compare component lists between SBOMs | JSON Patch |
+| `vulnerability-diff` | Compare vulnerability lists | JSON Patch |
+| `vex-diff` | Compare VEX statements | JSON Patch |
+| `full-diff` | Complete SBOM/VEX comparison | Combined JSON Patch |
+
+### JSON Patch Format
+
+Diff output uses RFC 6902 JSON Patch format:
+
+```json
+{
+  "patch": [
+    {
+      "op": "add",
+      "path": "/components/2",
+      "value": {
+        "type": "library",
+        "name": "new-lib",
+        "version": "1.0.0",
+        "purl": "pkg:npm/new-lib@1.0.0"
+      }
+    },
+    {
+      "op": "remove",
+      "path": "/components/0"
+    },
+    {
+      "op": "replace",
+      "path": "/components/1/version",
+      "value": "2.0.0"
+    }
+  ],
+  "meta": {
+    "source": "sbom-v1.json",
+    "target": "sbom-v2.json",
+    "sourceHash": "b3:...",
+    "targetHash": "b3:...",
+    "patchHash": "b3:...",
+    "timestamp": "2025-12-04T00:00:00Z"
+  }
+}
+```
+
+## Determinism Rules
+
+### Ordering
+
+1. **Operations**: `remove` first (descending path order), then `replace`, then `add`
+2. **Paths**: Lexicographic sort within operation type
+3. **Array indices**: Stable indices based on sort keys (purl for components, id for vulns)
+
+### Canonical Comparison
+
+When comparing elements for diff:
+
+| Element Type | Sort Keys | Tie Breakers |
+|--------------|-----------|--------------|
+| Component | `purl` | `name`, `version` |
+| Vulnerability | `id` | `source.name`, `ratings[0].score` |
+| VEX Statement | `vulnerability` | `products[0].purl`, `timestamp` |
+| Service | `name` | `version` |
+| Property | `name` | - |
+
+### Hash Computation
+
+Diff output hash computed as:
+1. Serialize patch array to canonical JSON (sorted keys, no whitespace)
+2. Compute BLAKE3-256 over UTF-8 bytes
+3. Record in `meta.patchHash`
+
+## Component Diff
+
+### Input
+
+```json
+// sbom-v1.json
+{
+  "components": [
+    {"name": "lib-a", "version": "1.0.0", "purl": "pkg:npm/lib-a@1.0.0"},
+    {"name": "lib-b", "version": "1.0.0", "purl": "pkg:npm/lib-b@1.0.0"}
+  ]
+}
+
+// sbom-v2.json
+{
+  "components": [
+    {"name": "lib-a", "version": "1.0.1", "purl": "pkg:npm/lib-a@1.0.1"},
+    {"name": "lib-c", "version": "1.0.0", "purl": "pkg:npm/lib-c@1.0.0"}
+  ]
+}
+```
+
+### Output
+
+```json
+{
+  "patch": [
+    {
+      "op": "remove",
+      "path": "/components/1",
+      "comment": "removed pkg:npm/lib-b@1.0.0"
+    },
+    {
+      "op": "replace",
+      "path": "/components/0/version",
+      "value": "1.0.1",
+      "comment": "upgraded pkg:npm/lib-a@1.0.0 -> 1.0.1"
+    },
+    {
+      "op": "replace",
+      "path": "/components/0/purl",
+      "value": "pkg:npm/lib-a@1.0.1"
+    },
+    {
+      "op": "add",
+      "path": "/components/1",
+      "value": {"name": "lib-c", "version": "1.0.0", "purl": "pkg:npm/lib-c@1.0.0"},
+      "comment": "added pkg:npm/lib-c@1.0.0"
+    }
+  ]
+}
+```
+
+## Vulnerability Diff
+
+### Added/Removed Vulnerabilities
+
+```json
+{
+  "patch": [
+    {
+      "op": "add",
+      "path": "/vulnerabilities/-",
+      "value": {
+        "id": "CVE-2025-0002",
+        "ratings": [{"method": "CVSSv4", "score": 5.3}]
+      },
+      "comment": "new vulnerability CVE-2025-0002"
+    },
+    {
+      "op": "remove",
+      "path": "/vulnerabilities/0",
+      "comment": "resolved CVE-2025-0001"
+    }
+  ]
+}
+```
+
+### Rating Changes
+
+```json
+{
+  "patch": [
+    {
+      "op": "replace",
+      "path": "/vulnerabilities/0/ratings/0/score",
+      "value": 9.0,
+      "comment": "CVE-2025-0001 score updated 8.5 -> 9.0"
+    }
+  ]
+}
+```
+
+## VEX Diff
+
+### Statement Status Changes
+
+```json
+{
+  "patch": [
+    {
+      "op": "replace",
+      "path": "/statements/0/status",
+      "value": "not_affected",
+      "comment": "CVE-2025-0001 status changed affected -> not_affected"
+    },
+    {
+      "op": "add",
+      "path": "/statements/0/justification",
+      "value": {
+        "category": "vulnerable_code_not_present",
+        "details": "Function patched in v2.0.1"
+      }
+    }
+  ]
+}
+```
+
+## Fixtures
+
+### Directory Structure
+
+```
+docs/modules/policy/fixtures/diff-rules/
+├── component-diff/
+│   ├── input-v1.json
+│   ├── input-v2.json
+│   ├── expected-diff.json
+│   └── hashes.txt
+├── vulnerability-diff/
+│   ├── input-v1.json
+│   ├── input-v2.json
+│   ├── expected-diff.json
+│   └── hashes.txt
+├── vex-diff/
+│   ├── input-v1.json
+│   ├── input-v2.json
+│   ├── expected-diff.json
+│   └── hashes.txt
+└── full-diff/
+    ├── sbom-v1.json
+    ├── sbom-v2.json
+    ├── expected-diff.json
+    └── hashes.txt
+```
+
+### Sample Fixture (Component Diff)
+
+```json
+// docs/modules/policy/fixtures/diff-rules/component-diff/expected-diff.json
+{
+  "patch": [
+    {"op": "remove", "path": "/components/1"},
+    {"op": "replace", "path": "/components/0/version", "value": "1.0.1"},
+    {"op": "replace", "path": "/components/0/purl", "value": "pkg:npm/lib-a@1.0.1"},
+    {"op": "add", "path": "/components/1", "value": {"name": "lib-c", "version": "1.0.0", "purl": "pkg:npm/lib-c@1.0.0"}}
+  ],
+  "meta": {
+    "sourceHash": "b3:...",
+    "targetHash": "b3:...",
+    "patchHash": "b3:..."
+  }
+}
+```
+
+## CI Validation
+
+```bash
+#!/bin/bash
+# scripts/policy/validate-diff-fixtures.sh
+
+FIXTURE_DIR="docs/modules/policy/fixtures/diff-rules"
+
+for category in component-diff vulnerability-diff vex-diff full-diff; do
+    echo "Validating ${category}..."
+
+    # Run diff
+    stellaops-diff \
+        --source "${FIXTURE_DIR}/${category}/input-v1.json" \
+        --target "${FIXTURE_DIR}/${category}/input-v2.json" \
+        --output /tmp/actual-diff.json
+
+    # Compare with expected
+    expected_hash=$(grep "expected-diff.json" "${FIXTURE_DIR}/${category}/hashes.txt" | awk '{print $2}')
+    actual_hash=$(b3sum /tmp/actual-diff.json | cut -d' ' -f1)
+
+    if [[ "${actual_hash}" != "${expected_hash}" ]]; then
+        echo "FAIL: ${category} diff hash mismatch"
+        diff <(jq -S . "${FIXTURE_DIR}/${category}/expected-diff.json") <(jq -S . /tmp/actual-diff.json)
+        exit 1
+    fi
+
+    echo "PASS: ${category}"
+done
+```
+
+## API Integration
+
+### Diff Endpoint
+
+```http
+POST /api/v1/sbom/diff
+Content-Type: application/json
+
+{
+  "source": "<base64-encoded-sbom-v1>",
+  "target": "<base64-encoded-sbom-v2>",
+  "options": {
+    "includeComments": true,
+    "format": "json-patch"
+  }
+}
+```
+
+### Response
+
+```json
+{
+  "diff": {
+    "patch": [...],
+    "meta": {
+      "sourceHash": "b3:...",
+      "targetHash": "b3:...",
+      "patchHash": "b3:...",
+      "componentChanges": {
+        "added": 1,
+        "removed": 1,
+        "modified": 1
+      },
+      "vulnerabilityChanges": {
+        "added": 0,
+        "removed": 1,
+        "modified": 0
+      }
+    }
+  }
+}
+```
+
+## Links
+
+- Sprint: `docs/implplan/SPRINT_0186_0001_0001_record_deterministic_execution.md` (SP5)
+- Spine Versioning: `docs/modules/policy/contracts/spine-versioning-plan.md` (SP1)