Add signal contracts for reachability, exploitability, trust, and unknown symbols

- Introduced `ReachabilityState`, `RuntimeHit`, `ExploitabilitySignal`, `ReachabilitySignal`, `SignalEnvelope`, `SignalType`, `TrustSignal`, and `UnknownSymbolSignal` records to define various signal types and their properties. - Implemented JSON serialization attributes for proper data interchange. - Created project files for the new signal contracts library and corresponding test projects. - Added deterministic test fixtures for micro-interaction testing. - Included cryptographic keys for secure operations with cosign.
2025-12-05 00:27:00 +02:00
parent b018949a8d
commit 8768c27f30
192 changed files with 27569 additions and 2552 deletions
--- a/docs/modules/scanner/design/competitor-offline-ingest-kit.md
+++ b/docs/modules/scanner/design/competitor-offline-ingest-kit.md
@@ -0,0 +1,353 @@
+# Competitor Offline Ingest Kit (CM5)
+
+Status: Draft · Date: 2025-12-04
+Scope: Define offline ingest kit contents including DSSE-signed adapters, mappings, fixtures, and trust roots for air-gapped environments.
+
+## Objectives
+
+- Bundle all competitor ingest artifacts for offline use.
+- Sign kit with DSSE for integrity verification.
+- Include trust roots for signature verification.
+- Enable complete ingest workflow without network access.
+
+## Bundle Structure
+
+```
+out/offline/competitor-ingest-kit-v1/
+├── manifest.json                    # Bundle manifest with all hashes
+├── manifest.dsse                    # DSSE signature over manifest
+├── adapters/
+│   ├── syft/
+│   │   ├── mapping.csv             # Field mappings
+│   │   ├── version-map.json        # Version compatibility
+│   │   └── schema.json             # Expected input schema
+│   ├── trivy/
+│   │   ├── mapping.csv
+│   │   ├── version-map.json
+│   │   └── schema.json
+│   └── clair/
+│       ├── mapping.csv
+│       ├── version-map.json
+│       └── schema.json
+├── fixtures/
+│   ├── normalized-syft.json
+│   ├── normalized-trivy.json
+│   ├── normalized-clair.json
+│   └── hashes.txt
+├── trust/
+│   ├── root-ca.pem
+│   ├── keyring.json
+│   ├── revocation.json
+│   └── cosign-keys/
+│       ├── syft-release.pub
+│       ├── trivy-release.pub
+│       └── clair-release.pub
+├── coverage/
+│   └── coverage.csv
+├── policies/
+│   ├── signature-policy.json
+│   ├── fallback-policy.json
+│   └── retry-policy.json
+└── tools/
+    ├── versions.json
+    └── checksums.txt
+```
+
+## Manifest Format
+
+```json
+{
+  "version": "1.0.0",
+  "created": "2025-12-04T00:00:00Z",
+  "creator": "stellaops-scanner",
+  "type": "competitor-ingest-kit",
+  "artifacts": [
+    {
+      "path": "adapters/syft/mapping.csv",
+      "type": "adapter",
+      "tool": "syft",
+      "blake3": "...",
+      "sha256": "..."
+    },
+    {
+      "path": "fixtures/normalized-syft.json",
+      "type": "fixture",
+      "tool": "syft",
+      "blake3": "aa42c167d19535709a10df73dc39e6a50b8efbbb0ae596d17183ce62676fa85a",
+      "sha256": "3f8684ff341808dcb92e97dd2c10acca727baaff05182e81a4364bb3dad0eaa7"
+    },
+    {
+      "path": "trust/keyring.json",
+      "type": "trust",
+      "blake3": "...",
+      "sha256": "..."
+    }
+  ],
+  "supportedTools": {
+    "syft": {
+      "minVersion": "1.0.0",
+      "maxVersion": "1.99.99",
+      "tested": ["1.0.0", "1.5.0", "1.10.0"]
+    },
+    "trivy": {
+      "minVersion": "0.50.0",
+      "maxVersion": "0.59.99",
+      "tested": ["0.50.0", "0.55.0"]
+    },
+    "clair": {
+      "minVersion": "6.0.0",
+      "maxVersion": "6.99.99",
+      "tested": ["6.0.0", "6.1.0"]
+    }
+  },
+  "manifestHash": {
+    "blake3": "...",
+    "sha256": "..."
+  }
+}
+```
+
+## Adapter Contents
+
+### Mapping CSV Format
+
+```csv
+source_field,target_field,rule,required,notes
+artifacts[].name,components[].name,copy,yes,Component name
+artifacts[].version,components[].version,copy,yes,Component version
+artifacts[].purl,components[].purl,copy,yes,Package URL
+artifacts[].type,components[].type,map:package->library,yes,Type mapping
+artifacts[].licenses[],components[].licenses[],flatten,no,License list
+artifacts[].metadata.digest,components[].hashes[],transform:sha256,no,Hash extraction
+```
+
+### Version Map Format
+
+```json
+{
+  "tool": "syft",
+  "versionRanges": [
+    {
+      "range": ">=1.0.0 <1.5.0",
+      "schemaVersion": "v1",
+      "mappingFile": "mapping-v1.csv"
+    },
+    {
+      "range": ">=1.5.0 <2.0.0",
+      "schemaVersion": "v2",
+      "mappingFile": "mapping-v2.csv",
+      "breaking": ["artifacts.metadata renamed to artifacts.meta"]
+    }
+  ]
+}
+```
+
+## Policy Files
+
+### Signature Policy (CM2)
+
+```json
+{
+  "policy": {
+    "requireSignature": true,
+    "allowUnsigned": false,
+    "acceptedFormats": ["dsse", "cose", "jws"],
+    "acceptedAlgorithms": ["ed25519", "ecdsa-p256", "rsa-2048"],
+    "trustedIssuers": [
+      "https://github.com/anchore/syft",
+      "https://github.com/aquasecurity/trivy",
+      "https://github.com/quay/clair"
+    ],
+    "rejectReasons": [
+      "sig_invalid",
+      "sig_expired",
+      "key_unknown",
+      "key_expired",
+      "key_revoked",
+      "alg_unsupported"
+    ]
+  }
+}
+```
+
+### Fallback Policy (CM6)
+
+```json
+{
+  "fallback": {
+    "hierarchy": [
+      {
+        "level": 1,
+        "source": "signed_sbom",
+        "confidence": 1.0,
+        "requirements": ["valid_signature", "valid_provenance"]
+      },
+      {
+        "level": 2,
+        "source": "unsigned_sbom",
+        "confidence": 0.7,
+        "requirements": ["tool_metadata", "component_purl", "scan_timestamp"],
+        "warnings": ["provenance_unknown"]
+      },
+      {
+        "level": 3,
+        "source": "scan_only",
+        "confidence": 0.5,
+        "requirements": ["scan_timestamp"],
+        "warnings": ["degraded_confidence", "no_sbom"]
+      },
+      {
+        "level": 4,
+        "source": "reject",
+        "confidence": 0.0,
+        "requirements": [],
+        "action": "reject",
+        "reason": "no_evidence"
+      }
+    ]
+  }
+}
+```
+
+### Retry Policy (CM10)
+
+```json
+{
+  "retry": {
+    "retryable": [
+      {"code": "network_error", "maxRetries": 3, "backoff": "exponential"},
+      {"code": "rate_limit", "maxRetries": 5, "backoff": "linear"},
+      {"code": "transient_io", "maxRetries": 2, "backoff": "fixed"}
+    ],
+    "nonRetryable": [
+      "signature_invalid",
+      "schema_invalid",
+      "unsupported_version",
+      "no_evidence"
+    ],
+    "backoffConfig": {
+      "exponential": {"base": 1000, "factor": 2, "max": 60000},
+      "linear": {"initial": 1000, "increment": 1000, "max": 30000},
+      "fixed": {"delay": 5000}
+    }
+  }
+}
+```
+
+## Kit Generation
+
+### Build Script
+
+```bash
+#!/bin/bash
+# scripts/scanner/build-competitor-ingest-kit.sh
+
+set -euo pipefail
+
+KIT_DIR="out/offline/competitor-ingest-kit-v1"
+rm -rf "${KIT_DIR}"
+mkdir -p "${KIT_DIR}"
+
+# Copy adapters
+for tool in syft trivy clair; do
+    mkdir -p "${KIT_DIR}/adapters/${tool}"
+    cp "src/Scanner/Adapters/${tool}/"*.csv "${KIT_DIR}/adapters/${tool}/"
+    cp "src/Scanner/Adapters/${tool}/"*.json "${KIT_DIR}/adapters/${tool}/"
+done
+
+# Copy fixtures
+mkdir -p "${KIT_DIR}/fixtures"
+cp docs/modules/scanner/fixtures/competitor-adapters/fixtures/*.json "${KIT_DIR}/fixtures/"
+cp docs/modules/scanner/fixtures/competitor-adapters/fixtures/hashes.txt "${KIT_DIR}/fixtures/"
+
+# Copy trust roots
+mkdir -p "${KIT_DIR}/trust/cosign-keys"
+cp trust/root-ca.pem "${KIT_DIR}/trust/"
+cp trust/keyring.json "${KIT_DIR}/trust/"
+cp trust/revocation.json "${KIT_DIR}/trust/"
+cp trust/cosign-keys/*.pub "${KIT_DIR}/trust/cosign-keys/"
+
+# Copy coverage
+mkdir -p "${KIT_DIR}/coverage"
+cp docs/modules/scanner/fixtures/competitor-adapters/coverage.csv "${KIT_DIR}/coverage/"
+
+# Copy policies
+mkdir -p "${KIT_DIR}/policies"
+cp policies/competitor/*.json "${KIT_DIR}/policies/"
+
+# Copy tool versions
+mkdir -p "${KIT_DIR}/tools"
+cp tools/versions.json "${KIT_DIR}/tools/"
+cp tools/checksums.txt "${KIT_DIR}/tools/"
+
+# Generate manifest
+scripts/scanner/generate-competitor-manifest.sh "${KIT_DIR}"
+
+# Sign manifest
+scripts/scanner/sign-manifest.sh "${KIT_DIR}/manifest.json" "${KIT_DIR}/manifest.dsse"
+
+echo "Kit built: ${KIT_DIR}"
+```
+
+## Verification
+
+### Kit Verification Script
+
+```bash
+#!/bin/bash
+# scripts/scanner/verify-competitor-ingest-kit.sh
+
+set -euo pipefail
+
+KIT_DIR="${1:-out/offline/competitor-ingest-kit-v1}"
+
+# Verify DSSE signature
+stellaops-verify dsse \
+    --envelope "${KIT_DIR}/manifest.dsse" \
+    --trust-root "${KIT_DIR}/trust/root-ca.pem" \
+    --expected-payload-type "application/vnd.stellaops.competitor-ingest.manifest+json"
+
+# Extract and verify artifacts
+MANIFEST=$(stellaops-verify dsse --envelope "${KIT_DIR}/manifest.dsse" --extract-payload)
+
+for artifact in $(echo "${MANIFEST}" | jq -r '.artifacts[] | @base64'); do
+    path=$(echo "${artifact}" | base64 -d | jq -r '.path')
+    expected_blake3=$(echo "${artifact}" | base64 -d | jq -r '.blake3')
+
+    actual_blake3=$(b3sum "${KIT_DIR}/${path}" | cut -d' ' -f1)
+
+    if [[ "${actual_blake3}" != "${expected_blake3}" ]]; then
+        echo "FAIL: ${path}"
+        exit 1
+    fi
+    echo "PASS: ${path}"
+done
+
+echo "All artifacts verified"
+```
+
+## Usage
+
+### Offline Ingest Workflow
+
+```bash
+# Initialize from kit
+stellaops ingest init --kit out/offline/competitor-ingest-kit-v1
+
+# Import SBOM with offline verification
+stellaops ingest import \
+    --input external-sbom.json \
+    --tool syft \
+    --offline \
+    --trust-root out/offline/competitor-ingest-kit-v1/trust/keyring.json
+
+# Validate against fixtures
+stellaops ingest validate \
+    --fixtures out/offline/competitor-ingest-kit-v1/fixtures
+```
+
+## Links
+
+- Sprint: `docs/implplan/SPRINT_0186_0001_0001_record_deterministic_execution.md` (CM5)
+- Normalization: `docs/modules/scanner/design/competitor-ingest-normalization.md` (CM1)
+- Verification: `docs/modules/scanner/design/competitor-signature-verification.md` (CM2)