Add sample proof bundle configurations and verification script

- Introduced sample proof bundle configuration files for testing, including `sample-proof-bundle-config.dsse.json`, `sample-proof-bundle.dsse.json`, and `sample-proof-bundle.json`. - Implemented a verification script `test_verify_sample.sh` to validate proof bundles against specified schemas and catalogs. - Updated existing proof bundle configurations with new metadata, including versioning, created timestamps, and justification details. - Enhanced evidence entries with expiration dates and hashes for better integrity checks. - Ensured all new configurations adhere to the defined schema for consistency and reliability in testing.
2025-12-04 08:54:32 +02:00
parent e1262eb916
commit 4dc7cf834a
76 changed files with 3051 additions and 355 deletions
--- a/docs/modules/evidence-locker/bundle-packaging.md
+++ b/docs/modules/evidence-locker/bundle-packaging.md
@@ -44,6 +44,8 @@ Upcoming EB1–EB10 remediation (Sprint 0161; advisory `docs/product-advisories/
 - Ship an offline verifier script and golden bundles/replay fixtures to prove determinism.
 - Add incident-mode activation/exit records and redaction/tenant isolation guidance for portable bundles.

+Canonical schemas now live in `docs/modules/evidence-locker/schemas/` (EB1, EB2). Offline verification steps and the embeddable script are documented in `docs/modules/evidence-locker/verify-offline.md` (EB9); use the computed Merkle root as the DSSE subject for sealed and portable bundles.
+
 ### Merkle recipe (example)
 ```bash
 cd bundle
--- a/docs/modules/evidence-locker/eb-gaps-161-007-plan.md
+++ b/docs/modules/evidence-locker/eb-gaps-161-007-plan.md
@@ -0,0 +1,32 @@
+# EB1–EB10 Gap Closure Plan (EVID-GAPS-161-007)
+
+Purpose: track remediation items from the 28-Nov-2025 advisory so Evidence Locker bundles, replay payloads, and portable exports are provably deterministic and verifiable offline.
+
+Working directory: `docs/implplan` (sprint coordination) with artefacts in `docs/modules/evidence-locker` and `tests/EvidenceLocker`.
+
+## Scope Items
+| ID  | Deliverable | Artifact / Path | Owner(s) | Acceptance / Notes | Status |
+| --- | --- | --- | --- | --- | --- |
+| EB1 | Publish canonical manifest schema | `docs/modules/evidence-locker/schemas/bundle.manifest.schema.json` | Evidence Locker Guild | JSON Schema matches EvidenceBundleManifest (bundleId, tenantId, kind, metadata, entries) and captures replay/incident/redaction hooks. | Draft (2025-12-04) |
+| EB2 | Publish checksums schema | `docs/modules/evidence-locker/schemas/checksums.schema.json` | Evidence Locker Guild | Canonical map for `checksums.txt`; Merkle root + chunking metadata; sorted entry rule recorded. | Draft (2025-12-04) |
+| EB3 | Hash/Merkle recipe doc | `docs/modules/evidence-locker/bundle-packaging.md` (new section) | Evidence Locker Guild | Normative steps for Merkle root + DSSE subject; clarifies gzip/tar invariants and CAS compatibility. | TODO |
+| EB4 | Mandatory DSSE predicate/log policy | `docs/modules/evidence-locker/attestation-contract.md` | Evidence Locker Guild · Security Guild | Required claims + signing profiles; Rekor/log policy (optional vs required); aligns with crypto registry defaults. | TODO |
+| EB5 | Replay provenance block | `docs/modules/evidence-locker/replay-payload-contract.md` + manifest schema | Evidence Locker Guild · Replay Delivery Guild | Replay digest + DSSE envelope recorded; ordering rules match `DETERMINISTIC_REPLAY.md`; portable bundle retains linkage. | TODO |
+| EB6 | Chunking/CAS rules | `checksums.schema.json` + `bundle-packaging.md` | Evidence Locker Guild · Storage/DevOps | Defines chunk sizing, CAS digest, and stability guarantees; CI test to catch ordering changes. | TODO |
+| EB7 | Incident-mode signed activation/exit | `docs/modules/evidence-locker/incident-mode.md` | Evidence Locker Guild · Security Guild | Manifest/DSSE captures activation + deactivation events with signer identity; API/CLI steps documented. | TODO |
+| EB8 | Tenant isolation + redaction manifest | `bundle-packaging.md` + portable bundle guidance | Evidence Locker Guild · Privacy Guild | Portable bundles omit tenant identifiers; redaction map recorded; verifier asserts redacted fields absent. | TODO |
+| EB9 | Offline verifier script | `docs/modules/evidence-locker/verify-offline.md` | Evidence Locker Guild | POSIX script included; no network dependencies; emits Merkle root used by DSSE subject. | DONE (2025-12-04) |
+| EB10 | Golden bundles/replay fixtures + SemVer/changelog | `tests/EvidenceLocker/Bundles/Golden/` + release notes (TBD) | Evidence Locker Guild · CLI Guild | Golden sealed + portable bundles and replay NDJSON with expected roots; changelog bump covering EB1–EB9. | TODO |
+
+## Near-Term Actions (to move EB1–EB10 to DONE)
+- Wire schemas into EvidenceLocker CI (manifest + checksums validation) and surface in API/CLI OpenAPI/Help.
+- Update `attestation-contract.md` and `incident-mode.md` with DSSE predicate/log policy and signed incident toggles (EB4, EB7).
+- Extend replay contract with provenance block and ordering example, and mirror in manifest schema (EB5).
+- Add normative Merkle/CAS section to `bundle-packaging.md`, ensuring DSSE subject references the root hash (EB3, EB6).
+- Create golden fixtures under `tests/EvidenceLocker/Bundles/Golden/` with recorded expected hashes and replay traces; hook into xUnit tests (EB10).
+- Bump Evidence Locker and CLI SemVer and changelog once above artefacts are wired (EB10).
+
+## Dependencies and Links
+- Advisory: `docs/product-advisories/archived/27-Nov-2025-superseded/28-Nov-2025 - Evidence Bundle and Replay Contracts.md`
+- Replay rules: `docs/replay/DETERMINISTIC_REPLAY.md`
+- Sprint tracking: `docs/implplan/SPRINT_0161_0001_0001_evidencelocker.md` (EVID-GAPS-161-007)
--- a/docs/modules/evidence-locker/schemas/bundle.manifest.schema.json
+++ b/docs/modules/evidence-locker/schemas/bundle.manifest.schema.json
@@ -0,0 +1,142 @@
+{
+  "$schema": "https://json-schema.org/draft/2020-12/schema",
+  "$id": "https://stellaops.local/schemas/evidence/bundle.manifest.schema.json",
+  "title": "StellaOps Evidence Bundle Manifest (EB1)",
+  "description": "Canonical manifest for deterministic evidence bundles; aligns with EvidenceLocker build models and EB1–EB10 advisory gaps.",
+  "type": "object",
+  "additionalProperties": false,
+  "required": [
+    "bundleId",
+    "tenantId",
+    "kind",
+    "createdAt",
+    "metadata",
+    "entries"
+  ],
+  "properties": {
+    "bundleId": {
+      "type": "string",
+      "description": "Bundle identifier in UUID v4 N-format (no dashes).",
+      "pattern": "^[0-9a-fA-F]{32}$"
+    },
+    "tenantId": {
+      "type": "string",
+      "description": "Tenant identifier in UUID v4 N-format (no dashes).",
+      "pattern": "^[0-9a-fA-F]{32}$"
+    },
+    "kind": {
+      "description": "Bundle category; numeric values mirror EvidenceBundleKind enum.",
+      "oneOf": [
+        { "type": "string", "enum": ["evaluation", "job", "export"] },
+        { "type": "integer", "enum": [1, 2, 3] }
+      ]
+    },
+    "createdAt": {
+      "type": "string",
+      "format": "date-time",
+      "description": "Bundle creation timestamp (UTC, RFC3339)."
+    },
+    "metadata": {
+      "type": "object",
+      "description": "Arbitrary key/value metadata captured at bundle creation.",
+      "additionalProperties": { "type": "string" }
+    },
+    "entries": {
+      "type": "array",
+      "description": "Canonical file inventory used to derive checksums and Merkle root.",
+      "minItems": 1,
+      "items": { "$ref": "#/$defs/manifestEntry" }
+    },
+    "hashSummary": {
+      "type": "object",
+      "description": "Optional Merkle root summary that binds the manifest to checksums.txt.",
+      "additionalProperties": false,
+      "required": ["algorithm", "merkleRoot"],
+      "properties": {
+        "algorithm": { "type": "string", "enum": ["sha256"] },
+        "merkleRoot": { "type": "string", "pattern": "^[0-9a-f]{64}$" },
+        "checksumsPath": {
+          "type": "string",
+          "description": "Relative path to canonical checksums file inside the bundle.",
+          "default": "checksums.txt"
+        }
+      }
+    },
+    "replayProvenance": {
+      "type": "object",
+      "description": "Optional replay linkage proving how the bundle was produced for deterministic re-run.",
+      "additionalProperties": false,
+      "required": ["recordDigest"],
+      "properties": {
+        "recordDigest": { "type": "string", "pattern": "^sha256:[0-9a-f]{64}$" },
+        "sequence": { "type": "integer", "minimum": 0 },
+        "ledgerUri": { "type": "string", "format": "uri" },
+        "dsseEnvelope": {
+          "type": "string",
+          "description": "Base64-encoded DSSE envelope for replay record provenance.",
+          "contentEncoding": "base64"
+        },
+        "transparencyLog": {
+          "type": "object",
+          "additionalProperties": false,
+          "properties": {
+            "rekorUuid": { "type": "string" },
+            "logIndex": { "type": "integer", "minimum": 0 },
+            "inclusionProof": { "type": "string" }
+          }
+        }
+      }
+    },
+    "incident": {
+      "type": "object",
+      "description": "Incident-mode activation/exit records captured at bundle time.",
+      "additionalProperties": false,
+      "properties": {
+        "activatedAt": { "type": "string", "format": "date-time" },
+        "activatedBy": { "type": "string" },
+        "reason": { "type": "string" },
+        "deactivatedAt": { "type": "string", "format": "date-time" },
+        "deactivatedBy": { "type": "string" }
+      }
+    },
+    "redaction": {
+      "type": "object",
+      "description": "Portable-bundle redaction details to prove tenant isolation.",
+      "additionalProperties": false,
+      "properties": {
+        "portable": { "type": "boolean", "default": false },
+        "maskedFields": {
+          "type": "array",
+          "items": { "type": "string" }
+        },
+        "tenantToken": {
+          "type": "string",
+          "description": "Opaque token replacing tenantId in portable bundles."
+        }
+      }
+    }
+  },
+  "$defs": {
+    "manifestEntry": {
+      "type": "object",
+      "additionalProperties": false,
+      "required": ["section", "canonicalPath", "sha256", "sizeBytes", "mediaType"],
+      "properties": {
+        "section": { "type": "string", "minLength": 1 },
+        "canonicalPath": {
+          "type": "string",
+          "description": "Deterministic path within the bundle using '/' separators.",
+          "pattern": "^(?:[A-Za-z0-9_.-]+/)*[A-Za-z0-9_.-]+$"
+        },
+        "sha256": { "type": "string", "pattern": "^[0-9a-f]{64}$" },
+        "sizeBytes": { "type": "integer", "minimum": 0 },
+        "mediaType": { "type": "string" },
+        "attributes": {
+          "type": "object",
+          "description": "Section-specific attributes (e.g., sbom format, dsse predicate).",
+          "additionalProperties": { "type": "string" }
+        }
+      }
+    }
+  }
+}
--- a/docs/modules/evidence-locker/schemas/checksums.schema.json
+++ b/docs/modules/evidence-locker/schemas/checksums.schema.json
@@ -0,0 +1,47 @@
+{
+  "$schema": "https://json-schema.org/draft/2020-12/schema",
+  "$id": "https://stellaops.local/schemas/evidence/checksums.schema.json",
+  "title": "StellaOps Evidence Bundle Checksums (EB2)",
+  "description": "Canonical checksum map used to derive the Merkle root and DSSE subject for evidence bundles.",
+  "type": "object",
+  "additionalProperties": false,
+  "required": ["algorithm", "root", "entries"],
+  "properties": {
+    "algorithm": { "type": "string", "enum": ["sha256"] },
+    "root": { "type": "string", "pattern": "^[0-9a-f]{64}$" },
+    "generatedAt": { "type": "string", "format": "date-time" },
+    "bundleId": { "type": "string", "pattern": "^[0-9a-fA-F]{32}$" },
+    "tenantId": { "type": "string", "pattern": "^[0-9a-fA-F]{32}$" },
+    "entries": {
+      "type": "array",
+      "minItems": 1,
+      "description": "Sorted list of entry hashes; order must be lexicographic on canonicalPath.",
+      "items": { "$ref": "#/$defs/checksumEntry" }
+    },
+    "chunking": {
+      "type": "object",
+      "description": "Optional chunked/CAS hashing strategy for large payloads.",
+      "additionalProperties": false,
+      "properties": {
+        "strategy": { "type": "string", "enum": ["none", "fixed", "buzhash"] },
+        "chunkSizeBytes": { "type": "integer", "minimum": 1024 },
+        "casDigestAlgorithm": { "type": "string", "enum": ["sha256"] }
+      }
+    }
+  },
+  "$defs": {
+    "checksumEntry": {
+      "type": "object",
+      "additionalProperties": false,
+      "required": ["canonicalPath", "sha256", "sizeBytes"],
+      "properties": {
+        "canonicalPath": {
+          "type": "string",
+          "pattern": "^(?:[A-Za-z0-9_.-]+/)*[A-Za-z0-9_.-]+$"
+        },
+        "sha256": { "type": "string", "pattern": "^[0-9a-f]{64}$" },
+        "sizeBytes": { "type": "integer", "minimum": 0 }
+      }
+    }
+  }
+}
--- a/docs/modules/evidence-locker/verify-offline.md
+++ b/docs/modules/evidence-locker/verify-offline.md
@@ -0,0 +1,51 @@
+# Offline Verification Playbook (EB9)
+
+Purpose: allow auditors to validate Evidence Locker bundles without network access, using only POSIX tools. Applies to both sealed `bundle.tgz` and portable `portable-bundle-v1.tgz`.
+
+## Prerequisites
+- `tar`, `sha256sum` (or `shasum`), `awk`, `base64`.
+- Optional: `jq` for schema validation; `cosign` or `stella` CLI for DSSE verification if pre-loaded.
+
+## Quick steps (sealed bundle)
+1) `tar -xzf bundle.tgz -C /tmp/bundle`
+2) `cd /tmp/bundle`
+3) Validate checksums: `sha256sum -c checksums.txt`
+4) Derive Merkle root (matches DSSE subject): `sha256sum checksums.txt | awk '{print $1}'`
+5) Validate manifest against schema (if `jq` present): `jq -e 'input | type=="object"' manifest.json >/dev/null`
+6) Verify DSSE envelope (optional but recommended):
+   - `cat manifest.json | base64 | cosign verify-blob --key cosign.pub --bundle signature.json --bundleType dsse`
+   - or `stella evidence verify --bundle ../bundle.tgz --offline` once CLI supports offline mode.
+
+## Quick steps (portable bundle)
+Same as sealed, plus confirm redaction:
+- `jq -e 'has(\"redaction\") and .redaction.portable==true' manifest.json >/dev/null` (if `jq` available)
+- Confirm no tenant identifiers in `bundle.json` and `manifest.json`.
+
+## Embeddable verifier script
+Place the following script into `verify-offline.sh` when assembling portable bundles. It exits non-zero on any mismatch and prints the Merkle root used as DSSE subject.
+
+```bash
+#!/usr/bin/env bash
+set -euo pipefail
+BUNDLE="${1:-bundle.tgz}"
+WORKDIR="$(mktemp -d)"
+cleanup() { rm -rf "$WORKDIR"; }
+trap cleanup EXIT
+tar -xzf "$BUNDLE" -C "$WORKDIR"
+cd "$WORKDIR"
+sha256sum -c checksums.txt
+MERKLE=$(sha256sum checksums.txt | awk '{print $1}')
+printf "merkle_root=%s\n" "$MERKLE"
+if command -v jq >/dev/null; then
+  jq -e 'type=="object" and has("entries")' manifest.json >/dev/null
+fi
+```
+
+## Fixtures
+- Golden bundles and replay records live under `tests/EvidenceLocker/Bundles/Golden/`.
+- Expected Merkle roots and DSSE payload digests should be recorded alongside each fixture to keep CI deterministic.
+
+## References
+- Manifest schema: `docs/modules/evidence-locker/schemas/bundle.manifest.schema.json`
+- Checksums schema: `docs/modules/evidence-locker/schemas/checksums.schema.json`
+- Merkle recipe: see `docs/modules/evidence-locker/bundle-packaging.md`