true the date

2025-11-30 19:23:21 +02:00
parent 71e9a56cfd
commit 0bef705bcc
14 changed files with 0 additions and 0 deletions
--- a/docs/product-advisories/28-Nov-2025
+++ b/docs/product-advisories/28-Nov-2025
@@ -0,0 +1,338 @@
+# Evidence Bundle and Replay Contracts
+
+**Version:** 1.0
+**Date:** 2025-11-29
+**Status:** Canonical
+
+This advisory defines the product rationale, data contracts, and implementation strategy for the Evidence Locker module, covering deterministic bundle packaging, attestation contracts, replay payload ingestion, and incident mode operation.
+
+---
+
+## 1. Executive Summary
+
+The Evidence Locker provides **immutable, deterministic evidence bundles** for audit, compliance, and offline verification. Key capabilities:
+
+- **Deterministic Bundle Packaging** - Reproducible tar.gz archives with fixed timestamps and sorted entries
+- **DSSE Attestation Contracts** - In-toto predicates for bundle integrity verification
+- **Replay Payload Ingestion** - Scanner record capture for deterministic replay
+- **Incident Mode** - Extended retention and forensic capture during security incidents
+- **Portable Bundles** - Redacted exports for external auditors
+
+---
+
+## 2. Market Drivers
+
+### 2.1 Target Segments
+
+| Segment | Evidence Requirements | Use Case |
+|---------|----------------------|----------|
+| **Financial Services** | Immutable audit trails, SOC 2 Type II | Compliance evidence for regulators |
+| **Healthcare** | HIPAA audit requirements | PHI access logging and retention |
+| **Government/Defense** | Chain of custody, NIST 800-53 | Security incident forensics |
+| **Critical Infrastructure** | ICS/SCADA audit trails | Operational technology compliance |
+
+### 2.2 Competitive Positioning
+
+Most vulnerability scanning tools export findings as ephemeral reports. Stella Ops differentiates with:
+- **Cryptographically sealed bundles** with DSSE signatures
+- **Deterministic replay** for reproducing past scan states
+- **Offline verification** without network connectivity
+- **Incident mode** for automatic forensic preservation
+
+---
+
+## 3. Technical Architecture
+
+### 3.1 Bundle Layout (v1)
+
+```
+evidence-bundle-<id>.tar.gz
+├── manifest.json          # Bundle metadata and file inventory
+├── signature.json         # DSSE envelope with key metadata
+├── bundle.json            # Locker metadata (ids, status, root hash)
+├── checksums.txt          # SHA-256 per-entry hashes + Merkle root
+├── instructions.txt       # Offline verification steps
+├── observations.ndjson    # Advisory observations (sorted)
+├── linksets.ndjson        # Component linkages (sorted)
+└── timeline.ndjson        # Time anchors (optional)
+```
+
+### 3.2 Determinism Rules
+
+All bundles must be **bit-for-bit reproducible**:
+
+| Property | Rule |
+|----------|------|
+| **Gzip timestamp** | Pinned to `2025-01-01T00:00:00Z` |
+| **File permissions** | `0644` for all entries |
+| **Owner/Group** | `0:0` (root) |
+| **mtime/atime/ctime** | Fixed epoch value |
+| **JSON serialization** | `JsonSerializerDefaults.Web` + indent=2 |
+| **NDJSON ordering** | Sorted by `advisoryId`, then `component`, ascending |
+| **Hash format** | Lower-case hex, SHA-256 |
+| **Timestamps** | UTC ISO-8601 (RFC3339) |
+
+### 3.3 Attestation Contract (v1)
+
+**DSSE Envelope Structure:**
+
+```json
+{
+  "payloadType": "application/vnd.stellaops.evidence+json",
+  "payload": "<base64(manifest.json)>",
+  "signatures": [{
+    "keyid": "evidence-locker-ed25519-01",
+    "sig": "<base64(Ed25519 signature)>"
+  }]
+}
+```
+
+**Required Claim Set:**
+
+| Claim | Type | Description |
+|-------|------|-------------|
+| `bundle_id` | UUID v4 | Unique bundle identifier |
+| `produced_at` | ISO-8601 | UTC production timestamp |
+| `producer` | string | `evidence-locker:<region>` |
+| `subject_digest` | string | OCI digest of bundle |
+| `hashes` | map | `{ path: sha256 }` for each file |
+| `sbom` | array | SBOM digests and mediaTypes |
+| `vex` | array | VEX doc digests and versions |
+| `replay_manifest` | object | Optional replay digest + sequence |
+| `transparency` | object | Optional Rekor log entry |
+| `signing_profile` | string | `sovereign-default`, `fips`, `gost`, `pq-experimental` |
+
+### 3.4 Replay Payload Contract
+
+**NDJSON Record Structure:**
+
+```json
+{
+  "scanId": "uuid",
+  "tenantId": "string",
+  "subjectDigest": "sha256:...",
+  "scanKind": "sbom|vuln|policy",
+  "startedAtUtc": "ISO-8601",
+  "completedAtUtc": "ISO-8601",
+  "artifacts": [
+    { "type": "sbom|vex|log", "digest": "sha256:...", "uri": "..." }
+  ],
+  "provenance": {
+    "dsseEnvelope": "<base64>",
+    "transparencyLog": { "rekorUUID": "...", "logIndex": 123 }
+  },
+  "summary": { "findings": 42, "advisories": 15, "policies": 3 }
+}
+```
+
+**Ordering:** Sorted by `recordedAtUtc`, then `scanId` ascending.
+
+---
+
+## 4. Implementation Strategy
+
+### 4.1 Phase 1: Bundle Foundation (Complete)
+
+- [x] Postgres schema with RLS per tenant
+- [x] Object-store abstraction (local + S3 with optional WORM)
+- [x] Deterministic `bundle.tgz` packaging
+- [x] Incident mode with extended retention
+- [x] Portable bundle export with redacted metadata
+- [x] Crypto provider registry integration (`EVID-CRYPTO-90-001`)
+
+### 4.2 Phase 2: Attestation & Replay (In Progress)
+
+- [ ] DSSE envelope signing with configurable provider
+- [ ] Replay payload ingestion API
+- [ ] CLI `stella scan --record` integration
+- [ ] Bundle verification CLI (`stella verify --bundle`)
+- [ ] Rekor transparency log integration (optional)
+
+### 4.3 Phase 3: Automation & Integration (Planned)
+
+- [ ] Orchestrator hooks for automatic bundle creation
+- [ ] Export Center integration for mirror bundles
+- [ ] Timeline event emission for audit dashboards
+- [ ] Retention policy automation with legal hold support
+
+---
+
+## 5. API Surface
+
+### 5.1 Evidence Bundle APIs
+
+| Endpoint | Method | Scope | Description |
+|----------|--------|-------|-------------|
+| `/evidence` | GET | `evidence:read` | List bundles with filters |
+| `/evidence/{bundleId}` | GET | `evidence:read` | Get bundle metadata |
+| `/evidence/{bundleId}/download` | GET | `evidence:read` | Download sealed bundle |
+| `/evidence/{bundleId}/portable` | GET | `evidence:read` | Download portable bundle |
+| `/evidence` | POST | `evidence:create` | Create new bundle |
+| `/evidence/{bundleId}/hold` | POST | `evidence:hold` | Apply legal hold |
+
+### 5.2 Replay APIs
+
+| Endpoint | Method | Scope | Description |
+|----------|--------|-------|-------------|
+| `/replay/records` | POST | `replay:write` | Ingest replay records (NDJSON) |
+| `/replay/records` | GET | `replay:read` | Query replay records |
+| `/replay/{recordId}/verify` | POST | `replay:read` | Verify record integrity |
+| `/replay/{recordId}/diff` | POST | `replay:read` | Compare two records |
+
+### 5.3 CLI Commands
+
+```bash
+# Record scan for replay
+stella scan --record --output replay.ndjson <image>
+
+# Verify bundle integrity
+stella verify --bundle evidence-bundle.tar.gz
+
+# Replay past scan
+stella replay --record replay.ndjson --compare current.json
+
+# Diff two scan results
+stella replay diff --before before.ndjson --after after.ndjson
+```
+
+---
+
+## 6. Incident Mode
+
+### 6.1 Activation
+
+Incident mode is a **service-wide switch** for forensic fidelity during suspected compromise or SLO breach.
+
+**Configuration:**
+
+```yaml
+EvidenceLocker:
+  Incident:
+    Enabled: true
+    RetentionExtensionDays: 60
+    CaptureRequestSnapshot: true
+```
+
+### 6.2 Behavior Changes
+
+| Feature | Normal Mode | Incident Mode |
+|---------|-------------|---------------|
+| **Retention** | Standard policy | Extended by `RetentionExtensionDays` |
+| **Request capture** | None | Full request snapshots to object store |
+| **Manifest metadata** | Standard | Includes `incident.*` fields |
+| **Timeline events** | Standard | Activation/deactivation events |
+| **Audit verbosity** | Normal | Enhanced with `incident_reason` |
+
+### 6.3 Activation API
+
+```bash
+# Activate incident mode
+stella evidence incident activate --reason "Security event investigation"
+
+# Deactivate incident mode
+stella evidence incident deactivate --reason "Investigation complete"
+```
+
+---
+
+## 7. Offline Verification
+
+### 7.1 Portable Bundle Contents
+
+```
+portable-bundle-v1.tgz
+├── manifest.json         # Redacted (no tenant/storage identifiers)
+├── signature.json        # DSSE envelope
+├── checksums.txt         # SHA-256 hashes
+├── verify-offline.sh     # POSIX verification script
+├── observations.ndjson   # Advisory data
+├── linksets.ndjson       # Linkage data
+└── README.txt            # Verification instructions
+```
+
+### 7.2 Verification Steps
+
+1. Extract archive
+2. Run `./verify-offline.sh` (computes hashes, validates signature)
+3. Compare manifest hash with external attestation
+4. If Rekor transparency present, verify inclusion proof
+
+---
+
+## 8. Determinism Requirements
+
+All Evidence Locker operations must maintain determinism:
+
+1. **Same inputs = same bundle hash** for identical observations/linksets
+2. **Timestamps in UTC ISO-8601** format only
+3. **Tenant IDs lowercase** and included in manifest
+4. **Crypto provider version** recorded in audit material
+5. **NDJSON sorted** by canonical key ordering
+
+---
+
+## 9. Testing Strategy
+
+### 9.1 Unit Tests
+
+- Bundle packaging produces identical hash for identical inputs
+- DSSE envelope validates against registered keys
+- Incident mode extends retention correctly
+- Replay records sorted deterministically
+
+### 9.2 Integration Tests
+
+- Full bundle creation workflow with object store
+- CLI record/verify/replay cycle
+- Portable bundle extraction and verification
+- Cross-tenant isolation enforcement
+
+### 9.3 Golden Fixtures
+
+Located at `tests/EvidenceLocker/Fixtures/`:
+- `golden-bundle-v1.tar.gz` - Reference bundle with known hash
+- `golden-replay-records.ndjson` - Reference replay records
+- `golden-dsse-envelope.json` - Reference DSSE signature
+
+---
+
+## 10. Related Documentation
+
+| Resource | Location |
+|----------|----------|
+| Evidence Locker architecture | `docs/modules/evidence-locker/` |
+| Bundle packaging spec | `docs/modules/evidence-locker/bundle-packaging.md` |
+| Attestation contract | `docs/modules/evidence-locker/attestation-contract.md` |
+| Replay payload contract | `docs/modules/evidence-locker/replay-payload-contract.md` |
+| Incident mode spec | `docs/modules/evidence-locker/incident-mode.md` |
+| Crypto provider registry | `docs/security/crypto-registry-decision-2025-11-18.md` |
+
+---
+
+## 11. Sprint Mapping
+
+- **Primary Sprint:** SPRINT_0161_0001_0001_evidencelocker.md
+- **CLI Integration:** SPRINT_0187_0001_0001_evidence_locker_cli_integration.md
+- **Coordination:** SPRINT_0160_0001_0001_export_evidence.md
+
+**Key Task IDs:**
+- `EVID-OBS-54-002` - Finalize bundle packaging + DSSE layout
+- `EVID-REPLAY-187-001` - Implement replay ingestion/retention APIs
+- `CLI-REPLAY-187-002` - Add CLI record/verify/replay commands
+- `EVID-CRYPTO-90-001` - Crypto provider registry routing (DONE)
+
+---
+
+## 12. Success Metrics
+
+| Metric | Target |
+|--------|--------|
+| Bundle hash reproducibility | 100% (bit-identical) |
+| DSSE verification success rate | 100% for valid bundles |
+| Replay record ingestion latency | < 100ms p99 |
+| Incident mode activation time | < 1 second |
+| Offline verification success | Works without network |
+
+---
+
+*Last updated: 2025-11-29*