up

2025-11-29 11:08:08 +02:00
parent 7e7be4d2fd
commit 3488b22c0c
102 changed files with 18487 additions and 969 deletions
--- a/docs/product-advisories/29-Nov-2025
+++ b/docs/product-advisories/29-Nov-2025
@@ -0,0 +1,407 @@
+# Findings Ledger and Immutable Audit Trail
+
+**Version:** 1.0
+**Date:** 2025-11-29
+**Status:** Canonical
+
+This advisory defines the product rationale, ledger semantics, and implementation strategy for the Findings Ledger module, covering append-only events, Merkle anchoring, projections, and deterministic exports.
+
+---
+
+## 1. Executive Summary
+
+The Findings Ledger provides **immutable, auditable records** of all vulnerability findings and their state transitions. Key capabilities:
+
+- **Append-Only Events** - Every finding change recorded permanently
+- **Merkle Anchoring** - Cryptographic proof of event ordering
+- **Projections** - Materialized current state views
+- **Deterministic Exports** - Reproducible compliance archives
+- **Chain Integrity** - Hash-linked event sequences per tenant
+
+---
+
+## 2. Market Drivers
+
+### 2.1 Target Segments
+
+| Segment | Ledger Requirements | Use Case |
+|---------|---------------------|----------|
+| **Compliance** | Immutable audit trail | SOC 2, FedRAMP evidence |
+| **Security Teams** | Finding history | Investigation timelines |
+| **Legal/eDiscovery** | Tamper-proof records | Litigation support |
+| **Auditors** | Verifiable exports | Third-party attestation |
+
+### 2.2 Competitive Positioning
+
+Most vulnerability tools provide mutable databases. Stella Ops differentiates with:
+- **Append-only architecture** ensuring no record deletion
+- **Merkle trees** for cryptographic verification
+- **Chain integrity** with hash-linked events
+- **Deterministic exports** for reproducible audits
+- **Air-gap support** with signed bundles
+
+---
+
+## 3. Event Model
+
+### 3.1 Ledger Event Structure
+
+```json
+{
+  "id": "uuid",
+  "type": "finding.status.changed",
+  "tenant": "acme-corp",
+  "chainId": "chain-uuid",
+  "sequence": 12345,
+  "policyVersion": "sha256:abc...",
+  "finding": {
+    "id": "artifact:sha256:...|pkg:npm/lodash",
+    "artifactId": "sha256:...",
+    "vulnId": "CVE-2025-12345"
+  },
+  "actor": {
+    "id": "user:jane@acme.com",
+    "type": "human"
+  },
+  "occurredAt": "2025-11-29T12:00:00Z",
+  "recordedAt": "2025-11-29T12:00:01Z",
+  "payload": {
+    "previousStatus": "open",
+    "newStatus": "triaged",
+    "reason": "Under investigation"
+  },
+  "evidenceBundleRef": "bundle://tenant/2025/11/29/...",
+  "eventHash": "sha256:...",
+  "previousHash": "sha256:...",
+  "merkleLeafHash": "sha256:..."
+}
+```
+
+### 3.2 Event Types
+
+| Type | Trigger | Payload |
+|------|---------|---------|
+| `finding.discovered` | New finding | severity, purl, advisory |
+| `finding.status.changed` | State transition | old/new status, reason |
+| `finding.verdict.changed` | Policy decision | verdict, rules matched |
+| `finding.vex.applied` | VEX override | status, justification |
+| `finding.assigned` | Owner change | assignee, team |
+| `finding.commented` | Annotation | comment text (redacted) |
+| `finding.resolved` | Resolution | resolution type, version |
+
+### 3.3 Chain Semantics
+
+- Each tenant has one or more event chains
+- Events are strictly ordered by sequence number
+- `previousHash` links to prior event for integrity
+- Chain forks are prohibited (409 on conflict)
+
+---
+
+## 4. Merkle Anchoring
+
+### 4.1 Tree Structure
+
+```
+              Root Hash
+             /         \
+        Hash(A+B)     Hash(C+D)
+        /     \       /     \
+      H(E1)  H(E2)  H(E3)  H(E4)
+        |      |      |      |
+      Event1 Event2 Event3 Event4
+```
+
+### 4.2 Anchoring Process
+
+1. **Batch collection** - Events accumulate in windows (default 15 min)
+2. **Tree construction** - Leaves are event hashes
+3. **Root computation** - Merkle root represents batch
+4. **Anchor record** - Root stored with timestamp
+5. **Optional external** - Root can be published to external ledger
+
+### 4.3 Configuration
+
+```yaml
+findings:
+  ledger:
+    merkle:
+      batchSize: 1000
+      windowDuration: 00:15:00
+      algorithm: sha256
+      externalAnchor:
+        enabled: false
+        type: rekor  # or custom
+```
+
+---
+
+## 5. Projections
+
+### 5.1 Purpose
+
+Projections provide **current state** views derived from event history. They are:
+- Materialized for fast queries
+- Reconstructible from events
+- Validated via `cycleHash`
+
+### 5.2 Finding Projection
+
+```json
+{
+  "tenantId": "acme-corp",
+  "findingId": "artifact:sha256:...|pkg:npm/lodash@4.17.20",
+  "policyVersion": "sha256:5f38c...",
+  "status": "triaged",
+  "severity": 6.7,
+  "riskScore": 85.2,
+  "riskSeverity": "high",
+  "riskProfileVersion": "v2.1",
+  "labels": {
+    "kev": true,
+    "runtime": "exposed"
+  },
+  "currentEventId": "uuid",
+  "cycleHash": "sha256:...",
+  "policyRationale": [
+    "explain://tenant/findings/...",
+    "policy://tenant/policy-v1/rationale/accepted"
+  ],
+  "updatedAt": "2025-11-29T12:00:00Z"
+}
+```
+
+### 5.3 Projection Refresh
+
+| Trigger | Action |
+|---------|--------|
+| New event | Incremental update |
+| Policy change | Full recalculation |
+| Manual request | On-demand rebuild |
+| Scheduled | Periodic validation |
+
+---
+
+## 6. Export Capabilities
+
+### 6.1 Export Shapes
+
+| Shape | Description | Use Case |
+|-------|-------------|----------|
+| `canonical` | Full event detail | Complete audit |
+| `compact` | Summary fields only | Quick reports |
+
+### 6.2 Export Types
+
+**Findings Export:**
+```json
+{
+  "eventSequence": 12345,
+  "observedAt": "2025-11-29T12:00:00Z",
+  "findingId": "artifact:...|pkg:...",
+  "policyVersion": "sha256:...",
+  "status": "triaged",
+  "severity": 6.7,
+  "cycleHash": "sha256:...",
+  "evidenceBundleRef": "bundle://...",
+  "provenance": {
+    "policyVersion": "sha256:...",
+    "cycleHash": "sha256:...",
+    "ledgerEventHash": "sha256:..."
+  }
+}
+```
+
+### 6.3 Export Formats
+
+- **JSON** - Paged API responses
+- **NDJSON** - Streaming exports
+- **Bundle** - Signed archive packages
+
+---
+
+## 7. Implementation Strategy
+
+### 7.1 Phase 1: Core Ledger (Complete)
+
+- [x] Append-only event store
+- [x] Hash-linked chains
+- [x] Basic projection engine
+- [x] REST API surface
+
+### 7.2 Phase 2: Merkle & Exports (In Progress)
+
+- [x] Merkle tree construction
+- [x] Batch anchoring
+- [ ] External anchor integration (LEDGER-MERKLE-50-001)
+- [ ] Deterministic NDJSON exports (LEDGER-EXPORT-51-001)
+
+### 7.3 Phase 3: Advanced Features (Planned)
+
+- [ ] Chain integrity verification CLI
+- [ ] Projection replay tooling
+- [ ] Cross-tenant federation
+- [ ] Long-term archival
+
+---
+
+## 8. API Surface
+
+### 8.1 Events
+
+| Endpoint | Method | Scope | Description |
+|----------|--------|-------|-------------|
+| `/v1/ledger/events` | GET | `vuln:audit` | List ledger events |
+| `/v1/ledger/events` | POST | `vuln:operate` | Append event |
+
+### 8.2 Projections
+
+| Endpoint | Method | Scope | Description |
+|----------|--------|-------|-------------|
+| `/v1/ledger/projections/findings` | GET | `vuln:view` | List projections |
+
+### 8.3 Exports
+
+| Endpoint | Method | Scope | Description |
+|----------|--------|-------|-------------|
+| `/v1/ledger/export/findings` | GET | `vuln:audit` | Export findings |
+| `/v1/ledger/export/vex` | GET | `vuln:audit` | Export VEX |
+| `/v1/ledger/export/advisories` | GET | `vuln:audit` | Export advisories |
+| `/v1/ledger/export/sboms` | GET | `vuln:audit` | Export SBOMs |
+
+### 8.4 Attestations
+
+| Endpoint | Method | Scope | Description |
+|----------|--------|-------|-------------|
+| `/v1/ledger/attestations` | GET | `vuln:audit` | List verifications |
+
+---
+
+## 9. Storage Model
+
+### 9.1 Collections
+
+| Collection | Purpose | Key Indexes |
+|------------|---------|-------------|
+| `ledger_events` | Append-only events | `{tenant, chainId, sequence}` |
+| `ledger_chains` | Chain metadata | `{tenant, chainId}` |
+| `ledger_merkle_roots` | Anchor records | `{tenant, batchId, anchoredAt}` |
+| `finding_projections` | Current state | `{tenant, findingId}` |
+
+### 9.2 Integrity Constraints
+
+- Events are append-only (no update/delete)
+- Sequence numbers strictly monotonic
+- Hash chain validated on write
+- Merkle roots immutable
+
+---
+
+## 10. Observability
+
+### 10.1 Metrics
+
+- `ledger.events.appended_total{tenant,type}`
+- `ledger.events.rejected_total{reason}`
+- `ledger.merkle.batches_total`
+- `ledger.merkle.anchor_latency_seconds`
+- `ledger.projection.updates_total`
+- `ledger.projection.staleness_seconds`
+- `ledger.export.rows_total{type,shape}`
+
+### 10.2 SLO Targets
+
+| Metric | Target |
+|--------|--------|
+| Event append latency | < 50ms p95 |
+| Projection freshness | < 5 seconds |
+| Merkle anchor window | 15 minutes |
+| Export throughput | 10k rows/sec |
+
+---
+
+## 11. Security Considerations
+
+### 11.1 Immutability Guarantees
+
+- No UPDATE/DELETE operations exposed
+- Admin override requires audit event
+- Merkle roots provide tamper evidence
+- External anchoring for non-repudiation
+
+### 11.2 Access Control
+
+- `vuln:view` - Read projections
+- `vuln:investigate` - Triage actions
+- `vuln:operate` - State transitions
+- `vuln:audit` - Export and verify
+
+### 11.3 Data Protection
+
+- Sensitive payloads redacted in exports
+- Comment text hashed, not stored
+- PII filtered at ingest
+- Tenant isolation enforced
+
+---
+
+## 12. Air-Gap Support
+
+### 12.1 Offline Bundles
+
+- Signed NDJSON exports
+- Merkle proofs included
+- Time anchors from trusted source
+- Bundle verification CLI
+
+### 12.2 Staleness Tracking
+
+```yaml
+airgap:
+  staleness:
+    warningThresholdDays: 7
+    blockThresholdDays: 30
+    riskCriticalExportsBlocked: true
+```
+
+---
+
+## 13. Related Documentation
+
+| Resource | Location |
+|----------|----------|
+| Ledger schema | `docs/modules/findings-ledger/schema.md` |
+| OpenAPI spec | `docs/modules/findings-ledger/openapi/` |
+| Export guide | `docs/modules/findings-ledger/exports.md` |
+
+---
+
+## 14. Sprint Mapping
+
+- **Primary Sprint:** SPRINT_0186_0001_0001_record_deterministic_execution.md
+- **Related Sprints:**
+  - SPRINT_0120_0000_0001_policy_reasoning.md
+  - SPRINT_311_docs_tasks_md_xi.md
+
+**Key Task IDs:**
+- `LEDGER-CORE-40-001` - Event store (DONE)
+- `LEDGER-PROJ-41-001` - Projections (DONE)
+- `LEDGER-MERKLE-50-001` - Merkle anchoring (IN PROGRESS)
+- `LEDGER-EXPORT-51-001` - Deterministic exports (IN PROGRESS)
+- `LEDGER-AIRGAP-56-001` - Bundle provenance (TODO)
+
+---
+
+## 15. Success Metrics
+
+| Metric | Target |
+|--------|--------|
+| Event durability | 100% (no data loss) |
+| Chain integrity | 100% hash verification |
+| Projection accuracy | 100% event replay match |
+| Export determinism | 100% hash reproducibility |
+| Audit compliance | SOC 2 Type II |
+
+---
+
+*Last updated: 2025-11-29*