Implement ledger metrics for observability and add tests for Ruby packages endpoints
Some checks failed
Docs CI / lint-and-preview (push) Has been cancelled

- Added `LedgerMetrics` class to record write latency and total events for ledger operations.
- Created comprehensive tests for Ruby packages endpoints, covering scenarios for missing inventory, successful retrieval, and identifier handling.
- Introduced `TestSurfaceSecretsScope` for managing environment variables during tests.
- Developed `ProvenanceMongoExtensions` for attaching DSSE provenance and trust information to event documents.
- Implemented `EventProvenanceWriter` and `EventWriter` classes for managing event provenance in MongoDB.
- Established MongoDB indexes for efficient querying of events based on provenance and trust.
- Added models and JSON parsing logic for DSSE provenance and trust information.
This commit is contained in:
master
2025-11-13 09:29:09 +02:00
parent 151f6b35cc
commit 61f963fd52
101 changed files with 5881 additions and 1776 deletions

View File

@@ -0,0 +1,61 @@
# Findings Ledger — Air-Gap Provenance Extensions (LEDGER-AIRGAP-56/57/58)
> **Scope:** How ledger events capture mirror bundle provenance, staleness metrics, evidence snapshots, and sealed-mode timeline events for air-gapped deployments.
## 1. Requirements recap
- **LEDGER-AIRGAP-56-001:** Record mirror bundle metadata (`bundle_id`, `merkle_root`, `time_anchor`, `source_region`) whenever advisories/VEX/policies are imported offline. Tie import provenance to each affected ledger event.
- **LEDGER-AIRGAP-56-002:** Surface staleness metrics and enforce risk-critical export blocks when imported data exceeds freshness SLAs; emit remediation guidance.
- **LEDGER-AIRGAP-57-001:** Link findings evidence snapshots (portable bundles) so cross-enclave verification can attest to the same ledger hash.
- **LEDGER-AIRGAP-58-001:** Emit sealed-mode timeline events describing bundle impacts (new findings, remediation deltas) for Console and Notify.
## 2. Schema additions
| Entity | Field | Type | Notes |
| --- | --- | --- | --- |
| `ledger_events.event_body` | `airgap.bundle` | object | `{ "bundleId", "merkleRoot", "timeAnchor", "sourceRegion", "importedAt", "importOperator" }` recorded on import events. |
| `ledger_events.event_body` | `airgap.evidenceSnapshot` | object | `{ "bundleUri", "dsseDigest", "expiresAt" }` for findings evidence bundles. |
| `ledger_projection` | `airgap.stalenessSeconds` | integer | Age of newest data feeding the finding projection. |
| `ledger_projection` | `airgap.bundleId` | string | Last bundle influencing the projection row. |
| `timeline_events` (new view) | `airgapImpact` | object | Materials needed for LEDGER-AIRGAP-58-001 timeline feed (finding counts, severity deltas). |
Canonical JSON must sort object keys (`bundleId`, `importOperator`, …) to keep hashes deterministic.
## 3. Import workflow
1. **Mirror bundle validation:** AirGap controller verifies bundle signature/manifest before ingest; saves metadata for ledger enrichment.
2. **Event enrichment:** The importer populates `airgap.bundle` fields on each event produced from the bundle. `bundleId` equals manifest digest (SHA-256). `merkleRoot` is the bundles manifest Merkle root; `timeAnchor` is the authoritative timestamp from the bundle.
3. **Anchoring:** Merkle batching includes bundle metadata; anchor references in `ledger_merkle_roots.anchor_reference` use format `airgap::<bundleId>` when not externally anchored.
4. **Projection staleness:** Projector updates `airgap.stalenessSeconds` comparing current time with `bundle.timeAnchor` per artifact scope; CLI + Console read the value to display freshness indicators.
## 4. Staleness enforcement
- Config option `AirGapPolicies:FreshnessThresholdSeconds` (default 604800 = 7days) sets allowable age.
- Export workflows check `airgap.stalenessSeconds`; when over threshold the service raises `ERR_AIRGAP_STALE` and supplies remediation message referencing the last bundle (`bundleId`, `timeAnchor`, `importOperator`).
- Metrics (`ledger_airgap_staleness_seconds`) track distribution per tenant for dashboards.
## 5. Evidence snapshots
- Evidence bundles (`airgap.evidenceSnapshot`) reference portable DSSE packages stored in Evidence Locker (`bundleUri` like `file://offline/evidence/<bundleId>.tar`).
- CLI command `stella ledger evidence link` attaches evidence snapshots to findings after bundle generation; ledger event records both DSSE digest and expiration.
- Timeline entries and Console detail views display “Evidence snapshot available” with download instructions suited for sealed environments.
## 6. Timeline events (LEDGER-AIRGAP-58-001)
- New derived view `timeline_airgap_impacts` emits JSON objects such as:
```json
{
"tenant": "tenant-a",
"bundleId": "bundle-sha256:…",
"newFindings": 42,
"resolvedFindings": 18,
"criticalDelta": +5,
"timeAnchor": "2025-10-30T11:00:00Z",
"sealedMode": true
}
```
- Console + Notify subscribe to `ledger.airgap.timeline` events to show sealed-mode summaries.
## 7. Offline kit considerations
- Include bundle provenance schema, staleness policy config, CLI scripts (`stella airgap bundle import`, `stella ledger evidence link`), and sample manifests.
- Provide validation script `scripts/ledger/validate-airgap-bundle.sh` verifying manifest signatures, timestamps, and ledger enrichment before ingest.
- Document sealed-mode toggles ensuring no external egress occurs when importing bundles.
---
*Draft 2025-11-13 for LEDGER-AIRGAP-56/57/58 planning.*