Implement ledger metrics for observability and add tests for Ruby packages endpoints
Some checks failed
Docs CI / lint-and-preview (push) Has been cancelled

- Added `LedgerMetrics` class to record write latency and total events for ledger operations.
- Created comprehensive tests for Ruby packages endpoints, covering scenarios for missing inventory, successful retrieval, and identifier handling.
- Introduced `TestSurfaceSecretsScope` for managing environment variables during tests.
- Developed `ProvenanceMongoExtensions` for attaching DSSE provenance and trust information to event documents.
- Implemented `EventProvenanceWriter` and `EventWriter` classes for managing event provenance in MongoDB.
- Established MongoDB indexes for efficient querying of events based on provenance and trust.
- Added models and JSON parsing logic for DSSE provenance and trust information.
This commit is contained in:
master
2025-11-13 09:29:09 +02:00
parent 151f6b35cc
commit 61f963fd52
101 changed files with 5881 additions and 1776 deletions

View File

@@ -0,0 +1,45 @@
# Implementation Plan — Findings Ledger (Sprint 120)
## Phase 1 Observability baselines (LEDGER-29-007)
- Instrument writer/projector with metrics listed in `observability.md` (`ledger_write_latency_seconds`, `ledger_events_total`, `ledger_projection_lag_seconds`, etc.).
- Emit structured logs (Serilog JSON) including chain/sequence/hash metadata.
- Wire OTLP exporters, ensure `/metrics` endpoint exposes histogram buckets with exemplars.
- Publish Grafana dashboards + alert rules (Policy SLO pack).
- Deliver doc updates + sample Grafana JSON in repo (`docs/observability/dashboards/findings-ledger/`).
## Phase 2 Determinism harness (LEDGER-29-008)
- Finalize NDJSON fixtures for ≥5M findings/tenant (per tenant/test scenario).
- Implement `tools/LedgerReplayHarness` CLI as specified in `replay-harness.md`.
- Add GitHub/Gitea pipeline job(s) running nightly (1M) + weekly (5M) harness plus DSSE signing.
- Capture CPU/memory/latency metrics and commit signed reports for validation.
- Provide runbook for QA + Ops to rerun harness in their environments.
## Phase 3 Deployment & backup collateral (LEDGER-29-009)
- Integrate ledger service into Compose (`docker-compose.prod.yaml`) and Helm values.
- Automate PostgreSQL migrations (DatabaseMigrator invocation pre-start).
- Document backup cadence (pg_dump + WAL archiving) and projection rebuild process (call harness).
- Ensure Offline Kit packaging pulls binaries, migrations, harness, and default dashboards.
## Phase 4 Provenance & air-gap extensions
- LEDGER-34-101: ingest orchestrator run export metadata, index by artifact hash, expose audit endpoint.
- LEDGER-AIRGAP-56/57/58: extend ledger events to capture bundle provenance, staleness metrics, timeline events.
- LEDGER-ATTEST-73-001: store attestation pointers (DSSE IDs, Rekor metadata) for explainability.
- For each extension, update schema doc + workflow inference doc to describe newly recorded fields and tenant-safe defaults.
## Dependencies & sequencing
1. AdvisoryAI Sprint 110.A completion (raw findings parity).
2. Observability schema approval (Nov15) to unblock Phase 1 instrumentation.
3. QA lab capacity for 5M replay (Nov18 checkpoint).
4. DevOps review of Compose/Helm overlays (Nov20).
5. Orchestrator export schema freeze (Nov25) for provenance linkage.
## Deliverables checklist
- [ ] Metrics/logging/tracing implementation merged, dashboards exported.
- [ ] Harness CLI + fixtures + signed reports committed.
- [ ] Compose/Helm overlays + backup/restore runbooks validated.
- [ ] Air-gap provenance fields documented + implemented.
- [ ] Sprint tracker and release notes updated after each phase.
---
*Draft: 2025-11-13. Update when sequencing or dependencies change.*