Add tests and implement timeline ingestion options with NATS and Redis subscribers

- Introduced `BinaryReachabilityLifterTests` to validate binary lifting functionality.
- Created `PackRunWorkerOptions` for configuring worker paths and execution persistence.
- Added `TimelineIngestionOptions` for configuring NATS and Redis ingestion transports.
- Implemented `NatsTimelineEventSubscriber` for subscribing to NATS events.
- Developed `RedisTimelineEventSubscriber` for reading from Redis Streams.
- Added `TimelineEnvelopeParser` to normalize incoming event envelopes.
- Created unit tests for `TimelineEnvelopeParser` to ensure correct field mapping.
- Implemented `TimelineAuthorizationAuditSink` for logging authorization outcomes.
This commit is contained in:
StellaOps Bot
2025-12-03 09:46:48 +02:00
parent e923880694
commit 35c8f9216f
520 changed files with 4416 additions and 31492 deletions

View File

@@ -1,17 +1,41 @@
# Sprint 121 - Ingestion & Evidence · 110.C) Excititor.III
# Sprint 0121 · Excititor Ingestion & Evidence (Phase III)
Active items only. Completed/historic work now resides in docs/implplan/archived/tasks.md (updated 2025-11-08).
## Topic & Scope
- Deliver Link-Not-Merge foundations in Excititor: storage, conflict annotations, event emission, and read APIs.
- Expose observation/linkset endpoints with deterministic pagination and no derived verdicts; document contracts and SDK examples.
- Publish ingest observability (latency, conflict rate, signature verification) and SLO alerts for AOC evidence freshness.
- **Working directory:** `src/Excititor` (Storage.Mongo, Core, WebService) and related docs.
[Ingestion & Evidence] 110.C) Excititor.III
Depends on: Sprint 110.C - Excititor.II
Summary: Ingestion & Evidence focus on Excititor (phase III).
> **Prep:** Read `docs/modules/excititor/architecture.md` and the Excititor component `AGENTS.md` guidance before acting on these tasks (requirement carried over from the component boards).
Task ID | State | Task description | Owners (Source)
--- | --- | --- | ---
EXCITITOR-LNM-21-001 `Observation & linkset stores` | DONE | Stand up `vex_observations` and `vex_linksets` collections with shard keys, tenant guards, and migrations that retire any residual merge-era data without mutating raw content. | Excititor Storage Guild (src/Excititor/__Libraries/StellaOps.Excititor.Storage.Mongo)
EXCITITOR-LNM-21-002 `Conflict annotations` | DONE | Capture disagreement metadata (status + justification deltas) directly inside linksets with confidence scores so downstream consumers can highlight conflicts without Excititor choosing winners. Depends on EXCITITOR-LNM-21-001. | Excititor Core Guild (src/Excititor/__Libraries/StellaOps.Excititor.Core)
EXCITITOR-LNM-21-003 `Event emission` | DONE | Emit `vex.linkset.updated` events and describe payload shape (observation ids, confidence, conflict summary) so Policy/Lens/UI can subscribe while Excititor stays aggregation-only. Depends on EXCITITOR-LNM-21-002. | Excititor Core Guild, Platform Events Guild (src/Excititor/__Libraries/StellaOps.Excititor.Core)
EXCITITOR-LNM-21-201 `Observation APIs` | DONE | Ship `/vex/observations` read endpoints with filters for advisory/product/issuer, strict RBAC, and deterministic pagination (no derived verdict fields). Depends on EXCITITOR-LNM-21-003. | Excititor WebService Guild (src/Excititor/StellaOps.Excititor.WebService)
EXCITITOR-LNM-21-202 `Linkset APIs` | DONE | Provide `/vex/linksets` + export endpoints that surface alias mappings, conflict markers, and provenance proofs exactly as stored; errors must map to `ERR_AGG_*`. Depends on EXCITITOR-LNM-21-201. | Excititor WebService Guild (src/Excititor/StellaOps.Excititor.WebService)
EXCITITOR-LNM-21-203 `Docs & SDK examples` | DONE | Update OpenAPI, SDK smoke tests, and documentation to cover the new observation/linkset endpoints with realistic examples Advisory AI/Lens teams can rely on. Depends on EXCITITOR-LNM-21-202. | Excititor WebService Guild, Docs Guild (src/Excititor/StellaOps.Excititor.WebService)
EXCITITOR-OBS-51-001 `Metrics & SLOs` | DONE | Publish ingest latency, scope resolution success, conflict rate, and signature verification metrics plus SLO burn alerts so we can prove Excititor meets the AOC "evidence freshness" mission. | Excititor Core Guild, DevOps Guild (src/Excititor/__Libraries/StellaOps.Excititor.Core)
## Dependencies & Concurrency
- Depends on Sprint 110.C (Excititor II) groundwork.
- Concurrency: storage → conflict annotations → events → APIs → docs; observability can proceed in parallel after storage is stable.
## Documentation Prerequisites
- docs/modules/excititor/architecture.md
- docs/modules/excititor/implementation_plan.md
- Component AGENTS.md under `src/Excititor/**`
## Delivery Tracker
| # | Task ID | Status | Key dependency / next step | Owners | Task Definition |
| --- | --- | --- | --- | --- | --- |
| 1 | EXCITITOR-LNM-21-001 | DONE | None (prereq storage) | Excititor Storage Guild (`src/Excititor/__Libraries/StellaOps.Excititor.Storage.Mongo`) | Stand up `vex_observations` and `vex_linksets` collections with shard keys, tenant guards, migrations that retire merge-era data without mutating raw content. |
| 2 | EXCITITOR-LNM-21-002 | DONE | Depends on 21-001 | Excititor Core Guild (`src/Excititor/__Libraries/StellaOps.Excititor.Core`) | Capture disagreement metadata (status + justification deltas) inside linksets with confidence scores; Excititor stays aggregation-only. |
| 3 | EXCITITOR-LNM-21-003 | DONE | Depends on 21-002 | Excititor Core Guild · Platform Events Guild | Emit `vex.linkset.updated` events describing payload (observation ids, confidence, conflict summary) for Policy/Lens/UI subscription. |
| 4 | EXCITITOR-LNM-21-201 | DONE | Depends on 21-003 | Excititor WebService Guild (`src/Excititor/StellaOps.Excititor.WebService`) | `/vex/observations` read endpoints with advisory/product/issuer filters, strict RBAC, deterministic pagination, no derived verdict fields. |
| 5 | EXCITITOR-LNM-21-202 | DONE | Depends on 21-201 | Excititor WebService Guild | `/vex/linksets` + export endpoints surfacing alias mappings, conflict markers, provenance proofs; errors map to `ERR_AGG_*`. |
| 6 | EXCITITOR-LNM-21-203 | DONE | Depends on 21-202 | Excititor WebService Guild · Docs Guild | Update OpenAPI, SDK smoke tests, and docs with realistic examples for observation/linkset endpoints. |
| 7 | EXCITITOR-OBS-51-001 | DONE | Parallel after storage ready | Excititor Core Guild · DevOps Guild | Publish ingest latency, scope-resolution success, conflict rate, signature verification metrics + SLO burn alerts. |
## Execution Log
| Date (UTC) | Update | Owner |
| --- | --- | --- |
| 2025-12-03 | Normalised sprint to standard template; preserved task statuses; working directory set. | Planning |
## Decisions & Risks
- Excititor remains aggregation-only; conflict data recorded but no verdict selection.
- Pagination deterministic: enforce stable ordering on observation/linkset reads to avoid replay drift.
- Event payloads must not leak tenant data across scopes; ensure RBAC on `/vex/*` endpoints and event topics.
## Next Checkpoints
- Confirm metrics dashboards remain in sync with OBS-51 signal names after later phases.
- Re-run SDK/example coverage when downstream services add new advisory/product fields.