Add tests and implement timeline ingestion options with NATS and Redis subscribers

- Introduced `BinaryReachabilityLifterTests` to validate binary lifting functionality.
- Created `PackRunWorkerOptions` for configuring worker paths and execution persistence.
- Added `TimelineIngestionOptions` for configuring NATS and Redis ingestion transports.
- Implemented `NatsTimelineEventSubscriber` for subscribing to NATS events.
- Developed `RedisTimelineEventSubscriber` for reading from Redis Streams.
- Added `TimelineEnvelopeParser` to normalize incoming event envelopes.
- Created unit tests for `TimelineEnvelopeParser` to ensure correct field mapping.
- Implemented `TimelineAuthorizationAuditSink` for logging authorization outcomes.
This commit is contained in:
StellaOps Bot
2025-12-03 09:46:48 +02:00
parent e923880694
commit 35c8f9216f
520 changed files with 4416 additions and 31492 deletions

View File

@@ -1,17 +1,41 @@
# Sprint 124 - Ingestion & Evidence · 110.C) Excititor.VI
# Sprint 0124 · Excititor Ingestion & Evidence (Phase VI)
Active items only. Completed/historic work now resides in docs/implplan/archived/tasks.md (updated 2025-11-08).
## Topic & Scope
- Expose streaming, evidence, and attestation APIs with OpenAPI discovery/examples while keeping aggregation-only semantics.
- Add mirror bundle import telemetry and crypto provider abstraction for deterministic verification profiles.
- **Working directory:** `src/Excititor` (WebService) with AirGap/Policy coordination.
[Ingestion & Evidence] 110.C) Excititor.VI
Depends on: Sprint 110.C - Excititor.V
Summary: Ingestion & Evidence focus on Excititor (phase VI).
> **Prep:** Read `docs/modules/excititor/architecture.md` and the Excititor component `AGENTS.md` files before working any items listed below.
Task ID | State | Task description | Owners (Source)
--- | --- | --- | ---
EXCITITOR-WEB-OBS-52-001 `Timeline streaming` | DONE | Provide SSE/WebSocket bridges for VEX timeline events with tenant filters, pagination anchors, and guardrails so downstream consoles can monitor raw evidence changes in real time. Depends on EXCITITOR-OBS-52-001. | Excititor WebService Guild (src/Excititor/StellaOps.Excititor.WebService)
EXCITITOR-WEB-OBS-53-001 `Evidence APIs` | DONE | Expose `/evidence/vex/*` endpoints that fetch locker bundles, enforce scopes, and surface verification metadata without synthesizing verdicts. Depends on EXCITITOR-WEB-OBS-52-001. | Excititor WebService Guild, Evidence Locker Guild (src/Excititor/StellaOps.Excititor.WebService)
EXCITITOR-WEB-OBS-54-001 `Attestation APIs` | DONE | Add `/attestations/vex/*` endpoints returning DSSE verification state, builder identity, and chain-of-custody links so consumers never need direct datastore access. Depends on EXCITITOR-WEB-OBS-53-001. | Excititor WebService Guild (src/Excititor/StellaOps.Excititor.WebService)
EXCITITOR-WEB-OAS-61-001 `OpenAPI discovery` | DONE | Implement `/.well-known/openapi` with spec version metadata plus standard error envelopes, then update controller/unit tests accordingly. | Excititor WebService Guild (src/Excititor/StellaOps.Excititor.WebService)
EXCITITOR-WEB-OAS-62-001 `Examples & deprecation headers` | DONE | Publish curated examples for the new evidence/attestation/timeline endpoints, emit deprecation headers for legacy routes, and align SDK docs. Depends on EXCITITOR-WEB-OAS-61-001. | Excititor WebService Guild, API Governance Guild (src/Excititor/StellaOps.Excititor.WebService)
EXCITITOR-WEB-AIRGAP-58-001 `Bundle import telemetry` | DONE | Emit timeline events + audit logs for mirror bundle imports (bundle ID, scope, actor) and map sealed-mode violations to actionable remediation guidance. | Excititor WebService Guild, AirGap Importer/Policy Guilds (src/Excititor/StellaOps.Excititor.WebService)
EXCITITOR-CRYPTO-90-001 `Crypto provider abstraction` | DONE | Replace ad-hoc hashing/signing in connectors/exporters/OpenAPI discovery with `ICryptoProviderRegistry` implementations approved by security so evidence verification stays deterministic across crypto profiles. | Excititor WebService Guild, Security Guild (src/Excititor/StellaOps.Excititor.WebService)
## Dependencies & Concurrency
- Depends on Phase V outputs (portable bundles, mirror registration) and Evidence Locker manifests.
- Concurrency: OpenAPI discovery/examples and streaming can proceed in parallel; crypto provider registry relies on security contract.
## Documentation Prerequisites
- docs/modules/excititor/architecture.md
- docs/modules/excititor/implementation_plan.md
- docs/modules/excititor/observability/locker-manifest.md
- Excititor WebService AGENTS.md
## Delivery Tracker
| # | Task ID | Status | Key dependency / next step | Owners | Task Definition |
| --- | --- | --- | --- | --- | --- |
| 1 | EXCITITOR-WEB-OBS-52-001 | DONE | Depends on OBS-52 schema | Excititor WebService Guild | Provide SSE/WebSocket bridges for VEX timeline events with tenant filters, pagination anchors, guardrails. |
| 2 | EXCITITOR-WEB-OBS-53-001 | DONE | Depends on 52-001; locker manifest available | Excititor WebService Guild · Evidence Locker Guild | `/evidence/vex/*` endpoints fetching locker bundles, enforcing scopes, surfacing verification metadata; no verdicts. |
| 3 | EXCITITOR-WEB-OBS-54-001 | DONE | Depends on 53-001; DSSE manifests available | Excititor WebService Guild | `/attestations/vex/*` endpoints returning DSSE verification state, builder identity, chain-of-custody links. |
| 4 | EXCITITOR-WEB-OAS-61-001 | DONE | None | Excititor WebService Guild | Implement `/.well-known/openapi` with spec metadata + standard error envelope; update controller/unit tests. |
| 5 | EXCITITOR-WEB-OAS-62-001 | DONE | Depends on 61-001 | Excititor WebService Guild · API Governance Guild | Publish curated examples + deprecation headers for evidence/attestation/timeline endpoints; align SDK docs. |
| 6 | EXCITITOR-WEB-AIRGAP-58-001 | DONE | Depends on mirror thin bundle schema | Excititor WebService Guild · AirGap Importer/Policy Guilds | Emit timeline events + audit logs for mirror bundle imports (bundle ID, scope, actor); map sealed-mode violations to remediation. |
| 7 | EXCITITOR-CRYPTO-90-001 | DONE | Security-approved registry contract | Excititor WebService Guild · Security Guild | Replace ad-hoc hashing/signing with `ICryptoProviderRegistry` implementations for deterministic verification across profiles. |
## Execution Log
| Date (UTC) | Update | Owner |
| --- | --- | --- |
| 2025-12-03 | Normalised sprint to standard template; preserved task details/statuses. | Planning |
## Decisions & Risks
- Aggregation-only stance preserved across streaming/evidence/attestation endpoints.
- Crypto provider registry must remain deterministic; benchmark and feature-flag per profile.
- Mirror bundle telemetry relies on thin bundle schema; revisit if schema changes to avoid telemetry mismatch.
## Next Checkpoints
- Re-validate `/evidence` and `/attestations` responses after any locker/DSSE manifest updates.
- Run OpenAPI discovery/examples regression when new routes land or headers change.