7.9 KiB
7.9 KiB
Sprint 0165-0001-0001 · Timeline Indexer (Export & Evidence 160.C)
Topic & Scope
- Bootstrap Timeline Indexer service: migrations/RLS, ingestion, query APIs, and evidence linkage.
- Keep ordering deterministic and tenant-scoped; link timeline events to evidence bundle digests/attestations.
- Working directory:
src/TimelineIndexer/StellaOps.TimelineIndexer.
Dependencies & Concurrency
- Upstream: AdvisoryAI (110.A), AirGap (120.A), Scanner (130.A), Orchestrator (150.A) schemas required for event payloads.
- Concurrency: execute tasks in listed order; evidence linkage follows ingestion and API/RLS work.
Documentation Prerequisites
- docs/README.md
- docs/07_HIGH_LEVEL_ARCHITECTURE.md
- docs/modules/platform/architecture-overview.md
- docs/modules/export-center/architecture.md (for evidence linkage)
- src/TimelineIndexer/StellaOps.TimelineIndexer/AGENTS.md (if present)
Delivery Tracker
| # | Task ID | Status | Key dependency / next step | Owners | Task Definition |
|---|---|---|---|---|---|
| 1 | TIMELINE-OBS-52-001 | DONE (2025-11-30) | Postgres schema + RLS committed; evidence linkage table aligned to bundle contract | Timeline Indexer Guild (src/TimelineIndexer/StellaOps.TimelineIndexer) |
Bootstrap service; Postgres migrations for timeline_events, timeline_event_details, timeline_event_digests; enable RLS scaffolding and deterministic migration scripts. |
| 2 | TIMELINE-OBS-52-002 | DOING (2025-11-30) | Ingestion worker + metrics + subscriber abstraction implemented; waiting on NATS/Redis subject/schema to bind transports | Timeline Indexer Guild | Implement event ingestion pipeline (NATS/Redis consumers) with ordering guarantees, dedupe (event_id, tenant_id), trace-ID correlation, backpressure metrics. |
| 3 | TIMELINE-OBS-52-003 | BLOCKED (2025-11-30) | BLOCKED by 52-002; ingestion contract not defined yet. | Timeline Indexer Guild | Expose REST/gRPC APIs for timeline queries (GET /timeline, /timeline/{id}) with filters, pagination, tenant enforcement; provide OpenAPI + contract tests. |
| 4 | TIMELINE-OBS-52-004 | BLOCKED (2025-11-30) | Blocked by 52-003; upstream schema + RLS review pending. | Timeline Indexer Guild · Security Guild | Finalize RLS policies, scope checks (timeline:read), audit logging; integration tests for cross-tenant isolation and legal hold markers. |
| 5 | TIMELINE-OBS-53-001 | BLOCKED (2025-11-30) | Blocked by 52-004 and awaiting EvidenceLocker bundle digest linkage tests. | Timeline Indexer Guild · Evidence Locker Guild | Link timeline events to evidence bundle digests + attestation subjects; expose /timeline/{id}/evidence returning signed manifest references. |
Wave Coordination
- Wave 1: TIMELINE-OBS-52 chain (service bootstrap → ingestion → APIs → RLS/policies).
- Wave 2: Evidence linkage (TIMELINE-OBS-53-001) after digest schema lands and RLS is approved.
Wave Detail Snapshots
- Wave 1 deliverable: tenant-scoped timeline service with deterministic ingestion, pagination, and RLS/audit logging ready for Security review.
- Wave 2 deliverable: evidence linkage endpoint returning signed manifest references tied to EvidenceLocker digests/attestations.
Interlocks
| Dependency | Impacts | Status / Next signal |
|---|---|---|
| Orchestrator/Notifications event schema | Tasks 2–4 | Pending; required to finalize ingestion payload and API fields. |
| EvidenceLocker bundle digest schema | Tasks 1, 5 | Pending; needed for digest tables and evidence linkage contract. |
| Security/Compliance RLS review | Task 4 | Pending; schedule once RLS proposal is drafted. |
Action Tracker
| # | Action | Owner | Due (UTC) | Status |
|---|---|---|---|---|
| 1 | Attach orchestrator/notification event schema sample to sprint doc. | Timeline Indexer Guild | 2025-12-02 | OPEN |
| 2 | Obtain EvidenceLocker digest schema/sample manifest for linkage design. | Timeline Indexer Guild · Evidence Locker Guild | 2025-12-02 | OPEN |
| 3 | Draft RLS/migration proposal and route to Security/Compliance for approval. | Timeline Indexer Guild | 2025-12-04 | OPEN |
Upcoming Checkpoints
- Schema drop ETA for orchestrator/notification events (TBD).
- EvidenceLocker digest schema publication (TBD).
- Security/Compliance review for RLS proposal (TBD).
Decisions & Risks
| Risk / Decision | Impact | Mitigation / Next step | Status |
|---|---|---|---|
| Orchestrator/notification schemas not yet published. | Blocks ingestion and API field definitions (TIMELINE-OBS-52-002/003). | Track Action 1; keep tasks BLOCKED until schema attached. | OPEN |
| EvidenceLocker digest schema pending. | Blocks digest table shape and evidence linkage (TIMELINE-OBS-53-001). | Track Action 2; keep tasks BLOCKED. | OPEN |
| RLS review not scheduled. | Could delay production readiness of policies (TIMELINE-OBS-52-004). | Track Action 3; schedule with Security once draft ready. | OPEN |
Baseline docs may change (docs/modules/orchestrator/event-envelope.md, docs/modules/evidence-locker/prep/2025-11-24-evidence-locker-contract.md). |
Schema drift could invalidate migrations. | Monitor upstream doc updates; re-run schema diff before coding resumes. | OPEN |
Risk table
| Risk | Severity | Mitigation / Owner |
|---|---|---|
| Orchestrator/notification schema slip. | High | Action 1 to secure sample; keep Wave 1 blocked until delivered. Owner: Timeline Indexer Guild. |
| EvidenceLocker digest schema slip. | High | Action 2 to obtain schema; block evidence linkage until received. Owner: Timeline Indexer Guild · Evidence Locker Guild. |
| RLS review delayed. | Medium | Action 3 to draft and schedule review with Security/Compliance. Owner: Timeline Indexer Guild. |
| Schema drift after migrations drafted. | Medium | Re-run schema diff against upstream docs before coding resumes. Owner: Timeline Indexer Guild. |
Execution Log
| Date (UTC) | Update | Owner |
|---|---|---|
| 2025-11-30 | Ran TimelineIndexer unit tests (TimelineIndexer.Tests) successfully after ingestion worker + query option fixes; still waiting on transport schema to wire NATS/Redis. | Implementer |
| 2025-11-30 | Implemented TimelineIngestionWorker with subscriber abstraction, session dedupe, and metrics counters; awaiting NATS/Redis subject schema to wire real transports. | Implementer |
| 2025-11-30 | Started TIMELINE-OBS-52-002: added ingestion service, Postgres store, and deterministic payload hashing; queue bindings pending schema alignment. | Implementer |
| 2025-11-30 | Normalized sprint to AGENTS template (Wave/Interlocks/Action tracker) while keeping prior content intact. | Implementer |
| 2025-11-30 | Completed TIMELINE-OBS-52-001: added Postgres schema/RLS migrations, DataSource + migration runner wiring; test run attempted for module but cancelled due to long solution restore—manual rerun needed. | Implementer |
| 2025-11-30 | Located orchestrator event envelope draft and Evidence Locker bundle contract; unblocked migrations and RLS design for TIMELINE-OBS-52-001 and started implementation. | Implementer |
| 2025-11-30 | Built TimelineIndexer solution successfully after query options fix; dotnet test on TimelineIndexer.Tests now passing (9 tests). |
Implementer |
| 2025-11-30 | Re-checked for orchestrator/notification schema and EvidenceLocker bundle digest; none landed in docs/events or docs/modules/evidence-locker, so keeping all tasks blocked. |
Implementer |
| 2025-11-25 | Marked TIMELINE-OBS-52-001 BLOCKED: missing orchestrator/notification event schema and EvidenceLocker digest schema prevent drafting migrations/RLS. | Implementer |
| 2025-11-12 | Captured task snapshot and blockers; waiting on orchestrator/notifications schema and EvidenceLocker digest schema. | Planning |
| 2025-11-19 | Normalized sprint to standard template and renamed from SPRINT_165_timelineindexer.md to SPRINT_0165_0001_0001_timelineindexer.md; content preserved. |
Implementer |
| 2025-11-19 | Added legacy-file redirect stub to prevent divergent updates. | Implementer |