128 lines
6.2 KiB
Markdown
128 lines
6.2 KiB
Markdown
# Timeline Forensics Guide
|
|
|
|
> **Imposed rule:** Timeline is append-only; events may not be deleted or rewritten. Redactions require creating a compensating `redaction_notice` event that references the original ULID.
|
|
|
|
The Timeline Indexer service aggregates structured events (scanner runs, policy verdicts, runtime posture, evidence locker activity) so operators can audit changes over time. This guide summarizes the event schema, query surfaces, persistence implementation, and integration points.
|
|
|
|
## 1. Event Model
|
|
|
|
| Field | Description |
|
|
| --- | --- |
|
|
| `event_id` | ULID identifying the event. |
|
|
| `tenant` | Tenant scope. |
|
|
| `timestamp` | UTC ISO-8601 time the event occurred. |
|
|
| `category` | Logical grouping (scanner, policy, runtime, evidence). |
|
|
| `details` | JSON payload describing the event; contract defined per producer. |
|
|
| `trace_id` | Optional distributed trace correlation ID. |
|
|
|
|
Events are stored append-only with tenant-specific partitions. Producers include Scanner WebService, Policy Engine, Zastava Observer, Evidence Locker, and Notify.
|
|
|
|
### Event kinds (normative)
|
|
|
|
- `scan.completed` - scanner job finished; includes SBOM digest, findings counts, determinism score.
|
|
- `policy.verdict` - policy engine decision with overlay cache version and allow/deny rationale.
|
|
- `attestation.verified` - attestation verification result with Rekor UUID and bundle digest.
|
|
- `evidence.ingested` - Evidence Locker write with WORM vault identifier.
|
|
- `notify.sent` - outbound notification with target channel and template id.
|
|
- `runtime.alert` - runtime enforcement or observation event from Zastava Observer.
|
|
- `redaction_notice` - compensating entry when data is logically withdrawn; must include `supersedes` ULID.
|
|
|
|
## 2. APIs
|
|
|
|
Native endpoints:
|
|
- `GET /timeline` - query timeline entries with filter parameters.
|
|
- `GET /timeline/{eventId}` - fetch a single timeline entry.
|
|
- `GET /timeline/{eventId}/evidence` - fetch evidence linked to a timeline entry.
|
|
- `POST /timeline/events` - ingestion ack endpoint.
|
|
|
|
Router/Gateway aliases for microservice transport routing:
|
|
- `GET /api/v1/timeline`
|
|
- `GET /api/v1/timeline/{eventId}`
|
|
- `GET /api/v1/timeline/{eventId}/evidence`
|
|
- `POST /api/v1/timeline/events`
|
|
|
|
API headers: `X-Stella-Tenant`, optional `X-Stella-TraceId`, and `If-None-Match` for cache revalidation.
|
|
|
|
## 3. Query Tips
|
|
|
|
- Use `category` + `trace_id` to follow a scan-to-policy-to-notification flow.
|
|
- Combine `tenant` and `timestamp` filters for SLA audits.
|
|
- CLI command `stella timeline list` mirrors the API for automation.
|
|
- For WORM verification, filter `category=evidence` and join on Evidence Locker bundle digest.
|
|
- Use `category=attestation.verified` and `details.rekor_uuid` to reconcile transparency proofs.
|
|
|
|
Example queries:
|
|
|
|
```sh
|
|
# Recent scan -> policy -> notify chain for a digest
|
|
stella timeline list --tenant acme --category scan.completed --subject sha256:abc... --limit 5
|
|
stella timeline list --tenant acme --category policy.verdict --trace-id <trace>
|
|
stella timeline list --tenant acme --category notify.sent --trace-id <trace>
|
|
|
|
# Export window for audit
|
|
curl -H "X-Stella-Tenant: acme" \
|
|
"https://console.example/api/v1/timeline/export?from=2025-11-01T00:00:00Z&to=2025-11-02T00:00:00Z" \
|
|
-o timeline-2025-11-01.ndjson
|
|
```
|
|
|
|
## 4. Persistence (EF Core)
|
|
|
|
- TimelineIndexer DAL is EF Core 10 based (`StellaOps.TimelineIndexer.Infrastructure/Db`).
|
|
- Database-first scaffolded context/models are under:
|
|
- `src/TimelineIndexer/StellaOps.TimelineIndexer/StellaOps.TimelineIndexer.Infrastructure/EfCore/Context`
|
|
- `src/TimelineIndexer/StellaOps.TimelineIndexer/StellaOps.TimelineIndexer.Infrastructure/EfCore/Models`
|
|
- Keep generated files regeneration-safe. Place manual fixes in partial classes/files.
|
|
|
|
Scaffold command used for baseline generation:
|
|
|
|
```sh
|
|
dotnet ef dbcontext scaffold "Host=localhost;Port=55433;Database=postgres;Username=postgres;Password=postgres" \
|
|
Npgsql.EntityFrameworkCore.PostgreSQL \
|
|
--project src/TimelineIndexer/StellaOps.TimelineIndexer/StellaOps.TimelineIndexer.Infrastructure/StellaOps.TimelineIndexer.Infrastructure.csproj \
|
|
--startup-project src/TimelineIndexer/StellaOps.TimelineIndexer/StellaOps.TimelineIndexer.WebService/StellaOps.TimelineIndexer.WebService.csproj \
|
|
--context TimelineIndexerDbContext \
|
|
--context-dir EfCore/Context \
|
|
--output-dir EfCore/Models \
|
|
--schema timeline \
|
|
--table timeline_events \
|
|
--table timeline_event_details \
|
|
--table timeline_event_digests \
|
|
--no-onconfiguring \
|
|
--use-database-names \
|
|
--force
|
|
```
|
|
|
|
Expected scaffold caveat: enum `timeline.event_severity` and composite FKs on `(event_id, tenant_id)` are not fully inferred by the scaffold command; these are restored in partial mapping files under `EfCore/Context` and `EfCore/Models`.
|
|
|
|
Generate/refresh compiled EF model artifacts:
|
|
|
|
```sh
|
|
dotnet ef dbcontext optimize \
|
|
--project src/TimelineIndexer/StellaOps.TimelineIndexer/StellaOps.TimelineIndexer.Infrastructure/StellaOps.TimelineIndexer.Infrastructure.csproj \
|
|
--startup-project src/TimelineIndexer/StellaOps.TimelineIndexer/StellaOps.TimelineIndexer.WebService/StellaOps.TimelineIndexer.WebService.csproj \
|
|
--context TimelineIndexerDbContext \
|
|
--output-dir EfCore/CompiledModels \
|
|
--namespace StellaOps.TimelineIndexer.Infrastructure.EfCore.CompiledModels
|
|
```
|
|
|
|
Design-time connection can be overridden with `STELLAOPS_TIMELINEINDEXER_EF_CONNECTION`.
|
|
|
|
Runtime usage is explicit in `TimelineIndexerDbContextFactory` via `UseModel(TimelineIndexerDbContextModel.Instance)` so the static compiled model module is always used by DAL stores.
|
|
|
|
## 5. Integration
|
|
|
|
- Evidence Locker attaches evidence bundle digests; the console links from timeline to evidence viewer.
|
|
- Notifier creates acknowledgement events for incident workflows.
|
|
- Offline kits package timeline exports for compliance reviews.
|
|
|
|
Retention: events are retained per-tenant for at least `timeline.retention_days` (default 400 days) and replicated to cold storage weekly. Index rebuilds must preserve ordering and ULIDs.
|
|
|
|
Privacy/PII: producers must redact PII before emission; once emitted, redactions occur via `redaction_notice` only.
|
|
|
|
## 6. References
|
|
|
|
- `docs/modules/telemetry/architecture.md`
|
|
- `docs/modules/zastava/architecture.md`
|
|
- `docs/modules/export-center/architecture.md`
|
|
- `src/TimelineIndexer/StellaOps.TimelineIndexer`
|