feat: Implement Filesystem and MongoDB provenance writers for PackRun execution context
- Added `FilesystemPackRunProvenanceWriter` to write provenance manifests to the filesystem. - Introduced `MongoPackRunArtifactReader` to read artifacts from MongoDB. - Created `MongoPackRunProvenanceWriter` to store provenance manifests in MongoDB. - Developed unit tests for filesystem and MongoDB provenance writers. - Established `ITimelineEventStore` and `ITimelineIngestionService` interfaces for timeline event handling. - Implemented `TimelineIngestionService` to validate and persist timeline events with hashing. - Created PostgreSQL schema and migration scripts for timeline indexing. - Added dependency injection support for timeline indexer services. - Developed tests for timeline ingestion and schema validation.
This commit is contained in:
@@ -8,6 +8,7 @@ Notify evaluates operator-defined rules against platform events and dispatches c
|
||||
- [Architecture](./architecture.md)
|
||||
- [Implementation plan](./implementation_plan.md)
|
||||
- [Task board](./TASKS.md)
|
||||
- [Observability runbook](./operations/observability.md) (offline import friendly)
|
||||
|
||||
## How to get started
|
||||
1. Open sprint file `/docs/implplan/SPRINT_*.md` and locate the stories referencing this module.
|
||||
|
||||
@@ -2,6 +2,11 @@
|
||||
|
||||
Notify (Notifications Studio) converts platform events into tenant-scoped alerts with deterministic delivery, offline parity, and a full audit trail. The service is split between the reusable tooling in `src/Notify/*` and the runtime host in `src/Notifier/*` (decision recorded 2025-11-02) so downstream systems can embed the rules engine without inheriting the Studio UI.
|
||||
|
||||
## Latest updates (2025-11-30)
|
||||
- Sprint tracker `docs/implplan/SPRINT_322_docs_modules_notify.md` and module `TASKS.md` added to mirror status.
|
||||
- Observability runbook stub and Grafana placeholder added under `operations/` (offline import); finalize after next demo.
|
||||
- NOTIFY-DOCS-0002 remains blocked pending NOTIFY-SVC-39-001..004 outputs (correlation/digests/simulation/quiet hours).
|
||||
|
||||
## Scope & responsibilities
|
||||
- Apply tenant-scoped rules to events from Scanner, Scheduler, VEX Lens, Attestor, Task Runner, and Zastava.
|
||||
- Render channel-specific payloads (Slack, Teams, Email, webhook) using deterministic templates with localisation safeguards.
|
||||
@@ -40,6 +45,7 @@ Status for these items is tracked in `src/Notifier/StellaOps.Notifier/TASKS.md`
|
||||
- Schema fixtures live in `./resources/schemas`; event and delivery samples live in `./resources/samples` for contract tests and UI mocks.
|
||||
- Offline Kit bundles ship plug-ins, default templates, and seed rules; update manifests under `ops/offline-kit/` when connectors change.
|
||||
- Dashboards and alert references depend on `DEVOPS-NOTIFY-39-002`; coordinate before renaming metrics or labels.
|
||||
- Observability assets: `operations/observability.md` and `operations/dashboards/notify-observability.json` (offline import).
|
||||
- When releasing new rule or connector features, mirror guidance into `docs/notifications/*.md` and checklists in `docs/updates/2025-10-29-notify-docs.md` until the follow-ups are closed.
|
||||
|
||||
## Epic alignment
|
||||
|
||||
10
docs/modules/notify/TASKS.md
Normal file
10
docs/modules/notify/TASKS.md
Normal file
@@ -0,0 +1,10 @@
|
||||
# Notify · TASKS (status mirror)
|
||||
|
||||
| Task ID | Status | Owner(s) | Notes / Evidence |
|
||||
| --- | --- | --- | --- |
|
||||
| NOTIFY-DOCS-0001 | DONE (2025-11-05) | Docs Guild | README refreshed for Notifications Studio pivot + release notes. |
|
||||
| NOTIFY-OPS-0001 | BLOCKED (2025-11-30) | Ops Guild | Await next demo outputs; observability runbook stub added. |
|
||||
| NOTIFY-ENG-0001 | DONE (2025-11-27) | Module Team | Implementation plan readiness tracker aligned with SPRINT_171..173. |
|
||||
| NOTIFY-DOCS-0002 | BLOCKED (2025-11-30) | Docs Guild | Pending NOTIFY-SVC-39-001..004 correlation/digests/simulation/quiet hours evidence. |
|
||||
|
||||
> Keep this table in lockstep with `docs/implplan/SPRINT_322_docs_modules_notify.md` (TODO/DOING/DONE/BLOCKED updates go to both files).
|
||||
@@ -384,6 +384,7 @@ Authority signs ack tokens using keys configured under `notifications.ackTokens`
|
||||
* `notify.sent_total{channel}` / `notify.failed_total{channel,code}`
|
||||
* `notify.delivery_latency_seconds{channel}` (end‑to‑end)
|
||||
* **Tracing**: spans `ingest`, `match`, `render`, `send`; correlation id = `eventId`.
|
||||
- Runbook + dashboard stub (offline import): `operations/observability.md`, `operations/dashboards/notify-observability.json` (to be populated after next demo).
|
||||
|
||||
**SLO targets**
|
||||
|
||||
|
||||
@@ -56,13 +56,18 @@
|
||||
- **Security:** RBAC tests, tenant isolation, secret reference validation, DSSE signature verification.
|
||||
- **Offline:** export/import round-trips, Offline Kit deployment, manual delivery replay.
|
||||
|
||||
## Definition of done
|
||||
- Notify service, workers, connectors, Console/CLI, observability, and Offline Kit assets shipped with documentation and runbooks.
|
||||
- Compliance checklist appended to docs; ./TASKS.md and ../../TASKS.md updated with progress.
|
||||
|
||||
---
|
||||
|
||||
## Sprint readiness tracker
|
||||
## Definition of done
|
||||
- Notify service, workers, connectors, Console/CLI, observability, and Offline Kit assets shipped with documentation and runbooks.
|
||||
- Compliance checklist appended to docs; ./TASKS.md and ../../TASKS.md updated with progress.
|
||||
|
||||
## Sprint alignment (2025-11-30)
|
||||
- Docs sprint: `docs/implplan/SPRINT_322_docs_modules_notify.md`; statuses mirrored in `docs/modules/notify/TASKS.md`.
|
||||
- Observability evidence stub: `operations/observability.md` and `operations/dashboards/notify-observability.json` (to be populated after next demo outputs).
|
||||
- NOTIFY-DOCS-0002 remains blocked pending NOTIFY-SVC-39-001..004 (correlation/digests/simulation/quiet hours); keep sprint/TASKS synced when those land.
|
||||
|
||||
---
|
||||
|
||||
## Sprint readiness tracker
|
||||
|
||||
> Last updated: 2025-11-27 (NOTIFY-ENG-0001)
|
||||
|
||||
|
||||
@@ -0,0 +1,6 @@
|
||||
{
|
||||
"_note": "Placeholder Grafana dashboard stub for Notify. Replace panels when metrics endpoints are available; keep offline-import friendly.",
|
||||
"schemaVersion": 39,
|
||||
"title": "Notify Observability (stub)",
|
||||
"panels": []
|
||||
}
|
||||
38
docs/modules/notify/operations/observability.md
Normal file
38
docs/modules/notify/operations/observability.md
Normal file
@@ -0,0 +1,38 @@
|
||||
# Notify observability runbook (stub · 2025-11-29 demo)
|
||||
|
||||
## Dashboards (offline import)
|
||||
- Grafana JSON: `docs/modules/notify/operations/dashboards/notify-observability.json` (import locally; no external data sources assumed).
|
||||
- Planned panels: enqueue/dequeue rate, delivery latency p95/p99, channel error rate, retry/dead-letter counts, rule evaluation latency, tenant isolation breaches (should stay 0), and notification simulation outcomes.
|
||||
|
||||
## Key metrics
|
||||
- `notify_enqueue_total{channel}` — notifications enqueued by channel.
|
||||
- `notify_delivery_latency_seconds_bucket{channel}` — delivery latency per channel.
|
||||
- `notify_delivery_failures_total{channel,reason}` — failed deliveries.
|
||||
- `notify_retry_total{channel}` and `notify_deadletter_total{channel}` — retries and dead letters.
|
||||
- `notify_rule_eval_duration_seconds_bucket` — rule evaluation latency.
|
||||
- `notify_simulation_total{result}` — simulation outcomes when quiet hours/correlation rules applied.
|
||||
|
||||
## Logs & traces
|
||||
- Correlate by `notificationId`, `ruleId`, `tenant`, `channel`. Include `quietHoursApplied`, `correlationKey`, `retries` fields.
|
||||
- Traces disabled by default for air-gap; enable by pointing OTLP exporter to on-prem collector.
|
||||
|
||||
## Health/diagnostics
|
||||
- `/health/liveness` and `/health/readiness` check queue backend reachability and channel provider credentials.
|
||||
- `/status` exposes build version, commit, feature flags; verify against offline bundle manifest.
|
||||
- Simulation probe: `/api/notify/simulate` with sample rule set to validate correlation/digest wiring once NOTIFY-SVC-39-001..004 land.
|
||||
|
||||
## Alert hints
|
||||
- Delivery latency p99 > 1.5s for email/webhook channels.
|
||||
- Dead-letter queue growth > threshold.
|
||||
- Rule evaluation latency p99 > 500ms.
|
||||
- Correlation/quiet-hours simulation failures once enabled.
|
||||
|
||||
## Offline verification steps
|
||||
1) Import Grafana JSON locally; point to Prometheus scrape labeled `notify`.
|
||||
2) Run `stella notify simulate --rules samples/rules.yaml --dry-run` (once available) and ensure metrics/logs emit locally.
|
||||
3) Fetch `/status` and compare commit/version to offline bundle manifest.
|
||||
|
||||
## Evidence locations
|
||||
- Sprint tracker: `docs/implplan/SPRINT_322_docs_modules_notify.md`.
|
||||
- Module docs: `README.md`, `architecture.md`, `implementation_plan.md`.
|
||||
- Dashboard stub: `operations/dashboards/notify-observability.json`.
|
||||
Reference in New Issue
Block a user