bc4318ef97
- Created `StellaOps.TestKit.Tests` project for unit tests related to determinism. - Implemented `DeterminismManifestTests` to validate deterministic output for canonical bytes and strings, file read/write operations, and error handling for invalid schema versions. - Added `SbomDeterminismTests` to ensure identical inputs produce consistent SBOMs across SPDX 3.0.1 and CycloneDX 1.6/1.7 formats, including parallel execution tests. - Updated project references in `StellaOps.Integration.Determinism` to include the new determinism testing library.
1.2 KiB
1.2 KiB
Notifications runbook
Purpose
- Deploy and operate the Notifications WebService and Worker.
Pre-flight
- Secrets stored in Authority (SMTP, Slack, webhook HMAC).
- Outbound allowlist configured for channels.
- PostgreSQL and Valkey reachable; health checks pass.
- Offline kit loaded with templates and rule seeds.
Deploy
- Deploy images with digests pinned.
- Set Notify Postgres, Redis, Authority, and allowlist settings.
- Warm caches via /api/v1/notify/admin/warm when needed.
Monitor
- notify_delivery_attempts_total by status and channel.
- notify_escalation_stage_total and notify_rule_eval_seconds.
- Logs include tenant, ruleId, deliveryId, channel, status.
Common operations
- List failed deliveries and replay.
- Pause a tenant without dropping audit events.
- Rotate channel secrets via refresh endpoints.
Failure recovery
- Validate templates and Redis connectivity for worker crashes.
- Replay deliveries after database recovery.
- Disable channels during upstream outages.
Determinism safeguards
- Rule snapshots versioned per tenant.
- Template rendering uses deterministic helpers.
- UTC time sources for quiet hours.
Related references
- notifications/overview.md
- notifications/rules.md
- docs/operations/notifier-runbook.md