# Sprint 170 - Notifications & Telemetry

> **BLOCKED Tasks:** Before working on BLOCKED tasks, review [BLOCKED_DEPENDENCY_TREE.md](./BLOCKED_DEPENDENCY_TREE.md) for root blockers and dependencies.

Active items only. Completed/historic work now resides in docs/implplan/archived/tasks.md (updated 2025-11-08).

This file now only tracks the notifications & telemetry status snapshot. Active backlog lives in Sprint 171+ files.

# Wave coordination

| Wave | Guild owners | Shared prerequisites | Status | Notes |
| --- | --- | --- | --- | --- |
| 170.A Notifier | Notifications Service Guild · Attestor Service Guild · Observability Guild | Sprint 150.A – Orchestrator | **DONE (2025-12-04)** | All 14 tasks DONE (NOTIFY-GAPS-171-014 signed with dev key `notify-dev-hmac-001`; production HSM re-signing deferred). Tracked in `SPRINT_0171_0001_0001_notifier_i.md`. |
| 170.B Telemetry | Telemetry Core Guild · Observability Guild · Security Guild | Sprint 150.A – Orchestrator | **DONE (2025-11-27)** | All 6 tasks complete (TELEMETRY-OBS-50-001 through 56-001). Tracked in `SPRINT_0174_0001_0001_telemetry.md`. |

# Sprint 170 - Notifications & Telemetry

## Wave 170.A – Notifier readiness

### Scope & goals
- Deliver attestation/key-rotation alert templates plus routing so Attestor/Signer incidents surface immediately (NOTIFY-ATTEST-74-001/002).
- Refresh Notifier OpenAPI/SDK surface (`NOTIFY-OAS-61-001` → `NOTIFY-OAS-63-001`) so Console/CLI teams can self-serve the new endpoints.
- Wire SLO/incident inputs into rules (NOTIFY-OBS-51-001/55-001) and extend risk-profile routing (NOTIFY-RISK-66-001 → NOTIFY-RISK-68-001) without regressing quiet-hours/dedup.
- Preserve Offline Kit and documentation parity (NOTIFY-DOC-70-001 — done, NOTIFY-AIRGAP-56-002 — done) while adding the new rule surfaces.

### Entry criteria
- Orchestrator job attest events flowing to Notify bus (Sprint 150.A dependency) with test fixtures approved by Attestor Guild.
- Quiet-hours/digest backlog reconciled (no pending blockers in `docs/notifications/*.md`).
- Observability Guild sign-off on telemetry fields reused by Notifier SLO webhooks.

### Exit criteria
- All NOTIFY-ATTEST/OAS/OBS/RISK tasks in `SPRINT_171_notifier_i.md` moved to DONE with accompanying doc updates.
- Templates promoted to Offline Kit manifests and sample payloads stored under `docs/notifications/templates.md`.
- Incident mode notifications exercised in staging with audit logs + DSSE evidence attached.

### Task clusters & owners

| Cluster | Linked tasks | Owners | Status snapshot | Notes |
| --- | --- | --- | --- | --- |
| Attestation / key lifecycle alerts | NOTIFY-ATTEST-74-001/74-002 | Notifications Service Guild · Attestor Service Guild | TODO → DOING (prep) | Template scaffolding drafted; awaiting Rekor witness payload contract freeze. |
| API/OAS refresh & SDK parity | NOTIFY-OAS-61-001 → NOTIFY-OAS-63-001 | Notifications Service Guild · API Contracts Guild · SDK Generator Guild | TODO | Contract doc outline in review; SDK generator blocked on `/notifications/rules` schema finalize date (target 2025-11-15). |
| Observability-driven triggers | NOTIFY-OBS-51-001/55-001 | Notifications Service Guild · Observability Guild | TODO | Depends on Telemetry team exposing SLO webhook payload shape (see TELEMETRY-OBS-51-001). |
| Risk profile routing | NOTIFY-RISK-66-001 → NOTIFY-RISK-68-001 | Notifications Service Guild · Risk Engine Guild · Policy Guild | TODO | Requires Policy’s risk profile metadata (POLICY-RISK-40-002) export; follow up in Sprint 175. |
| Docs & offline parity | NOTIFY-DOC-70-001, NOTIFY-AIRGAP-56-002 | Notifications Service Guild · DevOps Guild | DONE | Remains reference for GA checklists; keep untouched unless new surfaces appear. |

### Observability checkpoints
- Align metric names/labels with `docs/notifications/architecture.md#12-observability-prometheus--otel` before promoting new dashboards.
- Ensure Notifier spans/logs include tenant, ruleId, actionId, and `attestation_event_id` for attestation-triggered templates.
- Capture incident notification smoke tests via `ops/devops/telemetry/tenant_isolation_smoke.py` once Telemetry wave lands.

## Wave 170.B – Telemetry bootstrap

### Scope & goals
- Ship `StellaOps.Telemetry.Core` bootstrap + propagation helpers (TELEMETRY-OBS-50-001/50-002).
- Provide golden-signal helpers + scrubbing/PII safety nets (TELEMETRY-OBS-51-001/51-002) so service teams can onboard without bespoke plumbing.
- Implement incident + sealed-mode toggles (TELEMETRY-OBS-55-001/56-001) and document the integration contract for Orchestrator, Policy, Task Runner, Gateway (`WEB-OBS-50-001`).

### Entry criteria
- Orchestrator + Policy hosts expose extension points for telemetry bootstrap (tracked via Sprint 150.A and IDs ORCH-OBS-50-001 / POLICY-OBS-50-001).
- Observability Guild reviewed storage footprint impacts for Prometheus/Tempo/Loki per module (docs/modules/telemetry/architecture.md §2).
- Security Guild signs off on redaction defaults + tenant override audit logging.

### Exit criteria
- Core library published to `/local-nugets` and referenced by at least Orchestrator & Policy in integration branches.
- Context propagation middleware validated through HTTP/gRPC/job smoke tests with deterministic trace IDs.
- Incident/sealed-mode toggles wired into CLI + Notify hooks (NOTIFY-OBS-55-001) with runbooks updated under `docs/notifications/architecture.md`.

### Task clusters & owners

| Cluster | Linked tasks | Owners | Status snapshot | Notes |
| --- | --- | --- | --- | --- |
| Bootstrap & propagation | TELEMETRY-OBS-50-001/50-002 | Telemetry Core Guild | TODO → DOING (scaffolding) | Collector profile templates staged; need service metadata detector + sample host integration PRs. |
| Metrics helpers + scrubbing | TELEMETRY-OBS-51-001/51-002 | Telemetry Core Guild · Observability Guild · Security Guild | TODO | Roslyn analyzer spec drafted; waiting on scrub policy from Security (POLICY-SEC-42-003). |
| Incident & sealed-mode controls | TELEMETRY-OBS-55-001/56-001 | Telemetry Core Guild · Observability Guild | TODO | Requires CLI toggle contract (CLI-OBS-12-001) and Notify incident payload spec (NOTIFY-OBS-55-001). |

### Tooling & validation
- Smoke: `ops/devops/telemetry/smoke_otel_collector.py` + `tenant_isolation_smoke.py` to run for each profile (default/forensic/airgap).
- Offline bundle packaging: `ops/devops/telemetry/package_offline_bundle.py` to include updated collectors, dashboards, manifest digests.
- Incident simulation: reuse `ops/devops/telemetry/generate_dev_tls.sh` for local collector certs during sealed-mode testing.

## Shared milestones & dependencies

| Target date | Milestone | Owners | Dependency notes |
| --- | --- | --- | --- |
| 2025-11-13 | Finalize attestation payload schema + template variables | Notifications Service Guild · Attestor Service Guild | Unblocks NOTIFY-ATTEST-74-001/002 + Telemetry incident span labels. |
| 2025-11-15 | Publish draft Notifier OAS + SDK snippets | Notifications Service Guild · API Contracts Guild | Required for CLI/UI adoption; prereq for NOTIFY-OAS-61/62 series. |
| 2025-11-18 | Land Telemetry.Core bootstrap sample in Orchestrator | Telemetry Core Guild · Orchestrator Guild | Demonstrates TELEMETRY-OBS-50-001 viability; prerequisite for Policy adoption + Notify SLO hooks. |
| 2025-11-20 | Incident/quiet-hour end-to-end rehearsal | Notifications Service Guild · Telemetry Core Guild · Observability Guild | Validates TELEMETRY-OBS-55-001 + NOTIFY-OBS-55-001 + CLI toggle contract. |
| 2025-11-22 | Offline kit bundle refresh (notifications + telemetry assets) | DevOps Guild · Notifications Service Guild · Telemetry Core Guild | Ensure docs/ops/offline-kit manifests reference new templates/configs. |

## Risks & mitigations
- **Telemetry data drift in sealed mode.** Mitigate by enforcing `IEgressPolicy` checks (TELEMETRY-OBS-56-001) and documenting fallback exporters; schedule smoke runs after each config change.
- **Template/API divergence.** Maintain single source of truth in `SPRINT_171_notifier_i.md` tasks; require API Contracts review before merging SDK updates to avoid drift with UI consumers.
- **Observability storage overhead.** Coordinate with Ops Guild to project Prometheus/Tempo growth when SLO webhooks + incident toggles increase cardinality; adjust retention per docs/modules/telemetry/architecture.md §2.
- **Cross-sprint dependency churn.** Track ORCH-OBS-50-001, POLICY-OBS-50-001, WEB-OBS-50-001 weekly; if they slip, re-baseline Telemetry wave deliverables or gate Notifier observability triggers accordingly.

## Task mirror snapshot (reference: Sprint 171 & 174 trackers)

### Wave 170.A – Notifier (Sprint 171 mirror)
- **Open tasks:** 0.
- **Done tasks:** 14 (all NOTIFY-ATTEST, NOTIFY-OAS, NOTIFY-OBS, NOTIFY-RISK, NOTIFY-DOC, NOTIFY-AIRGAP, NOTIFY-GAPS series complete).

| Category | Task IDs | Current state | Notes |
| --- | --- | --- | --- |
| Attestation + key lifecycle | NOTIFY-ATTEST-74-001/002 | **DONE** | Templates and wiring complete (2025-11-16/27). |
| API/OAS + SDK refresh | NOTIFY-OAS-61-001 → 63-001 | **DONE** | All OAS/SDK tasks complete (2025-11-17). |
| Observability-driven triggers | NOTIFY-OBS-51-001/55-001 | **DONE** | SLO webhook + incident mode templates shipped (2025-11-22). |
| Risk routing | NOTIFY-RISK-66-001 → 68-001 | **DONE** | Risk-events endpoint + routing seeds shipped (2025-11-24); POLICY-RISK-40-002 metadata export now available. |
| Gap remediation | NOTIFY-GAPS-171-014 | **DONE** | NR1-NR10 artifacts complete; DSSE signed with dev key `notify-dev-hmac-001` (2025-12-04). |
| Completed prerequisites | NOTIFY-DOC-70-001, NOTIFY-AIRGAP-56-002 | **DONE** | Documentation and offline-kit parity complete. |

### Wave 170.B – Telemetry (Sprint 174 mirror)
- **Open tasks:** 0.
- **Done tasks:** 6 (TELEMETRY-OBS-50/51/55/56 series all complete as of 2025-11-27).

| Category | Task IDs | Current state | Notes |
| --- | --- | --- | --- |
| Bootstrap & propagation | TELEMETRY-OBS-50-001/002 | **DONE** | Core bootstrap (50-001) and propagation middleware (50-002) complete (2025-11-19/27). |
| Metrics helpers & scrubbing | TELEMETRY-OBS-51-001/002 | **DONE** | Golden signal metrics with cardinality guards + scrubbing filters complete (2025-11-27). |
| Incident & sealed-mode controls | TELEMETRY-OBS-55-001/56-001 | **DONE** | Incident mode toggle and sealed-mode helpers complete (2025-11-27). |

## External dependency tracker

| Dependency | Source sprint / doc | Current state (as of 2025-11-12) | Impact on waves |
| --- | --- | --- | --- |
| Sprint 150.A – Orchestrator (wave table) | `SPRINT_150_scheduling_automation.md` | TODO | Blocks Notifier template wiring + Telemetry consumption of job events until orchestration telemetry lands. |
| ORCH-OBS-50-001 `orchestrator instrumentation` | `docs/implplan/archived/tasks.md` excerpt / Sprint 150 backlog | TODO | Needed for Telemetry.Core sample + Notify SLO hooks; monitor for slip. |
| POLICY-OBS-50-001 `policy instrumentation` | Sprint 150 backlog | TODO | Required before Telemetry helpers can be adopted by Policy + risk routing. |
| WEB-OBS-50-001 `gateway telemetry core adoption` | Sprint 214/215 backlogs | TODO | Ensures web/gateway emits trace IDs that Notify incident payload references. |
| POLICY-RISK-40-002 `risk profile metadata export` | Sprint 215+ (Policy) | DONE (2025-12-04) | Implemented `GET /api/risk/profiles/{id}/metadata` endpoint for notification enrichment. |

## Coordination log

| Date (UTC) | Update | Owner |
| --- | --- | --- |
| 2025-12-04 | Sprint 170 FULLY COMPLETE: Created dev signing key (`etc/secrets/dsse-dev.signing.json`) and signing utility (`scripts/notifications/sign-dsse.py`); signed DSSE files with `notify-dev-hmac-001`. NOTIFY-GAPS-171-014 now DONE. All 14 Notifier + 6 Telemetry tasks complete. | Implementer |
| 2025-12-04 | Sprint 170 complete: Wave 170.A marked DONE (12/13 tasks); Wave 170.B already DONE (6/6 tasks). Only NOTIFY-GAPS-171-014 remains BLOCKED on security infra (signing keys). | Implementer |
| 2025-12-04 | Implemented POLICY-RISK-40-002: Added `GET /api/risk/profiles/{id}/metadata` endpoint for notification enrichment. NOTIFY-RISK tasks unblocked. Only NOTIFY-GAPS-171-014 remains BLOCKED (signing keys). | Implementer |
| 2025-12-04 | Status refresh: Wave 170.B (Telemetry) marked DONE (all 6 tasks complete); Wave 170.A (Notifier) updated to show 9/13 done with 4 BLOCKED on external dependencies (POLICY-RISK-40-002, signing keys). Updated task mirror snapshots. | Project Mgmt |
| 2025-11-12 10:15 | Wave rows flipped to DOING; baseline scope/entry/exit criteria recorded for both waves. | Observability Guild · Notifications Service Guild |
| 2025-11-12 14:40 | Added task mirror + dependency tracker + milestone table to keep Sprint 170 snapshot aligned with Sprint 171/174 execution plans. | Observability Guild |
| 2025-11-12 18:05 | Marked NOTIFY-ATTEST-74-001, NOTIFY-OAS-61-001, and TELEMETRY-OBS-50-001 as DOING in their sprint trackers; added status notes reflecting in-flight work vs. gated follow-ups. | Notifications Service Guild · Telemetry Core Guild |
| 2025-11-12 19:20 | Documented attestation template suite (Section 7 in `docs/notifications/templates.md`) to unblock NOTIFY-ATTEST-74-001 deliverables and updated sprint mirrors accordingly. | Notifications Service Guild |
| 2025-11-12 19:32 | Synced notifications architecture doc to reference the new attestation template suite so downstream teams see the dependency in one place. | Notifications Service Guild |
| 2025-11-12 19:45 | Updated notifications overview + rules docs with `tmpl-attest-*` requirements so rule authors/operators share the same contract. | Notifications Service Guild |
| 2025-11-12 20:05 | Published baseline Offline Kit templates under `offline/notifier/templates/attestation/` for Slack/Email/Webhook so NOTIFY-ATTEST-74-002 wiring has ready-made artefacts. | Notifications Service Guild |