Implement ledger metrics for observability and add tests for Ruby packages endpoints

- Added `LedgerMetrics` class to record write latency and total events for ledger operations.
- Created comprehensive tests for Ruby packages endpoints, covering scenarios for missing inventory, successful retrieval, and identifier handling.
- Introduced `TestSurfaceSecretsScope` for managing environment variables during tests.
- Developed `ProvenanceMongoExtensions` for attaching DSSE provenance and trust information to event documents.
- Implemented `EventProvenanceWriter` and `EventWriter` classes for managing event provenance in MongoDB.
- Established MongoDB indexes for efficient querying of events based on provenance and trust.
- Added models and JSON parsing logic for DSSE provenance and trust information.

docs/notifications/architecture.md

@@ -62,11 +62,12 @@ Failures during evaluation are logged with correlation IDs and surfaced through
 
 ## 3. Rendering & connectors
 
-- **Template resolution.** The renderer picks the template in this order: action template → channel default template → locale fallback → built-in minimal template. Locale negotiation reduces `en-US` to `en-us`.
-- **Helpers & partials.** Exposed helpers mirror the list in [`notifications/templates.md`](templates.md#3-variables-helpers-and-context). Plug-ins may register additional helpers but must remain deterministic and side-effect free.
-- **Rendering output.** `NotifyDeliveryRendered` captures:
-  - `channelType`, `format`, `locale`
-  - `title`, `body`, optional `summary`, `textBody`
+- **Template resolution.** The renderer picks the template in this order: action template → channel default template → locale fallback → built-in minimal template. Locale negotiation reduces `en-US` to `en-us`.
+- **Helpers & partials.** Exposed helpers mirror the list in [`notifications/templates.md`](templates.md#3-variables-helpers-and-context). Plug-ins may register additional helpers but must remain deterministic and side-effect free.
+- **Attestation lifecycle suite.** Sprint 171 introduced dedicated `tmpl-attest-*` templates for verification failures, expiring attestations, key rotations, and transparency anomalies (see [`templates.md` §7](templates.md#7-attestation--signing-lifecycle-templates-notify-attest-74-001)). Rule actions referencing those templates must populate the attestation context fields so channels stay consistent online/offline.
+- **Rendering output.** `NotifyDeliveryRendered` captures:
+  - `channelType`, `format`, `locale`
+  - `title`, `body`, optional `summary`, `textBody`
   - `target` (redacted where necessary)
   - `attachments[]` (safe URLs or references)
   - `bodyHash` (lowercase SHA-256) for audit parity

docs/notifications/overview.md

@@ -21,7 +21,7 @@ Notifications Studio turns raw platform events into concise, tenant-scoped alert
 |------------|--------------|----------|
 | Rules engine | Declarative matchers for event kinds, severities, namespaces, VEX context, KEV flags, and more. | [`notifications/rules.md`](rules.md) |
 | Channel catalog | Slack, Teams, Email, Webhook connectors loaded via restart-time plug-ins; metadata stored without secrets. | [`notifications/architecture.md`](architecture.md) |
-| Templates | Locale-aware, deterministic rendering via safe helpers; channel defaults plus tenant-specific overrides. | [`notifications/templates.md`](templates.md) |
+| Templates | Locale-aware, deterministic rendering via safe helpers; channel defaults plus tenant-specific overrides, including the attestation lifecycle suite (`tmpl-attest-*`). | [`notifications/templates.md`](templates.md#7-attestation--signing-lifecycle-templates-notify-attest-74-001) |
 | Digests | Coalesce bursts into periodic summaries with deterministic IDs and audit trails. | [`notifications/digests.md`](digests.md) |
 | Delivery ledger | Tracks rendered payload hashes, attempts, throttles, and outcomes for every action. | [`modules/notify/architecture.md`](../modules/notify/architecture.md#7-data-model-mongo) |
 | Ack tokens | DSSE-signed acknowledgement tokens with webhook allowlists and escalation guardrails enforced by Authority. | [`modules/notify/architecture.md`](../modules/notify/architecture.md#81-ack-tokens--escalation-workflows) |
@@ -44,7 +44,7 @@ The Notify WebService fronts worker state with REST APIs used by the UI and CLI.
 | Area | Guidance |
 |------|----------|
 | **Tenancy** | Each rule, channel, template, and delivery belongs to exactly one tenant. Cross-tenant sharing is intentionally unsupported. |
-| **Determinism** | Configuration persistence normalises strings and sorts collections. Template rendering produces identical `bodyHash` values when inputs match. |
+| **Determinism** | Configuration persistence normalises strings and sorts collections. Template rendering produces identical `bodyHash` values when inputs match; attestation events always reference the canonical `tmpl-attest-*` keys documented in the template guide. |
 | **Scaling** | Workers scale horizontally; per-tenant rule snapshots are cached and refreshed from Mongo change streams. Redis (or equivalent) guards throttles and locks. |
 | **Offline** | Offline Kits include plug-ins, default templates, and seed rules. Operators can edit YAML/JSON manifests before air-gapped deployment. |
 | **Security** | Channel secrets use indirection (`secretRef`), Authority-protected OAuth clients secure API access, and delivery payloads are redacted before storage where required. |

docs/notifications/rules.md

@@ -81,11 +81,24 @@ Each rule requires at least one action. Actions are deduplicated and sorted by `
 | `throttle` | ISO8601 duration? | Optional throttle TTL (`PT300S`, `PT1H`). Prevents duplicate deliveries when the same idempotency hash appears before expiry. |
 | `locale` | string? | BCP-47 tag (stored lower-case). Template lookup falls back to channel locale then `en-us`. |
 | `enabled` | bool | Disabled actions skip rendering but remain stored. |
-| `metadata` | map<string,string> | Connector-specific hints (priority, layout, etc.). |
-
-### 4.1 Evaluation order
-
-1. Verify channel exists and is enabled; disabled channels mark the delivery as `Dropped`.
+| `metadata` | map<string,string> | Connector-specific hints (priority, layout, etc.). |
+
+### 4.0 Attestation lifecycle templates
+
+Rules targeting attestation/signing events (`attestor.verification.failed`, `attestor.attestation.expiring`, `authority.keys.revoked`, `attestor.transparency.anomaly`) must reference the dedicated template keys documented in [`notifications/templates.md` §7](templates.md#7-attestation--signing-lifecycle-templates-notify-attest-74-001) so payloads remain deterministic across channels and Offline Kits:
+
+| Event kind | Required template key | Notes |
+| --- | --- | --- |
+| `attestor.verification.failed` | `tmpl-attest-verify-fail` | Include failure code, Rekor UUID/index, last good attestation link. |
+| `attestor.attestation.expiring` | `tmpl-attest-expiry-warning` | Surface issued/expires timestamps, time remaining, renewal instructions. |
+| `authority.keys.revoked` / `authority.keys.rotated` | `tmpl-attest-key-rotation` | List rotation batch ID, impacted services, remediation steps. |
+| `attestor.transparency.anomaly` | `tmpl-attest-transparency-anomaly` | Highlight Rekor/witness metadata and anomaly classification. |
+
+Locale-specific variants keep the same template key while varying `locale`; rule actions shouldn't create ad-hoc templates for these events.
+
+### 4.1 Evaluation order
+
+1. Verify channel exists and is enabled; disabled channels mark the delivery as `Dropped`.
 2. Apply throttle idempotency key: `hash(ruleId|actionId|event.kind|scope.digest|delta.hash|dayBucket)`. Hits are logged as `Throttled`.
 3. If the action defines a digest window other than `instant`, append the event to the open window and defer delivery until flush.
 4. When delivery proceeds, the renderer resolves the template, locale, and metadata before invoking the connector.

docs/notifications/templates.md

@@ -19,7 +19,7 @@ Templates shape the payload rendered for each channel when a rule action fires.
 
 | Field | Type | Notes |
 |-------|------|-------|
-| `templateId` | string | Stable identifier (UUID/slug). |
+| `templateId` | string | Stable identifier (UUID/slug). |
 | `tenantId` | string | Must match the tenant header in API calls. |
 | `channelType` | [`NotifyChannelType`](../modules/notify/architecture.md#5-channels--connectors-plug-ins) | Determines connector payload envelope. |
 | `key` | string | Human-readable key referenced by rules (`tmpl-critical`). |
@@ -109,22 +109,95 @@ When delivering via email, connectors automatically attach a plain-text alternat
 
 ---
 
-## 5. Preview and validation
-
-- `POST /channels/{id}/test` accepts an optional `templateId` and sample payload to produce a rendered preview without dispatching the event. Results include channel type, target, title/summary, locale, body hash, and connector metadata.
-- UI previews rely on the same API and highlight connector fallbacks (e.g., Teams adaptive card vs. text fallback).
-- Offline Kit scenarios can call `/internal/notify/templates/normalize` to ensure bundled templates match the canonical schema before packaging.
-
----
-
-## 6. Best practices
-
-- Keep channel-specific limits in mind (Slack block/character quotas, Teams adaptive card size, email line length). Lean on digests to summarise long lists.
-- Provide locale-specific versions for high-volume tenants; Notify selects the closest locale, falling back to `en-us`.
-- Store connector-specific hints (`metadata.layout`, `metadata.emoji`) in template metadata rather than rules when they affect rendering.
-- Version template bodies through metadata (e.g., `metadata.revision: "2025-10-28"`) so tenants can track changes over time.
-- Run test previews whenever introducing new helpers to confirm body hashes remain stable across environments.
-
----
-
-> **Imposed rule reminder:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.
+## 5. Preview and validation
+
+- `POST /channels/{id}/test` accepts an optional `templateId` and sample payload to produce a rendered preview without dispatching the event. Results include channel type, target, title/summary, locale, body hash, and connector metadata.
+- UI previews rely on the same API and highlight connector fallbacks (e.g., Teams adaptive card vs. text fallback).
+- Offline Kit scenarios can call `/internal/notify/templates/normalize` to ensure bundled templates match the canonical schema before packaging.
+
+---
+
+## 6. Best practices
+
+- Keep channel-specific limits in mind (Slack block/character quotas, Teams adaptive card size, email line length). Lean on digests to summarise long lists.
+- Provide locale-specific versions for high-volume tenants; Notify selects the closest locale, falling back to `en-us`.
+- Store connector-specific hints (`metadata.layout`, `metadata.emoji`) in template metadata rather than rules when they affect rendering.
+- Version template bodies through metadata (e.g., `metadata.revision: "2025-10-28"`) so tenants can track changes over time.
+- Run test previews whenever introducing new helpers to confirm body hashes remain stable across environments.
+
+---
+
+## 7. Attestation & signing lifecycle templates (NOTIFY-ATTEST-74-001)
+
+Attestation lifecycle events (verification failures, expiring attestations, key revocations, transparency anomalies) reuse the same structural context so operators can differentiate urgency while reusing channels. Every template **must** surface:
+
+- **Subject** (`payload.subject.digest`, `payload.subject.repository`, `payload.subject.tag`).
+- **Attestation metadata** (`payload.attestation.kind`, `payload.attestation.id`, `payload.attestation.issuedAt`, `payload.attestation.expiresAt`).
+- **Signer/Key fingerprint** (`payload.signer.kid`, `payload.signer.algorithm`, `payload.signer.rotationId`).
+- **Traceability** (`payload.links.console`, `payload.links.rekor`, `payload.links.docs`).
+
+### 7.1 Template keys & channels
+
+| Event | Template key | Required channels | Optional channels | Notes |
+| --- | --- | --- | --- | --- |
+| Verification failure (`attestor.verification.failed`) | `tmpl-attest-verify-fail` | Slack `sec-alerts`, Email `supply-chain@`, Webhook (Pager/SOC) | Teams `risk-war-room`, Custom SIEM feed | Include failure code, Rekor UUID, last-known good attestation link. |
+| Expiring attestation (`attestor.attestation.expiring`) | `tmpl-attest-expiry-warning` | Email summary, Slack reminder | Digest window (daily) | Provide expiration window, renewal instructions, `expiresIn` helper. |
+| Key revocation/rotation (`authority.keys.revoked`, `authority.keys.rotated`) | `tmpl-attest-key-rotation` | Email + Webhook | Slack (if SOC watches channel) | Add rotation batch ID, impacted tenants/services, remediation steps. |
+| Transparency anomaly (`attestor.transparency.anomaly`) | `tmpl-attest-transparency-anomaly` | Slack high-priority, Webhook, PagerDuty | Email follow-up | Show Rekor index delta, witness ID, anomaly classification, recommended actions. |
+
+Assign these keys when creating templates so rule actions can reference them deterministically (`actions[].template: "tmpl-attest-verify-fail"`).
+
+### 7.2 Context helpers
+
+- `attestation_status_badge status`: renders ✅/⚠️/❌ depending on verdict (`valid`, `expiring`, `failed`).
+- `expires_in expiresAt now`: returns human-readable duration, constrained to deterministic units (h/d).
+- `fingerprint key`: shortens long key IDs/pems, exposing the last 10 characters.
+
+### 7.3 Slack sample (verification failure)
+
+```hbs
+:rotating_light: {{attestation_status_badge payload.failure.status}} verification failed for `{{payload.subject.digest}}`
+Signer: `{{fingerprint payload.signer.kid}}` ({{payload.signer.algorithm}})
+Reason: `{{payload.failure.reasonCode}}` — {{payload.failure.reason}}
+Last valid attestation: {{link "Console report" payload.links.console}}
+Rekor entry: {{link "Transparency log" payload.links.rekor}}
+```
+
+### 7.4 Email sample (expiring attestation)
+
+```hbs
+<h2>Attestation expiry notice</h2>
+<p>The attestation for <code>{{payload.subject.repository}}</code> (digest {{payload.subject.digest}}) expires on <strong>{{payload.attestation.expiresAt}}</strong>.</p>
+<ul>
+  <li>Issued: {{payload.attestation.issuedAt}}</li>
+  <li>Signer: {{payload.signer.kid}} ({{payload.signer.algorithm}})</li>
+  <li>Time remaining: {{expires_in payload.attestation.expiresAt event.ts}}</li>
+</ul>
+<p>Please rotate the attestation before expiry. Reference <a href="{{payload.links.docs}}">renewal steps</a>.</p>
+```
+
+### 7.5 Webhook sample (transparency anomaly)
+
+```json
+{
+  "event": "attestor.transparency.anomaly",
+  "tenantId": "{{event.tenant}}",
+  "subjectDigest": "{{payload.subject.digest}}",
+  "rekorIndex": "{{payload.transparency.rekorIndex}}",
+  "witnessId": "{{payload.transparency.witnessId}}",
+  "anomaly": "{{payload.transparency.classification}}",
+  "detailsUrl": "{{payload.links.console}}",
+  "recommendation": "{{payload.recommendation}}"
+}
+```
+
+### 7.6 Offline kit guidance
+
+- Bundle these templates (JSON export) under `offline/notifier/templates/attestation/`.
+- Baseline English templates for Slack, Email, and Webhook ship in the repository at `offline/notifier/templates/attestation/*.template.json`; copy and localise them per tenant as needed.
+- Provide localized variants for `en-us` and `de-de` at minimum; additional locales can be appended per customer.
+- Include preview fixtures in Offline Kit smoke tests to guarantee channel render parity when air-gapped.
+
+---
+
+> **Imposed rule reminder:** Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.