feat(rate-limiting): Implement core rate limiting functionality with configuration, decision-making, metrics, middleware, and service registration
- Add RateLimitConfig for configuration management with YAML binding support. - Introduce RateLimitDecision to encapsulate the result of rate limit checks. - Implement RateLimitMetrics for OpenTelemetry metrics tracking. - Create RateLimitMiddleware for enforcing rate limits on incoming requests. - Develop RateLimitService to orchestrate instance and environment rate limit checks. - Add RateLimitServiceCollectionExtensions for dependency injection registration.
This commit is contained in:
master
@@ -14,7 +14,7 @@ This dossier distils the Notify architecture into implementation-ready guidance
|
||||
└───────┬──────────┘
|
||||
│
|
||||
┌───────▼──────────┐ ┌───────────────┐
|
||||
│ Notify.WebService│◀──────▶│ MongoDB │
|
||||
│ Notify.WebService│◀──────▶│ PostgreSQL │
|
||||
Tenant API│ REST + gRPC WIP │ │ rules/channels│
|
||||
└───────▲──────────┘ │ deliveries │
|
||||
│ │ digests │
|
||||
@@ -31,14 +31,14 @@ Tenant API│ REST + gRPC WIP │ │ rules/channels│
|
||||
│ Connectors │──────▶│ Slack/Teams/... │
|
||||
│ (plug-ins) │ │ External targets │
|
||||
└─────────────┘ └──────────────────┘
|
||||
```
|
||||
|
||||
- **2025-11-02 decision — module boundaries.** Keep `src/Notify/` as the shared notification toolkit (engine, storage, queue, connectors) that multiple hosts can consume. `src/Notifier/` remains the Notifications Studio runtime (WebService + Worker) composed from those libraries. Do not collapse the directories until a packaging RFC covers build impacts, offline kit parity, and imposed-rule propagation.
|
||||
- **WebService** hosts REST endpoints (`/channels`, `/rules`, `/templates`, `/deliveries`, `/digests`, `/stats`) and handles schema normalisation, validation, and Authority enforcement.
|
||||
- **Worker** subscribes to the platform event bus, evaluates rules per tenant, applies throttles/digests, renders payloads, writes ledger entries, and invokes connectors.
|
||||
- **Plug-ins** live under `plugins/notify/` and are loaded deterministically at service start (`orderedPlugins` list). Each implements connector contracts and optional health/test-preview providers.
|
||||
```
|
||||
|
||||
Both services share options via `notify.yaml` (see `etc/notify.yaml.sample`). For dev/test scenarios, an in-memory repository exists but production requires Mongo + Redis/NATS for durability and coordination.
|
||||
- **2025-11-02 decision — module boundaries.** Keep `src/Notify/` as the shared notification toolkit (engine, storage, queue, connectors) that multiple hosts can consume. `src/Notifier/` remains the Notifications Studio runtime (WebService + Worker) composed from those libraries. Do not collapse the directories until a packaging RFC covers build impacts, offline kit parity, and imposed-rule propagation.
|
||||
- **WebService** hosts REST endpoints (`/channels`, `/rules`, `/templates`, `/deliveries`, `/digests`, `/stats`) and handles schema normalisation, validation, and Authority enforcement.
|
||||
- **Worker** subscribes to the platform event bus, evaluates rules per tenant, applies throttles/digests, renders payloads, writes ledger entries, and invokes connectors.
|
||||
- **Plug-ins** live under `plugins/notify/` and are loaded deterministically at service start (`orderedPlugins` list). Each implements connector contracts and optional health/test-preview providers.
|
||||
|
||||
Both services share options via `notify.yaml` (see `etc/notify.yaml.sample`). For dev/test scenarios, an in-memory repository exists but production requires PostgreSQL + Redis/NATS for durability and coordination.
|
||||
|
||||
---
|
||||
|
||||
@@ -46,7 +46,7 @@ Both services share options via `notify.yaml` (see `etc/notify.yaml.sample`). Fo
|
||||
|
||||
1. **Subscription.** Workers attach to the internal bus (Redis Streams or NATS JetStream). Each partition key is `tenantId|scope.digest|event.kind` to preserve order for a given artefact.
|
||||
2. **Normalisation.** Incoming events are hydrated into `NotifyEvent` envelopes. Payload JSON is normalised (sorted object keys) to preserve determinism and enable hashing.
|
||||
3. **Rule snapshot.** Per-tenant rule sets are cached in memory. Change streams from Mongo trigger snapshot refreshes without restart.
|
||||
3. **Rule snapshot.** Per-tenant rule sets are cached in memory. PostgreSQL LISTEN/NOTIFY triggers snapshot refreshes without restart.
|
||||
4. **Match pipeline.**
|
||||
- Tenant check (`rule.tenantId` vs. event tenant).
|
||||
- Kind/namespace/repository/digest filters.
|
||||
@@ -62,39 +62,39 @@ Failures during evaluation are logged with correlation IDs and surfaced through
|
||||
|
||||
## 3. Rendering & connectors
|
||||
|
||||
- **Template resolution.** The renderer picks the template in this order: action template → channel default template → locale fallback → built-in minimal template. Locale negotiation reduces `en-US` to `en-us`.
|
||||
- **Helpers & partials.** Exposed helpers mirror the list in [`notifications/templates.md`](templates.md#3-variables-helpers-and-context). Plug-ins may register additional helpers but must remain deterministic and side-effect free.
|
||||
- **Attestation lifecycle suite.** Sprint 171 introduced dedicated `tmpl-attest-*` templates for verification failures, expiring attestations, key rotations, and transparency anomalies (see [`templates.md` §7](templates.md#7-attestation--signing-lifecycle-templates-notify-attest-74-001)). Rule actions referencing those templates must populate the attestation context fields so channels stay consistent online/offline.
|
||||
- **Rendering output.** `NotifyDeliveryRendered` captures:
|
||||
- `channelType`, `format`, `locale`
|
||||
- `title`, `body`, optional `summary`, `textBody`
|
||||
- **Template resolution.** The renderer picks the template in this order: action template → channel default template → locale fallback → built-in minimal template. Locale negotiation reduces `en-US` to `en-us`.
|
||||
- **Helpers & partials.** Exposed helpers mirror the list in [`notifications/templates.md`](templates.md#3-variables-helpers-and-context). Plug-ins may register additional helpers but must remain deterministic and side-effect free.
|
||||
- **Attestation lifecycle suite.** Sprint 171 introduced dedicated `tmpl-attest-*` templates for verification failures, expiring attestations, key rotations, and transparency anomalies (see [`templates.md` §7](templates.md#7-attestation--signing-lifecycle-templates-notify-attest-74-001)). Rule actions referencing those templates must populate the attestation context fields so channels stay consistent online/offline.
|
||||
- **Rendering output.** `NotifyDeliveryRendered` captures:
|
||||
- `channelType`, `format`, `locale`
|
||||
- `title`, `body`, optional `summary`, `textBody`
|
||||
- `target` (redacted where necessary)
|
||||
- `attachments[]` (safe URLs or references)
|
||||
- `bodyHash` (lowercase SHA-256) for audit parity
|
||||
- **Connector contract.** Connectors implement `INotifyConnector` (send + health) and can implement `INotifyChannelTestProvider` for `/channels/{id}/test`. All plugs are single-tenant aware; secrets are pulled via references at send time and never persisted in Mongo.
|
||||
- **Connector contract.** Connectors implement `INotifyConnector` (send + health) and can implement `INotifyChannelTestProvider` for `/channels/{id}/test`. All plugs are single-tenant aware; secrets are pulled via references at send time and never persisted in the database.
|
||||
- **Retries.** Workers track attempts with exponential jitter. On permanent failure, deliveries are marked `Failed` with `statusReason`, and optional DLQ fan-out is slated for Sprint 40.
|
||||
|
||||
---
|
||||
|
||||
## 4. Persistence model
|
||||
|
||||
| Collection | Purpose | Key fields & indexes |
|
||||
|------------|---------|----------------------|
|
||||
| `rules` | Tenant rule definitions. | `_id`, `tenantId`, `enabled`; index on `{tenantId, enabled}`. |
|
||||
| `channels` | Channel metadata + config references. | `_id`, `tenantId`, `type`; index on `{tenantId, type}`. |
|
||||
| `templates` | Locale-specific render bodies. | `_id`, `tenantId`, `channelType`, `key`; index on `{tenantId, channelType, key}`. |
|
||||
| `deliveries` | Ledger of rendered notifications. | `_id`, `tenantId`, `sentAt`; compound index on `{tenantId, sentAt:-1}` for history queries. |
|
||||
| `digests` | Open digest windows per action. | `_id` (`tenantId:actionKey:window`), `status`; index on `{tenantId, actionKey}`. |
|
||||
| `throttles` | Short-lived throttle tokens (Mongo or Redis). | Key format `idem:<hash>` with TTL aligned to throttle duration. |
|
||||
| Table | Purpose | Key fields & indexes |
|
||||
|-------|---------|----------------------|
|
||||
| `rules` | Tenant rule definitions. | `id`, `tenant_id`, `enabled`; index on `(tenant_id, enabled)`. |
|
||||
| `channels` | Channel metadata + config references. | `id`, `tenant_id`, `type`; index on `(tenant_id, type)`. |
|
||||
| `templates` | Locale-specific render bodies. | `id`, `tenant_id`, `channel_type`, `key`; index on `(tenant_id, channel_type, key)`. |
|
||||
| `deliveries` | Ledger of rendered notifications. | `id`, `tenant_id`, `sent_at`; compound index on `(tenant_id, sent_at DESC)` for history queries. |
|
||||
| `digests` | Open digest windows per action. | `id` (`tenant_id:action_key:window`), `status`; index on `(tenant_id, action_key)`. |
|
||||
| `throttles` | Short-lived throttle tokens (PostgreSQL or Redis). | Key format `idem:<hash>` with TTL aligned to throttle duration. |
|
||||
|
||||
Documents are stored using the canonical JSON serializer (`NotifyCanonicalJsonSerializer`) to preserve property ordering and casing. Schema migration helpers upgrade stored documents when new versions ship.
|
||||
Records are stored using the canonical JSON serializer (`NotifyCanonicalJsonSerializer`) to preserve property ordering and casing. Schema migration helpers upgrade stored records when new versions ship.
|
||||
|
||||
---
|
||||
|
||||
## 5. Deployment & configuration
|
||||
|
||||
- **Configuration sources.** YAML files feed typed options (`NotifyMongoOptions`, `NotifyWorkerOptions`, etc.). Environment variables can override connection strings and rate limits for production.
|
||||
- **Authority integration.** Two OAuth clients (`notify-web`, `notify-web-dev`) with scopes `notify.viewer`, `notify.operator`, and (for dev/admin flows) `notify.admin` are required. Authority enforcement can be disabled for air-gapped dev use by providing `developmentSigningKey`.
|
||||
- **Configuration sources.** YAML files feed typed options (`NotifyPostgresOptions`, `NotifyWorkerOptions`, etc.). Environment variables can override connection strings and rate limits for production.
|
||||
- **Authority integration.** Two OAuth clients (`notify-web`, `notify-web-dev`) with scopes `notify.viewer`, `notify.operator`, and (for dev/admin flows) `notify.admin` are required. Authority enforcement can be disabled for air-gapped dev use by providing `developmentSigningKey`.
|
||||
- **Plug-in management.** `plugins.baseDirectory` and `orderedPlugins` guarantee deterministic loading. Offline Kits copy the plug-in tree verbatim; operations must keep the order aligned across environments.
|
||||
- **Observability.** Workers expose structured logs (`ruleId`, `actionId`, `eventId`, `throttleKey`). Metrics include:
|
||||
- `notify_rule_matches_total{tenant,eventKind}`
|
||||
@@ -111,7 +111,7 @@ Documents are stored using the canonical JSON serializer (`NotifyCanonicalJsonSe
|
||||
|---------|--------------------|
|
||||
| `NOTIFY-SVC-38-001` | Standardise event envelope publication (idempotency keys) – ensure bus bindings use the documented key format. |
|
||||
| `NOTIFY-SVC-38-002..004` | Introduce simulation endpoints and throttle dashboards – expect additional `/internal/notify/simulate` routes and metrics; update once merged. |
|
||||
| `NOTIFY-SVC-39-001..004` | Correlation engine, digests generator, simulation API, quiet hours – anticipate new Mongo documents (`quietHours`, correlation caches) and connector metadata (quiet mode hints). Review this guide when implementations land. |
|
||||
| `NOTIFY-SVC-39-001..004` | Correlation engine, digests generator, simulation API, quiet hours – anticipate new PostgreSQL tables (`quiet_hours`, correlation caches) and connector metadata (quiet mode hints). Review this guide when implementations land. |
|
||||
|
||||
Action: schedule a documentation sync with the Notifications Service Guild immediately after `NOTIFY-SVC-39-001..004` merge to confirm schema adjustments (e.g., correlation edge storage, quiet hour calendars) and add any new persistence or API details here.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user