feat(rate-limiting): Implement core rate limiting functionality with configuration, decision-making, metrics, middleware, and service registration
- Add RateLimitConfig for configuration management with YAML binding support. - Introduce RateLimitDecision to encapsulate the result of rate limit checks. - Implement RateLimitMetrics for OpenTelemetry metrics tracking. - Create RateLimitMiddleware for enforcing rate limits on incoming requests. - Develop RateLimitService to orchestrate instance and environment rate limit checks. - Add RateLimitServiceCollectionExtensions for dependency injection registration.
This commit is contained in:
master
@@ -10,7 +10,7 @@
|
||||
|
||||
* Notify **does not make policy decisions** and **does not rescan**; it **consumes** events from Scanner/Scheduler/Excitor/Conselier/Attestor/Zastava and routes them.
|
||||
* Attachments are **links** (UI/attestation pages); Notify **does not** attach SBOMs or large blobs to messages.
|
||||
* Secrets for channels (Slack tokens, SMTP creds) are **referenced**, not stored raw in Mongo.
|
||||
* Secrets for channels (Slack tokens, SMTP creds) are **referenced**, not stored raw in the database.
|
||||
* **2025-11-02 module boundary.** Maintain `src/Notify/` as the reusable notification toolkit (engine, storage, queue, connectors) and `src/Notifier/` as the Notifications Studio host that composes those libraries. Do not merge directories without an approved packaging RFC that covers build impacts, offline kit parity, and cross-module governance.
|
||||
|
||||
---
|
||||
@@ -26,7 +26,6 @@ src/
|
||||
├─ StellaOps.Notify.Engine/ # rules engine, templates, idempotency, digests, throttles
|
||||
├─ StellaOps.Notify.Models/ # DTOs (Rule, Channel, Event, Delivery, Template)
|
||||
├─ StellaOps.Notify.Storage.Postgres/ # canonical persistence (notify schema)
|
||||
├─ StellaOps.Notify.Storage.Mongo/ # legacy shim kept only for data export/migrations
|
||||
├─ StellaOps.Notify.Queue/ # bus client (Redis Streams/NATS JetStream)
|
||||
└─ StellaOps.Notify.Tests.* # unit/integration/e2e
|
||||
```
|
||||
@@ -36,7 +35,7 @@ src/
|
||||
* **Notify.WebService** (stateless API)
|
||||
* **Notify.Worker** (horizontal scale)
|
||||
|
||||
**Dependencies**: Authority (OpToks; DPoP/mTLS), **PostgreSQL** (notify schema), Redis/NATS (bus), HTTP egress to Slack/Teams/Webhooks, SMTP relay for Email. MongoDB remains only for archival/export tooling until Phase 7 cleanup.
|
||||
**Dependencies**: Authority (OpToks; DPoP/mTLS), **PostgreSQL** (notify schema), Redis/NATS (bus), HTTP egress to Slack/Teams/Webhooks, SMTP relay for Email.
|
||||
|
||||
> **Configuration.** Notify.WebService bootstraps from `notify.yaml` (see `etc/notify.yaml.sample`). Use `storage.driver: postgres` and provide `postgres.notify` options (`connectionString`, `schemaName`, pool sizing, timeouts). Authority settings follow the platform defaults—when running locally without Authority, set `authority.enabled: false` and supply `developmentSigningKey` so JWTs can be validated offline.
|
||||
>
|
||||
@@ -240,11 +239,11 @@ public interface INotifyConnector {
|
||||
|
||||
---
|
||||
|
||||
## 7) Data model (Mongo)
|
||||
## 7) Data model (PostgreSQL)
|
||||
|
||||
Canonical JSON Schemas for rules/channels/events live in `docs/modules/notify/resources/schemas/`. Sample payloads intended for tests/UI mock responses are captured in `docs/modules/notify/resources/samples/`.
|
||||
|
||||
**Database**: `notify`
|
||||
**Database**: `stellaops_notify` (PostgreSQL)
|
||||
|
||||
* `rules`
|
||||
|
||||
@@ -289,11 +288,11 @@ Canonical JSON Schemas for rules/channels/events live in `docs/modules/notify/re
|
||||
|
||||
Base path: `/api/v1/notify` (Authority OpToks; scopes: `notify.admin` for write, `notify.read` for view).
|
||||
|
||||
*All* REST calls require the tenant header `X-StellaOps-Tenant` (matches the canonical `tenantId` stored in Mongo). Payloads are normalised via `NotifySchemaMigration` before persistence to guarantee schema version pinning.
|
||||
*All* REST calls require the tenant header `X-StellaOps-Tenant` (matches the canonical `tenantId` stored in PostgreSQL). Payloads are normalised via `NotifySchemaMigration` before persistence to guarantee schema version pinning.
|
||||
|
||||
Authentication today is stubbed with Bearer tokens (`Authorization: Bearer <token>`). When Authority wiring lands, this will switch to OpTok validation + scope enforcement, but the header contract will remain the same.
|
||||
|
||||
Service configuration exposes `notify:auth:*` keys (issuer, audience, signing key, scope names) so operators can wire the Authority JWKS or (in dev) a symmetric test key. `notify:storage:*` keys cover Mongo URI/database/collection overrides. Both sets are required for the new API surface.
|
||||
Service configuration exposes `notify:auth:*` keys (issuer, audience, signing key, scope names) so operators can wire the Authority JWKS or (in dev) a symmetric test key. `notify:storage:*` keys cover PostgreSQL connection/schema overrides. Both sets are required for the new API surface.
|
||||
|
||||
Internal tooling can hit `/internal/notify/<entity>/normalize` to upgrade legacy JSON and return canonical output used in the docs fixtures.
|
||||
|
||||
@@ -347,7 +346,7 @@ Authority signs ack tokens using keys configured under `notifications.ackTokens`
|
||||
|
||||
* **Ingestor**: N consumers with per‑key ordering (key = tenant|digest|namespace).
|
||||
* **RuleMatcher**: loads active rules snapshot for tenant into memory; vectorized predicate check.
|
||||
* **Throttle/Dedupe**: consult Redis + Mongo `throttles`; if hit → record `status=throttled`.
|
||||
* **Throttle/Dedupe**: consult Redis + PostgreSQL `throttles`; if hit → record `status=throttled`.
|
||||
* **DigestCoalescer**: append to open digest window or flush when timer expires.
|
||||
* **Renderer**: select template (channel+locale), inject variables, enforce length limits, compute `bodyHash`.
|
||||
* **Connector**: send; handle provider‑specific rate limits and backoffs; `maxAttempts` with exponential jitter; overflow → DLQ (dead‑letter topic) + UI surfacing.
|
||||
@@ -367,7 +366,7 @@ Authority signs ack tokens using keys configured under `notifications.ackTokens`
|
||||
## 11) Security & privacy
|
||||
|
||||
* **AuthZ**: all APIs require **Authority** OpToks; actions scoped by tenant.
|
||||
* **Secrets**: `secretRef` only; Notify fetches just‑in‑time from Authority Secret proxy or K8s Secret (mounted). No plaintext secrets in Mongo.
|
||||
* **Secrets**: `secretRef` only; Notify fetches just‑in‑time from Authority Secret proxy or K8s Secret (mounted). No plaintext secrets in database.
|
||||
* **Egress TLS**: validate SSL; pin domains per channel config; optional CA bundle override for on‑prem SMTP.
|
||||
* **Webhook signing**: HMAC or Ed25519 signatures in `X-StellaOps-Signature` + replay‑window timestamp; include canonical body hash in header.
|
||||
* **Redaction**: deliveries store **hashes** of bodies, not full payloads for chat/email to minimize PII retention (configurable).
|
||||
@@ -456,7 +455,7 @@ notify:
|
||||
| Invalid channel secret | Mark channel unhealthy; suppress sends; surface in UI |
|
||||
| Rule explosion (matches everything) | Safety valve: per‑tenant RPM caps; auto‑pause rule after X drops; UI alert |
|
||||
| Bus outage | Buffer to local queue (bounded); resume consuming when healthy |
|
||||
| Mongo slowness | Fall back to Redis throttles; batch write deliveries; shed low‑priority notifications |
|
||||
| PostgreSQL slowness | Fall back to Redis throttles; batch write deliveries; shed low‑priority notifications |
|
||||
|
||||
---
|
||||
|
||||
@@ -530,7 +529,7 @@ Bootstrap Pack. The artefacts live under `bootstrap/notify/` after running the
|
||||
Offline Kit builder and include:
|
||||
|
||||
- `notify.yaml` — configuration derived from `etc/notify.airgap.yaml`, pointing
|
||||
to the sealed MongoDB/Authority endpoints and loading connectors from the
|
||||
to the sealed PostgreSQL/Authority endpoints and loading connectors from the
|
||||
local plug-in directory.
|
||||
- `notify-web.secret.example` — template for the Authority client secret,
|
||||
intended to be renamed to `notify-web.secret` before deployment.
|
||||
|
||||
Reference in New Issue
Block a user