feat(rate-limiting): Implement core rate limiting functionality with configuration, decision-making, metrics, middleware, and service registration

- Add RateLimitConfig for configuration management with YAML binding support. - Introduce RateLimitDecision to encapsulate the result of rate limit checks. - Implement RateLimitMetrics for OpenTelemetry metrics tracking. - Create RateLimitMiddleware for enforcing rate limits on incoming requests. - Develop RateLimitService to orchestrate instance and environment rate limit checks. - Add RateLimitServiceCollectionExtensions for dependency injection registration.
2025-12-17 18:02:37 +02:00
parent 394b57f6bf
commit 8bbfe4d2d2
211 changed files with 47179 additions and 1590 deletions
--- a/docs/modules/issuer-directory/architecture.md
+++ b/docs/modules/issuer-directory/architecture.md
@@ -10,16 +10,16 @@ Issuer Directory centralises trusted VEX/CSAF publisher metadata so downstream s

 - **Service name:** `stellaops/issuer-directory`
 - **Framework:** ASP.NET Core minimal APIs (`net10.0`)
- **Persistence:** MongoDB (`issuer-directory.issuers`, `issuer-directory.issuer_keys`, `issuer-directory.issuer_audit`)
+- **Persistence:** PostgreSQL (`issuer_directory.issuers`, `issuer_directory.issuer_keys`, `issuer_directory.issuer_audit`)
 - **AuthZ:** StellaOps resource server scopes (`issuer-directory:read`, `issuer-directory:write`, `issuer-directory:admin`)
 - **Audit:** Every create/update/delete emits an audit record with actor, reason, and context.
 - **Bootstrap:** On startup, the service imports `data/csaf-publishers.json` into the global tenant (`@global`) and records a `seeded` audit the first time each publisher is added.
 - **Key lifecycle:** API validates Ed25519 public keys, X.509 certificates, and DSSE public keys, enforces future expiries, deduplicates fingerprints, and records audit entries for create/rotate/revoke actions.

 ```
-Clients ──> Authority (DPoP/JWT) ──> IssuerDirectory WebService ──> MongoDB
+Clients ──> Authority (DPoP/JWT) ──> IssuerDirectory WebService ──> PostgreSQL
                                            │
-                                            └─> Audit sink (Mongo)
+                                            └─> Audit sink (PostgreSQL)
 ```

 ## 3. Configuration
@@ -42,12 +42,12 @@ IssuerDirectory:
  tenantHeader: X-StellaOps-Tenant
  seedCsafPublishers: true
  csafSeedPath: data/csaf-publishers.json
-  Mongo:
-    connectionString: mongodb://localhost:27017
-    database: issuer-directory
-    issuersCollection: issuers
-    issuerKeysCollection: issuer_keys
-    auditCollection: issuer_audit
+  Postgres:
+    connectionString: Host=localhost;Port=5432;Database=issuer_directory;Username=stellaops;Password=secret
+    schema: issuer_directory
+    issuersTable: issuers
+    issuerKeysTable: issuer_keys
+    auditTable: issuer_audit
 ```

 ## 4. API Surface (v0)
@@ -74,7 +74,7 @@ Payloads follow the contract in `Contracts/IssuerDtos.cs` and align with domain
 ## 5. Dependencies & Reuse

 - `StellaOps.IssuerDirectory.Core` — domain model (`IssuerRecord`, `IssuerKeyRecord`) + application services.
- `StellaOps.IssuerDirectory.Infrastructure` — MongoDB persistence, audit sink, seed loader.
+- `StellaOps.IssuerDirectory.Infrastructure` — PostgreSQL persistence, audit sink, seed loader.
 - `StellaOps.IssuerDirectory.WebService` — minimal API host, authentication wiring.
 - Shared libraries: `StellaOps.Configuration`, `StellaOps.Auth.ServerIntegration`.

--- a/docs/modules/issuer-directory/operations/backup-restore.md
+++ b/docs/modules/issuer-directory/operations/backup-restore.md
@@ -2,18 +2,18 @@

 ## Scope
 - **Applies to:** Issuer Directory when deployed via Docker Compose (`deploy/compose/docker-compose.*.yaml`) or the Helm chart (`deploy/helm/stellaops`).
- **Artifacts covered:** MongoDB database `issuer-directory`, service configuration (`etc/issuer-directory.yaml`), CSAF seed file (`data/csaf-publishers.json`), and secret material for the Mongo connection string.
+- **Artifacts covered:** PostgreSQL database `issuer_directory`, service configuration (`etc/issuer-directory.yaml`), CSAF seed file (`data/csaf-publishers.json`), and secret material for the PostgreSQL connection string.
 - **Frequency:** Take a hot backup before every upgrade and at least daily in production. Keep encrypted copies off-site/air-gapped according to your compliance program.

 ## Inventory checklist
 | Component | Location (Compose default) | Notes |
 | --- | --- | --- |
-| Mongo data | `mongo-data` volume (`/var/lib/docker/volumes/.../mongo-data`) | Contains `issuers`, `issuer_keys`, `issuer_trust_overrides`, and `issuer_audit` collections. |
+| PostgreSQL data | `postgres-data` volume (`/var/lib/docker/volumes/.../postgres-data`) | Contains `issuers`, `issuer_keys`, `issuer_trust_overrides`, and `issuer_audit` tables in the `issuer_directory` schema. |
 | Configuration | `etc/issuer-directory.yaml` | Mounted read-only at `/etc/issuer-directory.yaml` inside the container. |
 | CSAF seed file | `src/IssuerDirectory/StellaOps.IssuerDirectory/data/csaf-publishers.json` | Ensure customised seeds are part of the backup; regenerate if you ship regional overrides. |
-| Mongo secret | `.env` entry `ISSUER_DIRECTORY_MONGO_CONNECTION_STRING` or secret store export | Required to restore connectivity; treat as sensitive. |
+| PostgreSQL secret | `.env` entry `ISSUER_DIRECTORY_POSTGRES_CONNECTION_STRING` or secret store export | Required to restore connectivity; treat as sensitive. |

-> **Tip:** Export the secret via `kubectl get secret issuer-directory-secrets -o yaml` (sanitize before storage) or copy the Compose `.env` file into an encrypted vault.
+> **Tip:** Export the secret via `kubectl get secret issuer-directory-secrets -o yaml` (sanitize before storage) or copy the Compose `.env` file into an encrypted vault. For PostgreSQL credentials, consider using `pg_dump` with connection info from environment variables.

 ## Hot backup (no downtime)
 1. **Create output directory**
@@ -21,16 +21,17 @@
   BACKUP_DIR=backup/issuer-directory/$(date +%Y-%m-%dT%H%M%S)
   mkdir -p "$BACKUP_DIR"
   ```
-2. **Dump Mongo collections**
+2. **Dump PostgreSQL tables**
   ```bash
-   docker compose -f deploy/compose/docker-compose.prod.yaml exec mongo \
-     mongodump --archive=/dump/issuer-directory-$(date +%Y%m%dT%H%M%SZ).gz \
-     --gzip --db issuer-directory
+   docker compose -f deploy/compose/docker-compose.prod.yaml exec postgres \
+     pg_dump --format=custom --compress=9 \
+     --file=/dump/issuer-directory-$(date +%Y%m%dT%H%M%SZ).dump \
+     --schema=issuer_directory issuer_directory

   docker compose -f deploy/compose/docker-compose.prod.yaml cp \
-     mongo:/dump/issuer-directory-$(date +%Y%m%dT%H%M%SZ).gz "$BACKUP_DIR/"
+     postgres:/dump/issuer-directory-$(date +%Y%m%dT%H%M%SZ).dump "$BACKUP_DIR/"
   ```
-   For Kubernetes, run the same `mongodump` command inside the `stellaops-mongo` pod and copy the archive via `kubectl cp`.
+   For Kubernetes, run the same `pg_dump` command inside the `stellaops-postgres` pod and copy the archive via `kubectl cp`.
 3. **Capture configuration and seeds**
   ```bash
   cp etc/issuer-directory.yaml "$BACKUP_DIR/"
@@ -38,8 +39,8 @@
   ```
 4. **Capture secrets**
   ```bash
-   grep '^ISSUER_DIRECTORY_MONGO_CONNECTION_STRING=' dev.env > "$BACKUP_DIR/issuer-directory.mongo.secret"
-   chmod 600 "$BACKUP_DIR/issuer-directory.mongo.secret"
+   grep '^ISSUER_DIRECTORY_POSTGRES_CONNECTION_STRING=' dev.env > "$BACKUP_DIR/issuer-directory.postgres.secret"
+   chmod 600 "$BACKUP_DIR/issuer-directory.postgres.secret"
   ```
 5. **Generate checksums and encrypt**
   ```bash
@@ -57,21 +58,21 @@
   (For Helm: `kubectl scale deploy stellaops-issuer-directory --replicas=0`.)
 3. Snapshot volumes:
   ```bash
-   docker run --rm -v mongo-data:/data \
-     -v "$(pwd)":/backup busybox tar czf /backup/mongo-data-$(date +%Y%m%d).tar.gz -C /data .
+   docker run --rm -v postgres-data:/data \
+     -v "$(pwd)":/backup busybox tar czf /backup/postgres-data-$(date +%Y%m%d).tar.gz -C /data .
   ```
 4. Copy configuration, seeds, and secrets as in the hot backup.
 5. Restart services and confirm `/health/live` returns `200 OK`.

 ## Restore procedure
 1. **Provision clean volumes**
-   - Compose: `docker volume rm mongo-data` (optional) then `docker compose up -d mongo`.
-   - Helm: delete the Mongo PVC or attach a fresh volume snapshot.
-2. **Restore Mongo**
+   - Compose: `docker volume rm postgres-data` (optional) then `docker compose up -d postgres`.
+   - Helm: delete the PostgreSQL PVC or attach a fresh volume snapshot.
+2. **Restore PostgreSQL**
   ```bash
-   docker compose exec -T mongo \
-     mongorestore --archive \
-     --gzip --drop < issuer-directory-YYYYMMDDTHHMMSSZ.gz
+   docker compose exec -T postgres \
+     pg_restore --format=custom --clean --if-exists \
+     --dbname=issuer_directory < issuer-directory-YYYYMMDDTHHMMSSZ.dump
   ```
 3. **Restore configuration/secrets**
   - Copy `issuer-directory.yaml` into `etc/`.
@@ -87,7 +88,7 @@
 6. **Validate**
   - `curl -fsSL https://localhost:8447/health/live`
   - Issue an access token and list issuers to confirm results.
-   - Check Mongo counts match expectations (`db.issuers.countDocuments()`, etc.).
+   - Check PostgreSQL counts match expectations (`SELECT COUNT(*) FROM issuer_directory.issuers;`, etc.).
   - Confirm Prometheus scrapes `issuer_directory_changes_total` and `issuer_directory_key_operations_total` for the tenants you restored.

 ## Disaster recovery notes
@@ -98,7 +99,7 @@

 ## Verification checklist
 - [ ] `/health/live` returns `200 OK`.
- [ ] Mongo collections (`issuers`, `issuer_keys`, `issuer_trust_overrides`) have expected counts.
+- [ ] PostgreSQL tables (`issuers`, `issuer_keys`, `issuer_trust_overrides`) have expected counts.
 - [ ] `issuer_directory_changes_total`, `issuer_directory_key_operations_total`, and `issuer_directory_key_validation_failures_total` metrics resume within 1 minute.
 - [ ] Audit entries exist for post-restore CRUD activity.
 - [ ] Client integrations (VEX Lens, Excititor) resolve issuers successfully.
--- a/docs/modules/issuer-directory/operations/deployment.md
+++ b/docs/modules/issuer-directory/operations/deployment.md
@@ -7,34 +7,34 @@

 ## 1 · Prerequisites
 - Authority must be running and reachable at the issuer URL you configure (default Compose host: `https://authority:8440`).
- MongoDB 4.2+ with credentials for the `issuer-directory` database (Compose defaults to the root user defined in `.env`).
- Network access to Authority, MongoDB, and (optionally) Prometheus if you scrape metrics.
+- PostgreSQL 14+ with credentials for the `issuer_directory` database (Compose defaults to the user defined in `.env`).
+- Network access to Authority, PostgreSQL, and (optionally) Prometheus if you scrape metrics.
 - Issuer Directory configuration file `etc/issuer-directory.yaml` checked and customised for your environment (tenant header, audiences, telemetry level, CSAF seed path).

-> **Secrets:** Use `etc/secrets/issuer-directory.mongo.secret.example` as a template. Store the real connection string in an untracked file or secrets manager and reference it via environment variables (`ISSUER_DIRECTORY_MONGO_CONNECTION_STRING`) rather than committing credentials.
+> **Secrets:** Use `etc/secrets/issuer-directory.postgres.secret.example` as a template. Store the real connection string in an untracked file or secrets manager and reference it via environment variables (`ISSUER_DIRECTORY_POSTGRES_CONNECTION_STRING`) rather than committing credentials.

 ## 2 · Deploy with Docker Compose
 1. **Prepare environment variables**
   ```bash
   cp deploy/compose/env/dev.env.example dev.env
-   cp etc/secrets/issuer-directory.mongo.secret.example issuer-directory.mongo.env
-   # Edit dev.env and issuer-directory.mongo.env with production-ready secrets.
+   cp etc/secrets/issuer-directory.postgres.secret.example issuer-directory.postgres.env
+   # Edit dev.env and issuer-directory.postgres.env with production-ready secrets.
   ```

 2. **Inspect the merged configuration**
   ```bash
   docker compose \
     --env-file dev.env \
-     --env-file issuer-directory.mongo.env \
+     --env-file issuer-directory.postgres.env \
     -f deploy/compose/docker-compose.dev.yaml config
   ```
-   The command confirms the new `issuer-directory` service resolves the port (`${ISSUER_DIRECTORY_PORT:-8447}`) and the Mongo connection string is in place.
+   The command confirms the new `issuer-directory` service resolves the port (`${ISSUER_DIRECTORY_PORT:-8447}`) and the PostgreSQL connection string is in place.

 3. **Launch the stack**
   ```bash
   docker compose \
     --env-file dev.env \
-     --env-file issuer-directory.mongo.env \
+     --env-file issuer-directory.postgres.env \
     -f deploy/compose/docker-compose.dev.yaml up -d issuer-directory
   ```
   Compose automatically mounts `../../etc/issuer-directory.yaml` into the container at `/etc/issuer-directory.yaml`, seeds CSAF publishers, and exposes the API on `https://localhost:8447`.
@@ -43,7 +43,7 @@
 | Variable | Purpose | Default |
 | --- | --- | --- |
 | `ISSUER_DIRECTORY_PORT` | Host port that maps to container port `8080`. | `8447` |
-| `ISSUER_DIRECTORY_MONGO_CONNECTION_STRING` | Injected into `ISSUERDIRECTORY__MONGO__CONNECTIONSTRING`; should contain credentials. | `mongodb://${MONGO_INITDB_ROOT_USERNAME}:${MONGO_INITDB_ROOT_PASSWORD}@mongo:27017` |
+| `ISSUER_DIRECTORY_POSTGRES_CONNECTION_STRING` | Injected into `ISSUERDIRECTORY__POSTGRES__CONNECTIONSTRING`; should contain credentials. | `Host=postgres;Port=5432;Database=issuer_directory;Username=${POSTGRES_USER};Password=${POSTGRES_PASSWORD}` |
 | `ISSUER_DIRECTORY_SEED_CSAF` | Toggles CSAF bootstrap on startup. Set to `false` after the first production import if you manage issuers manually. | `true` |

 4. **Smoke test**
@@ -63,7 +63,7 @@
 1. **Create or update the secret**
   ```bash
   kubectl create secret generic issuer-directory-secrets \
-     --from-literal=ISSUERDIRECTORY__MONGO__CONNECTIONSTRING='mongodb://stellaops:<password>@stellaops-mongo:27017' \
+     --from-literal=ISSUERDIRECTORY__POSTGRES__CONNECTIONSTRING='Host=stellaops-postgres;Port=5432;Database=issuer_directory;Username=stellaops;Password=<password>' \
     --dry-run=client -o yaml | kubectl apply -f -
   ```
   Add optional overrides (e.g. `ISSUERDIRECTORY__AUTHORITY__ISSUER`) if your Authority issuer differs from the default.
@@ -95,7 +95,7 @@
   ```bash
   kubectl exec deploy/stellaops-issuer-directory -- \
     curl -sf http://127.0.0.1:8080/health/live
-   kubectl logs deploy/stellaops-issuer-directory | grep 'IssuerDirectory Mongo connected'
+   kubectl logs deploy/stellaops-issuer-directory | grep 'IssuerDirectory PostgreSQL connected'
   ```
   Prometheus should begin scraping `issuer_directory_changes_total` and related metrics (labels: `tenant`, `issuer`, `action`).