feat(rate-limiting): Implement core rate limiting functionality with configuration, decision-making, metrics, middleware, and service registration

- Add RateLimitConfig for configuration management with YAML binding support.
- Introduce RateLimitDecision to encapsulate the result of rate limit checks.
- Implement RateLimitMetrics for OpenTelemetry metrics tracking.
- Create RateLimitMiddleware for enforcing rate limits on incoming requests.
- Develop RateLimitService to orchestrate instance and environment rate limit checks.
- Add RateLimitServiceCollectionExtensions for dependency injection registration.
This commit is contained in:
master
2025-12-17 18:02:37 +02:00
parent 394b57f6bf
commit 8bbfe4d2d2
211 changed files with 47179 additions and 1590 deletions

View File

@@ -11,7 +11,7 @@
| Component | Requirement | Notes |
|-----------|-------------|-------|
| Runtime | .NET 10.0+ | LTS recommended |
| Database | MongoDB 6.0+ | For projections and issuer directory |
| Database | PostgreSQL 15.0+ | For projections and issuer directory |
| Cache | Redis 7.0+ (optional) | For caching consensus results |
| Memory | 512MB minimum | 2GB recommended for production |
| CPU | 2 cores minimum | 4 cores for high throughput |
@@ -43,13 +43,12 @@ VEXLENS_TRUST_ALLOW_UNKNOWN_ISSUERS=true
VEXLENS_TRUST_UNKNOWN_ISSUER_PENALTY=0.5
# Storage
VEXLENS_STORAGE_MONGODB_CONNECTION_STRING=mongodb://localhost:27017
VEXLENS_STORAGE_MONGODB_DATABASE=vexlens
VEXLENS_STORAGE_POSTGRESQL_CONNECTION_STRING=Host=localhost;Database=vexlens;Username=stellaops;Password=secret
VEXLENS_STORAGE_PROJECTION_RETENTION_DAYS=365
VEXLENS_STORAGE_EVENT_RETENTION_DAYS=90
# Issuer Directory
VEXLENS_ISSUER_DIRECTORY_SOURCE=mongodb
VEXLENS_ISSUER_DIRECTORY_SOURCE=postgresql
VEXLENS_ISSUER_DIRECTORY_REFRESH_INTERVAL_MINUTES=60
# Observability
@@ -86,16 +85,15 @@ vexlens:
ProductAuthority: 0.05
storage:
mongodb:
connectionString: mongodb://localhost:27017
database: vexlens
projectionsCollection: consensus_projections
issuersCollection: issuers
postgresql:
connectionString: Host=localhost;Database=vexlens;Username=stellaops;Password=secret
projectionsTable: consensus_projections
issuersTable: issuers
projectionRetentionDays: 365
eventRetentionDays: 90
issuerDirectory:
source: mongodb
source: postgresql
refreshIntervalMinutes: 60
seedFile: /etc/vexlens/issuers.json
@@ -126,7 +124,7 @@ docker run -d \
--name vexlens \
-p 8080:8080 \
-v /etc/vexlens:/etc/vexlens:ro \
-e VEXLENS_STORAGE_MONGODB_CONNECTION_STRING=mongodb://mongo:27017 \
-e VEXLENS_STORAGE_POSTGRESQL_CONNECTION_STRING="Host=postgres;Database=vexlens;Username=stellaops;Password=secret" \
stellaops/vexlens:latest
```
@@ -154,11 +152,11 @@ spec:
ports:
- containerPort: 8080
env:
- name: VEXLENS_STORAGE_MONGODB_CONNECTION_STRING
- name: VEXLENS_STORAGE_POSTGRESQL_CONNECTION_STRING
valueFrom:
secretKeyRef:
name: vexlens-secrets
key: mongodb-connection-string
key: postgresql-connection-string
resources:
requests:
memory: "512Mi"
@@ -205,7 +203,7 @@ spec:
```bash
helm install vexlens stellaops/vexlens \
--namespace stellaops \
--set mongodb.connectionString=mongodb://mongo:27017 \
--set postgresql.connectionString="Host=postgres;Database=vexlens;Username=stellaops;Password=secret" \
--set replicas=2 \
--set resources.requests.memory=512Mi \
--set resources.limits.memory=2Gi
@@ -293,7 +291,7 @@ curl http://vexlens:8080/health/live
```bash
curl http://vexlens:8080/health/ready
# Response: {"status": "Healthy", "checks": {"mongodb": "Healthy", "issuerDirectory": "Healthy"}}
# Response: {"status": "Healthy", "checks": {"postgresql": "Healthy", "issuerDirectory": "Healthy"}}
```
### 5.3 Detailed Health
@@ -358,11 +356,10 @@ groups:
### 7.1 Backup Projections
```bash
# MongoDB backup
mongodump --uri="mongodb://localhost:27017" \
--db=vexlens \
--collection=consensus_projections \
--out=/backup/vexlens-$(date +%Y%m%d)
# PostgreSQL backup
pg_dump -h localhost -U stellaops -d vexlens \
-t consensus_projections \
-F c -f /backup/vexlens-projections-$(date +%Y%m%d).dump
```
### 7.2 Backup Issuer Directory
@@ -376,10 +373,9 @@ curl http://vexlens:8080/api/v1/vexlens/issuers?limit=1000 \
### 7.3 Restore
```bash
# Restore MongoDB
mongorestore --uri="mongodb://localhost:27017" \
--db=vexlens \
/backup/vexlens-20251206/
# Restore PostgreSQL
pg_restore -h localhost -U stellaops -d vexlens \
/backup/vexlens-projections-20251206.dump
# Re-seed issuers if needed
# Issuers are automatically loaded from seed file on startup
@@ -408,10 +404,10 @@ vexlens:
batchTimeoutMs: 50
storage:
mongodb:
postgresql:
# Connection pool
maxConnectionPoolSize: 100
minConnectionPoolSize: 10
maxPoolSize: 100
minPoolSize: 10
caching:
enabled: true