Gaps fill up, fixes, ui restructuring
This commit is contained in:
118
docs/modules/telemetry/federation-architecture.md
Normal file
118
docs/modules/telemetry/federation-architecture.md
Normal file
@@ -0,0 +1,118 @@
|
||||
# Federated Telemetry Architecture
|
||||
|
||||
## Overview
|
||||
|
||||
The Federated Telemetry subsystem enables privacy-preserving sharing of runtime exploit intelligence across Stella Ops instances in a federation mesh. It uses differential privacy (Laplacian noise) and k-anonymity to ensure that individual tenant data cannot be reconstructed from shared aggregates.
|
||||
|
||||
## Data Flow
|
||||
|
||||
```
|
||||
Tenant Runtime -> TelemetryFact Buffer -> TelemetryAggregator
|
||||
-> k-Anonymity Filter -> Laplacian Noise -> AggregationResult
|
||||
-> ConsentManager Check -> BundleBuilder -> DSSE-Signed Bundle
|
||||
-> EgressPolicy Check -> Federation Mesh Sync
|
||||
-> ExploitIntelligenceMerger <- Incoming Bundles from Peers
|
||||
```
|
||||
|
||||
## Privacy Model
|
||||
|
||||
### Differential Privacy (Epsilon Budget)
|
||||
|
||||
Each aggregation cycle consumes a portion of the total epsilon budget. The budget resets on a configurable period (default: 24 hours).
|
||||
|
||||
- **Epsilon per bucket**: `total_budget / number_of_buckets`
|
||||
- **Laplacian noise**: `-(sensitivity / epsilon) * sign(u) * ln(1 - 2|u|)` where u ~ Uniform(-0.5, 0.5)
|
||||
- **Budget exhaustion**: When remaining epsilon reaches zero, all further aggregation is suppressed until the next reset period.
|
||||
|
||||
### K-Anonymity
|
||||
|
||||
Buckets (grouped by CVE ID) with fewer than `k` distinct artifact digests are suppressed entirely. The default threshold is k=5, configurable via `FederatedTelemetryOptions.KAnonymityThreshold`.
|
||||
|
||||
## Consent Lifecycle
|
||||
|
||||
1. **Not Granted** (default) -- no federation data leaves the instance.
|
||||
2. **Granted** -- admin explicitly grants consent with optional TTL. A DSSE-signed consent proof is created.
|
||||
3. **Expired** -- consent with a TTL automatically reverts to Not Granted after expiry.
|
||||
4. **Revoked** -- admin explicitly revokes consent.
|
||||
|
||||
Consent state is checked at the start of each sync cycle. No bundles are built or transmitted without active consent.
|
||||
|
||||
## Sync Service
|
||||
|
||||
`FederatedTelemetrySyncService` is a `BackgroundService` that runs on a configurable interval (default: 15 minutes).
|
||||
|
||||
Each cycle:
|
||||
1. Check sealed mode -- skip if active.
|
||||
2. Check privacy budget -- skip if exhausted.
|
||||
3. Check consent -- skip if not granted.
|
||||
4. Drain fact buffer.
|
||||
5. Aggregate facts with k-anonymity and Laplacian noise.
|
||||
6. Build DSSE-signed bundle.
|
||||
7. Check egress policy.
|
||||
8. Transmit to federation mesh.
|
||||
|
||||
## Intelligence Merging
|
||||
|
||||
Incoming bundles from federation peers are processed by `ExploitIntelligenceMerger`:
|
||||
- Entries are normalized (CVE ID uppercase, timestamps UTC, site IDs lowercase).
|
||||
- Deduplication by `(CveId, SourceSiteId)` composite key.
|
||||
- Conflict resolution: latest `ObservedAt` wins.
|
||||
|
||||
## Bundle Format
|
||||
|
||||
A `FederatedBundle` contains:
|
||||
- Unique ID (GUID)
|
||||
- Source site identifier
|
||||
- Aggregation result (buckets with noisy counts, suppression flags)
|
||||
- Consent DSSE digest (proof that consent was active)
|
||||
- Bundle DSSE digest (integrity verification)
|
||||
- DSSE envelope (signed payload)
|
||||
- Creation timestamp
|
||||
|
||||
## Sealed Mode
|
||||
|
||||
When `FederatedTelemetryOptions.SealedModeEnabled` is true:
|
||||
- Sync service skips all cycles.
|
||||
- No outbound traffic is generated.
|
||||
- Local aggregation still functions for internal analytics.
|
||||
- Intelligence merging is paused.
|
||||
|
||||
## Configuration
|
||||
|
||||
```json
|
||||
{
|
||||
"FederatedTelemetry": {
|
||||
"KAnonymityThreshold": 5,
|
||||
"EpsilonBudget": 1.0,
|
||||
"BudgetResetPeriod": "24:00:00",
|
||||
"AggregationInterval": "00:15:00",
|
||||
"SealedModeEnabled": false,
|
||||
"ConsentPredicateType": "stella.ops/federatedConsent@v1",
|
||||
"BundlePredicateType": "stella.ops/federatedTelemetry@v1",
|
||||
"SiteId": "site-001"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## API Surface
|
||||
|
||||
See `src/Platform/StellaOps.Platform.WebService/Endpoints/FederationTelemetryEndpoints.cs` for the full REST API.
|
||||
|
||||
| Method | Path | Auth Policy | Description |
|
||||
|--------|------|-------------|-------------|
|
||||
| GET | /api/v1/telemetry/federation/consent | FederationRead | Get consent state |
|
||||
| POST | /api/v1/telemetry/federation/consent/grant | FederationManage | Grant consent |
|
||||
| POST | /api/v1/telemetry/federation/consent/revoke | FederationManage | Revoke consent |
|
||||
| GET | /api/v1/telemetry/federation/status | FederationRead | Federation status |
|
||||
| GET | /api/v1/telemetry/federation/bundles | FederationRead | List bundles |
|
||||
| GET | /api/v1/telemetry/federation/bundles/{id} | FederationRead | Bundle detail |
|
||||
| GET | /api/v1/telemetry/federation/intelligence | FederationRead | Exploit corpus |
|
||||
| GET | /api/v1/telemetry/federation/privacy-budget | FederationRead | Budget snapshot |
|
||||
| POST | /api/v1/telemetry/federation/trigger | FederationManage | Trigger aggregation |
|
||||
|
||||
## Source Files
|
||||
|
||||
- Project: `src/Telemetry/StellaOps.Telemetry.Federation/`
|
||||
- Tests: `src/Telemetry/StellaOps.Telemetry.Federation.Tests/`
|
||||
- API: `src/Platform/StellaOps.Platform.WebService/Endpoints/FederationTelemetryEndpoints.cs`
|
||||
- UI: `src/Web/StellaOps.Web/src/app/features/platform-ops/federation-telemetry/`
|
||||
Reference in New Issue
Block a user