499 lines
14 KiB
Markdown
499 lines
14 KiB
Markdown
# Federation Setup and Operations
|
|
|
|
Per SPRINT_8200_0014_0003.
|
|
|
|
> **Related:** [Bundle Export Format](federation-bundle-export.md) for detailed bundle schema.
|
|
|
|
## Overview
|
|
|
|
Federation enables secure, cursor-based synchronization of canonical vulnerability advisories between StellaOps sites. It supports:
|
|
|
|
- **Delta exports**: Only changed records since the last cursor are included
|
|
- **Air-gap transfers**: Bundles can be written to files for offline transfer
|
|
- **Multi-site topology**: Multiple sites can synchronize independently
|
|
- **Cryptographic verification**: DSSE signatures ensure bundle authenticity
|
|
|
|
## Bundle Format
|
|
|
|
Federation bundles are ZST-compressed TAR archives containing:
|
|
|
|
| File | Description |
|
|
|------|-------------|
|
|
| `MANIFEST.json` | Bundle metadata, cursor, counts, hash |
|
|
| `canonicals.ndjson` | Canonical advisories (one per line) |
|
|
| `edges.ndjson` | Source edges linking advisories to sources |
|
|
| `deletions.ndjson` | Withdrawn/deleted advisory IDs |
|
|
| `SIGNATURE.json` | Optional DSSE signature envelope |
|
|
|
|
## Cursor Format
|
|
|
|
Cursors use ISO-8601 timestamp with sequence number:
|
|
|
|
```
|
|
{ISO-8601 timestamp}#{sequence number}
|
|
|
|
Examples:
|
|
2025-01-15T10:00:00.000Z#0001
|
|
2025-01-15T10:00:00.000Z#0002
|
|
```
|
|
|
|
- Cursors are site-specific (each site maintains independent cursors)
|
|
- Sequence numbers distinguish concurrent exports
|
|
- Cursors are monotonically increasing within a site
|
|
|
|
## Architecture
|
|
|
|
```
|
|
┌─────────────────────────────────────────────────────────────────────────────┐
|
|
│ Federation Topology │
|
|
├─────────────────────────────────────────────────────────────────────────────┤
|
|
│ │
|
|
│ ┌──────────────────┐ ┌──────────────────┐ │
|
|
│ │ Site A (HQ) │ │ Site B (Branch) │ │
|
|
│ │ │ │ │ │
|
|
│ │ ┌────────────┐ │ Export │ ┌────────────┐ │ │
|
|
│ │ │ Concelier │──┼──────────►│ │ Concelier │ │ │
|
|
│ │ │ │ │ Bundle │ │ │ │ │
|
|
│ │ └────────────┘ │ │ └────────────┘ │ │
|
|
│ │ │ │ │ │ │ │
|
|
│ │ ▼ │ │ ▼ │ │
|
|
│ │ ┌────────────┐ │ │ ┌────────────┐ │ │
|
|
│ │ │ PostgreSQL │ │ │ │ PostgreSQL │ │ │
|
|
│ │ └────────────┘ │ │ └────────────┘ │ │
|
|
│ └──────────────────┘ └──────────────────┘ │
|
|
│ │
|
|
│ ┌──────────────────┐ │
|
|
│ │ Site C (Air-Gap)│ │
|
|
│ │ │ │
|
|
│ │ ┌────────────┐ │ USB/Secure │
|
|
│ │ │ Concelier │◄─┼───Transfer │
|
|
│ │ │ │ │ │
|
|
│ │ └────────────┘ │ │
|
|
│ └──────────────────┘ │
|
|
│ │
|
|
└─────────────────────────────────────────────────────────────────────────────┘
|
|
```
|
|
|
|
## Setup
|
|
|
|
### 1. Enable Federation
|
|
|
|
Configure federation in `concelier.yaml`:
|
|
|
|
```yaml
|
|
Federation:
|
|
Enabled: true
|
|
SiteId: "site-us-west-1" # Unique identifier for this site
|
|
DefaultCompressionLevel: 3
|
|
DefaultMaxItems: 10000
|
|
RequireSignature: true
|
|
|
|
FederationImport:
|
|
AllowedSites:
|
|
- "site-us-east-1"
|
|
- "site-eu-central-1"
|
|
MaxBundleSizeBytes: 104857600 # 100 MB
|
|
SkipSignatureOnTrustedSites: false
|
|
```
|
|
|
|
### 2. Configure Site Policies
|
|
|
|
Create site policies for each trusted federation partner:
|
|
|
|
```bash
|
|
# Add trusted site
|
|
stella feedser sites add site-us-east-1 \
|
|
--display-name "US East Production" \
|
|
--enabled
|
|
|
|
# Configure policy
|
|
stella feedser sites policy site-us-east-1 \
|
|
--max-bundle-size 100MB \
|
|
--allowed-sources nvd,ghsa,debian
|
|
```
|
|
|
|
### 3. Generate Signing Keys
|
|
|
|
For signed bundles, configure Authority keys:
|
|
|
|
```bash
|
|
# Generate federation signing key
|
|
stella authority keys generate \
|
|
--name federation-signer \
|
|
--algorithm ES256 \
|
|
--purpose federation
|
|
|
|
# Export public key for distribution
|
|
stella authority keys export federation-signer --public
|
|
```
|
|
|
|
## Import Operations
|
|
|
|
### API Import
|
|
|
|
```
|
|
POST /api/v1/federation/import
|
|
Content-Type: application/zstd
|
|
```
|
|
|
|
**Query Parameters:**
|
|
|
|
| Parameter | Type | Default | Description |
|
|
|-----------|------|---------|-------------|
|
|
| `dry_run` | bool | false | Validate without importing |
|
|
| `skip_signature` | bool | false | Skip signature verification (requires trust) |
|
|
| `on_conflict` | enum | prefer_remote | `prefer_remote`, `prefer_local`, `fail` |
|
|
| `force` | bool | false | Import even if cursor is not after current |
|
|
|
|
**Response:**
|
|
|
|
```json
|
|
{
|
|
"success": true,
|
|
"bundle_hash": "sha256:a1b2c3...",
|
|
"imported_cursor": "2025-01-15T10:30:00.000Z#0042",
|
|
"counts": {
|
|
"canonical_created": 100,
|
|
"canonical_updated": 25,
|
|
"canonical_skipped": 10,
|
|
"edges_added": 200,
|
|
"deletions_processed": 5
|
|
},
|
|
"conflicts": [],
|
|
"duration_ms": 1234
|
|
}
|
|
```
|
|
|
|
### CLI Import
|
|
|
|
```bash
|
|
# Import from file
|
|
stella feedser bundle import ./bundle.zst
|
|
|
|
# Import with dry run
|
|
stella feedser bundle import ./bundle.zst --dry-run
|
|
|
|
# Import from stdin (for pipes)
|
|
cat bundle.zst | stella feedser bundle import -
|
|
|
|
# Import without signature verification (testing only)
|
|
stella feedser bundle import ./bundle.zst --skip-signature
|
|
|
|
# Force import (override cursor check)
|
|
stella feedser bundle import ./bundle.zst --force
|
|
```
|
|
|
|
### Conflict Resolution
|
|
|
|
When conflicts occur between local and remote values:
|
|
|
|
| Strategy | Behavior |
|
|
|----------|----------|
|
|
| `prefer_remote` | Remote value wins (default) |
|
|
| `prefer_local` | Local value preserved |
|
|
| `fail` | Abort import on first conflict |
|
|
|
|
Conflicts are logged with full details:
|
|
|
|
```json
|
|
{
|
|
"merge_hash": "sha256:abc...",
|
|
"field": "severity",
|
|
"local_value": "high",
|
|
"remote_value": "critical",
|
|
"resolution": "prefer_remote"
|
|
}
|
|
```
|
|
|
|
## Site Management
|
|
|
|
### List Sites
|
|
|
|
```bash
|
|
stella feedser sites list
|
|
```
|
|
|
|
Output:
|
|
```
|
|
SITE ID STATUS LAST SYNC CURSOR
|
|
───────────────────────── ──────── ─────────────────── ──────────────────────────
|
|
site-us-east-1 enabled 2025-01-15 10:30 2025-01-15T10:30:00Z#0042
|
|
site-eu-central-1 enabled 2025-01-15 09:15 2025-01-15T09:15:00Z#0038
|
|
site-asia-pacific-1 disabled never -
|
|
```
|
|
|
|
### View Site History
|
|
|
|
```bash
|
|
stella feedser sites history site-us-east-1 --limit 10
|
|
```
|
|
|
|
### Update Site Policy
|
|
|
|
```bash
|
|
stella feedser sites policy site-us-east-1 \
|
|
--enabled false # Disable imports from this site
|
|
```
|
|
|
|
## Air-Gap Operations
|
|
|
|
For sites without network connectivity:
|
|
|
|
### Export for Transfer
|
|
|
|
```bash
|
|
# On connected site
|
|
stella feedser bundle export \
|
|
-c "2025-01-14T00:00:00Z#0000" \
|
|
-o ./delta-2025-01-15.zst
|
|
|
|
# Transfer via USB/secure media
|
|
```
|
|
|
|
### Import on Air-Gap Site
|
|
|
|
```bash
|
|
# On air-gapped site
|
|
stella feedser bundle import ./delta-2025-01-15.zst
|
|
|
|
# Verify import
|
|
stella feedser sites list
|
|
```
|
|
|
|
### Full Sync Workflow
|
|
|
|
1. **Initial Sync:**
|
|
```bash
|
|
# Export full dataset
|
|
stella feedser bundle export -o ./full-sync.zst
|
|
```
|
|
|
|
2. **Transfer to air-gap site**
|
|
|
|
3. **Import on air-gap:**
|
|
```bash
|
|
stella feedser bundle import ./full-sync.zst
|
|
```
|
|
|
|
4. **Subsequent Delta Syncs:**
|
|
```bash
|
|
# Get current cursor from air-gap site
|
|
stella feedser sites list # Note the cursor
|
|
|
|
# On connected site, export delta
|
|
stella feedser bundle export -c "{cursor}" -o ./delta.zst
|
|
|
|
# Transfer and import on air-gap
|
|
```
|
|
|
|
## Verification
|
|
|
|
### Validate Bundle Without Import
|
|
|
|
```bash
|
|
stella feedser bundle validate ./bundle.zst
|
|
```
|
|
|
|
Output:
|
|
```
|
|
Bundle: bundle.zst
|
|
Version: feedser-bundle/1.0
|
|
Site: site-us-east-1
|
|
Cursor: 2025-01-15T10:30:00.000Z#0042
|
|
|
|
Counts:
|
|
Canonicals: 1,234
|
|
Edges: 3,456
|
|
Deletions: 12
|
|
Total: 4,702
|
|
|
|
Verification:
|
|
Hash: ✓ Valid
|
|
Signature: ✓ Valid (key: sha256:abc...)
|
|
Format: ✓ Valid
|
|
|
|
Ready for import.
|
|
```
|
|
|
|
### Preview Import Impact
|
|
|
|
```bash
|
|
stella feedser bundle import ./bundle.zst --dry-run --json
|
|
```
|
|
|
|
## Monitoring
|
|
|
|
### Sync Status Endpoint
|
|
|
|
```
|
|
GET /api/v1/federation/sync/status
|
|
```
|
|
|
|
Response:
|
|
```json
|
|
{
|
|
"sites": [
|
|
{
|
|
"site_id": "site-us-east-1",
|
|
"enabled": true,
|
|
"last_sync_at": "2025-01-15T10:30:00Z",
|
|
"last_cursor": "2025-01-15T10:30:00.000Z#0042",
|
|
"bundles_imported": 156,
|
|
"total_items_imported": 45678
|
|
}
|
|
],
|
|
"local_cursor": "2025-01-15T10:35:00.000Z#0044"
|
|
}
|
|
```
|
|
|
|
### Event Stream
|
|
|
|
Import events are published to the `canonical-imported` stream:
|
|
|
|
```json
|
|
{
|
|
"canonical_id": "uuid",
|
|
"cve": "CVE-2024-1234",
|
|
"affects_key": "pkg:npm/express@4.0.0",
|
|
"merge_hash": "sha256:...",
|
|
"action": "Created",
|
|
"bundle_hash": "sha256:...",
|
|
"site_id": "site-us-east-1",
|
|
"imported_at": "2025-01-15T10:30:15Z"
|
|
}
|
|
```
|
|
|
|
### Cache Invalidation
|
|
|
|
After import, cache indexes are automatically updated:
|
|
- PURL index updated for affected packages
|
|
- CVE index updated for vulnerability lookups
|
|
- Existing cache entries invalidated for refresh
|
|
|
|
## Troubleshooting
|
|
|
|
### Common Issues
|
|
|
|
| Issue | Cause | Solution |
|
|
|-------|-------|----------|
|
|
| "Cursor not after current" | Bundle is stale | Use `--force` or export newer bundle |
|
|
| "Signature verification failed" | Key mismatch | Verify signing key is trusted |
|
|
| "Site not allowed" | Policy restriction | Add site to `AllowedSites` config |
|
|
| "Bundle too large" | Size limit exceeded | Increase `MaxBundleSizeBytes` or export smaller delta |
|
|
|
|
### Debug Logging
|
|
|
|
Enable verbose logging for federation operations:
|
|
|
|
```yaml
|
|
Logging:
|
|
LogLevel:
|
|
StellaOps.Concelier.Federation: Debug
|
|
```
|
|
|
|
### Verify Sync State
|
|
|
|
```bash
|
|
# Check local vs remote cursor
|
|
stella feedser sites status site-us-east-1
|
|
|
|
# List recent imports
|
|
stella feedser sites history site-us-east-1 --limit 5
|
|
|
|
# Verify specific canonical was imported
|
|
stella feedser canonical get sha256:mergehash...
|
|
```
|
|
|
|
## Best Practices
|
|
|
|
1. **Regular Sync Schedule:** Configure automated delta exports/imports on a schedule (e.g., hourly)
|
|
|
|
2. **Monitor Cursor Drift:** Alert if cursor falls too far behind
|
|
|
|
3. **Verify Signatures:** Only disable signature verification in development
|
|
|
|
4. **Size Bundles Appropriately:** For large deltas, split into multiple bundles
|
|
|
|
5. **Test Import Before Production:** Use `--dry-run` to validate bundles
|
|
|
|
6. **Maintain Key Trust:** Regularly rotate and verify federation signing keys
|
|
|
|
7. **Document Site Policies:** Keep a registry of trusted sites and their policies
|
|
|
|
## Multi-Site Topologies
|
|
|
|
### Hub-and-Spoke Topology
|
|
|
|
```
|
|
┌─────────────┐
|
|
│ Hub Site │
|
|
│ (Primary) │
|
|
└──────┬──────┘
|
|
│
|
|
┌──────────┼──────────┐
|
|
▼ ▼ ▼
|
|
┌──────────┐ ┌──────────┐ ┌──────────┐
|
|
│ Site A │ │ Site B │ │ Site C │
|
|
│ (Spoke) │ │ (Spoke) │ │ (Spoke) │
|
|
└──────────┘ └──────────┘ └──────────┘
|
|
```
|
|
|
|
### Mesh Topology
|
|
|
|
Each site can import from multiple sources for redundancy:
|
|
|
|
```yaml
|
|
federation:
|
|
import:
|
|
allowed_sites:
|
|
- "hub-primary"
|
|
- "hub-secondary" # Redundancy
|
|
```
|
|
|
|
## Verification Details
|
|
|
|
### Hash Verification
|
|
|
|
Bundle hash is computed over compressed content:
|
|
|
|
```
|
|
SHA256(compressed bundle content)
|
|
```
|
|
|
|
### DSSE Signature Format
|
|
|
|
DSSE envelope contains:
|
|
|
|
```json
|
|
{
|
|
"payloadType": "application/stellaops.federation.bundle+json",
|
|
"payload": "base64(bundle_hash + site_id + cursor)",
|
|
"signatures": [
|
|
{
|
|
"keyId": "signing-key-001",
|
|
"algorithm": "ES256",
|
|
"signature": "base64(signature)"
|
|
}
|
|
]
|
|
}
|
|
```
|
|
|
|
## Monitoring Metrics
|
|
|
|
### Key Prometheus Metrics
|
|
|
|
- `federation_export_duration_seconds` - Export time
|
|
- `federation_import_duration_seconds` - Import time
|
|
- `federation_bundle_size_bytes` - Bundle sizes
|
|
- `federation_items_processed_total` - Items processed by type
|
|
- `federation_conflicts_total` - Merge conflicts encountered
|
|
|
|
## Security Considerations
|
|
|
|
1. **Never skip signature verification in production**
|
|
2. **Validate allowed_sites whitelist**
|
|
3. **Use TLS for API endpoints**
|
|
4. **Rotate signing keys periodically**
|
|
5. **Audit import events**
|
|
6. **Monitor for duplicate bundle imports**
|