# Federation Setup and Operations Per SPRINT_8200_0014_0003. > **Related:** [Bundle Export Format](federation-bundle-export.md) for detailed bundle schema. ## Overview Federation enables secure, cursor-based synchronization of canonical vulnerability advisories between StellaOps sites. It supports: - **Delta exports**: Only changed records since the last cursor are included - **Air-gap transfers**: Bundles can be written to files for offline transfer - **Multi-site topology**: Multiple sites can synchronize independently - **Cryptographic verification**: DSSE signatures ensure bundle authenticity ## Bundle Format Federation bundles are ZST-compressed TAR archives containing: | File | Description | |------|-------------| | `MANIFEST.json` | Bundle metadata, cursor, counts, hash | | `canonicals.ndjson` | Canonical advisories (one per line) | | `edges.ndjson` | Source edges linking advisories to sources | | `deletions.ndjson` | Withdrawn/deleted advisory IDs | | `SIGNATURE.json` | Optional DSSE signature envelope | ## Cursor Format Cursors use ISO-8601 timestamp with sequence number: ``` {ISO-8601 timestamp}#{sequence number} Examples: 2025-01-15T10:00:00.000Z#0001 2025-01-15T10:00:00.000Z#0002 ``` - Cursors are site-specific (each site maintains independent cursors) - Sequence numbers distinguish concurrent exports - Cursors are monotonically increasing within a site ## Architecture ``` ┌─────────────────────────────────────────────────────────────────────────────┐ │ Federation Topology │ ├─────────────────────────────────────────────────────────────────────────────┤ │ │ │ ┌──────────────────┐ ┌──────────────────┐ │ │ │ Site A (HQ) │ │ Site B (Branch) │ │ │ │ │ │ │ │ │ │ ┌────────────┐ │ Export │ ┌────────────┐ │ │ │ │ │ Concelier │──┼──────────►│ │ Concelier │ │ │ │ │ │ │ │ Bundle │ │ │ │ │ │ │ └────────────┘ │ │ └────────────┘ │ │ │ │ │ │ │ │ │ │ │ │ ▼ │ │ ▼ │ │ │ │ ┌────────────┐ │ │ ┌────────────┐ │ │ │ │ │ PostgreSQL │ │ │ │ PostgreSQL │ │ │ │ │ └────────────┘ │ │ └────────────┘ │ │ │ └──────────────────┘ └──────────────────┘ │ │ │ │ ┌──────────────────┐ │ │ │ Site C (Air-Gap)│ │ │ │ │ │ │ │ ┌────────────┐ │ USB/Secure │ │ │ │ Concelier │◄─┼───Transfer │ │ │ │ │ │ │ │ │ └────────────┘ │ │ │ └──────────────────┘ │ │ │ └─────────────────────────────────────────────────────────────────────────────┘ ``` ## Setup ### 1. Enable Federation Configure federation in `concelier.yaml`: ```yaml Federation: Enabled: true SiteId: "site-us-west-1" # Unique identifier for this site DefaultCompressionLevel: 3 DefaultMaxItems: 10000 RequireSignature: true FederationImport: AllowedSites: - "site-us-east-1" - "site-eu-central-1" MaxBundleSizeBytes: 104857600 # 100 MB SkipSignatureOnTrustedSites: false ``` ### 2. Configure Site Policies Create site policies for each trusted federation partner: ```bash # Add trusted site stella feedser sites add site-us-east-1 \ --display-name "US East Production" \ --enabled # Configure policy stella feedser sites policy site-us-east-1 \ --max-bundle-size 100MB \ --allowed-sources nvd,ghsa,debian ``` ### 3. Generate Signing Keys For signed bundles, configure Authority keys: ```bash # Generate federation signing key stella authority keys generate \ --name federation-signer \ --algorithm ES256 \ --purpose federation # Export public key for distribution stella authority keys export federation-signer --public ``` ## Import Operations ### API Import ``` POST /api/v1/federation/import Content-Type: application/zstd ``` **Query Parameters:** | Parameter | Type | Default | Description | |-----------|------|---------|-------------| | `dry_run` | bool | false | Validate without importing | | `skip_signature` | bool | false | Skip signature verification (requires trust) | | `on_conflict` | enum | prefer_remote | `prefer_remote`, `prefer_local`, `fail` | | `force` | bool | false | Import even if cursor is not after current | **Response:** ```json { "success": true, "bundle_hash": "sha256:a1b2c3...", "imported_cursor": "2025-01-15T10:30:00.000Z#0042", "counts": { "canonical_created": 100, "canonical_updated": 25, "canonical_skipped": 10, "edges_added": 200, "deletions_processed": 5 }, "conflicts": [], "duration_ms": 1234 } ``` ### CLI Import ```bash # Import from file stella feedser bundle import ./bundle.zst # Import with dry run stella feedser bundle import ./bundle.zst --dry-run # Import from stdin (for pipes) cat bundle.zst | stella feedser bundle import - # Import without signature verification (testing only) stella feedser bundle import ./bundle.zst --skip-signature # Force import (override cursor check) stella feedser bundle import ./bundle.zst --force ``` ### Conflict Resolution When conflicts occur between local and remote values: | Strategy | Behavior | |----------|----------| | `prefer_remote` | Remote value wins (default) | | `prefer_local` | Local value preserved | | `fail` | Abort import on first conflict | Conflicts are logged with full details: ```json { "merge_hash": "sha256:abc...", "field": "severity", "local_value": "high", "remote_value": "critical", "resolution": "prefer_remote" } ``` ## Site Management ### List Sites ```bash stella feedser sites list ``` Output: ``` SITE ID STATUS LAST SYNC CURSOR ───────────────────────── ──────── ─────────────────── ────────────────────────── site-us-east-1 enabled 2025-01-15 10:30 2025-01-15T10:30:00Z#0042 site-eu-central-1 enabled 2025-01-15 09:15 2025-01-15T09:15:00Z#0038 site-asia-pacific-1 disabled never - ``` ### View Site History ```bash stella feedser sites history site-us-east-1 --limit 10 ``` ### Update Site Policy ```bash stella feedser sites policy site-us-east-1 \ --enabled false # Disable imports from this site ``` ## Air-Gap Operations For sites without network connectivity: ### Export for Transfer ```bash # On connected site stella feedser bundle export \ -c "2025-01-14T00:00:00Z#0000" \ -o ./delta-2025-01-15.zst # Transfer via USB/secure media ``` ### Import on Air-Gap Site ```bash # On air-gapped site stella feedser bundle import ./delta-2025-01-15.zst # Verify import stella feedser sites list ``` ### Full Sync Workflow 1. **Initial Sync:** ```bash # Export full dataset stella feedser bundle export -o ./full-sync.zst ``` 2. **Transfer to air-gap site** 3. **Import on air-gap:** ```bash stella feedser bundle import ./full-sync.zst ``` 4. **Subsequent Delta Syncs:** ```bash # Get current cursor from air-gap site stella feedser sites list # Note the cursor # On connected site, export delta stella feedser bundle export -c "{cursor}" -o ./delta.zst # Transfer and import on air-gap ``` ## Verification ### Validate Bundle Without Import ```bash stella feedser bundle validate ./bundle.zst ``` Output: ``` Bundle: bundle.zst Version: feedser-bundle/1.0 Site: site-us-east-1 Cursor: 2025-01-15T10:30:00.000Z#0042 Counts: Canonicals: 1,234 Edges: 3,456 Deletions: 12 Total: 4,702 Verification: Hash: ✓ Valid Signature: ✓ Valid (key: sha256:abc...) Format: ✓ Valid Ready for import. ``` ### Preview Import Impact ```bash stella feedser bundle import ./bundle.zst --dry-run --json ``` ## Monitoring ### Sync Status Endpoint ``` GET /api/v1/federation/sync/status ``` Response: ```json { "sites": [ { "site_id": "site-us-east-1", "enabled": true, "last_sync_at": "2025-01-15T10:30:00Z", "last_cursor": "2025-01-15T10:30:00.000Z#0042", "bundles_imported": 156, "total_items_imported": 45678 } ], "local_cursor": "2025-01-15T10:35:00.000Z#0044" } ``` ### Event Stream Import events are published to the `canonical-imported` stream: ```json { "canonical_id": "uuid", "cve": "CVE-2024-1234", "affects_key": "pkg:npm/express@4.0.0", "merge_hash": "sha256:...", "action": "Created", "bundle_hash": "sha256:...", "site_id": "site-us-east-1", "imported_at": "2025-01-15T10:30:15Z" } ``` ### Cache Invalidation After import, cache indexes are automatically updated: - PURL index updated for affected packages - CVE index updated for vulnerability lookups - Existing cache entries invalidated for refresh ## Troubleshooting ### Common Issues | Issue | Cause | Solution | |-------|-------|----------| | "Cursor not after current" | Bundle is stale | Use `--force` or export newer bundle | | "Signature verification failed" | Key mismatch | Verify signing key is trusted | | "Site not allowed" | Policy restriction | Add site to `AllowedSites` config | | "Bundle too large" | Size limit exceeded | Increase `MaxBundleSizeBytes` or export smaller delta | ### Debug Logging Enable verbose logging for federation operations: ```yaml Logging: LogLevel: StellaOps.Concelier.Federation: Debug ``` ### Verify Sync State ```bash # Check local vs remote cursor stella feedser sites status site-us-east-1 # List recent imports stella feedser sites history site-us-east-1 --limit 5 # Verify specific canonical was imported stella feedser canonical get sha256:mergehash... ``` ## Best Practices 1. **Regular Sync Schedule:** Configure automated delta exports/imports on a schedule (e.g., hourly) 2. **Monitor Cursor Drift:** Alert if cursor falls too far behind 3. **Verify Signatures:** Only disable signature verification in development 4. **Size Bundles Appropriately:** For large deltas, split into multiple bundles 5. **Test Import Before Production:** Use `--dry-run` to validate bundles 6. **Maintain Key Trust:** Regularly rotate and verify federation signing keys 7. **Document Site Policies:** Keep a registry of trusted sites and their policies ## Multi-Site Topologies ### Hub-and-Spoke Topology ``` ┌─────────────┐ │ Hub Site │ │ (Primary) │ └──────┬──────┘ │ ┌──────────┼──────────┐ ▼ ▼ ▼ ┌──────────┐ ┌──────────┐ ┌──────────┐ │ Site A │ │ Site B │ │ Site C │ │ (Spoke) │ │ (Spoke) │ │ (Spoke) │ └──────────┘ └──────────┘ └──────────┘ ``` ### Mesh Topology Each site can import from multiple sources for redundancy: ```yaml federation: import: allowed_sites: - "hub-primary" - "hub-secondary" # Redundancy ``` ## Verification Details ### Hash Verification Bundle hash is computed over compressed content: ``` SHA256(compressed bundle content) ``` ### DSSE Signature Format DSSE envelope contains: ```json { "payloadType": "application/stellaops.federation.bundle+json", "payload": "base64(bundle_hash + site_id + cursor)", "signatures": [ { "keyId": "signing-key-001", "algorithm": "ES256", "signature": "base64(signature)" } ] } ``` ## Monitoring Metrics ### Key Prometheus Metrics - `federation_export_duration_seconds` - Export time - `federation_import_duration_seconds` - Import time - `federation_bundle_size_bytes` - Bundle sizes - `federation_items_processed_total` - Items processed by type - `federation_conflicts_total` - Merge conflicts encountered ## Security Considerations 1. **Never skip signature verification in production** 2. **Validate allowed_sites whitelist** 3. **Use TLS for API endpoints** 4. **Rotate signing keys periodically** 5. **Audit import events** 6. **Monitor for duplicate bundle imports**