17613acf57
- Exported BulkTriageViewComponent and its related types from findings module. - Created a new accessibility test suite for score components using axe-core. - Introduced design tokens for score components to standardize styling. - Enhanced score breakdown popover for mobile responsiveness with drag handle. - Added date range selector functionality to score history chart component. - Implemented unit tests for date range selector in score history chart. - Created Storybook stories for bulk triage view and score history chart with date range selector.
387 lines
11 KiB
Markdown
387 lines
11 KiB
Markdown
# Federation Setup and Operations
|
|
|
|
Per SPRINT_8200_0014_0003.
|
|
|
|
## Overview
|
|
|
|
Federation enables multi-site synchronization of canonical advisory data between Concelier instances. Sites can export bundles containing delta changes and import bundles from other sites to maintain synchronized vulnerability intelligence.
|
|
|
|
## Architecture
|
|
|
|
```
|
|
┌─────────────────────────────────────────────────────────────────────────────┐
|
|
│ Federation Topology │
|
|
├─────────────────────────────────────────────────────────────────────────────┤
|
|
│ │
|
|
│ ┌──────────────────┐ ┌──────────────────┐ │
|
|
│ │ Site A (HQ) │ │ Site B (Branch) │ │
|
|
│ │ │ │ │ │
|
|
│ │ ┌────────────┐ │ Export │ ┌────────────┐ │ │
|
|
│ │ │ Concelier │──┼──────────►│ │ Concelier │ │ │
|
|
│ │ │ │ │ Bundle │ │ │ │ │
|
|
│ │ └────────────┘ │ │ └────────────┘ │ │
|
|
│ │ │ │ │ │ │ │
|
|
│ │ ▼ │ │ ▼ │ │
|
|
│ │ ┌────────────┐ │ │ ┌────────────┐ │ │
|
|
│ │ │ PostgreSQL │ │ │ │ PostgreSQL │ │ │
|
|
│ │ └────────────┘ │ │ └────────────┘ │ │
|
|
│ └──────────────────┘ └──────────────────┘ │
|
|
│ │
|
|
│ ┌──────────────────┐ │
|
|
│ │ Site C (Air-Gap)│ │
|
|
│ │ │ │
|
|
│ │ ┌────────────┐ │ USB/Secure │
|
|
│ │ │ Concelier │◄─┼───Transfer │
|
|
│ │ │ │ │ │
|
|
│ │ └────────────┘ │ │
|
|
│ └──────────────────┘ │
|
|
│ │
|
|
└─────────────────────────────────────────────────────────────────────────────┘
|
|
```
|
|
|
|
## Setup
|
|
|
|
### 1. Enable Federation
|
|
|
|
Configure federation in `concelier.yaml`:
|
|
|
|
```yaml
|
|
Federation:
|
|
Enabled: true
|
|
SiteId: "site-us-west-1" # Unique identifier for this site
|
|
DefaultCompressionLevel: 3
|
|
DefaultMaxItems: 10000
|
|
RequireSignature: true
|
|
|
|
FederationImport:
|
|
AllowedSites:
|
|
- "site-us-east-1"
|
|
- "site-eu-central-1"
|
|
MaxBundleSizeBytes: 104857600 # 100 MB
|
|
SkipSignatureOnTrustedSites: false
|
|
```
|
|
|
|
### 2. Configure Site Policies
|
|
|
|
Create site policies for each trusted federation partner:
|
|
|
|
```bash
|
|
# Add trusted site
|
|
stella feedser sites add site-us-east-1 \
|
|
--display-name "US East Production" \
|
|
--enabled
|
|
|
|
# Configure policy
|
|
stella feedser sites policy site-us-east-1 \
|
|
--max-bundle-size 100MB \
|
|
--allowed-sources nvd,ghsa,debian
|
|
```
|
|
|
|
### 3. Generate Signing Keys
|
|
|
|
For signed bundles, configure Authority keys:
|
|
|
|
```bash
|
|
# Generate federation signing key
|
|
stella authority keys generate \
|
|
--name federation-signer \
|
|
--algorithm ES256 \
|
|
--purpose federation
|
|
|
|
# Export public key for distribution
|
|
stella authority keys export federation-signer --public
|
|
```
|
|
|
|
## Import Operations
|
|
|
|
### API Import
|
|
|
|
```
|
|
POST /api/v1/federation/import
|
|
Content-Type: application/zstd
|
|
```
|
|
|
|
**Query Parameters:**
|
|
|
|
| Parameter | Type | Default | Description |
|
|
|-----------|------|---------|-------------|
|
|
| `dry_run` | bool | false | Validate without importing |
|
|
| `skip_signature` | bool | false | Skip signature verification (requires trust) |
|
|
| `on_conflict` | enum | prefer_remote | `prefer_remote`, `prefer_local`, `fail` |
|
|
| `force` | bool | false | Import even if cursor is not after current |
|
|
|
|
**Response:**
|
|
|
|
```json
|
|
{
|
|
"success": true,
|
|
"bundle_hash": "sha256:a1b2c3...",
|
|
"imported_cursor": "2025-01-15T10:30:00.000Z#0042",
|
|
"counts": {
|
|
"canonical_created": 100,
|
|
"canonical_updated": 25,
|
|
"canonical_skipped": 10,
|
|
"edges_added": 200,
|
|
"deletions_processed": 5
|
|
},
|
|
"conflicts": [],
|
|
"duration_ms": 1234
|
|
}
|
|
```
|
|
|
|
### CLI Import
|
|
|
|
```bash
|
|
# Import from file
|
|
stella feedser bundle import ./bundle.zst
|
|
|
|
# Import with dry run
|
|
stella feedser bundle import ./bundle.zst --dry-run
|
|
|
|
# Import from stdin (for pipes)
|
|
cat bundle.zst | stella feedser bundle import -
|
|
|
|
# Import without signature verification (testing only)
|
|
stella feedser bundle import ./bundle.zst --skip-signature
|
|
|
|
# Force import (override cursor check)
|
|
stella feedser bundle import ./bundle.zst --force
|
|
```
|
|
|
|
### Conflict Resolution
|
|
|
|
When conflicts occur between local and remote values:
|
|
|
|
| Strategy | Behavior |
|
|
|----------|----------|
|
|
| `prefer_remote` | Remote value wins (default) |
|
|
| `prefer_local` | Local value preserved |
|
|
| `fail` | Abort import on first conflict |
|
|
|
|
Conflicts are logged with full details:
|
|
|
|
```json
|
|
{
|
|
"merge_hash": "sha256:abc...",
|
|
"field": "severity",
|
|
"local_value": "high",
|
|
"remote_value": "critical",
|
|
"resolution": "prefer_remote"
|
|
}
|
|
```
|
|
|
|
## Site Management
|
|
|
|
### List Sites
|
|
|
|
```bash
|
|
stella feedser sites list
|
|
```
|
|
|
|
Output:
|
|
```
|
|
SITE ID STATUS LAST SYNC CURSOR
|
|
───────────────────────── ──────── ─────────────────── ──────────────────────────
|
|
site-us-east-1 enabled 2025-01-15 10:30 2025-01-15T10:30:00Z#0042
|
|
site-eu-central-1 enabled 2025-01-15 09:15 2025-01-15T09:15:00Z#0038
|
|
site-asia-pacific-1 disabled never -
|
|
```
|
|
|
|
### View Site History
|
|
|
|
```bash
|
|
stella feedser sites history site-us-east-1 --limit 10
|
|
```
|
|
|
|
### Update Site Policy
|
|
|
|
```bash
|
|
stella feedser sites policy site-us-east-1 \
|
|
--enabled false # Disable imports from this site
|
|
```
|
|
|
|
## Air-Gap Operations
|
|
|
|
For sites without network connectivity:
|
|
|
|
### Export for Transfer
|
|
|
|
```bash
|
|
# On connected site
|
|
stella feedser bundle export \
|
|
-c "2025-01-14T00:00:00Z#0000" \
|
|
-o ./delta-2025-01-15.zst
|
|
|
|
# Transfer via USB/secure media
|
|
```
|
|
|
|
### Import on Air-Gap Site
|
|
|
|
```bash
|
|
# On air-gapped site
|
|
stella feedser bundle import ./delta-2025-01-15.zst
|
|
|
|
# Verify import
|
|
stella feedser sites list
|
|
```
|
|
|
|
### Full Sync Workflow
|
|
|
|
1. **Initial Sync:**
|
|
```bash
|
|
# Export full dataset
|
|
stella feedser bundle export -o ./full-sync.zst
|
|
```
|
|
|
|
2. **Transfer to air-gap site**
|
|
|
|
3. **Import on air-gap:**
|
|
```bash
|
|
stella feedser bundle import ./full-sync.zst
|
|
```
|
|
|
|
4. **Subsequent Delta Syncs:**
|
|
```bash
|
|
# Get current cursor from air-gap site
|
|
stella feedser sites list # Note the cursor
|
|
|
|
# On connected site, export delta
|
|
stella feedser bundle export -c "{cursor}" -o ./delta.zst
|
|
|
|
# Transfer and import on air-gap
|
|
```
|
|
|
|
## Verification
|
|
|
|
### Validate Bundle Without Import
|
|
|
|
```bash
|
|
stella feedser bundle validate ./bundle.zst
|
|
```
|
|
|
|
Output:
|
|
```
|
|
Bundle: bundle.zst
|
|
Version: feedser-bundle/1.0
|
|
Site: site-us-east-1
|
|
Cursor: 2025-01-15T10:30:00.000Z#0042
|
|
|
|
Counts:
|
|
Canonicals: 1,234
|
|
Edges: 3,456
|
|
Deletions: 12
|
|
Total: 4,702
|
|
|
|
Verification:
|
|
Hash: ✓ Valid
|
|
Signature: ✓ Valid (key: sha256:abc...)
|
|
Format: ✓ Valid
|
|
|
|
Ready for import.
|
|
```
|
|
|
|
### Preview Import Impact
|
|
|
|
```bash
|
|
stella feedser bundle import ./bundle.zst --dry-run --json
|
|
```
|
|
|
|
## Monitoring
|
|
|
|
### Sync Status Endpoint
|
|
|
|
```
|
|
GET /api/v1/federation/sync/status
|
|
```
|
|
|
|
Response:
|
|
```json
|
|
{
|
|
"sites": [
|
|
{
|
|
"site_id": "site-us-east-1",
|
|
"enabled": true,
|
|
"last_sync_at": "2025-01-15T10:30:00Z",
|
|
"last_cursor": "2025-01-15T10:30:00.000Z#0042",
|
|
"bundles_imported": 156,
|
|
"total_items_imported": 45678
|
|
}
|
|
],
|
|
"local_cursor": "2025-01-15T10:35:00.000Z#0044"
|
|
}
|
|
```
|
|
|
|
### Event Stream
|
|
|
|
Import events are published to the `canonical-imported` stream:
|
|
|
|
```json
|
|
{
|
|
"canonical_id": "uuid",
|
|
"cve": "CVE-2024-1234",
|
|
"affects_key": "pkg:npm/express@4.0.0",
|
|
"merge_hash": "sha256:...",
|
|
"action": "Created",
|
|
"bundle_hash": "sha256:...",
|
|
"site_id": "site-us-east-1",
|
|
"imported_at": "2025-01-15T10:30:15Z"
|
|
}
|
|
```
|
|
|
|
### Cache Invalidation
|
|
|
|
After import, cache indexes are automatically updated:
|
|
- PURL index updated for affected packages
|
|
- CVE index updated for vulnerability lookups
|
|
- Existing cache entries invalidated for refresh
|
|
|
|
## Troubleshooting
|
|
|
|
### Common Issues
|
|
|
|
| Issue | Cause | Solution |
|
|
|-------|-------|----------|
|
|
| "Cursor not after current" | Bundle is stale | Use `--force` or export newer bundle |
|
|
| "Signature verification failed" | Key mismatch | Verify signing key is trusted |
|
|
| "Site not allowed" | Policy restriction | Add site to `AllowedSites` config |
|
|
| "Bundle too large" | Size limit exceeded | Increase `MaxBundleSizeBytes` or export smaller delta |
|
|
|
|
### Debug Logging
|
|
|
|
Enable verbose logging for federation operations:
|
|
|
|
```yaml
|
|
Logging:
|
|
LogLevel:
|
|
StellaOps.Concelier.Federation: Debug
|
|
```
|
|
|
|
### Verify Sync State
|
|
|
|
```bash
|
|
# Check local vs remote cursor
|
|
stella feedser sites status site-us-east-1
|
|
|
|
# List recent imports
|
|
stella feedser sites history site-us-east-1 --limit 5
|
|
|
|
# Verify specific canonical was imported
|
|
stella feedser canonical get sha256:mergehash...
|
|
```
|
|
|
|
## Best Practices
|
|
|
|
1. **Regular Sync Schedule:** Configure automated delta exports/imports on a schedule (e.g., hourly)
|
|
|
|
2. **Monitor Cursor Drift:** Alert if cursor falls too far behind
|
|
|
|
3. **Verify Signatures:** Only disable signature verification in development
|
|
|
|
4. **Size Bundles Appropriately:** For large deltas, split into multiple bundles
|
|
|
|
5. **Test Import Before Production:** Use `--dry-run` to validate bundles
|
|
|
|
6. **Maintain Key Trust:** Regularly rotate and verify federation signing keys
|
|
|
|
7. **Document Site Policies:** Keep a registry of trusted sites and their policies
|