feat: add bulk triage view component and related stories
- Exported BulkTriageViewComponent and its related types from findings module. - Created a new accessibility test suite for score components using axe-core. - Introduced design tokens for score components to standardize styling. - Enhanced score breakdown popover for mobile responsiveness with drag handle. - Added date range selector functionality to score history chart component. - Implemented unit tests for date range selector in score history chart. - Created Storybook stories for bulk triage view and score history chart with date range selector.
This commit is contained in:
StellaOps Bot
332
docs/modules/concelier/federation-setup.md
Normal file
332
docs/modules/concelier/federation-setup.md
Normal file
@@ -0,0 +1,332 @@
|
||||
# Federation Setup and Operations Guide
|
||||
|
||||
This guide covers the setup and operation of StellaOps federation for multi-site vulnerability data synchronization.
|
||||
|
||||
## Overview
|
||||
|
||||
Federation enables secure, cursor-based synchronization of canonical vulnerability advisories between StellaOps sites. It supports:
|
||||
|
||||
- **Delta exports**: Only changed records since the last cursor are included
|
||||
- **Air-gap transfers**: Bundles can be written to files for offline transfer
|
||||
- **Multi-site topology**: Multiple sites can synchronize independently
|
||||
- **Cryptographic verification**: DSSE signatures ensure bundle authenticity
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
┌─────────────┐ ┌─────────────┐ ┌─────────────┐
|
||||
│ Site A │────▶│ Bundle │────▶│ Site B │
|
||||
│ (Export) │ │ (.zst) │ │ (Import) │
|
||||
└─────────────┘ └─────────────┘ └─────────────┘
|
||||
│
|
||||
▼
|
||||
┌───────────┐
|
||||
│ Site C │
|
||||
│ (Import) │
|
||||
└───────────┘
|
||||
```
|
||||
|
||||
## Bundle Format
|
||||
|
||||
Federation bundles are ZST-compressed TAR archives containing:
|
||||
|
||||
| File | Description |
|
||||
|------|-------------|
|
||||
| `MANIFEST.json` | Bundle metadata, cursor, counts, hash |
|
||||
| `canonicals.ndjson` | Canonical advisories (one per line) |
|
||||
| `edges.ndjson` | Source edges linking advisories to sources |
|
||||
| `deletions.ndjson` | Withdrawn/deleted advisory IDs |
|
||||
| `SIGNATURE.json` | Optional DSSE signature envelope |
|
||||
|
||||
## Configuration
|
||||
|
||||
### Export Site Configuration
|
||||
|
||||
```yaml
|
||||
# concelier.yaml
|
||||
federation:
|
||||
enabled: true
|
||||
site_id: "us-west-1" # Unique site identifier
|
||||
export:
|
||||
enabled: true
|
||||
default_compression_level: 3 # ZST level (1-19)
|
||||
sign_bundles: true # Sign exported bundles
|
||||
max_items_per_bundle: 10000 # Maximum items per export
|
||||
```
|
||||
|
||||
### Import Site Configuration
|
||||
|
||||
```yaml
|
||||
# concelier.yaml
|
||||
federation:
|
||||
enabled: true
|
||||
site_id: "eu-central-1"
|
||||
import:
|
||||
enabled: true
|
||||
skip_signature_verification: false # NEVER set true in production
|
||||
allowed_sites: # Trusted site IDs
|
||||
- "us-west-1"
|
||||
- "ap-south-1"
|
||||
conflict_resolution: "prefer_remote" # prefer_remote | prefer_local | fail
|
||||
force_cursor_validation: true # Reject out-of-order imports
|
||||
```
|
||||
|
||||
## API Endpoints
|
||||
|
||||
### Export Endpoints
|
||||
|
||||
```bash
|
||||
# Export delta bundle since cursor
|
||||
GET /api/v1/federation/export?since_cursor={cursor}
|
||||
|
||||
# Preview export (counts only)
|
||||
GET /api/v1/federation/export/preview?since_cursor={cursor}
|
||||
|
||||
# Get federation status
|
||||
GET /api/v1/federation/status
|
||||
```
|
||||
|
||||
### Import Endpoints
|
||||
|
||||
```bash
|
||||
# Import bundle
|
||||
POST /api/v1/federation/import
|
||||
Content-Type: application/zstd
|
||||
|
||||
# Validate bundle without importing
|
||||
POST /api/v1/federation/validate
|
||||
Content-Type: application/zstd
|
||||
|
||||
# List federated sites
|
||||
GET /api/v1/federation/sites
|
||||
|
||||
# Update site policy
|
||||
PUT /api/v1/federation/sites/{site_id}/policy
|
||||
```
|
||||
|
||||
## CLI Commands
|
||||
|
||||
### Export Operations
|
||||
|
||||
```bash
|
||||
# Export full bundle (no cursor = all data)
|
||||
feedser bundle export --output bundle.zst
|
||||
|
||||
# Export delta since last cursor
|
||||
feedser bundle export --since-cursor "2025-01-15T10:00:00Z#0001" --output delta.zst
|
||||
|
||||
# Preview export without creating bundle
|
||||
feedser bundle preview --since-cursor "2025-01-15T10:00:00Z#0001"
|
||||
|
||||
# Export without signing (testing only)
|
||||
feedser bundle export --no-sign --output unsigned.zst
|
||||
```
|
||||
|
||||
### Import Operations
|
||||
|
||||
```bash
|
||||
# Import bundle
|
||||
feedser bundle import bundle.zst
|
||||
|
||||
# Dry run (validate without importing)
|
||||
feedser bundle import bundle.zst --dry-run
|
||||
|
||||
# Import from stdin (pipe)
|
||||
cat bundle.zst | feedser bundle import -
|
||||
|
||||
# Force import (skip cursor validation)
|
||||
feedser bundle import bundle.zst --force
|
||||
```
|
||||
|
||||
### Site Management
|
||||
|
||||
```bash
|
||||
# List federated sites
|
||||
feedser sites list
|
||||
|
||||
# Show site details
|
||||
feedser sites show us-west-1
|
||||
|
||||
# Enable/disable site
|
||||
feedser sites enable ap-south-1
|
||||
feedser sites disable ap-south-1
|
||||
```
|
||||
|
||||
## Cursor Format
|
||||
|
||||
Cursors use ISO-8601 timestamp with sequence number:
|
||||
|
||||
```
|
||||
{ISO-8601 timestamp}#{sequence number}
|
||||
|
||||
Examples:
|
||||
2025-01-15T10:00:00.000Z#0001
|
||||
2025-01-15T10:00:00.000Z#0002
|
||||
```
|
||||
|
||||
- Cursors are site-specific (each site maintains independent cursors)
|
||||
- Sequence numbers distinguish concurrent exports
|
||||
- Cursors are monotonically increasing within a site
|
||||
|
||||
## Air-Gap Transfer Workflow
|
||||
|
||||
For environments without network connectivity:
|
||||
|
||||
```bash
|
||||
# On Source Site (connected to authority)
|
||||
feedser bundle export --since-cursor "$LAST_CURSOR" --output /media/usb/bundle.zst
|
||||
feedser bundle preview --since-cursor "$LAST_CURSOR" > /media/usb/manifest.txt
|
||||
|
||||
# Transfer media to target site...
|
||||
|
||||
# On Target Site (air-gapped)
|
||||
feedser bundle import /media/usb/bundle.zst --dry-run # Validate first
|
||||
feedser bundle import /media/usb/bundle.zst # Import
|
||||
```
|
||||
|
||||
## Multi-Site Synchronization
|
||||
|
||||
### Hub-and-Spoke Topology
|
||||
|
||||
```
|
||||
┌─────────────┐
|
||||
│ Hub Site │
|
||||
│ (Primary) │
|
||||
└──────┬──────┘
|
||||
│
|
||||
┌──────────┼──────────┐
|
||||
▼ ▼ ▼
|
||||
┌──────────┐ ┌──────────┐ ┌──────────┐
|
||||
│ Site A │ │ Site B │ │ Site C │
|
||||
│ (Spoke) │ │ (Spoke) │ │ (Spoke) │
|
||||
└──────────┘ └──────────┘ └──────────┘
|
||||
```
|
||||
|
||||
### Mesh Topology
|
||||
|
||||
Each site can import from multiple sources:
|
||||
|
||||
```yaml
|
||||
federation:
|
||||
import:
|
||||
allowed_sites:
|
||||
- "hub-primary"
|
||||
- "hub-secondary" # Redundancy
|
||||
```
|
||||
|
||||
## Merge Behavior
|
||||
|
||||
### Conflict Resolution
|
||||
|
||||
When importing, conflicts are resolved based on configuration:
|
||||
|
||||
| Strategy | Behavior |
|
||||
|----------|----------|
|
||||
| `prefer_remote` | Remote (bundle) value wins (default) |
|
||||
| `prefer_local` | Local value preserved |
|
||||
| `fail` | Import aborts on any conflict |
|
||||
|
||||
### Merge Actions
|
||||
|
||||
| Action | Description |
|
||||
|--------|-------------|
|
||||
| `Created` | New canonical added |
|
||||
| `Updated` | Existing canonical updated |
|
||||
| `Skipped` | No change needed (identical) |
|
||||
|
||||
## Verification
|
||||
|
||||
### Hash Verification
|
||||
|
||||
Bundle hash is computed over compressed content:
|
||||
|
||||
```
|
||||
SHA256(compressed bundle content)
|
||||
```
|
||||
|
||||
### Signature Verification
|
||||
|
||||
DSSE envelope contains:
|
||||
|
||||
```json
|
||||
{
|
||||
"payloadType": "application/stellaops.federation.bundle+json",
|
||||
"payload": "base64(bundle_hash + site_id + cursor)",
|
||||
"signatures": [
|
||||
{
|
||||
"keyId": "signing-key-001",
|
||||
"algorithm": "ES256",
|
||||
"signature": "base64(signature)"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
## Monitoring
|
||||
|
||||
### Key Metrics
|
||||
|
||||
- `federation_export_duration_seconds` - Export time
|
||||
- `federation_import_duration_seconds` - Import time
|
||||
- `federation_bundle_size_bytes` - Bundle sizes
|
||||
- `federation_items_processed_total` - Items processed by type
|
||||
- `federation_conflicts_total` - Merge conflicts encountered
|
||||
|
||||
### Health Checks
|
||||
|
||||
```bash
|
||||
# Check federation status
|
||||
curl http://localhost:5000/api/v1/federation/status
|
||||
|
||||
# Response
|
||||
{
|
||||
"site_id": "us-west-1",
|
||||
"export_enabled": true,
|
||||
"import_enabled": true,
|
||||
"last_export": "2025-01-15T10:00:00Z",
|
||||
"last_import": "2025-01-15T09:30:00Z",
|
||||
"sites_synced": 2
|
||||
}
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Common Issues
|
||||
|
||||
**Import fails with "cursor validation failed"**
|
||||
- Bundle cursor is not after current site cursor
|
||||
- Use `--force` to override (not recommended)
|
||||
- Check if bundle was already imported
|
||||
|
||||
**Signature verification failed**
|
||||
- Signing key not trusted on target site
|
||||
- Key expired or revoked
|
||||
- Use `--skip-signature` for testing only
|
||||
|
||||
**Large bundle timeout**
|
||||
- Increase `federation.export.timeout`
|
||||
- Use smaller `max_items_per_bundle`
|
||||
- Stream directly to file
|
||||
|
||||
### Debug Logging
|
||||
|
||||
```yaml
|
||||
logging:
|
||||
level:
|
||||
StellaOps.Concelier.Federation: Debug
|
||||
```
|
||||
|
||||
## Security Considerations
|
||||
|
||||
1. **Never skip signature verification in production**
|
||||
2. **Validate allowed_sites whitelist**
|
||||
3. **Use TLS for API endpoints**
|
||||
4. **Rotate signing keys periodically**
|
||||
5. **Audit import events**
|
||||
6. **Monitor for duplicate bundle imports**
|
||||
|
||||
## Related Documentation
|
||||
|
||||
- [Bundle Export Format](federation-bundle-export.md)
|
||||
- [Sync Ledger Schema](../db/sync-ledger.md)
|
||||
- [Signing Configuration](../security/signing.md)
|
||||
Reference in New Issue
Block a user