feat: add bulk triage view component and related stories
- Exported BulkTriageViewComponent and its related types from findings module. - Created a new accessibility test suite for score components using axe-core. - Introduced design tokens for score components to standardize styling. - Enhanced score breakdown popover for mobile responsiveness with drag handle. - Added date range selector functionality to score history chart component. - Implemented unit tests for date range selector in score history chart. - Created Storybook stories for bulk triage view and score history chart with date range selector.
This commit is contained in:
StellaOps Bot
386
docs/modules/concelier/federation-operations.md
Normal file
386
docs/modules/concelier/federation-operations.md
Normal file
@@ -0,0 +1,386 @@
|
||||
# Federation Setup and Operations
|
||||
|
||||
Per SPRINT_8200_0014_0003.
|
||||
|
||||
## Overview
|
||||
|
||||
Federation enables multi-site synchronization of canonical advisory data between Concelier instances. Sites can export bundles containing delta changes and import bundles from other sites to maintain synchronized vulnerability intelligence.
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────────────────┐
|
||||
│ Federation Topology │
|
||||
├─────────────────────────────────────────────────────────────────────────────┤
|
||||
│ │
|
||||
│ ┌──────────────────┐ ┌──────────────────┐ │
|
||||
│ │ Site A (HQ) │ │ Site B (Branch) │ │
|
||||
│ │ │ │ │ │
|
||||
│ │ ┌────────────┐ │ Export │ ┌────────────┐ │ │
|
||||
│ │ │ Concelier │──┼──────────►│ │ Concelier │ │ │
|
||||
│ │ │ │ │ Bundle │ │ │ │ │
|
||||
│ │ └────────────┘ │ │ └────────────┘ │ │
|
||||
│ │ │ │ │ │ │ │
|
||||
│ │ ▼ │ │ ▼ │ │
|
||||
│ │ ┌────────────┐ │ │ ┌────────────┐ │ │
|
||||
│ │ │ PostgreSQL │ │ │ │ PostgreSQL │ │ │
|
||||
│ │ └────────────┘ │ │ └────────────┘ │ │
|
||||
│ └──────────────────┘ └──────────────────┘ │
|
||||
│ │
|
||||
│ ┌──────────────────┐ │
|
||||
│ │ Site C (Air-Gap)│ │
|
||||
│ │ │ │
|
||||
│ │ ┌────────────┐ │ USB/Secure │
|
||||
│ │ │ Concelier │◄─┼───Transfer │
|
||||
│ │ │ │ │ │
|
||||
│ │ └────────────┘ │ │
|
||||
│ └──────────────────┘ │
|
||||
│ │
|
||||
└─────────────────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
## Setup
|
||||
|
||||
### 1. Enable Federation
|
||||
|
||||
Configure federation in `concelier.yaml`:
|
||||
|
||||
```yaml
|
||||
Federation:
|
||||
Enabled: true
|
||||
SiteId: "site-us-west-1" # Unique identifier for this site
|
||||
DefaultCompressionLevel: 3
|
||||
DefaultMaxItems: 10000
|
||||
RequireSignature: true
|
||||
|
||||
FederationImport:
|
||||
AllowedSites:
|
||||
- "site-us-east-1"
|
||||
- "site-eu-central-1"
|
||||
MaxBundleSizeBytes: 104857600 # 100 MB
|
||||
SkipSignatureOnTrustedSites: false
|
||||
```
|
||||
|
||||
### 2. Configure Site Policies
|
||||
|
||||
Create site policies for each trusted federation partner:
|
||||
|
||||
```bash
|
||||
# Add trusted site
|
||||
stella feedser sites add site-us-east-1 \
|
||||
--display-name "US East Production" \
|
||||
--enabled
|
||||
|
||||
# Configure policy
|
||||
stella feedser sites policy site-us-east-1 \
|
||||
--max-bundle-size 100MB \
|
||||
--allowed-sources nvd,ghsa,debian
|
||||
```
|
||||
|
||||
### 3. Generate Signing Keys
|
||||
|
||||
For signed bundles, configure Authority keys:
|
||||
|
||||
```bash
|
||||
# Generate federation signing key
|
||||
stella authority keys generate \
|
||||
--name federation-signer \
|
||||
--algorithm ES256 \
|
||||
--purpose federation
|
||||
|
||||
# Export public key for distribution
|
||||
stella authority keys export federation-signer --public
|
||||
```
|
||||
|
||||
## Import Operations
|
||||
|
||||
### API Import
|
||||
|
||||
```
|
||||
POST /api/v1/federation/import
|
||||
Content-Type: application/zstd
|
||||
```
|
||||
|
||||
**Query Parameters:**
|
||||
|
||||
| Parameter | Type | Default | Description |
|
||||
|-----------|------|---------|-------------|
|
||||
| `dry_run` | bool | false | Validate without importing |
|
||||
| `skip_signature` | bool | false | Skip signature verification (requires trust) |
|
||||
| `on_conflict` | enum | prefer_remote | `prefer_remote`, `prefer_local`, `fail` |
|
||||
| `force` | bool | false | Import even if cursor is not after current |
|
||||
|
||||
**Response:**
|
||||
|
||||
```json
|
||||
{
|
||||
"success": true,
|
||||
"bundle_hash": "sha256:a1b2c3...",
|
||||
"imported_cursor": "2025-01-15T10:30:00.000Z#0042",
|
||||
"counts": {
|
||||
"canonical_created": 100,
|
||||
"canonical_updated": 25,
|
||||
"canonical_skipped": 10,
|
||||
"edges_added": 200,
|
||||
"deletions_processed": 5
|
||||
},
|
||||
"conflicts": [],
|
||||
"duration_ms": 1234
|
||||
}
|
||||
```
|
||||
|
||||
### CLI Import
|
||||
|
||||
```bash
|
||||
# Import from file
|
||||
stella feedser bundle import ./bundle.zst
|
||||
|
||||
# Import with dry run
|
||||
stella feedser bundle import ./bundle.zst --dry-run
|
||||
|
||||
# Import from stdin (for pipes)
|
||||
cat bundle.zst | stella feedser bundle import -
|
||||
|
||||
# Import without signature verification (testing only)
|
||||
stella feedser bundle import ./bundle.zst --skip-signature
|
||||
|
||||
# Force import (override cursor check)
|
||||
stella feedser bundle import ./bundle.zst --force
|
||||
```
|
||||
|
||||
### Conflict Resolution
|
||||
|
||||
When conflicts occur between local and remote values:
|
||||
|
||||
| Strategy | Behavior |
|
||||
|----------|----------|
|
||||
| `prefer_remote` | Remote value wins (default) |
|
||||
| `prefer_local` | Local value preserved |
|
||||
| `fail` | Abort import on first conflict |
|
||||
|
||||
Conflicts are logged with full details:
|
||||
|
||||
```json
|
||||
{
|
||||
"merge_hash": "sha256:abc...",
|
||||
"field": "severity",
|
||||
"local_value": "high",
|
||||
"remote_value": "critical",
|
||||
"resolution": "prefer_remote"
|
||||
}
|
||||
```
|
||||
|
||||
## Site Management
|
||||
|
||||
### List Sites
|
||||
|
||||
```bash
|
||||
stella feedser sites list
|
||||
```
|
||||
|
||||
Output:
|
||||
```
|
||||
SITE ID STATUS LAST SYNC CURSOR
|
||||
───────────────────────── ──────── ─────────────────── ──────────────────────────
|
||||
site-us-east-1 enabled 2025-01-15 10:30 2025-01-15T10:30:00Z#0042
|
||||
site-eu-central-1 enabled 2025-01-15 09:15 2025-01-15T09:15:00Z#0038
|
||||
site-asia-pacific-1 disabled never -
|
||||
```
|
||||
|
||||
### View Site History
|
||||
|
||||
```bash
|
||||
stella feedser sites history site-us-east-1 --limit 10
|
||||
```
|
||||
|
||||
### Update Site Policy
|
||||
|
||||
```bash
|
||||
stella feedser sites policy site-us-east-1 \
|
||||
--enabled false # Disable imports from this site
|
||||
```
|
||||
|
||||
## Air-Gap Operations
|
||||
|
||||
For sites without network connectivity:
|
||||
|
||||
### Export for Transfer
|
||||
|
||||
```bash
|
||||
# On connected site
|
||||
stella feedser bundle export \
|
||||
-c "2025-01-14T00:00:00Z#0000" \
|
||||
-o ./delta-2025-01-15.zst
|
||||
|
||||
# Transfer via USB/secure media
|
||||
```
|
||||
|
||||
### Import on Air-Gap Site
|
||||
|
||||
```bash
|
||||
# On air-gapped site
|
||||
stella feedser bundle import ./delta-2025-01-15.zst
|
||||
|
||||
# Verify import
|
||||
stella feedser sites list
|
||||
```
|
||||
|
||||
### Full Sync Workflow
|
||||
|
||||
1. **Initial Sync:**
|
||||
```bash
|
||||
# Export full dataset
|
||||
stella feedser bundle export -o ./full-sync.zst
|
||||
```
|
||||
|
||||
2. **Transfer to air-gap site**
|
||||
|
||||
3. **Import on air-gap:**
|
||||
```bash
|
||||
stella feedser bundle import ./full-sync.zst
|
||||
```
|
||||
|
||||
4. **Subsequent Delta Syncs:**
|
||||
```bash
|
||||
# Get current cursor from air-gap site
|
||||
stella feedser sites list # Note the cursor
|
||||
|
||||
# On connected site, export delta
|
||||
stella feedser bundle export -c "{cursor}" -o ./delta.zst
|
||||
|
||||
# Transfer and import on air-gap
|
||||
```
|
||||
|
||||
## Verification
|
||||
|
||||
### Validate Bundle Without Import
|
||||
|
||||
```bash
|
||||
stella feedser bundle validate ./bundle.zst
|
||||
```
|
||||
|
||||
Output:
|
||||
```
|
||||
Bundle: bundle.zst
|
||||
Version: feedser-bundle/1.0
|
||||
Site: site-us-east-1
|
||||
Cursor: 2025-01-15T10:30:00.000Z#0042
|
||||
|
||||
Counts:
|
||||
Canonicals: 1,234
|
||||
Edges: 3,456
|
||||
Deletions: 12
|
||||
Total: 4,702
|
||||
|
||||
Verification:
|
||||
Hash: ✓ Valid
|
||||
Signature: ✓ Valid (key: sha256:abc...)
|
||||
Format: ✓ Valid
|
||||
|
||||
Ready for import.
|
||||
```
|
||||
|
||||
### Preview Import Impact
|
||||
|
||||
```bash
|
||||
stella feedser bundle import ./bundle.zst --dry-run --json
|
||||
```
|
||||
|
||||
## Monitoring
|
||||
|
||||
### Sync Status Endpoint
|
||||
|
||||
```
|
||||
GET /api/v1/federation/sync/status
|
||||
```
|
||||
|
||||
Response:
|
||||
```json
|
||||
{
|
||||
"sites": [
|
||||
{
|
||||
"site_id": "site-us-east-1",
|
||||
"enabled": true,
|
||||
"last_sync_at": "2025-01-15T10:30:00Z",
|
||||
"last_cursor": "2025-01-15T10:30:00.000Z#0042",
|
||||
"bundles_imported": 156,
|
||||
"total_items_imported": 45678
|
||||
}
|
||||
],
|
||||
"local_cursor": "2025-01-15T10:35:00.000Z#0044"
|
||||
}
|
||||
```
|
||||
|
||||
### Event Stream
|
||||
|
||||
Import events are published to the `canonical-imported` stream:
|
||||
|
||||
```json
|
||||
{
|
||||
"canonical_id": "uuid",
|
||||
"cve": "CVE-2024-1234",
|
||||
"affects_key": "pkg:npm/express@4.0.0",
|
||||
"merge_hash": "sha256:...",
|
||||
"action": "Created",
|
||||
"bundle_hash": "sha256:...",
|
||||
"site_id": "site-us-east-1",
|
||||
"imported_at": "2025-01-15T10:30:15Z"
|
||||
}
|
||||
```
|
||||
|
||||
### Cache Invalidation
|
||||
|
||||
After import, cache indexes are automatically updated:
|
||||
- PURL index updated for affected packages
|
||||
- CVE index updated for vulnerability lookups
|
||||
- Existing cache entries invalidated for refresh
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Common Issues
|
||||
|
||||
| Issue | Cause | Solution |
|
||||
|-------|-------|----------|
|
||||
| "Cursor not after current" | Bundle is stale | Use `--force` or export newer bundle |
|
||||
| "Signature verification failed" | Key mismatch | Verify signing key is trusted |
|
||||
| "Site not allowed" | Policy restriction | Add site to `AllowedSites` config |
|
||||
| "Bundle too large" | Size limit exceeded | Increase `MaxBundleSizeBytes` or export smaller delta |
|
||||
|
||||
### Debug Logging
|
||||
|
||||
Enable verbose logging for federation operations:
|
||||
|
||||
```yaml
|
||||
Logging:
|
||||
LogLevel:
|
||||
StellaOps.Concelier.Federation: Debug
|
||||
```
|
||||
|
||||
### Verify Sync State
|
||||
|
||||
```bash
|
||||
# Check local vs remote cursor
|
||||
stella feedser sites status site-us-east-1
|
||||
|
||||
# List recent imports
|
||||
stella feedser sites history site-us-east-1 --limit 5
|
||||
|
||||
# Verify specific canonical was imported
|
||||
stella feedser canonical get sha256:mergehash...
|
||||
```
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. **Regular Sync Schedule:** Configure automated delta exports/imports on a schedule (e.g., hourly)
|
||||
|
||||
2. **Monitor Cursor Drift:** Alert if cursor falls too far behind
|
||||
|
||||
3. **Verify Signatures:** Only disable signature verification in development
|
||||
|
||||
4. **Size Bundles Appropriately:** For large deltas, split into multiple bundles
|
||||
|
||||
5. **Test Import Before Production:** Use `--dry-run` to validate bundles
|
||||
|
||||
6. **Maintain Key Trust:** Regularly rotate and verify federation signing keys
|
||||
|
||||
7. **Document Site Policies:** Keep a registry of trusted sites and their policies
|
||||
Reference in New Issue
Block a user