Files
git.stella-ops.org/docs/modules/concelier/federation-operations.md
StellaOps Bot 17613acf57 feat: add bulk triage view component and related stories
- Exported BulkTriageViewComponent and its related types from findings module.
- Created a new accessibility test suite for score components using axe-core.
- Introduced design tokens for score components to standardize styling.
- Enhanced score breakdown popover for mobile responsiveness with drag handle.
- Added date range selector functionality to score history chart component.
- Implemented unit tests for date range selector in score history chart.
- Created Storybook stories for bulk triage view and score history chart with date range selector.
2025-12-26 01:01:35 +02:00

11 KiB

Federation Setup and Operations

Per SPRINT_8200_0014_0003.

Overview

Federation enables multi-site synchronization of canonical advisory data between Concelier instances. Sites can export bundles containing delta changes and import bundles from other sites to maintain synchronized vulnerability intelligence.

Architecture

┌─────────────────────────────────────────────────────────────────────────────┐
│                            Federation Topology                              │
├─────────────────────────────────────────────────────────────────────────────┤
│                                                                             │
│    ┌──────────────────┐           ┌──────────────────┐                     │
│    │  Site A (HQ)     │           │  Site B (Branch) │                     │
│    │                  │           │                  │                     │
│    │  ┌────────────┐  │  Export   │  ┌────────────┐  │                     │
│    │  │ Concelier  │──┼──────────►│  │ Concelier  │  │                     │
│    │  │            │  │  Bundle   │  │            │  │                     │
│    │  └────────────┘  │           │  └────────────┘  │                     │
│    │        │         │           │        │         │                     │
│    │        ▼         │           │        ▼         │                     │
│    │  ┌────────────┐  │           │  ┌────────────┐  │                     │
│    │  │ PostgreSQL │  │           │  │ PostgreSQL │  │                     │
│    │  └────────────┘  │           │  └────────────┘  │                     │
│    └──────────────────┘           └──────────────────┘                     │
│                                                                             │
│    ┌──────────────────┐                                                     │
│    │  Site C (Air-Gap)│                                                     │
│    │                  │                                                     │
│    │  ┌────────────┐  │   USB/Secure                                       │
│    │  │ Concelier  │◄─┼───Transfer                                         │
│    │  │            │  │                                                     │
│    │  └────────────┘  │                                                     │
│    └──────────────────┘                                                     │
│                                                                             │
└─────────────────────────────────────────────────────────────────────────────┘

Setup

1. Enable Federation

Configure federation in concelier.yaml:

Federation:
  Enabled: true
  SiteId: "site-us-west-1"  # Unique identifier for this site
  DefaultCompressionLevel: 3
  DefaultMaxItems: 10000
  RequireSignature: true

FederationImport:
  AllowedSites:
    - "site-us-east-1"
    - "site-eu-central-1"
  MaxBundleSizeBytes: 104857600  # 100 MB
  SkipSignatureOnTrustedSites: false

2. Configure Site Policies

Create site policies for each trusted federation partner:

# Add trusted site
stella feedser sites add site-us-east-1 \
  --display-name "US East Production" \
  --enabled

# Configure policy
stella feedser sites policy site-us-east-1 \
  --max-bundle-size 100MB \
  --allowed-sources nvd,ghsa,debian

3. Generate Signing Keys

For signed bundles, configure Authority keys:

# Generate federation signing key
stella authority keys generate \
  --name federation-signer \
  --algorithm ES256 \
  --purpose federation

# Export public key for distribution
stella authority keys export federation-signer --public

Import Operations

API Import

POST /api/v1/federation/import
Content-Type: application/zstd

Query Parameters:

Parameter Type Default Description
dry_run bool false Validate without importing
skip_signature bool false Skip signature verification (requires trust)
on_conflict enum prefer_remote prefer_remote, prefer_local, fail
force bool false Import even if cursor is not after current

Response:

{
  "success": true,
  "bundle_hash": "sha256:a1b2c3...",
  "imported_cursor": "2025-01-15T10:30:00.000Z#0042",
  "counts": {
    "canonical_created": 100,
    "canonical_updated": 25,
    "canonical_skipped": 10,
    "edges_added": 200,
    "deletions_processed": 5
  },
  "conflicts": [],
  "duration_ms": 1234
}

CLI Import

# Import from file
stella feedser bundle import ./bundle.zst

# Import with dry run
stella feedser bundle import ./bundle.zst --dry-run

# Import from stdin (for pipes)
cat bundle.zst | stella feedser bundle import -

# Import without signature verification (testing only)
stella feedser bundle import ./bundle.zst --skip-signature

# Force import (override cursor check)
stella feedser bundle import ./bundle.zst --force

Conflict Resolution

When conflicts occur between local and remote values:

Strategy Behavior
prefer_remote Remote value wins (default)
prefer_local Local value preserved
fail Abort import on first conflict

Conflicts are logged with full details:

{
  "merge_hash": "sha256:abc...",
  "field": "severity",
  "local_value": "high",
  "remote_value": "critical",
  "resolution": "prefer_remote"
}

Site Management

List Sites

stella feedser sites list

Output:

SITE ID                    STATUS    LAST SYNC            CURSOR
─────────────────────────  ────────  ───────────────────  ──────────────────────────
site-us-east-1             enabled   2025-01-15 10:30     2025-01-15T10:30:00Z#0042
site-eu-central-1          enabled   2025-01-15 09:15     2025-01-15T09:15:00Z#0038
site-asia-pacific-1        disabled  never                -

View Site History

stella feedser sites history site-us-east-1 --limit 10

Update Site Policy

stella feedser sites policy site-us-east-1 \
  --enabled false  # Disable imports from this site

Air-Gap Operations

For sites without network connectivity:

Export for Transfer

# On connected site
stella feedser bundle export \
  -c "2025-01-14T00:00:00Z#0000" \
  -o ./delta-2025-01-15.zst

# Transfer via USB/secure media

Import on Air-Gap Site

# On air-gapped site
stella feedser bundle import ./delta-2025-01-15.zst

# Verify import
stella feedser sites list

Full Sync Workflow

  1. Initial Sync:

    # Export full dataset
    stella feedser bundle export -o ./full-sync.zst
    
  2. Transfer to air-gap site

  3. Import on air-gap:

    stella feedser bundle import ./full-sync.zst
    
  4. Subsequent Delta Syncs:

    # Get current cursor from air-gap site
    stella feedser sites list  # Note the cursor
    
    # On connected site, export delta
    stella feedser bundle export -c "{cursor}" -o ./delta.zst
    
    # Transfer and import on air-gap
    

Verification

Validate Bundle Without Import

stella feedser bundle validate ./bundle.zst

Output:

Bundle: bundle.zst
  Version: feedser-bundle/1.0
  Site: site-us-east-1
  Cursor: 2025-01-15T10:30:00.000Z#0042

Counts:
  Canonicals: 1,234
  Edges: 3,456
  Deletions: 12
  Total: 4,702

Verification:
  Hash: ✓ Valid
  Signature: ✓ Valid (key: sha256:abc...)
  Format: ✓ Valid

Ready for import.

Preview Import Impact

stella feedser bundle import ./bundle.zst --dry-run --json

Monitoring

Sync Status Endpoint

GET /api/v1/federation/sync/status

Response:

{
  "sites": [
    {
      "site_id": "site-us-east-1",
      "enabled": true,
      "last_sync_at": "2025-01-15T10:30:00Z",
      "last_cursor": "2025-01-15T10:30:00.000Z#0042",
      "bundles_imported": 156,
      "total_items_imported": 45678
    }
  ],
  "local_cursor": "2025-01-15T10:35:00.000Z#0044"
}

Event Stream

Import events are published to the canonical-imported stream:

{
  "canonical_id": "uuid",
  "cve": "CVE-2024-1234",
  "affects_key": "pkg:npm/express@4.0.0",
  "merge_hash": "sha256:...",
  "action": "Created",
  "bundle_hash": "sha256:...",
  "site_id": "site-us-east-1",
  "imported_at": "2025-01-15T10:30:15Z"
}

Cache Invalidation

After import, cache indexes are automatically updated:

  • PURL index updated for affected packages
  • CVE index updated for vulnerability lookups
  • Existing cache entries invalidated for refresh

Troubleshooting

Common Issues

Issue Cause Solution
"Cursor not after current" Bundle is stale Use --force or export newer bundle
"Signature verification failed" Key mismatch Verify signing key is trusted
"Site not allowed" Policy restriction Add site to AllowedSites config
"Bundle too large" Size limit exceeded Increase MaxBundleSizeBytes or export smaller delta

Debug Logging

Enable verbose logging for federation operations:

Logging:
  LogLevel:
    StellaOps.Concelier.Federation: Debug

Verify Sync State

# Check local vs remote cursor
stella feedser sites status site-us-east-1

# List recent imports
stella feedser sites history site-us-east-1 --limit 5

# Verify specific canonical was imported
stella feedser canonical get sha256:mergehash...

Best Practices

  1. Regular Sync Schedule: Configure automated delta exports/imports on a schedule (e.g., hourly)

  2. Monitor Cursor Drift: Alert if cursor falls too far behind

  3. Verify Signatures: Only disable signature verification in development

  4. Size Bundles Appropriately: For large deltas, split into multiple bundles

  5. Test Import Before Production: Use --dry-run to validate bundles

  6. Maintain Key Trust: Regularly rotate and verify federation signing keys

  7. Document Site Policies: Keep a registry of trusted sites and their policies