git.stella-ops.org/docs/modules/concelier/federation-bundle-export.md

Federation Bundle Export

Per SPRINT_8200_0014_0002.

Overview

Federation bundles enable multi-site synchronization of canonical advisory data. Each bundle contains a delta of changes since a specified cursor position, allowing incremental sync between federated Concelier instances.

Bundle Format

Bundles use a TAR archive compressed with ZStandard (ZST):

feedser-bundle-v1.zst
├── MANIFEST.json           # Bundle metadata
├── canonicals.ndjson       # Canonical advisories (one per line)
├── edges.ndjson            # Source edges (one per line)
├── deletions.ndjson        # Withdrawn/deleted canonical IDs
└── SIGNATURE.json          # DSSE envelope (optional)

MANIFEST.json

{
  "version": "feedser-bundle/1.0",
  "site_id": "site-us-west-1",
  "export_cursor": "2025-01-15T10:30:00.000Z#0042",
  "since_cursor": "2025-01-14T00:00:00.000Z#0000",
  "exported_at": "2025-01-15T10:30:15.123Z",
  "counts": {
    "canonicals": 1234,
    "edges": 3456,
    "deletions": 12,
    "total": 4702
  },
  "bundle_hash": "sha256:a1b2c3d4..."
}

Field	Type	Description
version	string	Bundle format version identifier
site_id	string	Identifier of the exporting site
export_cursor	string	Cursor position after this export
since_cursor	string?	Cursor position from which changes were exported (null for full export)
exported_at	ISO8601	Timestamp when bundle was created
counts	object	Item counts by type
bundle_hash	string	SHA256 hash of compressed bundle content

canonicals.ndjson

Each line contains a canonical advisory record:

{"id":"uuid","cve":"CVE-2024-1234","affects_key":"pkg:npm/express@4.0.0","merge_hash":"a1b2c3...","status":"active","severity":"high","title":"..."}

edges.ndjson

Each line contains a source edge linking a canonical to its source advisory:

{"id":"uuid","canonical_id":"uuid","source":"nvd","source_advisory_id":"CVE-2024-1234","vendor_status":"affected"}

deletions.ndjson

Each line contains a deletion record for withdrawn or deleted canonicals:

{"canonical_id":"uuid","deleted_at":"2025-01-15T10:00:00Z","reason":"withdrawn"}

SIGNATURE.json

When signing is enabled, contains a DSSE envelope over the bundle hash:

{
  "payload_type": "application/vnd.stellaops.bundle-hash+json",
  "payload": "eyJidW5kbGVfaGFzaCI6InNoYTI1NjphMWIy..."}",
  "signatures": [
    {
      "keyid": "sha256:xyz...",
      "sig": "MEUCIQD..."
    }
  ]
}

API Endpoints

Export Bundle

GET /api/v1/federation/export

Exports a delta bundle for federation sync.

Query Parameters:

Parameter	Type	Default	Description
since_cursor	string	null	Export changes since this cursor (null = full export)
sign	bool	true	Sign the bundle with Authority key
max_items	int	10000	Maximum items per bundle (1-100000)
compress_level	int	3	ZST compression level (1-19)

Response Headers:

Header	Description
Content-Type	application/zstd
Content-Disposition	attachment; filename="feedser-bundle-{timestamp}.zst"
X-Bundle-Hash	SHA256 hash of bundle content
X-Export-Cursor	Cursor position after this export
X-Items-Count	Total items in bundle

Response: Streaming ZST-compressed TAR archive.

Errors:

Status	Code	Description
400	VALIDATION_FAILED	Invalid parameter values
503	FEDERATION_DISABLED	Federation is not enabled

Preview Export

GET /api/v1/federation/export/preview

Preview export statistics without creating a bundle.

Query Parameters:

Parameter	Type	Default	Description
since_cursor	string	null	Preview changes since this cursor

Response:

{
  "since_cursor": "2025-01-14T00:00:00Z#0000",
  "estimated_canonicals": 1234,
  "estimated_edges": 3456,
  "estimated_deletions": 12,
  "estimated_size_bytes": 5242880,
  "estimated_size_mb": 5.0
}

Federation Status

GET /api/v1/federation/status

Get federation configuration status.

Response:

{
  "enabled": true,
  "site_id": "site-us-west-1",
  "default_compression_level": 3,
  "default_max_items": 10000
}

CLI Commands

Export Bundle

stella feedser bundle export [options]

Options:

Option	Short	Default	Description
--since-cursor	-c	null	Export changes since cursor
--output	-o	stdout	Output file path
--sign	-s	true	Sign bundle with Authority key
--compress-level	-l	3	ZST compression level (1-19)
--max-items	-m	10000	Maximum items per bundle
--json		false	Output metadata as JSON

Examples:

# Full export to file
stella feedser bundle export -o ./bundle.zst

# Delta export since cursor
stella feedser bundle export -c "2025-01-14T00:00:00Z#0000" -o ./delta.zst

# Export without signing (for testing)
stella feedser bundle export --sign=false -o ./unsigned.zst

# High compression for archival
stella feedser bundle export -l 19 -o ./archived.zst

Preview Export

stella feedser bundle preview [options]

Options:

Option	Short	Description
--since-cursor	-c	Preview changes since cursor
--json		Output as JSON

Example:

stella feedser bundle preview -c "2025-01-14T00:00:00Z#0000"

Configuration

Federation is configured in concelier.yaml:

Federation:
  Enabled: true
  SiteId: "site-us-west-1"
  DefaultCompressionLevel: 3
  DefaultMaxItems: 10000
  RequireSignature: true

Setting	Type	Default	Description
Enabled	bool	false	Enable federation endpoints
SiteId	string	"default"	Identifier for this site
DefaultCompressionLevel	int	3	Default ZST compression level
DefaultMaxItems	int	10000	Default max items per bundle
RequireSignature	bool	true	Require bundle signatures

Cursor Format

Cursors encode a timestamp and sequence number:

{ISO8601}#{sequence}

Example: 2025-01-15T10:30:00.000Z#0042

• Timestamp: When the change was recorded
• Sequence: Monotonically increasing within timestamp

Cursors are opaque to consumers and should be passed through unchanged.

Determinism

Bundles are deterministic:

• Same cursor range produces identical bundle content
• Same content produces identical bundle hash
• Suitable for caching and deduplication

Security

• Bundles can be signed with DSSE for integrity verification
• Signatures use Authority keys for cross-site trust
• Bundle hash prevents tampering during transit
• ZST compression is not encryption - bundles should be transferred over TLS