git.stella-ops.org/docs/modules/export-center/architecture.md

Export Center Architecture

Derived from Epic 10 – Export Center and the subsequent export adapter deep dives.

The Export Center is the dedicated service layer that packages StellaOps evidence and policy overlays into reproducible bundles. It runs as a multi-surface API backed by asynchronous workers and format adapters, enforcing Aggregation-Only Contract (AOC) guardrails while providing deterministic manifests, signing, and distribution paths.

Runtime topology

• Export Center API (StellaOps.ExportCenter.WebService). Receives profile CRUD, export run requests, status queries, and download streams through the unified Web API gateway. Enforces tenant scopes, RBAC, quotas, and concurrency guards.
• Export Center Worker (StellaOps.ExportCenter.Worker). Dequeues export jobs from the Orchestrator, resolves selectors, invokes adapters, and writes manifests and bundle artefacts. Stateless; scales horizontally.
• Backing stores.
  • PostgreSQL tables: export_profiles, export_runs, export_inputs, export_distributions, export_events.
  • Object storage bucket or filesystem for staging bundle payloads.
  • Optional registry/object storage credentials injected via Authority-scoped secrets.
• Current truthful runtime boundary.
  • Non-testing StellaOps.ExportCenter.WebService now requires PostgreSQL-backed canonical export repositories and a real EvidenceLocker:BaseUrl; it no longer falls back to Development in-memory repositories or an in-memory evidence locker client.
  • Non-testing export verification artifact readback plus export and promotion attestation readback/verification are currently explicit 501 problem+json gaps until a durable backend exists.
  • Non-testing incident management, risk bundle job orchestration, simulation export, audit bundle generation, and exception report generation also now fail with explicit 501 problem+json responses instead of keeping canonical state in-process.
  • Non-testing timeline publication no longer defaults to InMemoryExportNotificationSink; without a durable sink backend it now reports truthful delivery failure, while Testing can opt into the in-memory sink explicitly.
  • Testing can still opt into the in-memory host implementations, but only through explicit Export:UseInMemory* switches.

Truthful unsupported surface classification

The following Unsupported* services are intentional non-testing runtime behavior, not hidden fallback state. They are acceptable shipped truth because they fail closed instead of fabricating persisted success, but each still maps to a future durable-backend workstream.

Runtime surface	Current service	Current behavior outside Testing	Classification
Verification artifact readback	UnsupportedExportArtifactStore	Returns 501 problem+json for verification readback/verify flows.	Acceptable truthful fail-closed behavior until durable verification artifact storage exists.
Export attestation readback and verify	UnsupportedExportAttestationService	Returns 501 problem+json for export attestation readback/verification.	Acceptable truthful fail-closed behavior until durable attestation persistence exists.
Promotion attestation assembly/readback	UnsupportedPromotionAttestationAssembler	Returns 501 problem+json for promotion attestation assembly/readback.	Acceptable truthful fail-closed behavior until durable promotion attestation persistence exists.
Incident management endpoints	UnsupportedExportIncidentManager	Returns 501 problem+json for incident queries/actions.	Acceptable truthful fail-closed behavior until durable incident runtime exists.
Risk bundle job orchestration	UnsupportedRiskBundleJobHandler	Returns 501 problem+json for risk bundle job endpoints.	Acceptable truthful fail-closed behavior until durable risk bundle orchestration exists.
Simulation export	UnsupportedSimulationReportExporter	Returns 501 problem+json for simulation export endpoints.	Acceptable truthful fail-closed behavior until durable simulation export runtime exists.
Audit bundle generation	UnsupportedAuditBundleJobHandler	Returns 501 problem+json for audit bundle generation endpoints.	Acceptable truthful fail-closed behavior until durable audit bundle orchestration exists.
Exception report generation	UnsupportedExceptionReportGenerator	Returns 501 problem+json for exception report generation endpoints.	Acceptable truthful fail-closed behavior until durable exception-report runtime exists.
Timeline publication sink	UnsupportedExportNotificationSink	Publishes a truthful delivery failure instead of buffering in process.	Acceptable truthful fail-closed behavior until a durable timeline notification sink exists.

• Integration peers.
  • Findings Ledger for advisory, VEX, SBOM payload streaming.
  • Policy Engine for deterministic policy snapshots and evaluated findings.
  • Orchestrator for job scheduling, quotas, and telemetry fan-out.
  • Authority for tenant-aware access tokens and KMS key references.
  • Console & CLI as presentation surfaces consuming the API.

Gap remediation (EC1–EC10)

• Schemas: publish signed ExportProfile + manifest schemas with selector validation; keep in repo alongside OpenAPI docs.
• Determinism: per-adapter ordering/compression rules with rerun-hash CI; pin Trivy DB schema versions.
• Provenance: DSSE/SLSA attestations with log metadata for every export run; include tenant IDs in predicates.
• Integrity: require checksum/signature headers and OCI annotations; mirror delta/tombstone rules documented for adapters.
• Security: cross-tenant exports denied by default; enforce approval tokens and encryption recipient validation.
• Offline parity: provide export-kit packaging + verify script for air-gap consumers; include fixtures under src/ExportCenter/__fixtures.
• Advisory link: see docs/product/advisories/28-Nov-2025 - Export Center and Reporting Strategy.md (EC1–EC10) for original requirements and keep it alongside sprint tasks for implementers.

Job lifecycle

1. Profile selection. Operator or automation picks a profile (json:raw, json:policy, trivy:db, trivy:java-db, mirror:full, mirror:delta) and submits scope selectors (tenant, time window, products, SBOM subjects, ecosystems). See docs/modules/export-center/profiles.md for profile definitions and configuration fields.
2. Planner resolution. API validates selectors, expands include/exclude lists, and writes a pending export_run with immutable parameters and deterministic ordering hints.
3. Orchestrator dispatch. export_run triggers a job lease via Orchestrator with quotas per tenant/profile and concurrency caps (default 4 active per tenant).
4. Worker execution. Worker streams data from Findings Ledger and Policy Engine using pagination cursors. Adapters write canonical payloads to staging storage, compute checksums, and emit streaming progress events (SSE).
5. Manifest and provenance emission. Worker writes export.json and provenance.json, signs them with configured KMS keys (cosign-compatible), and uploads signatures alongside content.
6. Distribution registration. Worker records available distribution methods (download URL, OCI reference, object storage path), raises completion/failure events, and exposes metrics/logs.
7. Download & verification. Clients download bundles or pull OCI artefacts, verify signatures, and consume provenance to trace source artefacts.

Cancellation requests mark runs as aborted and cause workers to stop iterating sources; partially written files are destroyed and the run is marked with an audit entry.

Core components

API surface

• Detailed request and response payloads are catalogued in docs/modules/export-center/api.md.
• Profiles API.
  • GET /api/export/profiles: list tenant-scoped profiles.
  • POST /api/export/profiles: create custom profiles (variants of JSON, Trivy, mirror) with validated configuration schema.
  • PATCH /api/export/profiles/{id}: update metadata; config changes clone new revision to preserve determinism.
• Runs API.
  • POST /api/export/runs: submit export run for a profile with selectors and options (policy snapshot id, mirror base manifest).
  • GET /api/export/runs/{id}: status, progress counters, provenance summary.
  • GET /api/export/runs/{id}/events: server-sent events with state transitions, adapter milestones, signing status.
  • POST /api/export/runs/{id}/cancel: cooperative cancellation with audit logging.
• Downloads API.
  • GET /api/export/runs/{id}/download: streaming download with range support and checksum trailers.
  • GET /api/export/runs/{id}/manifest: signed export.json.
  • GET /api/export/runs/{id}/provenance: signed provenance.json.

All endpoints require Authority-issued JWT + DPoP tokens with scopes export:run, export:read, and tenant claim alignment. Rate-limiting and quotas surface via X-Stella-Quota-* headers.

Worker pipeline

• Input resolvers. Query Findings Ledger and Policy Engine using stable pagination (PostgreSQL id ascending, or cursor-based pagination). Selector expressions compile into PostgreSQL WHERE clauses and/or API query parameters.
• Adapter host. Adapter plugin loader (restart-time only) resolves profile variant to adapter implementation. Adapters present a deterministic RunAsync(context) contract with streaming writers and telemetry instrumentation.
• Content writers.
  • JSON adapters emit .jsonl.zst files with canonical ordering (tenant, subject, document id).
  • Trivy adapters materialise SQLite databases or tar archives matching Trivy DB expectations; schema version gates prevent unsupported outputs.
  • Mirror adapters assemble deterministic filesystem trees (manifests, indexes, payload subtrees) and, when configured, OCI artefact layers.
• Manifest generator. Aggregates counts, bytes, hash digests (SHA-256), profile metadata, and input references. Writes export.json and provenance.json using canonical JSON (sorted keys, RFC3339 UTC timestamps).
• Signing service. Integrates with platform KMS via Authority (default cosign signer). Produces in-toto SLSA attestations when configured. Supports detached signatures and optional in-bundle signatures.
• Distribution drivers. dist-http exposes staged files via download endpoint; dist-oci pushes artefacts to registries using ORAS with digest pinning; dist-objstore uploads to tenant-specific prefixes with immutability flags.
• Truthful unsupported paths. The current web host no longer pretends attestation/verification readback or the incident/risk/simulation/audit/exception job surfaces are durable. Until durable backends land, those endpoints fail with 501 problem+json outside Testing. Timeline publication also no longer silently buffers to an in-process sink outside Testing.

Data model snapshots

Collection	Purpose	Key fields	Notes
export_profiles	Profile definitions (kind, variant, config).	_id, tenant, name, kind, variant, config_json, created_by, created_at.	Config includes adapter parameters (included record types, compression, encryption).
export_runs	Run state machine and audit info.	_id, profile_id, tenant, status, requested_by, selectors, policy_snapshot_id, started_at, completed_at, duration_ms, error_code.	Immutable selectors; status transitions recorded in export_events.
export_inputs	Resolved input ranges.	run_id, source, cursor, count, hash.	Enables resumable retries and audit.
export_distributions	Distribution artefacts.	run_id, type (http, oci, object), location, sha256, size_bytes, expires_at.	expires_at used for retention policies and automatic pruning.
export_events	Timeline of state transitions and metrics.	run_id, event_type, message, at, metrics.	Feeds SSE stream and audit trails.

Audit bundles (immutable triage exports)

Audit bundles are a specialized Export Center output: a deterministic, immutable evidence pack for a single subject (and optional time window) suitable for audits and incident response.

• Schema: docs/modules/evidence-locker/schemas/audit-bundle-index.schema.json (bundle index/manifest with integrity hashes and referenced artefacts).
• The index must list Rekor entry ids and RFC3161 timestamp tokens when present; offline bundles record skip reasons in predicates.
• Core APIs:
  • POST /v1/audit-bundles - Create a new bundle (async generation).
  • GET /v1/audit-bundles - List previously created bundles.
  • GET /v1/audit-bundles/{bundleId} - Returns job metadata (Accept: application/json) or streams bundle bytes (Accept: application/octet-stream).
• Typical contents: vuln reports, SBOM(s), VEX decisions, policy evaluations, and DSSE attestations, plus an integrity root hash and optional OCI reference.
• Unified audit events + chain proof (Sprint SPRINT_20260408_004 AUDIT-007). When includeContent.auditEvents is true, the handler pulls the tenant's events from Timeline's /api/v1/audit/events over the bundle time window, writes them as audit/events.ndjson (AUDIT_EVENTS), and always attaches audit/chain-proof.json (AUDIT_CHAIN_PROOF) sourced from /api/v1/audit/verify-chain.
• Audit bundle manifest + DSSE signature (AUDIT-007 criterion 3).
  • audit/manifest.json is a deterministic, canonicalised manifest (apiVersion: stella.ops/v1, kind: AuditBundleManifest) that binds the bundle id, tenant, subject, time window, audit event count, and SHA-256 digests of events.ndjson / chain-proof.json. Property names are serialised in ordinal-ascending order so identical inputs produce byte-identical payloads across runs and platforms.
  • audit/manifest.dsse.json is a DSSE envelope over the canonical manifest bytes, payloadType application/vnd.stellaops.audit-bundle-manifest+json. Signing reuses IExportAttestationSigner (local ECDSA by default, or KMS via AddExportAttestationWithKms) so audit bundles share the same verification path as promotion and export attestations.
  • Signing is controlled by includeContent.signManifest (default true). When false, only the manifest artifact is emitted. When true but the signer is unavailable (typical offline/air-gap deployments without the Attestor stack), the handler logs a warning, records the status in the job summary, and produces the bundle without an envelope — the manifest.json artifact is still present so consumers can reverify against trusted public keys later.
• Reference: docs/product/advisories/archived/27-Nov-2025-superseded/28-Nov-2025 - Vulnerability Triage UX & VEX-First Decisioning.md.

The Web Export Center quick action for Export StellaBundle is expected to use this audit-bundle surface directly. On successful completion the UI must carry the canonical bundleId through the /evidence/exports/bundles handoff, not a synthetic export-run placeholder, so the operator lands on the real generated bundle inventory and can immediately download, verify, or inspect provenance.

Adapter responsibilities

• JSON (json:raw, json:policy).
  • Ensures canonical casing, timezone normalization, and linkset preservation.
  • Policy variant embeds policy snapshot metadata (policy_version, inputs_hash, decision_trace fingerprint) and emits evaluated findings as separate files.
  • Enforces AOC guardrails: no derived modifications to raw evidence fields.
• Trivy (trivy:db, trivy:java-db).
  • Maps StellaOps advisory schema to Trivy DB format, handling namespace collisions and ecosystem-specific ranges.
  • Validates compatibility against supported Trivy schema versions; run fails fast if mismatch.
  • Emits optional manifest summarising package counts and severity distribution.
• Mirror (mirror:full, mirror:delta).
  • Builds self-contained filesystem layout (/manifests, /data/raw, /data/policy, /indexes).
  • Delta variant compares against base manifest (base_export_id) to write only changed artefacts; records removed entries for cleanup.
  • Supports optional encryption of /data subtree (age/AES-GCM) with key wrapping stored in provenance.json.
• DevPortal (devportal:offline).
  • Packages developer portal static assets, OpenAPI specs, SDK releases, and changelog content into a reproducible archive with manifest/checksum pairs.
  • Emits manifest.json, checksums.txt, helper scripts, and a DSSE signature document (manifest.dsse.json) as described in DevPortal Offline Bundle Specification.
  • Stores artefacts under <storagePrefix>/<bundleId>/ and signs manifests via the Export Center signing adapter (HMAC-SHA256 v1, tenant scoped).

Adapters expose structured telemetry events (adapter.start, adapter.chunk, adapter.complete) with record counts and byte totals per chunk. Failures emit adapter.error with reason codes.

Signing and provenance

• Manifest schema. export.json contains run metadata, profile descriptor, selector summary, counts, SHA-256 digests, compression hints, and distribution list. Deterministic field ordering and normalized timestamps.
• Provenance schema. provenance.json captures in-toto subject listing (bundle digest, manifest digest), referenced inputs (findings ledger queries, policy snapshot ids, SBOM identifiers), tool version (exporter_version, adapter versions), and KMS key identifiers.
• Attestation. Cosign SLSA Level 2 template by default; optional SLSA Level 3 when supply chain attestations are enabled. Detached signatures stored alongside manifests; CLI/Console encourage cosign verify --key <tenant-key> workflow.
• Audit trail. Each run stores success/failure status, signature identifiers, and verification hints for downstream automation (CI pipelines, offline verification scripts).

OCI Referrer Discovery

Mirror bundles automatically discover and include OCI referrer artifacts (SBOMs, attestations, signatures, VEX statements) linked to container images via the OCI 1.1 referrers API.

Discovery Flow

┌─────────────────┐     ┌───────────────────────┐     ┌─────────────────┐
│  MirrorAdapter  │────▶│  IReferrerDiscovery   │────▶│   OCI Registry  │
│                 │     │  Service              │     │                 │
│  1. Detect      │     │  2. Probe registry    │     │  3. Query       │
│     images      │     │     capabilities      │     │     referrers   │
│                 │     │                       │     │     API         │
└─────────────────┘     └───────────────────────┘     └─────────────────┘
                                   │
                                   ▼
                        ┌───────────────────────┐
                        │  Fallback: Tag-based  │
                        │  discovery for older  │
                        │  registries (GHCR)    │
                        └───────────────────────┘

Capability Probing

Before starting referrer discovery, the export flow probes each unique registry to determine capabilities:

• OCI 1.1+ registries: Native referrers API (/v2/{repo}/referrers/{digest})
• OCI 1.0 registries: Fallback to tag-based discovery (sha256-{digest}.* tags)

Capabilities are cached per registry host with a 1-hour TTL.

Logging at export start:

[INFO] Probing 3 registries for OCI referrer capabilities before export
[INFO] Registry registry.example.com: OCI 1.1 (referrers API supported, version=OCI-Distribution/2.1, probe_ms=42)
[WARN] Registry ghcr.io: OCI 1.0 (using fallback tag discovery, version=registry/2.0, probe_ms=85)

Telemetry Metrics

Metric	Description	Tags
export_registry_capabilities_probed_total	Registry capability probe operations	registry, api_supported
export_referrer_discovery_method_total	Discovery operations by method	registry, method (native/fallback)
export_referrers_discovered_total	Referrers discovered	registry, artifact_type
export_referrer_discovery_failures_total	Discovery failures	registry, error_type

Artifact Type Mapping

OCI Artifact Type	Bundle Category	Example
application/vnd.cyclonedx+json	sbom	CycloneDX SBOM
application/vnd.spdx+json	sbom	SPDX SBOM
application/vnd.openvex+json	vex	OpenVEX statement
application/vnd.csaf+json	vex	CSAF document
application/vnd.in-toto+json	attestation	in-toto attestation
application/vnd.dsse.envelope+json	attestation	DSSE envelope
application/vnd.slsa.provenance+json	attestation	SLSA provenance

Error Handling

• If referrer discovery fails for a single image, the export logs a warning and continues with other images
• Network failures do not block the entire export
• Missing referrer artifacts are validated during bundle import (see ImportValidator)

Related Documentation

• Registry Compatibility Matrix
• Offline Bundle Format
• Registry Referrer Troubleshooting

Distribution flows

• HTTP download. Console and CLI stream bundles via chunked transfer; supports range requests and resumable downloads. Response includes X-Export-Digest, X-Export-Length, and optional encryption metadata.
• OCI push. Worker uses ORAS to publish bundles as OCI artefacts with annotations describing profile, tenant, manifest digest, and provenance reference. Supports multi-tenant registries with repository-per-tenant naming.
• Object storage. Writes to tenant-prefixed paths (s3://stella-exports/{tenant}/{run-id}/...) with immutable retention policies. Retention scheduler purges expired runs based on profile configuration.
• Offline Kit seeding. Mirror bundles optionally staged into Offline Kit assembly pipelines, inheriting the same manifests and signatures.

Observability

• Metrics. Emits exporter_run_duration_seconds, exporter_run_bytes_total{profile}, exporter_run_failures_total{error_code}, exporter_active_runs{tenant}, exporter_distribution_push_seconds{type}.
• Logs. Structured logs with fields run_id, tenant, profile_kind, adapter, phase, correlation_id, error_code. Phases include plan, resolve, adapter, manifest, sign, distribute.
• Traces. Optional OpenTelemetry spans (export.plan, export.fetch, export.write, export.sign, export.distribute) for cross-service correlation.
• Dashboards & alerts. DevOps pipeline seeds Grafana dashboards summarising throughput, size, failure ratios, and distribution latency. Alert thresholds: failure rate >5% per profile, median run duration >p95 baseline, signature verification failures >0. Runbook + dashboard stub for offline import: operations/observability.md, operations/dashboards/export-center-observability.json.

Security posture

• Tenant claim enforced at every query and distribution path; cross-tenant selectors rejected unless explicit cross-tenant mirror feature toggled with signed approval.
• RBAC scopes: export:profile:manage, export:run, export:read, export:download. Console hides actions without scope; CLI returns 401/403.
• Encryption options configurable per profile; keys derived from Authority-managed KMS. Mirror encryption uses tenant-specific recipients; JSON/Trivy rely on transport security plus optional encryption at rest.
• Restart-only plugin loading ensures adapters and distribution drivers are vetted at deployment time, reducing runtime injection risks.
• Deterministic output ensures tamper detection via content hashes; provenance links to source runs and policy snapshots to maintain auditability.

Deployment considerations

• Packaged as separate API and worker containers. Helm chart and compose overlays define horizontal scaling, worker concurrency, queue leases, and object storage credentials.
• Requires Authority client credentials for KMS and optional registry credentials stored via sealed secrets.
• Offline-first deployments disable OCI distribution by default and provide local object storage endpoints; HTTP downloads served via internal gateway.
• Health endpoints: /health/ready validates PostgreSQL connectivity, object storage access, adapter registry integrity, and KMS signer readiness.

Compliance checklist

• Profiles and runs enforce tenant scoping; cross-tenant exports disabled unless approved.
• Manifests and provenance files are generated with deterministic hashes and signed via configured KMS.
• Adapters run with restart-time registration only; no runtime plugin loading.
• Distribution drivers respect allowlist; OCI push disabled when offline mode is active.
• Metrics, logs, and traces follow observability guidelines; dashboards and alerts configured.
• Retention policies and pruning jobs configured for staged bundles.

Imposed rule: Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.

Client Surfacing of Hidden Backend Capabilities

The ExportSurfacingClient extends the existing ExportCenterClient by exposing backend capabilities that were previously not surfaced to CLI/UI consumers.

Surfaced Capabilities

Capability	Interface Method	Route
Profile CRUD	CreateProfileAsync, UpdateProfileAsync, ArchiveProfileAsync	POST/PUT/DELETE /v1/exports/profiles
Run Lifecycle	StartRunAsync, CancelRunAsync	POST /profiles/{id}/runs, POST /runs/{id}/cancel
Artifact Browsing	ListArtifactsAsync, GetArtifactAsync, DownloadArtifactAsync	GET /runs/{id}/artifacts
Verification	VerifyRunAsync, GetManifestAsync, GetAttestationStatusAsync	POST /runs/{id}/verify, GET .../manifest, GET .../attestation
Discovery	DiscoverCapabilitiesAsync	Local (15 known capabilities)

Key Types

Type	Location	Purpose
IExportSurfacingClient	Client/IExportSurfacingClient.cs	Interface for extended operations
ExportSurfacingClient	Client/ExportSurfacingClient.cs	HTTP implementation
ExportSurfacingModels.cs	Client/Models/	DTOs for profile CRUD, artifacts, verification, attestation status, capability discovery

DI Registration

AddExportSurfacingClient(Action<ExportCenterClientOptions>) in ServiceCollectionExtensions.cs — reuses the same ExportCenterClientOptions.

Test Coverage (37 tests)

• Models: CreateExportProfileRequest defaults, UpdateExportProfileRequest nulls, StartExportRunRequest defaults, ExportArtifact roundtrip, empty list, VerifyExportRunRequest defaults, ExportVerificationResult, HashVerificationEntry match/mismatch, SignatureVerificationEntry, ExportManifest, ExportAttestationStatus, ExportCapability, ExportCapabilitySummary, StartExportRunResponse
• Client: Constructor null guards (2), DiscoverCapabilities (all/profiles/verification/audit-bundles/openapi-anonymous), argument validation (8 — CreateProfile/ArchiveProfile/CancelRun/StartRun/ListArtifacts/GetArtifact/VerifyRun/GetManifest/GetAttestationStatus)