Files
git.stella-ops.org/docs/export-center/profiles.md
root 68da90a11a
Some checks failed
Docs CI / lint-and-preview (push) Has been cancelled
Restructure solution layout by module
2025-10-28 15:10:40 +02:00

9.0 KiB

Export Center Profiles

Export Center profiles define what data is collected, how it is encoded, and which distribution paths are enabled for a run. Profiles are tenant-scoped and deterministic: identical selectors and source data produce identical bundles. This guide summarises built-in profiles, configuration fields, schema conventions, and compatibility notes.

Profile catalogue

Profile Kind / Variant Output artefacts Primary use cases
json:raw json / raw Canonical JSONL archives of advisories, VEX, SBOMs Evidence escrow, analytics pipelines
json:policy json / policy json:raw artefacts plus policy snapshot + evaluated findings Audit, compliance attestations
trivy:db trivy / db Trivy-compatible vulnerability database Feeding external scanners / CI
trivy:java-db trivy / java-db Java ecosystem supplement for Trivy Supply Java CVE data to Trivy
mirror:full mirror / full Complete mirror bundle (raw, policy, indexes, provenance) Air-gap deployments, disaster recovery
mirror:delta mirror / delta Incremental changes relative to a prior manifest Efficient mirror updates

Profiles can be cloned and customised; configuration is immutable per revision to keep runs reproducible.

Common configuration fields

Field Description Applies to Notes
name Human-readable identifier displayed in Console/CLI All Unique per tenant.
kind Logical family (json, trivy, mirror) All Determines eligible adapters.
variant Specific export flavour (see table above) All Controls adapter behaviour.
include Record types to include (advisories, vex, sboms, findings) JSON, mirror Defaults depend on variant.
policySnapshotMode required, optional, or none JSON policy, mirror required forces a policy snapshot id when creating runs.
distribution Enabled distribution drivers (http, oci, object) All Offline installs typically disable oci.
compression Compression settings (zstd, level) JSON, mirror Trivy adapters manage compression internally.
encryption Mirror encryption options (enabled, recipientKeys, strict) Mirror When enabled, only /data subtree is encrypted; manifests remain plaintext.
retention Retention policy (days or never) All Drives pruning jobs for staged bundles.

Selectors (time windows, tenants, products, SBOM subjects, ecosystems) are supplied per run, not stored in the profile.

JSON profiles

json:raw

  • Content: Exports raw advisories, VEX statements, and SBOMs as newline-delimited JSON (.jsonl.zst).
  • Schema: Follows canonical StellaOps schema with casing and timestamps normalised. Each record includes tenant, source, linkset, and content fields.
  • Options:
    • include defaults to ["advisories", "vex", "sboms"].
    • compression defaults to zstd level 9.
    • maxRecordsPerFile (optional) splits outputs for large datasets.
  • Compatibility: Intended for analytics platforms, data escrow, or feeding downstream normalisation pipelines.
  • Sample manifest excerpt:
{
  "profile": { "kind": "json", "variant": "raw" },
  "outputs": [
    { "type": "advisories.jsonl.zst", "sha256": "...", "count": 15234 },
    { "type": "vex.jsonl.zst", "sha256": "...", "count": 3045 },
    { "type": "sboms.jsonl.zst", "sha256": "...", "count": 872 }
  ],
  "selectors": { "tenant": "acme", "products": ["registry.example/app"] }
}

json:policy

  • Content: Everything from json:raw plus:
    • policy_snapshot.json (policy metadata, version, hash).
    • findings.policy.jsonl.zst (evaluated findings with decision, rationale, rule id, inputs hash).
  • Determinism: Requires a policy snapshot id; runs fail if snapshot is missing or non-deterministic mode is active.
  • Use cases: Compliance exports, auditor packages, policy attestation archives.
  • Guardrails: AOC boundaries preserved: policy outputs are clearly partitioned from raw evidence.

Trivy profiles

trivy:db

  • Detailed adapter behaviour is documented in docs/export-center/trivy-adapter.md.
  • Content: Produces a Trivy DB-compatible bundle (SQLite database or tarball as required by Trivy version).
  • Mapping rules:
    • Advisory namespaces mapped to Trivy vendor IDs (e.g., ubuntu, debian, npm).
    • Version ranges translated into Trivy's semantic version syntax.
    • Severity mapped to Trivy standard (UNKNOWN, LOW, MEDIUM, HIGH, CRITICAL).
  • Validation: Adapter enforces supported Trivy schema versions; configuration includes targetSchemaVersion.
  • Distribution: Typically pushed to OCI or object storage for downstream scanners; Console download remains available.

trivy:java-db

  • Refer to docs/export-center/trivy-adapter.md for ecosystem-specific notes.
  • Content: Optional Java ecosystem supplement for Trivy (matching Trivy's separate Java DB).
  • Dependencies: Requires Java advisories in Findings Ledger; run fails with ERR_EXPORT_EMPTY if no Java data present and allowEmpty=false.
  • Compatibility: Intended for organisations using Trivy's Java plugin or hardened pipelines that split general and Java feeds.

Mirror profiles

mirror:full

  • Bundle structure and delta strategy are covered in docs/export-center/mirror-bundles.md.
  • Content: Complete export with:
    • Raw advisories, VEX, SBOMs (/data/raw).
    • Policy overlays (/data/policy), including evaluated findings and policy snapshots.
    • Indexes for fast lookup (/indexes/advisories.pb, /indexes/sboms.pb).
    • Manifests and provenance (/manifests/export.json, /manifests/provenance.json).
  • Layout: Deterministic directory structure with hashed filenames to reduce duplication.
  • Encryption: Optional encryption block enables age/AES-GCM encryption of /data. strict=true encrypts everything except export.json.
  • Use cases: Air-gap replication, disaster recovery drills, Offline Kit seeding.

mirror:delta

  • See docs/export-center/mirror-bundles.md for delta mechanics and application order.
  • Content: Includes only changes relative to a base manifest (specified by baseExportId when running).
    • changed, added, removed lists in manifests/delta.json.
    • Incremental indexes capturing only updated subjects.
  • Constraints: Requires the base manifest to exist in object storage or artifact registry accessible to the worker. Fails with ERR_EXPORT_BASE_MISSING otherwise.
  • Workflow: Ideal for frequent updates to mirrored environments with limited bandwidth.

Compatibility and guardrails

  • Aggregation-Only Contract: All profiles respect AOC boundaries: raw evidence is never mutated. Policy outputs are appended separately with clear provenance.
  • Tenant scoping: Profiles are tenant-specific. Cross-tenant exports require explicit administrative approval and signed justification.
  • Retriable runs: Re-running a profile with identical selectors yields matching manifests and hashes, facilitating verify-on-download workflows.
  • Offline operation: JSON and mirror profiles function in offline mode without additional configuration. Trivy profiles require pre-seeded schema metadata shipped via Offline Kit.
  • Quota integration: Profiles can define run quotas (per tenant per day). Quota exhaustion surfaces as 429 Too Many Requests with X-Stella-Quota-* hints.

Example profile definition (CLI)

{
  "name": "daily-json-raw",
  "kind": "json",
  "variant": "raw",
  "include": ["advisories", "vex", "sboms"],
  "distribution": ["http", "object"],
  "compression": { "codec": "zstd", "level": 9 },
  "retention": { "mode": "days", "value": 14 }
}

Create via stella export profile create --file profile.json (CLI command documented separately).

Verification workflow

  • Download bundle via Console/CLI and extract export.json and provenance.json.
  • Run cosign verify --key <tenant-key> export.json (for detatched signatures use --signature export.json.sig).
  • Validate Trivy bundles with trivy --cache-dir <temp> --debug --db-repository <oci-ref> or local trivy module db import.
  • For mirror bundles, run internal mirror verify script (bundled in Offline Kit) to ensure directory layout and digests match manifest.

Extending profiles

  • Use API/CLI to clone an existing profile and adjust include, distribution, or retention policies.
  • Adapter plug-ins can introduce new variants (e.g., json:raw-lite, mirror:policy-only). Plug-ins must be registered at service restart and documented under /docs/export-center/profiles.md.
  • Any new profile must append the imposed rule line and follow determinism and guardrail requirements.

Imposed rule: Work of this type or tasks of this type on this component must also be applied everywhere else it should be applied.