Add sample proof bundle configurations and verification script
Some checks failed
AOC Guard CI / aoc-guard (push) Has been cancelled
AOC Guard CI / aoc-verify (push) Has been cancelled
Concelier Attestation Tests / attestation-tests (push) Has been cancelled
Console CI / console-ci (push) Has been cancelled
Docs CI / lint-and-preview (push) Has been cancelled
Export Center CI / export-ci (push) Has been cancelled
VEX Proof Bundles / verify-bundles (push) Has been cancelled
Some checks failed
AOC Guard CI / aoc-guard (push) Has been cancelled
AOC Guard CI / aoc-verify (push) Has been cancelled
Concelier Attestation Tests / attestation-tests (push) Has been cancelled
Console CI / console-ci (push) Has been cancelled
Docs CI / lint-and-preview (push) Has been cancelled
Export Center CI / export-ci (push) Has been cancelled
VEX Proof Bundles / verify-bundles (push) Has been cancelled
- Introduced sample proof bundle configuration files for testing, including `sample-proof-bundle-config.dsse.json`, `sample-proof-bundle.dsse.json`, and `sample-proof-bundle.json`. - Implemented a verification script `test_verify_sample.sh` to validate proof bundles against specified schemas and catalogs. - Updated existing proof bundle configurations with new metadata, including versioning, created timestamps, and justification details. - Enhanced evidence entries with expiration dates and hashes for better integrity checks. - Ensured all new configurations adhere to the defined schema for consistency and reliability in testing.
This commit is contained in:
@@ -44,6 +44,8 @@ Upcoming EB1–EB10 remediation (Sprint 0161; advisory `docs/product-advisories/
|
||||
- Ship an offline verifier script and golden bundles/replay fixtures to prove determinism.
|
||||
- Add incident-mode activation/exit records and redaction/tenant isolation guidance for portable bundles.
|
||||
|
||||
Canonical schemas now live in `docs/modules/evidence-locker/schemas/` (EB1, EB2). Offline verification steps and the embeddable script are documented in `docs/modules/evidence-locker/verify-offline.md` (EB9); use the computed Merkle root as the DSSE subject for sealed and portable bundles.
|
||||
|
||||
### Merkle recipe (example)
|
||||
```bash
|
||||
cd bundle
|
||||
|
||||
32
docs/modules/evidence-locker/eb-gaps-161-007-plan.md
Normal file
32
docs/modules/evidence-locker/eb-gaps-161-007-plan.md
Normal file
@@ -0,0 +1,32 @@
|
||||
# EB1–EB10 Gap Closure Plan (EVID-GAPS-161-007)
|
||||
|
||||
Purpose: track remediation items from the 28-Nov-2025 advisory so Evidence Locker bundles, replay payloads, and portable exports are provably deterministic and verifiable offline.
|
||||
|
||||
Working directory: `docs/implplan` (sprint coordination) with artefacts in `docs/modules/evidence-locker` and `tests/EvidenceLocker`.
|
||||
|
||||
## Scope Items
|
||||
| ID | Deliverable | Artifact / Path | Owner(s) | Acceptance / Notes | Status |
|
||||
| --- | --- | --- | --- | --- | --- |
|
||||
| EB1 | Publish canonical manifest schema | `docs/modules/evidence-locker/schemas/bundle.manifest.schema.json` | Evidence Locker Guild | JSON Schema matches EvidenceBundleManifest (bundleId, tenantId, kind, metadata, entries) and captures replay/incident/redaction hooks. | Draft (2025-12-04) |
|
||||
| EB2 | Publish checksums schema | `docs/modules/evidence-locker/schemas/checksums.schema.json` | Evidence Locker Guild | Canonical map for `checksums.txt`; Merkle root + chunking metadata; sorted entry rule recorded. | Draft (2025-12-04) |
|
||||
| EB3 | Hash/Merkle recipe doc | `docs/modules/evidence-locker/bundle-packaging.md` (new section) | Evidence Locker Guild | Normative steps for Merkle root + DSSE subject; clarifies gzip/tar invariants and CAS compatibility. | TODO |
|
||||
| EB4 | Mandatory DSSE predicate/log policy | `docs/modules/evidence-locker/attestation-contract.md` | Evidence Locker Guild · Security Guild | Required claims + signing profiles; Rekor/log policy (optional vs required); aligns with crypto registry defaults. | TODO |
|
||||
| EB5 | Replay provenance block | `docs/modules/evidence-locker/replay-payload-contract.md` + manifest schema | Evidence Locker Guild · Replay Delivery Guild | Replay digest + DSSE envelope recorded; ordering rules match `DETERMINISTIC_REPLAY.md`; portable bundle retains linkage. | TODO |
|
||||
| EB6 | Chunking/CAS rules | `checksums.schema.json` + `bundle-packaging.md` | Evidence Locker Guild · Storage/DevOps | Defines chunk sizing, CAS digest, and stability guarantees; CI test to catch ordering changes. | TODO |
|
||||
| EB7 | Incident-mode signed activation/exit | `docs/modules/evidence-locker/incident-mode.md` | Evidence Locker Guild · Security Guild | Manifest/DSSE captures activation + deactivation events with signer identity; API/CLI steps documented. | TODO |
|
||||
| EB8 | Tenant isolation + redaction manifest | `bundle-packaging.md` + portable bundle guidance | Evidence Locker Guild · Privacy Guild | Portable bundles omit tenant identifiers; redaction map recorded; verifier asserts redacted fields absent. | TODO |
|
||||
| EB9 | Offline verifier script | `docs/modules/evidence-locker/verify-offline.md` | Evidence Locker Guild | POSIX script included; no network dependencies; emits Merkle root used by DSSE subject. | DONE (2025-12-04) |
|
||||
| EB10 | Golden bundles/replay fixtures + SemVer/changelog | `tests/EvidenceLocker/Bundles/Golden/` + release notes (TBD) | Evidence Locker Guild · CLI Guild | Golden sealed + portable bundles and replay NDJSON with expected roots; changelog bump covering EB1–EB9. | TODO |
|
||||
|
||||
## Near-Term Actions (to move EB1–EB10 to DONE)
|
||||
- Wire schemas into EvidenceLocker CI (manifest + checksums validation) and surface in API/CLI OpenAPI/Help.
|
||||
- Update `attestation-contract.md` and `incident-mode.md` with DSSE predicate/log policy and signed incident toggles (EB4, EB7).
|
||||
- Extend replay contract with provenance block and ordering example, and mirror in manifest schema (EB5).
|
||||
- Add normative Merkle/CAS section to `bundle-packaging.md`, ensuring DSSE subject references the root hash (EB3, EB6).
|
||||
- Create golden fixtures under `tests/EvidenceLocker/Bundles/Golden/` with recorded expected hashes and replay traces; hook into xUnit tests (EB10).
|
||||
- Bump Evidence Locker and CLI SemVer and changelog once above artefacts are wired (EB10).
|
||||
|
||||
## Dependencies and Links
|
||||
- Advisory: `docs/product-advisories/archived/27-Nov-2025-superseded/28-Nov-2025 - Evidence Bundle and Replay Contracts.md`
|
||||
- Replay rules: `docs/replay/DETERMINISTIC_REPLAY.md`
|
||||
- Sprint tracking: `docs/implplan/SPRINT_0161_0001_0001_evidencelocker.md` (EVID-GAPS-161-007)
|
||||
142
docs/modules/evidence-locker/schemas/bundle.manifest.schema.json
Normal file
142
docs/modules/evidence-locker/schemas/bundle.manifest.schema.json
Normal file
@@ -0,0 +1,142 @@
|
||||
{
|
||||
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
||||
"$id": "https://stellaops.local/schemas/evidence/bundle.manifest.schema.json",
|
||||
"title": "StellaOps Evidence Bundle Manifest (EB1)",
|
||||
"description": "Canonical manifest for deterministic evidence bundles; aligns with EvidenceLocker build models and EB1–EB10 advisory gaps.",
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": [
|
||||
"bundleId",
|
||||
"tenantId",
|
||||
"kind",
|
||||
"createdAt",
|
||||
"metadata",
|
||||
"entries"
|
||||
],
|
||||
"properties": {
|
||||
"bundleId": {
|
||||
"type": "string",
|
||||
"description": "Bundle identifier in UUID v4 N-format (no dashes).",
|
||||
"pattern": "^[0-9a-fA-F]{32}$"
|
||||
},
|
||||
"tenantId": {
|
||||
"type": "string",
|
||||
"description": "Tenant identifier in UUID v4 N-format (no dashes).",
|
||||
"pattern": "^[0-9a-fA-F]{32}$"
|
||||
},
|
||||
"kind": {
|
||||
"description": "Bundle category; numeric values mirror EvidenceBundleKind enum.",
|
||||
"oneOf": [
|
||||
{ "type": "string", "enum": ["evaluation", "job", "export"] },
|
||||
{ "type": "integer", "enum": [1, 2, 3] }
|
||||
]
|
||||
},
|
||||
"createdAt": {
|
||||
"type": "string",
|
||||
"format": "date-time",
|
||||
"description": "Bundle creation timestamp (UTC, RFC3339)."
|
||||
},
|
||||
"metadata": {
|
||||
"type": "object",
|
||||
"description": "Arbitrary key/value metadata captured at bundle creation.",
|
||||
"additionalProperties": { "type": "string" }
|
||||
},
|
||||
"entries": {
|
||||
"type": "array",
|
||||
"description": "Canonical file inventory used to derive checksums and Merkle root.",
|
||||
"minItems": 1,
|
||||
"items": { "$ref": "#/$defs/manifestEntry" }
|
||||
},
|
||||
"hashSummary": {
|
||||
"type": "object",
|
||||
"description": "Optional Merkle root summary that binds the manifest to checksums.txt.",
|
||||
"additionalProperties": false,
|
||||
"required": ["algorithm", "merkleRoot"],
|
||||
"properties": {
|
||||
"algorithm": { "type": "string", "enum": ["sha256"] },
|
||||
"merkleRoot": { "type": "string", "pattern": "^[0-9a-f]{64}$" },
|
||||
"checksumsPath": {
|
||||
"type": "string",
|
||||
"description": "Relative path to canonical checksums file inside the bundle.",
|
||||
"default": "checksums.txt"
|
||||
}
|
||||
}
|
||||
},
|
||||
"replayProvenance": {
|
||||
"type": "object",
|
||||
"description": "Optional replay linkage proving how the bundle was produced for deterministic re-run.",
|
||||
"additionalProperties": false,
|
||||
"required": ["recordDigest"],
|
||||
"properties": {
|
||||
"recordDigest": { "type": "string", "pattern": "^sha256:[0-9a-f]{64}$" },
|
||||
"sequence": { "type": "integer", "minimum": 0 },
|
||||
"ledgerUri": { "type": "string", "format": "uri" },
|
||||
"dsseEnvelope": {
|
||||
"type": "string",
|
||||
"description": "Base64-encoded DSSE envelope for replay record provenance.",
|
||||
"contentEncoding": "base64"
|
||||
},
|
||||
"transparencyLog": {
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"rekorUuid": { "type": "string" },
|
||||
"logIndex": { "type": "integer", "minimum": 0 },
|
||||
"inclusionProof": { "type": "string" }
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"incident": {
|
||||
"type": "object",
|
||||
"description": "Incident-mode activation/exit records captured at bundle time.",
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"activatedAt": { "type": "string", "format": "date-time" },
|
||||
"activatedBy": { "type": "string" },
|
||||
"reason": { "type": "string" },
|
||||
"deactivatedAt": { "type": "string", "format": "date-time" },
|
||||
"deactivatedBy": { "type": "string" }
|
||||
}
|
||||
},
|
||||
"redaction": {
|
||||
"type": "object",
|
||||
"description": "Portable-bundle redaction details to prove tenant isolation.",
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"portable": { "type": "boolean", "default": false },
|
||||
"maskedFields": {
|
||||
"type": "array",
|
||||
"items": { "type": "string" }
|
||||
},
|
||||
"tenantToken": {
|
||||
"type": "string",
|
||||
"description": "Opaque token replacing tenantId in portable bundles."
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"$defs": {
|
||||
"manifestEntry": {
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": ["section", "canonicalPath", "sha256", "sizeBytes", "mediaType"],
|
||||
"properties": {
|
||||
"section": { "type": "string", "minLength": 1 },
|
||||
"canonicalPath": {
|
||||
"type": "string",
|
||||
"description": "Deterministic path within the bundle using '/' separators.",
|
||||
"pattern": "^(?:[A-Za-z0-9_.-]+/)*[A-Za-z0-9_.-]+$"
|
||||
},
|
||||
"sha256": { "type": "string", "pattern": "^[0-9a-f]{64}$" },
|
||||
"sizeBytes": { "type": "integer", "minimum": 0 },
|
||||
"mediaType": { "type": "string" },
|
||||
"attributes": {
|
||||
"type": "object",
|
||||
"description": "Section-specific attributes (e.g., sbom format, dsse predicate).",
|
||||
"additionalProperties": { "type": "string" }
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
47
docs/modules/evidence-locker/schemas/checksums.schema.json
Normal file
47
docs/modules/evidence-locker/schemas/checksums.schema.json
Normal file
@@ -0,0 +1,47 @@
|
||||
{
|
||||
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
||||
"$id": "https://stellaops.local/schemas/evidence/checksums.schema.json",
|
||||
"title": "StellaOps Evidence Bundle Checksums (EB2)",
|
||||
"description": "Canonical checksum map used to derive the Merkle root and DSSE subject for evidence bundles.",
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": ["algorithm", "root", "entries"],
|
||||
"properties": {
|
||||
"algorithm": { "type": "string", "enum": ["sha256"] },
|
||||
"root": { "type": "string", "pattern": "^[0-9a-f]{64}$" },
|
||||
"generatedAt": { "type": "string", "format": "date-time" },
|
||||
"bundleId": { "type": "string", "pattern": "^[0-9a-fA-F]{32}$" },
|
||||
"tenantId": { "type": "string", "pattern": "^[0-9a-fA-F]{32}$" },
|
||||
"entries": {
|
||||
"type": "array",
|
||||
"minItems": 1,
|
||||
"description": "Sorted list of entry hashes; order must be lexicographic on canonicalPath.",
|
||||
"items": { "$ref": "#/$defs/checksumEntry" }
|
||||
},
|
||||
"chunking": {
|
||||
"type": "object",
|
||||
"description": "Optional chunked/CAS hashing strategy for large payloads.",
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"strategy": { "type": "string", "enum": ["none", "fixed", "buzhash"] },
|
||||
"chunkSizeBytes": { "type": "integer", "minimum": 1024 },
|
||||
"casDigestAlgorithm": { "type": "string", "enum": ["sha256"] }
|
||||
}
|
||||
}
|
||||
},
|
||||
"$defs": {
|
||||
"checksumEntry": {
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": ["canonicalPath", "sha256", "sizeBytes"],
|
||||
"properties": {
|
||||
"canonicalPath": {
|
||||
"type": "string",
|
||||
"pattern": "^(?:[A-Za-z0-9_.-]+/)*[A-Za-z0-9_.-]+$"
|
||||
},
|
||||
"sha256": { "type": "string", "pattern": "^[0-9a-f]{64}$" },
|
||||
"sizeBytes": { "type": "integer", "minimum": 0 }
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
51
docs/modules/evidence-locker/verify-offline.md
Normal file
51
docs/modules/evidence-locker/verify-offline.md
Normal file
@@ -0,0 +1,51 @@
|
||||
# Offline Verification Playbook (EB9)
|
||||
|
||||
Purpose: allow auditors to validate Evidence Locker bundles without network access, using only POSIX tools. Applies to both sealed `bundle.tgz` and portable `portable-bundle-v1.tgz`.
|
||||
|
||||
## Prerequisites
|
||||
- `tar`, `sha256sum` (or `shasum`), `awk`, `base64`.
|
||||
- Optional: `jq` for schema validation; `cosign` or `stella` CLI for DSSE verification if pre-loaded.
|
||||
|
||||
## Quick steps (sealed bundle)
|
||||
1) `tar -xzf bundle.tgz -C /tmp/bundle`
|
||||
2) `cd /tmp/bundle`
|
||||
3) Validate checksums: `sha256sum -c checksums.txt`
|
||||
4) Derive Merkle root (matches DSSE subject): `sha256sum checksums.txt | awk '{print $1}'`
|
||||
5) Validate manifest against schema (if `jq` present): `jq -e 'input | type=="object"' manifest.json >/dev/null`
|
||||
6) Verify DSSE envelope (optional but recommended):
|
||||
- `cat manifest.json | base64 | cosign verify-blob --key cosign.pub --bundle signature.json --bundleType dsse`
|
||||
- or `stella evidence verify --bundle ../bundle.tgz --offline` once CLI supports offline mode.
|
||||
|
||||
## Quick steps (portable bundle)
|
||||
Same as sealed, plus confirm redaction:
|
||||
- `jq -e 'has(\"redaction\") and .redaction.portable==true' manifest.json >/dev/null` (if `jq` available)
|
||||
- Confirm no tenant identifiers in `bundle.json` and `manifest.json`.
|
||||
|
||||
## Embeddable verifier script
|
||||
Place the following script into `verify-offline.sh` when assembling portable bundles. It exits non-zero on any mismatch and prints the Merkle root used as DSSE subject.
|
||||
|
||||
```bash
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
BUNDLE="${1:-bundle.tgz}"
|
||||
WORKDIR="$(mktemp -d)"
|
||||
cleanup() { rm -rf "$WORKDIR"; }
|
||||
trap cleanup EXIT
|
||||
tar -xzf "$BUNDLE" -C "$WORKDIR"
|
||||
cd "$WORKDIR"
|
||||
sha256sum -c checksums.txt
|
||||
MERKLE=$(sha256sum checksums.txt | awk '{print $1}')
|
||||
printf "merkle_root=%s\n" "$MERKLE"
|
||||
if command -v jq >/dev/null; then
|
||||
jq -e 'type=="object" and has("entries")' manifest.json >/dev/null
|
||||
fi
|
||||
```
|
||||
|
||||
## Fixtures
|
||||
- Golden bundles and replay records live under `tests/EvidenceLocker/Bundles/Golden/`.
|
||||
- Expected Merkle roots and DSSE payload digests should be recorded alongside each fixture to keep CI deterministic.
|
||||
|
||||
## References
|
||||
- Manifest schema: `docs/modules/evidence-locker/schemas/bundle.manifest.schema.json`
|
||||
- Checksums schema: `docs/modules/evidence-locker/schemas/checksums.schema.json`
|
||||
- Merkle recipe: see `docs/modules/evidence-locker/bundle-packaging.md`
|
||||
@@ -1,34 +1,58 @@
|
||||
# Export Center Determinism & Rerun Hash Guide
|
||||
|
||||
Advisory: `docs/product-advisories/28-Nov-2025 - Export Center and Reporting Strategy.md` (EC1–EC10).
|
||||
Advisory anchor: `docs/product-advisories/archived/27-Nov-2025-superseded/28-Nov-2025 - Export Center and Reporting Strategy.md` (EC1–EC10).
|
||||
|
||||
## Adapter settings (runnable example)
|
||||
- JSON adapters: `--compression zstd --compression-level 19 --deterministic-order`
|
||||
- Mirror adapter: sort descriptors by digest, emit annotations in lexicographic order, disable mtime in tar (`--mtime 0`).
|
||||
- Delta adapter: include `baseManifestHash` and sorted `added`/`removed` lists; tombstones must be explicit.
|
||||
## EC1 — Signed schemas
|
||||
- Export profile schema: `docs/modules/export-center/schemas/export-profile.schema.json` (selectors, approvals, quotas).
|
||||
- Export manifest schema: `docs/modules/export-center/schemas/export-manifest.schema.json` (rerunHash, integrity headers, attestations, quotas/backpressure).
|
||||
- Both schemas must be signed (DSSE) alongside publication; DSSE envelopes live next to the schema files when generated in CI.
|
||||
|
||||
## Rerun-hash check
|
||||
## EC2 — Per-adapter determinism and rerun hash
|
||||
- JSON adapters: canonical JSONL, sorted keys, zstd level 19; filenames stable (`advisories-<shard>.jsonl.zst`); gzip forbidden.
|
||||
- Trivy adapters: pin schema version (see `trivy-adapter.md`), normalize namespaces, ordered records by `(namespace, package, vulnerabilityId)`.
|
||||
- Mirror full: tar with `--sort=name --mtime=@0 --owner=0 --group=0 --numeric-owner`; manifest entries sorted by path; indexes stable.
|
||||
- Mirror delta: include `baseManifestDigest`, sorted `added`/`removed`, explicit `tombstones`; reject deltas without tombstones for removed entries.
|
||||
- Rerun hash algorithm: SHA-256 over newline-joined, sorted `contents[*].digest` values; stored in `manifest.rerunHash` and asserted in CI.
|
||||
- CI harness: `docs/modules/export-center/operations/verify-export-kit.sh` recomputes rerun hash and schema-consistent integrity hints.
|
||||
|
||||
## EC3 — DSSE + SLSA attestation with log metadata
|
||||
- All manifests and provenance files carry DSSE envelopes; provenance must include SLSA v1 builder metadata plus log proof (`kind`, `logId`, `logIndex`, `entryDigest`, `timestamp`).
|
||||
- Provenance subjects list both `manifests/export.json` and bundle tar/OCI digest; log metadata is mandatory even when transparency uploads are deferred.
|
||||
|
||||
## EC4 — Cross-tenant approval flow
|
||||
- `selectors.tenants` must contain the profile tenant; when selectors include additional tenants or wildcards, `approval.required=true` with `approvedBy` and `ticket` is mandatory (validated by the verify script).
|
||||
|
||||
## EC5 — Distribution integrity headers and OCI annotations
|
||||
- HTTP: `Digest: sha-256=<base64>` derived from bundle digest; `X-Stella-Signature: dsse-b64:<envelope>`; `X-Stella-Immutability: true` for immutable responses.
|
||||
- OCI: annotations must include `io.stellaops.export.profile`, `io.stellaops.export.run`, `io.stellaops.export.manifest-digest`, `io.stellaops.export.provenance-ref`, and `org.opencontainers.image.ref.name`.
|
||||
|
||||
## EC6 — Trivy schema pinning
|
||||
- Schema compatibility is pinned in `trivy-adapter.md`; CI rejects versions above the pinned set and emits `ERR_EXPORT_UNSUPPORTED_SCHEMA`.
|
||||
- Mirror/export manifests must record the targeted `schemaVersion` so rerun-hash and consumers can enforce deterministic decoding.
|
||||
|
||||
## EC7 — Mirror delta/tombstone rules
|
||||
- Deltas MUST include tombstones for all removals and a `baseManifestDigest` that matches the referenced baseline; omitted tombstones fail verification.
|
||||
- `delta.added/removed` are sorted, and `resetBaseline=false` unless explicitly set; consumers apply deltas in order and refuse out-of-order manifests.
|
||||
|
||||
## EC8 — Encryption/recipient policy
|
||||
- Only `age` or `aes-gcm` envelopes; recipients enumerated with `fingerprint` and optional `wrappedKey` in manifest and provenance.
|
||||
- `strict=true` encrypts everything except manifest/provenance; defaults to `false` to keep discovery metadata plaintext.
|
||||
|
||||
## EC9 — Quotas and backpressure
|
||||
- Manifest `quotas` block captures `maxActiveRuns`, `maxQueuedRuns`, `backpressureMode` (`reject`|`defer`|`throttle`), and optional `cpuThrottlePercent`.
|
||||
- CI verifies presence of quotas; operators surface `429` with `X-Stella-Quota-*` hints when limits engage.
|
||||
|
||||
## EC10 — Offline export kit + verify script
|
||||
- Fixtures: `src/ExportCenter/__fixtures/export-kit/*` (manifest, manifest.sha256, manifest.dsse, provenance).
|
||||
- Verifier: `docs/modules/export-center/operations/verify-export-kit.sh`
|
||||
- Validates manifest hash against `manifest.sha256`.
|
||||
- Recomputes rerun hash.
|
||||
- Confirms integrity headers align with OCI annotations.
|
||||
- Enforces approval + quota presence for cross-tenant selectors.
|
||||
- Confirms provenance references manifest digest and carries log metadata.
|
||||
- Tar flags for offline kit assembly: `tar --sort=name --mtime=@0 --owner=0 --group=0 --numeric-owner`.
|
||||
|
||||
## Quick rerun-hash smoke (uses fixtures)
|
||||
```bash
|
||||
set -euo pipefail
|
||||
run_id=$(uuidgen)
|
||||
stella export run --profile demo --run-id "$run_id" --out /tmp/export1
|
||||
sha256sum /tmp/export1/manifest.json > /tmp/export1/manifest.sha256
|
||||
# second run
|
||||
run_id2=$(uuidgen)
|
||||
stella export run --profile demo --run-id "$run_id2" --out /tmp/export2
|
||||
sha256sum /tmp/export2/manifest.json > /tmp/export2/manifest.sha256
|
||||
diff -u /tmp/export1/manifest.sha256 /tmp/export2/manifest.sha256
|
||||
./docs/modules/export-center/operations/verify-export-kit.sh src/ExportCenter/__fixtures/export-kit
|
||||
```
|
||||
|
||||
## Integrity headers (HTTP example)
|
||||
- `Digest: sha-256=<base64>`
|
||||
- `X-Stella-Signature: dsse-b64=<payload>`
|
||||
- `X-Stella-Immutability: true`
|
||||
|
||||
## Offline kit packaging
|
||||
- Tar flags: `tar --sort=name --mtime=@0 --owner=0 --group=0 --numeric-owner`
|
||||
- Include `export-kit/manifest.json` + `manifest.dsse`; add `verify-export-kit.sh` to check hashes and signatures.
|
||||
|
||||
## Where to place fixtures
|
||||
- `src/ExportCenter/__fixtures/` for deterministic manifests/outputs used by tests.
|
||||
- Add rerun-hash CI to compare fixture hash against regenerated outputs.
|
||||
|
||||
@@ -92,9 +92,11 @@ delta/
|
||||
manifest.diff.json # summary of counts, hashes, base export metadata
|
||||
```
|
||||
|
||||
- **Base lookup:** The worker verifies that the base export is reachable (download path or OCI reference). If missing, the run fails with `ERR_EXPORT_BASE_MISSING`.
|
||||
- **Change detection:** Uses deterministic hashing of normalized records to compute additions/updates. Indexes are regenerated only for affected subjects.
|
||||
- **Application order:** Consumers apply deltas sequentially. A `resetBaseline=true` flag instructs them to drop cached state and apply the bundle as a full refresh.
|
||||
- **Base lookup:** The worker verifies that the base export is reachable (download path or OCI reference). If missing, the run fails with `ERR_EXPORT_BASE_MISSING`.
|
||||
- **Change detection:** Uses deterministic hashing of normalized records to compute additions/updates. Indexes are regenerated only for affected subjects.
|
||||
- **Application order:** Consumers apply deltas sequentially. A `resetBaseline=true` flag instructs them to drop cached state and apply the bundle as a full refresh.
|
||||
- **Tombstones required:** Every removal must emit a tombstone entry plus the `removed` list; deltas without tombstones fail verification (`verify-export-kit.sh`).
|
||||
- **Integrity headers:** Each delta bundle exports `Digest`, `X-Stella-Signature`, and `X-Stella-Immutability` derived from the OCI annotation `io.stellaops.export.manifest-digest`. Consumers must validate before applying.
|
||||
|
||||
Example `manifest.diff.json` (delta):
|
||||
|
||||
@@ -195,6 +197,7 @@ sequenceDiagram
|
||||
- Post-import checks:
|
||||
- Recompute SHA256 for `manifest.yaml` and a sample data file; compare to manifest.
|
||||
- Run `mirror verify` (Offline Kit) and confirm zero mismatches.
|
||||
- Confirm OCI annotations `io.stellaops.export.profile/run/manifest-digest/provenance-ref` match the bundle being applied.
|
||||
|
||||
## 8. Troubleshooting
|
||||
|
||||
|
||||
99
docs/modules/export-center/operations/verify-export-kit.sh
Normal file
99
docs/modules/export-center/operations/verify-export-kit.sh
Normal file
@@ -0,0 +1,99 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
# Offline verifier for Export Center bundles (EC10)
|
||||
# Usage: VERIFY_FIXTURE=1 ./verify-export-kit.sh [kit_dir]
|
||||
|
||||
kit_dir="${1:-src/ExportCenter/__fixtures/export-kit}"
|
||||
|
||||
for f in manifest.json manifest.sha256 provenance.json manifest.dsse; do
|
||||
if [[ ! -f "${kit_dir}/${f}" ]]; then
|
||||
echo "missing ${kit_dir}/${f}" >&2
|
||||
exit 1
|
||||
fi
|
||||
done
|
||||
|
||||
manifest_sha="$(sha256sum "${kit_dir}/manifest.json" | awk '{print $1}')"
|
||||
expected_sha="$(awk '{print $1}' "${kit_dir}/manifest.sha256")"
|
||||
|
||||
if [[ "${manifest_sha}" != "${expected_sha}" ]]; then
|
||||
echo "manifest hash mismatch: expected ${expected_sha} got ${manifest_sha}" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
python - <<'PY' "${kit_dir}" "${manifest_sha}"
|
||||
import base64, hashlib, json, sys, pathlib
|
||||
|
||||
kit = pathlib.Path(sys.argv[1])
|
||||
manifest_sha = sys.argv[2]
|
||||
manifest = json.loads(kit.joinpath("manifest.json").read_text())
|
||||
|
||||
errors = []
|
||||
selectors = manifest.get("selectors", {})
|
||||
tenants = selectors.get("tenants", [])
|
||||
tenant = manifest.get("tenant")
|
||||
|
||||
if tenant and tenants and tenant not in tenants and "*" not in tenants[0]:
|
||||
errors.append(f"tenant {tenant} not included in selectors.tenants {tenants}")
|
||||
|
||||
digests = []
|
||||
for item in manifest.get("contents", []):
|
||||
digest = item.get("digest", "")
|
||||
if not digest.startswith("sha256:"):
|
||||
errors.append(f"invalid digest format for {item.get('path')}")
|
||||
continue
|
||||
digests.append(digest.split("sha256:")[1])
|
||||
|
||||
rerun_calc = hashlib.sha256("\n".join(sorted(digests)).encode()).hexdigest()
|
||||
rerun_expected = manifest.get("rerunHash")
|
||||
if rerun_expected != f"sha256:{rerun_calc}":
|
||||
errors.append(f"rerunHash mismatch: expected sha256:{rerun_calc} got {rerun_expected}")
|
||||
|
||||
annotations = manifest["integrity"]["oci"]["annotations"]
|
||||
manifest_digest_hex = annotations["io.stellaops.export.manifest-digest"].split("sha256:")[1]
|
||||
digest_hdr = manifest["integrity"]["httpHeaders"]["Digest"]
|
||||
expected_hdr = "sha-256=" + base64.b64encode(bytes.fromhex(manifest_digest_hex)).decode()
|
||||
if digest_hdr != expected_hdr:
|
||||
errors.append("Digest header does not match manifest digest annotation")
|
||||
|
||||
log_meta = manifest.get("attestations", {}).get("log")
|
||||
if not log_meta:
|
||||
errors.append("attestation log metadata missing")
|
||||
|
||||
if not manifest.get("quotas"):
|
||||
errors.append("quotas/backpressure block missing")
|
||||
|
||||
if len(set(tenants)) > 1 and not manifest.get("approval", {}).get("required"):
|
||||
errors.append("cross-tenant approval required but not present")
|
||||
|
||||
if errors:
|
||||
for err in errors:
|
||||
print(err)
|
||||
sys.exit(1)
|
||||
|
||||
print("manifest checks ok")
|
||||
PY
|
||||
|
||||
python - <<'PY' "${kit_dir}" "${manifest_sha}"
|
||||
import json, sys, pathlib
|
||||
|
||||
kit = pathlib.Path(sys.argv[1])
|
||||
manifest_sha = sys.argv[2]
|
||||
prov = json.loads(kit.joinpath("provenance.json").read_text())
|
||||
|
||||
manifest_subject = prov["subject"][0]["digest"]["sha256"]
|
||||
if manifest_subject != manifest_sha:
|
||||
print(f"provenance manifest digest mismatch: {manifest_subject} vs {manifest_sha}")
|
||||
sys.exit(1)
|
||||
|
||||
log = prov["predicate"]["environment"]["logs"]
|
||||
required = ["kind", "logId", "logIndex", "entryDigest", "timestamp"]
|
||||
missing = [k for k in required if k not in log]
|
||||
if missing:
|
||||
print(f"provenance log metadata missing keys: {missing}")
|
||||
sys.exit(1)
|
||||
|
||||
print("provenance checks ok")
|
||||
PY
|
||||
|
||||
echo "verify-export-kit: PASS"
|
||||
@@ -102,12 +102,12 @@ Selectors (time windows, tenants, products, SBOM subjects, ecosystems) are suppl
|
||||
- **Constraints:** Requires the base manifest to exist in object storage or artifact registry accessible to the worker. Fails with `ERR_EXPORT_BASE_MISSING` otherwise.
|
||||
- **Workflow:** Ideal for frequent updates to mirrored environments with limited bandwidth.
|
||||
|
||||
## Compatibility and guardrails
|
||||
- **Aggregation-Only Contract:** All profiles respect AOC boundaries: raw evidence is never mutated. Policy outputs are appended separately with clear provenance.
|
||||
- **Tenant scoping:** Profiles are tenant-specific. Cross-tenant exports require explicit administrative approval and signed justification.
|
||||
- **Retriable runs:** Re-running a profile with identical selectors yields matching manifests and hashes, facilitating verify-on-download workflows.
|
||||
- **Offline operation:** JSON and mirror profiles function in offline mode without additional configuration. Trivy profiles require pre-seeded schema metadata shipped via Offline Kit.
|
||||
- **Quota integration:** Profiles can define run quotas (per tenant per day). Quota exhaustion surfaces as `429 Too Many Requests` with `X-Stella-Quota-*` hints.
|
||||
## Compatibility and guardrails
|
||||
- **Aggregation-Only Contract:** All profiles respect AOC boundaries: raw evidence is never mutated. Policy outputs are appended separately with clear provenance.
|
||||
- **Tenant scoping + approvals:** Profiles are tenant-specific. When `selectors.tenants` includes additional tenants or wildcards, `approval.required=true` plus `approvedBy` and `ticket` must be present (validated by `verify-export-kit.sh` and schema).
|
||||
- **Retriable runs:** Re-running a profile with identical selectors yields matching manifests and hashes, facilitating verify-on-download workflows.
|
||||
- **Offline operation:** JSON and mirror profiles function in offline mode without additional configuration. Trivy profiles require pre-seeded schema metadata shipped via Offline Kit.
|
||||
- **Quota integration and backpressure:** Profiles declare `limits.maxActiveRuns`, `limits.maxQueuedRuns`, and `backpressureMode` (`reject`|`defer`|`throttle`). When limits trigger, exporters emit `429` with `X-Stella-Quota-*` plus `Retry-After` to keep retries deterministic.
|
||||
|
||||
## Example profile definition (CLI)
|
||||
|
||||
|
||||
@@ -12,6 +12,7 @@ Export Center runs emit deterministic manifests, provenance records, and signatu
|
||||
- **Traceability.** Provenance links each bundle to the inputs that produced it: tenant, findings ledger queries, policy snapshots, SBOM identifiers, adapter versions, and encryption recipients.
|
||||
- **Determinism.** Canonical JSON (sorted keys, RFC 3339 UTC timestamps, normalized numbers) guarantees byte-for-byte stability across reruns with identical input.
|
||||
- **Portability.** Signatures and attestations travel with filesystem bundles, OCI artefacts, and Offline Kit staging trees. Verification does not require online Authority access when the bundle includes the cosign public key.
|
||||
- **Transparency metadata.** DSSE/SLSA artefacts must embed log metadata (hashedrekord/rekor-style `logId`, `logIndex`, `entryDigest`, `timestamp`) so offline kits can prove submission intent even without online verification.
|
||||
|
||||
---
|
||||
|
||||
@@ -38,8 +39,9 @@ All digests use lowercase hex SHA-256 (`sha256:<digest>`). When bundle encryptio
|
||||
- Provenance `subjects[]` contains both manifest hash and bundle/archive hash.
|
||||
3. **Key retrieval.** Worker obtains a short-lived signing token from Authority’s KMS client using tenant-scoped credentials (`export.sign` scope). Keys live in Authority or tenant-specific HSMs depending on deployment.
|
||||
4. **Signature emission.** Cosign generates detached signatures (`*.sig`). If DSSE is enabled, cosign wraps payload bytes in a DSSE envelope (`*.dsse`). Attestations follow the SLSA Level 2 provenance template; Level 3 requires builder metadata (`EXPORT-SVC-37-002` optional feature flag).
|
||||
5. **Storage & distribution.** Signatures and attestations are written alongside manifests in object storage, included in filesystem bundles, and attached as OCI artefact layers/annotations.
|
||||
6. **Audit trail.** Run metadata captures signer identity (`signing_key_id`), cosign certificate serial, signature timestamps, and verification hints. Console/CLI surface these details for downstream automation.
|
||||
5. **Log metadata.** DSSE/SLSA outputs record log hints: `{kind: "hashedrekord", logId, logIndex, entryDigest, timestamp}`. For air-gap deployments the hints ride inside the attestation; when online, the same values come from Rekor receipts.
|
||||
6. **Storage & distribution.** Signatures and attestations are written alongside manifests in object storage, included in filesystem bundles, and attached as OCI artefact layers/annotations.
|
||||
7. **Audit trail.** Run metadata captures signer identity (`signing_key_id`), cosign certificate serial, signature timestamps, log hints, and verification outcomes. Console/CLI surface these details for downstream automation.
|
||||
|
||||
> **Key management.** Secrets and key references are configured per tenant via `export.signing`, pointing to Authority clients or external HSM aliases. Offline deployments pre-load cosign public keys into the bundle (`signatures/pubkeys/{tenant}.pem`).
|
||||
|
||||
@@ -61,6 +63,7 @@ All digests use lowercase hex SHA-256 (`sha256:<digest>`). When bundle encryptio
|
||||
| `predicate.metadata.reproducible` | Always `true`—workers guarantee determinism. |
|
||||
| `predicate.environment.encryption` | Records encryption recipients, wrapped keys, algorithm (`age` or `aes-gcm`). |
|
||||
| `predicate.environment.kms` | Signing key identifier (`authority://tenant/export-signing-key`) and certificate chain fingerprints. |
|
||||
| `predicate.environment.logs` | Transparency metadata `{kind,logId,logIndex,entryDigest,timestamp}` required by EC3 to keep DSSE verifiable offline. |
|
||||
|
||||
Sample (abridged):
|
||||
|
||||
|
||||
254
docs/modules/export-center/schemas/export-manifest.schema.json
Normal file
254
docs/modules/export-center/schemas/export-manifest.schema.json
Normal file
@@ -0,0 +1,254 @@
|
||||
{
|
||||
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
||||
"$id": "https://stellaops.io/schemas/export-center/export-manifest.schema.json",
|
||||
"title": "StellaOps Export Manifest",
|
||||
"description": "Schema for run manifests, attestations, integrity headers, and quota/backpressure metadata (EC2–EC9).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"schema",
|
||||
"version",
|
||||
"exportId",
|
||||
"profile",
|
||||
"tenant",
|
||||
"selectors",
|
||||
"generatedAt",
|
||||
"contents"
|
||||
],
|
||||
"properties": {
|
||||
"schema": { "type": "string", "const": "https://stellaops.io/export-center/manifest/v1alpha2" },
|
||||
"version": { "type": "string", "pattern": "^1\\.1\\.[0-9]+$" },
|
||||
"exportId": { "type": "string", "pattern": "^[a-z0-9-]{6,64}$" },
|
||||
"runId": { "type": "string", "pattern": "^[a-z0-9-]{6,64}$" },
|
||||
"profile": {
|
||||
"type": "object",
|
||||
"required": ["kind", "variant", "name"],
|
||||
"properties": {
|
||||
"kind": { "type": "string", "enum": ["json", "trivy", "mirror", "devportal", "attestation"] },
|
||||
"variant": {
|
||||
"type": "string",
|
||||
"enum": ["raw", "policy", "db", "java-db", "full", "delta", "offline", "bundle"]
|
||||
},
|
||||
"name": { "type": "string", "pattern": "^[a-z0-9]([a-z0-9-]*[a-z0-9])?$" },
|
||||
"revision": { "type": "string", "pattern": "^r[0-9]+$" }
|
||||
},
|
||||
"additionalProperties": false
|
||||
},
|
||||
"tenant": { "type": "string", "pattern": "^[a-z0-9]([a-z0-9-]*[a-z0-9])?$" },
|
||||
"selectors": { "$ref": "#/$defs/selectors" },
|
||||
"generatedAt": { "type": "string", "format": "date-time" },
|
||||
"rerunHash": { "type": "string", "pattern": "^sha256:[a-f0-9]{64}$" },
|
||||
"contents": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "object",
|
||||
"required": ["path", "digest", "bytes"],
|
||||
"properties": {
|
||||
"path": { "type": "string", "pattern": "^[A-Za-z0-9._/-]+$" },
|
||||
"digest": { "$ref": "#/$defs/digest" },
|
||||
"bytes": { "type": "integer", "minimum": 0 },
|
||||
"records": { "type": "integer", "minimum": 0 },
|
||||
"contentType": { "type": "string" }
|
||||
},
|
||||
"additionalProperties": false
|
||||
}
|
||||
},
|
||||
"delta": {
|
||||
"type": "object",
|
||||
"required": ["baseExportId", "baseManifestDigest", "tombstones"],
|
||||
"properties": {
|
||||
"baseExportId": { "type": "string", "pattern": "^[a-z0-9-]{6,64}$" },
|
||||
"baseManifestDigest": { "$ref": "#/$defs/digest" },
|
||||
"tombstones": {
|
||||
"type": "array",
|
||||
"items": { "type": "string", "pattern": "^[A-Za-z0-9._/-]+$" }
|
||||
},
|
||||
"added": {
|
||||
"type": "array",
|
||||
"items": { "type": "string", "pattern": "^[A-Za-z0-9._/-]+$" }
|
||||
},
|
||||
"removed": {
|
||||
"type": "array",
|
||||
"items": { "type": "string", "pattern": "^[A-Za-z0-9._/-]+$" }
|
||||
}
|
||||
},
|
||||
"additionalProperties": false
|
||||
},
|
||||
"integrity": {
|
||||
"type": "object",
|
||||
"required": ["httpHeaders", "oci"],
|
||||
"properties": {
|
||||
"httpHeaders": {
|
||||
"type": "object",
|
||||
"required": ["Digest", "X-Stella-Signature"],
|
||||
"properties": {
|
||||
"Digest": { "type": "string", "pattern": "^sha-256=[A-Za-z0-9+/=]+$" },
|
||||
"X-Stella-Signature": { "type": "string" },
|
||||
"X-Stella-Immutability": { "type": "string" }
|
||||
},
|
||||
"additionalProperties": false
|
||||
},
|
||||
"oci": {
|
||||
"type": "object",
|
||||
"required": ["annotations"],
|
||||
"properties": {
|
||||
"annotations": {
|
||||
"type": "object",
|
||||
"required": [
|
||||
"io.stellaops.export.profile",
|
||||
"io.stellaops.export.run",
|
||||
"io.stellaops.export.manifest-digest",
|
||||
"io.stellaops.export.provenance-ref"
|
||||
],
|
||||
"properties": {
|
||||
"io.stellaops.export.profile": { "type": "string" },
|
||||
"io.stellaops.export.run": { "type": "string" },
|
||||
"io.stellaops.export.manifest-digest": { "$ref": "#/$defs/digest" },
|
||||
"io.stellaops.export.provenance-ref": { "type": "string" },
|
||||
"org.opencontainers.image.ref.name": { "type": "string" }
|
||||
}
|
||||
}
|
||||
},
|
||||
"additionalProperties": false
|
||||
}
|
||||
},
|
||||
"additionalProperties": false
|
||||
},
|
||||
"attestations": {
|
||||
"type": "object",
|
||||
"required": ["provenanceRef", "dsseEnvelope", "slsaLevel", "log"],
|
||||
"properties": {
|
||||
"provenanceRef": { "type": "string" },
|
||||
"dsseEnvelope": { "type": "string" },
|
||||
"slsaLevel": { "type": "string" },
|
||||
"log": {
|
||||
"type": "object",
|
||||
"required": ["kind", "logId", "logIndex", "entryDigest", "timestamp"],
|
||||
"properties": {
|
||||
"kind": { "type": "string", "enum": ["hashedrekord", "rekor"] },
|
||||
"logId": { "type": "string" },
|
||||
"logIndex": { "type": "integer", "minimum": 0 },
|
||||
"entryDigest": { "$ref": "#/$defs/digest" },
|
||||
"timestamp": { "type": "string", "format": "date-time" }
|
||||
},
|
||||
"additionalProperties": false
|
||||
}
|
||||
},
|
||||
"additionalProperties": false
|
||||
},
|
||||
"distribution": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"http": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"enabled": { "type": "boolean" },
|
||||
"retentionDays": { "type": "integer", "minimum": 1, "maximum": 3650 },
|
||||
"etag": { "type": "string" },
|
||||
"rangeRequests": { "type": "boolean" }
|
||||
},
|
||||
"additionalProperties": false
|
||||
},
|
||||
"oci": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"enabled": { "type": "boolean" },
|
||||
"reference": { "type": "string" }
|
||||
},
|
||||
"additionalProperties": false
|
||||
},
|
||||
"object": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"enabled": { "type": "boolean" },
|
||||
"bucket": { "type": "string" },
|
||||
"prefix": { "type": "string" }
|
||||
},
|
||||
"additionalProperties": false
|
||||
}
|
||||
},
|
||||
"additionalProperties": false
|
||||
},
|
||||
"encryption": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"mode": { "type": "string", "enum": ["age", "aes-gcm"] },
|
||||
"recipients": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "object",
|
||||
"required": ["keyId", "fingerprint"],
|
||||
"properties": {
|
||||
"keyId": { "type": "string" },
|
||||
"fingerprint": { "type": "string" },
|
||||
"wrappedKey": { "type": "string" }
|
||||
},
|
||||
"additionalProperties": false
|
||||
}
|
||||
},
|
||||
"strict": { "type": "boolean" }
|
||||
},
|
||||
"additionalProperties": false
|
||||
},
|
||||
"approval": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"required": { "type": "boolean" },
|
||||
"reason": { "type": "string" },
|
||||
"approvedBy": { "type": "string" },
|
||||
"ticket": { "type": "string" }
|
||||
},
|
||||
"additionalProperties": false
|
||||
},
|
||||
"quotas": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"maxActiveRuns": { "type": "integer", "minimum": 1, "maximum": 32 },
|
||||
"maxQueuedRuns": { "type": "integer", "minimum": 1, "maximum": 512 },
|
||||
"backpressureMode": {
|
||||
"type": "string",
|
||||
"enum": ["reject", "defer", "throttle"]
|
||||
},
|
||||
"cpuThrottlePercent": { "type": "integer", "minimum": 1, "maximum": 100 }
|
||||
},
|
||||
"additionalProperties": false
|
||||
}
|
||||
},
|
||||
"additionalProperties": false,
|
||||
"$defs": {
|
||||
"digest": { "type": "string", "pattern": "^sha256:[a-f0-9]{64}$" },
|
||||
"selectors": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"tenants": {
|
||||
"type": "array",
|
||||
"items": { "type": "string", "pattern": "^[a-z0-9*.-]+$" },
|
||||
"uniqueItems": true
|
||||
},
|
||||
"products": {
|
||||
"type": "array",
|
||||
"items": { "type": "string", "pattern": "^pkg:[A-Za-z0-9.+\\-_/:@*]+$" }
|
||||
},
|
||||
"timeWindow": {
|
||||
"oneOf": [
|
||||
{ "type": "string", "pattern": "^[0-9]+d$" },
|
||||
{ "type": "string", "pattern": "^[0-9]{4}-[0-9]{2}-[0-9]{2}/[0-9]{4}-[0-9]{2}-[0-9]{2}$" }
|
||||
]
|
||||
},
|
||||
"severities": {
|
||||
"type": "array",
|
||||
"items": { "type": "string", "enum": ["critical", "high", "medium", "low", "info"] },
|
||||
"uniqueItems": true
|
||||
},
|
||||
"ecosystems": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "string",
|
||||
"enum": ["npm", "maven", "pypi", "nuget", "go", "cargo", "rpm", "deb", "apk", "java"]
|
||||
},
|
||||
"uniqueItems": true
|
||||
}
|
||||
},
|
||||
"additionalProperties": false
|
||||
}
|
||||
}
|
||||
}
|
||||
206
docs/modules/export-center/schemas/export-profile.schema.json
Normal file
206
docs/modules/export-center/schemas/export-profile.schema.json
Normal file
@@ -0,0 +1,206 @@
|
||||
{
|
||||
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
||||
"$id": "https://stellaops.io/schemas/export-center/export-profile.schema.json",
|
||||
"title": "StellaOps ExportProfile",
|
||||
"description": "Canonical schema for Export Center profile definitions with selector and approval guardrails (EC1, EC4, EC9).",
|
||||
"type": "object",
|
||||
"required": ["apiVersion", "kind", "metadata", "spec"],
|
||||
"properties": {
|
||||
"apiVersion": {
|
||||
"type": "string",
|
||||
"const": "stellaops.io/export.v1"
|
||||
},
|
||||
"kind": {
|
||||
"type": "string",
|
||||
"const": "ExportProfile"
|
||||
},
|
||||
"metadata": {
|
||||
"type": "object",
|
||||
"required": ["name", "tenant"],
|
||||
"properties": {
|
||||
"name": {
|
||||
"type": "string",
|
||||
"minLength": 3,
|
||||
"maxLength": 64,
|
||||
"pattern": "^[a-z0-9]([a-z0-9-]*[a-z0-9])?$"
|
||||
},
|
||||
"tenant": {
|
||||
"type": "string",
|
||||
"pattern": "^[a-z0-9]([a-z0-9-]*[a-z0-9])?$"
|
||||
},
|
||||
"revision": {
|
||||
"type": "string",
|
||||
"pattern": "^r[0-9]+$"
|
||||
},
|
||||
"labels": {
|
||||
"type": "object",
|
||||
"additionalProperties": {
|
||||
"type": "string",
|
||||
"maxLength": 128
|
||||
}
|
||||
}
|
||||
},
|
||||
"additionalProperties": false
|
||||
},
|
||||
"spec": {
|
||||
"type": "object",
|
||||
"required": ["kind", "variant", "distribution"],
|
||||
"properties": {
|
||||
"kind": {
|
||||
"type": "string",
|
||||
"enum": ["json", "trivy", "mirror", "devportal", "attestation"]
|
||||
},
|
||||
"variant": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"raw",
|
||||
"policy",
|
||||
"db",
|
||||
"java-db",
|
||||
"full",
|
||||
"delta",
|
||||
"offline",
|
||||
"bundle"
|
||||
]
|
||||
},
|
||||
"distribution": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "string",
|
||||
"enum": ["http", "oci", "object"]
|
||||
},
|
||||
"uniqueItems": true,
|
||||
"minItems": 1
|
||||
},
|
||||
"compression": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"codec": {
|
||||
"type": "string",
|
||||
"enum": ["zstd", "gzip", "none"]
|
||||
},
|
||||
"level": {
|
||||
"type": "integer",
|
||||
"minimum": 1,
|
||||
"maximum": 22
|
||||
}
|
||||
},
|
||||
"additionalProperties": false
|
||||
},
|
||||
"encryption": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"enabled": { "type": "boolean" },
|
||||
"mode": { "type": "string", "enum": ["age", "aes-gcm"] },
|
||||
"recipientKeys": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "string",
|
||||
"pattern": "^(age1|kms://)"
|
||||
}
|
||||
},
|
||||
"strict": { "type": "boolean" }
|
||||
},
|
||||
"additionalProperties": false
|
||||
},
|
||||
"retention": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"mode": { "type": "string", "enum": ["days", "never"] },
|
||||
"value": { "type": "integer", "minimum": 1, "maximum": 3650 }
|
||||
},
|
||||
"additionalProperties": false
|
||||
},
|
||||
"limits": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"maxActiveRuns": { "type": "integer", "minimum": 1, "maximum": 32 },
|
||||
"maxQueuedRuns": { "type": "integer", "minimum": 1, "maximum": 512 },
|
||||
"backpressureMode": {
|
||||
"type": "string",
|
||||
"enum": ["reject", "defer", "throttle"]
|
||||
}
|
||||
},
|
||||
"additionalProperties": false
|
||||
},
|
||||
"selectors": { "$ref": "#/$defs/selectors" },
|
||||
"approval": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"required": { "type": "boolean" },
|
||||
"reason": { "type": "string", "maxLength": 256 },
|
||||
"ticket": { "type": "string", "maxLength": 64 },
|
||||
"approver": { "type": "string", "maxLength": 64 }
|
||||
},
|
||||
"additionalProperties": false
|
||||
},
|
||||
"schemaVersion": {
|
||||
"type": "string",
|
||||
"enum": ["1.1.0"],
|
||||
"default": "1.1.0"
|
||||
}
|
||||
},
|
||||
"additionalProperties": false
|
||||
}
|
||||
},
|
||||
"additionalProperties": false,
|
||||
"$defs": {
|
||||
"selectors": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"tenants": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "string",
|
||||
"pattern": "^[a-z0-9*.-]+$"
|
||||
},
|
||||
"uniqueItems": true
|
||||
},
|
||||
"products": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "string",
|
||||
"pattern": "^pkg:[A-Za-z0-9.+\\-_/:@*]+$"
|
||||
}
|
||||
},
|
||||
"ecosystems": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"npm",
|
||||
"maven",
|
||||
"pypi",
|
||||
"nuget",
|
||||
"go",
|
||||
"cargo",
|
||||
"rpm",
|
||||
"deb",
|
||||
"apk",
|
||||
"java"
|
||||
]
|
||||
},
|
||||
"uniqueItems": true
|
||||
},
|
||||
"timeWindow": {
|
||||
"oneOf": [
|
||||
{ "type": "string", "pattern": "^[0-9]+d$" },
|
||||
{
|
||||
"type": "string",
|
||||
"pattern": "^[0-9]{4}-[0-9]{2}-[0-9]{2}/[0-9]{4}-[0-9]{2}-[0-9]{2}$"
|
||||
}
|
||||
]
|
||||
},
|
||||
"severities": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "string",
|
||||
"enum": ["critical", "high", "medium", "low", "info"]
|
||||
},
|
||||
"uniqueItems": true
|
||||
}
|
||||
},
|
||||
"additionalProperties": false
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -152,11 +152,11 @@ The Java supplement only includes ecosystems `maven`, `gradle`, `sbt`. Additiona
|
||||
|
||||
## 4. Compatibility matrix
|
||||
|
||||
| Trivy version | Schema version | Supported by adapter | Notes |
|
||||
|---------------|----------------|----------------------|-------|
|
||||
| 0.46.x | 2 | Yes | Baseline compatibility target. |
|
||||
| 0.50.x | 2 | Yes | Default validation target in CI. |
|
||||
| 0.51.x+ | 3 | Pending | Adapter throws `ERR_EXPORT_UNSUPPORTED_SCHEMA` until implemented. |
|
||||
| Trivy version | Schema version | Supported by adapter | Notes |
|
||||
|---------------|----------------|----------------------|-------|
|
||||
| 0.46.x | 2 (pinned) | Yes | Baseline compatibility target. |
|
||||
| 0.50.x | 2 (pinned) | Yes | Default validation target in CI and fixtures. |
|
||||
| 0.51.x+ | 3 | Pending | Adapter throws `ERR_EXPORT_UNSUPPORTED_SCHEMA` until implemented or explicitly overridden. |
|
||||
|
||||
Schema mismatches emit `adapter.trivy.unsupported_schema_version` and abort the run. Operators can pin the schema via `ExportCenter:Adapters:Trivy:SchemaVersion`.
|
||||
|
||||
@@ -169,10 +169,13 @@ Schema mismatches emit `adapter.trivy.unsupported_schema_version` and abort the
|
||||
- Generate bundle from fixture dataset.
|
||||
- Run `trivy module db import <bundle>` (Trivy CLI) to ensure the bundle is accepted.
|
||||
- For Java DB, run `trivy java-repo --db <bundle>` against sample repository.
|
||||
3. **CI smoke (`DEVOPS-EXPORT-36-001`)**:
|
||||
- Validate metadata fields using `jq`.
|
||||
- Ensure signatures verify with `cosign`.
|
||||
- Check runtime by invoking `trivy fs --cache-dir <temp> --skip-update --custom-db <bundle> fixtures/image`.
|
||||
3. **CI smoke (`DEVOPS-EXPORT-36-001`)**:
|
||||
- Validate metadata fields using `jq`.
|
||||
- Ensure signatures verify with `cosign`.
|
||||
- Check runtime by invoking `trivy fs --cache-dir <temp> --skip-update --custom-db <bundle> fixtures/image`.
|
||||
4. **Schema pinning (EC6)**:
|
||||
- CI enforces `ExportCenter:Adapters:Trivy:SchemaVersion=2`; higher versions fail fast with `adapter.trivy.unsupported_schema_version`.
|
||||
- Export manifests/OCI annotations record the pinned schema for rerun-hash stability.
|
||||
|
||||
Failures set the run status to `failed` with `errorCode="adapter-trivy"` so Console/CLI expose the reason.
|
||||
|
||||
|
||||
@@ -15,6 +15,8 @@
|
||||
- `sbom_snapshots` (immutable versions; tenant + artifact + digest + createdAt)
|
||||
- `sbom_projections` (materialised views keyed by snapshotId, entrypoint/service node flags)
|
||||
- `sbom_assets` (asset metadata, criticality/owner/env/exposure; append-only history)
|
||||
- `sbom_catalog` (console catalog surface; indexed by artifact, scope, license, assetTags.*, createdAt for deterministic pagination)
|
||||
- `sbom_component_neighbors` (component lookup graph edges; indexed by purl+artifact for cursor pagination)
|
||||
- `sbom_paths` (resolved dependency paths with runtime flags, blast-radius hints)
|
||||
- `sbom_events` (outbox for event delivery + watermark/backfill tracking)
|
||||
|
||||
@@ -84,7 +86,12 @@ Operational rules:
|
||||
- Logs: structured, include tenant + artifact digest + sbomVersion; classify ingest failures (schema, storage, orchestrator, validation).
|
||||
- Alerts: backlog thresholds for outbox/event delivery; high latency on path/timeline endpoints.
|
||||
|
||||
## 9) Open questions / dependencies
|
||||
## 9) Configuration (Mongo-backed catalog & lookup)
|
||||
- Enable Mongo storage for `/console/sboms` and `/components/lookup` by setting `SbomService:Mongo:ConnectionString` (env: `SBOM_SbomService__Mongo__ConnectionString`).
|
||||
- Optional overrides: `SbomService:Mongo:Database`, `SbomService:Mongo:CatalogCollection`, `SbomService:Mongo:ComponentLookupCollection`; defaults are `sbom_service`, `sbom_catalog`, `sbom_component_neighbors`.
|
||||
- When the connection string is absent the service falls back to fixture JSON or deterministic in-memory seeds to keep air-gapped workflows alive.
|
||||
|
||||
## 10) Open questions / dependencies
|
||||
- Confirm orchestrator pause/backfill contract (shared with Runtime & Signals 140-series).
|
||||
- Finalise storage collection names and indexes (compound on tenant+artifactDigest+version, TTL for transient staging).
|
||||
- Publish canonical LNM v1 fixtures and JSON schemas for projections and asset metadata.
|
||||
|
||||
Reference in New Issue
Block a user