35c8f9216f
- Introduced `BinaryReachabilityLifterTests` to validate binary lifting functionality. - Created `PackRunWorkerOptions` for configuring worker paths and execution persistence. - Added `TimelineIngestionOptions` for configuring NATS and Redis ingestion transports. - Implemented `NatsTimelineEventSubscriber` for subscribing to NATS events. - Developed `RedisTimelineEventSubscriber` for reading from Redis Streams. - Added `TimelineEnvelopeParser` to normalize incoming event envelopes. - Created unit tests for `TimelineEnvelopeParser` to ensure correct field mapping. - Implemented `TimelineAuthorizationAuditSink` for logging authorization outcomes.
86 lines
5.9 KiB
Markdown
86 lines
5.9 KiB
Markdown
# Deterministic SBOM Composition
|
|
|
|
> **Status:** Ready v1.0 (Sprint 136 linkage; fixtures dated 2025-12-03)
|
|
> **Owners:** Scanner Guild · DevEx/CLI Guild · UI Guild · Docs Guild · Security Guild
|
|
> **Related Tasks:** `SCANNER-SURFACE-04`, `SURFACE-FS-07`, `SCANNER-EMIT-15-001`, `SCANNER-SORT-02`, `CLI-SBOM-60-001`, `CLI-SBOM-60-002`, `UI-SBOM-DET-01`, `UI-POLICY-DET-01`, `DOCS-SCANNER-DET-01`, `DOCS-POLICY-DET-01`, `DOCS-CLI-DET-01`, `SCANNER-CRYPTO-90-002`, `SCANNER-CRYPTO-90-003`
|
|
|
|
## 1. Purpose
|
|
|
|
Guarantee that every container scan yields **provably deterministic** SBOM artifacts that can be verified offline. Each layer fragment is DSSE-signed before merge, `_composition.json` captures the canonical merge recipe, and the final CycloneDX inventory/usage SBOMs expose Merkle roots and `stella.contentHash` properties. CLI/UI/policy layers consume those signals to block non-deterministic releases and provide human-friendly diagnostics.
|
|
|
|
## 2. Scope
|
|
|
|
### 2.1 Fragment attestation
|
|
|
|
- Scanner Worker emits `layer.fragments` payloads as canonical JSON (lexicographic keys, compact whitespace, normalized timestamps, ordered component arrays).
|
|
- Each fragment is signed via DSSE (default Ed25519; PQ/Dilithium toggle routed through `ICryptoProviderRegistry`).
|
|
- `_composition.json` records `{layerDigest, fragmentSha256, dsseEnvelopeSha256}` per fragment alongside the overall Merkle root.
|
|
- Surface manifests append links to fragment DSSE envelopes so offline kits can fetch them without re-scan.
|
|
|
|
### 2.2 Canonical merge
|
|
|
|
- Merge order strictly follows `layerDigest` (ascending hex) and then `component.identity.purl` (fallback to `identity.key`).
|
|
- CycloneDX metadata gains:
|
|
- `properties["stellaops:stella.contentHash"]` for each fragment and composition root.
|
|
- `properties["stellaops:composition.manifest"]` referencing `_composition.json` CAS URI.
|
|
- `properties["stellaops:merkle.root"]` for the composed BOM.
|
|
- `ScannerTimestamps.Normalize` continues to zero fractional microseconds; SBOM timestamps should default to `"0001-01-01T00:00:00Z"` when no semantic timestamp is required.
|
|
|
|
### 2.3 Surface manifest extensions
|
|
|
|
- `SurfaceManifestArtifact` gains optional `attestations[]` with `{kind, mediaType, digest, uri}` for DSSE envelopes.
|
|
- `_composition.json` is published as an additional artifact kind `composition.recipe`.
|
|
- Surface reader/writer validate Merkle roots before caching manifest entries.
|
|
|
|
### 2.4 Tooling impacts
|
|
|
|
- **CLI (`stella sbomer ...`)**: adds `layer` and `compose` verbs, deterministic diff reporting, and offline verification per `_composition.json`.
|
|
- **UI/Policy**: determinism badge, drift diffs, and a policy gate that blocks releases when fragment DSSE/verifications fail.
|
|
- **Docs**: new guides under `docs/scanner` & `docs/cli` plus policy references detailing how to interpret determinism metadata.
|
|
- **Crypto**: PQ-friendly DSSE toggle delivered via `SCANNER-CRYPTO-90-002/003` so sovereign bundles can select Dilithium/Falcon.
|
|
|
|
## 3. Verification Flow (offline kit)
|
|
|
|
1. Verify DSSE on each fragment (using `verifiers.json`).
|
|
2. Recompute `sha256(c14n(fragment))` and compare with `_composition.json`.
|
|
3. Re-run composition locally (using canonical ordering) and compare `sha256(c14n(composed))` against `manifest.properties["stellaops:merkle.root"]`.
|
|
4. Optionally validate provided Merkle proofs (leaf → root) and attest that the UI/Policy gate marked the scan as deterministic.
|
|
|
|
### 3.1 Reference fixture (deterministic-compose)
|
|
- Path: `docs/modules/scanner/fixtures/deterministic-compose/` (generated 2025-12-03 by `generate.py`).
|
|
- Quick verify:
|
|
```bash
|
|
cd docs/modules/scanner/fixtures/deterministic-compose
|
|
sha256sum -c hashes.txt
|
|
jq -r '.payload' fragment-layer1.dsse.json | base64 -d | sha256sum
|
|
jq -r '.merkleRootSha256' _composition.json
|
|
jq -r '.properties[] | select(.name=="stellaops:merkle.root").value' bom.cdx.json
|
|
```
|
|
Expected Merkle root: `963e421d21be2db87895ea5fd973a0ad71aa638499c274308e013d2b6c8243f6` (matches `_composition.json` and `bom.cdx.json`).
|
|
- Regenerate deterministically: `python generate.py && sha256sum -c hashes.txt` (standard library only).
|
|
|
|
## 4. Deliverables Checklist
|
|
|
|
| Area | Deliverable |
|
|
| --- | --- |
|
|
| Scanner Worker | DSSE per fragment, `_composition.json`, canonical fragment serializer, Surface manifest updates |
|
|
| Emit pipeline | Layer-sorted composition, `stella.contentHash`, Merkle metadata, PQ-aware signing hooks |
|
|
| CLI | `stella sbomer layer/compose/drift`, verification commands, documentation |
|
|
| UI | Determinism badge, drift diagnostics, policy gate wiring |
|
|
| Docs | Updated scanner/cli/policy guides, offline kit instructions |
|
|
| Tests | Regression suites covering canonicalization, DSSE verification, PQ keypaths, Merkle roots |
|
|
|
|
## 5. Operational workflow (worker → CLI/UI/Policy)
|
|
- **Worker**: emit fragment DSSE + `_composition.json` into the surface manifest; persist `stellaops:composition.manifest` and `stellaops:merkle.root` properties on composed BOMs so downstream consumers do not recompute merges.
|
|
- **CLI**: verify bundles offline with `stella sbomer compose --recipe docs/modules/scanner/fixtures/deterministic-compose/_composition.json --fragments-dir docs/modules/scanner/fixtures/deterministic-compose --verify` (see `docs/cli/sbomer.md`). The command should fail if any DSSE signature, Merkle root, or BOM hash diverges.
|
|
- **UI / Policy**: render determinism badge using `stellaops:merkle.root`; block promotion when `_composition.json` is missing or hashes disagree; expose drift diagnostics by recomputing composition locally and comparing to BOM properties.
|
|
- **Export/Offline**: include `_composition.json`, fragment DSSEs, `bom.cdx.json`, and `hashes.txt` when building Offline Kit bundles so replay jobs can validate without network.
|
|
|
|
## 6. References
|
|
|
|
- `docs/modules/scanner/architecture.md`
|
|
- `docs/modules/scanner/design/surface-fs.md`
|
|
- `docs/replay/DETERMINISTIC_REPLAY.md`
|
|
- `docs/modules/cli/architecture.md`
|
|
- `docs/modules/policy/architecture.md`
|