save checkpoint
This commit is contained in:
master
@@ -55,6 +55,7 @@ Key settings:
|
||||
- Offline Kit: `../../OFFLINE_KIT.md`
|
||||
- Mirror: `../mirror/`
|
||||
- ExportCenter: `../export-center/`
|
||||
- Promotion Rekor tile runbook: `./guides/promotion-rekor-tile-verification.md`
|
||||
|
||||
## Evidence Bundles for Air-Gapped Verification
|
||||
|
||||
|
||||
@@ -0,0 +1,75 @@
|
||||
# Promotion Rekor Tile Verification (Air-Gap)
|
||||
|
||||
## Purpose
|
||||
|
||||
Operational runbook for using Rekor tile material in air-gapped promotion gates.
|
||||
|
||||
## Preconditions
|
||||
|
||||
- Offline bundle includes tile/proof artifacts and trust roots.
|
||||
- Promotion gate is configured to consume offline proof references.
|
||||
- Operator has tenant-scoped Authority credentials.
|
||||
|
||||
## Inputs
|
||||
|
||||
- Promotion identifier
|
||||
- Evidence bundle identifier
|
||||
- Rekor tile/proof bundle from offline sync
|
||||
- Trust root set (Fulcio/KMS roots and Rekor checkpoint material)
|
||||
|
||||
## Procedure
|
||||
|
||||
1. Validate bundle integrity.
|
||||
2. Import tile/proof files into the local Attestor cache.
|
||||
3. Run offline verification for referenced DSSE envelopes.
|
||||
4. Attach verification outputs to promotion gate input payload.
|
||||
5. Execute promotion gate evaluation.
|
||||
6. Persist decision record with proof references.
|
||||
|
||||
## Example Commands
|
||||
|
||||
```bash
|
||||
# 1) verify portable evidence bundle
|
||||
stella evidence verify --bundle portable-evidence-bundle.tgz --offline
|
||||
|
||||
# 2) import tile material
|
||||
stella rekor tiles import --bundle rekor-tiles.tgz
|
||||
|
||||
# 3) verify inclusion proofs offline
|
||||
stella rekor verify --offline --evidence-bundle-id <bundle-id>
|
||||
|
||||
# 4) run promotion gate preview with offline verification enabled
|
||||
stella promotion preview-gates --promotion <promotion-id> --offline-rekor
|
||||
```
|
||||
|
||||
## Failure Modes
|
||||
|
||||
| Failure mode | Expected gate behavior | Operator action |
|
||||
| --- | --- | --- |
|
||||
| Missing tile/proof files | Fail closed (deny or pending per policy) | Re-sync offline bundle and retry verification |
|
||||
| Invalid proof chain | Fail closed | Rotate trust roots or investigate tampering |
|
||||
| Expired trust roots | Fail closed | Import updated trust bundle from connected zone |
|
||||
| Break-glass enabled | Explicitly auditable non-standard path | Record reason/ticket and time-bound override |
|
||||
|
||||
## Offline QA Matrix (Deterministic)
|
||||
|
||||
1. Valid tile/proof bundle produces identical verification output hash across repeated runs.
|
||||
2. Missing tile segment fails closed with stable reason code.
|
||||
3. Tampered inclusion proof fails closed with stable reason code.
|
||||
4. Expired trust root fails closed with stable reason code.
|
||||
5. Break-glass path emits explicit marker and does not masquerade as standard verification.
|
||||
|
||||
## Audit Outputs
|
||||
|
||||
- Promotion decision record id
|
||||
- Policy decision digest
|
||||
- Evidence bundle id
|
||||
- Rekor verification report reference
|
||||
- Break-glass marker (if used)
|
||||
|
||||
## Related References
|
||||
|
||||
- `docs/modules/airgap/README.md`
|
||||
- `docs/modules/airgap/guides/proof-chain-verification.md`
|
||||
- `docs/modules/evidence-locker/promotion-evidence-contract.md`
|
||||
- `docs/modules/release-orchestrator/promotion-runtime-gap-closure-plan.md`
|
||||
@@ -7,6 +7,9 @@
|
||||
|
||||
The Cartographer service materializes immutable SBOM property graphs, precomputes layout tiles, and hydrates policy and VEX overlays so other services (API, UI, CLI) can navigate and reason about dependency relationships with context.
|
||||
|
||||
Boundary note: Cartographer is not the source of truth for environment topology
|
||||
or promotion lanes; those are owned by Release Orchestrator ENVMGR/PROMOT.
|
||||
|
||||
## Components
|
||||
|
||||
**Services:**
|
||||
|
||||
@@ -220,9 +220,10 @@ public interface IBaselineResolver
|
||||
|
||||
### 2.5 Verification
|
||||
|
||||
* `verify attestation --uuid <rekor-uuid> | --artifact <sha256> | --bundle <path>` — call **Attestor /verify** and print proof summary.
|
||||
* `verify referrers <digest>` — ask **Signer /verify/referrers** (is image Stella‑signed?).
|
||||
* `verify image-signature <ref|digest>` — standalone cosign verification (optional, local).
|
||||
* `verify attestation --uuid <rekor-uuid> | --artifact <sha256> | --bundle <path>` — call **Attestor /verify** and print proof summary.
|
||||
* `verify referrers <digest>` — ask **Signer /verify/referrers** (is image Stella‑signed?).
|
||||
* `verify image-signature <ref|digest>` — standalone cosign verification (optional, local).
|
||||
* `verify release <bundle> [--sbom <path>] [--vex <path>] [--trust-root <path>] [--checkpoint <path>]` — run promotion bundle verification and fail if source/build/rekor/signature links cannot be validated.
|
||||
|
||||
### 2.6 Runtime (Zastava helper)
|
||||
|
||||
|
||||
@@ -17,6 +17,7 @@ Concelier ingests signed advisories from **32 advisory connectors** and converts
|
||||
- Emit deterministic exports (JSON, Trivy DB) for downstream policy evaluation.
|
||||
- Coordinate offline/air-gap updates via Offline Kit bundles.
|
||||
- Serve paragraph-anchored advisory chunks for Advisory AI consumers without breaking the Aggregation-Only Contract.
|
||||
- Do not emit promotion PASS/FAIL decisions; promotion gate decisions are owned by Policy Engine.
|
||||
|
||||
## Key components
|
||||
- `StellaOps.Concelier.WebService` orchestration host.
|
||||
|
||||
@@ -39,6 +39,15 @@ Key settings:
|
||||
## Related Documentation
|
||||
|
||||
- Operations: `./operations/` (if exists)
|
||||
- Portable pack contract: `./portable-audit-pack-contract.md`
|
||||
- Portable manifest schema: `./schemas/portable-audit-pack-manifest.v1.schema.json`
|
||||
- Portable compatibility mapping: `./portable-audit-pack-compatibility.md`
|
||||
- Portable determinism profile: `./portable-audit-pack-determinism.md`
|
||||
- Portable Rekor offline profile: `./portable-audit-pack-rekor-offline.md`
|
||||
- Portable CLI runbook: `./portable-audit-pack-cli-runbook.md`
|
||||
- Portable Parquet profile: `./portable-audit-pack-parquet-profile.md`
|
||||
- Portable verification matrix: `./portable-audit-pack-test-matrix.md`
|
||||
- Promotion evidence contract: `./promotion-evidence-contract.md`
|
||||
- ExportCenter: `../export-center/`
|
||||
- Attestor: `../attestor/`
|
||||
- High-Level Architecture: `../../ARCHITECTURE_OVERVIEW.md`
|
||||
|
||||
@@ -276,6 +276,7 @@ Bundle N-1 Bundle N Bundle N+1
|
||||
* Attestation contract: `./attestation-contract.md`
|
||||
* Evidence bundle spec: `./evidence-bundle-v1.md`
|
||||
* Evidence pack schema: `./guides/evidence-pack-schema.md`
|
||||
* Promotion gate evidence contract: `./promotion-evidence-contract.md`
|
||||
* Audit bundle index schema: `./schemas/audit-bundle-index.schema.json`
|
||||
* ExportCenter: `../export-center/architecture.md`
|
||||
* Attestor: `../attestor/architecture.md`
|
||||
|
||||
@@ -93,3 +93,23 @@ Validation is fail-closed:
|
||||
- reject invalid Rekor index values
|
||||
|
||||
This contract is authoritative for Sprint 110 and blocks CONCELIER-ATTEST-73-001/002 and EXCITITOR-ATTEST-01-003/73-001/73-002.
|
||||
|
||||
## Gate Artifact Extension (v1.1, 2026-02-10)
|
||||
|
||||
Promotion evidence consumers now rely on additional optional fields for policy-gate semantics:
|
||||
|
||||
- `producer_bundle.evidence_score_value` (0-100 numeric score for threshold checks)
|
||||
- `producer_bundle.build_link.exists` (bool)
|
||||
- `producer_bundle.build_link.product_digest.sha256|sha512` (optional digest binding inputs)
|
||||
- `producer_bundle.artifact_digest.sha256|sha512` (optional explicit artifact digest)
|
||||
- `producer_bundle.dsse_signatures[]`:
|
||||
- `key_id`
|
||||
- `algorithm`
|
||||
- `valid`
|
||||
- `producer_bundle.rekor.checked_at` (UTC RFC3339 timestamp for freshness TTL checks)
|
||||
- `producer_bundle.human_decision_dsse_ref` (optional DSSE reference for signed escalation disposition)
|
||||
|
||||
Offline exports must retain enough metadata for air-gapped gate replay:
|
||||
- Rekor proof references (`tile_id`, `inclusion_proof_path`) and freshness timestamp.
|
||||
- DSSE signer evidence needed for k-of-n verification.
|
||||
- Human decision DSSE reference when escalation policy requires signed disposition.
|
||||
|
||||
@@ -0,0 +1,52 @@
|
||||
# Portable Audit Pack CLI Runbook
|
||||
|
||||
Status: Target behavior for implementation sprint handoff (2026-02-10).
|
||||
|
||||
## Objective
|
||||
Define expected parity between generation and verification CLI flows for portable audit packs in connected and air-gapped environments.
|
||||
|
||||
## Export workflow (target)
|
||||
```bash
|
||||
stella auditpack export \
|
||||
--artifact myorg/myapp@sha256:<digest> \
|
||||
--bom sbom.json \
|
||||
--vex vex/*.json \
|
||||
--out artifact-audit-pack.tzst \
|
||||
--profile portable-v1 \
|
||||
--rekor-tiles fetch \
|
||||
--sign-key ed25519:stella-bom-signer@2026Q1
|
||||
```
|
||||
|
||||
Expected behavior:
|
||||
- Emits manifest conforming to `portable-audit-pack-manifest.v1.schema.json`.
|
||||
- Produces deterministic archive metadata and ordered contents.
|
||||
- Emits stable machine-readable summary ordered by file path.
|
||||
|
||||
## Verify workflow (target)
|
||||
```bash
|
||||
stella auditpack verify artifact-audit-pack.tzst --offline --profile portable-v1
|
||||
```
|
||||
|
||||
Required checks:
|
||||
- Manifest signature verification.
|
||||
- File digest and size verification.
|
||||
- DSSE payload digest binding verification.
|
||||
- Rekor inclusion/root verification from bundled material.
|
||||
- Optional Parquet fingerprint verification when present.
|
||||
|
||||
## Output contract
|
||||
- Human output grouped in fixed order: manifest -> file digests -> DSSE -> Rekor -> optional index.
|
||||
- JSON output fields sorted lexicographically for deterministic diffing.
|
||||
- Non-zero exit and stable error codes on first failure.
|
||||
|
||||
## Air-gap operator sequence
|
||||
1. Transfer bundle to offline verifier host.
|
||||
2. Run `stella auditpack verify ... --offline`.
|
||||
3. Archive verification output with audit evidence.
|
||||
4. Record profile version and verifier key IDs in release record.
|
||||
|
||||
## Documentation dependency
|
||||
- Keep this runbook aligned with:
|
||||
- `portable-audit-pack-contract.md`
|
||||
- `portable-audit-pack-rekor-offline.md`
|
||||
- `portable-audit-pack-test-matrix.md`
|
||||
@@ -0,0 +1,40 @@
|
||||
# Portable Audit Pack Compatibility Mapping
|
||||
|
||||
Status: Draft frozen for implementation handoff (2026-02-10).
|
||||
|
||||
## Purpose
|
||||
Map current StellaOps evidence bundle contracts to the portable audit pack profile so writer/reader implementations use one required field model.
|
||||
|
||||
## Canonical contract source
|
||||
- Manifest schema: `docs/modules/evidence-locker/schemas/portable-audit-pack-manifest.v1.schema.json`
|
||||
- Profile contract: `docs/modules/evidence-locker/portable-audit-pack-contract.md`
|
||||
|
||||
## Required field mapping
|
||||
| Portable field | Existing source contract | Notes |
|
||||
| --- | --- | --- |
|
||||
| `spec_version` | `bundle.manifest.schema.json` `manifestVersion` | Portable uses fixed `1.0`. |
|
||||
| `artifact.digest.sha256` | `evidence-bundle-v1.md` subject digest | Required, lowercase hex without `sha256:` prefix in manifest payload fields. |
|
||||
| `files[*].sha256` | `checksums.schema.json` + bundle manifest entries | Portable stores per-file metadata directly in `files` map. |
|
||||
| `digests.canonical_bom_sha256` | `stellaops-evidence-pack.v1.schema.json` digest fields | New explicit top-level binding for BOM canonical bytes. |
|
||||
| `digests.dsse_payload_digest.sha256` | `attestation-contract.md` producer bundle digest linkage | Required preimage binding for DSSE payload verification. |
|
||||
| `rekor.tile_refs[]` | `attestor/transparency.md` + Rekor receipt inputs | Portable requires deterministic path references under `rekor/`. |
|
||||
| `rekor.root_hash` | Attestor checkpoint verification contract | Captured at inclusion checkpoint used by offline verifier. |
|
||||
| `verifiers.pubkeys[]` | Existing key bundle references | Portable manifest contains verifier key references used by CLI/offline verifier. |
|
||||
|
||||
## Legacy bundle compatibility
|
||||
- Legacy `evidence-bundle-<id>.tar.gz` and `portable-bundle-v1.tgz` remain valid for existing tooling.
|
||||
- Portable audit pack profile is additive and must not reinterpret legacy fields silently.
|
||||
- Readers should apply this precedence:
|
||||
1. If `spec_version` exists and equals `1.0`, validate against portable schema.
|
||||
2. Else if `manifestVersion` exists, validate against legacy `bundle.manifest.schema.json`.
|
||||
3. Else fail closed with `ERR_MANIFEST_PROFILE_UNKNOWN`.
|
||||
|
||||
## Writer/reader alignment rules
|
||||
- Writers MUST populate every required portable field in schema v1.
|
||||
- Readers MUST reject packs missing any required portable field.
|
||||
- Writers/readers MUST share the same portable schema artifact ID and hash in release notes.
|
||||
|
||||
## Migration notes
|
||||
- Maintain both parsers during transition.
|
||||
- Export paths should emit explicit profile indicator in logs and operator output.
|
||||
- Verification output should identify which profile was validated.
|
||||
106
docs/modules/evidence-locker/portable-audit-pack-contract.md
Normal file
106
docs/modules/evidence-locker/portable-audit-pack-contract.md
Normal file
@@ -0,0 +1,106 @@
|
||||
# Portable Audit Pack Contract (v1 Draft)
|
||||
|
||||
## Purpose
|
||||
Define a deterministic, offline-verifiable portable audit pack contract that unifies Stella Ops evidence export semantics across Attestor, EvidenceLocker, AuditPack, and CLI verification flows.
|
||||
|
||||
## Contract status
|
||||
- Status: Draft for implementation.
|
||||
- Source sprint: `docs-archived/implplan/2026-02-10-completed-sprints/SPRINT_20260210_003_DOCS_portable_audit_pack_translation.md`
|
||||
- Canonical schema: `docs/modules/evidence-locker/schemas/portable-audit-pack-manifest.v1.schema.json`
|
||||
|
||||
## Companion profile documents
|
||||
- Compatibility mapping: `docs/modules/evidence-locker/portable-audit-pack-compatibility.md`
|
||||
- Determinism profile: `docs/modules/evidence-locker/portable-audit-pack-determinism.md`
|
||||
- Rekor offline verification profile: `docs/modules/evidence-locker/portable-audit-pack-rekor-offline.md`
|
||||
- CLI runbook (target behavior): `docs/modules/evidence-locker/portable-audit-pack-cli-runbook.md`
|
||||
- Optional Parquet profile: `docs/modules/evidence-locker/portable-audit-pack-parquet-profile.md`
|
||||
- Verification test matrix: `docs/modules/evidence-locker/portable-audit-pack-test-matrix.md`
|
||||
|
||||
## Target bundle profile
|
||||
|
||||
### Required artifacts
|
||||
- `manifest.json` (JCS canonical JSON)
|
||||
- `manifest.sig` (DSSE envelope over canonical manifest, detached file)
|
||||
- `canonical_bom.json` (canonicalized BOM snapshot)
|
||||
- `dsse_envelope.json` (attestation envelope bound to BOM/subject)
|
||||
- `rekor/` proof material:
|
||||
- checkpoint note/signature
|
||||
- inclusion proof data
|
||||
- tile bundle reference material (`tile.tar` or equivalent deterministic bundle)
|
||||
|
||||
### Optional artifacts
|
||||
- `merged_vex.json` (canonical merged VEX view)
|
||||
- `components.parquet` (optional analytics profile)
|
||||
- `checksums.txt` / replay helper assets for operational workflows
|
||||
|
||||
## Manifest contract (portable profile)
|
||||
|
||||
### Core fields
|
||||
- `spec_version`
|
||||
- `created_utc`
|
||||
- `artifact` (`name`, `version`, `digest`, `media_type`)
|
||||
- `files` map with per-file:
|
||||
- `sha256`
|
||||
- `size`
|
||||
- `content_type`
|
||||
- profile-specific metadata (for example `compression`, `schema_fingerprint`)
|
||||
- `digests`:
|
||||
- `canonical_bom_sha256`
|
||||
- `dsse_payload_digest`
|
||||
- `rekor`:
|
||||
- `log_id`
|
||||
- `api_version`
|
||||
- `tile_refs`
|
||||
- `root_hash`
|
||||
- `timestamps`
|
||||
- `verifiers` (key references and trust metadata)
|
||||
|
||||
Schema note:
|
||||
- Required field set and allowed optional fields are frozen in `portable-audit-pack-manifest.v1.schema.json`.
|
||||
|
||||
### Determinism rules
|
||||
- JSON canonicalization MUST use RFC 8785/JCS-compatible canonical output.
|
||||
- Manifest signing input MUST be the canonical bytes of `manifest.json`.
|
||||
- File inventory MUST be sorted lexicographically by canonical path.
|
||||
- Archive metadata MUST be deterministic (mtime, uid/gid, mode, ordering).
|
||||
- Digests MUST be lowercase SHA-256 hex unless profile explicitly states otherwise.
|
||||
|
||||
## Verification contract
|
||||
1. Verify `manifest.sig` against canonical `manifest.json`.
|
||||
2. Verify every file digest/size in `manifest.files`.
|
||||
3. Verify DSSE envelope signature(s) and payload digest binding.
|
||||
4. Verify Rekor inclusion proof against checkpoint root using bundled proof/tile data.
|
||||
5. Verify artifact/BOM subject digest consistency.
|
||||
6. If `components.parquet` is present, validate schema fingerprint metadata.
|
||||
|
||||
Default policy is fail-closed for missing or invalid required verification inputs.
|
||||
|
||||
## Current state vs target (gap summary)
|
||||
- Implemented:
|
||||
- Detached `manifest.sig` support in audit bundle paths.
|
||||
- Rekor offline proof verification primitives.
|
||||
- EvidenceLocker fields for canonical BOM/payload digest and Rekor refs.
|
||||
- Gaps:
|
||||
- No single unified portable manifest schema with full required field set.
|
||||
- Non-uniform canonicalization implementations across pack writers.
|
||||
- Determinism not fully enforced across all packaging flows.
|
||||
- Optional Parquet profile not defined in portable pack contract.
|
||||
|
||||
## Ownership map
|
||||
- `Attestor`: DSSE/Rekor proof verification contract and tile/checkpoint binding.
|
||||
- `EvidenceLocker`: persistence/export schema and portable bundle profile publication.
|
||||
- `StellaOps.AuditPack`: deterministic pack write/read/sign/verify implementation.
|
||||
- `CLI`: pack generation and offline verification UX parity.
|
||||
- `QA`: deterministic fixtures, tamper matrix, replay verification.
|
||||
|
||||
## Implementation notes
|
||||
- Keep compatibility mapping for legacy bundle manifests; do not silently reinterpret fields.
|
||||
- Keep offline posture: no mandatory network calls in verification.
|
||||
- Prefer shared canonicalization libraries over local ad hoc JSON serializers.
|
||||
|
||||
## References
|
||||
- `docs/modules/attestor/repro-bundle-profile.md`
|
||||
- `docs/modules/attestor/transparency.md`
|
||||
- `docs/modules/evidence-locker/export-format.md`
|
||||
- `docs/modules/evidence-locker/schemas/audit-bundle-index.schema.json`
|
||||
- `docs/modules/evidence-locker/schemas/stellaops-evidence-pack.v1.schema.json`
|
||||
@@ -0,0 +1,46 @@
|
||||
# Portable Audit Pack Determinism Profile
|
||||
|
||||
Status: Draft frozen for implementation handoff (2026-02-10).
|
||||
|
||||
## Scope
|
||||
Deterministic requirements for portable pack generation (`manifest.json`, BOM, DSSE envelope, Rekor material, optional VEX/Parquet artifacts).
|
||||
|
||||
## Normative rules
|
||||
1. Canonical JSON MUST use RFC 8785/JCS-compatible serialization.
|
||||
2. File inventory in `manifest.files` MUST be lexicographically sorted by canonical path.
|
||||
3. Archive entries MUST have fixed metadata:
|
||||
- `mtime`: `2026-01-01T00:00:00Z`
|
||||
- `uid/gid`: `0/0`
|
||||
- file mode `0644`, directory mode `0755`
|
||||
4. Digests MUST be lowercase SHA-256 hex.
|
||||
5. Optional artifacts (`merged_vex.json`, `components.parquet`) MUST not change ordering of required files.
|
||||
6. Compression toolchain versions MUST be pinned in release manifests.
|
||||
|
||||
## Canonicalization conformance tests (required)
|
||||
- Nested object key ordering stability.
|
||||
- Unicode normalization and escaping stability.
|
||||
- Non-finite number rejection (`NaN`, `Infinity`).
|
||||
- DSSE payload preimage digest stability across repeated runs.
|
||||
|
||||
## Byte stability gate
|
||||
- CI must generate the same pack twice from identical frozen input fixtures.
|
||||
- Outputs must be byte-identical (`sha256sum pack1 == pack2`).
|
||||
- On mismatch, pipeline fails with `ERR_PACK_NON_DETERMINISTIC`.
|
||||
|
||||
## Deterministic fixture layout
|
||||
- `testvectors/portable-audit-pack/minimal/`
|
||||
- `testvectors/portable-audit-pack/with-vex/`
|
||||
- `testvectors/portable-audit-pack/with-parquet/`
|
||||
|
||||
Each fixture set should include:
|
||||
- inputs (`sbom.json`, optional `vex.json`)
|
||||
- expected canonical files
|
||||
- expected per-file SHA-256 digests
|
||||
- expected package archive digest
|
||||
|
||||
## Toolchain pin set (to be implemented)
|
||||
- JCS canonicalizer version
|
||||
- DSSE signer library version
|
||||
- tar implementation/version
|
||||
- compression implementation/version
|
||||
- Parquet writer version (if profile enabled)
|
||||
@@ -0,0 +1,43 @@
|
||||
# Portable Audit Pack Parquet Profile (Optional)
|
||||
|
||||
Status: Optional profile contract for implementation handoff (2026-02-10).
|
||||
|
||||
## Positioning
|
||||
`components.parquet` is optional and must not be required for baseline pack verification.
|
||||
|
||||
## Manifest integration
|
||||
When present, `manifest.files["components.parquet"]` must include:
|
||||
- `sha256`
|
||||
- `size`
|
||||
- `content_type` = `application/x-parquet`
|
||||
- `compression` = `snappy`
|
||||
- `schema_fingerprint`
|
||||
|
||||
## Recommended schema columns
|
||||
- `package_name` (STRING)
|
||||
- `package_version` (STRING)
|
||||
- `purl` (STRING)
|
||||
- `license` (STRING)
|
||||
- `component_hash_sha256` (STRING)
|
||||
- `artifact_digest_sha256` (STRING)
|
||||
- `cve_id` (STRING, nullable)
|
||||
- `vex_status` (STRING, nullable)
|
||||
- `introduced_range` (STRING, nullable)
|
||||
- `fixed_version` (STRING, nullable)
|
||||
- `source_bom_sha256` (STRING)
|
||||
|
||||
## Determinism rules
|
||||
- Stable row ordering: `(artifact_digest_sha256, package_name, package_version, purl)`.
|
||||
- Stable column ordering exactly as listed above.
|
||||
- Stable Parquet writer settings pinned by version and compression codec.
|
||||
- `schema_fingerprint` must be reproducible from logical schema definition.
|
||||
|
||||
## Feature gating
|
||||
- Default profile: disabled.
|
||||
- Enable only with explicit profile flag.
|
||||
- Verification ignores Parquet content when absent.
|
||||
- Verification fails with `ERR_PARQUET_FINGERPRINT_MISMATCH` when present but invalid.
|
||||
|
||||
## Operator guidance
|
||||
- Use Parquet profile for fleet-level offline analytics.
|
||||
- Keep analytics ingestion separate from baseline release gate verification.
|
||||
@@ -0,0 +1,40 @@
|
||||
# Portable Audit Pack Rekor Offline Verification Profile
|
||||
|
||||
Status: Draft frozen for implementation handoff (2026-02-10).
|
||||
|
||||
## Required Rekor material in pack
|
||||
At least one of:
|
||||
- `rekor/tile.tar`
|
||||
- `rekor/tiles.bundle`
|
||||
|
||||
And manifest references:
|
||||
- `rekor.log_id`
|
||||
- `rekor.api_version` (`2`)
|
||||
- `rekor.tile_refs[]`
|
||||
- `rekor.root_hash`
|
||||
|
||||
## Offline verification flow
|
||||
1. Validate manifest signature and manifest file inventory/digests.
|
||||
2. Load bundled tile material referenced by `rekor.tile_refs[]`.
|
||||
3. Reconstruct inclusion proof path for covered digests.
|
||||
4. Validate Merkle root equals `rekor.root_hash`.
|
||||
5. Validate checkpoint key material from `verifiers.rekor_pub` when present.
|
||||
6. Fail closed on any missing tile/proof/checkpoint dependency.
|
||||
|
||||
## Stable failure codes
|
||||
- `ERR_REKOR_TILE_MISSING`
|
||||
- `ERR_REKOR_TILE_DIGEST_MISMATCH`
|
||||
- `ERR_REKOR_PROOF_INVALID`
|
||||
- `ERR_REKOR_CHECKPOINT_INVALID`
|
||||
- `ERR_REKOR_ROOT_MISMATCH`
|
||||
- `ERR_REKOR_REFERENCE_UNCOVERED`
|
||||
|
||||
## Tamper test requirements
|
||||
- Corrupt one tile byte -> `ERR_REKOR_TILE_DIGEST_MISMATCH`.
|
||||
- Modify inclusion path node -> `ERR_REKOR_PROOF_INVALID`.
|
||||
- Alter checkpoint signature -> `ERR_REKOR_CHECKPOINT_INVALID`.
|
||||
- Alter `rekor.root_hash` in manifest -> `ERR_REKOR_ROOT_MISMATCH`.
|
||||
|
||||
## Compatibility notes
|
||||
- Existing Rekor receipt contracts remain valid for legacy bundle profiles.
|
||||
- Portable profile requires deterministic file references under `rekor/` in the manifest.
|
||||
@@ -0,0 +1,31 @@
|
||||
# Portable Audit Pack Verification Test Matrix
|
||||
|
||||
Status: QA handoff matrix for implementation sprint (2026-02-10).
|
||||
|
||||
## Coverage matrix
|
||||
| ID | Layer | Scenario | Expected result |
|
||||
| --- | --- | --- | --- |
|
||||
| PAP-T01 | Unit | Manifest canonicalization stable for nested objects/unicode | PASS |
|
||||
| PAP-T02 | Unit | Manifest missing required field | `ERR_MANIFEST_SCHEMA` |
|
||||
| PAP-T03 | Unit | Manifest signature mismatch | `ERR_MANIFEST_SIGNATURE` |
|
||||
| PAP-T04 | Unit | DSSE payload digest mismatch | `ERR_DSSE_PAYLOAD_DIGEST` |
|
||||
| PAP-T05 | Unit | File digest mismatch for required file | `ERR_FILE_DIGEST_MISMATCH` |
|
||||
| PAP-T06 | Integration | Rekor tile corruption | `ERR_REKOR_TILE_DIGEST_MISMATCH` |
|
||||
| PAP-T07 | Integration | Rekor proof node corruption | `ERR_REKOR_PROOF_INVALID` |
|
||||
| PAP-T08 | Integration | Checkpoint signature corruption | `ERR_REKOR_CHECKPOINT_INVALID` |
|
||||
| PAP-T09 | Integration | Root hash mismatch | `ERR_REKOR_ROOT_MISMATCH` |
|
||||
| PAP-T10 | Integration | Required Rekor material missing | `ERR_REKOR_TILE_MISSING` |
|
||||
| PAP-T11 | E2E | Full offline verification with valid pack | PASS |
|
||||
| PAP-T12 | E2E | Repeat generation from frozen input twice | identical SHA-256 output |
|
||||
| PAP-T13 | E2E | Optional Parquet absent | PASS |
|
||||
| PAP-T14 | E2E | Optional Parquet present with invalid fingerprint | `ERR_PARQUET_FINGERPRINT_MISMATCH` |
|
||||
|
||||
## Fixture expectations
|
||||
- Golden fixtures under `testvectors/portable-audit-pack/`.
|
||||
- Expected digests tracked in fixture metadata and CI assertions.
|
||||
- All fixtures must be network-independent and deterministic.
|
||||
|
||||
## Execution log template
|
||||
| Date (UTC) | Fixture set | Command | Result | Owner |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| 2026-02-10 | Planning only | Matrix drafted for implementation sprint handoff. | N/A | QA/Test Automation |
|
||||
95
docs/modules/evidence-locker/promotion-evidence-contract.md
Normal file
95
docs/modules/evidence-locker/promotion-evidence-contract.md
Normal file
@@ -0,0 +1,95 @@
|
||||
# Promotion Evidence Contract
|
||||
|
||||
## Purpose
|
||||
|
||||
This contract defines the evidence payload that promotion gates consume. It freezes
|
||||
ownership boundaries and required fields for deterministic and offline-capable
|
||||
release decisions.
|
||||
|
||||
## Ownership Boundaries
|
||||
|
||||
| Capability | Owning module | Notes |
|
||||
| --- | --- | --- |
|
||||
| Evidence storage and retrieval | EvidenceLocker | Stores immutable evidence; does not decide allow/deny. |
|
||||
| DSSE signing | Signer | Produces signed envelopes and signer metadata. |
|
||||
| Transparency logging and proof material | Attestor | Produces Rekor UUID/log index/proof references and verifies proofs. |
|
||||
| PASS/FAIL policy decisioning | Policy Engine | Evaluates gate logic and emits decision outputs. |
|
||||
| Promotion orchestration | Release Orchestrator | Consumes evidence + policy outputs to progress promotion state. |
|
||||
|
||||
## Canonical Promotion Gate Input
|
||||
|
||||
The promotion gate input MUST be serializable as canonical JSON with stable
|
||||
key order and UTC timestamps.
|
||||
|
||||
```json
|
||||
{
|
||||
"artifact": {
|
||||
"artifactId": "string",
|
||||
"digest": "sha256:..."
|
||||
},
|
||||
"evidence": {
|
||||
"evidenceScore": "lowercase-hex-sha256",
|
||||
"bundleId": "guid",
|
||||
"bundleDigest": "sha256:..."
|
||||
},
|
||||
"sbom": {
|
||||
"canonicalDigest": "sha256:...",
|
||||
"format": "cyclonedx|spdx"
|
||||
},
|
||||
"attestations": [
|
||||
{
|
||||
"predicateType": "string",
|
||||
"dsseDigest": "sha256:...",
|
||||
"signerKeyId": "string"
|
||||
}
|
||||
],
|
||||
"transparency": {
|
||||
"rekorUuid": "string",
|
||||
"logIndex": 0,
|
||||
"proofRef": "cas://..."
|
||||
},
|
||||
"vex": {
|
||||
"mergedDigest": "sha256:...",
|
||||
"statusSummary": "affected|not_affected|under_investigation|mixed"
|
||||
},
|
||||
"inToto": {
|
||||
"layoutDigest": "sha256:...",
|
||||
"linkDigests": ["sha256:..."]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## EvidenceLocker API Mapping
|
||||
|
||||
| Endpoint | Contract fields populated |
|
||||
| --- | --- |
|
||||
| `POST /evidence` | `artifact.*`, `evidence.evidenceScore` |
|
||||
| `GET /evidence/score?artifact_id=...` | `evidence.evidenceScore` |
|
||||
| `POST /evidence/snapshot` | `evidence.bundleId`, `evidence.bundleDigest` |
|
||||
| `GET /evidence/{bundleId}` | `sbom.*`, `attestations[]`, `vex.*`, `inToto.*` |
|
||||
| `POST /evidence/verify` | `transparency.*` verification status and proof linkage |
|
||||
| `GET /evidence/{bundleId}/portable` | Offline bundle material for air-gapped verification |
|
||||
|
||||
## Determinism Requirements
|
||||
|
||||
- Serialize gate input with stable ordering and invariant casing.
|
||||
- Use UTC RFC3339 timestamps only.
|
||||
- Sort all arrays that do not carry semantic order by deterministic key:
|
||||
- `attestations[]` by `predicateType`, then `dsseDigest`
|
||||
- `inToto.linkDigests[]` lexicographically
|
||||
- Treat digest values as lowercase.
|
||||
- Fail closed if required evidence fields are missing.
|
||||
|
||||
## Offline Verification Requirements
|
||||
|
||||
- Promotion gates MUST accept proof references from offline bundles.
|
||||
- Rekor verification may use tile/proof material from local mirrors.
|
||||
- If transparency data is unavailable, gate outcome must be explicit policy output
|
||||
(for example deny, or break-glass path with auditable reason).
|
||||
|
||||
## Implementation References
|
||||
|
||||
- EvidenceLocker architecture: `docs/modules/evidence-locker/architecture.md`
|
||||
- EvidenceLocker attestation contract: `docs/modules/evidence-locker/attestation-contract.md`
|
||||
- Policy ownership contract: `docs/modules/policy/promotion-gate-ownership-contract.md`
|
||||
- Release Orchestrator runtime gap plan: `docs/modules/release-orchestrator/promotion-runtime-gap-closure-plan.md`
|
||||
@@ -0,0 +1,331 @@
|
||||
{
|
||||
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
||||
"$id": "https://stellaops.io/schemas/evidence-locker/portable-audit-pack-manifest.v1.schema.json",
|
||||
"title": "StellaOps Portable Audit Pack Manifest v1",
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": [
|
||||
"spec_version",
|
||||
"created_utc",
|
||||
"artifact",
|
||||
"files",
|
||||
"digests",
|
||||
"rekor",
|
||||
"timestamps",
|
||||
"verifiers"
|
||||
],
|
||||
"properties": {
|
||||
"spec_version": {
|
||||
"type": "string",
|
||||
"const": "1.0"
|
||||
},
|
||||
"created_utc": {
|
||||
"type": "string",
|
||||
"format": "date-time"
|
||||
},
|
||||
"artifact": {
|
||||
"$ref": "#/$defs/artifact"
|
||||
},
|
||||
"files": {
|
||||
"type": "object",
|
||||
"minProperties": 3,
|
||||
"required": [
|
||||
"canonical_bom.json",
|
||||
"dsse_envelope.json",
|
||||
"manifest.sig"
|
||||
],
|
||||
"propertyNames": {
|
||||
"type": "string",
|
||||
"minLength": 1,
|
||||
"pattern": "^[^\\\\]+$"
|
||||
},
|
||||
"additionalProperties": {
|
||||
"$ref": "#/$defs/fileEntry"
|
||||
}
|
||||
},
|
||||
"digests": {
|
||||
"$ref": "#/$defs/digests"
|
||||
},
|
||||
"rekor": {
|
||||
"$ref": "#/$defs/rekor"
|
||||
},
|
||||
"timestamps": {
|
||||
"$ref": "#/$defs/timestamps"
|
||||
},
|
||||
"verifiers": {
|
||||
"$ref": "#/$defs/verifiers"
|
||||
},
|
||||
"compatibility": {
|
||||
"$ref": "#/$defs/compatibility"
|
||||
}
|
||||
},
|
||||
"$defs": {
|
||||
"sha256": {
|
||||
"type": "string",
|
||||
"pattern": "^[a-f0-9]{64}$"
|
||||
},
|
||||
"artifact": {
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": [
|
||||
"name",
|
||||
"version",
|
||||
"digest",
|
||||
"media_type"
|
||||
],
|
||||
"properties": {
|
||||
"name": {
|
||||
"type": "string",
|
||||
"minLength": 1
|
||||
},
|
||||
"version": {
|
||||
"type": "string",
|
||||
"minLength": 1
|
||||
},
|
||||
"digest": {
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": [
|
||||
"sha256"
|
||||
],
|
||||
"properties": {
|
||||
"sha256": {
|
||||
"$ref": "#/$defs/sha256"
|
||||
}
|
||||
}
|
||||
},
|
||||
"media_type": {
|
||||
"type": "string",
|
||||
"minLength": 1
|
||||
}
|
||||
}
|
||||
},
|
||||
"fileEntry": {
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": [
|
||||
"sha256",
|
||||
"size",
|
||||
"content_type"
|
||||
],
|
||||
"properties": {
|
||||
"sha256": {
|
||||
"$ref": "#/$defs/sha256"
|
||||
},
|
||||
"size": {
|
||||
"type": "integer",
|
||||
"minimum": 0
|
||||
},
|
||||
"content_type": {
|
||||
"type": "string",
|
||||
"minLength": 1
|
||||
},
|
||||
"compression": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"none",
|
||||
"gzip",
|
||||
"zstd",
|
||||
"snappy"
|
||||
]
|
||||
},
|
||||
"schema_fingerprint": {
|
||||
"type": "string",
|
||||
"minLength": 1
|
||||
}
|
||||
}
|
||||
},
|
||||
"digests": {
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": [
|
||||
"canonical_bom_sha256",
|
||||
"dsse_payload_digest"
|
||||
],
|
||||
"properties": {
|
||||
"canonical_bom_sha256": {
|
||||
"$ref": "#/$defs/sha256"
|
||||
},
|
||||
"dsse_payload_digest": {
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": [
|
||||
"sha256"
|
||||
],
|
||||
"properties": {
|
||||
"sha256": {
|
||||
"$ref": "#/$defs/sha256"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"rekor": {
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": [
|
||||
"log_id",
|
||||
"api_version",
|
||||
"tile_refs",
|
||||
"root_hash"
|
||||
],
|
||||
"properties": {
|
||||
"log_id": {
|
||||
"type": "string",
|
||||
"minLength": 1
|
||||
},
|
||||
"api_version": {
|
||||
"type": "string",
|
||||
"const": "2"
|
||||
},
|
||||
"tile_refs": {
|
||||
"type": "array",
|
||||
"minItems": 1,
|
||||
"items": {
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": [
|
||||
"path",
|
||||
"covers"
|
||||
],
|
||||
"properties": {
|
||||
"path": {
|
||||
"type": "string",
|
||||
"pattern": "^rekor/"
|
||||
},
|
||||
"covers": {
|
||||
"type": "array",
|
||||
"minItems": 1,
|
||||
"items": {
|
||||
"type": "string",
|
||||
"pattern": "^SHA256:[A-Fa-f0-9]{64}$"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"root_hash": {
|
||||
"$ref": "#/$defs/sha256"
|
||||
}
|
||||
}
|
||||
},
|
||||
"timestamps": {
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": [
|
||||
"bom_canonicalized",
|
||||
"dsse_signed",
|
||||
"rekor_included"
|
||||
],
|
||||
"properties": {
|
||||
"bom_canonicalized": {
|
||||
"type": "string",
|
||||
"format": "date-time"
|
||||
},
|
||||
"dsse_signed": {
|
||||
"type": "string",
|
||||
"format": "date-time"
|
||||
},
|
||||
"rekor_included": {
|
||||
"type": "string",
|
||||
"format": "date-time"
|
||||
}
|
||||
}
|
||||
},
|
||||
"verifiers": {
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": [
|
||||
"pubkeys"
|
||||
],
|
||||
"properties": {
|
||||
"pubkeys": {
|
||||
"type": "array",
|
||||
"minItems": 1,
|
||||
"items": {
|
||||
"$ref": "#/$defs/pubkey"
|
||||
}
|
||||
},
|
||||
"rekor_pub": {
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": [
|
||||
"type",
|
||||
"key_material"
|
||||
],
|
||||
"properties": {
|
||||
"type": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"rekor-checkpoint",
|
||||
"rekor-key-hash"
|
||||
]
|
||||
},
|
||||
"key_material": {
|
||||
"type": "string",
|
||||
"minLength": 1
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"pubkey": {
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": [
|
||||
"id",
|
||||
"type",
|
||||
"public_key",
|
||||
"usage"
|
||||
],
|
||||
"properties": {
|
||||
"id": {
|
||||
"type": "string",
|
||||
"minLength": 1
|
||||
},
|
||||
"type": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"ed25519",
|
||||
"ecdsa-p256",
|
||||
"rsa-4096"
|
||||
]
|
||||
},
|
||||
"public_key": {
|
||||
"type": "string",
|
||||
"minLength": 1
|
||||
},
|
||||
"usage": {
|
||||
"type": "array",
|
||||
"minItems": 1,
|
||||
"items": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"dsse",
|
||||
"manifest-signing",
|
||||
"checkpoint-verification"
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"compatibility": {
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"legacy_manifest_version": {
|
||||
"type": "string"
|
||||
},
|
||||
"legacy_bundle_id": {
|
||||
"type": "string"
|
||||
},
|
||||
"migration_notes": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -7,6 +7,7 @@ Policy Engine compiles and evaluates Stella DSL policies deterministically, prod
|
||||
- Join advisories, VEX evidence, and SBOM inventories to derive effective findings.
|
||||
- Expose simulation and diff APIs for UI/CLI workflows.
|
||||
- Emit change-stream driven events for Notify/Scheduler integrations.
|
||||
- Own promotion gate PASS/FAIL decision outputs consumed by Release Orchestrator.
|
||||
|
||||
## Key components
|
||||
- `StellaOps.Policy.Engine` service host.
|
||||
@@ -21,6 +22,7 @@ Policy Engine compiles and evaluates Stella DSL policies deterministically, prod
|
||||
- DSL grammar and lifecycle docs in ../../policy/.
|
||||
- Observability guidance in ../../observability/policy.md.
|
||||
- Governance and scope mapping in ../../security/policy-governance.md.
|
||||
- Promotion gate ownership contract: ./promotion-gate-ownership-contract.md.
|
||||
- Readiness briefs: ../policy/secret-leak-detection-readiness.md, ../policy/windows-package-readiness.md.
|
||||
- Readiness briefs: ../scanner/design/macos-analyzer.md, ../scanner/design/windows-analyzer.md, ../policy/secret-leak-detection-readiness.md, ../policy/windows-package-readiness.md.
|
||||
- Ruby capability predicates design: ./design/ruby-capability-predicates.md.
|
||||
|
||||
66
docs/modules/policy/promotion-gate-ownership-contract.md
Normal file
66
docs/modules/policy/promotion-gate-ownership-contract.md
Normal file
@@ -0,0 +1,66 @@
|
||||
# Promotion Gate Ownership Contract
|
||||
|
||||
## Purpose
|
||||
|
||||
This contract freezes ownership for promotion gate decisions and defines the
|
||||
Policy-to-Orchestrator interface.
|
||||
|
||||
## Ownership Rules
|
||||
|
||||
- Policy Engine owns PASS/FAIL gate evaluation semantics.
|
||||
- Concelier owns advisory ingestion and linkset publication only.
|
||||
- Release Orchestrator executes promotion state transitions using Policy outputs.
|
||||
- Authority enforces identity/scope boundaries for all callers.
|
||||
|
||||
## Explicit Non-Goals for Concelier
|
||||
|
||||
- No PASS/FAIL decisioning for promotion gates.
|
||||
- No direct production of promotion allow/deny verdicts.
|
||||
- No mutation of Policy-derived effective findings.
|
||||
|
||||
## Policy Evaluation Interface
|
||||
|
||||
```json
|
||||
{
|
||||
"request": {
|
||||
"tenantId": "string",
|
||||
"promotionId": "guid",
|
||||
"targetEnvironment": "string",
|
||||
"artifactDigest": "sha256:...",
|
||||
"evidenceRef": "guid",
|
||||
"policyBundleDigest": "sha256:..."
|
||||
},
|
||||
"response": {
|
||||
"decision": "allow|deny|pending",
|
||||
"reasonCodes": ["string"],
|
||||
"policyDigest": "sha256:...",
|
||||
"determinismHash": "sha256:...",
|
||||
"evaluatedAtUtc": "2026-02-10T00:00:00Z"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Determinism and Fail-Closed Rules
|
||||
|
||||
- Identical request payloads must produce identical decision outputs.
|
||||
- Missing or invalid policy inputs MUST return explicit deny or pending according
|
||||
to policy profile; no implicit allow.
|
||||
- Reason codes must be stable and sortable for replay/audit.
|
||||
|
||||
## Required Test Coverage
|
||||
|
||||
Promotion/Orchestrator side:
|
||||
- `src/ReleaseOrchestrator/__Tests/StellaOps.ReleaseOrchestrator.Promotion.Tests/Gate/PolicyGateTests.cs`
|
||||
- `src/ReleaseOrchestrator/__Tests/StellaOps.ReleaseOrchestrator.Promotion.Tests/Decision/DecisionEngineTests.cs`
|
||||
- `src/ReleaseOrchestrator/__Tests/StellaOps.ReleaseOrchestrator.Promotion.Tests/Gate/GateEvaluatorTests.cs`
|
||||
|
||||
Policy side:
|
||||
- `src/Policy/__Tests/StellaOps.Policy.Engine.Tests/Gates/PolicyGateEvaluatorTests.cs`
|
||||
- `src/Policy/__Tests/StellaOps.Policy.Engine.Tests/Determinism/PolicyEngineDeterminismTests.cs`
|
||||
- `src/Policy/__Tests/StellaOps.Policy.Engine.Tests/Gates/Determinization/DeterminizationGateTests.cs`
|
||||
|
||||
## Integration References
|
||||
|
||||
- Evidence contract: `docs/modules/evidence-locker/promotion-evidence-contract.md`
|
||||
- Promotion APIs: `docs/modules/release-orchestrator/api/promotions.md`
|
||||
- Runtime closure plan: `docs/modules/release-orchestrator/promotion-runtime-gap-closure-plan.md`
|
||||
@@ -54,6 +54,7 @@ The Release Orchestrator extends Stella Ops from a vulnerability scanning platfo
|
||||
- [Environment APIs](api/environments.md) — Environment endpoints
|
||||
- [Release APIs](api/releases.md) — Release endpoints
|
||||
- [Promotion APIs](api/promotions.md) — Promotion endpoints
|
||||
- [Promotion Runtime Gap Closure Plan](promotion-runtime-gap-closure-plan.md) — Docs-to-runtime delivery sequence
|
||||
- [Workflow APIs](api/workflows.md) — Workflow endpoints
|
||||
- [Agent APIs](api/agents.md) — Agent endpoints
|
||||
- [WebSocket APIs](api/websockets.md) — Real-time endpoints
|
||||
@@ -62,6 +63,7 @@ The Release Orchestrator extends Stella Ops from a vulnerability scanning platfo
|
||||
- [Template Structure](workflow/templates.md) — Workflow template specification
|
||||
- [Execution State Machine](workflow/execution.md) — Workflow state machine
|
||||
- [Promotion State Machine](workflow/promotion.md) — Promotion state machine
|
||||
- [Evidence-Based Release Gates](workflow/evidence-based-release-gates.md) — Data-driven evidence gate contract and outcomes
|
||||
|
||||
### Security
|
||||
- [Security Overview](security/overview.md) — Security principles
|
||||
@@ -109,6 +111,7 @@ The Release Orchestrator extends Stella Ops from a vulnerability scanning platfo
|
||||
- [Configuration Reference](appendices/config.md) — Configuration options
|
||||
- [Error Codes](appendices/errors.md) — API error codes
|
||||
- [Evidence Schema](appendices/evidence-schema.md) — Evidence packet format
|
||||
- [Optional Promotion Capsule](appendices/promotion-capsule-optional.md) — Optional DSSE capsule and `human_decision` envelope profile
|
||||
|
||||
## Quick Reference
|
||||
|
||||
|
||||
@@ -6,6 +6,9 @@
|
||||
**Source:** [Architecture Advisory Section 6.3.2](../../../product/advisories/09-Jan-2026%20-%20Stella%20Ops%20Orchestrator%20Architecture.md)
|
||||
**Related Modules:** [Environment Manager](../modules/environment-manager.md), [Agents](../modules/agents.md)
|
||||
|
||||
> Runtime note: environment APIs are documented here; controller implementation
|
||||
> sequencing is tracked in `../promotion-runtime-gap-closure-plan.md`.
|
||||
|
||||
## Overview
|
||||
|
||||
The Environment Management API provides CRUD operations for environments, target groups, deployment targets, agents, freeze windows, and inventory synchronization. All endpoints require authentication and respect tenant isolation via Row-Level Security.
|
||||
|
||||
@@ -6,10 +6,17 @@
|
||||
**Source:** [Architecture Advisory Section 6.3.5](../../../product/advisories/09-Jan-2026%20-%20Stella%20Ops%20Orchestrator%20Architecture.md)
|
||||
**Related Modules:** [Promotion Manager Module](../modules/promotion-manager.md), [Workflow Promotion](../workflow/promotion.md)
|
||||
|
||||
> Runtime note: promotion libraries and tests exist, while the API host controllers
|
||||
> for these endpoints are tracked in `../promotion-runtime-gap-closure-plan.md`.
|
||||
|
||||
## Overview
|
||||
|
||||
The Promotion API provides endpoints for requesting release promotions between environments, managing approvals, and evaluating promotion gates. Promotions enforce separation of duties (SoD) and require configured approvals before deployment proceeds.
|
||||
|
||||
Gate contract references:
|
||||
- `../../policy/promotion-gate-ownership-contract.md`
|
||||
- `../../evidence-locker/promotion-evidence-contract.md`
|
||||
|
||||
---
|
||||
|
||||
## Promotion Endpoints
|
||||
|
||||
@@ -0,0 +1,56 @@
|
||||
# Optional Promotion Capsule and `human_decision` Envelope
|
||||
|
||||
## Status
|
||||
|
||||
Optional profile. This appendix must not block baseline promotion delivery.
|
||||
|
||||
## Promotion Capsule (Optional)
|
||||
|
||||
The optional promotion capsule is a DSSE-wrapped bundle containing:
|
||||
|
||||
- Promotion identity (`promotionId`, source/target environment, artifact digests)
|
||||
- Policy inputs (policy digest/version, gate input digest)
|
||||
- Evidence references (evidence bundle id, attestation digests, Rekor refs)
|
||||
- Decision output (allow/deny/pending + reason codes)
|
||||
- Signatures and verification metadata
|
||||
|
||||
## Suggested Envelope Type
|
||||
|
||||
- Media type: `application/vnd.stellaops.promotion-capsule+json`
|
||||
- Predicate type: `stella.ops/promotionCapsule@v1`
|
||||
|
||||
## Optional `human_decision` DSSE Envelope
|
||||
|
||||
For exception paths, the optional envelope captures accountable human override
|
||||
decisions and links them to the promotion record.
|
||||
|
||||
Required fields:
|
||||
|
||||
- `decisionId` (stable id)
|
||||
- `promotionId`
|
||||
- `requestId` (Policy exception approval request id)
|
||||
- `actorId`
|
||||
- `decision` (`approve|reject|cancel`)
|
||||
- `reason`
|
||||
- `ticket`
|
||||
- `expiresAtUtc` (if temporary override)
|
||||
- `recordedAtUtc`
|
||||
|
||||
## Binding to Existing Approval Workflows
|
||||
|
||||
- Policy exception workflow APIs remain source of truth for request lifecycle.
|
||||
- Optional DSSE `human_decision` envelope references Policy request/audit ids.
|
||||
- Promotion decision records may include `humanDecisionEnvelopeId` when present.
|
||||
|
||||
## SLA and Governance Notes
|
||||
|
||||
- `human_decision` should be time-bounded and non-default.
|
||||
- Override paths should require explicit scope and reason metadata.
|
||||
- Expired override envelopes must not authorize future promotions.
|
||||
|
||||
## Related References
|
||||
|
||||
- `src/Policy/StellaOps.Policy.Gateway/Endpoints/ExceptionApprovalEndpoints.cs`
|
||||
- `src/Policy/StellaOps.Policy.Gateway/Services/ApprovalWorkflowService.cs`
|
||||
- `docs/product/decision-capsules.md`
|
||||
- `docs/modules/release-orchestrator/workflow/promotion.md`
|
||||
@@ -111,6 +111,10 @@ interface Approval {
|
||||
| **Data Entities** | `DecisionRecord`, `GateResult` |
|
||||
| **Events Produced** | `decision.evaluated`, `decision.recorded` |
|
||||
|
||||
Policy ownership note: PASS/FAIL promotion gate semantics are owned by Policy
|
||||
Engine and consumed by promotion workflows. Concelier remains ingestion-only for
|
||||
advisory/linkset data.
|
||||
|
||||
**Decision Record Structure**:
|
||||
```typescript
|
||||
interface DecisionRecord {
|
||||
|
||||
@@ -0,0 +1,62 @@
|
||||
# Promotion Runtime Gap Closure Plan
|
||||
|
||||
## Scope
|
||||
|
||||
Close the gap between documented Release Orchestrator promotion/environment APIs
|
||||
and currently implemented runtime controllers.
|
||||
|
||||
## Current Runtime Reality
|
||||
|
||||
Implemented HTTP controllers:
|
||||
|
||||
| Runtime surface | Implemented controller |
|
||||
| --- | --- |
|
||||
| Compliance APIs | `src/ReleaseOrchestrator/StellaOps.ReleaseOrchestrator.Api/Controllers/ComplianceController.cs` |
|
||||
| Rollback intelligence APIs | `src/ReleaseOrchestrator/__Apps/StellaOps.ReleaseOrchestrator.WebApi/Controllers/RollbackIntelligenceController.cs` |
|
||||
|
||||
Promotion libraries are implemented and test-covered, but promotion/environment
|
||||
HTTP controllers are not yet present in these API hosts.
|
||||
|
||||
## Gap Inventory (Docs vs Runtime)
|
||||
|
||||
| API group | Doc status | Runtime status | Priority |
|
||||
| --- | --- | --- | --- |
|
||||
| `/api/v1/environments` | Documented | Missing controller | P0 |
|
||||
| `/api/v1/promotions` | Documented | Missing controller | P0 |
|
||||
| `/api/v1/promotions/{id}/decision` | Documented | Missing controller | P0 |
|
||||
| `/api/v1/promotions/{id}/evidence` | Documented | Missing controller | P1 |
|
||||
| `/api/v1/approval-policies` | Documented | Missing controller | P1 |
|
||||
|
||||
## Delivery Sequence
|
||||
|
||||
1. Add Environment API controller surface backed by `IEnvironmentService`.
|
||||
2. Add Promotion API controller surface backed by `IPromotionManager` and
|
||||
decision services.
|
||||
3. Add decision/evidence endpoints with deterministic response models.
|
||||
4. Add approval-policy endpoints and SoD enforcement bindings.
|
||||
5. Add integration tests for end-to-end promotion request -> decision retrieval.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- Endpoint group implementation is tracked by API group with owning project path.
|
||||
- Promotion state transitions match `docs/modules/release-orchestrator/workflow/promotion.md`.
|
||||
- Decision records include policy digest and evidence references.
|
||||
- Fail-closed behavior is enforced when gate providers error.
|
||||
- Replay-oriented deterministic assertions are present in tests.
|
||||
|
||||
## Test Targets
|
||||
|
||||
- `src/ReleaseOrchestrator/__Tests/StellaOps.ReleaseOrchestrator.Promotion.Tests/`
|
||||
- `src/ReleaseOrchestrator/__Tests/StellaOps.ReleaseOrchestrator.Environment.Tests/`
|
||||
- `src/ReleaseOrchestrator/__Tests/StellaOps.ReleaseOrchestrator.Integration.Tests/`
|
||||
|
||||
Minimum acceptance test mapping:
|
||||
- Policy gate and decision flow: `Gate/PolicyGateTests.cs`, `Decision/DecisionEngineTests.cs`
|
||||
- Deterministic gate evaluation ordering: `Gate/GateEvaluatorTests.cs`
|
||||
- Environment promotion sequencing: `Environment.Tests/Services/EnvironmentServiceTests.cs`
|
||||
|
||||
## Related Contracts
|
||||
|
||||
- Policy ownership: `docs/modules/policy/promotion-gate-ownership-contract.md`
|
||||
- Evidence contract: `docs/modules/evidence-locker/promotion-evidence-contract.md`
|
||||
- Optional capsule profile: `docs/modules/release-orchestrator/appendices/promotion-capsule-optional.md`
|
||||
@@ -0,0 +1,95 @@
|
||||
# Evidence-Based Release Gates Contract
|
||||
|
||||
**Status:** Implemented baseline in promotion runtime (2026-02-10)
|
||||
**Related:** `docs/modules/release-orchestrator/workflow/promotion.md`, `docs/modules/attestor/repro-bundle-profile.md`, `docs/modules/evidence-locker/architecture.md`
|
||||
|
||||
## Purpose
|
||||
|
||||
Define a deterministic, policy-driven promotion gate model where release decisions are made from verifiable evidence, not ad hoc logic. This contract translates the evidence-based gate advisory into Stella Ops module boundaries and delivery tasks.
|
||||
|
||||
## Gate Policy Model (Data-Driven)
|
||||
|
||||
Gate policy is treated as versioned data (for example CUE compiled to JSON, or equivalent policy-pack JSON). At minimum, the gate policy profile must declare:
|
||||
|
||||
- `name`, `environment`
|
||||
- `evidence.min_score`, `evidence.required_links`, `evidence.product_digest_alg`
|
||||
- `signatures.k_of_n.k`, `signatures.k_of_n.n`, `signatures.allowed_keys`, `signatures.dsse_algos`
|
||||
- `rekor.required`, `rekor.instance_url`, `rekor.max_fresh_secs`
|
||||
- `retry.backoff_initial_ms`, `retry.backoff_factor`, `retry.max_retries`
|
||||
- `escalation.mode`, `escalation.human_queue`, `escalation.require_signed_human_decision`
|
||||
- `slo.p50_ms`, `slo.p90_ms`, `slo.p99_ms`, `slo.async_hold_sla_hours`, `slo.evidence_ttl_hours`
|
||||
|
||||
## Required Promotion Checks
|
||||
|
||||
Promotion gate evaluation must enforce all enabled checks below:
|
||||
|
||||
1. Evidence score threshold
|
||||
- Require `evidence_score >= policy.evidence.min_score`.
|
||||
|
||||
2. Rekor inclusion and freshness
|
||||
- Require inclusion proof present and verified when `policy.rekor.required=true`.
|
||||
- Require proof freshness within `policy.rekor.max_fresh_secs`.
|
||||
|
||||
3. In-toto build link + digest binding
|
||||
- Require `build` link when configured in `policy.evidence.required_links`.
|
||||
- Require build product digest to exactly match release artifact digest under `policy.evidence.product_digest_alg`.
|
||||
|
||||
4. DSSE signer threshold (k-of-n)
|
||||
- Count only valid signatures whose key IDs and algorithms are in policy allowlists.
|
||||
- Require `valid_unique_signers >= policy.signatures.k_of_n.k`.
|
||||
|
||||
## Decision Workflow Contract
|
||||
|
||||
Promotion flow for evidence gates:
|
||||
|
||||
`collect_evidence -> evaluate_gate_sync -> { approve | hold_async | escalate }`
|
||||
|
||||
Decision semantics:
|
||||
|
||||
- `approve`: all enabled checks pass.
|
||||
- `hold_async`: heavyweight evidence still pending; re-evaluate on evidence arrival or TTL expiry.
|
||||
- `escalate`: retries exhausted or policy requires human disposition; require DSSE-signed human decision when configured.
|
||||
|
||||
Runtime mapping:
|
||||
- `hold_async` is persisted as `DecisionOutcome.HoldAsync` (legacy `PendingGate` remains for compatibility).
|
||||
- `escalate` is persisted as `DecisionOutcome.Escalate`.
|
||||
|
||||
## Lane Defaults
|
||||
|
||||
- Sovereign and air-gap lanes default to `fail_closed`.
|
||||
- Dev lanes may use `fail_open_with_alert` only with mandatory audit record and alert emission.
|
||||
|
||||
## Audit and Replay Requirements
|
||||
|
||||
For each gate decision, persist immutable evidence pointers and gate inputs:
|
||||
|
||||
- policy identifier and version/digest
|
||||
- gate input hashes and evidence bundle references
|
||||
- DSSE envelope references and signer IDs counted toward k-of-n
|
||||
- Rekor UUIDs/log index/inclusion proof references and freshness timestamp
|
||||
- decision outcome, reason codes, retries, escalation routing
|
||||
- wall-clock timing metrics for SLO tracking
|
||||
- human decision DSSE reference when escalation occurs
|
||||
|
||||
## SLO Targets
|
||||
|
||||
Synchronous gate targets:
|
||||
|
||||
- P50 <= 200 ms
|
||||
- P90 <= 2 s
|
||||
- P99 <= 15 s
|
||||
|
||||
Asynchronous hold targets:
|
||||
|
||||
- `async_hold_sla_hours <= 12` (default profile)
|
||||
- `evidence_ttl_hours <= 24` with re-evaluation when stale
|
||||
|
||||
## Implementation Notes (2026-02-10)
|
||||
|
||||
- `SecurityGate` now enforces policy-configured checks for:
|
||||
- `minEvidenceScore` threshold (`SEC_REPRO_EVIDENCE_SCORE_THRESHOLD`)
|
||||
- in-toto build-link presence + digest binding (`SEC_REPRO_BUILD_LINK_MISSING`, `SEC_REPRO_BUILD_DIGEST_MISMATCH`)
|
||||
- DSSE k-of-n signer threshold (`SEC_REPRO_DSSE_THRESHOLD`)
|
||||
- Rekor freshness TTL with retry + escalation mode (`SEC_REPRO_REKOR_FRESHNESS_*`, `SEC_ESCALATION_*`)
|
||||
- Environment gate-specific policy config is now propagated into gate evaluation context by `DecisionEngine`.
|
||||
- Decision recording now captures SLO metadata (`p50/p90/p99`, async hold SLA, evidence TTL) and DSSE human decision reference when present.
|
||||
@@ -4,6 +4,12 @@
|
||||
|
||||
Promotions move releases through environments (Dev -> Staging -> Production). The promotion state machine manages the lifecycle from request to completion.
|
||||
|
||||
For policy-driven evidence gate inputs, checks, and outcomes, see `evidence-based-release-gates.md`.
|
||||
|
||||
Evidence-gate outcomes include explicit `hold_async` and `escalate` semantics in decision records:
|
||||
- `hold_async` maps to `DecisionOutcome.HoldAsync` for asynchronous evidence completion.
|
||||
- `escalate` maps to `DecisionOutcome.Escalate` when retry budgets are exhausted or lane policy requires human disposition.
|
||||
|
||||
## Promotion States
|
||||
|
||||
```
|
||||
|
||||
@@ -41,6 +41,8 @@ Scanner analyses container images layer-by-layer, producing deterministic SBOM f
|
||||
- ./analyzers-go.md
|
||||
- ./operations/secret-leak-detection.md
|
||||
- ./operations/dsse-rekor-operator-guide.md
|
||||
- ./operations/sbom-hot-lookup-operations.md
|
||||
- ./sbom-attestation-hot-lookup-profile.md
|
||||
- ./os-analyzers-evidence.md
|
||||
- ./design/macos-analyzer.md
|
||||
- ./design/windows-analyzer.md
|
||||
|
||||
@@ -3,7 +3,8 @@
|
||||
> Aligned with Epic 6 – Vulnerability Explorer and Epic 10 – Export Center.
|
||||
|
||||
> **Scope.** Implementation‑ready architecture for the **Scanner** subsystem: WebService, Workers, analyzers, SBOM assembly (inventory & usage), per‑layer caching, three‑way diffs, artifact catalog (RustFS default + PostgreSQL, S3-compatible fallback), attestation hand‑off, and scale/security posture. This document is the contract between the scanning plane and everything else (Policy, Excititor, Concelier, UI, CLI).
|
||||
> **Related:** `docs/modules/scanner/operations/ai-code-guard.md`
|
||||
> **Related:** `docs/modules/scanner/operations/ai-code-guard.md`
|
||||
> **Storage profile:** `docs/modules/scanner/sbom-attestation-hot-lookup-profile.md`
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -1,60 +1,76 @@
|
||||
# SLSA Source Track Capture (SC3)
|
||||
|
||||
Status: Draft · Date: 2025-12-03
|
||||
Scope: Define deterministic capture of SLSA Source Track data for replay bundles and CycloneDX 1.7 + CBOM exports. Aligns Scanner record/replay with provenance signals (build-id, repo/ref, provenance DSSE).
|
||||
Status: Active (partial implementation) | Last Updated: 2026-02-10
|
||||
Scope: Define deterministic capture of SLSA Source Track data for replay bundles and CycloneDX 1.7 + CBOM exports. Align scanner record/replay with source and build provenance signals.
|
||||
|
||||
## Objectives
|
||||
- Persist source provenance required by SLSA 1.2 Source Track: repo URI, resolved ref, digest of checked-out tree, invocation hash, builder ID, and reproducible build inputs.
|
||||
- Make data replayable offline: no network fetch; hashes + DSSE envelope paths must resolve locally.
|
||||
- Keep ordering/hashes deterministic: canonical JSON (sorted keys), BLAKE3-256 primary hash, SHA-256 secondary.
|
||||
- Persist source provenance required by SLSA 1.2 Source Track: repo URI, resolved ref, commit, source review controls, and policy snapshot signals.
|
||||
- Make data replayable offline with no network dependency.
|
||||
- Keep ordering and hashes deterministic with canonical JSON and explicit hash algorithm prefixes.
|
||||
|
||||
## Minimal fields (per build)
|
||||
- `source.repo`: canonical URI (https, ssh); normalized to lower-case host; trailing slash stripped.
|
||||
- `source.ref`: fully qualified ref (`refs/heads/main`, tag, or immutable commit).
|
||||
- `source.commit`: 40-hex commit digest.
|
||||
- `source.treeHash`: BLAKE3-256 of source tree snapshot (stable archive); optional SHA-256 mirror.
|
||||
- `build.invocation.hash`: BLAKE3-256 of normalized invocation (args/env/tool versions); also store `build.invocation.dsse` hash when signed.
|
||||
- `builder.id`: URI for builder identity (SLSA-style).
|
||||
- `provenance.dsse`: SHA-256 of DSSE envelope for provenance statement (e.g., in-toto SLSA provenance v1.0). Optionally include BLAKE3 and CAS URI.
|
||||
## Shipped Defaults (2026-02-10)
|
||||
- Build provenance policy supports Source Track controls:
|
||||
- `minimumReviewApprovals`
|
||||
- `requireNoSelfMerge`
|
||||
- `requireProtectedBranch`
|
||||
- `requireStatusChecksPassed`
|
||||
- `requirePolicyHash`
|
||||
- Source metadata is captured from build parameters using keys such as:
|
||||
- `sourceRef`
|
||||
- `sourceReviewCount` or `sourceApproverIds`
|
||||
- `sourceAuthorId` and `sourceMergedById`
|
||||
- `sourceBranchProtected`
|
||||
- `sourceStatusChecksPassed`
|
||||
- `sourcePolicyHash`
|
||||
- Source policy violations emit deterministic `SourcePolicyFailed` findings.
|
||||
- In-toto predicate output now includes source review and policy evidence fields.
|
||||
|
||||
## JSON shape (suggested)
|
||||
## Minimal Fields (Per Build)
|
||||
- `source.repo`: canonical repository URI.
|
||||
- `source.ref`: fully-qualified source ref (`refs/heads/main`, tag, or immutable commit).
|
||||
- `source.commit`: immutable source commit.
|
||||
- `source.review.count`: numeric review approval count.
|
||||
- `source.review.approvers`: sorted approver identity list.
|
||||
- `source.review.authorId`: source author identity.
|
||||
- `source.review.mergedById`: merge actor identity.
|
||||
- `source.branchProtected`: boolean signal from SCM policy enforcement.
|
||||
- `source.statusChecksPassed`: boolean signal for required CI checks.
|
||||
- `source.policyHash`: deterministic digest for branch/review policy snapshot.
|
||||
|
||||
## JSON Shape (Current Direction)
|
||||
```json
|
||||
{
|
||||
"source": {
|
||||
"repo": "https://example.invalid/demo",
|
||||
"ref": "refs/tags/v1.0.0",
|
||||
"ref": "refs/heads/main",
|
||||
"commit": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
|
||||
"treeHash": "b3:1111...",
|
||||
"builderId": "https://builder.stellaops.local/scanner",
|
||||
"invocationHash": "b3:2222...",
|
||||
"invocationDsse": "sha256:3333...",
|
||||
"provenance": {
|
||||
"dsse": "sha256:4444...",
|
||||
"cas": "cas://provenance/demo/v1.0.0.dsse"
|
||||
"policyHash": "sha256:policy123",
|
||||
"review": {
|
||||
"count": 2,
|
||||
"approvers": ["approver-a", "approver-b"],
|
||||
"authorId": "author-a",
|
||||
"mergedById": "approver-a",
|
||||
"branchProtected": true,
|
||||
"statusChecksPassed": true
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Where to store
|
||||
- CycloneDX 1.7 + CBOM: encode under `metadata.properties` using namespaced keys:
|
||||
- `source.repo`, `source.ref`, `source.commit`, `source.tree.hash`, `builder.id`, `build.invocation.hash`, `build.invocation.dsse`, `provenance.dsse`, `provenance.cas`.
|
||||
- Replay manifest: add `source` block mirroring the JSON shape above; include hashes in manifest subject list.
|
||||
- CAS: store provenance DSSE envelope under `cas://provenance/{component}/{version}.dsse`; store tree snapshot tarball under `cas://source/{commit}.tar.gz`.
|
||||
## Determinism Rules
|
||||
- Canonical JSON (lexicographic keys, UTF-8, no pretty-print) before hashing/signing.
|
||||
- UTC timestamps with `Z` suffix in exported provenance when timestamps are included.
|
||||
- Hash values must include algorithm prefix (`sha256:`, `b3:`).
|
||||
|
||||
## Determinism rules
|
||||
- Canonical JSON (lexicographic keys, UTF-8, no pretty-print) before hashing.
|
||||
- Timestamps in provenance statements must be UTC `Z`; strip milliseconds unless non-zero.
|
||||
- All hashes recorded with algorithm prefix (`b3:` for BLAKE3-256, `sha256:` for SHA-256).
|
||||
|
||||
## Verification
|
||||
- Verifier MUST: (1) schema-check fields are present; (2) recompute `treeHash` from tree tarball; (3) recompute `build.invocation.hash` from normalized invocation file; (4) verify DSSE envelope hash matches `provenance.dsse` and signature keys; (5) ensure repo/ref/commit are consistent (ref→commit mapping known or provided in bundle).
|
||||
- Fail closed on any mismatch; never fetch network.
|
||||
## Verification Expectations
|
||||
- Verifier fails closed when required Source Track controls are absent or violated.
|
||||
- Verifier links source control evidence (review, policy hash, branch/status signals) with build provenance identity.
|
||||
- No external fetch is allowed during verification.
|
||||
|
||||
## Fixtures
|
||||
- `docs/modules/scanner/fixtures/cdx17-cbom/source-track.sample.json` — deterministic example with placeholder hashes.
|
||||
- Future: add CAS tarball + invocation file under `tests/reachability/fixtures/source-track/` with recomputation script.
|
||||
- `docs/modules/scanner/fixtures/cdx17-cbom/source-track.sample.json`
|
||||
|
||||
## TODO (outside this doc)
|
||||
- Implement `scripts/scanner/verify_source_track.py` to validate source-track blocks and CAS payloads offline.
|
||||
- Extend replay manifest schema to include `source` block; add determinism tests in Scanner replay suite once manifest contract lands.
|
||||
## Remaining Work
|
||||
- Extend replay manifest schema to include source hash material (`treeHash`, invocation hash, DSSE hash) and offline recomputation assets.
|
||||
- Add a dedicated offline source-track verifier script for CAS-bound evidence inputs.
|
||||
- Add first-class SCM/CI attestation ingestion for source controls beyond parameter maps.
|
||||
|
||||
114
docs/modules/scanner/operations/sbom-hot-lookup-operations.md
Normal file
114
docs/modules/scanner/operations/sbom-hot-lookup-operations.md
Normal file
@@ -0,0 +1,114 @@
|
||||
# Scanner SBOM Hot Lookup Operations
|
||||
|
||||
Status: Active
|
||||
Last Updated: 2026-02-10
|
||||
Sprint: `SPRINT_20260210_001_DOCS_sbom_attestation_hot_lookup_contract` (`HOT-005`)
|
||||
|
||||
## Purpose
|
||||
|
||||
Operate the `scanner.artifact_boms` monthly partition set used by Scanner SBOM hot lookups:
|
||||
|
||||
- pre-create upcoming partitions to avoid month-boundary ingest failures
|
||||
- enforce retention windows by dropping old partitions
|
||||
- keep maintenance scoped to partition units (not whole-table rewrites)
|
||||
|
||||
## Required Inputs
|
||||
|
||||
- PostgreSQL DSN in `PG_DSN`
|
||||
- migration `025_artifact_boms_hot_lookup.sql` applied
|
||||
- permissions to execute:
|
||||
- `scanner.ensure_artifact_boms_future_partitions(int)`
|
||||
- `scanner.drop_artifact_boms_partitions_older_than(int, bool)`
|
||||
|
||||
## Manual Operations
|
||||
|
||||
Pre-create current + next month partition:
|
||||
|
||||
```bash
|
||||
PG_DSN="Host=...;Database=...;Username=...;Password=..." \
|
||||
./devops/scripts/scanner-artifact-boms-ensure-partitions.sh 1
|
||||
```
|
||||
|
||||
Retention dry-run (default keep 12 months):
|
||||
|
||||
```bash
|
||||
PG_DSN="Host=...;Database=...;Username=...;Password=..." \
|
||||
./devops/scripts/scanner-artifact-boms-retention.sh 12 true
|
||||
```
|
||||
|
||||
Retention execution:
|
||||
|
||||
```bash
|
||||
PG_DSN="Host=...;Database=...;Username=...;Password=..." \
|
||||
./devops/scripts/scanner-artifact-boms-retention.sh 12 false
|
||||
```
|
||||
|
||||
## Scheduled Jobs
|
||||
|
||||
### Cron example
|
||||
|
||||
```cron
|
||||
# first day each month: ensure next partition exists
|
||||
10 0 1 * * PG_DSN="..." /opt/stellaops/devops/scripts/scanner-artifact-boms-ensure-partitions.sh 1
|
||||
|
||||
# daily retention check
|
||||
15 0 * * * PG_DSN="..." /opt/stellaops/devops/scripts/scanner-artifact-boms-retention.sh 12 false
|
||||
```
|
||||
|
||||
### Systemd units
|
||||
|
||||
Install:
|
||||
|
||||
```bash
|
||||
sudo cp devops/scripts/systemd/scanner-artifact-boms-*.service /etc/systemd/system/
|
||||
sudo cp devops/scripts/systemd/scanner-artifact-boms-*.timer /etc/systemd/system/
|
||||
sudo systemctl daemon-reload
|
||||
sudo systemctl enable --now scanner-artifact-boms-ensure.timer
|
||||
sudo systemctl enable --now scanner-artifact-boms-retention.timer
|
||||
```
|
||||
|
||||
`/etc/stellaops/scanner-hotlookup.env` must define `PG_DSN`.
|
||||
|
||||
## Failure Modes and Rollback
|
||||
|
||||
### Missing upcoming partition
|
||||
|
||||
Symptom:
|
||||
- ingest errors near month boundary with partition routing failure.
|
||||
|
||||
Mitigation:
|
||||
1. Run `scanner-artifact-boms-ensure-partitions.sh 2`.
|
||||
2. Re-run failed ingest operations.
|
||||
|
||||
### Retention job dropped incorrect partition
|
||||
|
||||
Symptom:
|
||||
- historical hot-lookup rows unexpectedly missing.
|
||||
|
||||
Rollback:
|
||||
1. Restore dropped partition table from latest PostgreSQL backup.
|
||||
2. Attach restored table back to parent:
|
||||
```sql
|
||||
ALTER TABLE scanner.artifact_boms
|
||||
ATTACH PARTITION scanner.artifact_boms_YYYY_MM
|
||||
FOR VALUES FROM ('YYYY-MM-01') TO ('YYYY-MM-01'::date + INTERVAL '1 month');
|
||||
```
|
||||
3. Rebuild per-partition indexes if restore omitted them.
|
||||
|
||||
### Hot partition bloat
|
||||
|
||||
Symptom:
|
||||
- query latency regression on current month.
|
||||
|
||||
Mitigation:
|
||||
1. Run `VACUUM (ANALYZE) scanner.artifact_boms_YYYY_MM;`
|
||||
2. If needed, run `REINDEX TABLE scanner.artifact_boms_YYYY_MM;`
|
||||
3. For online reclaim workflows, use `pg_repack` partition-by-partition.
|
||||
|
||||
## References
|
||||
|
||||
- Schema + functions: `src/Scanner/__Libraries/StellaOps.Scanner.Storage/Postgres/Migrations/025_artifact_boms_hot_lookup.sql`
|
||||
- SQL job snippets: `devops/database/postgres-partitioning/003_scanner_artifact_boms_hot_lookup_jobs.sql`
|
||||
- Shell jobs:
|
||||
- `devops/scripts/scanner-artifact-boms-ensure-partitions.sh`
|
||||
- `devops/scripts/scanner-artifact-boms-retention.sh`
|
||||
204
docs/modules/scanner/sbom-attestation-hot-lookup-profile.md
Normal file
204
docs/modules/scanner/sbom-attestation-hot-lookup-profile.md
Normal file
@@ -0,0 +1,204 @@
|
||||
# Scanner SBOM and Attestation Hot Lookup Profile
|
||||
|
||||
Version: 0.1.0
|
||||
Status: Draft (Advisory translation)
|
||||
Last Updated: 2026-02-10
|
||||
|
||||
## Purpose
|
||||
|
||||
Define a Stella-compatible persistence profile for fast SBOM/attestation lookup queries in PostgreSQL without moving full replay/audit payloads out of CAS/object storage.
|
||||
|
||||
## Scope
|
||||
|
||||
This profile covers:
|
||||
- Hot OLTP lookup rows for digest, component, and pending-triage queries.
|
||||
- Partitioning, indexing, and retention for lookup tables.
|
||||
- Ingestion projection contracts from Scanner/Attestor outputs.
|
||||
|
||||
This profile does not replace:
|
||||
- CAS/object storage as source of truth for large immutable payloads.
|
||||
- Analytics star schema in `analytics.*`.
|
||||
- Existing proof bundle and witness contracts.
|
||||
|
||||
## Current Baseline (confirmed)
|
||||
|
||||
- Scanner stores full SBOM artifacts in object storage via `ArtifactStorageService` and `ArtifactObjectKeyBuilder`.
|
||||
- Scanner catalog metadata is stored in PostgreSQL (`scanner.artifacts`, `scanner.links`, related tables).
|
||||
- DSSE and proof metadata already use JSONB where needed (`proof_bundle.dsse_envelope`, `scanner.witnesses.dsse_envelope`).
|
||||
- High-volume partitioning currently exists for time-series style tables (for example EPSS and runtime samples), not for SBOM component lookup projections.
|
||||
- Component-level hot lookup acceleration is currently driven by the BOM-index sidecar contract.
|
||||
|
||||
## Advisory Fit Assessment
|
||||
|
||||
Aligned with current direction:
|
||||
- Keep exact-match routing keys narrow and indexed.
|
||||
- Use JSONB GIN indexes only on query paths that are actually hot.
|
||||
- Partition by time for deterministic retention.
|
||||
- Keep analytics and rollups away from Scanner OLTP hot paths.
|
||||
|
||||
Gap requiring implementation:
|
||||
- No explicit Scanner Postgres contract exists for a partitioned SBOM/attestation lookup projection that supports direct SQL lookups by payload digest, component PURL/name/version, and merged VEX pending state.
|
||||
|
||||
## Target Contract
|
||||
|
||||
### 1) Authoritative storage split
|
||||
|
||||
- Authoritative blobs:
|
||||
- `raw_bom`, canonical SBOM documents, DSSE envelopes, and merged VEX payloads remain in CAS/object storage.
|
||||
- PostgreSQL rows reference these via artifact IDs/URIs and store bounded JSONB projections for search.
|
||||
- Authoritative decisions:
|
||||
- Policy decisions remain in their existing modules.
|
||||
|
||||
### 2) Hot lookup table
|
||||
|
||||
Create a new append-only projection table:
|
||||
|
||||
```sql
|
||||
CREATE TABLE scanner.artifact_boms (
|
||||
build_id TEXT NOT NULL,
|
||||
canonical_bom_sha256 TEXT NOT NULL,
|
||||
payload_digest TEXT NOT NULL,
|
||||
inserted_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
|
||||
raw_bom_ref TEXT,
|
||||
canonical_bom_ref TEXT,
|
||||
dsse_envelope_ref TEXT,
|
||||
merged_vex_ref TEXT,
|
||||
|
||||
canonical_bom JSONB,
|
||||
merged_vex JSONB,
|
||||
attestations JSONB,
|
||||
|
||||
evidence_score INTEGER NOT NULL DEFAULT 0,
|
||||
rekor_tile_id TEXT,
|
||||
|
||||
PRIMARY KEY (build_id, inserted_at)
|
||||
) PARTITION BY RANGE (inserted_at);
|
||||
```
|
||||
|
||||
Partition policy:
|
||||
- Monthly range partitions.
|
||||
- `DEFAULT` partition optional for safety.
|
||||
- Retention by `DROP TABLE scanner.artifact_boms_YYYY_MM`.
|
||||
|
||||
### 3) Index profile
|
||||
|
||||
Required:
|
||||
|
||||
```sql
|
||||
CREATE INDEX IF NOT EXISTS ix_artifact_boms_payload_digest
|
||||
ON scanner.artifact_boms (payload_digest, inserted_at DESC);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS ix_artifact_boms_canonical_sha
|
||||
ON scanner.artifact_boms (canonical_bom_sha256);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS ix_artifact_boms_inserted_at
|
||||
ON scanner.artifact_boms (inserted_at DESC);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS ix_artifact_boms_canonical_gin
|
||||
ON scanner.artifact_boms USING GIN (canonical_bom jsonb_path_ops);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS ix_artifact_boms_merged_vex_gin
|
||||
ON scanner.artifact_boms USING GIN (merged_vex jsonb_path_ops);
|
||||
```
|
||||
|
||||
Optional partial index for pending triage:
|
||||
|
||||
```sql
|
||||
CREATE INDEX IF NOT EXISTS ix_artifact_boms_pending_vex
|
||||
ON scanner.artifact_boms USING GIN (merged_vex jsonb_path_ops)
|
||||
WHERE jsonb_path_exists(merged_vex, '$[*] ? (@.state == "unknown" || @.state == "triage_pending")');
|
||||
```
|
||||
|
||||
Uniqueness guard (optional):
|
||||
|
||||
```sql
|
||||
CREATE UNIQUE INDEX IF NOT EXISTS uq_artifact_boms_monthly_dedupe
|
||||
ON scanner.artifact_boms (canonical_bom_sha256, payload_digest, date_trunc('month', inserted_at));
|
||||
```
|
||||
|
||||
## Query Contracts
|
||||
|
||||
### Latest by payload digest
|
||||
|
||||
```sql
|
||||
SELECT build_id, inserted_at, evidence_score
|
||||
FROM scanner.artifact_boms
|
||||
WHERE payload_digest = $1
|
||||
ORDER BY inserted_at DESC
|
||||
LIMIT 1;
|
||||
```
|
||||
|
||||
### Component presence by PURL
|
||||
|
||||
```sql
|
||||
SELECT build_id, inserted_at
|
||||
FROM scanner.artifact_boms
|
||||
WHERE canonical_bom @? '$.components[*] ? (@.purl == $purl)'
|
||||
ORDER BY inserted_at DESC
|
||||
LIMIT 50;
|
||||
```
|
||||
|
||||
### Pending triage extraction
|
||||
|
||||
```sql
|
||||
SELECT build_id, inserted_at,
|
||||
jsonb_path_query_array(merged_vex, '$[*] ? (@.state == "unknown" || @.state == "triage_pending")') AS pending
|
||||
FROM scanner.artifact_boms
|
||||
WHERE jsonb_path_exists(merged_vex, '$[*] ? (@.state == "unknown" || @.state == "triage_pending")')
|
||||
ORDER BY inserted_at DESC
|
||||
LIMIT 100;
|
||||
```
|
||||
|
||||
## Implemented API Surface (Scanner WebService)
|
||||
|
||||
Base path: `/api/v1/sbom/hot-lookup`
|
||||
|
||||
- `GET /payload/{payloadDigest}/latest`
|
||||
- Returns latest projection row for digest.
|
||||
- `GET /components?purl=<purl>&limit=<n>&offset=<n>`
|
||||
- Component presence search by PURL.
|
||||
- `GET /components?name=<name>&minVersion=<version>&limit=<n>&offset=<n>`
|
||||
- Component presence search by normalized name and optional minimum version.
|
||||
- `GET /pending-triage?limit=<n>&offset=<n>`
|
||||
- Returns rows where merged VEX contains `unknown` or `triage_pending` states and includes extracted pending entries.
|
||||
|
||||
Pagination constraints:
|
||||
- `limit`: `1..200` (defaults: 50 for component searches, 100 for pending triage).
|
||||
- `offset`: `>= 0`.
|
||||
- Ordering is deterministic: `inserted_at DESC, build_id ASC`.
|
||||
|
||||
## Ingestion Contract
|
||||
|
||||
- Projectors should upsert on `(canonical_bom_sha256, payload_digest)` plus partition window.
|
||||
- `canonical_bom_sha256` must be computed from canonical JSON (stable ordering, UTF-8, deterministic normalization).
|
||||
- Projection rows must preserve deterministic timestamps (UTC) and stable JSON serialization.
|
||||
- If inline JSONB exceeds configured size thresholds, keep JSONB minimal and store full content only by CAS reference fields.
|
||||
|
||||
## Operations
|
||||
|
||||
- Autovacuum enabled per partition; tune by active partition write rate.
|
||||
- Reindex/repack operations should run per partition, never globally.
|
||||
- Partition creation job should pre-create at least one future month.
|
||||
- Retention job should drop old partitions according to policy (for example 90/180/365-day classes by environment).
|
||||
- Keep analytics workloads on `analytics.*`; export to external columnar systems when query volume exceeds OLTP SLO budgets.
|
||||
|
||||
Operational runbook and jobs:
|
||||
- Runbook: `docs/modules/scanner/operations/sbom-hot-lookup-operations.md`
|
||||
- SQL snippets: `devops/database/postgres-partitioning/003_scanner_artifact_boms_hot_lookup_jobs.sql`
|
||||
- Shell jobs:
|
||||
- `devops/scripts/scanner-artifact-boms-ensure-partitions.sh`
|
||||
- `devops/scripts/scanner-artifact-boms-retention.sh`
|
||||
- Systemd timers:
|
||||
- `devops/scripts/systemd/scanner-artifact-boms-ensure.timer`
|
||||
- `devops/scripts/systemd/scanner-artifact-boms-retention.timer`
|
||||
|
||||
## Security and Determinism Notes
|
||||
|
||||
- Do not store secrets in JSONB payloads.
|
||||
- DSSE and Rekor references must remain verifiable from immutable CAS/object artifacts.
|
||||
- Query responses used in policy decisions must preserve stable ordering and deterministic filtering rules.
|
||||
|
||||
## Delivery Link
|
||||
|
||||
- Implementation sprint: `docs/implplan/SPRINT_20260210_001_DOCS_sbom_attestation_hot_lookup_contract.md`
|
||||
Reference in New Issue
Block a user