up
Some checks failed
api-governance / spectral-lint (push) Has been cancelled
Docs CI / lint-and-preview (push) Has been cancelled
oas-ci / oas-validate (push) Has been cancelled
SDK Publish & Sign / sdk-publish (push) Has been cancelled
Policy Lint & Smoke / policy-lint (push) Has been cancelled
Policy Simulation / policy-simulate (push) Has been cancelled
devportal-offline / build-offline (push) Has been cancelled
Some checks failed
api-governance / spectral-lint (push) Has been cancelled
Docs CI / lint-and-preview (push) Has been cancelled
oas-ci / oas-validate (push) Has been cancelled
SDK Publish & Sign / sdk-publish (push) Has been cancelled
Policy Lint & Smoke / policy-lint (push) Has been cancelled
Policy Simulation / policy-simulate (push) Has been cancelled
devportal-offline / build-offline (push) Has been cancelled
This commit is contained in:
@@ -1,6 +1,6 @@
|
||||
# Scanner Determinism Score Guide
|
||||
|
||||
> **Status:** Draft – Sprint 186/202/203
|
||||
> **Status:** Stable (2025-11)
|
||||
> **Owners:** Scanner Guild · QA Guild · DevEx/CLI Guild · DevOps Guild
|
||||
|
||||
## 1. Goal
|
||||
@@ -22,36 +22,29 @@ Quantify how repeatable a scanner release is by re-running scans under frozen co
|
||||
|
||||
The harness persists the full run set under CAS, allowing regression tests and Offline kit inclusion.
|
||||
|
||||
## 3. Output artefacts (`SCAN-DETER-186-010`)
|
||||
## 3. determinism.json schema (`SCAN-DETER-186-010`)
|
||||
|
||||
* `determinism.json` – per-image runs, identical counts, score, policy/feed hashes.
|
||||
* `run_i/*.json` – canonicalised outputs for debugging.
|
||||
* `diffs/` – optional diff samples when runs diverge.
|
||||
Required fields:
|
||||
|
||||
Example `determinism.json`:
|
||||
- `version` (string, default `1`)
|
||||
- `release` (string; e.g., `scanner-0.14.3`)
|
||||
- `platform` (`os/arch`)
|
||||
- `policy_sha`, `feeds_sha`, `scanner_sha` (content-addressed digests)
|
||||
- `images[]`:
|
||||
- `digest` (image digest)
|
||||
- `runs` (int), `identical` (int), `score` (float 0–1)
|
||||
- `artifact_hashes` (map filename → sha256)
|
||||
- `non_deterministic` (array of artefact names when diverged)
|
||||
- `notes` (optional string)
|
||||
- `overall_score` (float 0–1)
|
||||
- `thresholds` { `image_min`:0.90, `overall_min`:0.95 }
|
||||
- `generated_at` (UTC ISO-8601)
|
||||
|
||||
```json
|
||||
{
|
||||
"release": "scanner-0.14.3",
|
||||
"platform": "linux/amd64",
|
||||
"policy_sha": "a1b2c3…",
|
||||
"feeds_sha": "d4e5f6…",
|
||||
"images": [
|
||||
{
|
||||
"digest": "sha256:abc…",
|
||||
"runs": 10,
|
||||
"identical": 10,
|
||||
"score": 1.0,
|
||||
"artifact_hashes": {
|
||||
"sbom.cdx.json": "sha256:11…",
|
||||
"vex.json": "sha256:22…",
|
||||
"findings.json": "sha256:33…"
|
||||
}
|
||||
}
|
||||
],
|
||||
"overall_score": 1.0
|
||||
}
|
||||
```
|
||||
Output bundle layout:
|
||||
|
||||
- `determinism.json` – schema above
|
||||
- `run_i/*.json` – canonicalised artefacts per run
|
||||
- `diffs/` – minimal diffs when divergence occurs
|
||||
|
||||
## 4. CI integration (`DEVOPS-SCAN-90-004`)
|
||||
|
||||
@@ -71,8 +64,12 @@ Example `determinism.json`:
|
||||
|
||||
## 7. Evidence & replay
|
||||
|
||||
* Include `determinism.json` and canonical run outputs in Replay bundles (`docs/replay/DETERMINISTIC_REPLAY.md`).
|
||||
* DSSE-sign determinism results before adding them to Evidence Locker.
|
||||
Replay instructions (deterministic, offline-friendly):
|
||||
|
||||
1. Fetch a published determinism bundle (`determinism.json` + `run_*`) and the corresponding scanner image.
|
||||
2. Run `scripts/scanner/determinism-run.sh --replay --bundle path/to/bundle --image sha256:...` (uses fixed clock/seed).
|
||||
3. Verify hashes match the recorded `artifact_hashes`; tool exits non-zero on mismatch.
|
||||
4. Attach the bundle to Evidence Locker with DSSE signature before export.
|
||||
|
||||
## 8. Implementation checklist
|
||||
|
||||
|
||||
Reference in New Issue
Block a user