# Scanner Determinism Score Guide

> **Status:** Stable (2025-11)  
> **Owners:** Scanner Guild · QA Guild · DevEx/CLI Guild · DevOps Guild

## 1. Goal

Quantify how repeatable a scanner release is by re-running scans under frozen conditions and reporting the ratio of bit-for-bit identical outputs. The determinism score lets customers and auditors confirm that Stella Ops scans are replayable and trustworthy.

## 2. Test harness overview (`SCAN-DETER-186-009`)

1. **Inputs:** image digests, policy bundle SHA, feed snapshot SHA, scanner container digest, platform (linux/amd64 by default).
2. **Execution loop:** run the scanner *N* times (default 10) with:
   * `--fixed-clock <timestamp>`
   * `RNG_SEED=1337`
   * `SCANNER_MAX_CONCURRENCY=1`
   * feeds/policy tarballs mounted read-only
   * `--network=none`, `--cpuset-cpus=0`, `--memory=2G`
3. **Canonicalisation:** normalise JSON outputs (SBOM, VEX, findings, logs) using the same serializer as production (`StellaOps.Scanner.Replay` helpers).
4. **Hashing:** compute SHA-256 for each canonical artefact per run.
5. **Score calculation:** `identical_runs / total_runs` (per image and overall). A run is “identical” if all artefact hashes match the baseline (run 1).

The harness persists the full run set under CAS, allowing regression tests and Offline kit inclusion.

## 3. determinism.json schema (`SCAN-DETER-186-010`)

Required fields:

- `version` (string, default `1`)
- `release` (string; e.g., `scanner-0.14.3`)
- `platform` (`os/arch`)
- `policy_sha`, `feeds_sha`, `scanner_sha` (content-addressed digests)
- `images[]`:
  - `digest` (image digest)
  - `runs` (int), `identical` (int), `score` (float 0–1)
  - `artifact_hashes` (map filename → sha256)
  - `non_deterministic` (array of artefact names when diverged)
  - `notes` (optional string)
- `overall_score` (float 0–1)
- `thresholds` { `image_min`:0.90, `overall_min`:0.95 }
- `generated_at` (UTC ISO-8601)

Output bundle layout:

- `determinism.json` – schema above
- `run_i/*.json` – canonicalised artefacts per run
- `diffs/` – minimal diffs when divergence occurs

## 4. CI integration (`DEVOPS-SCAN-90-004`)

* GitHub/Gitea pipeline stages run the determinism harness for the release matrix.
* Fail the job when `overall_score < threshold` (default 0.95) or any image falls below 0.90.
* Upload `determinism.json` and artefacts as build outputs; attach to release notes and Offline kits.

## 5. CLI support (`CLI-DETER-70-003/004`)

* `stella detscore run` – executes the harness locally, honoring the same frozen-clock and seed settings; exits non-zero when score falls below the configured threshold.
* `stella detscore report` – summarises one or more `determinism.json` files for release notes, showing per-image scores and detection of non-deterministic artefacts.

## 6. Policy & UI consumption

* Policy Engine can enforce determinism thresholds (e.g., block promotion if score < 0.95) using the `determinism.json` evidence.
* UI surfaces the score alongside scans (e.g., badge in scan detail view) referencing task `UI-SBOM-DET-01`.

## 7. Evidence & replay

Replay instructions (deterministic, offline-friendly):

1. Fetch a published determinism bundle (`determinism.json` + `run_*`) and the corresponding scanner image.
2. Run `scripts/scanner/determinism-run.sh --replay --bundle path/to/bundle --image sha256:...` (uses fixed clock/seed).
3. Verify hashes match the recorded `artifact_hashes`; tool exits non-zero on mismatch.
4. Attach the bundle to Evidence Locker with DSSE signature before export.

## 8. Implementation checklist

| Area | Task ID | Notes |
|------|---------|-------|
| Harness | `SCAN-DETER-186-009` | Deterministic execution + hashing |
| Artefacts | `SCAN-DETER-186-010` | Publish JSON, CAS storage |
| CLI | `CLI-DETER-70-003/004` | Local runs + reporting |
| DevOps | `DEVOPS-SCAN-90-004` | CI enforcement |
| Docs | `DOCS-DETER-70-002` | (this document) |

Update this guide with links to code once tasks move to **DONE**.