feat: Add Promotion-Time Attestations for Stella Ops
- Introduced a new document for promotion-time attestations, detailing the purpose, predicate schema, producer workflow, verification flow, APIs, and security considerations. - Implemented the `stella.ops/promotion@v1` predicate schema to capture promotion evidence including image digest, SBOM/VEX artifacts, and Rekor proof. - Defined producer responsibilities and workflows for CLI orchestration, signer responsibilities, and Export Center integration. - Added verification steps for auditors to validate promotion attestations offline. feat: Create Symbol Manifest v1 Specification - Developed a specification for Symbol Manifest v1 to provide a deterministic format for publishing debug symbols and source maps. - Defined the manifest structure, including schema, entries, source maps, toolchain, and provenance. - Outlined upload and verification processes, resolve APIs, runtime proxy, caching, and offline bundle generation. - Included security considerations and related tasks for implementation. chore: Add Ruby Analyzer with Git Sources - Created a Gemfile and Gemfile.lock for Ruby analyzer with dependencies on git-gem, httparty, and path-gem. - Implemented main application logic to utilize the defined gems and output their versions. - Added expected JSON output for the Ruby analyzer to validate the integration of the new gems and their functionalities. - Developed internal observation classes for Ruby packages, runtime edges, and capabilities, including serialization logic for observations. test: Add tests for Ruby Analyzer - Created test fixtures for Ruby analyzer, including Gemfile, Gemfile.lock, main application, and expected JSON output. - Ensured that the tests validate the correct integration and functionality of the Ruby analyzer with the specified gems.
This commit is contained in:
master
87
docs/modules/scanner/determinism-score.md
Normal file
87
docs/modules/scanner/determinism-score.md
Normal file
@@ -0,0 +1,87 @@
|
||||
# Scanner Determinism Score Guide
|
||||
|
||||
> **Status:** Draft – Sprint 186/202/203
|
||||
> **Owners:** Scanner Guild · QA Guild · DevEx/CLI Guild · DevOps Guild
|
||||
|
||||
## 1. Goal
|
||||
|
||||
Quantify how repeatable a scanner release is by re-running scans under frozen conditions and reporting the ratio of bit-for-bit identical outputs. The determinism score lets customers and auditors confirm that Stella Ops scans are replayable and trustworthy.
|
||||
|
||||
## 2. Test harness overview (`SCAN-DETER-186-009`)
|
||||
|
||||
1. **Inputs:** image digests, policy bundle SHA, feed snapshot SHA, scanner container digest, platform (linux/amd64 by default).
|
||||
2. **Execution loop:** run the scanner *N* times (default 10) with:
|
||||
* `--fixed-clock <timestamp>`
|
||||
* `RNG_SEED=1337`
|
||||
* `SCANNER_MAX_CONCURRENCY=1`
|
||||
* feeds/policy tarballs mounted read-only
|
||||
* `--network=none`, `--cpuset-cpus=0`, `--memory=2G`
|
||||
3. **Canonicalisation:** normalise JSON outputs (SBOM, VEX, findings, logs) using the same serializer as production (`StellaOps.Scanner.Replay` helpers).
|
||||
4. **Hashing:** compute SHA-256 for each canonical artefact per run.
|
||||
5. **Score calculation:** `identical_runs / total_runs` (per image and overall). A run is “identical” if all artefact hashes match the baseline (run 1).
|
||||
|
||||
The harness persists the full run set under CAS, allowing regression tests and Offline kit inclusion.
|
||||
|
||||
## 3. Output artefacts (`SCAN-DETER-186-010`)
|
||||
|
||||
* `determinism.json` – per-image runs, identical counts, score, policy/feed hashes.
|
||||
* `run_i/*.json` – canonicalised outputs for debugging.
|
||||
* `diffs/` – optional diff samples when runs diverge.
|
||||
|
||||
Example `determinism.json`:
|
||||
|
||||
```json
|
||||
{
|
||||
"release": "scanner-0.14.3",
|
||||
"platform": "linux/amd64",
|
||||
"policy_sha": "a1b2c3…",
|
||||
"feeds_sha": "d4e5f6…",
|
||||
"images": [
|
||||
{
|
||||
"digest": "sha256:abc…",
|
||||
"runs": 10,
|
||||
"identical": 10,
|
||||
"score": 1.0,
|
||||
"artifact_hashes": {
|
||||
"sbom.cdx.json": "sha256:11…",
|
||||
"vex.json": "sha256:22…",
|
||||
"findings.json": "sha256:33…"
|
||||
}
|
||||
}
|
||||
],
|
||||
"overall_score": 1.0
|
||||
}
|
||||
```
|
||||
|
||||
## 4. CI integration (`DEVOPS-SCAN-90-004`)
|
||||
|
||||
* GitHub/Gitea pipeline stages run the determinism harness for the release matrix.
|
||||
* Fail the job when `overall_score < threshold` (default 0.95) or any image falls below 0.90.
|
||||
* Upload `determinism.json` and artefacts as build outputs; attach to release notes and Offline kits.
|
||||
|
||||
## 5. CLI support (`CLI-DETER-70-003/004`)
|
||||
|
||||
* `stella detscore run` – executes the harness locally, honoring the same frozen-clock and seed settings; exits non-zero when score falls below the configured threshold.
|
||||
* `stella detscore report` – summarises one or more `determinism.json` files for release notes, showing per-image scores and detection of non-deterministic artefacts.
|
||||
|
||||
## 6. Policy & UI consumption
|
||||
|
||||
* Policy Engine can enforce determinism thresholds (e.g., block promotion if score < 0.95) using the `determinism.json` evidence.
|
||||
* UI surfaces the score alongside scans (e.g., badge in scan detail view) referencing task `UI-SBOM-DET-01`.
|
||||
|
||||
## 7. Evidence & replay
|
||||
|
||||
* Include `determinism.json` and canonical run outputs in Replay bundles (`docs/replay/DETERMINISTIC_REPLAY.md`).
|
||||
* DSSE-sign determinism results before adding them to Evidence Locker.
|
||||
|
||||
## 8. Implementation checklist
|
||||
|
||||
| Area | Task ID | Notes |
|
||||
|------|---------|-------|
|
||||
| Harness | `SCAN-DETER-186-009` | Deterministic execution + hashing |
|
||||
| Artefacts | `SCAN-DETER-186-010` | Publish JSON, CAS storage |
|
||||
| CLI | `CLI-DETER-70-003/004` | Local runs + reporting |
|
||||
| DevOps | `DEVOPS-SCAN-90-004` | CI enforcement |
|
||||
| Docs | `DOCS-DETER-70-002` | (this document) |
|
||||
|
||||
Update this guide with links to code once tasks move to **DONE**.
|
||||
Reference in New Issue
Block a user