Files

master c2c6b58b41 feat: Add Promotion-Time Attestations for Stella Ops

- Introduced a new document for promotion-time attestations, detailing the purpose, predicate schema, producer workflow, verification flow, APIs, and security considerations.
- Implemented the `stella.ops/promotion@v1` predicate schema to capture promotion evidence including image digest, SBOM/VEX artifacts, and Rekor proof.
- Defined producer responsibilities and workflows for CLI orchestration, signer responsibilities, and Export Center integration.
- Added verification steps for auditors to validate promotion attestations offline.

feat: Create Symbol Manifest v1 Specification

- Developed a specification for Symbol Manifest v1 to provide a deterministic format for publishing debug symbols and source maps.
- Defined the manifest structure, including schema, entries, source maps, toolchain, and provenance.
- Outlined upload and verification processes, resolve APIs, runtime proxy, caching, and offline bundle generation.
- Included security considerations and related tasks for implementation.

chore: Add Ruby Analyzer with Git Sources

- Created a Gemfile and Gemfile.lock for Ruby analyzer with dependencies on git-gem, httparty, and path-gem.
- Implemented main application logic to utilize the defined gems and output their versions.
- Added expected JSON output for the Ruby analyzer to validate the integration of the new gems and their functionalities.
- Developed internal observation classes for Ruby packages, runtime edges, and capabilities, including serialization logic for observations.

test: Add tests for Ruby Analyzer

- Created test fixtures for Ruby analyzer, including Gemfile, Gemfile.lock, main application, and expected JSON output.
- Ensured that the tests validate the correct integration and functionality of the Ruby analyzer with the specified gems.

2025-11-11 15:30:22 +02:00

3.6 KiB

Raw Blame History

Scanner Determinism Score Guide

Status: Draft – Sprint 186/202/203
Owners: Scanner Guild · QA Guild · DevEx/CLI Guild · DevOps Guild

1. Goal

Quantify how repeatable a scanner release is by re-running scans under frozen conditions and reporting the ratio of bit-for-bit identical outputs. The determinism score lets customers and auditors confirm that Stella Ops scans are replayable and trustworthy.

2. Test harness overview (`SCAN-DETER-186-009`)

Inputs: image digests, policy bundle SHA, feed snapshot SHA, scanner container digest, platform (linux/amd64 by default).
Execution loop: run the scanner N times (default 10) with:
- --fixed-clock <timestamp>
- RNG_SEED=1337
- SCANNER_MAX_CONCURRENCY=1
- feeds/policy tarballs mounted read-only
- --network=none, --cpuset-cpus=0, --memory=2G
Canonicalisation: normalise JSON outputs (SBOM, VEX, findings, logs) using the same serializer as production (StellaOps.Scanner.Replay helpers).
Hashing: compute SHA-256 for each canonical artefact per run.
Score calculation: identical_runs / total_runs (per image and overall). A run is “identical” if all artefact hashes match the baseline (run 1).

The harness persists the full run set under CAS, allowing regression tests and Offline kit inclusion.

3. Output artefacts (`SCAN-DETER-186-010`)

determinism.json – per-image runs, identical counts, score, policy/feed hashes.
run_i/*.json – canonicalised outputs for debugging.
diffs/ – optional diff samples when runs diverge.

Example determinism.json:

{
  "release": "scanner-0.14.3",
  "platform": "linux/amd64",
  "policy_sha": "a1b2c3…",
  "feeds_sha": "d4e5f6…",
  "images": [
    {
      "digest": "sha256:abc…",
      "runs": 10,
      "identical": 10,
      "score": 1.0,
      "artifact_hashes": {
        "sbom.cdx.json": "sha256:11…",
        "vex.json": "sha256:22…",
        "findings.json": "sha256:33…"
      }
    }
  ],
  "overall_score": 1.0
}

4. CI integration (`DEVOPS-SCAN-90-004`)

GitHub/Gitea pipeline stages run the determinism harness for the release matrix.
Fail the job when overall_score < threshold (default 0.95) or any image falls below 0.90.
Upload determinism.json and artefacts as build outputs; attach to release notes and Offline kits.

5. CLI support (`CLI-DETER-70-003/004`)

stella detscore run – executes the harness locally, honoring the same frozen-clock and seed settings; exits non-zero when score falls below the configured threshold.
stella detscore report – summarises one or more determinism.json files for release notes, showing per-image scores and detection of non-deterministic artefacts.

6. Policy & UI consumption

Policy Engine can enforce determinism thresholds (e.g., block promotion if score < 0.95) using the determinism.json evidence.
UI surfaces the score alongside scans (e.g., badge in scan detail view) referencing task UI-SBOM-DET-01.

7. Evidence & replay

Include determinism.json and canonical run outputs in Replay bundles (docs/replay/DETERMINISTIC_REPLAY.md).
DSSE-sign determinism results before adding them to Evidence Locker.

8. Implementation checklist

Area	Task ID	Notes
Harness	`SCAN-DETER-186-009`	Deterministic execution + hashing
Artefacts	`SCAN-DETER-186-010`	Publish JSON, CAS storage
CLI	`CLI-DETER-70-003/004`	Local runs + reporting
DevOps	`DEVOPS-SCAN-90-004`	CI enforcement
Docs	`DOCS-DETER-70-002`	(this document)

Update this guide with links to code once tasks move to DONE.

3.6 KiB Raw Blame History Unescape Escape