Here’s a tight, practical pattern to make your scanner’s vuln‑DB updates rock‑solid even when feeds hiccup: # Offline, verifiable update bundles (DSSE + Rekor v2) **Idea:** distribute DB updates as offline tarballs. Each tarball ships with: * a **DSSE‑signed** statement (e.g., in‑toto style) over the bundle hash * a **Rekor v2 receipt** proving the signature/statement was logged * a small **manifest.json** (version, created_at, content hashes) **Startup flow (happy path):** 1. Load latest tarball from your local `updates/` cache. 2. Verify DSSE signature against your trusted public keys. 3. Verify Rekor v2 receipt (inclusion proof) matches the DSSE payload hash. 4. If both pass, unpack/activate; record the bundle’s **trust_id** (e.g., statement digest). 5. If anything fails, **keep using the last good bundle**. No service disruption. **Why this helps** * **Air‑gap friendly:** no live network needed at activation time. * **Tamper‑evident:** DSSE + Rekor receipt proves provenance and transparency. * **Operational stability:** feed outages become non‑events—scanner just keeps the last good state. --- ## File layout inside each bundle ``` /bundle-2025-11-29/ manifest.json # { version, created_at, entries[], sha256s } payload.tar.zst # the actual DB/indices payload.tar.zst.sha256 statement.dsse.json # DSSE-wrapped statement over payload hash rekor-receipt.json # Rekor v2 inclusion/verification material ``` --- ## Acceptance/Activation rules * **Trust root:** pin one (or more) publisher public keys; rotate via separate, out‑of‑band process. * **Monotonicity:** only activate if `manifest.version > current.version` (or if trust policy explicitly allows replay for rollback testing). * **Atomic switch:** unpack to `db/staging/`, validate, then symlink‑flip to `db/active/`. * **Quarantine on failure:** move bad bundles to `updates/quarantine/` with a reason code. --- ## Minimal .NET 10 verifier sketch (C#) ```csharp public sealed record BundlePaths(string Dir) { public string Manifest => Path.Combine(Dir, "manifest.json"); public string Payload => Path.Combine(Dir, "payload.tar.zst"); public string Dsse => Path.Combine(Dir, "statement.dsse.json"); public string Receipt => Path.Combine(Dir, "rekor-receipt.json"); } public async Task ActivateBundleAsync(BundlePaths b, TrustConfig trust, string activeDir) { var manifest = await Manifest.LoadAsync(b.Manifest); if (!await Hashes.VerifyAsync(b.Payload, manifest.PayloadSha256)) return false; // 1) DSSE verify (publisher keys pinned in trust) var (okSig, dssePayloadDigest) = await Dsse.VerifyAsync(b.Dsse, trust.PublisherKeys); if (!okSig || dssePayloadDigest != manifest.PayloadSha256) return false; // 2) Rekor v2 receipt verify (inclusion + statement digest == dssePayloadDigest) if (!await RekorV2.VerifyReceiptAsync(b.Receipt, dssePayloadDigest, trust.RekorPub)) return false; // 3) Stage, validate, then atomically flip var staging = Path.Combine(activeDir, "..", "staging"); DirUtil.Empty(staging); await TarZstd.ExtractAsync(b.Payload, staging); if (!await LocalDbSelfCheck.RunAsync(staging)) return false; SymlinkUtil.AtomicSwap(source: staging, target: activeDir); State.WriteLastGood(manifest.Version, dssePayloadDigest); return true; } ``` --- ## Operational playbook * **On boot & daily at HH:MM:** try `ActivateBundleAsync()` on the newest bundle; on failure, log and continue. * **Telemetry (no PII):** reason codes (SIG_FAIL, RECEIPT_FAIL, HASH_MISMATCH, SELFTEST_FAIL), versions, last_good. * **Keys & rotation:** keep `publisher.pub` and `rekor.pub` in a root‑owned, read‑only path; rotate via a separate signed “trust bundle”. * **Defense‑in‑depth:** verify both the **payload hash** and each file’s hash listed in `manifest.entries[]`. * **Rollback:** allow `--force-activate ` for emergency testing, but mark as **non‑monotonic** in state. --- ## What to hand your release team * A Make/CI target that: 1. Builds `payload.tar.zst` and computes hashes 2. Generates `manifest.json` 3. Creates and signs the **DSSE statement** 4. Submits to Rekor (or your mirror) and saves the **v2 receipt** 5. Packages the bundle folder and publishes to your offline repo * A checksum file (`*.sha256sum`) for ops to verify out‑of‑band. --- If you want, I can turn this into a Stella Ops spec page (`docs/modules/scanner/offline-bundles.md`) plus a small reference implementation (C# library + CLI) that drops right into your Scanner service. Here’s a “drop‑in” Stella Ops dev guide for **DSSE‑signed Offline Scanner Updates** — written in the same spirit as the existing docs and sprint files. You can treat this as the seed for `docs/modules/scanner/development/dsse-offline-updates.md` (or similar). --- # DSSE‑Signed Offline Scanner Updates — Developer Guidelines > **Audience** > Scanner, Export Center, Attestor, CLI, and DevOps engineers implementing DSSE‑signed offline vulnerability updates and integrating them into the Offline Update Kit (OUK). > > **Context** > > * OUK already ships **signed, atomic offline update bundles** with merged vulnerability feeds, container images, and an attested manifest.([git.stella-ops.org][1]) > * DSSE + Rekor is already used for **scan evidence** (SBOM attestations, Rekor proofs).([git.stella-ops.org][2]) > * Sprints 160/162 add **attestation bundles** with manifest, checksums, DSSE signature, and optional transparency log segments, and integrate them into OUK and CLI flows.([git.stella-ops.org][3]) These guidelines tell you how to **wire all of that together** for “offline scanner updates” (feeds, rules, packs) in a way that matches Stella Ops’ determinism + sovereignty promises. --- ## 0. Mental model At a high level, you’re building this: ```text Advisory mirrors / Feeds builders │ ▼ ExportCenter.AttestationBundles (creates DSSE + Rekor evidence for each offline update snapshot) │ ▼ Offline Update Kit (OUK) builder (adds feeds + evidence to kit tarball) │ ▼ stella offline kit import / admin CLI (verifies Cosign + DSSE + Rekor segments, then atomically swaps scanner feeds) ``` Online, Rekor is live; offline, you rely on **bundled Rekor segments / snapshots** and the existing OUK mechanics (import is atomic, old feeds kept until new bundle is fully verified).([git.stella-ops.org][1]) --- ## 1. Goals & non‑goals ### Goals 1. **Authentic offline snapshots** Every offline scanner update (OUK or delta) must be verifiably tied to: * a DSSE envelope, * a certificate chain rooted in Stella’s Fulcio/KMS profile or BYO KMS/HSM, * *and* a Rekor v2 inclusion proof or bundled log segment.([Stella Ops][4]) 2. **Deterministic replay** Given: * a specific offline update kit (`stella-ops-offline-kit-.tgz` + `offline-manifest-.json`)([git.stella-ops.org][1]) * its DSSE attestation bundle + Rekor segments every verifier must reach the *same* verdict on integrity and contents — online or fully air‑gapped. 3. **Separation of concerns** * Export Center: build attestation bundles, no business logic about scanning.([git.stella-ops.org][5]) * Scanner: import & apply feeds; verify but not generate DSSE. * Signer / Attestor: own DSSE & Rekor integration.([git.stella-ops.org][2]) 4. **Operational safety** * Imports remain **atomic and idempotent**. * Old feeds stay live until the new update is **fully verified** (Cosign + DSSE + Rekor).([git.stella-ops.org][1]) ### Non‑goals * Designing new crypto or log formats. * Per‑feed DSSE envelopes (you can have more later, but the minimum contract is **bundle‑level** attestation). --- ## 2. Bundle contract for DSSE‑signed offline updates You’re extending the existing OUK contract: * OUK already packs: * merged vuln feeds (OSV, GHSA, optional NVD 2.0, CNNVD/CNVD, ENISA, JVN, BDU), * container images (`stella-ops`, Zastava, etc.), * provenance (Cosign signature, SPDX SBOM, in‑toto SLSA attestation), * `offline-manifest.json` + detached JWS signed during export.([git.stella-ops.org][1]) For **DSSE‑signed offline scanner updates**, add a new logical layer: ### 2.1. Files to ship Inside each offline kit (full or delta) you must produce: ```text /attestations/ offline-update.dsse.json # DSSE envelope offline-update.rekor.json # Rekor entry + inclusion proof (or segment descriptor) /manifest/ offline-manifest.json # existing manifest offline-manifest.json.jws # existing detached JWS /feeds/ ... # existing feed payloads ``` The exact paths can be adjusted, but keep: * **One DSSE bundle per kit** (min spec). * **One canonical Rekor proof file** per DSSE envelope. ### 2.2. DSSE payload contents (minimal) Define (or reuse) a predicate type such as: ```jsonc { "payloadType": "application/vnd.in-toto+json", "payload": { /* base64 */ } } ``` Decoded payload (in-toto statement) should **at minimum** contain: * **Subject** * `name`: `stella-ops-offline-kit-.tgz` * `digest.sha256`: tarball digest * **Predicate type** (recommendation) * `https://stella-ops.org/attestations/offline-update/1` * **Predicate fields** * `offline_manifest_sha256` – SHA‑256 of `offline-manifest.json` * `feeds` – array of feed entries such as `{ name, snapshot_date, archive_digest }` (mirrors `rules_and_feeds` style used in the moat doc).([Stella Ops][6]) * `builder` – CI workflow id / git commit / Export Center job id * `created_at` – UTC ISO‑8601 * `oukit_channel` – e.g., `edge`, `stable`, `fips-profile` **Guideline:** this DSSE payload is the **single canonical description** of “what this offline update snapshot is”. ### 2.3. Rekor material Attestor must: * Submit `offline-update.dsse.json` to Rekor v2, obtaining: * `uuid` * `logIndex` * inclusion proof (`rootHash`, `hashes`, `checkpoint`) * Serialize that to `offline-update.rekor.json` and store it in object storage + OUK staging, so it ships in the kit.([git.stella-ops.org][2]) For fully offline operation: * Either: * embed a **minimal log segment** containing that entry; or * rely on daily Rekor snapshot exports included elsewhere in the kit.([git.stella-ops.org][2]) --- ## 3. Implementation by module ### 3.1 Export Center — attestation bundles **Working directory:** `src/ExportCenter/StellaOps.ExportCenter.AttestationBundles`([git.stella-ops.org][7]) **Responsibilities** 1. **Compose attestation bundle job** (EXPORT‑ATTEST‑74‑001) * Input: a snapshot identifier (e.g., offline kit build id or feed snapshot date). * Read manifest and feed metadata from the Export Center’s storage.([git.stella-ops.org][5]) * Generate the DSSE payload structure described above. * Call `StellaOps.Signer` to wrap it in a DSSE envelope. * Call `StellaOps.Attestor` to submit DSSE → Rekor and get the inclusion proof.([git.stella-ops.org][2]) * Persist: * `offline-update.dsse.json` * `offline-update.rekor.json` * any log segment artifacts. 2. **Integrate into offline kit packaging** (EXPORT‑ATTEST‑74‑002 / 75‑001) * The OUK builder (Python script `ops/offline-kit/build_offline_kit.py`) already assembles artifacts & manifests.([Stella Ops][8]) * Extend that pipeline (or add an Export Center step) to: * fetch the attestation bundle for the snapshot, * place it under `/attestations/` in the kit staging dir, * ensure `offline-manifest.json` contains entries for the DSSE and Rekor files (name, sha256, size, capturedAt).([git.stella-ops.org][1]) 3. **Contracts & schemas** * Define a small JSON schema for `offline-update.rekor.json` (UUID, index, proof fields) and check it into `docs/11_DATA_SCHEMAS.md` or module‑local schemas. * Keep all new payload schemas **versioned**; avoid “shape drift”. **Do / Don’t** * ✅ **Do** treat attestation bundle job as *pure aggregation* (AOC guardrail: no modification of evidence).([git.stella-ops.org][5]) * ✅ **Do** rely on Signer + Attestor; don’t hand‑roll DSSE/Rekor logic in Export Center.([git.stella-ops.org][2]) * ❌ **Don’t** reach out to external networks from this job — it must run with the same offline‑ready posture as the rest of the platform. --- ### 3.2 Offline Update Kit builder **Working area:** `ops/offline-kit/*` + `docs/24_OFFLINE_KIT.md`([git.stella-ops.org][1]) Guidelines: 1. **Preserve current guarantees** * Imports must remain **idempotent and atomic**, with **old feeds kept until the new bundle is fully verified**. This now includes DSSE/Rekor checks in addition to Cosign + JWS.([git.stella-ops.org][1]) 2. **Staging layout** * When staging a kit, ensure the tree looks like: ```text out/offline-kit/staging/ feeds/... images/... manifest/offline-manifest.json attestations/offline-update.dsse.json attestations/offline-update.rekor.json ``` * Update `offline-manifest.json` so each new file appears with: * `name`, `sha256`, `size`, `capturedAt`.([git.stella-ops.org][1]) 3. **Deterministic ordering** * File lists in manifests must be in a stable order (e.g., lexical paths). * Timestamps = UTC ISO‑8601 only; never use local time. (Matches determinism guidance in AGENTS.md + policy/runs docs.)([git.stella-ops.org][9]) 4. **Delta kits** * For deltas (`stella-ouk-YYYY-MM-DD.delta.tgz`), DSSE should still cover: * the delta tarball digest, * the **logical state** (feeds & versions) after applying the delta. * Don’t shortcut by “attesting only the diff files” — the predicate must describe the resulting snapshot. --- ### 3.3 Scanner — import & activation **Working directory:** `src/Scanner/StellaOps.Scanner.WebService`, `StellaOps.Scanner.Worker`([git.stella-ops.org][9]) Scanner already exposes admin flows for: * **Offline kit import**, which: * validates the Cosign signature of the kit, * uses the attested manifest, * keeps old feeds until verification is done.([git.stella-ops.org][1]) Add DSSE/Rekor awareness as follows: 1. **Verification sequence (happy path)** On `import-offline-usage-kit`: 1. Validate **Cosign** signature of the tarball. 2. Validate `offline-manifest.json` with its JWS signature. 3. Verify **file digests** for all entries (including `/attestations/*`). 4. Verify **DSSE**: * Call `StellaOps.Attestor.Verify` (or CLI equivalent) with: * `offline-update.dsse.json` * `offline-update.rekor.json` * local Rekor log snapshot / segment (if configured)([git.stella-ops.org][2]) * Ensure the payload digest matches the kit tarball + manifest digests. 5. Only after all checks pass: * swap Scanner’s feed pointer to the new snapshot, * emit an audit event noting: * kit filename, tarball digest, * DSSE statement digest, * Rekor UUID + log index. 2. **Config surface** Add config keys (names illustrative): ```yaml scanner: offlineKit: requireDsse: true # fail import if DSSE/Rekor verification fails rekorOfflineMode: true # use local snapshots only attestationVerifier: https://attestor.internal ``` * Mirror them via ASP.NET Core config + env vars (`SCANNER__OFFLINEKIT__REQUIREDSSSE`, etc.), following the same pattern as the DSSE/Rekor operator guide.([git.stella-ops.org][2]) 3. **Failure behaviour** * **DSSE/Rekor fail, Cosign + manifest OK** * Keep old feeds active. * Mark import as failed; surface a `ProblemDetails` error via API/UI. * Log structured fields: `rekorUuid`, `attestationDigest`, `offlineKitHash`, `failureReason`.([git.stella-ops.org][2]) * **Config flag to soften during rollout** * When `requireDsse=false`, treat DSSE/Rekor failure as a warning and still allow the import (for initial observation phase), but emit alerts. This mirrors the “observe → enforce” pattern in the DSSE/Rekor operator guide.([git.stella-ops.org][2]) --- ### 3.4 Signer & Attestor You mostly **reuse** existing guidance:([git.stella-ops.org][2]) * Add a new predicate type & schema for offline updates in Signer. * Ensure Attestor: * can submit offline‑update DSSE envelopes to Rekor, * can emit verification routines (used by CLI and Scanner) that: * verify the DSSE signature, * check the certificate chain against the configured root pack (FIPS/eIDAS/GOST/SM, etc.),([Stella Ops][4]) * verify Rekor inclusion using either live log or local snapshot. * For fully air‑gapped installs: * rely on Rekor **snapshots mirrored** into Offline Kit (already recommended in the operator guide’s offline section).([git.stella-ops.org][2]) --- ### 3.5 CLI & UI Extend CLI with explicit verbs (matching EXPORT‑ATTEST sprints):([git.stella-ops.org][10]) * `stella attest bundle verify --bundle path/to/offline-kit.tgz --rekor-key rekor.pub` * `stella attest bundle import --bundle ...` (for sites that prefer a two‑step “verify then import” flow) * Wire UI Admin → Offline Kit screen so that: * verification status shows both **Cosign/JWS** and **DSSE/Rekor** state, * policy banners display kit generation time, manifest hash, and DSSE/Rekor freshness.([Stella Ops][11]) --- ## 4. Determinism & offline‑safety rules When touching any of this code, keep these rules front‑of‑mind (they align with the policy DSL and architecture docs):([Stella Ops][4]) 1. **No hidden network dependencies** * All verification **must work offline** given the kit + Rekor snapshots. * Any fallback to live Rekor / Fulcio endpoints must be explicitly toggled and never on by default for “offline mode”. 2. **Stable serialization** * DSSE payload JSON: * stable ordering of fields, * no float weirdness, * UTC timestamps. 3. **Replayable imports** * Running `import-offline-usage-kit` twice with the same bundle must be a no‑op after the first time. * The DSSE payload for a given snapshot must not change over time; if it does, bump the predicate or snapshot version. 4. **Explainability** * When verification fails, errors must explain **what** mismatched (kit digest, manifest digest, DSSE envelope hash, Rekor inclusion) so auditors can reason about it. --- ## 5. Testing & CI expectations Tie this into the existing CI workflows (`scanner-determinism.yml`, `attestation-bundle.yml`, `offline-kit` pipelines, etc.):([git.stella-ops.org][12]) ### 5.1 Unit & integration tests Write tests that cover: 1. **Happy paths** * Full kit import with valid: * Cosign, * manifest JWS, * DSSE, * Rekor proof (online and offline modes). 2. **Corruption scenarios** * Tampered feed file (hash mismatch). * Tampered `offline-manifest.json`. * Tampered DSSE payload (signature fails). * Mismatched Rekor entry (payload digest doesn’t match DSSE). 3. **Offline scenarios** * No network access, only Rekor snapshot: * DSSE verification still passes, * Rekor proof validates against local tree head. 4. **Roll‑back logic** * Import fails at DSSE/Rekor step: * scanner DB still points at previous feeds, * metrics/logs show failure and no partial state. ### 5.2 SLOs & observability Reuse metrics suggested by DSSE/Rekor guide and adapt to OUK imports:([git.stella-ops.org][2]) * `offlinekit_import_total{status="success|failed_dsse|failed_rekor|failed_cosign"}` * `offlinekit_attestation_verify_latency_seconds` (histogram) * `attestor_rekor_success_total`, `attestor_rekor_retry_total`, `rekor_inclusion_latency` * Dashboards: kit versions per environment, time since last kit, DSSE/Rekor health. --- ## 6. Developer checklist (TL;DR) When you pick up a task touching DSSE‑signed offline updates: 1. **Read the background** * `docs/modules/scanner/operations/dsse-rekor-operator-guide.md`([git.stella-ops.org][2]) * `docs/24_OFFLINE_KIT.md` (and public offline kit guide).([git.stella-ops.org][1]) * Relevant sprint file (`SPRINT_160_export_evidence`, `SPRINT_162_exportcenter_i`, etc.).([git.stella-ops.org][10]) 2. **Implement** * Generate DSSE payloads in Export Center only. * Call Signer & Attestor; persist DSSE + Rekor JSON next to manifests. * Extend OUK builder to include attestation bundle and list it in `offline-manifest.json`. * Update Scanner import flow to verify DSSE/Rekor before swapping feeds. 3. **Test** * Unit tests for bundle composition & schema. * Integration tests for import + rollback. * Determinism tests (same inputs → same DSSE payload). 4. **Wire telemetry** * Counters + latency histograms. * Logs with `offlineKitHash`, `attestationDigest`, `rekorUuid`. 5. **Document** * Update `docs/modules/export-center/architecture.md`, `docs/modules/scanner/architecture.md`, and the OUK docs where flows or contracts changed.([git.stella-ops.org][5]) --- If you tell me which module you’re actually coding in next (Scanner, Export Center, CLI, or Attestor), I can turn this into a very concrete “AGENTS.md‑style” section with exact file paths, class names, and a starter test layout for that module. [1]: https://git.stella-ops.org/stella-ops.org/git.stella-ops.org/src/commit/7bf40b8589c94078e8eadb240553c02f097a5127/docs/24_OFFLINE_KIT.md "git.stella-ops.org/24_OFFLINE_KIT.md at 7bf40b8589c94078e8eadb240553c02f097a5127 - git.stella-ops.org - Gitea: Git with a cup of tea" [2]: https://git.stella-ops.org/stella-ops.org/git.stella-ops.org/src/commit/13e4b53dda1575ba46c6188c794fd465ec6fdeec/docs/modules/scanner/operations/dsse-rekor-operator-guide.md "git.stella-ops.org/dsse-rekor-operator-guide.md at 13e4b53dda1575ba46c6188c794fd465ec6fdeec - git.stella-ops.org - Gitea: Git with a cup of tea" [3]: https://git.stella-ops.org/stella-ops.org/git.stella-ops.org/raw/commit/61f963fd52cd4d6bb2f86afc5a82eac04c04b00e/docs/implplan/SPRINT_162_exportcenter_i.md?utm_source=chatgpt.com "https://git.stella-ops.org/stella-ops.org/git.stel..." [4]: https://stella-ops.org/docs/07_high_level_architecture/index.html?utm_source=chatgpt.com "Open • Sovereign • Modular container security - Stella Ops" [5]: https://git.stella-ops.org/stella-ops.org/git.stella-ops.org/src/commit/d870da18ce194c6a5f1a6d71abea36205d9fb276/docs/export-center/architecture.md?utm_source=chatgpt.com "Export Center Architecture - Stella Ops" [6]: https://stella-ops.org/docs/moat/?utm_source=chatgpt.com "Open • Sovereign • Modular container security - Stella Ops" [7]: https://git.stella-ops.org/stella-ops.org/git.stella-ops.org/src/commit/79b8e53441e92dbc63684f42072434d40b80275f/src/ExportCenter?utm_source=chatgpt.com "Code - Stella Ops" [8]: https://stella-ops.org/docs/24_offline_kit/?utm_source=chatgpt.com "Offline Update Kit (OUK) — Air‑Gap Bundle - Stella Ops – Open ..." [9]: https://git.stella-ops.org/stella-ops.org/git.stella-ops.org/src/commit/7768555f2d107326050cc5ff7f5cb81b82b7ce5f/AGENTS.md "git.stella-ops.org/AGENTS.md at 7768555f2d107326050cc5ff7f5cb81b82b7ce5f - git.stella-ops.org - Gitea: Git with a cup of tea" [10]: https://git.stella-ops.org/stella-ops.org/git.stella-ops.org/src/commit/66cb6c4b8af58a33efa1521b7953dda834431497/docs/implplan/SPRINT_160_export_evidence.md?utm_source=chatgpt.com "git.stella-ops.org/SPRINT_160_export_evidence.md at ..." [11]: https://stella-ops.org/about/?utm_source=chatgpt.com "Signed Reachability · Deterministic Replay · Sovereign Crypto" [12]: https://git.stella-ops.org/stella-ops.org/git.stella-ops.org/actions/?actor=0&status=0&workflow=sdk-publish.yml&utm_source=chatgpt.com "Actions - git.stella-ops.org - Gitea: Git with a cup of tea"