docs re-org, audit fixes, build fixes

2026-01-05 09:35:33 +02:00
parent eca4e964d3
commit dfab8a29c3
173 changed files with 1276 additions and 560 deletions
--- a/docs/OFFLINE_KIT.md
+++ b/docs/OFFLINE_KIT.md
@@ -0,0 +1,661 @@
+# Offline Update Kit (OUK) — Air‑Gap Bundle
+
+<!--
+  Build‑time variable injection:
+    {{ quota_anon }}   = 33
+    {{ quota_token }}  = 333
+    {{ dotnet }}       = "10 LTS"
+-->
+
+The **Offline Update Kit** packages everything Stella Ops needs to run on a
+completely isolated network:
+
+| Component | Contents |
+|-----------|----------|
+| **Merged vulnerability feeds** | OSV, GHSA plus optional NVD 2.0, CNNVD, CNVD, ENISA, JVN and BDU |
+| **Container images** | `stella-ops`, *Zastava* sidecar, `advisory-ai-web`, and `advisory-ai-worker` (x86‑64 & arm64) |
+| **Provenance** | Cosign signature, SPDX 2.3 SBOM, in‑toto SLSA attestation |
+| **Attested manifest** | `offline-manifest.json` + detached JWS covering bundle metadata, signed during export. |
+| **Delta patches** | Daily diff bundles keep size \< 350 MB |
+| **Scanner plug-ins** | OS analyzers plus the Node.js, Go, .NET, Python, Ruby, Rust, and PHP language analyzers packaged under `plugins/scanner/analyzers/**` with manifests so Workers load deterministically offline. |
+| **Debug store** | `.debug` artefacts laid out under `debug/.build-id/<aa>/<rest>.debug` with `debug/debug-manifest.json` mapping build-ids to originating images for symbol retrieval. |
+| **Secret Detection Rules** | DSSE-signed rule bundles under `rules/secrets/<version>/` with manifest, JSONL rules, and signature envelope for air-gapped secret leak detection. |
+| **Telemetry collector bundle** | `telemetry/telemetry-offline-bundle.tar.gz` plus `.sha256`, containing OTLP collector config, Helm/Compose overlays, and operator instructions. |
+| **CLI + Task Packs** | `cli/` binaries from `release/cli`, Task Runner bootstrap (`bootstrap/task-runner/task-runner.yaml.sample`), and task-pack docs under `docs/task-packs/**` + `docs/modules/taskrunner/**`. |
+| **Orchestrator/Export/Notifier kits** | Orchestrator service, worker SDK, Postgres snapshot, dashboards (`orchestrator/**`), Export Center bundles (`export-center/**`), Notifier offline packs (`notifier/**`). |
+| **Container air-gap bundles** | Any tar/tgz under `containers/` or `images/` (mirrored registries) plus `docs/airgap/mirror-bundles.md`. |
+| **Surface.Secrets** | Encrypted secrets bundles and manifests (`surface-secrets/**`) for sealed-mode bootstrap. |
+
+**RU BDU note:** ship the official Russian Trusted Root/Sub CA bundle (`certificates/russian_trusted_bundle.pem`) inside the kit so `concelier:httpClients:source.bdu:trustedRootPaths` can resolve it when the service runs in an air‑gapped network. Drop the most recent `vulxml.zip` alongside the kit if operators need a cold-start cache.
+
+**Language analyzers:** the kit now carries the restart-only Node.js, Go, .NET, Python, Ruby, Rust, and PHP plug-ins (`plugins/scanner/analyzers/lang/StellaOps.Scanner.Analyzers.Lang.Node/`, `...Lang.Go/`, `...Lang.DotNet/`, `...Lang.Python/`, `...Lang.Ruby/`, `...Lang.Rust/`, `...Lang.Php/`). Drop the directories alongside Worker binaries so the unified plug-in catalog can load them without outbound fetches.
+
+**Ruby analyzer features:**
+- **Gemfile/Gemfile.lock** parsing with dependency edges (version constraints, PURLs)
+- **OCI container layer** support (`layers/`, `.layers/`, `layer/`) for VFS/container workspace discovery
+- **Ruby version detection** via `.ruby-version`, `.tool-versions`, Gemfile `ruby` directive, and binary paths
+- **Native extension detection** for `.so`, `.bundle`, `.dll` files in gem paths
+- **Web server config parsing** for Puma, Unicorn, and Passenger configurations
+- **AOC-compliant observations**: entrypoints (script/rack/rackup), dependency edges, runtime edges, jobs, configs, warnings
+- **Optional runtime evidence** via TracePoint; set `STELLA_RUBY_ENTRYPOINT` to enable runtime capture with SHA-256 path hashing for secure evidence correlation
+- **CLI inspection**: run `stella ruby inspect --root /path/to/app` to analyze a Ruby workspace locally
+
+The PHP analyzer parses `composer.lock` for Composer dependencies and supports optional runtime evidence via the `stella-trace.php` shim; set `STELLA_PHP_OPCACHE=1` to enable opcache statistics collection.
+
+**Secret Detection Rules:**
+
+The Offline Kit includes DSSE-signed rule bundles for secret leak detection, enabling fully offline scanning for exposed credentials, API keys, and other sensitive data.
+
+**Bundle Structure:**
+```
+rules/secrets/<version>/
+  secrets.ruleset.manifest.json    # Bundle metadata (version, rule count, signer)
+  secrets.ruleset.rules.jsonl      # Rule definitions (one JSON per line)
+  secrets.ruleset.dsse.json        # DSSE signature envelope
+  SHA256SUMS                       # File checksums
+```
+
+**Manifest Format:**
+```json
+{
+  "bundleId": "secrets.ruleset",
+  "bundleType": "secrets",
+  "version": "2026.01",
+  "ruleCount": 150,
+  "signerKeyId": "stellaops-secrets-signer",
+  "signedAt": "2026-01-04T00:00:00Z",
+  "files": [
+    {
+      "name": "secrets.ruleset.rules.jsonl",
+      "digest": "sha256:...",
+      "sizeBytes": 45678
+    }
+  ]
+}
+```
+
+**Installation:**
+```bash
+# Verify bundle signature using local attestor mirror
+export STELLA_ATTESTOR_URL="file:///mnt/offline-kit/attestor-mirror"
+devops/offline/scripts/install-secrets-bundle.sh \
+  /mnt/offline-kit/rules/secrets/2026.01 \
+  /opt/stellaops/plugins/scanner/analyzers/secrets
+```
+
+**Bundle Rotation:**
+```bash
+# Upgrade to new version with automatic backup
+devops/offline/scripts/rotate-secrets-bundle.sh \
+  /mnt/offline-kit/rules/secrets/2026.02
+```
+
+**Enable Feature:**
+```yaml
+scanner:
+  features:
+    experimental:
+      secret-leak-detection: true
+```
+
+**Verify Bundle is Loaded:**
+```bash
+kubectl logs -l app=scanner-worker --tail=100 | grep SecretsAnalyzerHost
+# Expected: SecretsAnalyzerHost: Loaded bundle 2026.01 with 150 rules
+```
+
+**Python analyzer features:**
+- **Wheel/sdist/editable** parsing with dependency edges from `METADATA`, `PKG-INFO`, `requirements.txt`, and `pyproject.toml`
+- **Virtual environment** support for virtualenv, venv, and conda prefix layouts
+- **PEP 420 namespace packages** with proper `importlib` resolution semantics across `sys.path`
+- **Python version detection** via `pyproject.toml`, `runtime.txt`, Dockerfile `FROM python:*`, `.python-version`
+- **Native extension detection** for `.so`, `.pyd`, CFFI modules, ctypes loaders, and embedded WASM
+- **Framework/config heuristics** for Django, Flask, FastAPI, Celery, AWS Lambda, Gunicorn, Click/Typer CLIs
+- **AOC-compliant observations**: entrypoints (module `__main__`, console_scripts, zipapp), components (modules/packages/native), edges (import, namespace, dynamic-hint, native-extension) with resolver traces
+- **Optional runtime evidence** via import hook; the bundled `stellaops_trace.py` module captures module load events with SHA-256 path hashing for secure evidence correlation
+- **CLI inspection**: run `stella python inspect --root /path/to/app` to analyze a Python workspace locally
+
+**Surface.Env configuration:** Scanner Worker and WebService components use the Surface.Env library for configuration discovery. In air-gapped deployments, configure the following environment variables (see `docs/modules/scanner/design/surface-env.md` for details):
+
+| Variable | Description | Air-gap Default |
+|----------|-------------|-----------------|
+| `SCANNER_SURFACE_FS_ENDPOINT` | Base URI for Surface.FS / RustFS storage | `http://rustfs:8080` |
+| `SCANNER_SURFACE_FS_BUCKET` | Bucket for manifests/artefacts | `surface-cache` |
+| `SCANNER_SURFACE_CACHE_ROOT` | Local cache directory | `/var/lib/stellaops/surface` |
+| `SCANNER_SURFACE_CACHE_QUOTA_MB` | Cache quota in MB (64-262144) | `4096` |
+| `SCANNER_SURFACE_PREFETCH_ENABLED` | Enable manifest prefetch | `false` |
+| `SCANNER_SURFACE_TENANT` | Tenant namespace | `default` |
+| `SCANNER_SURFACE_SECRETS_PROVIDER` | Secrets provider (`file`, `kubernetes`) | `file` |
+| `SCANNER_SURFACE_SECRETS_ROOT` | Root path for file provider | `/etc/stellaops/secrets` |
+| `SCANNER_SURFACE_SECRETS_ALLOW_INLINE` | Allow inline secrets | `false` |
+
+For Helm deployments, configure via `values.yaml`:
+```yaml
+surface:
+  fs:
+    endpoint: "http://rustfs:8080"
+    bucket: "surface-cache"
+  cache:
+    root: "/var/lib/stellaops/surface"
+    quotaMb: 4096
+  tenant: "default"
+  secrets:
+    provider: "file"
+    root: "/etc/stellaops/secrets"
+```
+
+For Docker Compose, these variables are pre-configured in `docker-compose.airgap.yaml` with sensible defaults.
+
+**Advisory AI volume primer:** ship a tarball containing empty `queue/`, `plans/`, and `outputs/` directories plus their ownership metadata. During import, extract it onto the RWX volume used by `advisory-ai-web` and `advisory-ai-worker` so pods start with the expected directory tree even on air-gapped nodes.
+
+*Scanner core:* C# 12 on **.NET {{ dotnet }}**.  
+*Imports are idempotent and atomic — no service downtime.*
+
+## 0 · Prepare the debug store
+
+Before packaging the Offline Kit, mirror the release debug artefacts (GNU build-id `.debug` files and the associated manifest) into the staging directory:
+
+```bash
+./ops/offline-kit/mirror_debug_store.py \
+  --release-dir out/release \
+  --offline-kit-dir out/offline-kit
+```
+
+The helper copies `debug/.build-id/**`, validates `debug/debug-manifest.json` against its recorded SHA-256, and writes `out/offline-kit/metadata/debug-store.json` with a short summary (platforms, artefact counts, sample build-ids). The command exits non-zero if an artefact referenced by the manifest is missing or has the wrong digest, so run it as part of every kit build.
+
+---
+
+## 0.1 · Automated packaging
+
+The packaging workflow is scripted via `ops/offline-kit/build_offline_kit.py`.  
+It verifies the release artefacts, runs the Python analyzer smoke suite, mirrors the debug store, and emits a deterministic tarball + manifest set.
+
+What it picks up automatically (if present under `--release-dir`):
+- `cli/**` → CLI binaries and installers.
+- `containers/**` or `images/**` → air-gap container bundles.
+- `orchestrator/{service,worker-sdk,postgres,dashboards}/**`.
+- `export-center/**`, `notifier/**`, `surface-secrets/**`.
+- Docs: `docs/task-packs/**`, `docs/modules/taskrunner/**`, `docs/airgap/mirror-bundles.md`.
+
+```bash
+python ops/offline-kit/build_offline_kit.py \
+  --version 2025.10.0 \
+  --channel edge \
+  --release-dir out/release \
+  --staging-dir out/offline-kit/staging \
+  --output-dir out/offline-kit/dist
+
+# Optional: regenerate the telemetry collector bundle prior to packaging.
+python ops/devops/telemetry/package_offline_bundle.py --output out/telemetry/telemetry-offline-bundle.tar.gz
+```
+
+Outputs:
+
+- `stella-ops-offline-kit-<version>-<channel>.tar.gz` — bundle (mtime/uid/gid forced to zero for reproducibility)  
+- `stella-ops-offline-kit-<version>-<channel>.tar.gz.sha256` — bundle digest  
+- `manifest/offline-manifest.json` + `.sha256` — inventories every file in the bundle  
+- `<bundle>.metadata.json` — descriptor consumed by the CLI/Console import tooling; includes `counts` for `cli`, `taskPacksDocs`, `containers`, `orchestrator`, `exportCenter`, `notifier`, `surfaceSecrets` so operators can sanity-check bundle composition without unpacking
+- `telemetry/telemetry-offline-bundle.tar.gz` + `.sha256` — packaged OTLP collector assets for environments without upstream access
+- `plugins/scanner/analyzers/lang/StellaOps.Scanner.Analyzers.Lang.Python/*.sig` (+ `.sha256`) — Cosign signatures for the Python analyzer DLL and manifest
+
+### Policy Gateway configuration bundle
+
+- Copy `etc/policy-gateway.yaml` (or the `*.sample` template if you expect operators to override values) into `config/policy-gateway/policy-gateway.yaml` within the staging tree.
+- Include the gateway DPoP private key under `secrets/policy-gateway/policy-gateway-dpop.pem` and reference the location inside the manifest notes. Set the permissions explicitly (`chmod 600 secrets/policy-gateway/policy-gateway-dpop.pem`) so only the kit importer can read it; the importer will refuse keys that are broader.
+- Document the gateway base URL and activation verification steps in `docs/policy/gateway.md` (bundled alongside the kit). Operators can use those curl snippets to smoke-test pack CRUD once the Offline Kit is imported.
+- Ensure the Prometheus snapshot captured during packaging contains `policy_gateway_activation_requests_total` so auditors can reconcile activation attempts performed via the gateway during the validation window.
+
+Provide `--cosign-key` / `--cosign-identity-token` (and optional `--cosign-password`) to generate Cosign signatures for both the tarball and manifest.
+
+---
+
+## 1 · Download & verify
+
+```bash
+curl -LO https://get.stella-ops.org/ouk/stella-ops-offline-kit-<DATE>.tgz
+curl -LO https://get.stella-ops.org/ouk/stella-ops-offline-kit-<DATE>.tgz.sig
+curl -LO https://get.stella-ops.org/ouk/offline-manifest-<DATE>.json
+curl -LO https://get.stella-ops.org/ouk/offline-manifest-<DATE>.json.jws
+
+cosign verify-blob \
+  --key https://stella-ops.org/keys/cosign.pub \
+  --signature stella-ops-offline-kit-<DATE>.tgz.sig \
+  stella-ops-offline-kit-<DATE>.tgz
+````
+
+**CLI shortcut.** `stellaops-cli offline kit pull --destination ./offline-kit` downloads the bundle, manifest, and detached signatures in one step, resumes partial transfers, and writes a `.metadata.json` summary for later import.
+
+Verification prints **OK** and the SHA‑256 digest; cross‑check against the
+[changelog](https://git.stella-ops.org/stella-ops/offline-kit/-/releases).
+
+Validate the attested manifest before distribution:
+
+```bash
+cosign verify-blob \
+  --key https://stella-ops.org/keys/cosign.pub \
+  --signature offline-manifest-<DATE>.json.jws \
+  offline-manifest-<DATE>.json
+
+jq '.artifacts[] | {name, sha256, size, capturedAt}' offline-manifest-<DATE>.json
+```
+
+The manifest enumerates every artefact (`name`, `sha256`, `size`, `capturedAt`) and is signed with the same key registry as Authority revocation bundles. Operators can ship the manifest alongside the tarball so downstream mirrors can re-verify without unpacking the kit.
+
+Example excerpt (2025-10-23 kit) showing the Go and .NET analyzer plug-in payloads:
+
+```json
+{
+  "name": "plugins/scanner/analyzers/lang/StellaOps.Scanner.Analyzers.Lang.Go/StellaOps.Scanner.Analyzers.Lang.Go.dll",
+  "sha256": "a6dc850fc51151c8967ef46a3c4730f08b549667e041079431f39a8a72d0b641",
+  "size": 33792,
+  "capturedAt": "2025-10-23T00:00:00Z"
+}
+{
+  "name": "plugins/scanner/analyzers/lang/StellaOps.Scanner.Analyzers.Lang.Go/StellaOps.Scanner.Analyzers.Lang.Go.pdb",
+  "sha256": "6cbdabf155282f458b89edf267e7f6bb2441a93029aad7aad45c8a9ec58b1b3b",
+  "size": 32152,
+  "capturedAt": "2025-10-23T00:00:00Z"
+}
+{
+  "name": "plugins/scanner/analyzers/lang/StellaOps.Scanner.Analyzers.Lang.Go/manifest.json",
+  "sha256": "c19bfca2fcbb7cb18f1082b5d0d5a8f15fc799c648b50e95fce8d8b109ce48c9",
+  "size": 622,
+  "capturedAt": "2025-10-23T00:00:00Z"
+}
+{
+  "name": "plugins/scanner/analyzers/lang/StellaOps.Scanner.Analyzers.Lang.DotNet/StellaOps.Scanner.Analyzers.Lang.DotNet.dll",
+  "sha256": "0734d23e33277ce2ccb596782d2d42cfe394b3d372dc34da9cb28b59df9b9d22",
+  "size": 70144,
+  "capturedAt": "2025-10-23T00:00:00Z"
+}
+{
+  "name": "plugins/scanner/analyzers/lang/StellaOps.Scanner.Analyzers.Lang.DotNet/StellaOps.Scanner.Analyzers.Lang.DotNet.pdb",
+  "sha256": "b853c1ff4b196715f5bd1447e1a13edeb4940917527ec9bf153b5048da49abaf",
+  "size": 40400,
+  "capturedAt": "2025-10-23T00:00:00Z"
+}
+{
+  "name": "plugins/scanner/analyzers/lang/StellaOps.Scanner.Analyzers.Lang.DotNet/manifest.json",
+  "sha256": "5d483885f825f01bfd9943dcf2889ec2e0beba38ede92ecfe67d4f506cf14e37",
+  "size": 647,
+  "capturedAt": "2025-10-23T00:00:00Z"
+}
+{
+  "name": "plugins/scanner/analyzers/lang/StellaOps.Scanner.Analyzers.Lang.Python/StellaOps.Scanner.Analyzers.Lang.Python.dll",
+  "sha256": "a4f558f363394096e3dd6263f35b180b93b4112f9cf616c05872da8a8657d518",
+  "size": 47104,
+  "capturedAt": "2025-10-26T00:00:00Z"
+}
+{
+  "name": "plugins/scanner/analyzers/lang/StellaOps.Scanner.Analyzers.Lang.Python/StellaOps.Scanner.Analyzers.Lang.Python.pdb",
+  "sha256": "ef2ad78bc2cd1d7e99bae000b92357aa9a9c32938501899e9033d001096196d0",
+  "size": 31896,
+  "capturedAt": "2025-10-26T00:00:00Z"
+}
+{
+  "name": "plugins/scanner/analyzers/lang/StellaOps.Scanner.Analyzers.Lang.Python/manifest.json",
+  "sha256": "668ad9a1a35485628677b639db4d996d1e25f62021680a81a22482483800e557",
+  "size": 648,
+  "capturedAt": "2025-10-26T00:00:00Z"
+}
+{
+  "name": "plugins/scanner/analyzers/lang/StellaOps.Scanner.Analyzers.Lang.Ruby/StellaOps.Scanner.Analyzers.Lang.Ruby.dll",
+  "sha256": "<computed-at-release>",
+  "size": 0,
+  "capturedAt": "2025-11-27T00:00:00Z"
+}
+{
+  "name": "plugins/scanner/analyzers/lang/StellaOps.Scanner.Analyzers.Lang.Ruby/StellaOps.Scanner.Analyzers.Lang.Ruby.pdb",
+  "sha256": "<computed-at-release>",
+  "size": 0,
+  "capturedAt": "2025-11-27T00:00:00Z"
+}
+{
+  "name": "plugins/scanner/analyzers/lang/StellaOps.Scanner.Analyzers.Lang.Ruby/manifest.json",
+  "sha256": "<computed-at-release>",
+  "size": 0,
+  "capturedAt": "2025-11-27T00:00:00Z"
+}
+{
+  "name": "plugins/scanner/analyzers/lang/StellaOps.Scanner.Analyzers.Lang.Rust/StellaOps.Scanner.Analyzers.Lang.Rust.dll",
+  "sha256": "d90ba8b6ace7d98db563b1dec178d57ac09df474e1342fa1daa38bd55e17b185",
+  "size": 54784,
+  "capturedAt": "2025-11-01T00:00:00Z"
+}
+{
+  "name": "plugins/scanner/analyzers/lang/StellaOps.Scanner.Analyzers.Lang.Rust/StellaOps.Scanner.Analyzers.Lang.Rust.pdb",
+  "sha256": "6fac88640a4980d2bb8f7ea2dd2f3d0a521b90fd30ae3a84981575d5f76fa3df",
+  "size": 36636,
+  "capturedAt": "2025-11-01T00:00:00Z"
+}
+{
+  "name": "plugins/scanner/analyzers/lang/StellaOps.Scanner.Analyzers.Lang.Rust/manifest.json",
+  "sha256": "1ec47d1a2103ad5eff23e903532cb76b1ed7ded85d301c1a6631ff21aa966ed4",
+  "size": 658,
+  "capturedAt": "2025-11-01T00:00:00Z"
+}
+```
+
+---
+
+## 2 · Import on the air‑gapped host
+
+```bash
+docker compose --env-file .env \
+  -f docker-compose.stella-ops.yml \
+  exec stella-ops \
+  stella admin import-offline-usage-kit stella-ops-offline-kit-<DATE>.tgz
+```
+
+Alternatively, run
+
+```bash
+stellaops-cli offline kit import stella-ops-offline-kit-<DATE>.tgz \
+  --manifest offline-manifest-<DATE>.json \
+  --bundle-signature stella-ops-offline-kit-<DATE>.tgz.sig \
+  --manifest-signature offline-manifest-<DATE>.json.jws
+```
+
+The CLI validates recorded digests (when `.metadata.json` is present) before streaming the multipart payload to `/api/offline-kit/import`.
+
+* The CLI validates the Cosign signature **before** activation.
+* Old feeds are kept until the new bundle is fully verified.
+* Import time on a SATA SSD: ≈ 25 s for a 300 MB kit.
+
+### 2.1 Validator + idempotency enablement (air-gap)
+
+The Offline Kit carries the same helper scripts under `scripts/`:
+
+1. **Duplicate audit:** run
+   ```bash
+   psql -d concelier -f ops/devops/scripts/check-advisory-raw-duplicates.sql -v LIMIT=200
+   ```
+   to verify no `(vendor, upstream_id, content_hash, tenant)` conflicts remain before enabling the idempotency index.
+2. **Apply validators:** execute `psql -d concelier -f ops/devops/scripts/apply-aoc-validators.sql` (and the Excititor equivalent) with `validationLevel: "moderate"` in maintenance mode.
+3. **Restart Concelier** so migrations `20251028_advisory_raw_idempotency_index` and `20251028_advisory_supersedes_backfill` run automatically. After the restart:
+   - Confirm `db.advisory` resolves to a view on `advisory_backup_20251028`.
+   - Spot-check a few `advisory_raw` entries to ensure `supersedes` chains are populated deterministically.
+4. **Smoke test:** run `stella sources ingest --dry-run --fixture advisory` (bundled fixtures) to confirm ingestion succeeds post-guard and the CLI reports zero violations.
+
+### Authority scope sanity check
+
+Offline installs rely on the bundled `etc/authority.yaml.sample`. Before promoting the kit, confirm the sample clients keep the Aggregation-Only guardrails:
+
+- `aoc-verifier` requests `aoc:verify`, `advisory:read`, and `vex:read`.
+- `signals-uploader` requests `signals:write`, `signals:read`, and `aoc:verify`.
+- `airgap-operator` requests `airgap:status:read`, `airgap:import`, and `airgap:seal`.
+- `task-runner` requests `packs.run` and `packs.read` for execution flows.
+- `pack-approver` requests `packs.approve` (plus `packs.read`) for automation that resumes runs after approvals.
+- `packs-registry` requests `packs.write` and `packs.read` for publishing bundles.
+
+Authority now rejects tokens that request `advisory:read`, `vex:read`, or any `signals:*` scope without `aoc:verify`; the sample has been updated to match. Air-gap scopes (`airgap:*`) also require an explicit tenant assignment—match the updated roles (`airgap-viewer`, `airgap-operator`, `airgap-admin`) so automation fails closed when misconfigured.
+
+**Quick smoke test:** before import, verify the tarball carries the Go analyzer plug-in:
+
+```bash
+tar -tzf stella-ops-offline-kit-<DATE>.tgz 'plugins/scanner/analyzers/lang/StellaOps.Scanner.Analyzers.Lang.Go/*' 'plugins/scanner/analyzers/lang/StellaOps.Scanner.Analyzers.Lang.DotNet/*' 'plugins/scanner/analyzers/lang/StellaOps.Scanner.Analyzers.Lang.Python/*' 'plugins/scanner/analyzers/lang/StellaOps.Scanner.Analyzers.Lang.Ruby/*' 'plugins/scanner/analyzers/lang/StellaOps.Scanner.Analyzers.Lang.Php/*'
+```
+
+The manifest lookup above and this `tar` listing should both surface the Go analyzer DLL, PDB, and manifest entries before the kit is promoted.
+
+> **Release guardrail.** The automated release pipeline now publishes the Python, Ruby, Rust, and PHP plug-ins from source and executes `dotnet run --project src/Tools/LanguageAnalyzerSmoke --configuration Release -- --repo-root <checkout> --analyzer <id>` to validate manifest integrity and cold/warm determinism within the < 30 s / < 5 s budgets (differences versus repository goldens are logged for triage). Run `ops/offline-kit/run-python-analyzer-smoke.sh`, `ops/offline-kit/run-ruby-analyzer-smoke.sh`, `ops/offline-kit/run-rust-analyzer-smoke.sh`, and `ops/offline-kit/run-php-analyzer-smoke.sh` locally before shipping a refreshed kit if you rebuild artefacts outside CI or when preparing the air-gap bundle.
+
+### Debug store mirror
+
+Offline symbols (`debug/.build-id/**`) must accompany every Offline Kit to keep symbol lookup deterministic. The release workflow is expected to emit `out/release/debug/` containing the build-id tree plus `debug-manifest.json` and its `.sha256` companion. After a release completes:
+
+```bash
+python ops/offline-kit/mirror_debug_store.py \
+  --release-dir out/release \
+  --offline-dir out/offline-kit \
+  --summary out/offline-kit/metadata/debug-store.json
+```
+
+The script mirrors the debug tree into the Offline Kit staging directory, verifies SHA-256 values against the manifest, and writes a summary under `metadata/debug-store.json` for audit logs. If the release pipeline does not populate `out/release/debug`, the tooling now logs a warning (`DEVOPS-REL-17-004`)—treat it as a build failure and re-run the release once symbol extraction is enabled.
+
+---
+## 2.2 · Reachability & Proof Bundle Extensions
+
+The Offline Kit supports deterministic replay and reachability analysis in air-gapped environments through additional bundle types.
+
+### Reachability Bundle Format
+
+```
+/offline/reachability/<scan-id>/
+  ├── callgraph.json.zst         # Compressed call-graph (cg_node + cg_edge)
+  ├── manifest.json               # Scan manifest with frozen feed hashes
+  ├── manifest.dsse.json          # DSSE signature envelope
+  ├── entrypoints.json            # Discovered entry points
+  └── proofs/
+      ├── score_proof.cbor        # Canonical CBOR proof ledger
+      ├── score_proof.dsse.json   # DSSE signature for proof
+      └── reachability.json       # Reachability verdicts per finding
+```
+
+**Bundle contents:**
+
+| File | Purpose | Format |
+|------|---------|--------|
+| `callgraph.json.zst` | Static call-graph extracted from artifact | Zstd-compressed JSON |
+| `manifest.json` | Scan parameters + frozen Concelier/Excititor snapshot hashes | JSON |
+| `manifest.dsse.json` | DSSE envelope signing the manifest | JSON (in-toto DSSE) |
+| `entrypoints.json` | Discovered entry points (controllers, handlers, etc.) | JSON array |
+| `proofs/score_proof.cbor` | Deterministic proof ledger with Merkle root | CBOR (RFC 8949) |
+| `proofs/score_proof.dsse.json` | DSSE signature attesting to proof integrity | JSON (in-toto DSSE) |
+| `proofs/reachability.json` | Reachability status per CVE/finding | JSON |
+
+### Ground-Truth Corpus Bundle
+
+For validation and regression testing of reachability analysis:
+
+```
+/offline/corpus/ground-truth-v1.tar.zst
+  ├── corpus-manifest.json        # Corpus metadata and sample count
+  ├── dotnet/                     # .NET test cases (10 samples)
+  │   ├── sample-001/
+  │   │   ├── artifact.tar.gz     # Source/binary artifact
+  │   │   ├── expected.json       # Ground-truth reachability verdicts
+  │   │   └── callgraph.json      # Expected call-graph
+  │   └── ...
+  └── java/                       # Java test cases (10 samples)
+      ├── sample-001/
+      └── ...
+```
+
+**Corpus validation:**
+```bash
+stella scan validate-corpus --corpus /offline/corpus/ground-truth-v1.tar.zst
+```
+
+Expected output:
+- Precision ≥ 80% on all samples
+- Recall ≥ 80% on all samples  
+- 100% bit-identical replay when re-running with same manifest
+
+### Proof Replay in Air-Gap Mode
+
+To replay a scan with frozen feeds:
+
+```bash
+# Import the reachability bundle
+stella admin import-reachability-bundle /offline/reachability/<scan-id>/
+
+# Replay the score calculation
+stella score replay --scan <scan-id> --verify-proof
+
+# Expected: "Proof root hash matches: <hash>"
+```
+
+The replay command:
+1. Loads the frozen Concelier/Excititor snapshots from the manifest
+2. Re-executes scoring with the same inputs
+3. Computes a new proof root hash
+4. Verifies it matches the original (bit-identical determinism)
+
+### CLI Commands for Reachability
+
+```bash
+# Extract call-graph from artifact
+stella scan graph --lang dotnet --sln /path/to/solution.sln --output callgraph.json
+
+# Run reachability analysis
+stella scan reachability --callgraph callgraph.json --sbom sbom.json --output reachability.json
+
+# Package for offline transfer
+stella scan export-bundle --scan <scan-id> --output /offline/reachability/<scan-id>/
+```
+
+---
+## 2.3 · Provcache Air-Gap Integration
+
+The Provenance Cache (Provcache) supports air-gapped environments through minimal proof bundles with lazy evidence fetching.
+
+### Proof Bundle Density Levels
+
+| Density | Contents | Typical Size | Air-Gap Usage |
+|---------|----------|--------------|---------------|
+| **Lite** | DecisionDigest + ProofRoot + Manifest | ~2 KB | Requires lazy fetch for evidence |
+| **Standard** | + First ~10% of evidence chunks | ~200 KB | Partial evidence, lazy fetch remaining |
+| **Strict** | + All evidence chunks | Variable | Full compliance, no network needed |
+
+### Export Workflow
+
+```bash
+# Export lite bundle for minimal transfer size
+stella prov export --verikey sha256:<key> --density lite --output proof-lite.json
+
+# Export standard bundle (balanced)
+stella prov export --verikey sha256:<key> --density standard --output proof-std.json
+
+# Export strict bundle with full evidence + signature
+stella prov export --verikey sha256:<key> --density strict --sign --output proof-full.json
+```
+
+### Evidence Chunk Export for Sneakernet
+
+For fully air-gapped environments using lite/standard bundles:
+
+```bash
+# Export all evidence chunks to directory for transport
+stella prov export-chunks --proof-root sha256:<root> --output /mnt/usb/evidence/
+
+# Output structure:
+/mnt/usb/evidence/
+  ├── sha256-<proof_root>/
+  │   ├── manifest.json
+  │   ├── 00000000.chunk
+  │   ├── 00000001.chunk
+  │   └── ...
+```
+
+### Import Workflow on Air-Gapped Host
+
+```bash
+# Import with lazy fetch from file directory (sneakernet)
+stella prov import proof-lite.json --lazy-fetch --chunks-dir /mnt/usb/evidence/
+
+# Import with lazy fetch from local server (isolated network)
+stella prov import proof-lite.json --lazy-fetch --backend http://provcache-server:8080
+
+# Import strict bundle (no network needed)
+stella prov import proof-full.json --verify
+```
+
+### Programmatic Lazy Fetch
+
+```csharp
+// File-based fetcher for air-gapped environments
+var fileFetcher = new FileChunkFetcher(
+    basePath: "/mnt/usb/evidence",
+    logger);
+
+var orchestrator = new LazyFetchOrchestrator(repository, logger);
+
+// Fetch and verify all missing chunks
+var result = await orchestrator.FetchAndStoreAsync(
+    proofRoot: "sha256:...",
+    fileFetcher,
+    new LazyFetchOptions
+    {
+        VerifyOnFetch = true,
+        BatchSize = 100
+    });
+
+if (result.Success)
+    Console.WriteLine($"Fetched {result.ChunksStored} chunks");
+```
+
+### Bundle Format (v1)
+
+```json
+{
+    "version": "v1",
+    "exportedAt": "2025-01-15T10:30:00Z",
+    "density": "standard",
+    "digest": {
+        "veriKey": "sha256:...",
+        "verdictHash": "sha256:...",
+        "proofRoot": "sha256:...",
+        "trustScore": 85
+    },
+    "manifest": {
+        "proofRoot": "sha256:...",
+        "totalChunks": 42,
+        "totalSize": 2752512,
+        "chunks": [...]
+    },
+    "chunks": [...],
+    "signature": {
+        "algorithm": "ECDSA-P256",
+        "signature": "base64...",
+        "signedAt": "2025-01-15T10:30:01Z"
+    }
+}
+```
+
+### Related Documentation
+
+- [Provcache Architecture](modules/provcache/architecture.md) — Detailed architecture and API reference
+- [Provcache README](modules/provcache/README.md) — Configuration and usage guide
+
+---## 3 · Delta patch workflow
+
+1. **Connected site** fetches `stella-ouk-YYYY‑MM‑DD.delta.tgz`.
+2. Transfer via any medium (USB, portable disk).
+3. `stella admin import-offline-usage-kit <delta>` applies only changed CVE rows & images.
+
+Daily deltas are **< 30 MB**; weekly roll‑up produces a fresh full kit.
+
+---
+
+## 4 · Quota behaviour offline
+
+The scanner enforces the same fair‑use limits offline:
+
+* **Anonymous:** {{ quota\_anon }} scans per UTC day
+* **Free JWT:** {{ quota\_token }} scans per UTC day
+
+Soft reminder at 200 scans; throttle above the ceiling but **never block**.
+See the quota enforcement flow in
+[`30_QUOTA_ENFORCEMENT_FLOW1.md`](30_QUOTA_ENFORCEMENT_FLOW1.md).
+
+---
+
+## 5 · Troubleshooting
+
+| Symptom                                | Explanation                              | Fix                                   |
+| -------------------------------------- | ---------------------------------------- | ------------------------------------- |
+| `could not verify SBOM hash`           | Bundle corrupted in transit              | Re‑download / re‑copy                 |
+| Import hangs at `Applying feeds…`      | Low disk space in `/var/lib/stella`      | Free ≥ 2 GiB before retry             |
+| `quota exceeded` same day after import | Import resets counters at UTC 00:00 only | Wait until next UTC day or load a JWT |
+
+---
+
+## 6 · Related documentation
+
+* **Install guide:** `/install/#air-gapped`
+* **Sovereign mode rationale:** `/sovereign/`
+* **Security policy:** `/security/#reporting-a-vulnerability`
+* **CERT-Bund snapshots:** `python src/Tools/certbund_offline_snapshot.py --help` (see `docs/modules/concelier/operations/connectors/certbund.md`)
+* **PostgreSQL operations:** `docs/operations/postgresql-guide.md` - performance tuning, monitoring, backup/restore, and scaling
+* **Database specification:** `docs/db/SPECIFICATION.md` - schema design, migration patterns, and module ownership