master
96d52884e8
Some checks failed
Docs CI / lint-and-preview (push) Has been cancelled
- Implemented PolicyDslValidator with command-line options for strict mode and JSON output. - Created PolicySchemaExporter to generate JSON schemas for policy-related models. - Developed PolicySimulationSmoke tool to validate policy simulations against expected outcomes. - Added project files and necessary dependencies for each tool. - Ensured proper error handling and usage instructions across tools.
256 lines
12 KiB
Markdown
Executable File
256 lines
12 KiB
Markdown
Executable File
# Offline Update Kit (OUK) — Air‑Gap Bundle
|
||
|
||
<!--
|
||
Build‑time variable injection:
|
||
{{ quota_anon }} = 33
|
||
{{ quota_token }} = 333
|
||
{{ dotnet }} = "10 LTS"
|
||
-->
|
||
|
||
The **Offline Update Kit** packages everything Stella Ops needs to run on a
|
||
completely isolated network:
|
||
|
||
| Component | Contents |
|
||
|-----------|----------|
|
||
| **Merged vulnerability feeds** | OSV, GHSA plus optional NVD 2.0, CNNVD, CNVD, ENISA, JVN and BDU |
|
||
| **Container images** | `stella-ops`, *Zastava* sidecar (x86‑64 & arm64) |
|
||
| **Provenance** | Cosign signature, SPDX 2.3 SBOM, in‑toto SLSA attestation |
|
||
| **Attested manifest** | `offline-manifest.json` + detached JWS covering bundle metadata, signed during export. |
|
||
| **Delta patches** | Daily diff bundles keep size \< 350 MB |
|
||
| **Scanner plug-ins** | OS analyzers plus the Node.js, Go, .NET, and Python language analyzers packaged under `plugins/scanner/analyzers/**` with manifests so Workers load deterministically offline. |
|
||
| **Debug store** | `.debug` artefacts laid out under `debug/.build-id/<aa>/<rest>.debug` with `debug/debug-manifest.json` mapping build-ids to originating images for symbol retrieval. |
|
||
| **Telemetry collector bundle** | `telemetry/telemetry-offline-bundle.tar.gz` plus `.sha256`, containing OTLP collector config, Helm/Compose overlays, and operator instructions. |
|
||
|
||
**RU BDU note:** ship the official Russian Trusted Root/Sub CA bundle (`certificates/russian_trusted_bundle.pem`) inside the kit so `concelier:httpClients:source.bdu:trustedRootPaths` can resolve it when the service runs in an air‑gapped network. Drop the most recent `vulxml.zip` alongside the kit if operators need a cold-start cache.
|
||
|
||
**Language analyzers:** the kit now carries the restart-only Node.js, Go, .NET, and Python analyzer plug-ins (`plugins/scanner/analyzers/lang/StellaOps.Scanner.Analyzers.Lang.Node/`, `...Lang.Go/`, `...Lang.DotNet/`, `...Lang.Python/`). Drop the directories alongside Worker binaries so the unified plug-in catalog can load them without outbound fetches; Rust remains on the Wave 4 roadmap.
|
||
|
||
*Scanner core:* C# 12 on **.NET {{ dotnet }}**.
|
||
*Imports are idempotent and atomic — no service downtime.*
|
||
|
||
## 0 · Prepare the debug store
|
||
|
||
Before packaging the Offline Kit, mirror the release debug artefacts (GNU build-id `.debug` files and the associated manifest) into the staging directory:
|
||
|
||
```bash
|
||
./ops/offline-kit/mirror_debug_store.py \
|
||
--release-dir out/release \
|
||
--offline-kit-dir out/offline-kit
|
||
```
|
||
|
||
The helper copies `debug/.build-id/**`, validates `debug/debug-manifest.json` against its recorded SHA-256, and writes `out/offline-kit/metadata/debug-store.json` with a short summary (platforms, artefact counts, sample build-ids). The command exits non-zero if an artefact referenced by the manifest is missing or has the wrong digest, so run it as part of every kit build.
|
||
|
||
---
|
||
|
||
## 0.1 · Automated packaging
|
||
|
||
The packaging workflow is scripted via `ops/offline-kit/build_offline_kit.py`.
|
||
It verifies the release artefacts, runs the Python analyzer smoke suite, mirrors the debug store, and emits a deterministic tarball + manifest set.
|
||
|
||
```bash
|
||
python ops/offline-kit/build_offline_kit.py \
|
||
--version 2025.10.0 \
|
||
--channel edge \
|
||
--release-dir out/release \
|
||
--staging-dir out/offline-kit/staging \
|
||
--output-dir out/offline-kit/dist
|
||
|
||
# Optional: regenerate the telemetry collector bundle prior to packaging.
|
||
python ops/devops/telemetry/package_offline_bundle.py --output out/telemetry/telemetry-offline-bundle.tar.gz
|
||
```
|
||
|
||
Outputs:
|
||
|
||
- `stella-ops-offline-kit-<version>-<channel>.tar.gz` — bundle (mtime/uid/gid forced to zero for reproducibility)
|
||
- `stella-ops-offline-kit-<version>-<channel>.tar.gz.sha256` — bundle digest
|
||
- `manifest/offline-manifest.json` + `.sha256` — inventories every file in the bundle
|
||
- `<bundle>.metadata.json` — descriptor consumed by the CLI/Console import tooling
|
||
- `telemetry/telemetry-offline-bundle.tar.gz` + `.sha256` — packaged OTLP collector assets for environments without upstream access
|
||
- `plugins/scanner/analyzers/lang/StellaOps.Scanner.Analyzers.Lang.Python/*.sig` (+ `.sha256`) — Cosign signatures for the Python analyzer DLL and manifest
|
||
|
||
Provide `--cosign-key` / `--cosign-identity-token` (and optional `--cosign-password`) to generate Cosign signatures for both the tarball and manifest.
|
||
|
||
---
|
||
|
||
## 1 · Download & verify
|
||
|
||
```bash
|
||
curl -LO https://get.stella-ops.org/ouk/stella-ops-offline-kit-<DATE>.tgz
|
||
curl -LO https://get.stella-ops.org/ouk/stella-ops-offline-kit-<DATE>.tgz.sig
|
||
curl -LO https://get.stella-ops.org/ouk/offline-manifest-<DATE>.json
|
||
curl -LO https://get.stella-ops.org/ouk/offline-manifest-<DATE>.json.jws
|
||
|
||
cosign verify-blob \
|
||
--key https://stella-ops.org/keys/cosign.pub \
|
||
--signature stella-ops-offline-kit-<DATE>.tgz.sig \
|
||
stella-ops-offline-kit-<DATE>.tgz
|
||
````
|
||
|
||
**CLI shortcut.** `stellaops-cli offline kit pull --destination ./offline-kit` downloads the bundle, manifest, and detached signatures in one step, resumes partial transfers, and writes a `.metadata.json` summary for later import.
|
||
|
||
Verification prints **OK** and the SHA‑256 digest; cross‑check against the
|
||
[changelog](https://git.stella-ops.org/stella-ops/offline-kit/-/releases).
|
||
|
||
Validate the attested manifest before distribution:
|
||
|
||
```bash
|
||
cosign verify-blob \
|
||
--key https://stella-ops.org/keys/cosign.pub \
|
||
--signature offline-manifest-<DATE>.json.jws \
|
||
offline-manifest-<DATE>.json
|
||
|
||
jq '.artifacts[] | {name, sha256, size, capturedAt}' offline-manifest-<DATE>.json
|
||
```
|
||
|
||
The manifest enumerates every artefact (`name`, `sha256`, `size`, `capturedAt`) and is signed with the same key registry as Authority revocation bundles. Operators can ship the manifest alongside the tarball so downstream mirrors can re-verify without unpacking the kit.
|
||
|
||
Example excerpt (2025-10-23 kit) showing the Go and .NET analyzer plug-in payloads:
|
||
|
||
```json
|
||
{
|
||
"name": "plugins/scanner/analyzers/lang/StellaOps.Scanner.Analyzers.Lang.Go/StellaOps.Scanner.Analyzers.Lang.Go.dll",
|
||
"sha256": "a6dc850fc51151c8967ef46a3c4730f08b549667e041079431f39a8a72d0b641",
|
||
"size": 33792,
|
||
"capturedAt": "2025-10-23T00:00:00Z"
|
||
}
|
||
{
|
||
"name": "plugins/scanner/analyzers/lang/StellaOps.Scanner.Analyzers.Lang.Go/StellaOps.Scanner.Analyzers.Lang.Go.pdb",
|
||
"sha256": "6cbdabf155282f458b89edf267e7f6bb2441a93029aad7aad45c8a9ec58b1b3b",
|
||
"size": 32152,
|
||
"capturedAt": "2025-10-23T00:00:00Z"
|
||
}
|
||
{
|
||
"name": "plugins/scanner/analyzers/lang/StellaOps.Scanner.Analyzers.Lang.Go/manifest.json",
|
||
"sha256": "c19bfca2fcbb7cb18f1082b5d0d5a8f15fc799c648b50e95fce8d8b109ce48c9",
|
||
"size": 622,
|
||
"capturedAt": "2025-10-23T00:00:00Z"
|
||
}
|
||
{
|
||
"name": "plugins/scanner/analyzers/lang/StellaOps.Scanner.Analyzers.Lang.DotNet/StellaOps.Scanner.Analyzers.Lang.DotNet.dll",
|
||
"sha256": "0734d23e33277ce2ccb596782d2d42cfe394b3d372dc34da9cb28b59df9b9d22",
|
||
"size": 70144,
|
||
"capturedAt": "2025-10-23T00:00:00Z"
|
||
}
|
||
{
|
||
"name": "plugins/scanner/analyzers/lang/StellaOps.Scanner.Analyzers.Lang.DotNet/StellaOps.Scanner.Analyzers.Lang.DotNet.pdb",
|
||
"sha256": "b853c1ff4b196715f5bd1447e1a13edeb4940917527ec9bf153b5048da49abaf",
|
||
"size": 40400,
|
||
"capturedAt": "2025-10-23T00:00:00Z"
|
||
}
|
||
{
|
||
"name": "plugins/scanner/analyzers/lang/StellaOps.Scanner.Analyzers.Lang.DotNet/manifest.json",
|
||
"sha256": "5d483885f825f01bfd9943dcf2889ec2e0beba38ede92ecfe67d4f506cf14e37",
|
||
"size": 647,
|
||
"capturedAt": "2025-10-23T00:00:00Z"
|
||
}
|
||
{
|
||
"name": "plugins/scanner/analyzers/lang/StellaOps.Scanner.Analyzers.Lang.Python/StellaOps.Scanner.Analyzers.Lang.Python.dll",
|
||
"sha256": "a4f558f363394096e3dd6263f35b180b93b4112f9cf616c05872da8a8657d518",
|
||
"size": 47104,
|
||
"capturedAt": "2025-10-26T00:00:00Z"
|
||
}
|
||
{
|
||
"name": "plugins/scanner/analyzers/lang/StellaOps.Scanner.Analyzers.Lang.Python/StellaOps.Scanner.Analyzers.Lang.Python.pdb",
|
||
"sha256": "ef2ad78bc2cd1d7e99bae000b92357aa9a9c32938501899e9033d001096196d0",
|
||
"size": 31896,
|
||
"capturedAt": "2025-10-26T00:00:00Z"
|
||
}
|
||
{
|
||
"name": "plugins/scanner/analyzers/lang/StellaOps.Scanner.Analyzers.Lang.Python/manifest.json",
|
||
"sha256": "668ad9a1a35485628677b639db4d996d1e25f62021680a81a22482483800e557",
|
||
"size": 648,
|
||
"capturedAt": "2025-10-26T00:00:00Z"
|
||
}
|
||
```
|
||
|
||
---
|
||
|
||
## 2 · Import on the air‑gapped host
|
||
|
||
```bash
|
||
docker compose --env-file .env \
|
||
-f docker-compose.stella-ops.yml \
|
||
exec stella-ops \
|
||
stella admin import-offline-usage-kit stella-ops-offline-kit-<DATE>.tgz
|
||
```
|
||
|
||
Alternatively, run
|
||
|
||
```bash
|
||
stellaops-cli offline kit import stella-ops-offline-kit-<DATE>.tgz \
|
||
--manifest offline-manifest-<DATE>.json \
|
||
--bundle-signature stella-ops-offline-kit-<DATE>.tgz.sig \
|
||
--manifest-signature offline-manifest-<DATE>.json.jws
|
||
```
|
||
|
||
The CLI validates recorded digests (when `.metadata.json` is present) before streaming the multipart payload to `/api/offline-kit/import`.
|
||
|
||
* The CLI validates the Cosign signature **before** activation.
|
||
* Old feeds are kept until the new bundle is fully verified.
|
||
* Import time on a SATA SSD: ≈ 25 s for a 300 MB kit.
|
||
|
||
**Quick smoke test:** before import, verify the tarball carries the Go analyzer plug-in:
|
||
|
||
```bash
|
||
tar -tzf stella-ops-offline-kit-<DATE>.tgz 'plugins/scanner/analyzers/lang/StellaOps.Scanner.Analyzers.Lang.Go/*' 'plugins/scanner/analyzers/lang/StellaOps.Scanner.Analyzers.Lang.DotNet/*' 'plugins/scanner/analyzers/lang/StellaOps.Scanner.Analyzers.Lang.Python/*'
|
||
```
|
||
|
||
The manifest lookup above and this `tar` listing should both surface the Go analyzer DLL, PDB, and manifest entries before the kit is promoted.
|
||
|
||
> **Release guardrail.** The automated release pipeline now publishes the Python plug-in from source and executes `dotnet run --project tools/LanguageAnalyzerSmoke --configuration Release -- --repo-root <checkout>` to validate manifest integrity and cold/warm determinism within the < 30 s / < 5 s budgets (differences versus repository goldens are logged for triage). Run `ops/offline-kit/run-python-analyzer-smoke.sh` locally before shipping a refreshed kit if you rebuild artefacts outside CI or when preparing the air-gap bundle.
|
||
|
||
### Debug store mirror
|
||
|
||
Offline symbols (`debug/.build-id/**`) must accompany every Offline Kit to keep symbol lookup deterministic. The release workflow is expected to emit `out/release/debug/` containing the build-id tree plus `debug-manifest.json` and its `.sha256` companion. After a release completes:
|
||
|
||
```bash
|
||
python ops/offline-kit/mirror_debug_store.py \
|
||
--release-dir out/release \
|
||
--offline-dir out/offline-kit \
|
||
--summary out/offline-kit/metadata/debug-store.json
|
||
```
|
||
|
||
The script mirrors the debug tree into the Offline Kit staging directory, verifies SHA-256 values against the manifest, and writes a summary under `metadata/debug-store.json` for audit logs. If the release pipeline does not populate `out/release/debug`, the tooling now logs a warning (`DEVOPS-REL-17-004`)—treat it as a build failure and re-run the release once symbol extraction is enabled.
|
||
|
||
---
|
||
|
||
## 3 · Delta patch workflow
|
||
|
||
1. **Connected site** fetches `stella-ouk-YYYY‑MM‑DD.delta.tgz`.
|
||
2. Transfer via any medium (USB, portable disk).
|
||
3. `stella admin import-offline-usage-kit <delta>` applies only changed CVE rows & images.
|
||
|
||
Daily deltas are **< 30 MB**; weekly roll‑up produces a fresh full kit.
|
||
|
||
---
|
||
|
||
## 4 · Quota behaviour offline
|
||
|
||
The scanner enforces the same fair‑use limits offline:
|
||
|
||
* **Anonymous:** {{ quota\_anon }} scans per UTC day
|
||
* **Free JWT:** {{ quota\_token }} scans per UTC day
|
||
|
||
Soft reminder at 200 scans; throttle above the ceiling but **never block**.
|
||
See the detailed rules in
|
||
[`33_333_QUOTA_OVERVIEW.md`](33_333_QUOTA_OVERVIEW.md).
|
||
|
||
---
|
||
|
||
## 5 · Troubleshooting
|
||
|
||
| Symptom | Explanation | Fix |
|
||
| -------------------------------------- | ---------------------------------------- | ------------------------------------- |
|
||
| `could not verify SBOM hash` | Bundle corrupted in transit | Re‑download / re‑copy |
|
||
| Import hangs at `Applying feeds…` | Low disk space in `/var/lib/stella` | Free ≥ 2 GiB before retry |
|
||
| `quota exceeded` same day after import | Import resets counters at UTC 00:00 only | Wait until next UTC day or load a JWT |
|
||
|
||
---
|
||
|
||
## 6 · Related documentation
|
||
|
||
* **Install guide:** `/install/#air-gapped`
|
||
* **Sovereign mode rationale:** `/sovereign/`
|
||
* **Security policy:** `/security/#reporting-a-vulnerability`
|
||
* **CERT-Bund snapshots:** `python tools/certbund_offline_snapshot.py --help` (see `docs/ops/concelier-certbund-operations.md`)
|