Add Policy DSL Validator, Schema Exporter, and Simulation Smoke tools
Some checks failed
Docs CI / lint-and-preview (push) Has been cancelled
Some checks failed
Docs CI / lint-and-preview (push) Has been cancelled
- Implemented PolicyDslValidator with command-line options for strict mode and JSON output. - Created PolicySchemaExporter to generate JSON schemas for policy-related models. - Developed PolicySimulationSmoke tool to validate policy simulations against expected outcomes. - Added project files and necessary dependencies for each tool. - Ensured proper error handling and usage instructions across tools.
This commit is contained in:
@@ -18,6 +18,8 @@ completely isolated network:
|
||||
| **Attested manifest** | `offline-manifest.json` + detached JWS covering bundle metadata, signed during export. |
|
||||
| **Delta patches** | Daily diff bundles keep size \< 350 MB |
|
||||
| **Scanner plug-ins** | OS analyzers plus the Node.js, Go, .NET, and Python language analyzers packaged under `plugins/scanner/analyzers/**` with manifests so Workers load deterministically offline. |
|
||||
| **Debug store** | `.debug` artefacts laid out under `debug/.build-id/<aa>/<rest>.debug` with `debug/debug-manifest.json` mapping build-ids to originating images for symbol retrieval. |
|
||||
| **Telemetry collector bundle** | `telemetry/telemetry-offline-bundle.tar.gz` plus `.sha256`, containing OTLP collector config, Helm/Compose overlays, and operator instructions. |
|
||||
|
||||
**RU BDU note:** ship the official Russian Trusted Root/Sub CA bundle (`certificates/russian_trusted_bundle.pem`) inside the kit so `concelier:httpClients:source.bdu:trustedRootPaths` can resolve it when the service runs in an air‑gapped network. Drop the most recent `vulxml.zip` alongside the kit if operators need a cold-start cache.
|
||||
|
||||
@@ -25,11 +27,53 @@ completely isolated network:
|
||||
|
||||
*Scanner core:* C# 12 on **.NET {{ dotnet }}**.
|
||||
*Imports are idempotent and atomic — no service downtime.*
|
||||
|
||||
---
|
||||
|
||||
## 1 · Download & verify
|
||||
|
||||
|
||||
## 0 · Prepare the debug store
|
||||
|
||||
Before packaging the Offline Kit, mirror the release debug artefacts (GNU build-id `.debug` files and the associated manifest) into the staging directory:
|
||||
|
||||
```bash
|
||||
./ops/offline-kit/mirror_debug_store.py \
|
||||
--release-dir out/release \
|
||||
--offline-kit-dir out/offline-kit
|
||||
```
|
||||
|
||||
The helper copies `debug/.build-id/**`, validates `debug/debug-manifest.json` against its recorded SHA-256, and writes `out/offline-kit/metadata/debug-store.json` with a short summary (platforms, artefact counts, sample build-ids). The command exits non-zero if an artefact referenced by the manifest is missing or has the wrong digest, so run it as part of every kit build.
|
||||
|
||||
---
|
||||
|
||||
## 0.1 · Automated packaging
|
||||
|
||||
The packaging workflow is scripted via `ops/offline-kit/build_offline_kit.py`.
|
||||
It verifies the release artefacts, runs the Python analyzer smoke suite, mirrors the debug store, and emits a deterministic tarball + manifest set.
|
||||
|
||||
```bash
|
||||
python ops/offline-kit/build_offline_kit.py \
|
||||
--version 2025.10.0 \
|
||||
--channel edge \
|
||||
--release-dir out/release \
|
||||
--staging-dir out/offline-kit/staging \
|
||||
--output-dir out/offline-kit/dist
|
||||
|
||||
# Optional: regenerate the telemetry collector bundle prior to packaging.
|
||||
python ops/devops/telemetry/package_offline_bundle.py --output out/telemetry/telemetry-offline-bundle.tar.gz
|
||||
```
|
||||
|
||||
Outputs:
|
||||
|
||||
- `stella-ops-offline-kit-<version>-<channel>.tar.gz` — bundle (mtime/uid/gid forced to zero for reproducibility)
|
||||
- `stella-ops-offline-kit-<version>-<channel>.tar.gz.sha256` — bundle digest
|
||||
- `manifest/offline-manifest.json` + `.sha256` — inventories every file in the bundle
|
||||
- `<bundle>.metadata.json` — descriptor consumed by the CLI/Console import tooling
|
||||
- `telemetry/telemetry-offline-bundle.tar.gz` + `.sha256` — packaged OTLP collector assets for environments without upstream access
|
||||
- `plugins/scanner/analyzers/lang/StellaOps.Scanner.Analyzers.Lang.Python/*.sig` (+ `.sha256`) — Cosign signatures for the Python analyzer DLL and manifest
|
||||
|
||||
Provide `--cosign-key` / `--cosign-identity-token` (and optional `--cosign-password`) to generate Cosign signatures for both the tarball and manifest.
|
||||
|
||||
---
|
||||
|
||||
## 1 · Download & verify
|
||||
|
||||
```bash
|
||||
curl -LO https://get.stella-ops.org/ouk/stella-ops-offline-kit-<DATE>.tgz
|
||||
curl -LO https://get.stella-ops.org/ouk/stella-ops-offline-kit-<DATE>.tgz.sig
|
||||
@@ -101,21 +145,21 @@ Example excerpt (2025-10-23 kit) showing the Go and .NET analyzer plug-in payloa
|
||||
}
|
||||
{
|
||||
"name": "plugins/scanner/analyzers/lang/StellaOps.Scanner.Analyzers.Lang.Python/StellaOps.Scanner.Analyzers.Lang.Python.dll",
|
||||
"sha256": "28b6e06c7cabf3b78f13f801cbb14962093f3d42c4ae9ec01babbcd14cda4644",
|
||||
"size": 53760,
|
||||
"capturedAt": "2025-10-23T00:00:00Z"
|
||||
"sha256": "a4f558f363394096e3dd6263f35b180b93b4112f9cf616c05872da8a8657d518",
|
||||
"size": 47104,
|
||||
"capturedAt": "2025-10-26T00:00:00Z"
|
||||
}
|
||||
{
|
||||
"name": "plugins/scanner/analyzers/lang/StellaOps.Scanner.Analyzers.Lang.Python/StellaOps.Scanner.Analyzers.Lang.Python.pdb",
|
||||
"sha256": "be4e34b4dc9a790fe1299e84213343b7c8ea90a2d22e5d7d1aa7585b8fedc946",
|
||||
"size": 34516,
|
||||
"capturedAt": "2025-10-23T00:00:00Z"
|
||||
"sha256": "ef2ad78bc2cd1d7e99bae000b92357aa9a9c32938501899e9033d001096196d0",
|
||||
"size": 31896,
|
||||
"capturedAt": "2025-10-26T00:00:00Z"
|
||||
}
|
||||
{
|
||||
"name": "plugins/scanner/analyzers/lang/StellaOps.Scanner.Analyzers.Lang.Python/manifest.json",
|
||||
"sha256": "bceea1e7542aae860b0ec5ba7b8b3aa960b21edc4d1efe60afc98ce289341ac3",
|
||||
"size": 671,
|
||||
"capturedAt": "2025-10-23T00:00:00Z"
|
||||
"sha256": "668ad9a1a35485628677b639db4d996d1e25f62021680a81a22482483800e557",
|
||||
"size": 648,
|
||||
"capturedAt": "2025-10-26T00:00:00Z"
|
||||
}
|
||||
```
|
||||
|
||||
@@ -153,6 +197,21 @@ tar -tzf stella-ops-offline-kit-<DATE>.tgz 'plugins/scanner/analyzers/lang/Stell
|
||||
|
||||
The manifest lookup above and this `tar` listing should both surface the Go analyzer DLL, PDB, and manifest entries before the kit is promoted.
|
||||
|
||||
> **Release guardrail.** The automated release pipeline now publishes the Python plug-in from source and executes `dotnet run --project tools/LanguageAnalyzerSmoke --configuration Release -- --repo-root <checkout>` to validate manifest integrity and cold/warm determinism within the < 30 s / < 5 s budgets (differences versus repository goldens are logged for triage). Run `ops/offline-kit/run-python-analyzer-smoke.sh` locally before shipping a refreshed kit if you rebuild artefacts outside CI or when preparing the air-gap bundle.
|
||||
|
||||
### Debug store mirror
|
||||
|
||||
Offline symbols (`debug/.build-id/**`) must accompany every Offline Kit to keep symbol lookup deterministic. The release workflow is expected to emit `out/release/debug/` containing the build-id tree plus `debug-manifest.json` and its `.sha256` companion. After a release completes:
|
||||
|
||||
```bash
|
||||
python ops/offline-kit/mirror_debug_store.py \
|
||||
--release-dir out/release \
|
||||
--offline-dir out/offline-kit \
|
||||
--summary out/offline-kit/metadata/debug-store.json
|
||||
```
|
||||
|
||||
The script mirrors the debug tree into the Offline Kit staging directory, verifies SHA-256 values against the manifest, and writes a summary under `metadata/debug-store.json` for audit logs. If the release pipeline does not populate `out/release/debug`, the tooling now logs a warning (`DEVOPS-REL-17-004`)—treat it as a build failure and re-run the release once symbol extraction is enabled.
|
||||
|
||||
---
|
||||
|
||||
## 3 · Delta patch workflow
|
||||
|
||||
Reference in New Issue
Block a user