DevOps Release Automation
The release workflow builds and signs the StellaOps service containers,
generates SBOM + provenance attestations, and emits a canonical
release.yaml. The logic lives under ops/devops/release/ and is invoked
by the new .gitea/workflows/release.yml pipeline.
Local dry run
./ops/devops/release/build_release.py \
--version 2025.10.0-edge \
--channel edge \
--dry-run
Outputs land under out/release/. Use --no-push to run full builds without
pushing to the registry.
After the build completes, run the verifier to validate recorded hashes and artefact presence:
python ops/devops/release/verify_release.py --release-dir out/release
Python analyzer smoke & signing
dotnet run --project tools/LanguageAnalyzerSmoke exercises the Python language
analyzer plug-in against the golden fixtures (cold/warm timings, determinism). The
release workflow runs this harness automatically and then produces Cosign
signatures + SHA-256 sidecars for StellaOps.Scanner.Analyzers.Lang.Python.dll
and its manifest.json. Keep COSIGN_KEY_REF/COSIGN_IDENTITY_TOKEN populated so
the step can sign the artefacts; the generated .sig/.sha256 files ship with the
Offline Kit bundle.
Required tooling
- Docker 25+ with Buildx
- .NET 10 preview SDK (builds container stages and the SBOM generator)
- Node.js 20 (Angular UI build)
- Helm 3.16+
- Cosign 2.2+
Supply signing material via environment variables:
COSIGN_KEY_REF– e.g.file:./keys/cosign.keyorazurekms://…COSIGN_PASSWORD– password protecting the above key
The workflow defaults to multi-arch (linux/amd64,linux/arm64), SBOM in
CycloneDX, and SLSA provenance (https://slsa.dev/provenance/v1).
Debug store extraction
build_release.py now exports stripped debug artefacts for every ELF discovered in the published images. The files land under out/release/debug/.build-id/<aa>/<rest>.debug, with metadata captured in debug/debug-manifest.json (and a .sha256 sidecar). Use jq to inspect the manifest or readelf -n to spot-check a build-id. Offline Kit packaging should reuse the debug/ directory as-is.
UI auth smoke (Playwright)
As part of DEVOPS-UI-13-006 the pipelines will execute the UI auth smoke
tests (npm run test:e2e) after building the Angular bundle. See
docs/ops/ui-auth-smoke.md for the job design, environment stubs, and
offline runner considerations.
NuGet preview bootstrap
.NET 10 preview packages (Microsoft.Extensions.*, JwtBearer 10.0 RC, Sqlite 9 RC)
ship from the public dotnet-public Azure DevOps feed. We mirror them into
./local-nuget so restores succeed inside Offline Kit.
- Run
./ops/devops/sync-preview-nuget.shwhenever you update the manifest. - The script now understands the optional
SourceBasecolumn (V3 flat container) and writes packages alongside their SHA-256 checks. NuGet.configregisters the mirror (local), dotnet-public, and nuget.org.
Use python3 ops/devops/validate_restore_sources.py to prove the repo still
prefers the local mirror and that Directory.Build.props enforces the same order.
The validator now runs automatically in the build-test-deploy and release
workflows so CI fails fast when a feed priority regression slips in.
Detailed operator instructions live in docs/ops/nuget-preview-bootstrap.md.
Telemetry collector tooling (DEVOPS-OBS-50-001)
ops/devops/telemetry/generate_dev_tls.sh– generates a development CA and client/server certificates for the OpenTelemetry collector overlay (mutual TLS).ops/devops/telemetry/smoke_otel_collector.py– sends OTLP traces/metrics/logs over TLS and validates that the collector increments its receiver counters.ops/devops/telemetry/package_offline_bundle.py– re-packages collector assets for the Offline Kit.deploy/compose/docker-compose.telemetry-storage.yaml– Prometheus/Tempo/Loki stack for staging validation.
Combine these helpers with deploy/compose/docker-compose.telemetry.yaml to run
a secured collector locally before rolling out the Helm-based deployment.