The release workflow builds and signs the StellaOps service containers, generates SBOM + provenance attestations, and emits a canonical release.yaml. The logic lives under ops/devops/release/ and is invoked by the new .gitea/workflows/release.yml pipeline.

Local dry run

./ops/devops/release/build_release.py \
  --version 2025.10.0-edge \
  --channel edge \
  --dry-run

Outputs land under out/release/. Use --no-push to run full builds without pushing to the registry.

After the build completes, run the verifier to validate recorded hashes and artefact presence:

python ops/devops/release/verify_release.py --release-dir out/release

Python analyzer smoke & signing

dotnet run --project src/Tools/LanguageAnalyzerSmoke exercises the Python language analyzer plug-in against the golden fixtures (cold/warm timings, determinism). The release workflow runs this harness automatically and then produces Cosign signatures + SHA-256 sidecars for StellaOps.Scanner.Analyzers.Lang.Python.dll and its manifest.json. Keep COSIGN_KEY_REF/COSIGN_IDENTITY_TOKEN populated so the step can sign the artefacts; the generated .sig/.sha256 files ship with the Offline Kit bundle.

Required tooling

Docker 25+ with Buildx
.NET 10 preview SDK (builds container stages and the SBOM generator)
Node.js 20 (Angular UI build)
Helm 3.16+
Cosign 2.2+

Supply signing material via environment variables:

COSIGN_KEY_REF – e.g. file:./keys/cosign.key or azurekms://…
COSIGN_PASSWORD – password protecting the above key

The workflow defaults to multi-arch (linux/amd64,linux/arm64), SBOM in CycloneDX, and SLSA provenance (https://slsa.dev/provenance/v1).

Debug store extraction

build_release.py now exports stripped debug artefacts for every ELF discovered in the published images. The files land under out/release/debug/.build-id/<aa>/<rest>.debug, with metadata captured in debug/debug-manifest.json (and a .sha256 sidecar). Use jq to inspect the manifest or readelf -n to spot-check a build-id. Offline Kit packaging should reuse the debug/ directory as-is.

UI auth smoke (Playwright)

As part of DEVOPS-UI-13-006 the pipelines will execute the UI auth smoke tests (npm run test:e2e) after building the Angular bundle. See docs/modules/ui/operations/auth-smoke.md for the job design, environment stubs, and offline runner considerations.

NuGet preview bootstrap

.NET 10 preview packages (Microsoft.Extensions.*, JwtBearer 10.0 RC, Sqlite 9 RC) ship from the public dotnet-public Azure DevOps feed. We mirror them into ./local-nuget so restores succeed inside Offline Kit.

Run ./ops/devops/sync-preview-nuget.sh whenever you update the manifest.
The script now understands the optional SourceBase column (V3 flat container) and writes packages alongside their SHA-256 checks.
NuGet.config registers the mirror (local), dotnet-public, and nuget.org.

Use python3 ops/devops/validate_restore_sources.py to prove the repo still prefers the local mirror and that Directory.Build.props enforces the same order. The validator now runs automatically in the build-test-deploy and release workflows so CI fails fast when a feed priority regression slips in.

Detailed operator instructions live in docs/modules/devops/runbooks/nuget-preview-bootstrap.md.

CI harnesses (offline-friendly)

Concelier: ops/devops/concelier-ci-runner/run-concelier-ci.sh builds concelier-webservice.slnf and runs WebService + Storage Mongo tests. Outputs binlog + TRX + summary under ops/devops/artifacts/concelier-ci/<ts>/.
Advisory AI: ops/devops/advisoryai-ci-runner/run-advisoryai-ci.sh builds src/AdvisoryAI/StellaOps.AdvisoryAI.sln, runs StellaOps.AdvisoryAI.Tests, and emits binlog + TRX + summary under ops/devops/artifacts/advisoryai-ci/<ts>/. For offline parity, configure a local NuGet feed in nuget.config.
Scanner: ops/devops/scanner-ci-runner/run-scanner-ci.sh builds src/Scanner/StellaOps.Scanner.sln and runs core/analyzer/web/worker test buckets with binlog + TRX outputs under ops/devops/artifacts/scanner-ci/<ts>/.

Telemetry collector tooling (DEVOPS-OBS-50-001)

ops/devops/telemetry/generate_dev_tls.sh – generates a development CA and client/server certificates for the OpenTelemetry collector overlay (mutual TLS).
ops/devops/telemetry/smoke_otel_collector.py – sends OTLP traces/metrics/logs over TLS and validates that the collector increments its receiver counters.
ops/devops/telemetry/package_offline_bundle.py – re-packages collector assets for the Offline Kit.
ops/devops/telemetry/tenant_isolation_smoke.py – verifies Tempo/Loki tenant isolation with mTLS and scoped headers.
deploy/compose/docker-compose.telemetry-storage.yaml – Prometheus/Tempo/Loki stack for staging validation.

Combine these helpers with deploy/compose/docker-compose.telemetry.yaml to run a secured collector locally before rolling out the Helm-based deployment.

README.md Unescape Escape

DevOps Release Automation