# DevOps Release Automation

The **release** workflow builds and signs the StellaOps service containers,
generates SBOM + provenance attestations, and emits a canonical
`release.yaml`. The logic lives under `ops/devops/release/` and is invoked
by the new `.gitea/workflows/release.yml` pipeline.

## Local dry run

```bash
./ops/devops/release/build_release.py \
  --version 2025.10.0-edge \
  --channel edge \
  --dry-run
```

Outputs land under `out/release/`. Use `--no-push` to run full builds without
pushing to the registry.

After the build completes, run the verifier to validate recorded hashes and artefact
presence:

```bash
python ops/devops/release/verify_release.py --release-dir out/release
```

## Python analyzer smoke & signing

`dotnet run --project tools/LanguageAnalyzerSmoke` exercises the Python language
analyzer plug-in against the golden fixtures (cold/warm timings, determinism). The
release workflow runs this harness automatically and then produces Cosign
signatures + SHA-256 sidecars for `StellaOps.Scanner.Analyzers.Lang.Python.dll`
and its `manifest.json`. Keep `COSIGN_KEY_REF`/`COSIGN_IDENTITY_TOKEN` populated so
the step can sign the artefacts; the generated `.sig`/`.sha256` files ship with the
Offline Kit bundle.

## Required tooling

- Docker 25+ with Buildx
- .NET 10 preview SDK (builds container stages and the SBOM generator)
- Node.js 20 (Angular UI build)
- Helm 3.16+
- Cosign 2.2+

Supply signing material via environment variables:

- `COSIGN_KEY_REF` – e.g. `file:./keys/cosign.key` or `azurekms://…`
- `COSIGN_PASSWORD` – password protecting the above key

The workflow defaults to multi-arch (`linux/amd64,linux/arm64`), SBOM in
CycloneDX, and SLSA provenance (`https://slsa.dev/provenance/v1`).

## Debug store extraction

`build_release.py` now exports stripped debug artefacts for every ELF discovered in the published images. The files land under `out/release/debug/.build-id/<aa>/<rest>.debug`, with metadata captured in `debug/debug-manifest.json` (and a `.sha256` sidecar). Use `jq` to inspect the manifest or `readelf -n` to spot-check a build-id. Offline Kit packaging should reuse the `debug/` directory as-is.

## UI auth smoke (Playwright)

As part of **DEVOPS-UI-13-006** the pipelines will execute the UI auth smoke
tests (`npm run test:e2e`) after building the Angular bundle. See
`docs/ops/ui-auth-smoke.md` for the job design, environment stubs, and
offline runner considerations.

## NuGet preview bootstrap

`.NET 10` preview packages (Microsoft.Extensions.*, JwtBearer 10.0 RC, Sqlite 9 RC)
ship from the public `dotnet-public` Azure DevOps feed. We mirror them into
`./local-nuget` so restores succeed inside Offline Kit.

1. Run `./ops/devops/sync-preview-nuget.sh` whenever you update the manifest.
2. The script now understands the optional `SourceBase` column (V3 flat container)
   and writes packages alongside their SHA-256 checks.
3. `NuGet.config` registers the mirror (`local`), dotnet-public, and nuget.org.

Use `python3 ops/devops/validate_restore_sources.py` to prove the repo still
prefers the local mirror and that `Directory.Build.props` enforces the same order.
The validator now runs automatically in the `build-test-deploy` and `release`
workflows so CI fails fast when a feed priority regression slips in.

Detailed operator instructions live in `docs/ops/nuget-preview-bootstrap.md`.

## Telemetry collector tooling (DEVOPS-OBS-50-001)

- `ops/devops/telemetry/generate_dev_tls.sh` – generates a development CA and
  client/server certificates for the OpenTelemetry collector overlay (mutual TLS).
- `ops/devops/telemetry/smoke_otel_collector.py` – sends OTLP traces/metrics/logs
  over TLS and validates that the collector increments its receiver counters.
- `ops/devops/telemetry/package_offline_bundle.py` – re-packages collector assets for the Offline Kit.
- `deploy/compose/docker-compose.telemetry-storage.yaml` – Prometheus/Tempo/Loki stack for staging validation.

Combine these helpers with `deploy/compose/docker-compose.telemetry.yaml` to run
a secured collector locally before rolling out the Helm-based deployment.