# Release Pipelines > Complete guide to StellaOps release automation including suite releases, module publishing, and promotion workflows. --- ## Release Strategy Overview StellaOps uses a **dual-versioning strategy**: 1. **Suite Releases** - Ubuntu-style `YYYY.MM` versioning with codenames 2. **Module Releases** - Semantic versioning `MAJOR.MINOR.PATCH` per module ### Release Channels | Channel | Purpose | Stability | Update Frequency | |---------|---------|-----------|------------------| | **Edge** | Latest features, early adopters | Beta | Every merge to main | | **Stable** | Production-ready, tested | Production | Bi-weekly | | **LTS** | Long-term support, enterprise | Enterprise | Quarterly | --- ## Suite Release Pipeline ### Trigger ```bash # Tag-based trigger git tag suite-2026.04 git push origin suite-2026.04 # Or manual trigger via Gitea Actions UI # Workflow: release-suite.yml # Inputs: version, codename, channel, skip_tests, dry_run ``` ### Workflow: `release-suite.yml` ``` ┌─────────────────────────────────────────────────────────────────┐ │ SUITE RELEASE PIPELINE │ │ │ │ ┌──────────────┐ │ │ │ parse-tag │ (if triggered by tag push) │ │ │ or validate │ (if triggered manually) │ │ └──────┬───────┘ │ │ │ │ │ ▼ │ │ ┌──────────────┐ │ │ │ test-gate │ (optional, skipped with skip_tests=true) │ │ └──────┬───────┘ │ │ │ │ │ ┌────┴────────────────────────────────────────┐ │ │ │ BUILD PHASE │ │ │ │ │ │ │ │ ┌─────────────────┐ ┌─────────────────┐ │ │ │ │ │ build-modules │ │ build-containers│ │ │ │ │ │ (9 in parallel)│ │ (9 in parallel)│ │ │ │ │ └─────────────────┘ └─────────────────┘ │ │ │ │ │ │ │ │ ┌─────────────────┐ ┌─────────────────┐ │ │ │ │ │ build-cli │ │ build-helm │ │ │ │ │ │ (5 platforms) │ │ │ │ │ │ │ └─────────────────┘ └─────────────────┘ │ │ │ │ │ │ │ └─────────────────────┬────────────────────────┘ │ │ │ │ │ ▼ │ │ ┌───────────────────────────────────────────────┐ │ │ │ release-manifest │ │ │ │ - Binary manifest with SHA256 checksums │ │ │ │ - SBOM generation (CycloneDX, SPDX) │ │ │ │ - Provenance attestation (in-toto/DSSE) │ │ │ └───────────────────────┬────────────────────────┘ │ │ │ │ │ ┌─────────────────────┴─────────────────────────┐ │ │ │ │ │ │ ▼ ▼ ▼ │ │ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │ │ │ changelog │ │ suite-docs │ │ compose │ │ │ │ generation │ │ generation │ │ generation │ │ │ └──────────────┘ └──────────────┘ └──────────────┘ │ │ │ │ │ ▼ │ │ ┌───────────────────────────────────────────────┐ │ │ │ create-release │ │ │ │ - Upload artifacts to Gitea Releases │ │ │ │ - Sign with Cosign (keyless Sigstore) │ │ │ │ - Publish to container registry │ │ │ └───────────────────────┬────────────────────────┘ │ │ │ │ │ ▼ │ │ ┌───────────────────────────────────────────────┐ │ │ │ commit-docs │ │ │ │ - Update docs/releases/ │ │ │ │ - Update devops/compose/ │ │ │ └───────────────────────────────────────────────┘ │ │ │ └─────────────────────────────────────────────────────────────────┘ ``` ### Suite Versioning | Component | Format | Example | |-----------|--------|---------| | Suite Version | `YYYY.MM` | `2026.04` | | Codename | Alpha name | `Nova`, `Orion`, `Phoenix` | | Full Tag | `suite-YYYY.MM` | `suite-2026.04` | | Docker Tag | `YYYY.MM-channel` | `2026.04-stable` | ### Modules Built | Module | NuGet Package | Container Image | |--------|---------------|-----------------| | Authority | `StellaOps.Authority` | `stellaops/authority` | | Scanner | `StellaOps.Scanner` | `stellaops/scanner` | | Concelier | `StellaOps.Concelier` | `stellaops/concelier` | | Excititor | `StellaOps.Excititor` | `stellaops/excititor` | | SbomService | `StellaOps.SbomService` | `stellaops/sbom-service` | | EvidenceLocker | `StellaOps.EvidenceLocker` | `stellaops/evidence-locker` | | Policy | `StellaOps.Policy` | `stellaops/policy` | | Attestor | `StellaOps.Attestor` | `stellaops/attestor` | | VexLens | `StellaOps.VexLens` | `stellaops/vexlens` | ### CLI Platforms | Runtime ID | OS | Architecture | Binary Name | |------------|-----|--------------|-------------| | `linux-x64` | Linux | x86_64 | `stellaops-linux-x64` | | `linux-arm64` | Linux | ARM64 | `stellaops-linux-arm64` | | `win-x64` | Windows | x86_64 | `stellaops-win-x64.exe` | | `osx-x64` | macOS | Intel | `stellaops-osx-x64` | | `osx-arm64` | macOS | Apple Silicon | `stellaops-osx-arm64` | --- ## Module Release Pipeline ### Trigger ```bash # Tag-based trigger git tag module-authority-v1.2.3 git push origin module-authority-v1.2.3 # Or manual trigger via Gitea Actions UI # Workflow: module-publish.yml # Inputs: module, version, publish_nuget, publish_container, prerelease ``` ### Workflow: `module-publish.yml` ``` ┌─────────────────────────────────────────────────────────────────┐ │ MODULE PUBLISH PIPELINE │ │ │ │ ┌──────────────┐ │ │ │ parse-tag │ Extract module name and version from tag │ │ │ or validate │ Normalize manual inputs │ │ └──────┬───────┘ │ │ │ │ │ ┌────┴────────────────────────────────────────┐ │ │ │ │ │ │ ▼ ▼ │ │ ┌──────────────┐ ┌──────────────┐ │ │ │publish-nuget │ (if flag set) │publish-cont. │ │ │ │ │ │ (if flag set)│ │ │ │ - Pack │ │ - Build │ │ │ │ - Sign │ │ - Scan │ │ │ │ - Push │ │ - Sign │ │ │ └──────────────┘ │ - Push │ │ │ └──────────────┘ │ │ │ │ │ OR (if module=CLI) │ │ │ ▼ │ │ ┌──────────────────────────────────────────────────┐ │ │ │ publish-cli │ │ │ │ - Build for 5 platforms │ │ │ │ - Native AOT compilation │ │ │ │ - Code sign binaries │ │ │ │ - Generate checksums │ │ │ │ - Upload to release │ │ │ └──────────────────────────────────────────────────┘ │ │ │ │ │ ▼ │ │ ┌──────────────────────────────────────────────────┐ │ │ │ summary │ │ │ │ - Release notes │ │ │ │ - Artifact links │ │ │ │ - SBOM references │ │ │ └──────────────────────────────────────────────────┘ │ │ │ └─────────────────────────────────────────────────────────────────┘ ``` ### Module Tag Format ``` module--v Examples: module-authority-v1.2.3 module-scanner-v2.0.0 module-cli-v3.1.0-beta.1 ``` ### Available Modules | Module Name | NuGet | Container | CLI | |-------------|-------|-----------|-----| | `authority` | Yes | Yes | No | | `scanner` | Yes | Yes | No | | `concelier` | Yes | Yes | No | | `excititor` | Yes | Yes | No | | `sbomservice` | Yes | Yes | No | | `evidencelocker` | Yes | Yes | No | | `policy` | Yes | Yes | No | | `attestor` | Yes | Yes | No | | `vexlens` | Yes | Yes | No | | `cli` | No | No | Yes (multi-platform) | --- ## Bundle Release Pipeline ### Trigger ```bash # Tag-based trigger git tag v2025.12.1 git push origin v2025.12.1 # Channel-specific tags git tag v2025.12.0-edge git tag v2025.12.0-stable git tag v2025.12.0-lts ``` ### Workflow: `release.yml` Creates deterministic release bundles with: - Signed container images - SBOM generation - Provenance attestations - CLI parity verification --- ## Rollback Pipeline ### Trigger ```bash # Manual trigger only via Gitea Actions UI # Workflow: rollback.yml # Inputs: environment, service, version, reason ``` ### Workflow: `rollback.yml` ``` ┌─────────────────────────────────────────────────────────────────┐ │ ROLLBACK PIPELINE │ │ (SLA Target: < 5 min) │ │ │ │ ┌──────────────┐ │ │ │ validate │ Verify inputs and permissions │ │ └──────┬───────┘ │ │ │ │ │ ▼ │ │ ┌──────────────┐ │ │ │ fetch-prev │ Download previous version artifacts │ │ │ version │ │ │ └──────┬───────┘ │ │ │ │ │ ▼ │ │ ┌──────────────┐ │ │ │ execute │ Run rollback via Helm/kubectl │ │ │ rollback │ │ │ └──────┬───────┘ │ │ │ │ │ ▼ │ │ ┌──────────────┐ │ │ │health-check │ Verify service health post-rollback │ │ └──────┬───────┘ │ │ │ │ │ ▼ │ │ ┌──────────────┐ │ │ │ notify │ Send notification (Slack/Teams/Webhook) │ │ └──────────────┘ │ │ │ └─────────────────────────────────────────────────────────────────┘ ``` ### Rollback Parameters | Parameter | Type | Description | |-----------|------|-------------| | `environment` | choice | `staging`, `production` | | `service` | choice | Service to rollback (or `all`) | | `version` | string | Target version to rollback to | | `reason` | string | Reason for rollback (audit log) | | `dry_run` | boolean | Simulate without executing | --- ## Promotion Pipeline ### Trigger ```bash # Manual trigger only via Gitea Actions UI # Workflow: promote.yml # Inputs: from_environment, to_environment, version ``` ### Promotion Flow ``` ┌─────────────┐ │ Edge │ (Automatic on main merge) └──────┬──────┘ │ │ promote.yml (manual) ▼ ┌─────────────┐ │ Stable │ (After testing period) └──────┬──────┘ │ │ promote.yml (manual) ▼ ┌─────────────┐ │ LTS │ (After extended validation) └─────────────┘ ``` ### Promotion Checklist (Automated) 1. **Pre-Flight Checks** - All tests passing in source environment - No critical vulnerabilities - Performance SLOs met - Documentation complete 2. **Promotion Steps** - Re-tag containers with new channel - Update Helm chart values - Deploy to target environment - Run smoke tests 3. **Post-Promotion** - Health check verification - Update release documentation - Notify stakeholders --- ## Artifact Signing ### Cosign Integration All release artifacts are signed using Cosign with Sigstore keyless signing: ```bash # Verify container signature cosign verify \ --certificate-identity-regexp=".*github.com/stellaops.*" \ --certificate-oidc-issuer="https://token.actions.githubusercontent.com" \ ghcr.io/stellaops/scanner:2026.04 # Verify SBOM cosign verify-attestation \ --type spdxjson \ --certificate-identity-regexp=".*github.com/stellaops.*" \ ghcr.io/stellaops/scanner:2026.04 ``` ### Signature Artifacts | Artifact Type | Signature Location | |---------------|-------------------| | Container Image | OCI registry (same repo) | | CLI Binary | `.sig` file alongside binary | | SBOM | Attestation on OCI image | | Provenance | Attestation on OCI image | --- ## Release Artifacts ### Per-Release Artifacts | Artifact | Format | Location | |----------|--------|----------| | Release Notes | Markdown | Gitea Release | | Changelog | `CHANGELOG.md` | Gitea Release, `docs/releases/` | | Binary Checksums | `SHA256SUMS.txt` | Gitea Release | | SBOM (CycloneDX) | JSON | Gitea Release, OCI attestation | | SBOM (SPDX) | JSON | Gitea Release | | Provenance | in-toto/DSSE | OCI attestation | | Docker Compose | YAML | `devops/compose/` | | Helm Chart | TGZ | OCI registry | ### Artifact Retention | Environment | Retention Period | |-------------|------------------| | PR/Preview | 7 days | | Edge | 30 days | | Stable | 1 year | | LTS | 3 years | --- ## Creating a Release ### Suite Release ```bash # 1. Ensure main is stable git checkout main git pull # 2. Create and push tag git tag suite-2026.04 git push origin suite-2026.04 # 3. Monitor release pipeline # Gitea Actions → release-suite.yml # 4. Verify artifacts # - Check Gitea Releases page # - Verify container images pushed # - Validate SBOM and signatures ``` ### Module Release ```bash # 1. Update module version # Edit src//version.txt or .csproj # 2. Create and push tag git tag module-authority-v1.2.3 git push origin module-authority-v1.2.3 # 3. Monitor release pipeline # Gitea Actions → module-publish.yml ``` ### Hotfix Release ```bash # 1. Create hotfix branch from release tag git checkout -b hotfix/v2025.12.1 v2025.12.0 # 2. Apply fix # ... make changes ... git commit -m "Fix: critical security issue" # 3. Create hotfix tag git tag v2025.12.1 git push origin hotfix/v2025.12.1 v2025.12.1 # 4. Fast-track through pipeline # Workflow will run with reduced test scope ``` --- ## Troubleshooting Releases ### Release Pipeline Failed 1. **Check build logs** - Gitea Actions → failed job 2. **Verify tag format** - Must match expected pattern 3. **Check secrets** - Registry credentials, signing keys 4. **Review test failures** - May need to skip with `skip_tests=true` ### Container Not Published 1. **Check registry authentication** - `REGISTRY_TOKEN` secret 2. **Verify image name** - Check for typos in workflow 3. **Check rate limits** - May need to wait and retry 4. **Review scan results** - Image may be blocked by vulnerability scan ### Signature Verification Failed 1. **Check Sigstore availability** - May have temporary outage 2. **Verify certificate identity** - Workflow must match expected pattern 3. **Check OIDC issuer** - Must be GitHub/Gitea Actions ### Rollback Failed 1. **Verify target version exists** - Check artifact storage 2. **Check Helm/kubectl access** - Cluster credentials 3. **Review health check** - Service may need manual intervention 4. **Check resource constraints** - May need to scale down first --- ## Related Documentation - [README - CI/CD Overview](./README.md) - [Workflow Triggers](./workflow-triggers.md) - [Versioning Guide](../releases/VERSIONING.md) - [Container Registry Guide](../operations/container-registry.md) - [Helm Deployment Guide](../operations/helm-deployment.md)