22 KiB
22 KiB
Release Pipelines
Complete guide to StellaOps release automation including suite releases, module publishing, and promotion workflows.
Release Strategy Overview
StellaOps uses a dual-versioning strategy:
- Suite Releases - Ubuntu-style
YYYY.MMversioning with codenames - Module Releases - Semantic versioning
MAJOR.MINOR.PATCHper module
Release Channels
| Channel | Purpose | Stability | Update Frequency |
|---|---|---|---|
| Edge | Latest features, early adopters | Beta | Every merge to main |
| Stable | Production-ready, tested | Production | Bi-weekly |
| LTS | Long-term support, enterprise | Enterprise | Quarterly |
Suite Release Pipeline
Trigger
# Tag-based trigger
git tag suite-2026.04
git push origin suite-2026.04
# Or manual trigger via Gitea Actions UI
# Workflow: release-suite.yml
# Inputs: version, codename, channel, skip_tests, dry_run
Workflow: release-suite.yml
┌─────────────────────────────────────────────────────────────────┐
│ SUITE RELEASE PIPELINE │
│ │
│ ┌──────────────┐ │
│ │ parse-tag │ (if triggered by tag push) │
│ │ or validate │ (if triggered manually) │
│ └──────┬───────┘ │
│ │ │
│ ▼ │
│ ┌──────────────┐ │
│ │ test-gate │ (optional, skipped with skip_tests=true) │
│ └──────┬───────┘ │
│ │ │
│ ┌────┴────────────────────────────────────────┐ │
│ │ BUILD PHASE │ │
│ │ │ │
│ │ ┌─────────────────┐ ┌─────────────────┐ │ │
│ │ │ build-modules │ │ build-containers│ │ │
│ │ │ (9 in parallel)│ │ (9 in parallel)│ │ │
│ │ └─────────────────┘ └─────────────────┘ │ │
│ │ │ │
│ │ ┌─────────────────┐ ┌─────────────────┐ │ │
│ │ │ build-cli │ │ build-helm │ │ │
│ │ │ (5 platforms) │ │ │ │ │
│ │ └─────────────────┘ └─────────────────┘ │ │
│ │ │ │
│ └─────────────────────┬────────────────────────┘ │
│ │ │
│ ▼ │
│ ┌───────────────────────────────────────────────┐ │
│ │ release-manifest │ │
│ │ - Binary manifest with SHA256 checksums │ │
│ │ - SBOM generation (CycloneDX, SPDX) │ │
│ │ - Provenance attestation (in-toto/DSSE) │ │
│ └───────────────────────┬────────────────────────┘ │
│ │ │
│ ┌─────────────────────┴─────────────────────────┐ │
│ │ │ │
│ ▼ ▼ ▼ │
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │
│ │ changelog │ │ suite-docs │ │ compose │ │
│ │ generation │ │ generation │ │ generation │ │
│ └──────────────┘ └──────────────┘ └──────────────┘ │
│ │ │
│ ▼ │
│ ┌───────────────────────────────────────────────┐ │
│ │ create-release │ │
│ │ - Upload artifacts to Gitea Releases │ │
│ │ - Sign with Cosign (keyless Sigstore) │ │
│ │ - Publish to container registry │ │
│ └───────────────────────┬────────────────────────┘ │
│ │ │
│ ▼ │
│ ┌───────────────────────────────────────────────┐ │
│ │ commit-docs │ │
│ │ - Update docs/releases/ │ │
│ │ - Update devops/compose/ │ │
│ └───────────────────────────────────────────────┘ │
│ │
└─────────────────────────────────────────────────────────────────┘
Suite Versioning
| Component | Format | Example |
|---|---|---|
| Suite Version | YYYY.MM |
2026.04 |
| Codename | Alpha name | Nova, Orion, Phoenix |
| Full Tag | suite-YYYY.MM |
suite-2026.04 |
| Docker Tag | YYYY.MM-channel |
2026.04-stable |
Modules Built
| Module | NuGet Package | Container Image |
|---|---|---|
| Authority | StellaOps.Authority |
stellaops/authority |
| Scanner | StellaOps.Scanner |
stellaops/scanner |
| Concelier | StellaOps.Concelier |
stellaops/concelier |
| Excititor | StellaOps.Excititor |
stellaops/excititor |
| SbomService | StellaOps.SbomService |
stellaops/sbom-service |
| EvidenceLocker | StellaOps.EvidenceLocker |
stellaops/evidence-locker |
| Policy | StellaOps.Policy |
stellaops/policy |
| Attestor | StellaOps.Attestor |
stellaops/attestor |
| VexLens | StellaOps.VexLens |
stellaops/vexlens |
CLI Platforms
| Runtime ID | OS | Architecture | Binary Name |
|---|---|---|---|
linux-x64 |
Linux | x86_64 | stellaops-linux-x64 |
linux-arm64 |
Linux | ARM64 | stellaops-linux-arm64 |
win-x64 |
Windows | x86_64 | stellaops-win-x64.exe |
osx-x64 |
macOS | Intel | stellaops-osx-x64 |
osx-arm64 |
macOS | Apple Silicon | stellaops-osx-arm64 |
Module Release Pipeline
Trigger
# Tag-based trigger
git tag module-authority-v1.2.3
git push origin module-authority-v1.2.3
# Or manual trigger via Gitea Actions UI
# Workflow: module-publish.yml
# Inputs: module, version, publish_nuget, publish_container, prerelease
Workflow: module-publish.yml
┌─────────────────────────────────────────────────────────────────┐
│ MODULE PUBLISH PIPELINE │
│ │
│ ┌──────────────┐ │
│ │ parse-tag │ Extract module name and version from tag │
│ │ or validate │ Normalize manual inputs │
│ └──────┬───────┘ │
│ │ │
│ ┌────┴────────────────────────────────────────┐ │
│ │ │ │
│ ▼ ▼ │
│ ┌──────────────┐ ┌──────────────┐ │
│ │publish-nuget │ (if flag set) │publish-cont. │ │
│ │ │ │ (if flag set)│ │
│ │ - Pack │ │ - Build │ │
│ │ - Sign │ │ - Scan │ │
│ │ - Push │ │ - Sign │ │
│ └──────────────┘ │ - Push │ │
│ └──────────────┘ │
│ │ │
│ OR (if module=CLI) │ │
│ ▼ │
│ ┌──────────────────────────────────────────────────┐ │
│ │ publish-cli │ │
│ │ - Build for 5 platforms │ │
│ │ - Native AOT compilation │ │
│ │ - Code sign binaries │ │
│ │ - Generate checksums │ │
│ │ - Upload to release │ │
│ └──────────────────────────────────────────────────┘ │
│ │ │
│ ▼ │
│ ┌──────────────────────────────────────────────────┐ │
│ │ summary │ │
│ │ - Release notes │ │
│ │ - Artifact links │ │
│ │ - SBOM references │ │
│ └──────────────────────────────────────────────────┘ │
│ │
└─────────────────────────────────────────────────────────────────┘
Module Tag Format
module-<name>-v<semver>
Examples:
module-authority-v1.2.3
module-scanner-v2.0.0
module-cli-v3.1.0-beta.1
Available Modules
| Module Name | NuGet | Container | CLI |
|---|---|---|---|
authority |
Yes | Yes | No |
scanner |
Yes | Yes | No |
concelier |
Yes | Yes | No |
excititor |
Yes | Yes | No |
sbomservice |
Yes | Yes | No |
evidencelocker |
Yes | Yes | No |
policy |
Yes | Yes | No |
attestor |
Yes | Yes | No |
vexlens |
Yes | Yes | No |
cli |
No | No | Yes (multi-platform) |
Bundle Release Pipeline
Trigger
# Tag-based trigger
git tag v2025.12.1
git push origin v2025.12.1
# Channel-specific tags
git tag v2025.12.0-edge
git tag v2025.12.0-stable
git tag v2025.12.0-lts
Workflow: release.yml
Creates deterministic release bundles with:
- Signed container images
- SBOM generation
- Provenance attestations
- CLI parity verification
Rollback Pipeline
Trigger
# Manual trigger only via Gitea Actions UI
# Workflow: rollback.yml
# Inputs: environment, service, version, reason
Workflow: rollback.yml
┌─────────────────────────────────────────────────────────────────┐
│ ROLLBACK PIPELINE │
│ (SLA Target: < 5 min) │
│ │
│ ┌──────────────┐ │
│ │ validate │ Verify inputs and permissions │
│ └──────┬───────┘ │
│ │ │
│ ▼ │
│ ┌──────────────┐ │
│ │ fetch-prev │ Download previous version artifacts │
│ │ version │ │
│ └──────┬───────┘ │
│ │ │
│ ▼ │
│ ┌──────────────┐ │
│ │ execute │ Run rollback via Helm/kubectl │
│ │ rollback │ │
│ └──────┬───────┘ │
│ │ │
│ ▼ │
│ ┌──────────────┐ │
│ │health-check │ Verify service health post-rollback │
│ └──────┬───────┘ │
│ │ │
│ ▼ │
│ ┌──────────────┐ │
│ │ notify │ Send notification (Slack/Teams/Webhook) │
│ └──────────────┘ │
│ │
└─────────────────────────────────────────────────────────────────┘
Rollback Parameters
| Parameter | Type | Description |
|---|---|---|
environment |
choice | staging, production |
service |
choice | Service to rollback (or all) |
version |
string | Target version to rollback to |
reason |
string | Reason for rollback (audit log) |
dry_run |
boolean | Simulate without executing |
Promotion Pipeline
Trigger
# Manual trigger only via Gitea Actions UI
# Workflow: promote.yml
# Inputs: from_environment, to_environment, version
Promotion Flow
┌─────────────┐
│ Edge │ (Automatic on main merge)
└──────┬──────┘
│
│ promote.yml (manual)
▼
┌─────────────┐
│ Stable │ (After testing period)
└──────┬──────┘
│
│ promote.yml (manual)
▼
┌─────────────┐
│ LTS │ (After extended validation)
└─────────────┘
Promotion Checklist (Automated)
-
Pre-Flight Checks
- All tests passing in source environment
- No critical vulnerabilities
- Performance SLOs met
- Documentation complete
-
Promotion Steps
- Re-tag containers with new channel
- Update Helm chart values
- Deploy to target environment
- Run smoke tests
-
Post-Promotion
- Health check verification
- Update release documentation
- Notify stakeholders
Artifact Signing
Cosign Integration
All release artifacts are signed using Cosign with Sigstore keyless signing:
# Verify container signature
cosign verify \
--certificate-identity-regexp=".*github.com/stellaops.*" \
--certificate-oidc-issuer="https://token.actions.githubusercontent.com" \
ghcr.io/stellaops/scanner:2026.04
# Verify SBOM
cosign verify-attestation \
--type spdxjson \
--certificate-identity-regexp=".*github.com/stellaops.*" \
ghcr.io/stellaops/scanner:2026.04
Signature Artifacts
| Artifact Type | Signature Location |
|---|---|
| Container Image | OCI registry (same repo) |
| CLI Binary | .sig file alongside binary |
| SBOM | Attestation on OCI image |
| Provenance | Attestation on OCI image |
Release Artifacts
Per-Release Artifacts
| Artifact | Format | Location |
|---|---|---|
| Release Notes | Markdown | Gitea Release |
| Changelog | CHANGELOG.md |
Gitea Release, docs/releases/ |
| Binary Checksums | SHA256SUMS.txt |
Gitea Release |
| SBOM (CycloneDX) | JSON | Gitea Release, OCI attestation |
| SBOM (SPDX) | JSON | Gitea Release |
| Provenance | in-toto/DSSE | OCI attestation |
| Docker Compose | YAML | devops/compose/ |
| Helm Chart | TGZ | OCI registry |
Artifact Retention
| Environment | Retention Period |
|---|---|
| PR/Preview | 7 days |
| Edge | 30 days |
| Stable | 1 year |
| LTS | 3 years |
Creating a Release
Suite Release
# 1. Ensure main is stable
git checkout main
git pull
# 2. Create and push tag
git tag suite-2026.04
git push origin suite-2026.04
# 3. Monitor release pipeline
# Gitea Actions → release-suite.yml
# 4. Verify artifacts
# - Check Gitea Releases page
# - Verify container images pushed
# - Validate SBOM and signatures
Module Release
# 1. Update module version
# Edit src/<Module>/version.txt or .csproj
# 2. Create and push tag
git tag module-authority-v1.2.3
git push origin module-authority-v1.2.3
# 3. Monitor release pipeline
# Gitea Actions → module-publish.yml
Hotfix Release
# 1. Create hotfix branch from release tag
git checkout -b hotfix/v2025.12.1 v2025.12.0
# 2. Apply fix
# ... make changes ...
git commit -m "Fix: critical security issue"
# 3. Create hotfix tag
git tag v2025.12.1
git push origin hotfix/v2025.12.1 v2025.12.1
# 4. Fast-track through pipeline
# Workflow will run with reduced test scope
Troubleshooting Releases
Release Pipeline Failed
- Check build logs - Gitea Actions → failed job
- Verify tag format - Must match expected pattern
- Check secrets - Registry credentials, signing keys
- Review test failures - May need to skip with
skip_tests=true
Container Not Published
- Check registry authentication -
REGISTRY_TOKENsecret - Verify image name - Check for typos in workflow
- Check rate limits - May need to wait and retry
- Review scan results - Image may be blocked by vulnerability scan
Signature Verification Failed
- Check Sigstore availability - May have temporary outage
- Verify certificate identity - Workflow must match expected pattern
- Check OIDC issuer - Must be GitHub/Gitea Actions
Rollback Failed
- Verify target version exists - Check artifact storage
- Check Helm/kubectl access - Cluster credentials
- Review health check - Service may need manual intervention
- Check resource constraints - May need to scale down first