CI/CD Infrastructure Overview
Sprint: CI/CD Enhancement - Documentation Last Updated: 2025-12-28 Workflow Count: 100 workflows
Quick Links
- Workflow Triggers & Dependencies
- Release Pipelines
- Security Scanning
- Test Strategy
- Troubleshooting Guide
Architecture Overview
The StellaOps CI/CD infrastructure uses Gitea Actions (GitHub Actions compatible) with a sophisticated multi-tier triggering strategy designed for:
- Determinism & Reproducibility - Identical builds across runs
- Offline-First Operation - Air-gap compatible pipelines
- Supply Chain Security - SLSA Level 2-3 compliance
- Developer Velocity - Fast PR feedback with comprehensive nightly testing
Pipeline Tiers
┌─────────────────────────────────────────────────────────────────────────┐
│ TRIGGER HIERARCHY │
├─────────────────────────────────────────────────────────────────────────┤
│ │
│ TIER 1: PR GATING (Every Pull Request) │
│ ├── test-matrix.yml (Unit, Architecture, Contract, Integration, │
│ │ Security, Golden) │
│ ├── build-test-deploy.yml (Build verification) │
│ ├── policy-lint.yml (Policy file validation) │
│ ├── sast-scan.yml (Static security analysis) │
│ └── docs.yml (Documentation validation) │
│ │
│ TIER 2: MAIN BRANCH (Post-Merge) │
│ ├── All Tier 1 workflows │
│ ├── build-test-deploy.yml → Deploy stage (staging environment) │
│ ├── integration-tests-gate.yml → Extended coverage │
│ └── coverage-report (Full coverage analysis) │
│ │
│ TIER 3: SCHEDULED (Nightly/Weekly) │
│ ├── nightly-regression.yml (2:00 AM UTC daily) │
│ ├── test-matrix.yml → Extended tests (5:00 AM UTC daily) │
│ ├── dependency-security-scan.yml (2:00 AM UTC Sunday) │
│ ├── renovate.yml (3:00 AM & 3:00 PM UTC daily) │
│ ├── sast-scan.yml (3:30 AM UTC Monday) │
│ └── migration-test.yml (4:30 AM UTC daily) │
│ │
│ TIER 4: RELEASE (Tag-Triggered) │
│ ├── release-suite.yml (suite-YYYY.MM tags) │
│ ├── release.yml (v* tags) │
│ └── module-publish.yml (module-*-v* tags) │
│ │
│ TIER 5: MANUAL (On-Demand) │
│ ├── cli-build.yml, scanner-determinism.yml │
│ ├── rollback.yml, promote.yml │
│ └── 20+ specialized test/debug workflows │
│ │
└─────────────────────────────────────────────────────────────────────────┘
Workflow Categories
1. Core Build & Test (12 workflows)
| Workflow | Purpose | Triggers |
|---|---|---|
build-test-deploy.yml |
Main build pipeline | PR, main push, daily, manual |
test-matrix.yml |
Unified test execution | PR, main push, daily, manual |
integration-tests-gate.yml |
Extended integration testing | PR, main push, manual |
nightly-regression.yml |
Comprehensive nightly suite | Daily 2 AM UTC |
migration-test.yml |
Database migration validation | PR (migrations), daily |
2. Release Automation (8 workflows)
| Workflow | Purpose | Triggers |
|---|---|---|
release-suite.yml |
Ubuntu-style suite releases | suite-* tags, manual |
release.yml |
Version bundle releases | v* tags, manual |
module-publish.yml |
Per-module publishing | module-*-v* tags, manual |
cli-build.yml |
Multi-platform CLI builds | Manual only |
promote.yml |
Environment promotion | Manual only |
rollback.yml |
Emergency rollback | Manual only |
3. Security Scanning (6 workflows)
| Workflow | Purpose | Triggers |
|---|---|---|
sast-scan.yml |
Static code analysis | PR, main push, weekly |
secrets-scan.yml |
Credential detection | PR, main push |
container-scan.yml |
Image vulnerability scanning | Dockerfile changes, daily |
dependency-security-scan.yml |
NuGet/npm vulnerability audit | Weekly, PR (deps) |
dependency-license-gate.yml |
License compliance | PR (deps) |
4. Quality Assurance (15 workflows)
| Workflow | Purpose | Triggers |
|---|---|---|
policy-lint.yml |
Policy file validation | PR, main push |
docs.yml |
Documentation linting | docs/** changes |
scanner-determinism.yml |
Output reproducibility | Manual only |
determinism-gate.yml |
Build determinism | Manual only |
cross-platform-determinism.yml |
Multi-OS verification | Manual only |
5. Module-Specific (30+ workflows)
Specialized workflows for individual modules (Scanner, Concelier, Authority, etc.)
Trigger Quick Reference
Branch Patterns
| Pattern | Example | Workflows Triggered |
|---|---|---|
Push to main |
Direct commit or merge | All Tier 1 + Tier 2 |
Push to develop |
Feature integration | Selected gating workflows |
| Pull Request | Any PR to main/develop | All Tier 1 (gating) |
Push to feature/* |
Feature branches | None (PR required) |
Push to release/* |
Release prep branches | Selected validation |
Tag Patterns
| Pattern | Example | Workflow |
|---|---|---|
v* |
v2025.12.1 |
release.yml |
suite-* |
suite-2026.04 |
release-suite.yml |
module-*-v* |
module-authority-v1.2.3 |
module-publish.yml |
Schedule Summary
| Time (UTC) | Frequency | Workflow |
|---|---|---|
| 2:00 AM | Daily | nightly-regression.yml |
| 2:00 AM | Sunday | dependency-security-scan.yml |
| 3:00 AM | Daily | renovate.yml |
| 3:30 AM | Monday | sast-scan.yml |
| 4:30 AM | Daily | migration-test.yml |
| 5:00 AM | Daily | build-test-deploy.yml, test-matrix.yml |
| 3:00 PM | Daily | renovate.yml |
Environment Flow
┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐
│ PR │───▶│ Staging │───▶│ Stable │───▶│ LTS │
│ (Preview)│ │ (Edge) │ │ (Tested) │ │(Long-Term)│
└──────────┘ └──────────┘ └──────────┘ └──────────┘
│ │ │ │
│ │ │ │
▼ ▼ ▼ ▼
PR tests Auto-deploy promote.yml promote.yml
(gating) on main merge (manual) (manual)
Environment Matrix
| Environment | Branch/Tag | Auto-Deploy | Rollback |
|---|---|---|---|
| Preview | PR | Yes (ephemeral) | N/A |
| Staging (Edge) | main |
Yes | rollback.yml |
| Stable | v* tags |
Manual | rollback.yml |
| LTS | suite-* tags |
Manual | rollback.yml |
Key Features
1. PR-Gating Tests
Required tests that must pass before merge:
- Unit Tests - Fast, isolated tests
- Architecture Tests - Dependency rule enforcement
- Contract Tests - API compatibility
- Integration Tests - PostgreSQL integration
- Security Tests - Security-focused assertions
- Golden Tests - Corpus-based validation
2. Determinism Verification
All builds produce identical outputs:
- Binary checksums compared across runs
- UTC timezone enforcement (
TZ: UTC) - Stable JSON serialization
- Reproducible SBOM generation
3. Supply Chain Security
- SBOM Generation - Syft for CycloneDX/SPDX
- Artifact Signing - Cosign/Sigstore integration
- Provenance - in-toto/DSSE attestations
- Dependency Scanning - Automated vulnerability detection
4. Rollback Automation
Emergency rollback via rollback.yml:
- Target: < 5 minute SLA
- Helm-based deployment rollback
- Health check verification
- Notification integration
Directory Structure
.gitea/
├── workflows/ # 100 workflow files
│ ├── build-test-deploy.yml
│ ├── test-matrix.yml
│ ├── release-suite.yml
│ └── ...
├── scripts/ # CI/CD scripts
│ ├── build/ # Build orchestration
│ ├── test/ # Test execution
│ ├── release/ # Release automation
│ ├── sign/ # Signing operations
│ └── validate/ # Validation scripts
└── docs/ # CI-specific docs
├── architecture.md
├── scripts.md
└── troubleshooting.md
devops/
├── scripts/
│ └── lib/ # Shared bash libraries
│ ├── logging.sh
│ ├── exit-codes.sh
│ ├── git-utils.sh
│ ├── path-utils.sh
│ └── hash-utils.sh
├── compose/ # Docker Compose profiles
├── helm/ # Helm charts
└── docker/ # Dockerfiles
Getting Started
Running Workflows Locally
# Run test matrix locally
./devops/scripts/test-local.sh
# Validate compose files
./devops/scripts/validate-compose.sh
# Run a specific test category
./.gitea/scripts/test/run-test-category.sh Unit
Triggering Manual Workflows
# Via Gitea UI: Actions → Workflow → Run workflow
# Or via API:
curl -X POST \
-H "Authorization: token $GITEA_TOKEN" \
"$GITEA_URL/api/v1/repos/owner/repo/actions/workflows/rollback.yml/dispatches" \
-d '{"ref":"main","inputs":{"environment":"staging","version":"v2025.12.0"}}'
Creating a Release
-
Module Release:
git tag module-authority-v1.2.3 git push origin module-authority-v1.2.3 -
Suite Release:
git tag suite-2026.04 git push origin suite-2026.04 -
Bundle Release:
git tag v2025.12.1 git push origin v2025.12.1
Related Documentation
- Workflow Triggers Deep Dive
- Release Pipeline Details
- Security Scanning Guide
- Test Strategy
- CI Quality Gates
- Troubleshooting
- Script Reference
Metrics & Monitoring
Key Metrics Tracked
| Metric | Target | Measurement |
|---|---|---|
| PR Build Time | < 15 min | Workflow duration |
| Main Build Time | < 20 min | Workflow duration |
| Test Flakiness | < 1% | Flaky test detection |
| Security Scan Coverage | 100% | SAST/DAST coverage |
| Rollback SLA | < 5 min | Rollback workflow duration |
Dashboard Links
- Workflow Runs (Gitea Actions UI)
- Test Results (TRX/JUnit artifacts)
- Coverage Reports (Generated nightly)