stella-ops.org/git.stella-ops.org
1
0
Fork 0
Code Issues Pull Requests Actions 3 Releases Wiki Activity
Files
c2b9cd8d1f1a81af32c757063a460815ff4949a4
git.stella-ops.org/docs/cicd/README.md

330 lines
12 KiB
Markdown
Raw Blame History

# CI/CD Infrastructure Overview
> **Sprint:** CI/CD Enhancement - Documentation
> **Last Updated:** 2025-12-28
> **Workflow Count:** 100 workflows
## Quick Links
- [Workflow Triggers & Dependencies](./workflow-triggers.md)
- [Release Pipelines](./release-pipelines.md)
- [Security Scanning](./security-scanning.md)
- [Test Strategy](./test-strategy.md)
- [Troubleshooting Guide](../.gitea/docs/troubleshooting.md)
---
## Architecture Overview
The StellaOps CI/CD infrastructure uses **Gitea Actions** (GitHub Actions compatible) with a sophisticated multi-tier triggering strategy designed for:
- **Determinism & Reproducibility** - Identical builds across runs
- **Offline-First Operation** - Air-gap compatible pipelines
- **Supply Chain Security** - SLSA Level 2-3 compliance
- **Developer Velocity** - Fast PR feedback with comprehensive nightly testing
### Pipeline Tiers
```
┌─────────────────────────────────────────────────────────────────────────┐
│ TRIGGER HIERARCHY │
├─────────────────────────────────────────────────────────────────────────┤
│ │
│ TIER 1: PR GATING (Every Pull Request) │
│ ├── test-matrix.yml (Unit, Architecture, Contract, Integration, │
│ │ Security, Golden) │
│ ├── build-test-deploy.yml (Build verification) │
│ ├── policy-lint.yml (Policy file validation) │
│ ├── sast-scan.yml (Static security analysis) │
│ └── docs.yml (Documentation validation) │
│ │
│ TIER 2: MAIN BRANCH (Post-Merge) │
│ ├── All Tier 1 workflows │
│ ├── build-test-deploy.yml → Deploy stage (staging environment) │
│ ├── integration-tests-gate.yml → Extended coverage │
│ └── coverage-report (Full coverage analysis) │
│ │
│ TIER 3: SCHEDULED (Nightly/Weekly) │
│ ├── nightly-regression.yml (2:00 AM UTC daily) │
│ ├── test-matrix.yml → Extended tests (5:00 AM UTC daily) │
│ ├── dependency-security-scan.yml (2:00 AM UTC Sunday) │
│ ├── renovate.yml (3:00 AM & 3:00 PM UTC daily) │
│ ├── sast-scan.yml (3:30 AM UTC Monday) │
│ └── migration-test.yml (4:30 AM UTC daily) │
│ │
│ TIER 4: RELEASE (Tag-Triggered) │
│ ├── release-suite.yml (suite-YYYY.MM tags) │
│ ├── release.yml (v* tags) │
│ └── module-publish.yml (module-*-v* tags) │
│ │
│ TIER 5: MANUAL (On-Demand) │
│ ├── cli-build.yml, scanner-determinism.yml │
│ ├── rollback.yml, promote.yml │
│ └── 20+ specialized test/debug workflows │
│ │
└─────────────────────────────────────────────────────────────────────────┘
```
---
## Workflow Categories
### 1. Core Build & Test (12 workflows)
| Workflow | Purpose | Triggers |
|----------|---------|----------|
| `build-test-deploy.yml` | Main build pipeline | PR, main push, daily, manual |
| `test-matrix.yml` | Unified test execution | PR, main push, daily, manual |
| `integration-tests-gate.yml` | Extended integration testing | PR, main push, manual |
| `nightly-regression.yml` | Comprehensive nightly suite | Daily 2 AM UTC |
| `migration-test.yml` | Database migration validation | PR (migrations), daily |
### 2. Release Automation (8 workflows)
| Workflow | Purpose | Triggers |
|----------|---------|----------|
| `release-suite.yml` | Ubuntu-style suite releases | `suite-*` tags, manual |
| `release.yml` | Version bundle releases | `v*` tags, manual |
| `module-publish.yml` | Per-module publishing | `module-*-v*` tags, manual |
| `cli-build.yml` | Multi-platform CLI builds | Manual only |
| `promote.yml` | Environment promotion | Manual only |
| `rollback.yml` | Emergency rollback | Manual only |
### 3. Security Scanning (6 workflows)
| Workflow | Purpose | Triggers |
|----------|---------|----------|
| `sast-scan.yml` | Static code analysis | PR, main push, weekly |
| `secrets-scan.yml` | Credential detection | PR, main push |
| `container-scan.yml` | Image vulnerability scanning | Dockerfile changes, daily |
| `dependency-security-scan.yml` | NuGet/npm vulnerability audit | Weekly, PR (deps) |
| `dependency-license-gate.yml` | License compliance | PR (deps) |
### 4. Quality Assurance (15 workflows)
| Workflow | Purpose | Triggers |
|----------|---------|----------|
| `policy-lint.yml` | Policy file validation | PR, main push |
| `docs.yml` | Documentation linting | docs/** changes |
| `scanner-determinism.yml` | Output reproducibility | Manual only |
| `determinism-gate.yml` | Build determinism | Manual only |
| `cross-platform-determinism.yml` | Multi-OS verification | Manual only |
### 5. Module-Specific (30+ workflows)
Specialized workflows for individual modules (Scanner, Concelier, Authority, etc.)
---
## Trigger Quick Reference
### Branch Patterns
| Pattern | Example | Workflows Triggered |
|---------|---------|---------------------|
| Push to `main` | Direct commit or merge | All Tier 1 + Tier 2 |
| Push to `develop` | Feature integration | Selected gating workflows |
| Pull Request | Any PR to main/develop | All Tier 1 (gating) |
| Push to `feature/*` | Feature branches | None (PR required) |
| Push to `release/*` | Release prep branches | Selected validation |
### Tag Patterns
| Pattern | Example | Workflow |
|---------|---------|----------|
| `v*` | `v2025.12.1` | `release.yml` |
| `suite-*` | `suite-2026.04` | `release-suite.yml` |
| `module-*-v*` | `module-authority-v1.2.3` | `module-publish.yml` |
### Schedule Summary
| Time (UTC) | Frequency | Workflow |
|------------|-----------|----------|
| 2:00 AM | Daily | `nightly-regression.yml` |
| 2:00 AM | Sunday | `dependency-security-scan.yml` |
| 3:00 AM | Daily | `renovate.yml` |
| 3:30 AM | Monday | `sast-scan.yml` |
| 4:30 AM | Daily | `migration-test.yml` |
| 5:00 AM | Daily | `build-test-deploy.yml`, `test-matrix.yml` |
| 3:00 PM | Daily | `renovate.yml` |
---
## Environment Flow
```
┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐
│ PR │───▶│ Staging │───▶│ Stable │───▶│ LTS │
│ (Preview)│ │ (Edge) │ │ (Tested) │ │(Long-Term)│
└──────────┘ └──────────┘ └──────────┘ └──────────┘
│ │ │ │
│ │ │ │
▼ ▼ ▼ ▼
PR tests Auto-deploy promote.yml promote.yml
(gating) on main merge (manual) (manual)
```
### Environment Matrix
| Environment | Branch/Tag | Auto-Deploy | Rollback |
|-------------|------------|-------------|----------|
| Preview | PR | Yes (ephemeral) | N/A |
| Staging (Edge) | `main` | Yes | `rollback.yml` |
| Stable | `v*` tags | Manual | `rollback.yml` |
| LTS | `suite-*` tags | Manual | `rollback.yml` |
---
## Key Features
### 1. PR-Gating Tests
Required tests that must pass before merge:
- **Unit Tests** - Fast, isolated tests
- **Architecture Tests** - Dependency rule enforcement
- **Contract Tests** - API compatibility
- **Integration Tests** - PostgreSQL integration
- **Security Tests** - Security-focused assertions
- **Golden Tests** - Corpus-based validation
### 2. Determinism Verification
All builds produce identical outputs:
- Binary checksums compared across runs
- UTC timezone enforcement (`TZ: UTC`)
- Stable JSON serialization
- Reproducible SBOM generation
### 3. Supply Chain Security
- **SBOM Generation** - Syft for CycloneDX/SPDX
- **Artifact Signing** - Cosign/Sigstore integration
- **Provenance** - in-toto/DSSE attestations
- **Dependency Scanning** - Automated vulnerability detection
### 4. Rollback Automation
Emergency rollback via `rollback.yml`:
- Target: < 5 minute SLA
- Helm-based deployment rollback
- Health check verification
- Notification integration
---
## Directory Structure
```
.gitea/
├── workflows/ # 100 workflow files
│ ├── build-test-deploy.yml
│ ├── test-matrix.yml
│ ├── release-suite.yml
│ └── ...
├── scripts/ # CI/CD scripts
│ ├── build/ # Build orchestration
│ ├── test/ # Test execution
│ ├── release/ # Release automation
│ ├── sign/ # Signing operations
│ └── validate/ # Validation scripts
└── docs/ # CI-specific docs
├── architecture.md
├── scripts.md
└── troubleshooting.md
devops/
├── scripts/
│ └── lib/ # Shared bash libraries
│ ├── logging.sh
│ ├── exit-codes.sh
│ ├── git-utils.sh
│ ├── path-utils.sh
│ └── hash-utils.sh
├── compose/ # Docker Compose profiles
├── helm/ # Helm charts
└── docker/ # Dockerfiles
```
---
## Getting Started
### Running Workflows Locally
```bash
# Run test matrix locally
./devops/scripts/test-local.sh
# Validate compose files
./devops/scripts/validate-compose.sh
# Run a specific test category
./.gitea/scripts/test/run-test-category.sh Unit
```
### Triggering Manual Workflows
```bash
# Via Gitea UI: Actions → Workflow → Run workflow
# Or via API:
curl -X POST \
-H "Authorization: token $GITEA_TOKEN" \
"$GITEA_URL/api/v1/repos/owner/repo/actions/workflows/rollback.yml/dispatches" \
-d '{"ref":"main","inputs":{"environment":"staging","version":"v2025.12.0"}}'
```
### Creating a Release
1. **Module Release:**
```bash
git tag module-authority-v1.2.3
git push origin module-authority-v1.2.3
```
2. **Suite Release:**
```bash
git tag suite-2026.04
git push origin suite-2026.04
```
3. **Bundle Release:**
```bash
git tag v2025.12.1
git push origin v2025.12.1
```
---
## Related Documentation
- [Workflow Triggers Deep Dive](./workflow-triggers.md)
- [Release Pipeline Details](./release-pipelines.md)
- [Security Scanning Guide](./security-scanning.md)
- [Test Strategy](./test-strategy.md)
- [CI Quality Gates](../testing/ci-quality-gates.md)
- [Troubleshooting](../.gitea/docs/troubleshooting.md)
- [Script Reference](../.gitea/docs/scripts.md)
---
## Metrics & Monitoring
### Key Metrics Tracked
| Metric | Target | Measurement |
|--------|--------|-------------|
| PR Build Time | < 15 min | Workflow duration |
| Main Build Time | < 20 min | Workflow duration |
| Test Flakiness | < 1% | Flaky test detection |
| Security Scan Coverage | 100% | SAST/DAST coverage |
| Rollback SLA | < 5 min | Rollback workflow duration |
### Dashboard Links
- [Workflow Runs](../../.gitea/workflows/) (Gitea Actions UI)
- [Test Results](./test-results/) (TRX/JUnit artifacts)
- [Coverage Reports](./coverage/) (Generated nightly)
Reference in New Issue View Git Blame Copy Permalink