429 lines
8.9 KiB
Markdown
429 lines
8.9 KiB
Markdown
# Local CI Testing Guide
|
|
|
|
> Run CI/CD pipelines locally before pushing to verify your changes.
|
|
|
|
---
|
|
|
|
## Quick Start
|
|
|
|
### Prerequisites
|
|
|
|
- **Docker Desktop** (Windows/macOS) or Docker Engine (Linux)
|
|
- **.NET 10 SDK** ([download](https://dot.net/download))
|
|
- **Git**
|
|
- **Bash** (Linux/macOS native, Windows via WSL2 or Git Bash)
|
|
- **act** (optional, for workflow simulation) - [install](https://github.com/nektos/act)
|
|
|
|
### Installation
|
|
|
|
1. **Copy environment template:**
|
|
```bash
|
|
cp devops/ci-local/.env.local.sample devops/ci-local/.env.local
|
|
```
|
|
|
|
2. **(Optional) Install act for workflow simulation:**
|
|
```bash
|
|
# macOS
|
|
brew install act
|
|
|
|
# Linux
|
|
curl -sSL https://raw.githubusercontent.com/nektos/act/master/install.sh | sudo bash
|
|
|
|
# Windows (via Chocolatey)
|
|
choco install act-cli
|
|
```
|
|
|
|
3. **(Optional) Build CI Docker image:**
|
|
```bash
|
|
docker build -t stellaops-ci:local -f devops/docker/Dockerfile.ci .
|
|
```
|
|
|
|
---
|
|
|
|
## Usage
|
|
|
|
### Basic Commands
|
|
|
|
```bash
|
|
# Quick smoke test (~2 min)
|
|
./devops/scripts/local-ci.sh smoke
|
|
|
|
# Full PR-gating suite (~15 min)
|
|
./devops/scripts/local-ci.sh pr
|
|
|
|
# Test specific module
|
|
./devops/scripts/local-ci.sh module --module Scanner
|
|
|
|
# Auto-detect changed modules
|
|
./devops/scripts/local-ci.sh module
|
|
|
|
# Simulate workflow
|
|
./devops/scripts/local-ci.sh workflow --workflow test-matrix
|
|
|
|
# Release dry-run
|
|
./devops/scripts/local-ci.sh release --dry-run
|
|
|
|
# Full test suite (~45 min)
|
|
./devops/scripts/local-ci.sh full
|
|
```
|
|
|
|
### Windows (PowerShell)
|
|
|
|
```powershell
|
|
# Quick smoke test
|
|
.\devops\scripts\local-ci.ps1 smoke
|
|
|
|
# Full PR check
|
|
.\devops\scripts\local-ci.ps1 pr
|
|
|
|
# With options
|
|
.\devops\scripts\local-ci.ps1 pr -Verbose -Docker
|
|
```
|
|
|
|
---
|
|
|
|
## Modes
|
|
|
|
### smoke (~2 minutes)
|
|
Quick validation before pushing. Runs only Unit tests.
|
|
|
|
```bash
|
|
./devops/scripts/local-ci.sh smoke
|
|
```
|
|
|
|
**What it does:**
|
|
1. Builds the solution
|
|
2. Runs Unit tests
|
|
3. Reports pass/fail
|
|
|
|
### pr (~15 minutes)
|
|
Full PR-gating suite matching CI exactly. Run this before opening a PR.
|
|
|
|
```bash
|
|
./devops/scripts/local-ci.sh pr
|
|
```
|
|
|
|
**What it does:**
|
|
1. Starts CI services (PostgreSQL, Valkey)
|
|
2. Builds with warnings-as-errors
|
|
3. Runs all PR-gating categories:
|
|
- Unit
|
|
- Architecture
|
|
- Contract
|
|
- Integration
|
|
- Security
|
|
- Golden
|
|
4. Generates TRX reports
|
|
5. Stops CI services
|
|
|
|
### module
|
|
Test only the modules you've changed. Detects changes via git diff.
|
|
|
|
```bash
|
|
# Auto-detect changed modules
|
|
./devops/scripts/local-ci.sh module
|
|
|
|
# Test specific module
|
|
./devops/scripts/local-ci.sh module --module Scanner
|
|
./devops/scripts/local-ci.sh module --module Concelier
|
|
./devops/scripts/local-ci.sh module --module Authority
|
|
```
|
|
|
|
**Available modules:**
|
|
Scanner, Concelier, Authority, Policy, Attestor, EvidenceLocker, ExportCenter,
|
|
Findings, SbomService, Notify, Router, Cryptography, AirGap, Cli, AdvisoryAI,
|
|
ReachGraph, Orchestrator, PacksRegistry, Replay, Aoc, IssuerDirectory, Telemetry, Signals
|
|
|
|
### workflow
|
|
Simulate a specific Gitea Actions workflow using `act`.
|
|
|
|
```bash
|
|
# Simulate test-matrix workflow
|
|
./devops/scripts/local-ci.sh workflow --workflow test-matrix
|
|
|
|
# Simulate build-test-deploy
|
|
./devops/scripts/local-ci.sh workflow --workflow build-test-deploy
|
|
```
|
|
|
|
**Requirements:**
|
|
- `act` must be installed
|
|
- CI Docker image built (`stellaops-ci:local`)
|
|
|
|
### release
|
|
Dry-run release pipeline to verify release artifacts.
|
|
|
|
```bash
|
|
./devops/scripts/local-ci.sh release --dry-run
|
|
```
|
|
|
|
**What it does:**
|
|
1. Builds all modules
|
|
2. Validates CLI packaging
|
|
3. Validates Helm chart
|
|
4. Shows what would be released
|
|
5. **Does NOT publish anything**
|
|
|
|
### full (~45 minutes)
|
|
Complete test suite including extended categories.
|
|
|
|
```bash
|
|
./devops/scripts/local-ci.sh full
|
|
```
|
|
|
|
**Categories run:**
|
|
- PR-Gating: Unit, Architecture, Contract, Integration, Security, Golden
|
|
- Extended: Performance, Benchmark, AirGap, Chaos, Determinism, Resilience, Observability
|
|
|
|
---
|
|
|
|
## Options
|
|
|
|
| Option | Description |
|
|
|--------|-------------|
|
|
| `--category <cat>` | Run specific test category |
|
|
| `--module <name>` | Test specific module |
|
|
| `--workflow <name>` | Workflow to simulate |
|
|
| `--docker` | Force Docker execution |
|
|
| `--native` | Force native execution |
|
|
| `--act` | Force act execution |
|
|
| `--parallel <n>` | Parallel test runners |
|
|
| `--verbose` | Verbose output |
|
|
| `--dry-run` | Show what would run |
|
|
| `--rebuild` | Force rebuild CI image |
|
|
| `--no-services` | Skip CI services |
|
|
| `--keep-services` | Keep services running after tests |
|
|
|
|
---
|
|
|
|
## Test Categories
|
|
|
|
### PR-Gating (Required for merge)
|
|
|
|
| Category | Purpose | Time |
|
|
|----------|---------|------|
|
|
| **Unit** | Component isolation tests | ~3 min |
|
|
| **Architecture** | Dependency rules | ~2 min |
|
|
| **Contract** | API compatibility | ~2 min |
|
|
| **Integration** | Database/service tests | ~8 min |
|
|
| **Security** | Security assertions | ~3 min |
|
|
| **Golden** | Corpus-based validation | ~3 min |
|
|
|
|
### Extended (On-demand)
|
|
|
|
| Category | Purpose | Time |
|
|
|----------|---------|------|
|
|
| **Performance** | Latency/throughput | ~20 min |
|
|
| **Benchmark** | BenchmarkDotNet | ~30 min |
|
|
| **Determinism** | Reproducibility | ~15 min |
|
|
| **AirGap** | Offline operation | ~15 min |
|
|
| **Chaos** | Resilience testing | ~20 min |
|
|
|
|
---
|
|
|
|
## CI Services
|
|
|
|
The local CI uses Docker Compose to run required services.
|
|
|
|
### Services Started
|
|
|
|
| Service | Port | Purpose |
|
|
|---------|------|---------|
|
|
| postgres-ci | 5433 | PostgreSQL 16 for tests |
|
|
| valkey-ci | 6380 | Cache/messaging tests |
|
|
| nats-ci | 4223 | Message queue tests |
|
|
| mock-registry | 5001 | Container registry |
|
|
| minio-ci | 9100 | S3-compatible storage |
|
|
|
|
### Manual Service Management
|
|
|
|
```bash
|
|
# Start services
|
|
docker compose -f devops/compose/docker-compose.ci.yaml up -d
|
|
|
|
# Check status
|
|
docker compose -f devops/compose/docker-compose.ci.yaml ps
|
|
|
|
# View logs
|
|
docker compose -f devops/compose/docker-compose.ci.yaml logs postgres-ci
|
|
|
|
# Stop services
|
|
docker compose -f devops/compose/docker-compose.ci.yaml down
|
|
|
|
# Stop and remove volumes
|
|
docker compose -f devops/compose/docker-compose.ci.yaml down -v
|
|
```
|
|
|
|
---
|
|
|
|
## Configuration
|
|
|
|
### Environment Variables
|
|
|
|
Copy the template and customize:
|
|
|
|
```bash
|
|
cp devops/ci-local/.env.local.sample devops/ci-local/.env.local
|
|
```
|
|
|
|
Key variables:
|
|
|
|
```bash
|
|
# Database
|
|
STELLAOPS_TEST_POSTGRES_CONNECTION="Host=localhost;Port=5433;..."
|
|
|
|
# Cache
|
|
VALKEY_CONNECTION_STRING="localhost:6380"
|
|
|
|
# Execution
|
|
LOCAL_CI_MODE=docker # docker, native, act
|
|
LOCAL_CI_PARALLEL=4 # Parallel jobs
|
|
LOCAL_CI_VERBOSE=false # Verbose output
|
|
```
|
|
|
|
### Act Configuration
|
|
|
|
The `.actrc` file configures `act` for workflow simulation:
|
|
|
|
```ini
|
|
--platform ubuntu-22.04=stellaops-ci:local
|
|
--env-file devops/ci-local/.env.local
|
|
--bind
|
|
--reuse
|
|
```
|
|
|
|
---
|
|
|
|
## Troubleshooting
|
|
|
|
### Docker Issues
|
|
|
|
```bash
|
|
# Reset CI services
|
|
docker compose -f devops/compose/docker-compose.ci.yaml down -v
|
|
|
|
# Rebuild CI image
|
|
docker build --no-cache -t stellaops-ci:local -f devops/docker/Dockerfile.ci .
|
|
|
|
# Check Docker is running
|
|
docker info
|
|
```
|
|
|
|
### Test Failures
|
|
|
|
```bash
|
|
# Run with verbose output
|
|
./devops/scripts/local-ci.sh pr --verbose
|
|
|
|
# Run specific category
|
|
./devops/scripts/local-ci.sh pr --category Unit
|
|
|
|
# Check logs
|
|
cat out/local-ci/logs/Unit-*.log
|
|
```
|
|
|
|
### Act Issues
|
|
|
|
```bash
|
|
# List available workflows
|
|
act -l
|
|
|
|
# Dry run workflow
|
|
act -n pull_request -W .gitea/workflows/test-matrix.yml
|
|
|
|
# Debug mode
|
|
act --verbose pull_request
|
|
```
|
|
|
|
### Windows Issues
|
|
|
|
```powershell
|
|
# Check WSL is installed
|
|
wsl --status
|
|
|
|
# Install WSL if needed
|
|
wsl --install
|
|
|
|
# Check Git Bash
|
|
& "C:\Program Files\Git\bin\bash.exe" --version
|
|
```
|
|
|
|
---
|
|
|
|
## Results
|
|
|
|
Test results are saved to `out/local-ci/`:
|
|
|
|
```
|
|
out/local-ci/
|
|
├── trx/ # TRX test reports
|
|
│ ├── Unit-20250128_120000.trx
|
|
│ ├── Integration-20250128_120000.trx
|
|
│ └── ...
|
|
└── logs/ # Execution logs
|
|
├── Unit-20250128_120000.log
|
|
└── ...
|
|
```
|
|
|
|
### Viewing Results
|
|
|
|
```bash
|
|
# List TRX files
|
|
ls -la out/local-ci/trx/
|
|
|
|
# View test log
|
|
cat out/local-ci/logs/Unit-*.log | less
|
|
```
|
|
|
|
---
|
|
|
|
## Examples
|
|
|
|
### Pre-Push Workflow
|
|
|
|
```bash
|
|
# 1. Quick validation
|
|
./devops/scripts/local-ci.sh smoke
|
|
|
|
# 2. If smoke passes, run full PR check
|
|
./devops/scripts/local-ci.sh pr
|
|
|
|
# 3. Push your changes
|
|
git push origin feature/my-branch
|
|
```
|
|
|
|
### Module Development
|
|
|
|
```bash
|
|
# 1. Make changes to Scanner module
|
|
|
|
# 2. Run module-specific tests
|
|
./devops/scripts/local-ci.sh module --module Scanner
|
|
|
|
# 3. If passes, run full PR check before PR
|
|
./devops/scripts/local-ci.sh pr
|
|
```
|
|
|
|
### Debugging Test Failures
|
|
|
|
```bash
|
|
# 1. Run with verbose output
|
|
./devops/scripts/local-ci.sh pr --verbose --category Unit
|
|
|
|
# 2. Check the log
|
|
cat out/local-ci/logs/Unit-*.log
|
|
|
|
# 3. Run specific test directly
|
|
dotnet test src/Scanner/__Tests/StellaOps.Scanner.Tests/StellaOps.Scanner.Tests.csproj \
|
|
--filter "Category=Unit" \
|
|
--verbosity detailed
|
|
```
|
|
|
|
---
|
|
|
|
## Related Documentation
|
|
|
|
- [CI/CD Overview](../cicd/README.md)
|
|
- [Test Strategy](../cicd/test-strategy.md)
|
|
- [Workflow Triggers](../cicd/workflow-triggers.md)
|
|
- [Path Filters](../cicd/path-filters.md)
|