737 lines
12 KiB
Markdown
737 lines
12 KiB
Markdown
# CI/CD Scripts Inventory
|
|
|
|
Complete documentation of all scripts in `.gitea/scripts/`.
|
|
|
|
## Directory Structure
|
|
|
|
```
|
|
.gitea/scripts/
|
|
├── build/ # Build orchestration
|
|
├── evidence/ # Evidence bundle management
|
|
├── metrics/ # Performance metrics
|
|
├── release/ # Release automation
|
|
├── sign/ # Artifact signing
|
|
├── test/ # Test execution
|
|
├── util/ # Utilities
|
|
└── validate/ # Validation scripts
|
|
```
|
|
|
|
## Exit Code Conventions
|
|
|
|
| Code | Meaning |
|
|
|------|---------|
|
|
| 0 | Success |
|
|
| 1 | General error |
|
|
| 2 | Missing configuration/key |
|
|
| 3 | Missing required file |
|
|
| 69 | Tool not found (EX_UNAVAILABLE) |
|
|
|
|
---
|
|
|
|
## Build Scripts (`scripts/build/`)
|
|
|
|
### build-cli.sh
|
|
|
|
Multi-platform CLI build with SBOM generation and signing.
|
|
|
|
**Usage:**
|
|
```bash
|
|
RIDS=linux-x64,win-x64,osx-arm64 ./build-cli.sh
|
|
```
|
|
|
|
**Environment Variables:**
|
|
|
|
| Variable | Default | Description |
|
|
|----------|---------|-------------|
|
|
| `RIDS` | `linux-x64,win-x64,osx-arm64` | Comma-separated runtime identifiers |
|
|
| `CONFIG` | `Release` | Build configuration |
|
|
| `SBOM_TOOL` | `syft` | SBOM generator (`syft` or `none`) |
|
|
| `SIGN` | `false` | Enable artifact signing |
|
|
| `COSIGN_KEY` | - | Path to Cosign key file |
|
|
|
|
**Output:**
|
|
```
|
|
out/cli/
|
|
├── linux-x64/
|
|
│ ├── publish/
|
|
│ ├── stella-cli-linux-x64.tar.gz
|
|
│ ├── stella-cli-linux-x64.tar.gz.sha256
|
|
│ └── stella-cli-linux-x64.tar.gz.sbom.json
|
|
├── win-x64/
|
|
│ ├── publish/
|
|
│ ├── stella-cli-win-x64.zip
|
|
│ └── ...
|
|
└── manifest.json
|
|
```
|
|
|
|
**Features:**
|
|
- Builds self-contained single-file executables
|
|
- Includes CLI plugins (Aoc, Symbols)
|
|
- Generates SHA-256 checksums
|
|
- Optional SBOM generation via Syft
|
|
- Optional Cosign signing
|
|
|
|
---
|
|
|
|
### build-multiarch.sh
|
|
|
|
Multi-architecture Docker image builds using buildx.
|
|
|
|
**Usage:**
|
|
```bash
|
|
IMAGE=scanner PLATFORMS=linux/amd64,linux/arm64 ./build-multiarch.sh
|
|
```
|
|
|
|
**Environment Variables:**
|
|
|
|
| Variable | Default | Description |
|
|
|----------|---------|-------------|
|
|
| `IMAGE` | - | Image name (required) |
|
|
| `PLATFORMS` | `linux/amd64,linux/arm64` | Target platforms |
|
|
| `REGISTRY` | `git.stella-ops.org` | Container registry |
|
|
| `TAG` | `latest` | Image tag |
|
|
| `PUSH` | `false` | Push to registry |
|
|
|
|
---
|
|
|
|
### build-airgap-bundle.sh
|
|
|
|
Build offline/air-gapped deployment bundle.
|
|
|
|
**Usage:**
|
|
```bash
|
|
VERSION=2026.04 ./build-airgap-bundle.sh
|
|
```
|
|
|
|
**Output:**
|
|
```
|
|
out/airgap/
|
|
├── images.tar # All container images
|
|
├── helm-charts.tar.gz # Helm charts
|
|
├── compose.tar.gz # Docker Compose files
|
|
├── checksums.txt
|
|
└── manifest.json
|
|
```
|
|
|
|
---
|
|
|
|
## Test Scripts (`scripts/test/`)
|
|
|
|
### determinism-run.sh
|
|
|
|
Run determinism verification tests.
|
|
|
|
**Usage:**
|
|
```bash
|
|
./determinism-run.sh
|
|
```
|
|
|
|
**Purpose:**
|
|
- Executes tests filtered by `Determinism` category
|
|
- Collects TRX test results
|
|
- Generates summary and artifacts archive
|
|
|
|
**Output:**
|
|
```
|
|
out/scanner-determinism/
|
|
├── determinism.trx
|
|
├── summary.txt
|
|
└── determinism-artifacts.tgz
|
|
```
|
|
|
|
---
|
|
|
|
### run-fixtures-check.sh
|
|
|
|
Validate test fixtures against expected schemas.
|
|
|
|
**Usage:**
|
|
```bash
|
|
./run-fixtures-check.sh [--update]
|
|
```
|
|
|
|
**Options:**
|
|
- `--update`: Update golden fixtures if mismatched
|
|
|
|
---
|
|
|
|
## Validation Scripts (`scripts/validate/`)
|
|
|
|
### validate-sbom.sh
|
|
|
|
Validate CycloneDX SBOM files.
|
|
|
|
**Usage:**
|
|
```bash
|
|
./validate-sbom.sh <sbom-file>
|
|
./validate-sbom.sh --all
|
|
./validate-sbom.sh --schema custom.json sample.json
|
|
```
|
|
|
|
**Options:**
|
|
|
|
| Option | Description |
|
|
|--------|-------------|
|
|
| `--all` | Validate all fixtures in `src/__Tests/__Benchmarks/golden-corpus/` |
|
|
| `--schema <path>` | Custom schema file |
|
|
|
|
**Dependencies:**
|
|
- `sbom-utility` (auto-installed if missing)
|
|
|
|
**Exit Codes:**
|
|
- `0`: All validations passed
|
|
- `1`: Validation failed
|
|
|
|
---
|
|
|
|
### validate-spdx.sh
|
|
|
|
Validate SPDX SBOM files.
|
|
|
|
**Usage:**
|
|
```bash
|
|
./validate-spdx.sh <spdx-file>
|
|
```
|
|
|
|
---
|
|
|
|
### validate-vex.sh
|
|
|
|
Validate VEX documents (OpenVEX, CSAF).
|
|
|
|
**Usage:**
|
|
```bash
|
|
./validate-vex.sh <vex-file>
|
|
```
|
|
|
|
---
|
|
|
|
### validate-helm.sh
|
|
|
|
Validate Helm charts.
|
|
|
|
**Usage:**
|
|
```bash
|
|
./validate-helm.sh [chart-path]
|
|
```
|
|
|
|
**Default Path:** `devops/helm/stellaops`
|
|
|
|
**Checks:**
|
|
- `helm lint`
|
|
- Template rendering
|
|
- Schema validation
|
|
|
|
---
|
|
|
|
### validate-compose.sh
|
|
|
|
Validate Docker Compose files.
|
|
|
|
**Usage:**
|
|
```bash
|
|
./validate-compose.sh [profile]
|
|
```
|
|
|
|
**Profiles:**
|
|
- `dev` - Development
|
|
- `stage` - Staging
|
|
- `prod` - Production
|
|
- `airgap` - Air-gapped
|
|
|
|
---
|
|
|
|
### validate-licenses.sh
|
|
|
|
Check dependency licenses for compliance.
|
|
|
|
**Usage:**
|
|
```bash
|
|
./validate-licenses.sh
|
|
```
|
|
|
|
**Checks:**
|
|
- NuGet packages via `dotnet-delice`
|
|
- npm packages via `license-checker`
|
|
- Reports blocked licenses (GPL-2.0-only, SSPL, etc.)
|
|
|
|
---
|
|
|
|
### validate-migrations.sh
|
|
|
|
Validate database migrations.
|
|
|
|
**Usage:**
|
|
```bash
|
|
./validate-migrations.sh
|
|
```
|
|
|
|
**Checks:**
|
|
- Migration naming conventions
|
|
- Forward/rollback pairs
|
|
- Idempotency
|
|
|
|
---
|
|
|
|
### validate-workflows.sh
|
|
|
|
Validate Gitea Actions workflow YAML files.
|
|
|
|
**Usage:**
|
|
```bash
|
|
./validate-workflows.sh
|
|
```
|
|
|
|
**Checks:**
|
|
- YAML syntax
|
|
- Required fields
|
|
- Action version pinning
|
|
|
|
---
|
|
|
|
### verify-binaries.sh
|
|
|
|
Verify binary integrity.
|
|
|
|
**Usage:**
|
|
```bash
|
|
./verify-binaries.sh <binary-path> [checksum-file]
|
|
```
|
|
|
|
---
|
|
|
|
## Signing Scripts (`scripts/sign/`)
|
|
|
|
### sign-signals.sh
|
|
|
|
Sign Signals artifacts with Cosign.
|
|
|
|
**Usage:**
|
|
```bash
|
|
./sign-signals.sh
|
|
```
|
|
|
|
**Environment Variables:**
|
|
|
|
| Variable | Description |
|
|
|----------|-------------|
|
|
| `COSIGN_KEY_FILE` | Path to signing key |
|
|
| `COSIGN_PRIVATE_KEY_B64` | Base64-encoded private key |
|
|
| `COSIGN_PASSWORD` | Key password |
|
|
| `COSIGN_ALLOW_DEV_KEY` | Allow development key (`1`) |
|
|
| `OUT_DIR` | Output directory |
|
|
|
|
**Key Resolution Order:**
|
|
1. `COSIGN_KEY_FILE` environment variable
|
|
2. `COSIGN_PRIVATE_KEY_B64` environment variable (decoded)
|
|
3. `tools/cosign/cosign.key`
|
|
4. `tools/cosign/cosign.dev.key` (if `COSIGN_ALLOW_DEV_KEY=1`)
|
|
|
|
**Signed Artifacts:**
|
|
- `confidence_decay_config.yaml`
|
|
- `unknowns_scoring_manifest.json`
|
|
- `heuristics.catalog.json`
|
|
|
|
**Output:**
|
|
```
|
|
evidence-locker/signals/{date}/
|
|
├── confidence_decay_config.sigstore.json
|
|
├── unknowns_scoring_manifest.sigstore.json
|
|
├── heuristics_catalog.sigstore.json
|
|
└── SHA256SUMS
|
|
```
|
|
|
|
---
|
|
|
|
### sign-policy.sh
|
|
|
|
Sign policy artifacts.
|
|
|
|
**Usage:**
|
|
```bash
|
|
./sign-policy.sh <policy-file>
|
|
```
|
|
|
|
---
|
|
|
|
### sign-authority-gaps.sh
|
|
|
|
Sign authority gap attestations.
|
|
|
|
**Usage:**
|
|
```bash
|
|
./sign-authority-gaps.sh
|
|
```
|
|
|
|
---
|
|
|
|
## Release Scripts (`scripts/release/`)
|
|
|
|
### build_release.py
|
|
|
|
Main release pipeline orchestration.
|
|
|
|
**Usage:**
|
|
```bash
|
|
python build_release.py --channel stable --version 2026.04
|
|
```
|
|
|
|
**Arguments:**
|
|
|
|
| Argument | Description |
|
|
|----------|-------------|
|
|
| `--channel` | Release channel (`stable`, `beta`, `nightly`) |
|
|
| `--version` | Version string |
|
|
| `--config` | Component config file |
|
|
| `--dry-run` | Don't push artifacts |
|
|
|
|
**Dependencies:**
|
|
- docker (with buildx)
|
|
- cosign
|
|
- helm
|
|
- npm/node
|
|
- dotnet SDK
|
|
|
|
---
|
|
|
|
### verify_release.py
|
|
|
|
Post-release verification.
|
|
|
|
**Usage:**
|
|
```bash
|
|
python verify_release.py --version 2026.04
|
|
```
|
|
|
|
---
|
|
|
|
### bump-service-version.py
|
|
|
|
Manage service versions in `Directory.Versions.props`.
|
|
|
|
**Usage:**
|
|
```bash
|
|
# Bump version
|
|
python bump-service-version.py --service scanner --bump minor
|
|
|
|
# Set explicit version
|
|
python bump-service-version.py --service scanner --version 2.0.0
|
|
|
|
# List versions
|
|
python bump-service-version.py --list
|
|
```
|
|
|
|
**Arguments:**
|
|
|
|
| Argument | Description |
|
|
|----------|-------------|
|
|
| `--service` | Service name (e.g., `scanner`, `authority`) |
|
|
| `--bump` | Bump type (`major`, `minor`, `patch`) |
|
|
| `--version` | Explicit version to set |
|
|
| `--list` | List all service versions |
|
|
| `--dry-run` | Don't write changes |
|
|
|
|
---
|
|
|
|
### read-service-version.sh
|
|
|
|
Read current service version.
|
|
|
|
**Usage:**
|
|
```bash
|
|
./read-service-version.sh scanner
|
|
```
|
|
|
|
**Output:**
|
|
```
|
|
1.2.3
|
|
```
|
|
|
|
---
|
|
|
|
### generate-docker-tag.sh
|
|
|
|
Generate Docker tag with datetime suffix.
|
|
|
|
**Usage:**
|
|
```bash
|
|
./generate-docker-tag.sh 1.2.3
|
|
```
|
|
|
|
**Output:**
|
|
```
|
|
1.2.3+20250128143022
|
|
```
|
|
|
|
---
|
|
|
|
### generate_changelog.py
|
|
|
|
AI-assisted changelog generation.
|
|
|
|
**Usage:**
|
|
```bash
|
|
python generate_changelog.py --version 2026.04 --codename Nova
|
|
```
|
|
|
|
**Environment Variables:**
|
|
|
|
| Variable | Description |
|
|
|----------|-------------|
|
|
| `AI_API_KEY` | AI service API key |
|
|
| `AI_API_URL` | AI service endpoint (optional) |
|
|
|
|
**Features:**
|
|
- Parses git commits since last release
|
|
- Categorizes by type (Breaking, Security, Features, Fixes)
|
|
- Groups by module
|
|
- AI-assisted summary generation
|
|
- Fallback to rule-based generation
|
|
|
|
---
|
|
|
|
### generate_suite_docs.py
|
|
|
|
Generate suite release documentation.
|
|
|
|
**Usage:**
|
|
```bash
|
|
python generate_suite_docs.py --version 2026.04 --codename Nova
|
|
```
|
|
|
|
**Output:**
|
|
```
|
|
docs/releases/2026.04/
|
|
├── README.md
|
|
├── CHANGELOG.md
|
|
├── services.md
|
|
├── upgrade-guide.md
|
|
├── checksums.txt
|
|
└── manifest.yaml
|
|
```
|
|
|
|
---
|
|
|
|
### generate_compose.py
|
|
|
|
Generate pinned Docker Compose files.
|
|
|
|
**Usage:**
|
|
```bash
|
|
python generate_compose.py --version 2026.04
|
|
```
|
|
|
|
**Output:**
|
|
- `docker-compose.yml` - Standard deployment
|
|
- `docker-compose.airgap.yml` - Air-gapped deployment
|
|
|
|
---
|
|
|
|
### collect_versions.py
|
|
|
|
Collect service versions from `Directory.Versions.props`.
|
|
|
|
**Usage:**
|
|
```bash
|
|
python collect_versions.py --format json
|
|
python collect_versions.py --format yaml
|
|
python collect_versions.py --format markdown
|
|
python collect_versions.py --format env
|
|
```
|
|
|
|
---
|
|
|
|
### check_cli_parity.py
|
|
|
|
Verify CLI version parity across platforms.
|
|
|
|
**Usage:**
|
|
```bash
|
|
python check_cli_parity.py
|
|
```
|
|
|
|
---
|
|
|
|
## Evidence Scripts (`scripts/evidence/`)
|
|
|
|
### upload-all-evidence.sh
|
|
|
|
Upload all evidence bundles to Evidence Locker.
|
|
|
|
**Usage:**
|
|
```bash
|
|
./upload-all-evidence.sh
|
|
```
|
|
|
|
---
|
|
|
|
### signals-upload-evidence.sh
|
|
|
|
Upload Signals evidence.
|
|
|
|
**Usage:**
|
|
```bash
|
|
./signals-upload-evidence.sh
|
|
```
|
|
|
|
---
|
|
|
|
### zastava-upload-evidence.sh
|
|
|
|
Upload Zastava evidence.
|
|
|
|
**Usage:**
|
|
```bash
|
|
./zastava-upload-evidence.sh
|
|
```
|
|
|
|
---
|
|
|
|
## Metrics Scripts (`scripts/metrics/`)
|
|
|
|
### compute-reachability-metrics.sh
|
|
|
|
Compute reachability analysis metrics.
|
|
|
|
**Usage:**
|
|
```bash
|
|
./compute-reachability-metrics.sh
|
|
```
|
|
|
|
**Output Metrics:**
|
|
- Total functions analyzed
|
|
- Reachable functions
|
|
- Coverage percentage
|
|
- Analysis duration
|
|
|
|
---
|
|
|
|
### compute-ttfs-metrics.sh
|
|
|
|
Compute Time-to-First-Scan metrics.
|
|
|
|
**Usage:**
|
|
```bash
|
|
./compute-ttfs-metrics.sh
|
|
```
|
|
|
|
---
|
|
|
|
### enforce-performance-slos.sh
|
|
|
|
Enforce performance SLOs.
|
|
|
|
**Usage:**
|
|
```bash
|
|
./enforce-performance-slos.sh
|
|
```
|
|
|
|
**Checked SLOs:**
|
|
- Build time < 30 minutes
|
|
- Test coverage > 80%
|
|
- TTFS < 60 seconds
|
|
|
|
---
|
|
|
|
## Utility Scripts (`scripts/util/`)
|
|
|
|
### cleanup-runner-space.sh
|
|
|
|
Clean up runner disk space.
|
|
|
|
**Usage:**
|
|
```bash
|
|
./cleanup-runner-space.sh
|
|
```
|
|
|
|
**Actions:**
|
|
- Remove Docker build cache
|
|
- Clean NuGet cache
|
|
- Remove old test results
|
|
- Prune unused images
|
|
|
|
---
|
|
|
|
### dotnet-filter.sh
|
|
|
|
Filter .NET projects for selective builds.
|
|
|
|
**Usage:**
|
|
```bash
|
|
./dotnet-filter.sh --changed
|
|
./dotnet-filter.sh --module Scanner
|
|
```
|
|
|
|
---
|
|
|
|
### enable-openssl11-shim.sh
|
|
|
|
Enable OpenSSL 1.1 compatibility shim.
|
|
|
|
**Usage:**
|
|
```bash
|
|
./enable-openssl11-shim.sh
|
|
```
|
|
|
|
**Purpose:**
|
|
Required for certain cryptographic operations on newer Linux distributions that have removed OpenSSL 1.1.
|
|
|
|
---
|
|
|
|
## Script Development Guidelines
|
|
|
|
### Required Elements
|
|
|
|
1. **Shebang:**
|
|
```bash
|
|
#!/usr/bin/env bash
|
|
```
|
|
|
|
2. **Strict Mode:**
|
|
```bash
|
|
set -euo pipefail
|
|
```
|
|
|
|
3. **Sprint Reference:**
|
|
```bash
|
|
# DEVOPS-XXX-YY-ZZZ: Description
|
|
# Sprint: SPRINT_XXXX_XXXX_XXXX - Topic
|
|
```
|
|
|
|
4. **Usage Documentation:**
|
|
```bash
|
|
# Usage:
|
|
# ./script.sh <required-arg> [optional-arg]
|
|
```
|
|
|
|
### Best Practices
|
|
|
|
1. **Use environment variables with defaults:**
|
|
```bash
|
|
CONFIG="${CONFIG:-Release}"
|
|
```
|
|
|
|
2. **Validate required tools:**
|
|
```bash
|
|
if ! command -v dotnet >/dev/null 2>&1; then
|
|
echo "dotnet CLI not found" >&2
|
|
exit 69
|
|
fi
|
|
```
|
|
|
|
3. **Use absolute paths:**
|
|
```bash
|
|
ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
|
|
```
|
|
|
|
4. **Handle cleanup:**
|
|
```bash
|
|
trap 'rm -f "$TMP_FILE"' EXIT
|
|
```
|
|
|
|
5. **Use logging functions:**
|
|
```bash
|
|
log_info() { echo "[INFO] $*"; }
|
|
log_error() { echo "[ERROR] $*" >&2; }
|
|
```
|