Files

StellaOps Bot c2b9cd8d1f Fix build and code structure improvements. New but essential UI functionality. CI improvements. Documentation improvements. AI module improvements.

2025-12-29 07:45:03 +02:00

12 KiB

Raw Blame History

CI/CD Scripts Inventory

Complete documentation of all scripts in .gitea/scripts/.

Directory Structure

.gitea/scripts/
├── build/           # Build orchestration
├── evidence/        # Evidence bundle management
├── metrics/         # Performance metrics
├── release/         # Release automation
├── sign/            # Artifact signing
├── test/            # Test execution
├── util/            # Utilities
└── validate/        # Validation scripts

Exit Code Conventions

Code	Meaning
0	Success
1	General error
2	Missing configuration/key
3	Missing required file
69	Tool not found (EX_UNAVAILABLE)

Build Scripts (`scripts/build/`)

build-cli.sh

Multi-platform CLI build with SBOM generation and signing.

Usage:

RIDS=linux-x64,win-x64,osx-arm64 ./build-cli.sh

Environment Variables:

Variable	Default	Description
`RIDS`	`linux-x64,win-x64,osx-arm64`	Comma-separated runtime identifiers
`CONFIG`	`Release`	Build configuration
`SBOM_TOOL`	`syft`	SBOM generator (`syft` or `none`)
`SIGN`	`false`	Enable artifact signing
`COSIGN_KEY`	-	Path to Cosign key file

Output:

out/cli/
├── linux-x64/
│   ├── publish/
│   ├── stella-cli-linux-x64.tar.gz
│   ├── stella-cli-linux-x64.tar.gz.sha256
│   └── stella-cli-linux-x64.tar.gz.sbom.json
├── win-x64/
│   ├── publish/
│   ├── stella-cli-win-x64.zip
│   └── ...
└── manifest.json

Features:

Builds self-contained single-file executables
Includes CLI plugins (Aoc, Symbols)
Generates SHA-256 checksums
Optional SBOM generation via Syft
Optional Cosign signing

build-multiarch.sh

Multi-architecture Docker image builds using buildx.

Usage:

IMAGE=scanner PLATFORMS=linux/amd64,linux/arm64 ./build-multiarch.sh

Environment Variables:

Variable	Default	Description
`IMAGE`	-	Image name (required)
`PLATFORMS`	`linux/amd64,linux/arm64`	Target platforms
`REGISTRY`	`git.stella-ops.org`	Container registry
`TAG`	`latest`	Image tag
`PUSH`	`false`	Push to registry

build-airgap-bundle.sh

Build offline/air-gapped deployment bundle.

Usage:

VERSION=2026.04 ./build-airgap-bundle.sh

Output:

out/airgap/
├── images.tar          # All container images
├── helm-charts.tar.gz  # Helm charts
├── compose.tar.gz      # Docker Compose files
├── checksums.txt
└── manifest.json

Test Scripts (`scripts/test/`)

determinism-run.sh

Run determinism verification tests.

Usage:

./determinism-run.sh

Purpose:

Executes tests filtered by Determinism category
Collects TRX test results
Generates summary and artifacts archive

Output:

out/scanner-determinism/
├── determinism.trx
├── summary.txt
└── determinism-artifacts.tgz

run-fixtures-check.sh

Validate test fixtures against expected schemas.

Usage:

./run-fixtures-check.sh [--update]

Options:

--update: Update golden fixtures if mismatched

Validation Scripts (`scripts/validate/`)

validate-sbom.sh

Validate CycloneDX SBOM files.

Usage:

./validate-sbom.sh <sbom-file>
./validate-sbom.sh --all
./validate-sbom.sh --schema custom.json sample.json

Options:

Option	Description
`--all`	Validate all fixtures in `src/__Tests/__Benchmarks/golden-corpus/`
`--schema <path>`	Custom schema file

Dependencies:

sbom-utility (auto-installed if missing)

Exit Codes:

0: All validations passed
1: Validation failed

validate-spdx.sh

Validate SPDX SBOM files.

Usage:

./validate-spdx.sh <spdx-file>

validate-vex.sh

Validate VEX documents (OpenVEX, CSAF).

Usage:

./validate-vex.sh <vex-file>

validate-helm.sh

Validate Helm charts.

Usage:

./validate-helm.sh [chart-path]

Default Path: devops/helm/stellaops

Checks:

helm lint
Template rendering
Schema validation

validate-compose.sh

Validate Docker Compose files.

Usage:

./validate-compose.sh [profile]

Profiles:

dev - Development
stage - Staging
prod - Production
airgap - Air-gapped

validate-licenses.sh

Check dependency licenses for compliance.

Usage:

./validate-licenses.sh

Checks:

NuGet packages via dotnet-delice
npm packages via license-checker
Reports blocked licenses (GPL-2.0-only, SSPL, etc.)

validate-migrations.sh

Validate database migrations.

Usage:

./validate-migrations.sh

Checks:

Migration naming conventions
Forward/rollback pairs
Idempotency

validate-workflows.sh

Validate Gitea Actions workflow YAML files.

Usage:

./validate-workflows.sh

Checks:

YAML syntax
Required fields
Action version pinning

verify-binaries.sh

Verify binary integrity.

Usage:

./verify-binaries.sh <binary-path> [checksum-file]

Signing Scripts (`scripts/sign/`)

sign-signals.sh

Sign Signals artifacts with Cosign.

Usage:

./sign-signals.sh

Environment Variables:

Variable	Description
`COSIGN_KEY_FILE`	Path to signing key
`COSIGN_PRIVATE_KEY_B64`	Base64-encoded private key
`COSIGN_PASSWORD`	Key password
`COSIGN_ALLOW_DEV_KEY`	Allow development key (`1`)
`OUT_DIR`	Output directory

Key Resolution Order:

COSIGN_KEY_FILE environment variable
COSIGN_PRIVATE_KEY_B64 environment variable (decoded)
tools/cosign/cosign.key
tools/cosign/cosign.dev.key (if COSIGN_ALLOW_DEV_KEY=1)

Signed Artifacts:

confidence_decay_config.yaml
unknowns_scoring_manifest.json
heuristics.catalog.json

Output:

evidence-locker/signals/{date}/
├── confidence_decay_config.sigstore.json
├── unknowns_scoring_manifest.sigstore.json
├── heuristics_catalog.sigstore.json
└── SHA256SUMS

sign-policy.sh

Sign policy artifacts.

Usage:

./sign-policy.sh <policy-file>

sign-authority-gaps.sh

Sign authority gap attestations.

Usage:

./sign-authority-gaps.sh

Release Scripts (`scripts/release/`)

build_release.py

Main release pipeline orchestration.

Usage:

python build_release.py --channel stable --version 2026.04

Arguments:

Argument	Description
`--channel`	Release channel (`stable`, `beta`, `nightly`)
`--version`	Version string
`--config`	Component config file
`--dry-run`	Don't push artifacts

Dependencies:

docker (with buildx)
cosign
helm
npm/node
dotnet SDK

verify_release.py

Post-release verification.

Usage:

python verify_release.py --version 2026.04

bump-service-version.py

Manage service versions in Directory.Versions.props.

Usage:

# Bump version
python bump-service-version.py --service scanner --bump minor

# Set explicit version
python bump-service-version.py --service scanner --version 2.0.0

# List versions
python bump-service-version.py --list

Arguments:

Argument	Description
`--service`	Service name (e.g., `scanner`, `authority`)
`--bump`	Bump type (`major`, `minor`, `patch`)
`--version`	Explicit version to set
`--list`	List all service versions
`--dry-run`	Don't write changes

read-service-version.sh

Read current service version.

Usage:

./read-service-version.sh scanner

Output:

1.2.3

generate-docker-tag.sh

Generate Docker tag with datetime suffix.

Usage:

./generate-docker-tag.sh 1.2.3

Output:

1.2.3+20250128143022

generate_changelog.py

AI-assisted changelog generation.

Usage:

python generate_changelog.py --version 2026.04 --codename Nova

Environment Variables:

Variable	Description
`AI_API_KEY`	AI service API key
`AI_API_URL`	AI service endpoint (optional)

Features:

Parses git commits since last release
Categorizes by type (Breaking, Security, Features, Fixes)
Groups by module
AI-assisted summary generation
Fallback to rule-based generation

generate_suite_docs.py

Generate suite release documentation.

Usage:

python generate_suite_docs.py --version 2026.04 --codename Nova

Output:

docs/releases/2026.04/
├── README.md
├── CHANGELOG.md
├── services.md
├── upgrade-guide.md
├── checksums.txt
└── manifest.yaml

generate_compose.py

Generate pinned Docker Compose files.

Usage:

python generate_compose.py --version 2026.04

Output:

docker-compose.yml - Standard deployment
docker-compose.airgap.yml - Air-gapped deployment

collect_versions.py

Collect service versions from Directory.Versions.props.

Usage:

python collect_versions.py --format json
python collect_versions.py --format yaml
python collect_versions.py --format markdown
python collect_versions.py --format env

check_cli_parity.py

Verify CLI version parity across platforms.

Usage:

python check_cli_parity.py

Evidence Scripts (`scripts/evidence/`)

upload-all-evidence.sh

Upload all evidence bundles to Evidence Locker.

Usage:

./upload-all-evidence.sh

signals-upload-evidence.sh

Upload Signals evidence.

Usage:

./signals-upload-evidence.sh

zastava-upload-evidence.sh

Upload Zastava evidence.

Usage:

./zastava-upload-evidence.sh

Metrics Scripts (`scripts/metrics/`)

compute-reachability-metrics.sh

Compute reachability analysis metrics.

Usage:

./compute-reachability-metrics.sh

Output Metrics:

Total functions analyzed
Reachable functions
Coverage percentage
Analysis duration

compute-ttfs-metrics.sh

Compute Time-to-First-Scan metrics.

Usage:

./compute-ttfs-metrics.sh

enforce-performance-slos.sh

Enforce performance SLOs.

Usage:

./enforce-performance-slos.sh

Checked SLOs:

Build time < 30 minutes
Test coverage > 80%
TTFS < 60 seconds

Utility Scripts (`scripts/util/`)

cleanup-runner-space.sh

Clean up runner disk space.

Usage:

./cleanup-runner-space.sh

Actions:

Remove Docker build cache
Clean NuGet cache
Remove old test results
Prune unused images

dotnet-filter.sh

Filter .NET projects for selective builds.

Usage:

./dotnet-filter.sh --changed
./dotnet-filter.sh --module Scanner

enable-openssl11-shim.sh

Enable OpenSSL 1.1 compatibility shim.

Usage:

./enable-openssl11-shim.sh

Purpose: Required for certain cryptographic operations on newer Linux distributions that have removed OpenSSL 1.1.

Script Development Guidelines

Required Elements

Shebang:
```
#!/usr/bin/env bash
```
Strict Mode:
```
set -euo pipefail
```

Sprint Reference:

# DEVOPS-XXX-YY-ZZZ: Description
# Sprint: SPRINT_XXXX_XXXX_XXXX - Topic

Usage Documentation:

# Usage:
#   ./script.sh <required-arg> [optional-arg]

Best Practices

Use environment variables with defaults:
```
CONFIG="${CONFIG:-Release}"
```

Validate required tools:

if ! command -v dotnet >/dev/null 2>&1; then
  echo "dotnet CLI not found" >&2
  exit 69
fi

Use absolute paths:

ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"

Handle cleanup:
```
trap 'rm -f "$TMP_FILE"' EXIT
```

Use logging functions:

log_info() { echo "[INFO] $*"; }
log_error() { echo "[ERROR] $*" >&2; }

12 KiB Raw Blame History

CI/CD Scripts Inventory

Directory Structure

Exit Code Conventions

Build Scripts (scripts/build/)

build-cli.sh

build-multiarch.sh

build-airgap-bundle.sh

Test Scripts (scripts/test/)

determinism-run.sh

run-fixtures-check.sh

Validation Scripts (scripts/validate/)

validate-sbom.sh

validate-spdx.sh

validate-vex.sh

validate-helm.sh

validate-compose.sh

validate-licenses.sh

validate-migrations.sh

validate-workflows.sh

verify-binaries.sh

Signing Scripts (scripts/sign/)

sign-signals.sh

sign-policy.sh

sign-authority-gaps.sh

Release Scripts (scripts/release/)

build_release.py

verify_release.py

bump-service-version.py

read-service-version.sh

generate-docker-tag.sh

generate_changelog.py

generate_suite_docs.py

generate_compose.py

collect_versions.py

check_cli_parity.py

Evidence Scripts (scripts/evidence/)

upload-all-evidence.sh

signals-upload-evidence.sh

zastava-upload-evidence.sh

Metrics Scripts (scripts/metrics/)

compute-reachability-metrics.sh

compute-ttfs-metrics.sh

enforce-performance-slos.sh

Utility Scripts (scripts/util/)

cleanup-runner-space.sh

dotnet-filter.sh

enable-openssl11-shim.sh

Script Development Guidelines

Required Elements

Best Practices

12 KiB

Raw Blame History

Build Scripts (`scripts/build/`)

Test Scripts (`scripts/test/`)

Validation Scripts (`scripts/validate/`)

Signing Scripts (`scripts/sign/`)

Release Scripts (`scripts/release/`)

Evidence Scripts (`scripts/evidence/`)

Metrics Scripts (`scripts/metrics/`)

Utility Scripts (`scripts/util/`)