git.stella-ops.org/docs/releases/offline-verification.md

Offline Verification Guide

This guide explains how to verify Stella Ops releases in air-gapped or offline environments without network access.

Overview

The Release Evidence Pack is designed for complete offline verification. All cryptographic materials and proofs are bundled together, allowing verification without contacting external services.

Verification Levels

Stella Ops supports multiple verification levels depending on your security requirements and available tools:

Level	Tools Required	Network	Security Assurance
1 - Checksum	sha256sum	None	Artifact integrity
2 - Signature	sha256sum + cosign	None	Authenticity + integrity
3 - Provenance	sha256sum + cosign + jq	None	Build chain verification
4 - Transparency	sha256sum + cosign + rekor-cli	Optional	Non-repudiation

Prerequisites

Minimal (Level 1)

Standard Unix tools available on most systems:

• sha256sum or shasum
• cat, diff

Full Verification (Levels 2-4)

Install cosign for signature verification:

# Linux
curl -sSL https://github.com/sigstore/cosign/releases/latest/download/cosign-linux-amd64 -o cosign
chmod +x cosign
sudo mv cosign /usr/local/bin/

# macOS
brew install cosign

# Windows (PowerShell)
scoop install cosign
# or download from GitHub releases

Quick Start

Using the Verification Script

The evidence pack includes a self-contained verification script:

# Extract the evidence pack
tar -xzf stella-release-2.5.0-evidence-pack.tgz
cd stella-release-2.5.0-evidence-pack

# Run verification
./verify.sh

# For verbose output
./verify.sh --verbose

# For JSON output (CI integration)
./verify.sh --json

On Windows (PowerShell 7+):

# Extract
Expand-Archive stella-release-2.5.0-evidence-pack.zip -DestinationPath .
cd stella-release-2.5.0-evidence-pack

# Run verification
./verify.ps1

Exit Codes

The verification scripts return specific exit codes:

Code	Meaning
0	All verifications passed
1	Checksum verification failed
2	Signature verification failed
3	Provenance verification failed
4	Configuration error

Manual Verification Steps

Level 1: Checksum Verification

Verify artifact integrity using SHA-256 checksums:

cd artifacts/
sha256sum -c ../checksums/SHA256SUMS

Expected output:

stella-2.5.0-linux-x64.tar.gz: OK
stella-2.5.0-linux-arm64.tar.gz: OK
stella-2.5.0-macos-universal.tar.gz: OK
stella-2.5.0-windows-x64.zip: OK

Level 2: Signature Verification

Verify that artifacts were signed by Stella Ops:

# Verify the checksums file signature
cosign verify-blob \
  --key cosign.pub \
  --signature checksums/SHA256SUMS.sig \
  checksums/SHA256SUMS

# Verify individual artifact signatures
cosign verify-blob \
  --key cosign.pub \
  --signature artifacts/stella-2.5.0-linux-x64.tar.gz.sig \
  artifacts/stella-2.5.0-linux-x64.tar.gz

Level 3: Provenance Verification

Verify SLSA provenance and inspect build details:

# Verify provenance signature
cosign verify-blob \
  --key cosign.pub \
  --signature provenance/stella-cli.slsa.intoto.jsonl.sig \
  provenance/stella-cli.slsa.intoto.jsonl

# Inspect provenance contents
cat provenance/stella-cli.slsa.intoto.jsonl | jq '.'

# Verify builder ID
BUILDER_ID=$(cat provenance/stella-cli.slsa.intoto.jsonl | \
  jq -r '.predicate.runDetails.builder.id')
echo "Builder: $BUILDER_ID"

# Verify it matches expected value
if [ "$BUILDER_ID" != "https://ci.stella-ops.org/builder/v1" ]; then
  echo "WARNING: Unexpected builder ID"
fi

# Check source commit
SOURCE_COMMIT=$(cat provenance/stella-cli.slsa.intoto.jsonl | \
  jq -r '.predicate.buildDefinition.resolvedDependencies[0].digest.gitCommit')
echo "Source commit: $SOURCE_COMMIT"

Level 4: Transparency Log Verification

Verify Rekor inclusion proofs (requires network OR pre-fetched proofs):

With Network Access

rekor-cli verify \
  --artifact artifacts/stella-2.5.0-linux-x64.tar.gz \
  --signature artifacts/stella-2.5.0-linux-x64.tar.gz.sig \
  --public-key cosign.pub

Offline (using bundled proofs)

The evidence pack includes pre-fetched Rekor proofs in rekor-proofs/:

# List included proofs
cat rekor-proofs/inclusion-proofs.json | jq '.proofs'

# View a specific entry
cat rekor-proofs/log-entries/<uuid>.json | jq '.'

SBOM Verification

Verify Software Bill of Materials:

# Verify SBOM signature
cosign verify-blob \
  --key cosign.pub \
  --signature sbom/stella-cli.cdx.json.sig \
  sbom/stella-cli.cdx.json

# Inspect SBOM contents
cat sbom/stella-cli.cdx.json | jq '.components | length'

Reproducible Build Verification

Stella Ops releases are reproducible. You can rebuild from source and compare:

# Get the SOURCE_DATE_EPOCH from manifest
SOURCE_DATE_EPOCH=$(cat manifest.json | jq -r '.sourceDateEpoch')
SOURCE_COMMIT=$(cat manifest.json | jq -r '.sourceCommit')

# Clone and checkout
git clone https://git.stella-ops.org/stella-ops.org/git.stella-ops.org.git
cd git.stella-ops.org
git checkout $SOURCE_COMMIT

# Set reproducible timestamp
export SOURCE_DATE_EPOCH

# Build
make release

# Compare checksums
sha256sum dist/stella-* | diff - path/to/evidence-pack/checksums/SHA256SUMS

Verification in CI/CD

For automated verification in pipelines:

# Download and verify in one step
curl -sSL https://releases.stella-ops.org/v2.5.0/evidence-pack.tgz | tar -xz
cd stella-release-2.5.0-evidence-pack

# Run verification with JSON output
./verify.sh --json > verification-result.json

# Check result
if [ "$(jq -r '.overall' verification-result.json)" != "PASS" ]; then
  echo "Verification failed!"
  jq '.steps[] | select(.status == "FAIL")' verification-result.json
  exit 1
fi

Troubleshooting

"cosign: command not found"

Install cosign from https://docs.sigstore.dev/cosign/installation/

Checksum Mismatch

1. Re-download the artifact
2. Verify download completed (check file size)
3. Try a different mirror if available
4. Check for file corruption during transfer

Signature Verification Failed

1. Ensure you're using cosign.pub from the evidence pack
2. Check the signature file hasn't been corrupted
3. Verify the artifact hasn't been modified

"Error: no matching entries in transparency log"

This can happen if:

• The artifact was signed with key-based signing (not keyless)
• The Rekor server is unreachable
• Use the bundled proofs in rekor-proofs/ instead

Security Considerations

1. Verify the evidence pack itself - Download only from official sources
2. Compare public key fingerprint - Verify cosign.pub fingerprint matches published key
3. Check provenance builder ID - Ensure it matches expected CI system
4. Review SBOM for known vulnerabilities - Scan dependencies before deployment

Related Documentation

• Release Evidence Pack
• SLSA Compliance
• Reproducible Builds