git.stella-ops.org/docs/airgap/score-proofs-reachability-airgap-runbook.md

Air-Gap Operations Runbook for Score Proofs & Reachability

Version: 1.0.0
Sprint: 3500.0004.0004
Last Updated: 2025-12-20

This runbook covers air-gapped operations for Score Proofs and Reachability features, including offline kit deployment, proof verification, and bundle management.

---

Table of Contents

1. Overview
2. Offline Kit Deployment
3. Score Proofs in Air-Gap Mode
4. Reachability in Air-Gap Mode
5. Bundle Import Operations
6. Proof Verification Offline
7. Troubleshooting
8. Monitoring & Alerting

---

1. Overview

Air-Gap Modes

Mode	Network	Use Case
Sealed	No external connectivity	Classified environments
Constrained	Limited egress (allowlist)	Regulated networks
Hybrid	Selective connectivity	Standard enterprise

Score Proofs Air-Gap Capabilities

Feature	Sealed	Constrained	Hybrid
Score computation	✅	✅	✅
Score replay	✅	✅	✅
Proof generation	✅	✅	✅
Proof verification	✅	✅	✅
Rekor logging	❌	🔶 (optional)	✅
Feed updates	Bundle	Bundle	Online

Reachability Air-Gap Capabilities

Feature	Sealed	Constrained	Hybrid
Call graph upload	✅	✅	✅
Reachability compute	✅	✅	✅
Explain queries	✅	✅	✅
Symbol resolution	Bundle	Bundle	Online

---

2. Offline Kit Deployment

2.1 Offline Kit Contents

The offline kit contains everything needed for air-gapped Score Proofs and Reachability:

offline-kit/
├── manifests/
│   ├── kit-manifest.json          # Kit metadata and versions
│   ├── feed-manifest.json         # Advisory feed snapshot
│   └── vex-manifest.json          # VEX data snapshot
├── feeds/
│   ├── concelier/                 # Advisory feed data
│   │   ├── advisories.ndjson
│   │   └── snapshot.dsse.json
│   └── excititor/                 # VEX data
│       ├── vex-statements.ndjson
│       └── snapshot.dsse.json
├── policies/
│   ├── scoring-policy.yaml
│   └── policy.dsse.json
├── trust/
│   ├── trust-anchors.json         # Public keys for verification
│   └── time-anchor.json           # Time attestation
├── symbols/
│   └── symbol-index.db            # Symbol resolution database
└── tools/
    ├── stella-cli                 # CLI binary
    └── verify-kit.sh              # Kit verification script

2.2 Verify Kit Integrity

Before deployment, always verify the offline kit:

# Verify kit signature
stella airgap verify-kit --kit /path/to/offline-kit

# Output:
# Kit manifest: VALID
# Feed snapshot: VALID (sha256:feed123...)
# VEX snapshot: VALID (sha256:vex456...)
# Policy: VALID (sha256:policy789...)
# Trust anchors: VALID (3 anchors)
# Time anchor: VALID (expires: 2025-12-31T00:00:00Z)
# 
# Kit verification: PASSED

# Verify individual components
stella airgap verify --file feeds/concelier/snapshot.dsse.json
stella airgap verify --file feeds/excititor/snapshot.dsse.json
stella airgap verify --file policies/policy.dsse.json

2.3 Deploy Offline Kit

# Deploy kit (sealed mode)
stella airgap deploy --kit /path/to/offline-kit --mode sealed

# Deploy kit (constrained mode with limited egress)
stella airgap deploy --kit /path/to/offline-kit \
  --mode constrained \
  --egress-allowlist https://rekor.sigstore.dev

# Verify deployment
stella airgap status

# Output:
# Mode: sealed
# Kit version: 2025.12.20
# Feed snapshot: sha256:feed123... (2025-12-20)
# VEX snapshot: sha256:vex456... (2025-12-20)
# Policy: sha256:policy789... (v1.2.3)
# Trust anchors: 3 active
# Time anchor: Valid until 2025-12-31
# Staleness: OK (0 days)

2.4 Kit Updates

# Check for kit updates (requires external access or new media)
stella airgap check-update --current-kit /path/to/current-kit

# Import new kit
stella airgap import-kit --kit /path/to/new-kit --validate

# Rollback to previous kit
stella airgap rollback --generation previous

---

3. Score Proofs in Air-Gap Mode

3.1 Create Scan (Air-Gap)

# Create scan referencing offline kit snapshots
stella scan create --artifact sha256:abc123... \
  --airgap \
  --feed-snapshot sha256:feed123... \
  --vex-snapshot sha256:vex456... \
  --policy-snapshot sha256:policy789...

# Or auto-detect from deployed kit
stella scan create --artifact sha256:abc123... --use-offline-kit

3.2 Score Replay (Air-Gap)

# Replay with offline kit snapshots
stella score replay --scan-id $SCAN_ID --offline

# Replay with specific bundle
stella score replay --scan-id $SCAN_ID \
  --offline \
  --bundle /path/to/proof-bundle.zip

# Compare with different kit versions
stella score replay --scan-id $SCAN_ID \
  --offline \
  --feed-snapshot sha256:newfeed... \
  --diff

3.3 Generate Proof Bundle (Air-Gap)

# Generate proof bundle for export
stella proof export --scan-id $SCAN_ID \
  --include-manifest \
  --include-chain \
  --output proof-bundle.zip

# Proof bundle contents (air-gap safe):
# - manifest.json (canonical)
# - manifest.dsse.json
# - score_proof.json
# - proof_root.dsse.json
# - meta.json
# - NO external references

# Generate portable bundle
stella proof export --scan-id $SCAN_ID \
  --portable \
  --include-trust-anchors \
  --output portable-proof.zip

---

4. Reachability in Air-Gap Mode

4.1 Call Graph Operations (Air-Gap)

# Upload call graph (works identically)
stella scan graph upload --scan-id $SCAN_ID --file callgraph.json

# Call graph processing is fully local
# No external network required

4.2 Compute Reachability (Air-Gap)

# Compute reachability (fully offline)
stella reachability compute --scan-id $SCAN_ID --offline

# Symbol resolution uses offline database
stella reachability compute --scan-id $SCAN_ID \
  --offline \
  --symbol-db /path/to/offline-kit/symbols/symbol-index.db

4.3 Explain Queries (Air-Gap)

# Explain queries work offline
stella reachability explain --scan-id $SCAN_ID \
  --cve CVE-2024-1234 \
  --purl "pkg:npm/lodash@4.17.20" \
  --offline

# Export explanations for external review
stella reachability explain-all --scan-id $SCAN_ID \
  --output explanations.json \
  --offline

---

5. Bundle Import Operations

5.1 Import Feed Updates

# Verify feed bundle before import
stella airgap verify --bundle feed-update.zip

# Dry-run import
stella airgap import --bundle feed-update.zip \
  --type feed \
  --dry-run

# Import feed bundle
stella airgap import --bundle feed-update.zip \
  --type feed \
  --generation 2025.12.21

# Verify import
stella airgap verify-import --generation 2025.12.21

5.2 Import VEX Updates

# Import VEX bundle
stella airgap import --bundle vex-update.zip \
  --type vex \
  --generation 2025.12.21

# Verify VEX statements
stella airgap vex-status

# Output:
# VEX statements: 15,432
# Last update: 2025-12-21
# Generation: 2025.12.21
# Signature: VALID

5.3 Import Trust Anchors

# Import new trust anchor (requires approval)
stella airgap import-anchor --file new-anchor.json \
  --reason "Key rotation Q4 2025" \
  --approver admin@example.com

# Verify anchor chain
stella airgap verify-anchors

# List active anchors
stella airgap anchors list

5.4 Import Checklist

Pre-Import:

• Verify bundle signature (DSSE)
• Verify bundle hash matches manifest
• Confirm sealed/constrained mode is set
• Backup current generation

Import:

• Run dry-run import
• Apply import
• Verify import succeeded

Post-Import:

• Verify timeline event emitted
• Update staleness dashboard
• Archive import manifest
• Update audit log

---

6. Proof Verification Offline

6.1 Verify Proof Bundle (Full Offline)

# Verify proof bundle without any network access
stella proof verify --bundle proof-bundle.zip \
  --offline \
  --trust-anchor /path/to/trust-anchors.json

# Verification checks (offline):
# ✅ Signature valid (DSSE)
# ✅ Content-addressed ID matches
# ✅ Merkle path valid
# ⏭️ Rekor inclusion (SKIPPED - offline mode)
# ✅ Time anchor valid

6.2 Verify with Portable Bundle

# Portable bundles include trust anchors
stella proof verify --bundle portable-proof.zip \
  --offline \
  --self-contained

# Output:
# Using embedded trust anchors
# Signature verification: PASS
# ID recomputation: PASS
# Merkle path: PASS
# Time anchor: VALID
# Overall: VERIFIED

6.3 Batch Verification

# Verify multiple bundles
stella proof verify-batch --dir /path/to/bundles/ \
  --offline \
  --trust-anchor /path/to/trust-anchors.json \
  --output verification-report.json

# Generate verification report
cat verification-report.json | jq '.summary'
# Output:
# {
#   "total": 100,
#   "verified": 98,
#   "failed": 2,
#   "skipped": 0
# }

6.4 Verification Without CLI

For environments without the CLI, manual verification is possible:

# 1. Extract bundle
unzip proof-bundle.zip -d ./verify/

# 2. Verify DSSE signature (using openssl)
# Extract payload from DSSE envelope
cat ./verify/manifest.dsse.json | jq -r '.payload' | base64 -d > payload.json

# Verify signature
cat ./verify/manifest.dsse.json | jq -r '.signatures[0].sig' | base64 -d > signature.bin
openssl dgst -sha256 -verify trust-anchor-pubkey.pem -signature signature.bin payload.json

# 3. Verify content-addressed ID
# Compute canonical hash
cat ./verify/manifest.json | jq -cS . | sha256sum
# Compare with manifestHash in bundle

# 4. Verify merkle path
# (See docs/airgap/proof-chain-verification.md for algorithm)

---

7. Troubleshooting

7.1 Kit Verification Failed

Symptom: stella airgap verify-kit fails.

Diagnosis:

# Check specific component
stella airgap verify-kit --verbose --component feeds

# Common errors:
# - "Signature verification failed": Key mismatch
# - "Hash mismatch": Bundle corrupted during transfer
# - "Time anchor expired": Anchor needs refresh

Resolution:

Error	Cause	Resolution
Signature failed	Wrong trust anchor	Import correct anchor
Hash mismatch	Corruption	Re-transfer bundle
Time anchor expired	Clock drift or expired	Import new time anchor

7.2 Staleness Alert

Symptom: Staleness warning/alert.

Diagnosis:

# Check staleness status
stella airgap staleness-status

# Output:
# Feed age: 5 days (threshold: 7 days)
# VEX age: 3 days (threshold: 7 days)
# Policy age: 30 days (threshold: 90 days)
# Status: AMBER (approaching threshold)

Resolution:

# Import updated bundles
stella airgap import --bundle /path/to/latest-feed.zip --type feed

# If bundles unavailable and breach imminent:
# - Raise amber alert (5-7 days)
# - If >7 days, raise red alert and halt new ingests
# - Request emergency bundle via secure channel

7.3 Proof Verification Fails Offline

Symptom: Proof verification fails in sealed mode.

Diagnosis:

# Check verification error
stella proof verify --bundle proof.zip --offline --verbose

# Common errors:
# - "Trust anchor not found": Missing anchor in offline kit
# - "Time anchor expired": Time validation failed
# - "Unsupported algorithm": Key algorithm not supported

Resolution:

# For missing trust anchor:
# Import the required anchor
stella airgap import-anchor --file required-anchor.json

# For expired time anchor:
# Import new time anchor
stella airgap import-time-anchor --file new-time-anchor.json

# For algorithm issues:
# Regenerate proof with supported algorithm
stella proof regenerate --scan-id $SCAN_ID --algorithm ECDSA-P256

7.4 Symbol Resolution Fails

Symptom: Reachability shows "symbol not found" errors.

Diagnosis:

# Check symbol database status
stella airgap symbols-status

# Output:
# Symbol DB: /path/to/symbols/symbol-index.db
# Version: 2025.12.15
# Entries: 5,234,567
# Coverage: Java, .NET, Python

Resolution:

# Import updated symbol database
stella airgap import --bundle symbol-update.zip --type symbols

# Recompute reachability with new symbols
stella reachability compute --scan-id $SCAN_ID --offline --force

---

8. Monitoring & Alerting

8.1 Key Metrics (Air-Gap)

Metric	Description	Alert Threshold
airgap_staleness_days	Days since last bundle import	> 5 (amber), > 7 (red)
airgap_time_anchor_validity_days	Days until time anchor expires	< 7
airgap_verification_failures	Offline verification failures	> 0
airgap_import_failures	Bundle import failures	> 0

8.2 Alerting Rules

groups:
  - name: airgap-operations
    rules:
      - alert: AirgapStalenessAmber
        expr: airgap_staleness_days > 5
        for: 1h
        labels:
          severity: warning
        annotations:
          summary: "Air-gap feed staleness approaching threshold"
          
      - alert: AirgapStalenessRed
        expr: airgap_staleness_days > 7
        for: 1h
        labels:
          severity: critical
        annotations:
          summary: "Air-gap feed staleness breach - halt new ingests"
          
      - alert: AirgapTimeAnchorExpiring
        expr: airgap_time_anchor_validity_days < 7
        for: 1h
        labels:
          severity: warning
        annotations:
          summary: "Time anchor expiring in {{ $value }} days"
          
      - alert: AirgapVerificationFailure
        expr: increase(airgap_verification_failures_total[1h]) > 0
        for: 5m
        labels:
          severity: critical
        annotations:
          summary: "Air-gap verification failures detected"

8.3 Audit Requirements

For air-gapped environments, maintain strict audit trails:

# Record every import
{
  "timestamp": "2025-12-20T10:00:00Z",
  "action": "import",
  "bundleType": "feed",
  "bundleHash": "sha256:feed123...",
  "generation": "2025.12.20",
  "actor": "operator@example.com",
  "mode": "sealed",
  "verification": "PASS"
}

# Daily audit log export
stella airgap audit-export --date today --output audit-$(date +%Y%m%d).json

# Verify audit log integrity
stella airgap audit-verify --log audit-20251220.json

---

Related Documentation

• Air-Gap Overview
• Offline Bundle Format
• Proof Chain Verification
• Time Anchor Schema
• Score Proofs Runbook
• Reachability Runbook

---

Last Updated: 2025-12-20
Version: 1.0.0
Sprint: 3500.0004.0004