stella-ops.org/git.stella-ops.org
1
0
Fork 0
Code Issues Pull Requests Actions 2 Releases Wiki Activity
Files
main
git.stella-ops.org/docs/operations/evidence-migration.md

6.5 KiB
Raw Permalink Blame History

Evidence Migration Guide

This guide covers evidence-specific migration procedures during upgrades, schema changes, or disaster recovery scenarios.

Overview

Evidence bundles are cryptographically linked data structures that must maintain integrity across upgrades. This guide ensures chain-of-custody is preserved during migrations.

Quick Reference

Scenario CLI Command Risk Level Downtime
Schema upgrade stella evidence migrate Medium Minutes
Reindex after algorithm change stella evidence reindex Low None
Cross-version continuity check stella evidence verify-continuity None None
Full evidence export stella evidence export --all None None

Pre-Migration Checklist

1. Capture Current State

# Record current evidence statistics
stella evidence stats --detailed > pre-migration-stats.json

# Export Merkle roots for all tenants
stella evidence roots-export --all > pre-migration-roots.json

# Verify existing evidence integrity
stella evidence verify-all --output pre-migration-verify.json
if [ $? -ne 0 ]; then
  echo "ABORT: Evidence integrity check failed"
  exit 1
fi

2. Create Evidence Backup

# Full evidence bundle export
stella evidence export \
  --all \
  --include-attestations \
  --include-proofs \
  --output /backup/evidence-$(date +%Y%m%d)/

# Verify export integrity
stella evidence verify-bundle /backup/evidence-*/

3. Document Chain-of-Custody

# Record the current root hashes
OLD_MERKLE_ROOT=$(stella evidence roots-export --format json | jq -r '.globalRoot')
echo "Pre-migration Merkle root: ${OLD_MERKLE_ROOT}" > custody-log.txt
date >> custody-log.txt

Migration Procedures

Schema Migration (Version Upgrade)

When upgrading between versions with schema changes:

# Step 1: Assess migration impact (dry-run)
stella evidence migrate \
  --from-version 1.0 \
  --to-version 2.0 \
  --dry-run

# Step 2: Review migration plan output
# Ensure all changes are expected

# Step 3: Execute migration
stella evidence migrate \
  --from-version 1.0 \
  --to-version 2.0

# Step 4: Verify migration
stella evidence verify-all

Evidence Reindex (Algorithm Change)

When the hashing algorithm or Merkle tree structure changes:

# Step 1: Assess reindex impact
stella evidence reindex \
  --dry-run \
  --output reindex-plan.json

# Review reindex-plan.json for:
# - Total records affected
# - Estimated duration
# - New schema version

# Step 2: Execute reindex with batching
stella evidence reindex \
  --batch-size 100 \
  --since 2026-01-01

# Step 3: Capture new root
NEW_MERKLE_ROOT=$(stella evidence roots-export --format json | jq -r '.globalRoot')
echo "Post-migration Merkle root: ${NEW_MERKLE_ROOT}" >> custody-log.txt
date >> custody-log.txt

Chain-of-Custody Verification

After any evidence migration, verify continuity:

# Verify that old proofs remain valid
stella evidence verify-continuity \
  --old-root "${OLD_MERKLE_ROOT}" \
  --new-root "${NEW_MERKLE_ROOT}" \
  --output continuity-report.html \
  --format html

# Check verification results
if grep -q "FAIL" continuity-report.html; then
  echo "ERROR: Chain-of-custody verification failed!"
  echo "Review continuity-report.html for details"
  exit 1
fi

Rollback Procedures

Immediate Rollback (Within Migration Window)

# If migration fails mid-way, rollback is automatic
# Check current migration state
stella evidence migrate --status

# Force rollback if needed
stella evidence migrate \
  --rollback \
  --from-version 2.0

Restore from Backup

# Step 1: Stop evidence-related services
kubectl scale deployment evidence-locker --replicas=0

# Step 2: Restore PostgreSQL evidence tables
pg_restore -d stellaops \
  --table='evidence.*' \
  /backup/stellaops-backup.dump

# Step 3: Restore evidence files
stella evidence import /backup/evidence-*/

# Step 4: Verify restoration
stella evidence verify-all

# Step 5: Restart services
kubectl scale deployment evidence-locker --replicas=3

Air-Gap Migration

For air-gapped environments without network access:

Export Phase (Online Environment)

# Create portable evidence bundle
stella evidence export \
  --all \
  --portable \
  --include-schemas \
  --output /media/airgap-evidence.tar.gz

# Generate checksums
sha256sum /media/airgap-evidence.tar.gz > /media/checksums.txt

Transfer Phase

  1. Copy to removable media
  2. Verify checksums at destination
  3. Scan media for security

Import Phase (Air-Gap Environment)

# Verify transfer integrity
sha256sum -c /media/checksums.txt

# Import evidence bundle
stella evidence import \
  --portable \
  /media/airgap-evidence.tar.gz

# Verify import
stella evidence verify-all

Troubleshooting

Migration Stuck or Timeout

# Check migration status
stella evidence migrate --status

# View migration logs
stella evidence migrate --logs

# Resume from last checkpoint
stella evidence migrate --resume

Root Hash Mismatch

If verification reports root hash mismatch:

  1. Do not proceed with upgrade
  2. Check for data corruption: 
    stella evidence integrity-check --deep
  3. Review recent changes to evidence store
  4. Contact support with integrity report

Missing Evidence Records

# Count records by type
stella evidence stats --by-type

# Find orphaned records
stella evidence orphans --list

# Reconcile with source systems
stella evidence reconcile --source attestor

Performance Issues

For large evidence stores (>1M records):

# Run reindex in parallel batches
stella evidence reindex \
  --parallel 4 \
  --batch-size 500 \
  --since 2026-01-01

# Monitor progress
stella evidence reindex --progress

Audit Trail Requirements

All evidence migrations must maintain audit trail:

Event Required Data Retention
Migration Start Timestamp, version, operator Permanent
Schema Change Before/after schema versions Permanent
Root Hash Change Old root, new root, cross-reference Permanent
Verification Pass/fail, anomalies, timestamps 7 years
Rollback Reason, restored version Permanent

Related Documents