sprints completion. new product advisories prepared

2026-01-16 16:30:03 +02:00
parent a927d924e3
commit 4ca3ce8fb4
255 changed files with 42434 additions and 1020 deletions
--- a/docs/operations/evidence-migration.md
+++ b/docs/operations/evidence-migration.md
@@ -0,0 +1,278 @@
+# Evidence Migration Guide
+
+This guide covers evidence-specific migration procedures during upgrades, schema changes, or disaster recovery scenarios.
+
+## Overview
+
+Evidence bundles are cryptographically linked data structures that must maintain integrity across upgrades. This guide ensures chain-of-custody is preserved during migrations.
+
+## Quick Reference
+
+| Scenario | CLI Command | Risk Level | Downtime |
+|----------|-------------|------------|----------|
+| Schema upgrade | `stella evidence migrate` | Medium | Minutes |
+| Reindex after algorithm change | `stella evidence reindex` | Low | None |
+| Cross-version continuity check | `stella evidence verify-continuity` | None | None |
+| Full evidence export | `stella evidence export --all` | None | None |
+
+## Pre-Migration Checklist
+
+### 1. Capture Current State
+
+```bash
+# Record current evidence statistics
+stella evidence stats --detailed > pre-migration-stats.json
+
+# Export Merkle roots for all tenants
+stella evidence roots-export --all > pre-migration-roots.json
+
+# Verify existing evidence integrity
+stella evidence verify-all --output pre-migration-verify.json
+if [ $? -ne 0 ]; then
+  echo "ABORT: Evidence integrity check failed"
+  exit 1
+fi
+```
+
+### 2. Create Evidence Backup
+
+```bash
+# Full evidence bundle export
+stella evidence export \
+  --all \
+  --include-attestations \
+  --include-proofs \
+  --output /backup/evidence-$(date +%Y%m%d)/
+
+# Verify export integrity
+stella evidence verify-bundle /backup/evidence-*/
+```
+
+### 3. Document Chain-of-Custody
+
+```bash
+# Record the current root hashes
+OLD_MERKLE_ROOT=$(stella evidence roots-export --format json | jq -r '.globalRoot')
+echo "Pre-migration Merkle root: ${OLD_MERKLE_ROOT}" > custody-log.txt
+date >> custody-log.txt
+```
+
+## Migration Procedures
+
+### Schema Migration (Version Upgrade)
+
+When upgrading between versions with schema changes:
+
+```bash
+# Step 1: Assess migration impact (dry-run)
+stella evidence migrate \
+  --from-version 1.0 \
+  --to-version 2.0 \
+  --dry-run
+
+# Step 2: Review migration plan output
+# Ensure all changes are expected
+
+# Step 3: Execute migration
+stella evidence migrate \
+  --from-version 1.0 \
+  --to-version 2.0
+
+# Step 4: Verify migration
+stella evidence verify-all
+```
+
+### Evidence Reindex (Algorithm Change)
+
+When the hashing algorithm or Merkle tree structure changes:
+
+```bash
+# Step 1: Assess reindex impact
+stella evidence reindex \
+  --dry-run \
+  --output reindex-plan.json
+
+# Review reindex-plan.json for:
+# - Total records affected
+# - Estimated duration
+# - New schema version
+
+# Step 2: Execute reindex with batching
+stella evidence reindex \
+  --batch-size 100 \
+  --since 2026-01-01
+
+# Step 3: Capture new root
+NEW_MERKLE_ROOT=$(stella evidence roots-export --format json | jq -r '.globalRoot')
+echo "Post-migration Merkle root: ${NEW_MERKLE_ROOT}" >> custody-log.txt
+date >> custody-log.txt
+```
+
+### Chain-of-Custody Verification
+
+After any evidence migration, verify continuity:
+
+```bash
+# Verify that old proofs remain valid
+stella evidence verify-continuity \
+  --old-root "${OLD_MERKLE_ROOT}" \
+  --new-root "${NEW_MERKLE_ROOT}" \
+  --output continuity-report.html \
+  --format html
+
+# Check verification results
+if grep -q "FAIL" continuity-report.html; then
+  echo "ERROR: Chain-of-custody verification failed!"
+  echo "Review continuity-report.html for details"
+  exit 1
+fi
+```
+
+## Rollback Procedures
+
+### Immediate Rollback (Within Migration Window)
+
+```bash
+# If migration fails mid-way, rollback is automatic
+# Check current migration state
+stella evidence migrate --status
+
+# Force rollback if needed
+stella evidence migrate \
+  --rollback \
+  --from-version 2.0
+```
+
+### Restore from Backup
+
+```bash
+# Step 1: Stop evidence-related services
+kubectl scale deployment evidence-locker --replicas=0
+
+# Step 2: Restore PostgreSQL evidence tables
+pg_restore -d stellaops \
+  --table='evidence.*' \
+  /backup/stellaops-backup.dump
+
+# Step 3: Restore evidence files
+stella evidence import /backup/evidence-*/
+
+# Step 4: Verify restoration
+stella evidence verify-all
+
+# Step 5: Restart services
+kubectl scale deployment evidence-locker --replicas=3
+```
+
+## Air-Gap Migration
+
+For air-gapped environments without network access:
+
+### Export Phase (Online Environment)
+
+```bash
+# Create portable evidence bundle
+stella evidence export \
+  --all \
+  --portable \
+  --include-schemas \
+  --output /media/airgap-evidence.tar.gz
+
+# Generate checksums
+sha256sum /media/airgap-evidence.tar.gz > /media/checksums.txt
+```
+
+### Transfer Phase
+
+1. Copy to removable media
+2. Verify checksums at destination
+3. Scan media for security
+
+### Import Phase (Air-Gap Environment)
+
+```bash
+# Verify transfer integrity
+sha256sum -c /media/checksums.txt
+
+# Import evidence bundle
+stella evidence import \
+  --portable \
+  /media/airgap-evidence.tar.gz
+
+# Verify import
+stella evidence verify-all
+```
+
+## Troubleshooting
+
+### Migration Stuck or Timeout
+
+```bash
+# Check migration status
+stella evidence migrate --status
+
+# View migration logs
+stella evidence migrate --logs
+
+# Resume from last checkpoint
+stella evidence migrate --resume
+```
+
+### Root Hash Mismatch
+
+If verification reports root hash mismatch:
+
+1. **Do not proceed** with upgrade
+2. Check for data corruption:
+   ```bash
+   stella evidence integrity-check --deep
+   ```
+3. Review recent changes to evidence store
+4. Contact support with integrity report
+
+### Missing Evidence Records
+
+```bash
+# Count records by type
+stella evidence stats --by-type
+
+# Find orphaned records
+stella evidence orphans --list
+
+# Reconcile with source systems
+stella evidence reconcile --source attestor
+```
+
+### Performance Issues
+
+For large evidence stores (>1M records):
+
+```bash
+# Run reindex in parallel batches
+stella evidence reindex \
+  --parallel 4 \
+  --batch-size 500 \
+  --since 2026-01-01
+
+# Monitor progress
+stella evidence reindex --progress
+```
+
+## Audit Trail Requirements
+
+All evidence migrations must maintain audit trail:
+
+| Event | Required Data | Retention |
+|-------|---------------|-----------|
+| Migration Start | Timestamp, version, operator | Permanent |
+| Schema Change | Before/after schema versions | Permanent |
+| Root Hash Change | Old root, new root, cross-reference | Permanent |
+| Verification | Pass/fail, anomalies, timestamps | 7 years |
+| Rollback | Reason, restored version | Permanent |
+
+## Related Documents
+
+- [Upgrade Runbook](upgrade-runbook.md) - Overall upgrade procedures
+- [Blue-Green Deployment](blue-green-deployment.md) - Zero-downtime deployment
+- [Evidence Locker Architecture](../modules/evidencelocker/architecture.md) - Technical design
+- [Air-Gap Operations](airgap-operations-runbook.md) - Offline deployment guide