# Artifact Store Migration Runbook

Sprint: SPRINT_20260118_017_Evidence_artifact_store_unification (AS-006)

## Overview

This runbook covers the migration of existing evidence from legacy artifact stores to the unified ArtifactStore.

## Migration Sources

| Source | Legacy Path | Description |
|--------|-------------|-------------|
| EvidenceLocker | `tenants/{tenantId}/bundles/{bundleId}/{sha256}-{name}` | Evidence bundles |
| Attestor | `attest/dsse/{bundleSha256}.json` | DSSE envelopes |
| Vex | `{prefix}/{format}/{digest}.{ext}` | VEX documents |

## Target Path Convention

All artifacts are migrated to: `/artifacts/{bom-ref-encoded}/{serialNumber}/{artifactId}.json`

## Pre-Migration Checklist

- [ ] Backup existing S3 buckets
- [ ] Verify PostgreSQL backup is current
- [ ] Ensure sufficient storage for duplicated data
- [ ] Review migration in dry-run mode first
- [ ] Notify stakeholders of potential service impact

## Running the Migration

### Dry Run (Recommended First Step)

```bash
stella artifacts migrate --source all --dry-run --output migration-preview.json
```

### Full Migration

```bash
# Migrate all sources with default settings
stella artifacts migrate --source all

# Migrate with increased parallelism
stella artifacts migrate --source all --parallelism 8 --batch-size 200

# Migrate specific source
stella artifacts migrate --source evidence --output migration-report.json

# Migrate specific tenant
stella artifacts migrate --source all --tenant <tenant-uuid>
```

### Resuming Failed Migration

```bash
# Use checkpoint ID from previous run
stella artifacts migrate --source all --resume-from <checkpoint-id>
```

## Progress Monitoring

The CLI displays real-time progress:

```
  Progress: 1500/10000 (15.0%) - Success: 1495, Failed: 3, Skipped: 2
```

## Rollback Procedure

### When to Rollback

- Migration corrupted data
- Performance degradation after migration
- Business-critical bug discovered

### Rollback Steps

#### 1. Stop New Writes to Unified Store

```bash
# Disable unified store in configuration
kubectl set env deployment/evidence-locker ARTIFACT_STORE_UNIFIED_ENABLED=false
kubectl set env deployment/attestor ARTIFACT_STORE_UNIFIED_ENABLED=false
```

#### 2. Revert Application Configuration

```yaml
# etc/appsettings.yaml
artifactStore:
  useUnifiedStore: false
  legacyMode: true
```

#### 3. Clear Unified Store Index

```sql
-- Clear PostgreSQL index (preserves S3 data)
TRUNCATE TABLE artifact_store.artifacts;
```

#### 4. (Optional) Remove Migrated S3 Objects

```bash
# Only if disk space is critical and you're certain about rollback
# WARNING: This is destructive!
aws s3 rm s3://artifacts-bucket/artifacts/ --recursive
```

#### 5. Restart Services

```bash
kubectl rollout restart deployment/evidence-locker
kubectl rollout restart deployment/attestor
```

#### 6. Verify Legacy Stores Work

```bash
# Test evidence retrieval
stella evidence get --bundle-id <test-bundle>

# Test attestation retrieval  
stella attestor get --digest <test-digest>
```

## Post-Migration Validation

### Verify Artifact Counts

```sql
-- Count migrated artifacts by source
SELECT 
  CASE 
    WHEN storage_key LIKE '%evidence%' THEN 'evidence'
    WHEN storage_key LIKE '%dsse%' THEN 'attestor'
    WHEN storage_key LIKE '%vex%' THEN 'vex'
    ELSE 'unknown'
  END as source,
  COUNT(*) as count
FROM artifact_store.artifacts
GROUP BY 1;
```

### Verify bom-ref Extraction

```sql
-- Check for artifacts with synthetic bom-refs (extraction failed)
SELECT COUNT(*) as synthetic_count
FROM artifact_store.artifacts
WHERE bom_ref LIKE 'sha256:%';
```

### Test Retrieval

```bash
# Query by bom-ref
curl "https://api.example.com/api/v1/artifacts?bom_ref=pkg:docker/acme/api@sha256:abc123"

# Verify content matches original
stella artifacts compare \
  --original tenants/xxx/bundles/yyy/sha256-sbom.json \
  --migrated /artifacts/encoded-ref/serial/artifact.json
```

## Troubleshooting

### Migration Stuck

```bash
# Check for stuck workers
ps aux | grep migrate

# Check migration checkpoints
cat /var/lib/stella/migration-checkpoint.json
```

### High Failure Rate

1. Check migration report for common errors
2. Verify source store connectivity
3. Check for corrupted source artifacts
4. Increase batch size for memory issues

### Slow Migration

1. Increase parallelism (up to CPU count)
2. Run during off-peak hours
3. Consider migrating by tenant in parallel
4. Verify network bandwidth to S3

## Representative Dataset Testing

Before production migration, test with representative dataset:

```bash
# Export sample from each source
stella evidence list --limit 100 --output sample-evidence.json
stella attestor list --limit 100 --output sample-attestor.json

# Create test environment with samples
stella artifacts migrate --source all --tenant test-tenant --output test-report.json

# Verify counts and content
diff <(cat sample-evidence.json | jq '.total') <(cat test-report.json | jq '.succeeded')
```

## Related Documentation

- [Artifact Store API](../api/artifact-store-api.yaml)
- [IArtifactStore Interface](../../src/__Libraries/StellaOps.Artifact.Core/IArtifactStore.cs)
- [PostgreSQL Index Schema](../../src/__Libraries/StellaOps.Artifact.Infrastructure/Migrations/001_artifact_index_schema.sql)