# Sprint Epic 5100 · Comprehensive Testing Strategy

## Overview

Epic 5100 implements the comprehensive testing strategy defined in the Testing Strategy advisory (20-Dec-2025). This epic transforms Stella Ops' testing moats into continuously verified guarantees through deterministic replay, offline compliance, interoperability contracts, and chaos resilience testing.

**IMPLID**: 5100 (Test Infrastructure)
**Total Sprints**: 12
**Total Tasks**: ~75

---

## Epic Structure

### Phase 0: Harness & Corpus Foundation
**Objective**: Standardize test artifacts and expand the golden corpus.

| Sprint | Name | Tasks | Priority |
|--------|------|-------|----------|
| 5100.0001.0001 | [Run Manifest Schema](SPRINT_5100_0001_0001_run_manifest_schema.md) | 7 | HIGH |
| 5100.0001.0002 | [Evidence Index Schema](SPRINT_5100_0001_0002_evidence_index_schema.md) | 7 | HIGH |
| 5100.0001.0003 | [Offline Bundle Manifest](SPRINT_5100_0001_0003_offline_bundle_manifest.md) | 7 | HIGH |
| 5100.0001.0004 | [Golden Corpus Expansion](SPRINT_5100_0001_0004_golden_corpus_expansion.md) | 10 | MEDIUM |

**Key Deliverables**:
- `RunManifest` schema capturing all replay inputs
- `EvidenceIndex` schema linking verdict to evidence chain
- `BundleManifest` for offline operation
- 50+ golden test corpus cases

---

### Phase 1: Determinism & Replay
**Objective**: Ensure byte-identical verdicts across time and machines.

| Sprint | Name | Tasks | Priority |
|--------|------|-------|----------|
| 5100.0002.0001 | [Canonicalization Utilities](SPRINT_5100_0002_0001_canonicalization_utilities.md) | 7 | HIGH |
| 5100.0002.0002 | [Replay Runner Service](SPRINT_5100_0002_0002_replay_runner_service.md) | 7 | HIGH |
| 5100.0002.0003 | [Delta-Verdict Generator](SPRINT_5100_0002_0003_delta_verdict_generator.md) | 7 | MEDIUM |

**Key Deliverables**:
- Canonical JSON serialization (RFC 8785 principles)
- Stable ordering for all collections
- Replay engine with frozen time/PRNG
- Delta-verdict for diff-aware release gates
- Property-based tests with FsCheck

---

### Phase 2: Offline E2E & Interop
**Objective**: Prove air-gap compliance and tool interoperability.

| Sprint | Name | Tasks | Priority |
|--------|------|-------|----------|
| 5100.0003.0001 | [SBOM Interop Round-Trip](SPRINT_5100_0003_0001_sbom_interop_roundtrip.md) | 7 | HIGH |
| 5100.0003.0002 | [No-Egress Enforcement](SPRINT_5100_0003_0002_no_egress_enforcement.md) | 6 | HIGH |

**Key Deliverables**:
- Syft → cosign → Grype round-trip tests
- CycloneDX 1.6 and SPDX 3.0.1 validation
- 95%+ findings parity with consumer tools
- Network-isolated test infrastructure
- `--network none` CI enforcement

---

### Phase 3: Unknowns Budgets CI Gates
**Objective**: Enforce unknowns-budget policy gates in CI/CD.

| Sprint | Name | Tasks | Priority |
|--------|------|-------|----------|
| 5100.0004.0001 | [Unknowns Budget CI Gates](SPRINT_5100_0004_0001_unknowns_budget_ci_gates.md) | 6 | HIGH |

**Key Deliverables**:
- `stella budget check` CLI command
- CI workflow with environment-based budgets
- PR comments with budget status
- UI budget visualization
- Attestation integration

---

### Phase 4: Backpressure & Chaos
**Objective**: Validate router resilience under load.

| Sprint | Name | Tasks | Priority |
|--------|------|-------|----------|
| 5100.0005.0001 | [Router Chaos Suite](SPRINT_5100_0005_0001_router_chaos_suite.md) | 6 | MEDIUM |

**Key Deliverables**:
- k6 load test harness
- 429/503 response verification
- Retry-After header compliance
- Recovery within 30 seconds
- Valkey failure injection tests

---

### Phase 5: Audit Packs & Time-Travel
**Objective**: Enable sealed export/import for auditors.

| Sprint | Name | Tasks | Priority |
|--------|------|-------|----------|
| 5100.0006.0001 | [Audit Pack Export/Import](SPRINT_5100_0006_0001_audit_pack_export_import.md) | 6 | MEDIUM |

**Key Deliverables**:
- Sealed audit pack format
- One-command replay verification
- Signature verification with included trust roots
- CLI commands for auditor workflow

---

## Dependency Graph

```
Phase 0 (Foundation)
├── 5100.0001.0001 (Run Manifest)
│   └── Phase 1 depends
├── 5100.0001.0002 (Evidence Index)
│   └── Phase 2, 5 depend
├── 5100.0001.0003 (Offline Bundle)
│   └── Phase 2 depends
└── 5100.0001.0004 (Golden Corpus)
    └── All phases use

Phase 1 (Determinism)
├── 5100.0002.0001 (Canonicalization)
│   └── 5100.0002.0002, 5100.0002.0003 depend
├── 5100.0002.0002 (Replay Runner)
│   └── Phase 5 depends
└── 5100.0002.0003 (Delta-Verdict)

Phase 2 (Offline & Interop)
├── 5100.0003.0001 (SBOM Interop)
└── 5100.0003.0002 (No-Egress)

Phase 3 (Unknowns Gates)
└── 5100.0004.0001 (CI Gates)
    └── Depends on 4100.0001.0002

Phase 4 (Chaos)
└── 5100.0005.0001 (Router Chaos)

Phase 5 (Audit Packs)
└── 5100.0006.0001 (Export/Import)
    └── Depends on Phase 0, Phase 1
```

---

## CI/CD Integration

### New Workflows

| Workflow | Trigger | Purpose |
|----------|---------|---------|
| `replay-verification.yml` | PR (scanner changes) | Verify deterministic replay |
| `interop-e2e.yml` | PR + Nightly | SBOM interoperability |
| `offline-e2e.yml` | PR + Nightly | Air-gap compliance |
| `unknowns-gate.yml` | PR + Push | Budget enforcement |
| `router-chaos.yml` | Nightly | Resilience testing |

### Release Blocking Gates

A release candidate is blocked if any of these fail:

1. **Replay Verification**: Zero non-deterministic diffs
2. **Interop Suite**: 95%+ findings parity
3. **Offline E2E**: All tests pass with no network
4. **Unknowns Budget**: Within budget for prod environment
5. **Performance**: No breach of p95/memory budgets

---

## Success Criteria

| Criteria | Metric | Gate |
|----------|--------|------|
| Full scan + attest + verify with no network | `offline-e2e` passes | Release |
| Re-running fixed input = identical verdict | 0 byte diff | Release |
| Grype from SBOM matches image scan | 95%+ parity | Release |
| Builds fail when unknowns > budget | Exit code 2 | PR |
| Router under burst emits correct Retry-After | 100% compliance | Nightly |
| Evidence index links complete | Validation passes | Release |

---

## Artifacts Standardized

| Artifact | Schema Location | Purpose |
|----------|-----------------|---------|
| Run Manifest | `StellaOps.Testing.Manifests` | Replay key |
| Evidence Index | `StellaOps.Evidence` | Verdict → evidence chain |
| Offline Bundle | `StellaOps.AirGap.Bundle` | Air-gap operation |
| Delta Verdict | `StellaOps.DeltaVerdict` | Diff-aware gates |
| Audit Pack | `StellaOps.AuditPack` | Compliance verification |

---

## Implementation Order

### Immediate (This Week)
1. **5100.0001.0001** - Run Manifest Schema
2. **5100.0002.0001** - Canonicalization Utilities
3. **5100.0004.0001** - Unknowns Budget CI Gates

### Short Term (Next 2 Sprints)
4. **5100.0001.0002** - Evidence Index Schema
5. **5100.0002.0002** - Replay Runner Service
6. **5100.0003.0001** - SBOM Interop Round-Trip

### Medium Term (Following Sprints)
7. **5100.0001.0003** - Offline Bundle Manifest
8. **5100.0003.0002** - No-Egress Enforcement
9. **5100.0002.0003** - Delta-Verdict Generator

### Later
10. **5100.0001.0004** - Golden Corpus Expansion
11. **5100.0005.0001** - Router Chaos Suite
12. **5100.0006.0001** - Audit Pack Export/Import

---

## Related Documentation

- [Test Suite Overview](../19_TEST_SUITE_OVERVIEW.md)
- [Testing Strategy Advisory](../product-advisories/20-Dec-2025%20-%20Testing%20strategy.md)
- [Offline Operation Guide](../24_OFFLINE_KIT.md)
- [tests/AGENTS.md](../../tests/AGENTS.md)

---

## Execution Log

| Date (UTC) | Update | Owner |
|------------|--------|-------|
| 2025-12-21 | Epic created from Testing Strategy advisory analysis. 12 sprints defined across 6 phases. | Agent |

---

**Epic Status**: PLANNING (0/12 sprints complete)