git.stella-ops.org/docs/implplan/SPRINT_5100_SUMMARY.md

Sprint Epic 5100 · Comprehensive Testing Strategy

Overview

Epic 5100 implements the comprehensive testing strategy defined in the Testing Strategy advisory (20-Dec-2025). This epic transforms Stella Ops' testing moats into continuously verified guarantees through deterministic replay, offline compliance, interoperability contracts, and chaos resilience testing.

IMPLID: 5100 (Test Infrastructure) Total Sprints: 12 Total Tasks: ~75

---

Epic Structure

Phase 0: Harness & Corpus Foundation

Objective: Standardize test artifacts and expand the golden corpus.

Sprint	Name	Tasks	Priority
5100.0001.0001	Run Manifest Schema	7	HIGH
5100.0001.0002	Evidence Index Schema	7	HIGH
5100.0001.0003	Offline Bundle Manifest	7	HIGH
5100.0001.0004	Golden Corpus Expansion	10	MEDIUM

Key Deliverables:

• RunManifest schema capturing all replay inputs
• EvidenceIndex schema linking verdict to evidence chain
• BundleManifest for offline operation
• 50+ golden test corpus cases

---

Phase 1: Determinism & Replay

Objective: Ensure byte-identical verdicts across time and machines.

Sprint	Name	Tasks	Priority
5100.0002.0001	Canonicalization Utilities	7	HIGH
5100.0002.0002	Replay Runner Service	7	HIGH
5100.0002.0003	Delta-Verdict Generator	7	MEDIUM

Key Deliverables:

• Canonical JSON serialization (RFC 8785 principles)
• Stable ordering for all collections
• Replay engine with frozen time/PRNG
• Delta-verdict for diff-aware release gates
• Property-based tests with FsCheck

---

Phase 2: Offline E2E & Interop

Objective: Prove air-gap compliance and tool interoperability.

Sprint	Name	Tasks	Priority
5100.0003.0001	SBOM Interop Round-Trip	7	HIGH
5100.0003.0002	No-Egress Enforcement	6	HIGH

Key Deliverables:

• Syft → cosign → Grype round-trip tests
• CycloneDX 1.6 and SPDX 3.0.1 validation
• 95%+ findings parity with consumer tools
• Network-isolated test infrastructure
• --network none CI enforcement

---

Phase 3: Unknowns Budgets CI Gates

Objective: Enforce unknowns-budget policy gates in CI/CD.

Sprint	Name	Tasks	Priority
5100.0004.0001	Unknowns Budget CI Gates	6	HIGH

Key Deliverables:

• stella budget check CLI command
• CI workflow with environment-based budgets
• PR comments with budget status
• UI budget visualization
• Attestation integration

---

Phase 4: Backpressure & Chaos

Objective: Validate router resilience under load.

Sprint	Name	Tasks	Priority
5100.0005.0001	Router Chaos Suite	6	MEDIUM

Key Deliverables:

• k6 load test harness
• 429/503 response verification
• Retry-After header compliance
• Recovery within 30 seconds
• Valkey failure injection tests

---

Phase 5: Audit Packs & Time-Travel

Objective: Enable sealed export/import for auditors.

Sprint	Name	Tasks	Priority
5100.0006.0001	Audit Pack Export/Import	6	MEDIUM

Key Deliverables:

• Sealed audit pack format
• One-command replay verification
• Signature verification with included trust roots
• CLI commands for auditor workflow

---

Dependency Graph

Phase 0 (Foundation)
├── 5100.0001.0001 (Run Manifest)
│   └── Phase 1 depends
├── 5100.0001.0002 (Evidence Index)
│   └── Phase 2, 5 depend
├── 5100.0001.0003 (Offline Bundle)
│   └── Phase 2 depends
└── 5100.0001.0004 (Golden Corpus)
    └── All phases use

Phase 1 (Determinism)
├── 5100.0002.0001 (Canonicalization)
│   └── 5100.0002.0002, 5100.0002.0003 depend
├── 5100.0002.0002 (Replay Runner)
│   └── Phase 5 depends
└── 5100.0002.0003 (Delta-Verdict)

Phase 2 (Offline & Interop)
├── 5100.0003.0001 (SBOM Interop)
└── 5100.0003.0002 (No-Egress)

Phase 3 (Unknowns Gates)
└── 5100.0004.0001 (CI Gates)
    └── Depends on 4100.0001.0002

Phase 4 (Chaos)
└── 5100.0005.0001 (Router Chaos)

Phase 5 (Audit Packs)
└── 5100.0006.0001 (Export/Import)
    └── Depends on Phase 0, Phase 1

---

CI/CD Integration

New Workflows

Workflow	Trigger	Purpose
replay-verification.yml	PR (scanner changes)	Verify deterministic replay
interop-e2e.yml	PR + Nightly	SBOM interoperability
offline-e2e.yml	PR + Nightly	Air-gap compliance
unknowns-gate.yml	PR + Push	Budget enforcement
router-chaos.yml	Nightly	Resilience testing

Release Blocking Gates

A release candidate is blocked if any of these fail:

1. Replay Verification: Zero non-deterministic diffs
2. Interop Suite: 95%+ findings parity
3. Offline E2E: All tests pass with no network
4. Unknowns Budget: Within budget for prod environment
5. Performance: No breach of p95/memory budgets

---

Success Criteria

Criteria	Metric	Gate
Full scan + attest + verify with no network	offline-e2e passes	Release
Re-running fixed input = identical verdict	0 byte diff	Release
Grype from SBOM matches image scan	95%+ parity	Release
Builds fail when unknowns > budget	Exit code 2	PR
Router under burst emits correct Retry-After	100% compliance	Nightly
Evidence index links complete	Validation passes	Release

---

Artifacts Standardized

Artifact	Schema Location	Purpose
Run Manifest	StellaOps.Testing.Manifests	Replay key
Evidence Index	StellaOps.Evidence	Verdict → evidence chain
Offline Bundle	StellaOps.AirGap.Bundle	Air-gap operation
Delta Verdict	StellaOps.DeltaVerdict	Diff-aware gates
Audit Pack	StellaOps.AuditPack	Compliance verification

---

Implementation Order

Immediate (This Week)

1. 5100.0001.0001 - Run Manifest Schema
2. 5100.0002.0001 - Canonicalization Utilities
3. 5100.0004.0001 - Unknowns Budget CI Gates

Short Term (Next 2 Sprints)

4. 5100.0001.0002 - Evidence Index Schema
5. 5100.0002.0002 - Replay Runner Service
6. 5100.0003.0001 - SBOM Interop Round-Trip

Medium Term (Following Sprints)

7. 5100.0001.0003 - Offline Bundle Manifest
8. 5100.0003.0002 - No-Egress Enforcement
9. 5100.0002.0003 - Delta-Verdict Generator

Later

10. 5100.0001.0004 - Golden Corpus Expansion
11. 5100.0005.0001 - Router Chaos Suite
12. 5100.0006.0001 - Audit Pack Export/Import

---

Related Documentation

• Test Suite Overview
• Testing Strategy Advisory
• Offline Operation Guide
• tests/AGENTS.md

---

Execution Log

Date (UTC)	Update	Owner
2025-12-21	Epic created from Testing Strategy advisory analysis. 12 sprints defined across 6 phases.	Agent

---

Epic Status: PLANNING (0/12 sprints complete)