Archived Advisory: Testing Strategy
Archived: 2025-12-21
Original: docs/product-advisories/20-Dec-2025 - Testing strategy.md
Processing Summary
This advisory was processed into Sprint Epic 5100 - Comprehensive Testing Strategy.
Artifacts Created
Sprint Files (12 sprints, ~75 tasks):
| Sprint | Name | Phase |
|---|---|---|
| 5100.0001.0001 | Run Manifest Schema | Phase 0 |
| 5100.0001.0002 | Evidence Index Schema | Phase 0 |
| 5100.0001.0003 | Offline Bundle Manifest | Phase 0 |
| 5100.0001.0004 | Golden Corpus Expansion | Phase 0 |
| 5100.0002.0001 | Canonicalization Utilities | Phase 1 |
| 5100.0002.0002 | Replay Runner Service | Phase 1 |
| 5100.0002.0003 | Delta-Verdict Generator | Phase 1 |
| 5100.0003.0001 | SBOM Interop Round-Trip | Phase 2 |
| 5100.0003.0002 | No-Egress Enforcement | Phase 2 |
| 5100.0004.0001 | Unknowns Budget CI Gates | Phase 3 |
| 5100.0005.0001 | Router Chaos Suite | Phase 4 |
| 5100.0006.0001 | Audit Pack Export/Import | Phase 5 |
Documentation Updated:
docs/implplan/SPRINT_5100_0000_0000_epic_summary.md- Master epic summarydocs/19_TEST_SUITE_OVERVIEW.md- Test suite documentationtests/AGENTS.md- AI agent guidance for tests directory
Key Concepts Implemented
- Deterministic Replay: Run Manifests capture all inputs for byte-identical verdict reproduction
- Canonical JSON: RFC 8785 principles for stable serialization
- Evidence Index: Linking verdicts to complete evidence chain
- Air-Gap Compliance: Network-isolated testing with
--network none - SBOM Interoperability: Round-trip testing with Syft, Grype, cosign
- Unknowns Budget Gates: Environment-based budget enforcement
- Router Backpressure: HTTP 429/503 with Retry-After validation
- Audit Packs: Sealed export/import for compliance verification
Release Blocking Gates
- Replay Verification: 0 byte diff
- Interop Suite: 95%+ findings parity
- Offline E2E: All pass with no network
- Unknowns Budget: Within configured limits
- Router Retry-After: 100% compliance
Processed by: Claude Code Date: 2025-12-21