# Testing Strategy Coverage Matrix

> **Purpose:** Visual map of test model requirements per module, quality gates, and sprint-to-model relationships.

---

## Module-to-Model Coverage Map

### Legend
- ✅ **Required** (from TEST_CATALOG.yml)
- 🟡 **Optional** (recommended but not mandatory)
- ⬜ **Not Applicable**

### Model Definitions (Quick Reference)
| Model | Description | Key Tests |
|-------|-------------|-----------|
| **L0** | Library/Core | Unit, property, snapshot, determinism |
| **S1** | Storage/Postgres | Integration, migrations, idempotency, query ordering |
| **T1** | Transport/Queue | Protocol roundtrip, fuzz invalid, delivery semantics, backpressure |
| **C1** | Connector/External | Fixtures, snapshot, resilience, security |
| **W1** | WebService/API | Contract, authz, OTel, negative |
| **WK1** | Worker/Indexer | End-to-end, retries, idempotency, OTel |
| **AN1** | Analyzer/SourceGen | Diagnostics, codefixes, golden generated |
| **CLI1** | Tool/CLI | Exit codes, golden output, determinism |
| **PERF** | Benchmarks | Benchmark, perf smoke, regression thresholds |

---

## Coverage Matrix

### Core Modules

| Module | L0 | S1 | T1 | C1 | W1 | WK1 | AN1 | CLI1 | PERF | Sprint | Tasks |
|--------|----|----|----|----|----|----|-----|------|------|--------|-------|
| **Scanner** | ✅ | ✅ | ✅ | ⬜ | ✅ | ✅ | ✅ | ⬜ | ✅ | 5100.0009.0001 | 25 |
| **Concelier** | ✅ | ✅ | ⬜ | ✅ | ✅ | ⬜ | ✅ | ⬜ | ⬜ | 5100.0009.0002 | 18 |
| **Excititor** | ✅ | ✅ | ⬜ | ✅ | ✅ | ✅ | ⬜ | ⬜ | ⬜ | 5100.0009.0003 | 21 |
| **Policy** | ✅ | ✅ | ⬜ | ⬜ | ✅ | ⬜ | ⬜ | ⬜ | ⬜ | 5100.0009.0004 | 15 |

### Security & Compliance Modules

| Module | L0 | S1 | T1 | C1 | W1 | WK1 | AN1 | CLI1 | PERF | Sprint | Tasks |
|--------|----|----|----|----|----|----|-----|------|------|--------|-------|
| **Authority** | ✅ | ⬜ | ⬜ | ✅ | ✅ | ⬜ | ⬜ | ⬜ | ⬜ | 5100.0009.0005 | 17 |
| **Signer** | ✅ | ⬜ | ⬜ | ✅ | ✅ | ⬜ | ⬜ | ⬜ | ⬜ | 5100.0009.0006 | 17 |
| **Attestor** | ✅ | ⬜ | ⬜ | ⬜ | ✅ | ⬜ | ⬜ | ⬜ | ⬜ | 5100.0009.0007 | 14 |

### Platform Services

| Module | L0 | S1 | T1 | C1 | W1 | WK1 | AN1 | CLI1 | PERF | Sprint | Tasks |
|--------|----|----|----|----|----|----|-----|------|------|--------|-------|
| **Scheduler** | ✅ | ✅ | ⬜ | ⬜ | ✅ | ✅ | ⬜ | ⬜ | ⬜ | 5100.0009.0008 | 14 |
| **Notify** | ✅ | ✅ | ⬜ | ✅ | ✅ | ✅ | ⬜ | ⬜ | ⬜ | 5100.0009.0009 | 18 |

### Client Interfaces

| Module | L0 | S1 | T1 | C1 | W1 | WK1 | AN1 | CLI1 | PERF | Sprint | Tasks |
|--------|----|----|----|----|----|----|-----|------|------|--------|-------|
| **CLI** | ⬜ | ⬜ | ⬜ | ⬜ | ⬜ | ⬜ | ⬜ | ✅ | ⬜ | 5100.0009.0010 | 13 |
| **UI** | ⬜ | ⬜ | ⬜ | ⬜ | ✅ | ⬜ | ⬜ | ⬜ | ⬜ | 5100.0009.0011 | 13 |

### Infrastructure & Platform

| Module | L0 | S1 | T1 | C1 | W1 | WK1 | AN1 | CLI1 | PERF | Sprint | Tasks |
|--------|----|----|----|----|----|----|-----|------|------|--------|-------|
| **EvidenceLocker** | ✅ | ✅ | ⬜ | ⬜ | ✅ | ⬜ | ⬜ | ⬜ | ⬜ | 5100.0010.0001 | 16 |
| **Graph/Timeline** | ✅ | ✅ | ⬜ | ⬜ | ✅ | ✅ | ⬜ | ⬜ | ⬜ | 5100.0010.0002 | 15 |
| **Router/Messaging** | ✅ | ✅ | ✅ | ⬜ | ✅ | ⬜ | ⬜ | ⬜ | ⬜ | 5100.0010.0003 | 14 |
| **AirGap** | ✅ | ✅ | ⬜ | ⬜ | ✅ | ⬜ | ✅ | ✅ | ⬜ | 5100.0010.0004 | 17 |

---

## Model Distribution Analysis

### Models by Usage Frequency

| Model | Modules Using | Percentage | Complexity |
|-------|---------------|------------|------------|
| **L0** (Library/Core) | 13/15 modules | 87% | HIGH (property tests, snapshots) |
| **W1** (WebService) | 13/15 modules | 87% | MEDIUM (contract tests, auth) |
| **S1** (Storage) | 10/15 modules | 67% | HIGH (migrations, idempotency) |
| **C1** (Connectors) | 5/15 modules | 33% | MEDIUM (fixtures, resilience) |
| **WK1** (Workers) | 5/15 modules | 33% | MEDIUM (end-to-end, retries) |
| **AN1** (Analyzers) | 3/15 modules | 20% | HIGH (Roslyn, diagnostics) |
| **T1** (Transport) | 2/15 modules | 13% | HIGH (protocol compliance) |
| **CLI1** (CLI Tools) | 2/15 modules | 13% | LOW (exit codes, snapshots) |
| **PERF** (Performance) | 1/15 modules | 7% | MEDIUM (benchmarks, regression) |

### Complexity Heatmap

**High Complexity (>15 tasks per sprint):**
- Scanner (25 tasks: L0+AN1+S1+T1+W1+WK1+PERF)
- Excititor (21 tasks: C1+L0+S1+W1+WK1)
- Concelier (18 tasks: C1+L0+S1+W1+AN1)
- Notify (18 tasks: L0+C1+S1+W1+WK1)
- Authority (17 tasks: L0+W1+C1)
- Signer (17 tasks: L0+W1+C1)
- AirGap (17 tasks: L0+AN1+S1+W1+CLI1)

**Medium Complexity (10-15 tasks):**
- Policy (15 tasks: L0+S1+W1)
- EvidenceLocker (16 tasks: L0+S1+W1)
- Graph/Timeline (15 tasks: L0+S1+W1+WK1)
- Scheduler (14 tasks: L0+S1+W1+WK1)
- Attestor (14 tasks: L0+W1)
- Router/Messaging (14 tasks: L0+T1+W1+S1)
- CLI (13 tasks: CLI1)
- UI (13 tasks: W1)

---

## Quality Gate Coverage

### Module-Specific Quality Gates (from TEST_CATALOG.yml)

| Module | Quality Gates | Enforced By |
|--------|---------------|-------------|
| **Scanner** | determinism, reachability_evidence, proof_spine | Sprint 5100.0009.0001 Tasks 7-10, 23-25 |
| **Concelier** | fixture_coverage, normalization_determinism, no_lattice_dependency | Sprint 5100.0009.0002 Tasks 1-7, 8-10, 18 |
| **Excititor** | preserve_prune_source, format_snapshots, no_lattice_dependency | Sprint 5100.0009.0003 Tasks 6-11, 21 |
| **Policy** | unknown_budget, verdict_snapshot | Sprint 5100.0009.0004 Tasks 2, 4, 14-15 |
| **Authority** | scope_enforcement, sign_verify | Sprint 5100.0009.0005 Tasks 3-5, 16-17 |
| **Signer** | canonical_payloads, sign_verify | Sprint 5100.0009.0006 Tasks 1-3, 15-17 |
| **Attestor** | rekor_receipts, dsse_verify | Sprint 5100.0009.0007 Tasks 6-8, 2 |
| **Scheduler** | idempotent_jobs, retry_backoff | Sprint 5100.0009.0008 Tasks 4, 3, 12 |
| **Notify** | connector_snapshots, retry_semantics | Sprint 5100.0009.0009 Tasks 1-6, 16 |
| **CLI** | exit_codes, stdout_snapshots | Sprint 5100.0009.0010 Tasks 1-4, 5-8 |
| **UI** | contract_snapshots, e2e_smoke | Sprint 5100.0009.0011 Tasks 1-2, 7-10 |

### Cross-Cutting Quality Gates

| Gate | Applies To | Enforced By |
|------|-----------|-------------|
| **Determinism Contract** | Scanner, Excititor, Signer, CLI, AirGap, Concelier | Sprint 5100.0007.0003 (Determinism Gate) |
| **Architecture Boundaries** | Concelier, Excititor (must NOT reference Scanner lattice) | Sprint 5100.0007.0007 (Architecture Tests) |
| **Contract Stability** | All WebServices (13 modules) | Sprint 5100.0007.0006 (WebService Contract) |
| **Storage Idempotency** | All S1 modules (10 modules) | Sprint 5100.0007.0004 (Storage Harness) |
| **Connector Resilience** | All C1 modules (5 modules) | Sprint 5100.0007.0005 (Connector Fixtures) |

---

## CI Lane Coverage

### Test Distribution Across CI Lanes

| CI Lane | Models | Modules | Sprint Tasks | Est. Runtime |
|---------|--------|---------|--------------|--------------|
| **Unit** | L0, AN1, CLI1 | All 15 modules | ~120 tasks | <5 min |
| **Contract** | W1 | 13 modules | ~50 tasks | <2 min |
| **Integration** | S1, WK1, T1 | 12 modules | ~100 tasks | 10-15 min |
| **Security** | C1 (security tests), W1 (auth tests) | 5 connectors + 13 WebServices | ~60 tasks | 5-10 min |
| **Performance** | PERF | Scanner only | ~3 tasks | 3-5 min |
| **Live** | C1 (live smoke tests) | Concelier, Excititor, Notify, Authority, Signer | ~5 tasks (opt-in) | 5-10 min (nightly) |

### CI Lane Dependencies

```
PR Gate (Must Pass):
├─ Unit Lane (L0, AN1, CLI1) ← Fast feedback
├─ Contract Lane (W1) ← API stability
├─ Architecture Lane (Sprint 5100.0007.0007) ← Boundary enforcement
└─ Integration Lane (S1, WK1, T1) ← Testcontainers

Merge Gate (Must Pass):
├─ All PR Gate lanes
├─ Security Lane (C1 security, W1 auth)
└─ Determinism Lane (Sprint 5100.0007.0003)

Nightly (Optional):
├─ Performance Lane (PERF)
└─ Live Lane (C1 live smoke)

Weekly (Optional):
└─ Competitor Parity (Sprint 5100.0008.0001)
```

---

## Epic-to-Model Coverage

### Epic Sprints Support Multiple Models

| Epic Sprint | Models Enabled | Consuming Modules | Tasks |
|-------------|----------------|-------------------|-------|
| **5100.0007.0002 (TestKit)** | ALL (L0, S1, T1, C1, W1, WK1, AN1, CLI1, PERF) | ALL 15 modules | 13 |
| **5100.0007.0003 (Determinism)** | L0 (determinism), CLI1 (determinism) | Scanner, Excititor, Signer, CLI, AirGap, Concelier | 12 |
| **5100.0007.0004 (Storage)** | S1 | 10 modules | 12 |
| **5100.0007.0005 (Connectors)** | C1 | Concelier, Excititor, Authority, Signer, Notify | 12 |
| **5100.0007.0006 (WebService)** | W1 | 13 modules | 12 |
| **5100.0007.0007 (Architecture)** | (Cross-cutting) | Concelier, Excititor | 17 |

---

## Test Type Distribution

### By Test Category (Trait)

| Test Category | Model Coverage | Estimated Test Count | CI Lane |
|---------------|----------------|----------------------|---------|
| **Unit** | L0, AN1 | ~150 tests across 13 modules | Unit |
| **Property** | L0 (subset) | ~40 tests (Scanner, Policy, Scheduler, Router) | Unit |
| **Snapshot** | L0, C1, CLI1 | ~80 tests (all modules with canonical outputs) | Unit/Contract |
| **Integration** | S1, WK1, T1 | ~120 tests across 12 modules | Integration |
| **Contract** | W1 | ~50 tests (13 WebServices × avg 4 endpoints) | Contract |
| **Security** | C1 (security), W1 (auth) | ~60 tests | Security |
| **Performance** | PERF | ~3 tests (Scanner only) | Performance |
| **Live** | C1 (live smoke) | ~5 tests (opt-in, nightly) | Live |

---

## Coverage Gaps & Recommendations

### Current Gaps

1. **Performance Testing:** Only Scanner has PERF model
   - **Recommendation:** Add PERF to Policy (policy evaluation latency), Concelier (merge performance), Scheduler (scheduling overhead)

2. **Transport Testing:** Only Router/Messaging has T1 model
   - **Recommendation:** Scanner has T1 in TEST_CATALOG.yml but should validate Valkey transport for job queues

3. **Live Connector Tests:** Only 5 modules have C1 live smoke tests (opt-in)
   - **Recommendation:** Run weekly, not nightly; treat as early warning system for schema drift

### Recommended Additions (Future Sprints)

| Module | Missing Model | Justification | Priority |
|--------|---------------|---------------|----------|
| Policy | PERF | Policy evaluation latency critical for real-time decisioning | HIGH |
| Concelier | PERF | Merge performance affects ingestion throughput | MEDIUM |
| Scheduler | PERF | Scheduling overhead affects job execution latency | MEDIUM |
| Scanner | T1 (validate) | Job queue transport (Valkey) should have compliance tests | HIGH |
| Authority | S1 | Token storage/revocation should have migration tests | MEDIUM |

---

## Summary Statistics

**Total Test Models:** 9
**Total Modules Covered:** 15
**Total Module Test Sprints:** 15 (11 module + 4 infrastructure)
**Total Epic Sprints:** 6
**Total Quality Gate Sprints:** 1 (Competitor Parity)

**Model Usage:**
- L0: 13 modules (87%)
- W1: 13 modules (87%)
- S1: 10 modules (67%)
- C1: 5 modules (33%)
- WK1: 5 modules (33%)
- AN1: 3 modules (20%)
- T1: 2 modules (13%)
- CLI1: 2 modules (13%)
- PERF: 1 module (7%)

**Estimated Total Tests:** ~500 tests across all modules and models

---

**Prepared by:** Project Management
**Date:** 2025-12-23
**Next Review:** 2026-01-06 (Week 1 kickoff)
**Source:** `docs/testing/TEST_CATALOG.yml`, Sprint files 5100.0009.* and 5100.0010.*