test fixes and new product advisories work

2026-01-28 02:30:48 +02:00
parent 82caceba56
commit 644887997c
288 changed files with 69101 additions and 375 deletions
--- a/docs/technical/testing/TESTING_MASTER_PLAN.md
+++ b/docs/technical/testing/TESTING_MASTER_PLAN.md
@@ -391,10 +391,12 @@ ONGOING: QUALITY GATES (Weeks 3-14+)

 1. **Advisory:** `docs/product/advisories/22-Dec-2026 - Better testing strategy.md`
 2. **Test Catalog:** `docs/technical/testing/TEST_CATALOG.yml`
-3. **Test Models:** `docs/technical/testing/testing-strategy-models.md`
+3. **Test Models:** `docs/technical/testing/testing-strategy-models.md` (includes Turn #6 enhancements: intent tagging, observability contracts, evidence traceability, longevity, interop)
 4. **Dependency Graph:** `docs/technical/testing/SPRINT_DEPENDENCY_GRAPH.md`
 5. **Coverage Matrix:** `docs/technical/testing/TEST_COVERAGE_MATRIX.md`
 6. **Execution Playbook:** `docs/technical/testing/SPRINT_EXECUTION_PLAYBOOK.md`
+7. **Testing Practices:** `docs/code-of-conduct/TESTING_PRACTICES.md` (Turn #6 mandatory practices)
+8. **CI Quality Gates:** `docs/technical/testing/ci-quality-gates.md` (Turn #6 gates)

 ### Appendix C: Budget Estimate (Preliminary)

--- a/docs/technical/testing/TEST_COVERAGE_MATRIX.md
+++ b/docs/technical/testing/TEST_COVERAGE_MATRIX.md
@@ -256,7 +256,84 @@ Weekly (Optional):

 ---

+## Turn #6 Testing Enhancements Coverage
+
+### New Coverage Dimensions (Sprint 0127.002)
+
+The following dimensions track adoption of Turn #6 testing practices across modules:
+
+| Dimension | Description | Target Coverage |
+|-----------|-------------|-----------------|
+| **Intent Tags** | Tests with `[Intent]` attribute declaring regulatory/safety/performance/competitive/operational | 100% non-trivial tests in Policy, Authority, Signer, Attestor |
+| **Observability Contracts** | W1 tests with OTel schema validation, log field contracts | 100% of W1 tests |
+| **Evidence Traceability** | Tests with `[Requirement]` attribute linking to requirements | 100% of regulatory-tagged tests |
+| **Longevity Tests** | Memory stability, counter drift, connection pool tests | Scanner, Scheduler, Notify workers |
+| **Interop Tests** | N-1/N+1 version compatibility tests | EvidenceLocker, Policy (schema-dependent) |
+| **Environment Skew** | Tests across infrastructure profiles (network latency, resource limits) | Integration tests |
+
+### Turn #6 Coverage Matrix
+
+| Module | Intent Tags | Observability | Evidence | Longevity | Interop | Skew |
+|--------|-------------|---------------|----------|-----------|---------|------|
+| **Policy** | ✅ Pilot | 🟡 | ✅ Pilot | 🟡 | 🟡 | ⬜ |
+| **EvidenceLocker** | 🟡 | 🟡 | ✅ Pilot | 🟡 | ✅ | 🟡 |
+| **Scanner** | 🟡 | ✅ Pilot | 🟡 | ✅ | 🟡 | 🟡 |
+| **Authority** | 🟡 | 🟡 | 🟡 | ⬜ | 🟡 | ⬜ |
+| **Signer** | 🟡 | 🟡 | 🟡 | ⬜ | 🟡 | ⬜ |
+| **Attestor** | 🟡 | 🟡 | 🟡 | ⬜ | 🟡 | ⬜ |
+| **Scheduler** | 🟡 | 🟡 | 🟡 | ✅ | ⬜ | 🟡 |
+| **Notify** | 🟡 | 🟡 | 🟡 | ✅ | ⬜ | ⬜ |
+
+**Legend:**
+- ✅ Pilot implementation complete
+- 🟡 Recommended, not yet implemented
+- ⬜ Not applicable
+
+### Turn #6 TestKit Components
+
+| Component | Location | Purpose | Status |
+|-----------|----------|---------|--------|
+| `IntentAttribute` | `TestKit/Traits/IntentAttribute.cs` | Tag tests with intent | ✅ Complete |
+| `IntentAnalyzer` | `TestKit.Analyzers/IntentAnalyzer.cs` | Detect missing intent tags | ✅ Complete |
+| `OTelContractAssert` | `TestKit/Observability/OTelContractAssert.cs` | Span/attribute validation | ✅ Complete |
+| `LogContractAssert` | `TestKit/Observability/LogContractAssert.cs` | Log field validation | ✅ Complete |
+| `MetricsContractAssert` | `TestKit/Observability/MetricsContractAssert.cs` | Cardinality bounds | ✅ Complete |
+| `RequirementAttribute` | `TestKit/Evidence/RequirementAttribute.cs` | Link tests to requirements | ✅ Complete |
+| `EvidenceChainAssert` | `TestKit/Evidence/EvidenceChainAssert.cs` | Hash/immutability validation | ✅ Complete |
+| `EvidenceChainReporter` | `TestKit/Evidence/EvidenceChainReporter.cs` | Traceability matrix | ✅ Complete |
+| `IncidentTestGenerator` | `TestKit/Incident/IncidentTestGenerator.cs` | Post-incident test scaffolds | ✅ Complete |
+| `SchemaVersionMatrix` | `TestKit/Interop/SchemaVersionMatrix.cs` | Version compatibility | ✅ Complete |
+| `VersionCompatibilityFixture` | `TestKit/Interop/VersionCompatibilityFixture.cs` | N-1/N+1 testing | ✅ Complete |
+| `StabilityMetrics` | `TestKit/Longevity/StabilityMetrics.cs` | Memory/counter tracking | ✅ Complete |
+| `StabilityTestRunner` | `TestKit/Longevity/StabilityTestRunner.cs` | Time-extended tests | ✅ Complete |
+| `EnvironmentProfile` | `TestKit/Environment/EnvironmentProfile.cs` | Infrastructure profiles | ✅ Complete |
+| `SkewTestRunner` | `TestKit/Environment/SkewTestRunner.cs` | Cross-profile testing | ✅ Complete |
+
+### Turn #6 Test Categories
+
+New categories added to `TestCategories.cs`:
+
+| Category | Filter | CI Lane | Gating |
+|----------|--------|---------|--------|
+| `PostIncident` | `Category=PostIncident` | Release | P1/P2 block |
+| `EvidenceChain` | `Category=EvidenceChain` | Merge | Block |
+| `Longevity` | `Category=Longevity` | Nightly | Warning |
+| `Interop` | `Category=Interop` | Release | Block |
+| `EnvironmentSkew` | `Category=EnvironmentSkew` | Nightly | Warning |
+
+### Coverage Targets (End of Q1 2026)
+
+| Dimension | Current Baseline | Target | Tracking |
+|-----------|------------------|--------|----------|
+| Intent Tags (Policy, Authority, Signer, Attestor) | 5 tests | 100% non-trivial | `IntentCoverageReport` |
+| Observability Contracts (W1 tests) | 5 tests | 100% | `OTelContractAssert` usage |
+| Evidence Traceability (Regulatory tests) | 3 tests | 100% | `EvidenceChainReporter` |
+| Longevity Tests (Worker modules) | 0 tests | 1 per worker | `StabilityTestRunner` usage |
+| Interop Tests (Schema modules) | 0 tests | 1 per schema | `SchemaVersionMatrix` usage |
+
+---
+
 **Prepared by:** Project Management
-**Date:** 2025-12-23
-**Next Review:** 2026-01-06 (Week 1 kickoff)
-**Source:** `docs/technical/testing/TEST_CATALOG.yml`, Sprint files 5100.0009.* and 5100.0010.*
+**Date:** 2026-01-27
+**Next Review:** 2026-02-03 (Turn #6 adoption review)
+**Source:** `docs/technical/testing/TEST_CATALOG.yml`, Sprint files 5100.0009.* and 5100.0010.*, SPRINT_0127_002_DOCS_testing_enhancements_turn6.md
--- a/docs/technical/testing/ci-quality-gates.md
+++ b/docs/technical/testing/ci-quality-gates.md
@@ -147,6 +147,158 @@ If baselines become stale:
 ./scripts/ci/compute-reachability-metrics.sh --update-baseline
 ```

+---
+
+## Turn #6 Quality Gates (2026-01-27)
+
+Source: Testing Enhancements (Automation Turn #6)
+Sprint: `docs/implplan/SPRINT_0127_002_DOCS_testing_enhancements_turn6.md`
+
+### Intent Violation Gate
+
+**Purpose:** Detect test changes that violate declared intent categories.
+
+**Script:** `scripts/ci/check-intent-violations.sh`
+
+| Check | Description | Action |
+|-------|-------------|--------|
+| Intent missing | Non-trivial test without Intent trait | Warning (regulatory modules: Error) |
+| Intent contradiction | Test behavior contradicts declared intent | Error |
+| Intent coverage drop | Module intent coverage decreased | Warning |
+
+**Enforcement:**
+- PR-gating for regulatory modules (Policy, Authority, Signer, Attestor, EvidenceLocker).
+- Warning-only for other modules (to allow gradual adoption).
+
+### Observability Contract Gate
+
+**Purpose:** Validate OTel spans, structured logs, and metrics contracts.
+
+**Script:** `scripts/ci/check-observability-contracts.sh`
+
+| Check | Description | Threshold |
+|-------|-------------|-----------|
+| Required spans missing | Core operation spans not emitted | Error |
+| Span attribute missing | Required attributes not present | Error |
+| High cardinality attribute | Label cardinality exceeds limit | Warning (> 50), Error (> 100) |
+| PII in logs | Sensitive data patterns in log output | Error |
+| Missing log fields | Required fields not present | Warning |
+
+**Enforcement:**
+- PR-gating for all W1 (WebService) modules.
+- Run as part of contract test lane.
+
+### Evidence Chain Gate
+
+**Purpose:** Verify requirement -> test -> artifact traceability.
+
+**Script:** `scripts/ci/check-evidence-chain.sh`
+
+| Check | Description | Action |
+|-------|-------------|--------|
+| Orphaned test | Regulatory test without Requirement attribute | Warning |
+| Artifact hash drift | Artifact hash changed unexpectedly | Error |
+| Artifact non-deterministic | Multiple runs produce different artifacts | Error |
+| Traceability gap | Requirement without test coverage | Warning |
+
+**Enforcement:**
+- PR-gating for regulatory modules.
+- Traceability report generated as CI artifact.
+
+### Longevity Gate (Release Gating)
+
+**Purpose:** Detect memory leaks, connection leaks, and counter drift under sustained load.
+
+**Script:** `scripts/ci/run-longevity-gate.sh`
+**Cadence:** Nightly + pre-release
+
+| Metric | Description | Threshold |
+|--------|-------------|-----------|
+| Memory growth rate | Memory increase per hour | ≤ 1% |
+| Connection pool leaks | Unreturned connections | 0 |
+| Counter drift | Counter value outside expected range | Error |
+| GC pressure | Gen2 collections per hour | ≤ 10 |
+
+**Enforcement:**
+- Not PR-gating (too slow).
+- Release-gating: longevity tests must pass before release.
+- Results stored for trend analysis.
+
+### Interop Gate (Release Gating)
+
+**Purpose:** Validate cross-version and environment compatibility.
+
+**Script:** `scripts/ci/run-interop-gate.sh`
+**Cadence:** Weekly + pre-release
+
+| Check | Description | Threshold |
+|-------|-------------|-----------|
+| N-1 compatibility | Current server with previous client | Must pass |
+| N+1 compatibility | Previous server with current client | Must pass |
+| Environment equivalence | Same results across infra profiles | ≤ 5% deviation |
+
+**Profiles Tested:**
+- `standard`: default Testcontainers configuration.
+- `high-latency`: +100ms network latency.
+- `low-bandwidth`: 10 Mbps limit.
+- `packet-loss`: 1% packet loss (Linux only).
+
+**Enforcement:**
+- Not PR-gating (requires multi-version infrastructure).
+- Release-gating: interop tests must pass before release.
+
+### Post-Incident Gate
+
+**Purpose:** Ensure incident-derived tests are maintained and passing.
+
+**Script:** `scripts/ci/check-post-incident-tests.sh`
+
+| Check | Description | Action |
+|-------|-------------|--------|
+| Incident test failing | PostIncident test not passing | Error (P1/P2), Warning (P3) |
+| Incident test missing metadata | Missing IncidentId or RootCause | Warning |
+| Incident coverage | P1/P2 incidents without tests | Error |
+
+**Enforcement:**
+- PR-gating: P1/P2 incident tests must pass.
+- Release-gating: all incident tests must pass.
+
+---
+
+## Gate Summary by Gating Level
+
+### PR-Gating (Must Pass for Merge)
+
+| Gate | Scope |
+|------|-------|
+| Reachability Quality | All |
+| TTFS Regression | All |
+| Intent Violation | Regulatory modules |
+| Observability Contract | W1 modules |
+| Evidence Chain | Regulatory modules |
+| Post-Incident (P1/P2) | All |
+
+### Release-Gating (Must Pass for Release)
+
+| Gate | Scope |
+|------|-------|
+| All PR gates | All |
+| Longevity | Worker modules |
+| Interop | Schema/API-dependent modules |
+| Post-Incident (all) | All |
+| Performance SLO | All |
+
+### Warning-Only (Informational)
+
+| Gate | Scope |
+|------|-------|
+| Intent missing | Non-regulatory modules |
+| Intent coverage drop | All |
+| Orphaned test | All |
+| Traceability gap | All |
+
+---
+
 ## Related Documentation

 - [Test Suite Overview](../TEST_SUITE_OVERVIEW.md)
@@ -155,3 +307,4 @@ If baselines become stale:
 - [Reachability Corpus Plan](../reachability/corpus-plan.md)
 - [Performance Workbook](../PERFORMANCE_WORKBOOK.md)
 - [Testing Quality Guardrails](./testing-quality-guardrails-implementation.md)
+- [Testing Practices](../../code-of-conduct/TESTING_PRACTICES.md)
--- a/docs/technical/testing/post-incident-testing-guide.md
+++ b/docs/technical/testing/post-incident-testing-guide.md
@@ -0,0 +1,324 @@
+# Post-Incident Testing Guide
+
+**Version:** 1.0
+**Status:** Turn #6 Implementation
+**Audience:** StellaOps developers, QA engineers, incident responders
+
+---
+
+## Overview
+
+Every production incident should produce a permanent regression test. This guide describes the infrastructure and workflow for generating, reviewing, and maintaining post-incident tests in the StellaOps codebase.
+
+### Key Principles
+
+1. **Permanent Regression**: Incidents that reach production indicate a gap in testing. That gap must be permanently closed.
+2. **Deterministic Replay**: Tests are generated from replay manifests captured during the incident.
+3. **Severity-Gated**: P1/P2 incident tests block releases; P3/P4 tests are warning-only.
+4. **Traceable**: Every incident test links back to the incident report and fix.
+
+---
+
+## Workflow
+
+### 1. Incident Triggers Replay Capture
+
+When an incident occurs, the replay infrastructure automatically captures:
+
+- Event sequences with correlation IDs
+- Input data (sanitized for PII)
+- System state at time of incident
+- Configuration and policy digests
+
+This produces a **replay manifest** stored in the Evidence Locker.
+
+### 2. Generate Test Scaffold
+
+Use the `IncidentTestGenerator` to create a test scaffold from the replay manifest:
+
+```csharp
+using StellaOps.TestKit.Incident;
+
+// Load the replay manifest
+var manifestJson = File.ReadAllText("incident-replay-manifest.json");
+
+// Create incident metadata
+var metadata = new IncidentMetadata
+{
+    IncidentId = "INC-2026-001",
+    OccurredAt = DateTimeOffset.Parse("2026-01-15T10:30:00Z"),
+    RootCause = "Race condition in concurrent bundle creation",
+    AffectedModules = ["EvidenceLocker", "Policy"],
+    Severity = IncidentSeverity.P1,
+    Title = "Evidence bundle duplication in high-concurrency scenario",
+    ReportUrl = "https://incidents.stella-ops.internal/INC-2026-001"
+};
+
+// Generate the test scaffold
+var generator = new IncidentTestGenerator();
+var scaffold = generator.GenerateFromManifestJson(manifestJson, metadata);
+
+// Output the generated test code
+var code = scaffold.GenerateTestCode();
+File.WriteAllText($"Tests/{scaffold.TestClassName}.cs", code);
+```
+
+### 3. Review and Complete Test
+
+The generated scaffold is a starting point. A human must:
+
+1. **Review fixtures**: Ensure input data is appropriate and sanitized.
+2. **Complete assertions**: Add specific assertions for the expected behavior.
+3. **Verify determinism**: Ensure the test produces consistent results.
+4. **Add to CI**: Include the test in the appropriate test project.
+
+### 4. Register for Tracking
+
+Register the incident test for reporting:
+
+```csharp
+generator.RegisterIncidentTest(metadata.IncidentId, scaffold);
+
+// Generate a summary report
+var report = generator.GenerateReport();
+Console.WriteLine($"Total incident tests: {report.TotalTests}");
+Console.WriteLine($"P1 tests: {report.BySeveority.GetValueOrDefault(IncidentSeverity.P1, 0)}");
+```
+
+---
+
+## Incident Metadata
+
+The `IncidentMetadata` record captures essential incident context:
+
+| Property | Required | Description |
+|----------|----------|-------------|
+| `IncidentId` | Yes | Unique identifier from incident management system |
+| `OccurredAt` | Yes | When the incident occurred (UTC) |
+| `RootCause` | Yes | Brief description of the root cause |
+| `AffectedModules` | Yes | Modules impacted by the incident |
+| `Severity` | Yes | P1 (critical) through P4 (low impact) |
+| `Title` | No | Short descriptive title |
+| `ReportUrl` | No | Link to incident report or postmortem |
+| `ResolvedAt` | No | When the incident was resolved |
+| `CorrelationIds` | No | IDs for replay matching |
+| `FixTaskId` | No | Sprint task that implemented the fix |
+| `Tags` | No | Categorization tags |
+
+### Severity Levels
+
+| Severity | Description | CI Behavior |
+|----------|-------------|-------------|
+| P1 | Critical: service down, data loss, security breach | Blocks releases |
+| P2 | Major: significant degradation, partial outage | Blocks releases |
+| P3 | Minor: limited impact, workaround available | Warning only |
+| P4 | Low: cosmetic issues, minor bugs | Informational |
+
+---
+
+## Generated Test Structure
+
+The scaffold generates a test class with:
+
+```csharp
+[Trait("Category", TestCategories.PostIncident)]
+[Trait("Incident", "INC-2026-001")]
+[Trait("Severity", "P1")]
+public sealed class Incident_INC_2026_001_Tests
+{
+    private static readonly IncidentMetadata Incident = new()
+    {
+        IncidentId = "INC-2026-001",
+        OccurredAt = DateTimeOffset.Parse("2026-01-15T10:30:00Z"),
+        RootCause = "Race condition in concurrent bundle creation",
+        AffectedModules = ["EvidenceLocker", "Policy"],
+        Severity = IncidentSeverity.P1,
+        Title = "Evidence bundle duplication"
+    };
+
+    [Fact]
+    public async Task Validates_RaceCondition_Fix()
+    {
+        // Arrange
+        // TODO: Load fixtures from replay manifest
+
+        // Act
+        // TODO: Execute the scenario that triggered the incident
+
+        // Assert
+        // TODO: Verify the fix prevents the incident condition
+    }
+}
+```
+
+---
+
+## CI Integration
+
+### Test Filtering
+
+Filter post-incident tests in CI:
+
+```bash
+# Run all post-incident tests
+dotnet test --filter "Category=PostIncident"
+
+# Run only P1/P2 tests (release-gating)
+dotnet test --filter "Category=PostIncident&(Severity=P1|Severity=P2)"
+
+# Run tests for a specific incident
+dotnet test --filter "Incident=INC-2026-001"
+
+# Run tests for a specific module
+dotnet test --filter "Category=PostIncident&Module:EvidenceLocker=true"
+```
+
+### CI Lanes
+
+| Lane | Filter | Trigger | Behavior |
+|------|--------|---------|----------|
+| PR Gate | `Category=PostIncident&(Severity=P1\|Severity=P2)` | Pull requests | Blocks merge |
+| Release Gate | `Category=PostIncident` | Release builds | P1/P2 block, P3/P4 warn |
+| Nightly | `Category=PostIncident` | Scheduled | Full run, report only |
+
+### Example CI Configuration
+
+```yaml
+# .gitea/workflows/post-incident-tests.yml
+name: Post-Incident Tests
+on:
+  pull_request:
+  release:
+    types: [created]
+
+jobs:
+  post-incident:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-dotnet@v4
+        with:
+          dotnet-version: '10.0.x'
+
+      - name: Run P1/P2 Incident Tests
+        run: |
+          dotnet test --filter "Category=PostIncident&(Severity=P1|Severity=P2)" \
+            --logger "trx;LogFileName=incident-results.trx"
+
+      - name: Upload Results
+        uses: actions/upload-artifact@v4
+        with:
+          name: incident-test-results
+          path: '**/incident-results.trx'
+```
+
+---
+
+## Best Practices
+
+### 1. Sanitize Fixtures
+
+Remove or mask any PII or sensitive data from replay fixtures:
+
+```csharp
+// Before storing fixture
+var sanitizedFixture = fixture
+    .Replace(userEmail, "user@example.com")
+    .Replace(apiKey, "REDACTED");
+```
+
+### 2. Use Deterministic Infrastructure
+
+Ensure incident tests use TestKit's deterministic primitives:
+
+```csharp
+// Use deterministic time
+using var time = new DeterministicTime(Incident.OccurredAt);
+
+// Use deterministic random if needed
+var random = new DeterministicRandom(seed: 42);
+```
+
+### 3. Document the Incident
+
+Include comprehensive documentation in the test:
+
+```csharp
+/// <summary>
+/// Regression test for incident INC-2026-001: Evidence bundle duplication.
+/// </summary>
+/// <remarks>
+/// Root cause: Race condition in concurrent bundle creation.
+///
+/// The incident occurred when multiple workers attempted to create the same
+/// evidence bundle simultaneously. The fix added optimistic locking with
+/// a unique constraint on (tenant_id, bundle_id).
+///
+/// Report: https://incidents.stella-ops.internal/INC-2026-001
+/// Fix: PR #1234
+/// </remarks>
+```
+
+### 4. Link to Sprint Tasks
+
+Connect incident tests to the fix implementation:
+
+```csharp
+[Fact]
+[Trait("SprintTask", "EVIDENCE-0115-001")]
+public async Task Validates_RaceCondition_Fix()
+```
+
+### 5. Evolve Tests Over Time
+
+Incident tests may need updates as the codebase evolves:
+
+- Update fixtures when schemas change
+- Adjust assertions when behavior intentionally changes
+- Add new scenarios discovered during subsequent incidents
+
+---
+
+## Troubleshooting
+
+### Manifest Not Available
+
+If the replay manifest wasn't captured:
+
+1. Check Evidence Locker for any captured events
+2. Reconstruct the scenario from logs and metrics
+3. Create a synthetic manifest for testing
+
+### Flaky Incident Tests
+
+If the test is non-deterministic:
+
+1. Identify non-deterministic inputs (time, random, external state)
+2. Replace with TestKit deterministic primitives
+3. Add retry logic only as a last resort
+
+### Test No Longer Relevant
+
+If the fix makes the scenario impossible:
+
+1. Document why the test is no longer applicable
+2. Move to an "archived incidents" test category
+3. Keep the test for documentation purposes
+
+---
+
+## Related Documentation
+
+- [TestKit Usage Guide](testkit-usage-guide.md)
+- [Testing Practices](../../code-of-conduct/TESTING_PRACTICES.md)
+- [CI Quality Gates](ci-quality-gates.md)
+- [Replay Infrastructure](../../modules/replay/architecture.md)
+
+---
+
+## Changelog
+
+### v1.0 (2026-01-27)
+- Initial release: IncidentTestGenerator, IncidentMetadata, TestScaffold
+- CI integration patterns
+- Best practices and troubleshooting
--- a/docs/technical/testing/testing-strategy-models.md
+++ b/docs/technical/testing/testing-strategy-models.md
@@ -50,3 +50,115 @@ Supersedes/extends: `docs/product/advisories/archived/2025-12-21-testing-strateg
 - Test suite overview: `docs/technical/testing/TEST_SUITE_OVERVIEW.md`
 - Quality guardrails: `docs/technical/testing/testing-quality-guardrails-implementation.md`
 - Code samples from the advisory: `docs/benchmarks/testing/better-testing-strategy-samples.md`
+
+---
+
+## Turn #6 Enhancements (2026-01-27)
+
+Source advisory: Testing Enhancements (Automation Turn #6)
+Sprint: `docs/implplan/SPRINT_0127_002_DOCS_testing_enhancements_turn6.md`
+
+### New test intent categories
+
+Every non-trivial test must declare intent. Intent clarifies *why* the behavior exists.
+
+```csharp
+public static class TestIntents
+{
+    public const string Regulatory = "Regulatory";   // Compliance, audit, legal
+    public const string Safety = "Safety";           // Security, fail-secure, crypto
+    public const string Performance = "Performance"; // Latency, throughput, resources
+    public const string Competitive = "Competitive"; // Parity with competitor tools
+    public const string Operational = "Operational"; // Observability, operability
+}
+
+// Usage
+[Trait("Intent", TestIntents.Safety)]
+[Trait("Category", "Unit")]
+public void Signer_RejectsExpiredCertificate() { /* ... */ }
+```
+
+### New test trait categories
+
+| Category | Purpose | Example Usage |
+|----------|---------|---------------|
+| `Intent` | Test intent classification | `[Trait("Intent", "Safety")]` |
+| `Evidence` | Evidence chain validation | `[Trait("Category", "Evidence")]` |
+| `Observability` | OTel/log/metrics contracts | `[Trait("Category", "Observability")]` |
+| `Longevity` | Time-extended stability tests | `[Trait("Category", "Longevity")]` |
+| `Interop` | Cross-version/environment skew | `[Trait("Category", "Interop")]` |
+| `PostIncident` | Tests from production incidents | `[Trait("Category", "PostIncident")]` |
+
+### Updated test model requirements
+
+| Model | Turn #6 Additions |
+|-------|-------------------|
+| L0 (Library/Core) | + Intent trait required for non-trivial tests |
+| S1 (Storage/Postgres) | + Interop tests for schema version migrations |
+| W1 (WebService/API) | + Observability contract tests (OTel spans, log fields, metrics) |
+| WK1 (Worker/Indexer) | + Longevity tests for memory/connection stability |
+| CLI1 (Tool/CLI) | + PostIncident regression tests |
+
+### New CI lanes
+
+| Lane | Purpose | Cadence | Gating |
+|------|---------|---------|--------|
+| Evidence | Evidence chain validation, traceability | Per PR | PR-gating for regulatory modules |
+| Longevity | Time-extended stability tests | Nightly | Release-gating |
+| Interop | Cross-version compatibility | Weekly + pre-release | Release-gating |
+
+### Observability contract requirements (W1 model)
+
+WebService tests must validate:
+- **OTel spans**: required spans exist, attributes present, cardinality bounded.
+- **Structured logs**: required fields present, no PII, appropriate log levels.
+- **Metrics**: required metrics exist, label cardinality bounded, counters monotonic.
+
+```csharp
+[Trait("Category", "Observability")]
+[Trait("Intent", "Operational")]
+public async Task Scanner_EmitsRequiredTelemetry()
+{
+    using var otel = new OtelCapture();
+    await sut.ScanAsync(request);
+
+    OTelContractAssert.HasRequiredSpans(otel, "ScanImage", "ExtractLayers", "AnalyzeSBOM");
+    OTelContractAssert.NoHighCardinalityAttributes(otel, threshold: 100);
+}
+```
+
+### Evidence traceability requirements
+
+Regulatory tests must link to requirements:
+```csharp
+[Requirement("REQ-EVIDENCE-001")]
+[Trait("Intent", "Regulatory")]
+public void EvidenceBundle_IsImmutableAfterSigning() { /* ... */ }
+```
+
+CI generates traceability matrix: requirement -> test -> artifact.
+
+### Cross-version testing requirements (Interop)
+
+For modules with schema or API versioning:
+- Test N-1 compatibility (current server, previous client).
+- Test N+1 compatibility (previous server, current client).
+- Document compatibility matrix.
+
+### Time-extended testing requirements (Longevity)
+
+For worker modules (WK1 model):
+- Memory stability: verify no growth under sustained load.
+- Connection pool stability: verify no leaks.
+- Counter drift: verify values remain bounded.
+
+Run duration: 1+ hours for nightly, 4+ hours for release validation.
+
+### Post-incident testing requirements
+
+For P1/P2 production incidents:
+1. Capture event sequence via replay infrastructure.
+2. Generate test scaffold from replay manifest.
+3. Include incident metadata (ID, root cause, severity).
+4. Tag with `[Trait("Category", "PostIncident")]`.
+5. Test failures block releases.
--- a/docs/technical/testing/testkit-usage-guide.md
+++ b/docs/technical/testing/testkit-usage-guide.md
@@ -373,7 +373,216 @@ public async Task Test_TracingBehavior()

 ---

-### 9. Test Categories
+### 9. Observability Contract Testing (Turn #6)
+
+Contract assertions for treating logs, metrics, and traces as APIs:
+
+**OTel Contract Testing:**
+
+```csharp
+using StellaOps.TestKit.Observability;
+
+[Fact, Trait("Category", TestCategories.Contract)]
+public async Task Test_SpanContracts()
+{
+    using var capture = new OtelCapture("MyService");
+
+    await service.ProcessRequestAsync();
+
+    // Verify required spans are present
+    OTelContractAssert.HasRequiredSpans(capture, "ProcessRequest", "ValidateInput", "SaveResult");
+
+    // Verify span attributes
+    var span = capture.CapturedActivities.First();
+    OTelContractAssert.SpanHasAttributes(span, "user_id", "tenant_id", "correlation_id");
+
+    // Check attribute cardinality (prevent metric explosion)
+    OTelContractAssert.AttributeCardinality(capture, "http_method", maxCardinality: 10);
+
+    // Detect high-cardinality attributes globally
+    OTelContractAssert.NoHighCardinalityAttributes(capture, threshold: 100);
+}
+```
+
+**Log Contract Testing:**
+
+```csharp
+using StellaOps.TestKit.Observability;
+using System.Text.RegularExpressions;
+
+[Fact]
+public async Task Test_LogContracts()
+{
+    var logCapture = new List<CapturedLogRecord>();
+    // ... capture logs during test execution ...
+
+    // Verify required fields
+    LogContractAssert.HasRequiredFields(logCapture[0], "CorrelationId", "TenantId");
+
+    // Ensure no PII leakage
+    var piiPatterns = new[]
+    {
+        new Regex(@"\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,}\b"),  // Email
+        new Regex(@"\b\d{3}-\d{2}-\d{4}\b"),  // SSN
+    };
+    LogContractAssert.NoSensitiveData(logCapture, piiPatterns);
+
+    // Verify log level appropriateness
+    LogContractAssert.LogLevelAppropriate(logCapture[0], LogLevel.Information, LogLevel.Warning);
+
+    // Ensure error logs have correlation for troubleshooting
+    LogContractAssert.ErrorLogsHaveCorrelation(logCapture, "CorrelationId", "RequestId");
+}
+```
+
+**Metrics Contract Testing:**
+
+```csharp
+using StellaOps.TestKit.Observability;
+
+[Fact]
+public async Task Test_MetricsContracts()
+{
+    using var capture = new MetricsCapture("MyService");
+
+    await service.ProcessMultipleRequests();
+
+    // Verify required metrics exist
+    MetricsContractAssert.HasRequiredMetrics(capture, "requests_total", "request_duration_seconds");
+
+    // Check label cardinality bounds
+    MetricsContractAssert.LabelCardinalityBounded(capture, "http_requests_total", maxLabels: 50);
+
+    // Verify counter monotonicity
+    MetricsContractAssert.CounterMonotonic(capture, "processed_items_total");
+
+    // Verify gauge bounds
+    MetricsContractAssert.GaugeInBounds(capture, "active_connections", minValue: 0, maxValue: 1000);
+}
+```
+
+**API Reference:**
+- `OTelContractAssert.HasRequiredSpans(capture, spanNames)` - Verify spans exist
+- `OTelContractAssert.SpanHasAttributes(span, attrNames)` - Verify attributes
+- `OTelContractAssert.AttributeCardinality(capture, attr, max)` - Check cardinality
+- `OTelContractAssert.NoHighCardinalityAttributes(capture, threshold)` - Detect explosion
+- `LogContractAssert.HasRequiredFields(record, fields)` - Verify log fields
+- `LogContractAssert.NoSensitiveData(records, patterns)` - Check for PII
+- `MetricsContractAssert.MetricExists(capture, name)` - Verify metric
+- `MetricsContractAssert.LabelCardinalityBounded(capture, name, max)` - Check cardinality
+- `MetricsCapture` - Capture metrics during test execution
+- `ContractViolationException` - Thrown when contracts are violated
+
+---
+
+### 10. Evidence Chain Traceability (Turn #6)
+
+Link tests to requirements for regulatory compliance and audit trails:
+
+**Requirement Attribute:**
+
+```csharp
+using StellaOps.TestKit.Evidence;
+
+[Fact]
+[Requirement("REQ-AUTH-001", SprintTaskId = "AUTH-0127-001")]
+public async Task Test_UserAuthentication()
+{
+    // Verify authentication works as required
+}
+
+[Fact]
+[Requirement("REQ-AUDIT-002", SprintTaskId = "AUDIT-0127-003", ComplianceControl = "SOC2-AU-12")]
+public void Test_AuditLogImmutability()
+{
+    // Verify audit logs cannot be modified
+}
+```
+
+**Filtering tests by requirement:**
+
+```bash
+# Run tests for a specific requirement
+dotnet test --filter "Requirement=REQ-AUTH-001"
+
+# Run tests for a sprint task
+dotnet test --filter "SprintTask=AUTH-0127-001"
+
+# Run tests for a compliance control
+dotnet test --filter "ComplianceControl=SOC2-AU-12"
+```
+
+**Evidence Chain Assertions:**
+
+```csharp
+using StellaOps.TestKit.Evidence;
+
+[Fact]
+[Requirement("REQ-EVIDENCE-001")]
+public void Test_ArtifactHashStability()
+{
+    var artifact = GenerateEvidence(input);
+
+    // Verify artifact produces expected hash (golden master)
+    EvidenceChainAssert.ArtifactHashStable(artifact, "abc123...expected-sha256...");
+}
+
+[Fact]
+[Requirement("REQ-DETERMINISM-001")]
+public void Test_EvidenceImmutability()
+{
+    // Verify generator produces identical output across iterations
+    EvidenceChainAssert.ArtifactImmutable(() => GenerateEvidence(fixedInput), iterations: 100);
+}
+
+[Fact]
+[Requirement("REQ-TRACE-001")]
+public void Test_TraceabilityComplete()
+{
+    var requirementId = "REQ-EVIDENCE-001";
+    var testId = "MyTests.TestMethod";
+    var artifactHash = EvidenceChainAssert.ComputeSha256(artifact);
+
+    // Verify all traceability components present
+    EvidenceChainAssert.TraceabilityComplete(requirementId, testId, artifactHash);
+}
+```
+
+**Traceability Report Generation:**
+
+```csharp
+using StellaOps.TestKit.Evidence;
+
+// Generate traceability matrix from test assemblies
+var reporter = new EvidenceChainReporter();
+reporter.AddAssembly(typeof(MyTests).Assembly);
+var report = reporter.GenerateReport();
+
+// Output as Markdown
+Console.WriteLine(report.ToMarkdown());
+
+// Output as JSON
+Console.WriteLine(report.ToJson());
+```
+
+**API Reference:**
+- `RequirementAttribute(string requirementId)` - Link test to requirement
+- `RequirementAttribute.SprintTaskId` - Link to sprint task (optional)
+- `RequirementAttribute.ComplianceControl` - Link to compliance control (optional)
+- `EvidenceChainAssert.ArtifactHashStable(artifact, expectedHash)` - Verify hash
+- `EvidenceChainAssert.ArtifactImmutable(generator, iterations)` - Verify determinism
+- `EvidenceChainAssert.ComputeSha256(content)` - Compute SHA-256 hash
+- `EvidenceChainAssert.RequirementLinked(requirementId)` - Marker assertion
+- `EvidenceChainAssert.TraceabilityComplete(reqId, testId, artifactId)` - Verify chain
+- `EvidenceChainReporter.AddAssembly(assembly)` - Add assembly to scan
+- `EvidenceChainReporter.GenerateReport()` - Generate traceability report
+- `EvidenceChainReport.ToMarkdown()` - Markdown output
+- `EvidenceChainReport.ToJson()` - JSON output
+- `EvidenceTraceabilityException` - Thrown when evidence assertions fail
+
+---
+
+### 11. Test Categories

 Standardized trait constants for CI lane filtering:

@@ -412,6 +621,82 @@ dotnet test --filter "Category=Integration|Category=Contract"
 - `Security` - Cryptographic validation
 - `Performance` - Benchmarking, load tests
 - `Live` - Requires external services (disabled in CI by default)
+- `PostIncident` - Tests derived from production incidents (Turn #6)
+- `EvidenceChain` - Requirement traceability tests (Turn #6)
+- `Longevity` - Time-extended stability tests (Turn #6)
+- `Interop` - Cross-version compatibility tests (Turn #6)
+
+---
+
+### 12. Post-Incident Testing (Turn #6)
+
+Generate regression tests from production incidents:
+
+**Generate Test Scaffold from Incident:**
+
+```csharp
+using StellaOps.TestKit.Incident;
+
+// Create incident metadata
+var metadata = new IncidentMetadata
+{
+    IncidentId = "INC-2026-001",
+    OccurredAt = DateTimeOffset.Parse("2026-01-15T10:30:00Z"),
+    RootCause = "Race condition in concurrent bundle creation",
+    AffectedModules = ["EvidenceLocker", "Policy"],
+    Severity = IncidentSeverity.P1,
+    Title = "Evidence bundle duplication"
+};
+
+// Generate test scaffold from replay manifest
+var generator = new IncidentTestGenerator();
+var scaffold = generator.GenerateFromManifestJson(manifestJson, metadata);
+
+// Output generated test code
+var code = scaffold.GenerateTestCode();
+File.WriteAllText($"Tests/{scaffold.TestClassName}.cs", code);
+```
+
+**Generated Test Structure:**
+
+```csharp
+[Trait("Category", TestCategories.PostIncident)]
+[Trait("Incident", "INC-2026-001")]
+[Trait("Severity", "P1")]
+public sealed class Incident_INC_2026_001_Tests
+{
+    [Fact]
+    public async Task Validates_RaceCondition_Fix()
+    {
+        // Arrange - fixtures from replay manifest
+        // Act - execute the incident scenario
+        // Assert - verify fix prevents recurrence
+    }
+}
+```
+
+**Filter Post-Incident Tests:**
+
+```bash
+# Run all post-incident tests
+dotnet test --filter "Category=PostIncident"
+
+# Run only P1/P2 tests (release-gating)
+dotnet test --filter "Category=PostIncident&(Severity=P1|Severity=P2)"
+
+# Run tests for a specific incident
+dotnet test --filter "Incident=INC-2026-001"
+```
+
+**API Reference:**
+- `IncidentMetadata` - Incident context (ID, severity, root cause, modules)
+- `IncidentSeverity` - P1 (critical) through P4 (low impact)
+- `IncidentTestGenerator.GenerateFromManifestJson(json, metadata)` - Generate scaffold
+- `TestScaffold.GenerateTestCode()` - Output C# test code
+- `TestScaffold.ToJson()` / `FromJson()` - Serialize/deserialize scaffold
+- `IncidentTestGenerator.GenerateReport()` - Summary of registered incident tests
+
+See [Post-Incident Testing Guide](post-incident-testing-guide.md) for complete documentation.

 ---