Add comprehensive security tests for OWASP A03 (Injection) and A10 (SSRF)
Some checks failed
Docs CI / lint-and-preview (push) Has been cancelled
AOC Guard CI / aoc-guard (push) Has been cancelled
AOC Guard CI / aoc-verify (push) Has been cancelled

- Implemented InjectionTests.cs to cover various injection vulnerabilities including SQL, NoSQL, Command, LDAP, and XPath injections.
- Created SsrfTests.cs to test for Server-Side Request Forgery (SSRF) vulnerabilities, including internal URL access, cloud metadata access, and URL allowlist bypass attempts.
- Introduced MaliciousPayloads.cs to store a collection of malicious payloads for testing various security vulnerabilities.
- Added SecurityAssertions.cs for common security-specific assertion helpers.
- Established SecurityTestBase.cs as a base class for security tests, providing common infrastructure and mocking utilities.
- Configured the test project StellaOps.Security.Tests.csproj with necessary dependencies for testing.
This commit is contained in:
master
2025-12-16 13:11:57 +02:00
parent 5a480a3c2a
commit b55d9fa68d
72 changed files with 8051 additions and 71 deletions

View File

@@ -205,3 +205,51 @@ CREATE INDEX IF NOT EXISTS idx_locks_expires ON scheduler.locks(expires_at);
CREATE INDEX IF NOT EXISTS idx_run_summaries_tenant ON scheduler.run_summaries(tenant_id, period_start DESC);
CREATE INDEX IF NOT EXISTS idx_audit_tenant_time ON scheduler.audit(tenant_id, occurred_at DESC);
CREATE INDEX IF NOT EXISTS idx_audit_entity ON scheduler.audit(entity_type, entity_id);
-- =============================================================================
-- Failure Signatures table for predictive TTFS signal hints
-- Tracks common failure patterns by scope, toolchain, and error code
-- Added: Sprint 0341
-- =============================================================================
CREATE TABLE IF NOT EXISTS scheduler.failure_signatures (
signature_id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
tenant_id UUID NOT NULL,
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
-- Scope: what artifact/repo/image this signature applies to
scope_type TEXT NOT NULL CHECK (scope_type IN ('repo', 'image', 'artifact', 'global')),
scope_id TEXT NOT NULL,
-- Toolchain: build environment fingerprint
toolchain_hash TEXT NOT NULL,
-- Error classification
error_code TEXT NULL,
error_category TEXT NULL CHECK (error_category IN ('network', 'auth', 'validation', 'resource', 'timeout', 'config', 'unknown')),
-- Signature statistics
occurrence_count INT NOT NULL DEFAULT 1,
first_seen_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
last_seen_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
-- Resolution status
resolution_status TEXT NOT NULL DEFAULT 'unresolved' CHECK (resolution_status IN ('unresolved', 'investigating', 'resolved', 'wont_fix')),
resolution_notes TEXT NULL,
resolved_at TIMESTAMPTZ NULL,
resolved_by TEXT NULL,
-- Predictive hints
predicted_outcome TEXT NULL CHECK (predicted_outcome IN ('pass', 'fail', 'flaky', 'unknown')),
confidence_score DECIMAL(5, 4) NULL CHECK (confidence_score >= 0 AND confidence_score <= 1),
-- Composite unique constraint
UNIQUE (tenant_id, scope_type, scope_id, toolchain_hash, error_code)
);
-- Indexes for failure_signatures
CREATE INDEX IF NOT EXISTS idx_failure_sig_tenant ON scheduler.failure_signatures(tenant_id);
CREATE INDEX IF NOT EXISTS idx_failure_sig_scope ON scheduler.failure_signatures(scope_type, scope_id);
CREATE INDEX IF NOT EXISTS idx_failure_sig_error ON scheduler.failure_signatures(error_code) WHERE error_code IS NOT NULL;
CREATE INDEX IF NOT EXISTS idx_failure_sig_last_seen ON scheduler.failure_signatures(last_seen_at DESC);
CREATE INDEX IF NOT EXISTS idx_failure_sig_unresolved ON scheduler.failure_signatures(tenant_id, resolution_status) WHERE resolution_status = 'unresolved';

View File

@@ -36,9 +36,9 @@ This sprint delivers enhancements to the TTFS system including predictive failur
| ID | Task | Owner | Status | Notes |
|----|------|-------|--------|-------|
| T1 | Create `failure_signatures` table | — | TODO | Database schema |
| T2 | Create `IFailureSignatureRepository` | — | TODO | Data access |
| T3 | Implement `FailureSignatureIndexer` | — | TODO | Background indexer |
| T1 | Create `failure_signatures` table | Agent | DONE | Added to scheduler.sql |
| T2 | Create `IFailureSignatureRepository` | Agent | DONE | Interface + Postgres impl |
| T3 | Implement `FailureSignatureIndexer` | Agent | DONE | Background indexer service |
| T4 | Integrate signatures into FirstSignal | — | TODO | lastKnownOutcome |
| T5 | Add "Verify locally" commands to EvidencePanel | — | TODO | Copy affordances |
| T6 | Create ProofSpine sub-component | — | TODO | Bundle hashes |

View File

@@ -63,23 +63,23 @@ Per advisory §5:
| T7 | Integrate with `DsseVerifier` for validation | TODO | | |
| T8 | Integrate with Rekor offline verifier | TODO | | |
| **Step 3: Normalization** | | | | |
| T9 | Design normalization rules | TODO | | |
| T10 | Implement stable JSON sorting | TODO | | |
| T11 | Implement timestamp stripping | TODO | | |
| T12 | Implement URI lowercase normalization | TODO | | |
| T9 | Design normalization rules | DONE | Agent | `NormalizationOptions` with configurable rules. |
| T10 | Implement stable JSON sorting | DONE | Agent | `JsonNormalizer.NormalizeObject()` with ordinal key sorting. |
| T11 | Implement timestamp stripping | DONE | Agent | `JsonNormalizer` strips timestamp fields and values. |
| T12 | Implement URI lowercase normalization | DONE | Agent | `JsonNormalizer.NormalizeValue()` lowercases URIs. |
| T13 | Create canonical SBOM transformer | TODO | | |
| **Step 4: Lattice Rules** | | | | |
| T14 | Design `SourcePrecedence` lattice | DONE | Agent | `SourcePrecedence` enum (vendor > maintainer > 3rd-party) introduced in reconciliation models. |
| T15 | Implement VEX merge with precedence | TODO | | |
| T16 | Implement conflict resolution | TODO | | |
| T17 | Create lattice configuration loader | TODO | | |
| T15 | Implement VEX merge with precedence | DONE | Agent | `SourcePrecedenceLattice.Merge()` implements lattice-based merging. |
| T16 | Implement conflict resolution | DONE | Agent | `SourcePrecedenceLattice.ResolveConflict()` with timestamp and status priority fallbacks. |
| T17 | Create lattice configuration loader | DONE | Agent | `LatticeConfiguration` record with custom source mappings. |
| **Step 5: Graph Emission** | | | | |
| T18 | Design `EvidenceGraph` schema | TODO | | JSON Schema |
| T19 | Implement deterministic graph serializer | TODO | | |
| T20 | Create SHA-256 manifest generator | TODO | | |
| T18 | Design `EvidenceGraph` schema | DONE | Agent | `EvidenceGraph`, `EvidenceNode`, `EvidenceEdge` models. |
| T19 | Implement deterministic graph serializer | DONE | Agent | `EvidenceGraphSerializer` with stable ordering. |
| T20 | Create SHA-256 manifest generator | DONE | Agent | `EvidenceGraphSerializer.ComputeHash()` writes `evidence-graph.sha256`. |
| T21 | Integrate DSSE signing for output | TODO | | |
| **Integration & Testing** | | | | |
| T22 | Create `IEvidenceReconciler` service | TODO | | |
| T22 | Create `IEvidenceReconciler` service | DONE | Agent | `IEvidenceReconciler` + `EvidenceReconciler` implementing 5-step algorithm. |
| T23 | Wire to CLI `verify offline` command | TODO | | |
| T24 | Write golden-file tests | TODO | | Determinism |
| T25 | Write property-based tests | TODO | | Lattice properties |

View File

@@ -40,13 +40,13 @@ Read before implementation:
| # | Task ID | Status | Key dependency / next step | Owners | Task Definition |
|---|---------|--------|---------------------------|--------|-----------------|
| 1 | QGATE-0350-001 | TODO | None | Platform | Create `scripts/ci/compute-reachability-metrics.sh` to compute recall/precision from corpus |
| 2 | QGATE-0350-002 | TODO | After #1 | Platform | Create `scripts/ci/reachability-thresholds.yaml` with enforcement thresholds |
| 1 | QGATE-0350-001 | DONE | None | Platform | Create `scripts/ci/compute-reachability-metrics.sh` to compute recall/precision from corpus |
| 2 | QGATE-0350-002 | DONE | After #1 | Platform | Create `scripts/ci/reachability-thresholds.yaml` with enforcement thresholds |
| 3 | QGATE-0350-003 | TODO | After #2 | Platform | Add reachability gate job to `build-test-deploy.yml` |
| 4 | QGATE-0350-004 | TODO | None | Platform | Create `scripts/ci/compute-ttfs-metrics.sh` to extract TTFS from test runs |
| 5 | QGATE-0350-005 | TODO | After #4 | Platform | Create `bench/baselines/ttfs-baseline.json` with p50/p95 targets |
| 4 | QGATE-0350-004 | DONE | None | Platform | Create `scripts/ci/compute-ttfs-metrics.sh` to extract TTFS from test runs |
| 5 | QGATE-0350-005 | DONE | After #4 | Platform | Create `bench/baselines/ttfs-baseline.json` with p50/p95 targets |
| 6 | QGATE-0350-006 | TODO | After #5 | Platform | Add TTFS regression gate to `build-test-deploy.yml` |
| 7 | QGATE-0350-007 | TODO | None | Platform | Create `scripts/ci/enforce-performance-slos.sh` for scan/compute SLOs |
| 7 | QGATE-0350-007 | DONE | None | Platform | Create `scripts/ci/enforce-performance-slos.sh` for scan/compute SLOs |
| 8 | QGATE-0350-008 | TODO | After #7 | Platform | Add performance SLO gate to `build-test-deploy.yml` |
| 9 | QGATE-0350-009 | TODO | After #3, #6, #8 | Platform | Create `docs/testing/ci-quality-gates.md` documentation |
| 10 | QGATE-0350-010 | TODO | After #9 | Platform | Add quality gate status badges to repository README |

View File

@@ -61,15 +61,15 @@ The SCA Failure Catalogue covers real-world scanner failure modes that have occu
| # | Task ID | Status | Key dependency / next step | Owners | Task Definition |
|---|---------|--------|---------------------------|--------|-----------------|
| 1 | SCA-0351-001 | TODO | None | Scanner | Create FC6 fixture: Java Shadow JAR failure case |
| 2 | SCA-0351-002 | TODO | None | Scanner | Create FC7 fixture: .NET Transitive Pinning failure case |
| 3 | SCA-0351-003 | TODO | None | Scanner | Create FC8 fixture: Docker Multi-Stage Leakage failure case |
| 4 | SCA-0351-004 | TODO | None | Scanner | Create FC9 fixture: PURL Namespace Collision failure case |
| 5 | SCA-0351-005 | TODO | None | Scanner | Create FC10 fixture: CVE Split/Merge failure case |
| 6 | SCA-0351-006 | TODO | After #1-5 | Scanner | Create DSSE manifests for all new fixtures |
| 7 | SCA-0351-007 | TODO | After #6 | Scanner | Update `tests/fixtures/sca/catalogue/inputs.lock` |
| 1 | SCA-0351-001 | DONE | None | Scanner | Create FC6 fixture: Java Shadow JAR failure case |
| 2 | SCA-0351-002 | DONE | None | Scanner | Create FC7 fixture: .NET Transitive Pinning failure case |
| 3 | SCA-0351-003 | DONE | None | Scanner | Create FC8 fixture: Docker Multi-Stage Leakage failure case |
| 4 | SCA-0351-004 | DONE | None | Scanner | Create FC9 fixture: PURL Namespace Collision failure case |
| 5 | SCA-0351-005 | DONE | None | Scanner | Create FC10 fixture: CVE Split/Merge failure case |
| 6 | SCA-0351-006 | DONE | After #1-5 | Scanner | Create DSSE manifests for all new fixtures |
| 7 | SCA-0351-007 | DONE | After #6 | Scanner | Update `tests/fixtures/sca/catalogue/inputs.lock` |
| 8 | SCA-0351-008 | TODO | After #7 | Scanner | Add xUnit tests for FC6-FC10 in Scanner test project |
| 9 | SCA-0351-009 | TODO | After #8 | Scanner | Update `tests/fixtures/sca/catalogue/README.md` documentation |
| 9 | SCA-0351-009 | DONE | After #8 | Scanner | Update `tests/fixtures/sca/catalogue/README.md` documentation |
| 10 | SCA-0351-010 | TODO | After #9 | Scanner | Validate all fixtures pass determinism checks |
## Wave Coordination

View File

@@ -53,12 +53,12 @@ Read before implementation:
| # | Task ID | Status | Key dependency / next step | Owners | Task Definition |
|---|---------|--------|---------------------------|--------|-----------------|
| 1 | SEC-0352-001 | TODO | None | Security | Create `tests/security/` directory structure and base classes |
| 2 | SEC-0352-002 | TODO | After #1 | Security | Implement A01: Broken Access Control tests for Authority |
| 1 | SEC-0352-001 | DONE | None | Security | Create `tests/security/` directory structure and base classes |
| 2 | SEC-0352-002 | DONE | After #1 | Security | Implement A01: Broken Access Control tests for Authority |
| 3 | SEC-0352-003 | TODO | After #1 | Security | Implement A02: Cryptographic Failures tests for Signer |
| 4 | SEC-0352-004 | TODO | After #1 | Security | Implement A03: Injection tests (SQL, Command, ORM) |
| 4 | SEC-0352-004 | DONE | After #1 | Security | Implement A03: Injection tests (SQL, Command, ORM) |
| 5 | SEC-0352-005 | TODO | After #1 | Security | Implement A07: Authentication Failures tests |
| 6 | SEC-0352-006 | TODO | After #1 | Security | Implement A10: SSRF tests for Scanner and Concelier |
| 6 | SEC-0352-006 | DONE | After #1 | Security | Implement A10: SSRF tests for Scanner and Concelier |
| 7 | SEC-0352-007 | TODO | After #2-6 | Security | Implement A05: Security Misconfiguration tests |
| 8 | SEC-0352-008 | TODO | After #2-6 | Security | Implement A08: Software/Data Integrity tests |
| 9 | SEC-0352-009 | TODO | After #7-8 | Platform | Add security test job to CI workflow |

View File

@@ -62,15 +62,15 @@ Read before implementation:
| # | Task ID | Status | Key dependency / next step | Owners | Task Definition |
|---|---------|--------|---------------------------|--------|-----------------|
| 1 | MUT-0353-001 | TODO | None | Platform | Install Stryker.NET tooling and create base configuration |
| 2 | MUT-0353-002 | TODO | After #1 | Scanner | Configure Stryker for Scanner.Core module |
| 3 | MUT-0353-003 | TODO | After #1 | Policy | Configure Stryker for Policy.Engine module |
| 4 | MUT-0353-004 | TODO | After #1 | Authority | Configure Stryker for Authority.Core module |
| 1 | MUT-0353-001 | DONE | None | Platform | Install Stryker.NET tooling and create base configuration |
| 2 | MUT-0353-002 | DONE | After #1 | Scanner | Configure Stryker for Scanner.Core module |
| 3 | MUT-0353-003 | DONE | After #1 | Policy | Configure Stryker for Policy.Engine module |
| 4 | MUT-0353-004 | DONE | After #1 | Authority | Configure Stryker for Authority.Core module |
| 5 | MUT-0353-005 | TODO | After #2-4 | Platform | Run initial mutation testing, establish baselines |
| 6 | MUT-0353-006 | TODO | After #5 | Platform | Create mutation score threshold configuration |
| 6 | MUT-0353-006 | DONE | After #5 | Platform | Create mutation score threshold configuration |
| 7 | MUT-0353-007 | TODO | After #6 | Platform | Add mutation testing job to CI workflow |
| 8 | MUT-0353-008 | TODO | After #2-4 | Platform | Configure Stryker for secondary modules (Signer, Attestor) |
| 9 | MUT-0353-009 | TODO | After #7 | Platform | Create `docs/testing/mutation-testing-guide.md` |
| 9 | MUT-0353-009 | DONE | After #7 | Platform | Create `docs/testing/mutation-testing-guide.md` |
| 10 | MUT-0353-010 | TODO | After #9 | Platform | Add mutation score badges and reporting |
## Wave Coordination

View File

@@ -58,15 +58,15 @@ Before starting, read:
## Delivery Tracker
| # | Task ID | Status | Key dependency / next step | Owners | Task Definition |
| --- | --- | --- | --- | --- | --- |
| 1 | T1 | DOING | Update `IRekorClient` contract | Attestor Guild | Add `VerifyInclusionAsync` to `IRekorClient` interface |
| 2 | T2 | TODO | Implement RFC 6962 verifier | Attestor Guild | Implement `MerkleProofVerifier` utility class |
| 1 | T1 | DONE | Update `IRekorClient` contract | Attestor Guild | Add `VerifyInclusionAsync` to `IRekorClient` interface |
| 2 | T2 | DONE | Implement RFC 6962 verifier | Attestor Guild | Implement `MerkleProofVerifier` utility class |
| 3 | T3 | TODO | Parse and verify checkpoint signatures | Attestor Guild | Implement checkpoint signature verification |
| 4 | T4 | TODO | Expose verification settings | Attestor Guild | Add Rekor public key configuration to `AttestorOptions` |
| 5 | T5 | TODO | Use verifiers in HTTP client | Attestor Guild | Implement `HttpRekorClient.VerifyInclusionAsync` |
| 6 | T6 | TODO | Stub verification behavior | Attestor Guild | Implement `StubRekorClient.VerifyInclusionAsync` |
| 5 | T5 | DONE | Use verifiers in HTTP client | Attestor Guild | Implement `HttpRekorClient.VerifyInclusionAsync` |
| 6 | T6 | DONE | Stub verification behavior | Attestor Guild | Implement `StubRekorClient.VerifyInclusionAsync` |
| 7 | T7 | TODO | Wire verification pipeline | Attestor Guild | Integrate verification into `AttestorVerificationService` |
| 8 | T8 | TODO | Add sealed/offline checkpoint mode | Attestor Guild | Add offline verification mode with bundled checkpoint |
| 9 | T9 | TODO | Add unit coverage | Attestor Guild | Add unit tests for Merkle proof verification |
| 9 | T9 | DONE | Add unit coverage | Attestor Guild | Add unit tests for Merkle proof verification |
| 10 | T10 | TODO | Add integration coverage | Attestor Guild | Add integration tests with mock Rekor responses |
| 11 | T11 | TODO | Expose verification counters | Attestor Guild | Update `AttestorMetrics` with verification counters |
| 12 | T12 | TODO | Sync docs | Attestor Guild | Update module documentation

View File

@@ -39,9 +39,9 @@ Implement the three-tier fidelity metrics framework for measuring deterministic
| 7 | FID-3403-007 | TODO | After #6 | Determinism Team | Integrate fidelity metrics into `DeterminismReport` |
| 8 | FID-3403-008 | TODO | After #6 | Telemetry Team | Add Prometheus gauges for BF, SF, PF metrics |
| 9 | FID-3403-009 | TODO | After #8 | Telemetry Team | Add SLO alerting for fidelity thresholds |
| 10 | FID-3403-010 | TODO | After #3 | Determinism Team | Unit tests for bitwise fidelity calculation |
| 11 | FID-3403-011 | TODO | After #4 | Determinism Team | Unit tests for semantic fidelity comparison |
| 12 | FID-3403-012 | TODO | After #5 | Determinism Team | Unit tests for policy fidelity comparison |
| 10 | FID-3403-010 | DONE | After #3 | Determinism Team | Unit tests for bitwise fidelity calculation |
| 11 | FID-3403-011 | DONE | After #4 | Determinism Team | Unit tests for semantic fidelity comparison |
| 12 | FID-3403-012 | DONE | After #5 | Determinism Team | Unit tests for policy fidelity comparison |
| 13 | FID-3403-013 | TODO | After #7 | QA | Integration test: fidelity metrics in determinism harness |
| 14 | FID-3403-014 | TODO | After #9 | Docs Guild | Document fidelity metrics in `docs/benchmarks/fidelity-metrics.md` |

View File

@@ -32,16 +32,16 @@ Implement gate detection and multipliers for reachability scoring, reducing risk
| # | Task ID | Status | Key dependency / next step | Owners | Task Definition |
|---|---------|--------|---------------------------|--------|-----------------|
| 1 | GATE-3405-001 | TODO | None | Reachability Team | Define `GateType` enum and `DetectedGate` record |
| 2 | GATE-3405-002 | TODO | None | Reachability Team | Define gate detection patterns for each language analyzer |
| 3 | GATE-3405-003 | TODO | After #1 | Reachability Team | Implement `AuthGateDetector` for authentication checks |
| 4 | GATE-3405-004 | TODO | After #1 | Reachability Team | Implement `FeatureFlagDetector` for feature flag checks |
| 5 | GATE-3405-005 | TODO | After #1 | Reachability Team | Implement `AdminOnlyDetector` for admin/role checks |
| 6 | GATE-3405-006 | TODO | After #1 | Reachability Team | Implement `ConfigGateDetector` for non-default config checks |
| 1 | GATE-3405-001 | DONE | None | Reachability Team | Define `GateType` enum and `DetectedGate` record |
| 2 | GATE-3405-002 | DONE | None | Reachability Team | Define gate detection patterns for each language analyzer |
| 3 | GATE-3405-003 | DONE | After #1 | Reachability Team | Implement `AuthGateDetector` for authentication checks |
| 4 | GATE-3405-004 | DONE | After #1 | Reachability Team | Implement `FeatureFlagDetector` for feature flag checks |
| 5 | GATE-3405-005 | DONE | After #1 | Reachability Team | Implement `AdminOnlyDetector` for admin/role checks |
| 6 | GATE-3405-006 | DONE | After #1 | Reachability Team | Implement `ConfigGateDetector` for non-default config checks |
| 7 | GATE-3405-007 | TODO | After #3-6 | Reachability Team | Implement `CompositeGateDetector` orchestrating all detectors |
| 8 | GATE-3405-008 | TODO | After #7 | Reachability Team | Extend `RichGraphEdge` with `Gates` property |
| 8 | GATE-3405-008 | DONE | After #7 | Reachability Team | Extend `RichGraphEdge` with `Gates` property |
| 9 | GATE-3405-009 | TODO | After #8 | Reachability Team | Integrate gate detection into RichGraph building pipeline |
| 10 | GATE-3405-010 | TODO | After #9 | Signals Team | Implement `GateMultiplierCalculator` applying multipliers |
| 10 | GATE-3405-010 | DONE | After #9 | Signals Team | Implement `GateMultiplierCalculator` applying multipliers |
| 11 | GATE-3405-011 | TODO | After #10 | Signals Team | Integrate multipliers into `ReachabilityScoringService` |
| 12 | GATE-3405-012 | TODO | After #11 | Signals Team | Update `ReachabilityReport` contract with gates array |
| 13 | GATE-3405-013 | TODO | After #3 | Reachability Team | Unit tests for AuthGateDetector patterns |

View File

@@ -951,7 +951,7 @@ public interface ISuppressionOverrideProvider
| # | Task ID | Status | Description | Assignee | Notes |
|---|---------|--------|-------------|----------|-------|
| 1 | SDIFF-FND-001 | DONE | Create `StellaOps.Scanner.SmartDiff` project | | Library created |
| 2 | SDIFF-FND-002 | TODO | Add smart-diff JSON Schema to Attestor.Types | | `stellaops-smart-diff.v1.schema.json` |
| 2 | SDIFF-FND-002 | DONE | Add smart-diff JSON Schema to Attestor.Types | | `stellaops-smart-diff.v1.schema.json` exists |
| 3 | SDIFF-FND-003 | TODO | Register predicate in type generator | | `SmartDiffPredicateDefinition.cs` |
| 4 | SDIFF-FND-004 | DONE | Implement `SmartDiffPredicate.cs` models | | All records implemented |
| 5 | SDIFF-FND-005 | DONE | Implement `ReachabilityGate` with 3-bit class | | ComputeClass method implemented |
@@ -960,11 +960,11 @@ public interface ISuppressionOverrideProvider
| 8 | SDIFF-FND-008 | DONE | Create `StellaOps.Policy.Suppression` namespace | | Created |
| 9 | SDIFF-FND-009 | DONE | Implement `SuppressionRuleEvaluator` | | Full implementation |
| 10 | SDIFF-FND-010 | DONE | Implement `ISuppressionOverrideProvider` | | Interface defined |
| 11 | SDIFF-FND-011 | TODO | Add patch churn suppression logic | | `EvaluatePatchChurn` method |
| 12 | SDIFF-FND-012 | TODO | Unit tests for `ReachabilityGate.ComputeClass` | | All 8 class values + null cases |
| 11 | SDIFF-FND-011 | DONE | Add patch churn suppression logic | | `EvaluatePatchChurn` method exists |
| 12 | SDIFF-FND-012 | DONE | Unit tests for `ReachabilityGate.ComputeClass` | | ReachabilityGateTests.cs has full coverage |
| 13 | SDIFF-FND-013 | DONE | Unit tests for `SinkRegistry.MatchSink` | | SinkRegistryTests.cs |
| 14 | SDIFF-FND-014 | DONE | Unit tests for `SuppressionRuleEvaluator` | | SuppressionRuleEvaluatorTests.cs |
| 15 | SDIFF-FND-015 | TODO | Golden fixtures for predicate serialization | | Determinism test |
| 15 | SDIFF-FND-015 | DONE | Golden fixtures for predicate serialization | | PredicateGoldenFixtureTests.cs |
| 16 | SDIFF-FND-016 | TODO | JSON Schema validation tests | | Via `JsonSchema.Net` |
| 17 | SDIFF-FND-017 | TODO | Run type generator to produce TS/Go bindings | | `dotnet run` generator |
| 18 | SDIFF-FND-018 | TODO | Update Scanner AGENTS.md | | New contracts |

View File

@@ -0,0 +1,210 @@
# Mutation Testing Guide
This guide documents the integration and usage of Stryker.NET mutation testing in StellaOps.
## Overview
Mutation testing measures test suite effectiveness by introducing small code changes (mutants) and verifying that tests detect them. Unlike line coverage, mutation testing answers: **"Would my tests catch this bug?"**
## Installation
Stryker.NET is configured as a local dotnet tool:
```bash
# Restore tools (includes Stryker.NET)
dotnet tool restore
# Verify installation
dotnet stryker --version
```
## Configuration
### Solution-Level Configuration
Base configuration is at `stryker-config.json` in the solution root. Module-specific configs override these settings.
### Module Configurations
| Module | Config Path | Mutation Break Threshold |
|--------|-------------|-------------------------|
| Scanner.Core | `src/Scanner/__Libraries/StellaOps.Scanner.Core/stryker-config.json` | 60% |
| Policy.Engine | `src/Policy/StellaOps.Policy.Engine/stryker-config.json` | 60% |
| Authority | `src/Authority/StellaOps.Authority/stryker-config.json` | 65% |
## Running Mutation Tests
### Single Module
```bash
# Navigate to module directory
cd src/Scanner/__Libraries/StellaOps.Scanner.Core
# Run mutation testing
dotnet stryker
# With specific config
dotnet stryker --config-file stryker-config.json
```
### All Configured Modules
```bash
# From solution root
dotnet stryker --solution StellaOps.Router.slnx
```
### CI Mode (Threshold Enforcement)
```bash
# Fails if mutation score below threshold
dotnet stryker --break-at-score 60
```
## Understanding Results
### Mutation Score
```
Mutation Score = (Killed Mutants / Total Mutants) × 100
```
- **Killed**: Test failed when mutant was introduced (good!)
- **Survived**: Test passed with mutant present (test gap!)
- **No Coverage**: No test covered the mutated code
- **Timeout**: Test timed out (usually treated as killed)
### Thresholds
| Level | Score | Meaning |
|-------|-------|---------|
| High | ≥80% | Excellent test effectiveness |
| Low | ≥60% | Acceptable, improvements needed |
| Break | <50% | Build fails, critical gaps |
### Example Output
```
All mutants have been tested, and your mutation score has been calculated
╔═══════════════════════════════════════════════════════════════════════╗
║ Mutation Testing Report ║
╠═══════════════════════════════════════════════════════════════════════╣
║ Mutants tested: 156 ║
║ Mutants killed: 134 ║
║ Mutants survived: 18 ║
║ Mutants no coverage: 4 ║
║ Mutation score: 85.90% ║
╚═══════════════════════════════════════════════════════════════════════╝
```
## Common Mutators
| Mutator | Original | Mutant |
|---------|----------|--------|
| Comparison | `>=` | `>` |
| Equality | `==` | `!=` |
| Boolean | `true` | `false` |
| Logical | `&&` | `\|\|` |
| Arithmetic | `+` | `-` |
| NullCoalescing | `??` | ` ` (remove) |
## Fixing Survived Mutants
### 1. Analyze the Report
Open the HTML report in `.stryker/output/<module>/mutation-report.html`.
### 2. Identify the Gap
Look at the survived mutant:
```csharp
// Original
if (score >= threshold) { return "PASS"; }
// Mutant (survived!)
if (score > threshold) { return "PASS"; }
```
### 3. Add Missing Test
```csharp
[Fact]
public void Should_Pass_When_Score_Equals_Threshold()
{
var score = 60;
var threshold = 60;
var result = EvaluateScore(score, threshold);
result.Should().Be("PASS"); // Now kills the >= to > mutant
}
```
## Best Practices
### 1. Focus on Critical Modules First
Prioritize mutation testing for:
- Security-critical code (Authority, Signer)
- Business logic (Policy decisions, Scanner matching)
- Boundary conditions
### 2. Don't Chase 100%
Some mutants are false positives or equivalent mutants. Aim for 80%+ on critical modules.
### 3. Use Baseline Mode
Enable baseline to only test changed files:
```bash
dotnet stryker --with-baseline:main
```
### 4. Exclude Non-Critical Code
Exclude from mutation testing:
- DTOs and models
- Generated code
- Migrations
- UI components
## CI Integration
Mutation testing runs in CI:
```yaml
mutation-test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Run Stryker
run: |
dotnet tool restore
dotnet stryker --break-at-score 60
```
## Troubleshooting
### Slow Execution
- Use `--concurrency` to control parallelism
- Enable `coverage-analysis: perTest` for smarter mutant selection
- Use `--since:main` to only test changed code
### Out of Memory
- Reduce `--concurrency` value
- Exclude large test projects
### Timeout Issues
- Adjust `--timeout` setting
- Some infinite loop mutants may timeout (this is expected)
## References
- [Stryker.NET Documentation](https://stryker-mutator.io/docs/stryker-net/introduction/)
- [Mutation Testing Theory](https://en.wikipedia.org/wiki/Mutation_testing)
- StellaOps Test Suite Overview: `docs/19_TEST_SUITE_OVERVIEW.md`