git.stella-ops.org/docs/testing/mutation-testing-guide.md

# Mutation Testing Guide

This guide documents the integration and usage of Stryker.NET mutation testing in StellaOps.

## Overview

Mutation testing measures test suite effectiveness by introducing small code changes (mutants) and verifying that tests detect them. Unlike line coverage, mutation testing answers: **"Would my tests catch this bug?"**

## Installation

Stryker.NET is configured as a local dotnet tool:

```bash
# Restore tools (includes Stryker.NET)
dotnet tool restore

# Verify installation
dotnet stryker --version
```

## Configuration

### Solution-Level Configuration

Base configuration is at `stryker-config.json` in the solution root. Module-specific configs override these settings.

### Module Configurations

| Module | Config Path | Mutation Break Threshold |
|--------|-------------|-------------------------|
| Scanner.Core | `src/Scanner/__Libraries/StellaOps.Scanner.Core/stryker-config.json` | 60% |
| Policy.Engine | `src/Policy/StellaOps.Policy.Engine/stryker-config.json` | 60% |
| Authority | `src/Authority/StellaOps.Authority/stryker-config.json` | 65% |

## Running Mutation Tests

### Single Module

```bash
# Navigate to module directory
cd src/Scanner/__Libraries/StellaOps.Scanner.Core

# Run mutation testing
dotnet stryker

# With specific config
dotnet stryker --config-file stryker-config.json
```

### All Configured Modules

```bash
# From solution root
dotnet stryker --solution StellaOps.Router.slnx
```

### CI Mode (Threshold Enforcement)

```bash
# Fails if mutation score below threshold
dotnet stryker --break-at-score 60
```

## Understanding Results

### Mutation Score

```
Mutation Score = (Killed Mutants / Total Mutants) × 100
```

- **Killed**: Test failed when mutant was introduced (good!)
- **Survived**: Test passed with mutant present (test gap!)
- **No Coverage**: No test covered the mutated code
- **Timeout**: Test timed out (usually treated as killed)

### Thresholds

| Level | Score | Meaning |
|-------|-------|---------|
| High | ≥80% | Excellent test effectiveness |
| Low | ≥60% | Acceptable, improvements needed |
| Break | <50% | Build fails, critical gaps |

### Example Output

```
All mutants have been tested, and your mutation score has been calculated
╔═══════════════════════════════════════════════════════════════════════╗
║ Mutation Testing Report                                                ║
╠═══════════════════════════════════════════════════════════════════════╣
║ Mutants tested: 156                                                    ║
║ Mutants killed: 134                                                    ║
║ Mutants survived: 18                                                   ║
║ Mutants no coverage: 4                                                 ║
║ Mutation score: 85.90%                                                 ║
╚═══════════════════════════════════════════════════════════════════════╝
```

## Common Mutators

| Mutator | Original | Mutant |
|---------|----------|--------|
| Comparison | `>=` | `>` |
| Equality | `==` | `!=` |
| Boolean | `true` | `false` |
| Logical | `&&` | `\|\|` |
| Arithmetic | `+` | `-` |
| NullCoalescing | `??` | ` ` (remove) |

## Fixing Survived Mutants

### 1. Analyze the Report

Open the HTML report in `.stryker/output/<module>/mutation-report.html`.

### 2. Identify the Gap

Look at the survived mutant:

```csharp
// Original
if (score >= threshold) { return "PASS"; }

// Mutant (survived!)
if (score > threshold) { return "PASS"; }
```

### 3. Add Missing Test

```csharp
[Fact]
public void Should_Pass_When_Score_Equals_Threshold()
{
    var score = 60;
    var threshold = 60;
    
    var result = EvaluateScore(score, threshold);
    
    result.Should().Be("PASS"); // Now kills the >= to > mutant
}
```

## Best Practices

### 1. Focus on Critical Modules First

Prioritize mutation testing for:
- Security-critical code (Authority, Signer)
- Business logic (Policy decisions, Scanner matching)
- Boundary conditions

### 2. Don't Chase 100%

Some mutants are false positives or equivalent mutants. Aim for 80%+ on critical modules.

### 3. Use Baseline Mode

Enable baseline to only test changed files:

```bash
dotnet stryker --with-baseline:main
```

### 4. Exclude Non-Critical Code

Exclude from mutation testing:
- DTOs and models
- Generated code
- Migrations
- UI components

## CI Integration

Mutation testing runs in CI:

```yaml
mutation-test:
  runs-on: ubuntu-latest
  steps:
    - uses: actions/checkout@v4
    - name: Run Stryker
      run: |
        dotnet tool restore
        dotnet stryker --break-at-score 60
```

## Troubleshooting

### Slow Execution

- Use `--concurrency` to control parallelism
- Enable `coverage-analysis: perTest` for smarter mutant selection
- Use `--since:main` to only test changed code

### Out of Memory

- Reduce `--concurrency` value
- Exclude large test projects

### Timeout Issues

- Adjust `--timeout` setting
- Some infinite loop mutants may timeout (this is expected)

## References

- [Stryker.NET Documentation](https://stryker-mutator.io/docs/stryker-net/introduction/)
- [Mutation Testing Theory](https://en.wikipedia.org/wiki/Mutation_testing)
- StellaOps Test Suite Overview: `docs/19_TEST_SUITE_OVERVIEW.md`