Add comprehensive security tests for OWASP A03 (Injection) and A10 (SSRF)
- Implemented InjectionTests.cs to cover various injection vulnerabilities including SQL, NoSQL, Command, LDAP, and XPath injections. - Created SsrfTests.cs to test for Server-Side Request Forgery (SSRF) vulnerabilities, including internal URL access, cloud metadata access, and URL allowlist bypass attempts. - Introduced MaliciousPayloads.cs to store a collection of malicious payloads for testing various security vulnerabilities. - Added SecurityAssertions.cs for common security-specific assertion helpers. - Established SecurityTestBase.cs as a base class for security tests, providing common infrastructure and mocking utilities. - Configured the test project StellaOps.Security.Tests.csproj with necessary dependencies for testing.
This commit is contained in:
210
docs/testing/mutation-testing-guide.md
Normal file
210
docs/testing/mutation-testing-guide.md
Normal file
@@ -0,0 +1,210 @@
|
||||
# Mutation Testing Guide
|
||||
|
||||
This guide documents the integration and usage of Stryker.NET mutation testing in StellaOps.
|
||||
|
||||
## Overview
|
||||
|
||||
Mutation testing measures test suite effectiveness by introducing small code changes (mutants) and verifying that tests detect them. Unlike line coverage, mutation testing answers: **"Would my tests catch this bug?"**
|
||||
|
||||
## Installation
|
||||
|
||||
Stryker.NET is configured as a local dotnet tool:
|
||||
|
||||
```bash
|
||||
# Restore tools (includes Stryker.NET)
|
||||
dotnet tool restore
|
||||
|
||||
# Verify installation
|
||||
dotnet stryker --version
|
||||
```
|
||||
|
||||
## Configuration
|
||||
|
||||
### Solution-Level Configuration
|
||||
|
||||
Base configuration is at `stryker-config.json` in the solution root. Module-specific configs override these settings.
|
||||
|
||||
### Module Configurations
|
||||
|
||||
| Module | Config Path | Mutation Break Threshold |
|
||||
|--------|-------------|-------------------------|
|
||||
| Scanner.Core | `src/Scanner/__Libraries/StellaOps.Scanner.Core/stryker-config.json` | 60% |
|
||||
| Policy.Engine | `src/Policy/StellaOps.Policy.Engine/stryker-config.json` | 60% |
|
||||
| Authority | `src/Authority/StellaOps.Authority/stryker-config.json` | 65% |
|
||||
|
||||
## Running Mutation Tests
|
||||
|
||||
### Single Module
|
||||
|
||||
```bash
|
||||
# Navigate to module directory
|
||||
cd src/Scanner/__Libraries/StellaOps.Scanner.Core
|
||||
|
||||
# Run mutation testing
|
||||
dotnet stryker
|
||||
|
||||
# With specific config
|
||||
dotnet stryker --config-file stryker-config.json
|
||||
```
|
||||
|
||||
### All Configured Modules
|
||||
|
||||
```bash
|
||||
# From solution root
|
||||
dotnet stryker --solution StellaOps.Router.slnx
|
||||
```
|
||||
|
||||
### CI Mode (Threshold Enforcement)
|
||||
|
||||
```bash
|
||||
# Fails if mutation score below threshold
|
||||
dotnet stryker --break-at-score 60
|
||||
```
|
||||
|
||||
## Understanding Results
|
||||
|
||||
### Mutation Score
|
||||
|
||||
```
|
||||
Mutation Score = (Killed Mutants / Total Mutants) × 100
|
||||
```
|
||||
|
||||
- **Killed**: Test failed when mutant was introduced (good!)
|
||||
- **Survived**: Test passed with mutant present (test gap!)
|
||||
- **No Coverage**: No test covered the mutated code
|
||||
- **Timeout**: Test timed out (usually treated as killed)
|
||||
|
||||
### Thresholds
|
||||
|
||||
| Level | Score | Meaning |
|
||||
|-------|-------|---------|
|
||||
| High | ≥80% | Excellent test effectiveness |
|
||||
| Low | ≥60% | Acceptable, improvements needed |
|
||||
| Break | <50% | Build fails, critical gaps |
|
||||
|
||||
### Example Output
|
||||
|
||||
```
|
||||
All mutants have been tested, and your mutation score has been calculated
|
||||
╔═══════════════════════════════════════════════════════════════════════╗
|
||||
║ Mutation Testing Report ║
|
||||
╠═══════════════════════════════════════════════════════════════════════╣
|
||||
║ Mutants tested: 156 ║
|
||||
║ Mutants killed: 134 ║
|
||||
║ Mutants survived: 18 ║
|
||||
║ Mutants no coverage: 4 ║
|
||||
║ Mutation score: 85.90% ║
|
||||
╚═══════════════════════════════════════════════════════════════════════╝
|
||||
```
|
||||
|
||||
## Common Mutators
|
||||
|
||||
| Mutator | Original | Mutant |
|
||||
|---------|----------|--------|
|
||||
| Comparison | `>=` | `>` |
|
||||
| Equality | `==` | `!=` |
|
||||
| Boolean | `true` | `false` |
|
||||
| Logical | `&&` | `\|\|` |
|
||||
| Arithmetic | `+` | `-` |
|
||||
| NullCoalescing | `??` | ` ` (remove) |
|
||||
|
||||
## Fixing Survived Mutants
|
||||
|
||||
### 1. Analyze the Report
|
||||
|
||||
Open the HTML report in `.stryker/output/<module>/mutation-report.html`.
|
||||
|
||||
### 2. Identify the Gap
|
||||
|
||||
Look at the survived mutant:
|
||||
|
||||
```csharp
|
||||
// Original
|
||||
if (score >= threshold) { return "PASS"; }
|
||||
|
||||
// Mutant (survived!)
|
||||
if (score > threshold) { return "PASS"; }
|
||||
```
|
||||
|
||||
### 3. Add Missing Test
|
||||
|
||||
```csharp
|
||||
[Fact]
|
||||
public void Should_Pass_When_Score_Equals_Threshold()
|
||||
{
|
||||
var score = 60;
|
||||
var threshold = 60;
|
||||
|
||||
var result = EvaluateScore(score, threshold);
|
||||
|
||||
result.Should().Be("PASS"); // Now kills the >= to > mutant
|
||||
}
|
||||
```
|
||||
|
||||
## Best Practices
|
||||
|
||||
### 1. Focus on Critical Modules First
|
||||
|
||||
Prioritize mutation testing for:
|
||||
- Security-critical code (Authority, Signer)
|
||||
- Business logic (Policy decisions, Scanner matching)
|
||||
- Boundary conditions
|
||||
|
||||
### 2. Don't Chase 100%
|
||||
|
||||
Some mutants are false positives or equivalent mutants. Aim for 80%+ on critical modules.
|
||||
|
||||
### 3. Use Baseline Mode
|
||||
|
||||
Enable baseline to only test changed files:
|
||||
|
||||
```bash
|
||||
dotnet stryker --with-baseline:main
|
||||
```
|
||||
|
||||
### 4. Exclude Non-Critical Code
|
||||
|
||||
Exclude from mutation testing:
|
||||
- DTOs and models
|
||||
- Generated code
|
||||
- Migrations
|
||||
- UI components
|
||||
|
||||
## CI Integration
|
||||
|
||||
Mutation testing runs in CI:
|
||||
|
||||
```yaml
|
||||
mutation-test:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- name: Run Stryker
|
||||
run: |
|
||||
dotnet tool restore
|
||||
dotnet stryker --break-at-score 60
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Slow Execution
|
||||
|
||||
- Use `--concurrency` to control parallelism
|
||||
- Enable `coverage-analysis: perTest` for smarter mutant selection
|
||||
- Use `--since:main` to only test changed code
|
||||
|
||||
### Out of Memory
|
||||
|
||||
- Reduce `--concurrency` value
|
||||
- Exclude large test projects
|
||||
|
||||
### Timeout Issues
|
||||
|
||||
- Adjust `--timeout` setting
|
||||
- Some infinite loop mutants may timeout (this is expected)
|
||||
|
||||
## References
|
||||
|
||||
- [Stryker.NET Documentation](https://stryker-mutator.io/docs/stryker-net/introduction/)
|
||||
- [Mutation Testing Theory](https://en.wikipedia.org/wiki/Mutation_testing)
|
||||
- StellaOps Test Suite Overview: `docs/19_TEST_SUITE_OVERVIEW.md`
|
||||
Reference in New Issue
Block a user