# src/__Tests/AGENTS.md

## Purpose & Scope

This directory contains all global test infrastructure, benchmarks, datasets, and shared testing libraries for the StellaOps platform.

- **Working directory:** `src/__Tests/`
- **Roles:** QA engineer, performance/bench engineer, integration test developer, docs contributor

## Directory Structure

```
src/__Tests/
├── __Libraries/           # Shared testing libraries
│   ├── StellaOps.Infrastructure.Postgres.Testing/
│   ├── StellaOps.Messaging.Testing/
│   ├── StellaOps.Testing.AirGap/
│   ├── StellaOps.Testing.Determinism/
│   ├── StellaOps.Testing.Manifests/
│   ├── StellaOps.Concelier.Testing/
│   └── StellaOps.Router.Testing/
├── __Benchmarks/          # Golden corpus, CVE findings, determinism fixtures
│   ├── golden-corpus/     # Canonical test cases (severity, VEX, reachability)
│   ├── findings/          # CVE bundles with reachability evidence
│   ├── reachability-benchmark/  # Public multi-language benchmark
│   ├── determinism/       # Determinism test fixtures
│   └── tools/             # Verification utilities
├── __Datasets/            # Ground truth samples, schemas
│   └── reachability/      # Reachability ground truth
├── Integration/           # Cross-module integration tests
├── acceptance/            # Acceptance test packs
├── load/                  # k6 load tests
├── security/              # OWASP security tests
├── chaos/                 # Chaos engineering tests
├── AirGap/                # Offline operation tests
├── reachability/          # Reachability analysis tests
├── fixtures/              # Shared test fixtures (offline-bundle, images, sboms)
└── ...                    # Other test categories
```

## Required Reading

Before working in this directory:
- `docs/README.md`
- `docs/19_TEST_SUITE_OVERVIEW.md`
- `src/__Tests/__Benchmarks/README.md`
- Sprint-specific guidance for corpus/bench artifacts

## Test Categories

Use the standardized test categories from `StellaOps.TestKit.TestCategories`:

```csharp
using StellaOps.TestKit;

// PR-GATING TESTS (run on every push/PR)
[Trait("Category", TestCategories.Unit)]           // Fast, in-memory, no external dependencies
[Trait("Category", TestCategories.Architecture)]   // Module dependency rules, naming conventions
[Trait("Category", TestCategories.Contract)]       // API/WebService contract verification
[Trait("Category", TestCategories.Integration)]    // Testcontainers, PostgreSQL, Valkey
[Trait("Category", TestCategories.Security)]       // Cryptographic validation, vulnerability scanning
[Trait("Category", TestCategories.Golden)]         // Output comparison against known-good references

// SCHEDULED/ON-DEMAND TESTS
[Trait("Category", TestCategories.Performance)]    // Performance measurements, SLO enforcement
[Trait("Category", TestCategories.Benchmark)]      // BenchmarkDotNet measurements
[Trait("Category", TestCategories.AirGap)]         // Offline/air-gapped environment validation
[Trait("Category", TestCategories.Chaos)]          // Fault injection, failure recovery
[Trait("Category", TestCategories.Determinism)]    // Reproducibility, stable ordering, idempotency
[Trait("Category", TestCategories.Resilience)]     // Retry policies, circuit breakers, timeouts
[Trait("Category", TestCategories.Observability)]  // OpenTelemetry traces, metrics, logging

// OTHER CATEGORIES
[Trait("Category", TestCategories.Property)]       // FsCheck/generative testing for invariants
[Trait("Category", TestCategories.Snapshot)]       // Golden master regression testing
[Trait("Category", TestCategories.Live)]           // Require external services (Rekor, feeds)
```

### CI/CD Integration

Tests are discovered dynamically by `.gitea/workflows/test-matrix.yml` which runs all `*.Tests.csproj` files with Category filtering:
- **PR-Gating:** Unit, Architecture, Contract, Integration, Security, Golden
- **Scheduled:** Performance, Benchmark (daily)
- **On-Demand:** AirGap, Chaos, Determinism, Resilience, Observability

### Validation

Run the validation script to ensure all tests have Category traits:
```bash
python devops/scripts/validate-test-traits.py           # Report coverage
python devops/scripts/validate-test-traits.py --fix     # Add default Unit trait
python devops/scripts/validate-test-traits.py --json    # JSON output for CI
```

## Key Patterns

### 1. PostgreSQL Integration Tests

Use the shared fixture from `__Libraries/StellaOps.Infrastructure.Postgres.Testing`:

```csharp
public class MyIntegrationTests : IClassFixture<MyPostgresFixture>
{
    private readonly MyPostgresFixture _fixture;

    public MyIntegrationTests(MyPostgresFixture fixture)
    {
        _fixture = fixture;
    }

    [Fact]
    public async Task MyTest()
    {
        // _fixture.ConnectionString is available
        // _fixture.TruncateAllTablesAsync() for cleanup
    }
}
```

### 2. Air-Gap Tests

Inherit from `NetworkIsolatedTestBase` for network-free tests:

```csharp
[Trait("Category", "AirGap")]
public class OfflineTests : NetworkIsolatedTestBase
{
    [Fact]
    public async Task Test_WorksOffline()
    {
        AssertNoNetworkCalls();  // Fails if network accessed
    }

    protected string GetOfflineBundlePath() =>
        Path.Combine(AppContext.BaseDirectory, "fixtures", "offline-bundle");
}
```

### 3. Determinism Tests

Use `DeterminismVerifier` to ensure reproducibility:

```csharp
[Fact]
public void Output_IsDeterministic()
{
    var verifier = new DeterminismVerifier();
    var result = verifier.Verify(myObject, iterations: 10);
    result.IsDeterministic.Should().BeTrue();
}
```

### 4. Golden Corpus Tests

Reference cases from `__Benchmarks/golden-corpus/`:

```csharp
[Theory]
[MemberData(nameof(GetCorpusCases))]
public async Task Corpus_Case_Passes(string caseId)
{
    var testCase = CorpusLoader.Load(caseId);
    var result = await ProcessAsync(testCase.Input);
    result.Should().BeEquivalentTo(testCase.Expected);
}
```

## Working Agreements

1. **Determinism:** Stable ordering, fixed seeds, UTC timestamps
2. **Offline-first:** No network dependencies unless explicitly required
3. **Testcontainers:** Use PostgreSQL fixtures from `__Libraries/`
4. **Air-gap validation:** Inherit from `NetworkIsolatedTestBase`
5. **Golden corpus:** Reference cases from `__Benchmarks/golden-corpus/`
6. **Fixtures:** Keep ASCII and reproducible; avoid oversized binaries

## Module Tests vs Global Tests

- **Module tests:** Stay in `src/<Module>/__Tests/` - component-specific testing
- **Global tests:** Go in `src/__Tests/` - cross-cutting, infrastructure, benchmarks, integration

## Rules for Test Development

### DO:
1. Tag tests with appropriate categories for filtering
2. Use Testcontainers for infrastructure dependencies
3. Inherit from shared fixtures to avoid duplication
4. Assert no network calls in air-gap tests
5. Verify determinism for any serialization output
6. Use property-based tests (FsCheck) for invariants
7. Document test purpose in method names

### DON'T:
1. Don't skip tests without documenting why
2. Don't use `Thread.Sleep` - use proper async waits
3. Don't hardcode paths - use `AppContext.BaseDirectory`
4. Don't make network calls in non-interop tests
5. Don't depend on test execution order
6. Don't leave test data in shared databases

## Environment Variables

| Variable | Purpose | Default |
|----------|---------|---------|
| `STELLAOPS_OFFLINE_MODE` | Enable offline mode | `false` |
| `STELLAOPS_OFFLINE_BUNDLE` | Path to offline bundle | - |
| `STELLAOPS_TEST_POSTGRES` | PostgreSQL connection | Testcontainers |
| `STELLAOPS_TEST_VALKEY` | Valkey connection | Testcontainers |

## Related Documentation

- `docs/19_TEST_SUITE_OVERVIEW.md` - Comprehensive test taxonomy
- `docs/testing/webservice-test-discipline.md` - WebService test patterns
- `docs/testing/SPRINT_EXECUTION_PLAYBOOK.md` - Sprint execution guide
- `docs/dev/fixtures.md` - Fixture maintenance patterns