git.stella-ops.org/src/__Tests/AGENTS.md

src/__Tests/AGENTS.md

Purpose & Scope

This directory contains all global test infrastructure, benchmarks, datasets, and shared testing libraries for the StellaOps platform.

• Working directory: src/__Tests/
• Roles: QA engineer, performance/bench engineer, integration test developer, docs contributor

Directory Structure

src/__Tests/
├── __Libraries/           # Shared testing libraries
│   ├── StellaOps.Infrastructure.Postgres.Testing/
│   ├── StellaOps.Messaging.Testing/
│   ├── StellaOps.Testing.AirGap/
│   ├── StellaOps.Testing.Determinism/
│   ├── StellaOps.Testing.Manifests/
│   ├── StellaOps.Concelier.Testing/
│   └── StellaOps.Router.Testing/
├── __Benchmarks/          # Golden corpus, CVE findings, determinism fixtures
│   ├── golden-corpus/     # Canonical test cases (severity, VEX, reachability)
│   ├── findings/          # CVE bundles with reachability evidence
│   ├── reachability-benchmark/  # Public multi-language benchmark
│   ├── determinism/       # Determinism test fixtures
│   └── tools/             # Verification utilities
├── __Datasets/            # Ground truth samples, schemas
│   └── reachability/      # Reachability ground truth
├── Integration/           # Cross-module integration tests
├── acceptance/            # Acceptance test packs
├── load/                  # k6 load tests
├── security/              # OWASP security tests
├── chaos/                 # Chaos engineering tests
├── AirGap/                # Offline operation tests
├── reachability/          # Reachability analysis tests
├── fixtures/              # Shared test fixtures (offline-bundle, images, sboms)
└── ...                    # Other test categories

Required Reading

Before working in this directory:

• docs/README.md
• docs/19_TEST_SUITE_OVERVIEW.md
• src/__Tests/__Benchmarks/README.md
• Sprint-specific guidance for corpus/bench artifacts

Test Categories

When writing tests, use appropriate xUnit traits:

[Trait("Category", "Unit")]           // Fast, isolated unit tests
[Trait("Category", "Integration")]    // Tests requiring infrastructure
[Trait("Category", "E2E")]            // Full end-to-end workflows
[Trait("Category", "AirGap")]         // Must work without network
[Trait("Category", "Interop")]        // Third-party tool compatibility
[Trait("Category", "Performance")]    // Performance benchmarks
[Trait("Category", "Chaos")]          // Failure injection tests
[Trait("Category", "Security")]       // Security-focused tests

Key Patterns

1. PostgreSQL Integration Tests

Use the shared fixture from __Libraries/StellaOps.Infrastructure.Postgres.Testing:

public class MyIntegrationTests : IClassFixture<MyPostgresFixture>
{
    private readonly MyPostgresFixture _fixture;

    public MyIntegrationTests(MyPostgresFixture fixture)
    {
        _fixture = fixture;
    }

    [Fact]
    public async Task MyTest()
    {
        // _fixture.ConnectionString is available
        // _fixture.TruncateAllTablesAsync() for cleanup
    }
}

2. Air-Gap Tests

Inherit from NetworkIsolatedTestBase for network-free tests:

[Trait("Category", "AirGap")]
public class OfflineTests : NetworkIsolatedTestBase
{
    [Fact]
    public async Task Test_WorksOffline()
    {
        AssertNoNetworkCalls();  // Fails if network accessed
    }

    protected string GetOfflineBundlePath() =>
        Path.Combine(AppContext.BaseDirectory, "fixtures", "offline-bundle");
}

3. Determinism Tests

Use DeterminismVerifier to ensure reproducibility:

[Fact]
public void Output_IsDeterministic()
{
    var verifier = new DeterminismVerifier();
    var result = verifier.Verify(myObject, iterations: 10);
    result.IsDeterministic.Should().BeTrue();
}

4. Golden Corpus Tests

Reference cases from __Benchmarks/golden-corpus/:

[Theory]
[MemberData(nameof(GetCorpusCases))]
public async Task Corpus_Case_Passes(string caseId)
{
    var testCase = CorpusLoader.Load(caseId);
    var result = await ProcessAsync(testCase.Input);
    result.Should().BeEquivalentTo(testCase.Expected);
}

Working Agreements

1. Determinism: Stable ordering, fixed seeds, UTC timestamps
2. Offline-first: No network dependencies unless explicitly required
3. Testcontainers: Use PostgreSQL fixtures from __Libraries/
4. Air-gap validation: Inherit from NetworkIsolatedTestBase
5. Golden corpus: Reference cases from __Benchmarks/golden-corpus/
6. Fixtures: Keep ASCII and reproducible; avoid oversized binaries

Module Tests vs Global Tests

• Module tests: Stay in src/<Module>/__Tests/ - component-specific testing
• Global tests: Go in src/__Tests/ - cross-cutting, infrastructure, benchmarks, integration

Rules for Test Development

DO:

1. Tag tests with appropriate categories for filtering
2. Use Testcontainers for infrastructure dependencies
3. Inherit from shared fixtures to avoid duplication
4. Assert no network calls in air-gap tests
5. Verify determinism for any serialization output
6. Use property-based tests (FsCheck) for invariants
7. Document test purpose in method names

DON'T:

1. Don't skip tests without documenting why
2. Don't use Thread.Sleep - use proper async waits
3. Don't hardcode paths - use AppContext.BaseDirectory
4. Don't make network calls in non-interop tests
5. Don't depend on test execution order
6. Don't leave test data in shared databases

Environment Variables

Variable	Purpose	Default
STELLAOPS_OFFLINE_MODE	Enable offline mode	false
STELLAOPS_OFFLINE_BUNDLE	Path to offline bundle	-
STELLAOPS_TEST_POSTGRES	PostgreSQL connection	Testcontainers
STELLAOPS_TEST_VALKEY	Valkey connection	Testcontainers

Related Documentation

• docs/19_TEST_SUITE_OVERVIEW.md - Comprehensive test taxonomy
• docs/testing/webservice-test-discipline.md - WebService test patterns
• docs/testing/SPRINT_EXECUTION_PLAYBOOK.md - Sprint execution guide
• docs/dev/fixtures.md - Fixture maintenance patterns