git.stella-ops.org/tests/AGENTS.md

tests/AGENTS.md

Overview

This document provides guidance for AI agents and developers working in the tests/ directory of the StellaOps codebase.

Directory Structure

tests/
├── acceptance/           # Acceptance test suites
├── AirGap/              # Air-gap specific tests
├── authority/           # Authority module tests
├── chaos/               # Chaos engineering tests
├── e2e/                 # End-to-end test suites
├── EvidenceLocker/      # Evidence storage tests
├── fixtures/            # Shared test fixtures
│   ├── offline-bundle/  # Offline bundle for air-gap tests
│   ├── images/          # Container image tarballs
│   └── sboms/           # Sample SBOM documents
├── Graph/               # Graph module tests
├── integration/         # Integration test suites
├── interop/             # Interoperability tests
├── load/                # Load testing scripts
├── native/              # Native code tests
├── offline/             # Offline operation tests
├── plugins/             # Plugin tests
├── Policy/              # Policy module tests
├── Provenance/          # Provenance/attestation tests
├── reachability/        # Reachability analysis tests
├── Replay/              # Replay functionality tests
├── security/            # Security tests (OWASP)
├── shared/              # Shared test utilities
└── Vex/                 # VEX processing tests

Test Categories

When writing tests, use appropriate category traits:

[Trait("Category", "Unit")]           // Fast, isolated unit tests
[Trait("Category", "Integration")]    // Tests requiring infrastructure
[Trait("Category", "E2E")]            // Full end-to-end workflows
[Trait("Category", "AirGap")]         // Must work without network
[Trait("Category", "Interop")]        // Third-party tool compatibility
[Trait("Category", "Performance")]    // Performance benchmarks
[Trait("Category", "Chaos")]          // Failure injection tests
[Trait("Category", "Security")]       // Security-focused tests

Key Patterns

1. PostgreSQL Integration Tests

Use the shared fixture from StellaOps.Infrastructure.Postgres.Testing:

public class MyIntegrationTests : IClassFixture<MyPostgresFixture>
{
    private readonly MyPostgresFixture _fixture;

    public MyIntegrationTests(MyPostgresFixture fixture)
    {
        _fixture = fixture;
    }

    [Fact]
    public async Task MyTest()
    {
        // _fixture.ConnectionString is available
        // _fixture.TruncateAllTablesAsync() for cleanup
    }
}

2. Air-Gap Tests

Inherit from NetworkIsolatedTestBase for network-free tests:

[Trait("Category", "AirGap")]
public class OfflineTests : NetworkIsolatedTestBase
{
    [Fact]
    public async Task Test_WorksOffline()
    {
        // Test implementation
        AssertNoNetworkCalls();  // Fails if network accessed
    }

    protected string GetOfflineBundlePath() =>
        Path.Combine(AppContext.BaseDirectory, "fixtures", "offline-bundle");
}

3. Determinism Tests

Use DeterminismVerifier to ensure reproducibility:

[Fact]
public void Output_IsDeterministic()
{
    var verifier = new DeterminismVerifier();
    var result = verifier.Verify(myObject, iterations: 10);

    result.IsDeterministic.Should().BeTrue();
}

4. Golden Corpus Tests

Reference cases from bench/golden-corpus/:

[Theory]
[MemberData(nameof(GetCorpusCases))]
public async Task Corpus_Case_Passes(string caseId)
{
    var testCase = CorpusLoader.Load(caseId);
    var result = await ProcessAsync(testCase.Input);
    result.Should().BeEquivalentTo(testCase.Expected);
}

Rules for Test Development

DO:

1. Tag tests with appropriate categories for filtering
2. Use Testcontainers for infrastructure dependencies
3. Inherit from shared fixtures to avoid duplication
4. Assert no network calls in air-gap tests
5. Verify determinism for any serialization output
6. Use property-based tests (FsCheck) for invariants
7. Document test purpose in method names

DON'T:

1. Don't skip tests without documenting why
2. Don't use Thread.Sleep - use proper async waits
3. Don't hardcode paths - use AppContext.BaseDirectory
4. Don't make network calls in non-interop tests
5. Don't depend on test execution order
6. Don't leave test data in shared databases

Test Infrastructure

Required Services (CI)

services:
  postgres:
    image: postgres:16-alpine
    env:
      POSTGRES_PASSWORD: test
  valkey:
    image: valkey/valkey:7-alpine

Environment Variables

Variable	Purpose	Default
STELLAOPS_OFFLINE_MODE	Enable offline mode	false
STELLAOPS_OFFLINE_BUNDLE	Path to offline bundle	-
STELLAOPS_TEST_POSTGRES	PostgreSQL connection	Testcontainers
STELLAOPS_TEST_VALKEY	Valkey connection	Testcontainers

Related Sprints

Sprint	Topic
5100.0001.0001	Run Manifest Schema
5100.0001.0002	Evidence Index Schema
5100.0001.0004	Golden Corpus Expansion
5100.0002.0001	Canonicalization Utilities
5100.0002.0002	Replay Runner Service
5100.0003.0001	SBOM Interop Round-Trip
5100.0003.0002	No-Egress Enforcement
5100.0005.0001	Router Chaos Suite

Contact

For test infrastructure questions, see:

• docs/19_TEST_SUITE_OVERVIEW.md
• docs/implplan/SPRINT_5100_SUMMARY.md
• Sprint files in docs/implplan/SPRINT_5100_*.md