consolidate the tests locations
This commit is contained in:
StellaOps Bot
184
src/__Tests/AGENTS.md
Normal file
184
src/__Tests/AGENTS.md
Normal file
@@ -0,0 +1,184 @@
|
||||
# src/__Tests/AGENTS.md
|
||||
|
||||
## Purpose & Scope
|
||||
|
||||
This directory contains all global test infrastructure, benchmarks, datasets, and shared testing libraries for the StellaOps platform.
|
||||
|
||||
- **Working directory:** `src/__Tests/`
|
||||
- **Roles:** QA engineer, performance/bench engineer, integration test developer, docs contributor
|
||||
|
||||
## Directory Structure
|
||||
|
||||
```
|
||||
src/__Tests/
|
||||
├── __Libraries/ # Shared testing libraries
|
||||
│ ├── StellaOps.Infrastructure.Postgres.Testing/
|
||||
│ ├── StellaOps.Messaging.Testing/
|
||||
│ ├── StellaOps.Testing.AirGap/
|
||||
│ ├── StellaOps.Testing.Determinism/
|
||||
│ ├── StellaOps.Testing.Manifests/
|
||||
│ ├── StellaOps.Concelier.Testing/
|
||||
│ └── StellaOps.Router.Testing/
|
||||
├── __Benchmarks/ # Golden corpus, CVE findings, determinism fixtures
|
||||
│ ├── golden-corpus/ # Canonical test cases (severity, VEX, reachability)
|
||||
│ ├── findings/ # CVE bundles with reachability evidence
|
||||
│ ├── reachability-benchmark/ # Public multi-language benchmark
|
||||
│ ├── determinism/ # Determinism test fixtures
|
||||
│ └── tools/ # Verification utilities
|
||||
├── __Datasets/ # Ground truth samples, schemas
|
||||
│ └── reachability/ # Reachability ground truth
|
||||
├── Integration/ # Cross-module integration tests
|
||||
├── acceptance/ # Acceptance test packs
|
||||
├── load/ # k6 load tests
|
||||
├── security/ # OWASP security tests
|
||||
├── chaos/ # Chaos engineering tests
|
||||
├── AirGap/ # Offline operation tests
|
||||
├── reachability/ # Reachability analysis tests
|
||||
├── fixtures/ # Shared test fixtures (offline-bundle, images, sboms)
|
||||
└── ... # Other test categories
|
||||
```
|
||||
|
||||
## Required Reading
|
||||
|
||||
Before working in this directory:
|
||||
- `docs/README.md`
|
||||
- `docs/19_TEST_SUITE_OVERVIEW.md`
|
||||
- `src/__Tests/__Benchmarks/README.md`
|
||||
- Sprint-specific guidance for corpus/bench artifacts
|
||||
|
||||
## Test Categories
|
||||
|
||||
When writing tests, use appropriate xUnit traits:
|
||||
|
||||
```csharp
|
||||
[Trait("Category", "Unit")] // Fast, isolated unit tests
|
||||
[Trait("Category", "Integration")] // Tests requiring infrastructure
|
||||
[Trait("Category", "E2E")] // Full end-to-end workflows
|
||||
[Trait("Category", "AirGap")] // Must work without network
|
||||
[Trait("Category", "Interop")] // Third-party tool compatibility
|
||||
[Trait("Category", "Performance")] // Performance benchmarks
|
||||
[Trait("Category", "Chaos")] // Failure injection tests
|
||||
[Trait("Category", "Security")] // Security-focused tests
|
||||
```
|
||||
|
||||
## Key Patterns
|
||||
|
||||
### 1. PostgreSQL Integration Tests
|
||||
|
||||
Use the shared fixture from `__Libraries/StellaOps.Infrastructure.Postgres.Testing`:
|
||||
|
||||
```csharp
|
||||
public class MyIntegrationTests : IClassFixture<MyPostgresFixture>
|
||||
{
|
||||
private readonly MyPostgresFixture _fixture;
|
||||
|
||||
public MyIntegrationTests(MyPostgresFixture fixture)
|
||||
{
|
||||
_fixture = fixture;
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task MyTest()
|
||||
{
|
||||
// _fixture.ConnectionString is available
|
||||
// _fixture.TruncateAllTablesAsync() for cleanup
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### 2. Air-Gap Tests
|
||||
|
||||
Inherit from `NetworkIsolatedTestBase` for network-free tests:
|
||||
|
||||
```csharp
|
||||
[Trait("Category", "AirGap")]
|
||||
public class OfflineTests : NetworkIsolatedTestBase
|
||||
{
|
||||
[Fact]
|
||||
public async Task Test_WorksOffline()
|
||||
{
|
||||
AssertNoNetworkCalls(); // Fails if network accessed
|
||||
}
|
||||
|
||||
protected string GetOfflineBundlePath() =>
|
||||
Path.Combine(AppContext.BaseDirectory, "fixtures", "offline-bundle");
|
||||
}
|
||||
```
|
||||
|
||||
### 3. Determinism Tests
|
||||
|
||||
Use `DeterminismVerifier` to ensure reproducibility:
|
||||
|
||||
```csharp
|
||||
[Fact]
|
||||
public void Output_IsDeterministic()
|
||||
{
|
||||
var verifier = new DeterminismVerifier();
|
||||
var result = verifier.Verify(myObject, iterations: 10);
|
||||
result.IsDeterministic.Should().BeTrue();
|
||||
}
|
||||
```
|
||||
|
||||
### 4. Golden Corpus Tests
|
||||
|
||||
Reference cases from `__Benchmarks/golden-corpus/`:
|
||||
|
||||
```csharp
|
||||
[Theory]
|
||||
[MemberData(nameof(GetCorpusCases))]
|
||||
public async Task Corpus_Case_Passes(string caseId)
|
||||
{
|
||||
var testCase = CorpusLoader.Load(caseId);
|
||||
var result = await ProcessAsync(testCase.Input);
|
||||
result.Should().BeEquivalentTo(testCase.Expected);
|
||||
}
|
||||
```
|
||||
|
||||
## Working Agreements
|
||||
|
||||
1. **Determinism:** Stable ordering, fixed seeds, UTC timestamps
|
||||
2. **Offline-first:** No network dependencies unless explicitly required
|
||||
3. **Testcontainers:** Use PostgreSQL fixtures from `__Libraries/`
|
||||
4. **Air-gap validation:** Inherit from `NetworkIsolatedTestBase`
|
||||
5. **Golden corpus:** Reference cases from `__Benchmarks/golden-corpus/`
|
||||
6. **Fixtures:** Keep ASCII and reproducible; avoid oversized binaries
|
||||
|
||||
## Module Tests vs Global Tests
|
||||
|
||||
- **Module tests:** Stay in `src/<Module>/__Tests/` - component-specific testing
|
||||
- **Global tests:** Go in `src/__Tests/` - cross-cutting, infrastructure, benchmarks, integration
|
||||
|
||||
## Rules for Test Development
|
||||
|
||||
### DO:
|
||||
1. Tag tests with appropriate categories for filtering
|
||||
2. Use Testcontainers for infrastructure dependencies
|
||||
3. Inherit from shared fixtures to avoid duplication
|
||||
4. Assert no network calls in air-gap tests
|
||||
5. Verify determinism for any serialization output
|
||||
6. Use property-based tests (FsCheck) for invariants
|
||||
7. Document test purpose in method names
|
||||
|
||||
### DON'T:
|
||||
1. Don't skip tests without documenting why
|
||||
2. Don't use `Thread.Sleep` - use proper async waits
|
||||
3. Don't hardcode paths - use `AppContext.BaseDirectory`
|
||||
4. Don't make network calls in non-interop tests
|
||||
5. Don't depend on test execution order
|
||||
6. Don't leave test data in shared databases
|
||||
|
||||
## Environment Variables
|
||||
|
||||
| Variable | Purpose | Default |
|
||||
|----------|---------|---------|
|
||||
| `STELLAOPS_OFFLINE_MODE` | Enable offline mode | `false` |
|
||||
| `STELLAOPS_OFFLINE_BUNDLE` | Path to offline bundle | - |
|
||||
| `STELLAOPS_TEST_POSTGRES` | PostgreSQL connection | Testcontainers |
|
||||
| `STELLAOPS_TEST_VALKEY` | Valkey connection | Testcontainers |
|
||||
|
||||
## Related Documentation
|
||||
|
||||
- `docs/19_TEST_SUITE_OVERVIEW.md` - Comprehensive test taxonomy
|
||||
- `docs/testing/webservice-test-discipline.md` - WebService test patterns
|
||||
- `docs/testing/SPRINT_EXECUTION_PLAYBOOK.md` - Sprint execution guide
|
||||
- `docs/dev/fixtures.md` - Fixture maintenance patterns
|
||||
Reference in New Issue
Block a user