46 lines
4.9 KiB
Markdown
46 lines
4.9 KiB
Markdown
# Shared TestKit Library with Deterministic Infrastructure
|
|
|
|
## Module
|
|
__Libraries
|
|
|
|
## Status
|
|
VERIFIED
|
|
|
|
## Description
|
|
Comprehensive shared test kit with standardized test categories (Unit, Integration, Property, Snapshot, Contract, Security, Performance, Live, Architecture, Golden, Benchmark, AirGap, Chaos, Determinism, Resilience, Observability), blast-radius annotations for operational surfaces, deterministic helpers (DeterministicTime, DeterministicRandom), fixture support (PostgresFixture, ValkeyFixture, HttpFixtureServer, WebServiceFixture), assertion utilities (CanonicalJsonAssert, SnapshotAssert, EvidenceChainAssert, LogContractAssert, MetricsContractAssert, OTelContractAssert), and stability/incident testing infrastructure.
|
|
|
|
## Implementation Details
|
|
- **TestCategories**: `src/__Libraries/StellaOps.TestKit/TestCategories.cs` -- static class with 20+ category constants: `Unit`, `Property`, `Snapshot`, `Integration`, `Contract`, `Security`, `Performance`, `Live`, `Architecture`, `Golden`, `Benchmark`, `AirGap`, `Chaos`, `Determinism`, `Resilience`, `Observability`, `StorageConcurrency`, `StorageIdempotency`, `QueryDeterminism`, `StorageMigration`, `SchemaEvolution`, `ConfigDiff`, `HLC`, `Federation`, `Latency`, `Immutability`, `Parity`, `PostIncident`, `EvidenceChain`, `Longevity`, `Interop`, `EnvironmentSkew`; nested `BlastRadius` class with `Auth`, `Scanning`, `Evidence`, `Compliance`, `Advisories`, `RiskPolicy`, `Crypto`, `Integrations`, `Persistence`, `Api`
|
|
- **DeterministicTime**: `src/__Libraries/StellaOps.TestKit/Deterministic/DeterministicTime.cs` -- sealed class implementing `IDisposable`; constructor takes UTC `DateTime`; `UtcNow` property returns fixed time (thread-safe with lock); `Advance(TimeSpan)` moves time forward/backward; `SetTo(DateTime)` sets to specific time; `Reset(DateTime)` resets to new start
|
|
- **DeterministicRandom**: `src/__Libraries/StellaOps.TestKit/Deterministic/DeterministicRandom.cs` -- deterministic random number generator for reproducible test scenarios
|
|
- **Assertions**: `src/__Libraries/StellaOps.TestKit/CanonicalJsonAssert.cs` (canonical JSON equality), `SnapshotAssert.cs` (golden snapshot comparison), `EvidenceChainAssert.cs` (evidence chain verification), `LogContractAssert.cs` (structured log contract assertions), `MetricsContractAssert.cs` (metrics contract verification), `OTelContractAssert.cs` (OpenTelemetry contract assertions)
|
|
- **Fixtures**: `src/__Libraries/StellaOps.TestKit/PostgresFixture.cs` (Testcontainers PostgreSQL), `ValkeyFixture.cs` (Testcontainers Valkey/Redis), `HttpFixtureServer.cs` (HTTP test server), `WebServiceFixture.cs` (web service test fixture), `ConnectorHttpFixture.cs` (connector HTTP fixture)
|
|
- **Stability**: `src/__Libraries/StellaOps.TestKit/StabilityMetrics.cs` and `StabilityTestRunner.cs` -- stability measurement and runner
|
|
- **Blast Radius**: `src/__Libraries/StellaOps.TestKit/BlastRadiusTestRunner.cs` and `BlastRadiusValidator.cs` -- targeted blast-radius test execution
|
|
- **Incident Testing**: `src/__Libraries/StellaOps.TestKit/IncidentMetadata.cs` and `IncidentTestGenerator.cs` -- post-incident regression test generation
|
|
- **Schema/Version Compatibility**: `src/__Libraries/StellaOps.TestKit/SchemaVersionMatrix.cs` and `VersionCompatibilityFixture.cs` -- multi-version schema testing
|
|
- **Storage Testing**: `src/__Libraries/StellaOps.TestKit/CacheIdempotencyTests.cs`, `QueryDeterminismTests.cs`, `StorageConcurrencyTests.cs`, `StorageIdempotencyTests.cs` -- reusable storage test bases
|
|
- **Connector Testing**: `src/__Libraries/StellaOps.TestKit/ConnectorTestBase.cs`, `ConnectorLiveSchemaTestBase.cs`, `ConnectorResilienceTestBase.cs`, `ConnectorSecurityTestBase.cs` -- connector integration test bases
|
|
- **Observability**: `src/__Libraries/StellaOps.TestKit/OtelCapture.cs` -- OpenTelemetry capture for test assertions
|
|
- **Skew Testing**: `src/__Libraries/StellaOps.TestKit/SkewTestRunner.cs`, `EnvironmentProfile.cs` -- environment skew simulation
|
|
- **Intent Tracking**: `src/__Libraries/StellaOps.TestKit/IntentAttribute.cs`, `TestIntents.cs`, `IntentCoverageReport.cs`, `RequirementAttribute.cs` -- requirement-to-test traceability
|
|
- **Source**: Feature matrix scan
|
|
|
|
## E2E Test Plan
|
|
- [ ] Verify TestCategories constants enable filtering via `dotnet test --filter "Category=Unit"`
|
|
- [ ] Test DeterministicTime provides fixed UTC time and supports Advance/SetTo/Reset
|
|
- [ ] Verify DeterministicRandom produces reproducible sequences from same seed
|
|
- [ ] Test PostgresFixture creates and manages Testcontainers PostgreSQL instances
|
|
- [ ] Verify ValkeyFixture creates and manages Testcontainers Valkey instances
|
|
- [ ] Test CanonicalJsonAssert correctly compares JSON with key-order independence
|
|
- [ ] Verify SnapshotAssert detects golden snapshot deviations
|
|
- [ ] Test BlastRadiusTestRunner filters and runs tests by blast-radius annotation
|
|
|
|
|
|
## Verification
|
|
|
|
- **Verified**: 2026-02-13T20:30:00Z
|
|
- **Run**: run-001
|
|
- **Tier**: Tier 2d (Library/Internal)
|
|
- **Verdict**: PASS
|