Files

master 644887997c test fixes and new product advisories work

2026-01-28 02:30:48 +02:00

24 KiB

Raw Permalink Blame History

StellaOps.TestKit Usage Guide

Version: 1.0 Status: Pilot Release (Wave 4 Complete) Audience: StellaOps developers writing unit, integration, and contract tests

Overview

StellaOps.TestKit provides deterministic testing infrastructure for StellaOps modules. It eliminates flaky tests, provides reproducible test primitives, and standardizes fixtures for integration testing.

Key Features

Deterministic Time: Freeze and advance time for reproducible tests
Deterministic Random: Seeded random number generation
Canonical JSON Assertions: SHA-256 hash verification for determinism
Snapshot Testing: Golden master regression testing
PostgreSQL Fixture: Testcontainers-based PostgreSQL 16 for integration tests
Valkey Fixture: Redis-compatible caching tests
HTTP Fixture: In-memory API contract testing
OpenTelemetry Capture: Trace and span assertion helpers
Test Categories: Standardized trait constants for CI filtering

Installation

Add StellaOps.TestKit as a project reference to your test project:

<ItemGroup>
  <ProjectReference Include="../../../__Libraries/StellaOps.TestKit/StellaOps.TestKit.csproj" />
</ItemGroup>

Quick Start Examples

1. Deterministic Time

Eliminate flaky tests caused by time-dependent logic:

using StellaOps.TestKit.Deterministic;
using Xunit;

[Fact]
public void Test_ExpirationLogic()
{
    // Arrange: Fix time at a known UTC timestamp
    using var time = new DeterministicTime(new DateTime(2026, 1, 15, 10, 30, 0, DateTimeKind.Utc));

    var expiresAt = time.UtcNow.AddHours(24);

    // Act: Advance time to just before expiration
    time.Advance(TimeSpan.FromHours(23));
    Assert.False(time.UtcNow > expiresAt);

    // Advance past expiration
    time.Advance(TimeSpan.FromHours(2));
    Assert.True(time.UtcNow > expiresAt);
}

API Reference:

DeterministicTime(DateTime initialUtc) - Create with fixed start time
UtcNow - Get current deterministic time
Advance(TimeSpan duration) - Move time forward
SetTo(DateTime newUtc) - Jump to specific time

2. Deterministic Random

Reproducible random sequences for property tests and fuzzing:

using StellaOps.TestKit.Deterministic;

[Fact]
public void Test_RandomIdGeneration()
{
    // Arrange: Same seed produces same sequence
    var random1 = new DeterministicRandom(seed: 42);
    var random2 = new DeterministicRandom(seed: 42);

    // Act
    var guid1 = random1.NextGuid();
    var guid2 = random2.NextGuid();

    // Assert: Reproducible GUIDs
    Assert.Equal(guid1, guid2);
}

[Fact]
public void Test_Shuffling()
{
    var random = new DeterministicRandom(seed: 100);
    var array = new[] { 1, 2, 3, 4, 5 };

    random.Shuffle(array);

    // Deterministic shuffle order
    Assert.NotEqual(new[] { 1, 2, 3, 4, 5 }, array);
}

API Reference:

DeterministicRandom(int seed) - Create with seed
NextGuid() - Generate deterministic GUID
NextString(int length) - Generate alphanumeric string
NextInt(int min, int max) - Generate integer in range
Shuffle<T>(T[] array) - Fisher-Yates shuffle

3. Canonical JSON Assertions

Verify JSON determinism for SBOM, VEX, and attestation outputs:

using StellaOps.TestKit.Assertions;

[Fact]
public void Test_SbomDeterminism()
{
    var sbom = new
    {
        SpdxVersion = "SPDX-3.0.1",
        Name = "MySbom",
        Packages = new[] { new { Name = "Pkg1", Version = "1.0" } }
    };

    // Verify deterministic serialization
    CanonicalJsonAssert.IsDeterministic(sbom, iterations: 100);

    // Verify expected hash (golden master)
    var expectedHash = "abc123..."; // Precomputed SHA-256
    CanonicalJsonAssert.HasExpectedHash(sbom, expectedHash);
}

[Fact]
public void Test_JsonPropertyExists()
{
    var vex = new
    {
        Document = new { Id = "VEX-2026-001" },
        Statements = new[] { new { Vulnerability = "CVE-2026-1234" } }
    };

    // Deep property verification
    CanonicalJsonAssert.ContainsProperty(vex, "Document.Id", "VEX-2026-001");
    CanonicalJsonAssert.ContainsProperty(vex, "Statements[0].Vulnerability", "CVE-2026-1234");
}

API Reference:

IsDeterministic<T>(T value, int iterations) - Verify N serializations match
HasExpectedHash<T>(T value, string expectedSha256Hex) - Verify SHA-256 hash
ComputeCanonicalHash<T>(T value) - Compute hash for golden master
AreCanonicallyEqual<T>(T expected, T actual) - Compare canonical JSON
ContainsProperty<T>(T value, string propertyPath, object expectedValue) - Deep search

4. Snapshot Testing

Golden master regression testing for complex outputs:

using StellaOps.TestKit.Assertions;

[Fact, Trait("Category", TestCategories.Snapshot)]
public void Test_SbomGeneration()
{
    var sbom = GenerateSbom(); // Your SBOM generation logic

    // Snapshot will be stored in Snapshots/TestSbomGeneration.json
    SnapshotAssert.MatchesSnapshot(sbom, "TestSbomGeneration");
}

// Update snapshots when intentional changes occur:
// UPDATE_SNAPSHOTS=1 dotnet test

Text and Binary Snapshots:

[Fact]
public void Test_LicenseText()
{
    var licenseText = GenerateLicenseNotice();
    SnapshotAssert.MatchesTextSnapshot(licenseText, "LicenseNotice");
}

[Fact]
public void Test_SignatureBytes()
{
    var signature = SignDocument(document);
    SnapshotAssert.MatchesBinarySnapshot(signature, "DocumentSignature");
}

API Reference:

MatchesSnapshot<T>(T value, string snapshotName) - JSON snapshot
MatchesTextSnapshot(string value, string snapshotName) - Text snapshot
MatchesBinarySnapshot(byte[] value, string snapshotName) - Binary snapshot
Environment variable: UPDATE_SNAPSHOTS=1 to update baselines

5. PostgreSQL Fixture

Testcontainers-based PostgreSQL 16 for integration tests:

using StellaOps.TestKit.Fixtures;
using Xunit;

public class DatabaseTests : IClassFixture<PostgresFixture>
{
    private readonly PostgresFixture _fixture;

    public DatabaseTests(PostgresFixture fixture)
    {
        _fixture = fixture;
    }

    [Fact, Trait("Category", TestCategories.Integration)]
    public async Task Test_DatabaseOperations()
    {
        // Use _fixture.ConnectionString to connect
        using var connection = new NpgsqlConnection(_fixture.ConnectionString);
        await connection.OpenAsync();

        // Run migrations
        await _fixture.RunMigrationsAsync(connection);

        // Test database operations
        var result = await connection.QueryAsync("SELECT version()");
        Assert.NotEmpty(result);
    }
}

API Reference:

PostgresFixture - xUnit class fixture
ConnectionString - PostgreSQL connection string
RunMigrationsAsync(DbConnection) - Apply migrations
Requires Docker running locally

6. Valkey Fixture

Redis-compatible caching for integration tests:

using StellaOps.TestKit.Fixtures;

public class CacheTests : IClassFixture<ValkeyFixture>
{
    private readonly ValkeyFixture _fixture;

    [Fact, Trait("Category", TestCategories.Integration)]
    public async Task Test_CachingLogic()
    {
        var connection = await ConnectionMultiplexer.Connect(_fixture.ConnectionString);
        var db = connection.GetDatabase();

        await db.StringSetAsync("key", "value");
        var result = await db.StringGetAsync("key");

        Assert.Equal("value", result.ToString());
    }
}

API Reference:

ValkeyFixture - xUnit class fixture
ConnectionString - Redis connection string (host:port)
Host, Port - Connection details
Uses valkey/valkey:7.2-alpine image (Redis-compatible)

7. HTTP Fixture Server

In-memory API contract testing:

using StellaOps.TestKit.Fixtures;

public class ApiTests : IClassFixture<HttpFixtureServer<Program>>
{
    private readonly HttpClient _client;

    public ApiTests(HttpFixtureServer<Program> fixture)
    {
        _client = fixture.CreateClient();
    }

    [Fact, Trait("Category", TestCategories.Contract)]
    public async Task Test_HealthEndpoint()
    {
        var response = await _client.GetAsync("/health");
        response.EnsureSuccessStatusCode();

        var body = await response.Content.ReadAsStringAsync();
        Assert.Contains("healthy", body);
    }
}

HTTP Message Handler Stub (Hermetic Tests):

[Fact]
public async Task Test_ExternalApiCall()
{
    var handler = new HttpMessageHandlerStub()
        .WhenRequest("https://api.example.com/data", HttpStatusCode.OK, "{\"status\":\"ok\"}");

    var httpClient = new HttpClient(handler);
    var response = await httpClient.GetAsync("https://api.example.com/data");

    Assert.Equal(HttpStatusCode.OK, response.StatusCode);
}

API Reference:

HttpFixtureServer<TProgram> - WebApplicationFactory wrapper
CreateClient() - Get HttpClient for test server
HttpMessageHandlerStub - Stub external HTTP dependencies
WhenRequest(url, statusCode, content) - Configure stub responses

8. OpenTelemetry Capture

Trace and span assertion helpers:

using StellaOps.TestKit.Observability;

[Fact]
public async Task Test_TracingBehavior()
{
    using var capture = new OtelCapture();

    // Execute code that emits traces
    await MyService.DoWorkAsync();

    // Assert traces
    capture.AssertHasSpan("MyService.DoWork");
    capture.AssertHasTag("user_id", "123");
    capture.AssertSpanCount(expectedCount: 3);

    // Verify parent-child hierarchy
    capture.AssertHierarchy("ParentSpan", "ChildSpan");
}

API Reference:

OtelCapture(string? activitySourceName = null) - Create capture
AssertHasSpan(string spanName) - Verify span exists
AssertHasTag(string tagKey, string expectedValue) - Verify tag
AssertSpanCount(int expectedCount) - Verify span count
AssertHierarchy(string parentSpanName, string childSpanName) - Verify parent-child
CapturedActivities - Get all captured spans

9. Observability Contract Testing (Turn #6)

Contract assertions for treating logs, metrics, and traces as APIs:

OTel Contract Testing:

using StellaOps.TestKit.Observability;

[Fact, Trait("Category", TestCategories.Contract)]
public async Task Test_SpanContracts()
{
    using var capture = new OtelCapture("MyService");

    await service.ProcessRequestAsync();

    // Verify required spans are present
    OTelContractAssert.HasRequiredSpans(capture, "ProcessRequest", "ValidateInput", "SaveResult");

    // Verify span attributes
    var span = capture.CapturedActivities.First();
    OTelContractAssert.SpanHasAttributes(span, "user_id", "tenant_id", "correlation_id");

    // Check attribute cardinality (prevent metric explosion)
    OTelContractAssert.AttributeCardinality(capture, "http_method", maxCardinality: 10);

    // Detect high-cardinality attributes globally
    OTelContractAssert.NoHighCardinalityAttributes(capture, threshold: 100);
}

Log Contract Testing:

using StellaOps.TestKit.Observability;
using System.Text.RegularExpressions;

[Fact]
public async Task Test_LogContracts()
{
    var logCapture = new List<CapturedLogRecord>();
    // ... capture logs during test execution ...

    // Verify required fields
    LogContractAssert.HasRequiredFields(logCapture[0], "CorrelationId", "TenantId");

    // Ensure no PII leakage
    var piiPatterns = new[]
    {
        new Regex(@"\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,}\b"),  // Email
        new Regex(@"\b\d{3}-\d{2}-\d{4}\b"),  // SSN
    };
    LogContractAssert.NoSensitiveData(logCapture, piiPatterns);

    // Verify log level appropriateness
    LogContractAssert.LogLevelAppropriate(logCapture[0], LogLevel.Information, LogLevel.Warning);

    // Ensure error logs have correlation for troubleshooting
    LogContractAssert.ErrorLogsHaveCorrelation(logCapture, "CorrelationId", "RequestId");
}

Metrics Contract Testing:

using StellaOps.TestKit.Observability;

[Fact]
public async Task Test_MetricsContracts()
{
    using var capture = new MetricsCapture("MyService");

    await service.ProcessMultipleRequests();

    // Verify required metrics exist
    MetricsContractAssert.HasRequiredMetrics(capture, "requests_total", "request_duration_seconds");

    // Check label cardinality bounds
    MetricsContractAssert.LabelCardinalityBounded(capture, "http_requests_total", maxLabels: 50);

    // Verify counter monotonicity
    MetricsContractAssert.CounterMonotonic(capture, "processed_items_total");

    // Verify gauge bounds
    MetricsContractAssert.GaugeInBounds(capture, "active_connections", minValue: 0, maxValue: 1000);
}

API Reference:

OTelContractAssert.HasRequiredSpans(capture, spanNames) - Verify spans exist
OTelContractAssert.SpanHasAttributes(span, attrNames) - Verify attributes
OTelContractAssert.AttributeCardinality(capture, attr, max) - Check cardinality
OTelContractAssert.NoHighCardinalityAttributes(capture, threshold) - Detect explosion
LogContractAssert.HasRequiredFields(record, fields) - Verify log fields
LogContractAssert.NoSensitiveData(records, patterns) - Check for PII
MetricsContractAssert.MetricExists(capture, name) - Verify metric
MetricsContractAssert.LabelCardinalityBounded(capture, name, max) - Check cardinality
MetricsCapture - Capture metrics during test execution
ContractViolationException - Thrown when contracts are violated

10. Evidence Chain Traceability (Turn #6)

Link tests to requirements for regulatory compliance and audit trails:

Requirement Attribute:

using StellaOps.TestKit.Evidence;

[Fact]
[Requirement("REQ-AUTH-001", SprintTaskId = "AUTH-0127-001")]
public async Task Test_UserAuthentication()
{
    // Verify authentication works as required
}

[Fact]
[Requirement("REQ-AUDIT-002", SprintTaskId = "AUDIT-0127-003", ComplianceControl = "SOC2-AU-12")]
public void Test_AuditLogImmutability()
{
    // Verify audit logs cannot be modified
}

Filtering tests by requirement:

# Run tests for a specific requirement
dotnet test --filter "Requirement=REQ-AUTH-001"

# Run tests for a sprint task
dotnet test --filter "SprintTask=AUTH-0127-001"

# Run tests for a compliance control
dotnet test --filter "ComplianceControl=SOC2-AU-12"

Evidence Chain Assertions:

using StellaOps.TestKit.Evidence;

[Fact]
[Requirement("REQ-EVIDENCE-001")]
public void Test_ArtifactHashStability()
{
    var artifact = GenerateEvidence(input);

    // Verify artifact produces expected hash (golden master)
    EvidenceChainAssert.ArtifactHashStable(artifact, "abc123...expected-sha256...");
}

[Fact]
[Requirement("REQ-DETERMINISM-001")]
public void Test_EvidenceImmutability()
{
    // Verify generator produces identical output across iterations
    EvidenceChainAssert.ArtifactImmutable(() => GenerateEvidence(fixedInput), iterations: 100);
}

[Fact]
[Requirement("REQ-TRACE-001")]
public void Test_TraceabilityComplete()
{
    var requirementId = "REQ-EVIDENCE-001";
    var testId = "MyTests.TestMethod";
    var artifactHash = EvidenceChainAssert.ComputeSha256(artifact);

    // Verify all traceability components present
    EvidenceChainAssert.TraceabilityComplete(requirementId, testId, artifactHash);
}

Traceability Report Generation:

using StellaOps.TestKit.Evidence;

// Generate traceability matrix from test assemblies
var reporter = new EvidenceChainReporter();
reporter.AddAssembly(typeof(MyTests).Assembly);
var report = reporter.GenerateReport();

// Output as Markdown
Console.WriteLine(report.ToMarkdown());

// Output as JSON
Console.WriteLine(report.ToJson());

API Reference:

RequirementAttribute(string requirementId) - Link test to requirement
RequirementAttribute.SprintTaskId - Link to sprint task (optional)
RequirementAttribute.ComplianceControl - Link to compliance control (optional)
EvidenceChainAssert.ArtifactHashStable(artifact, expectedHash) - Verify hash
EvidenceChainAssert.ArtifactImmutable(generator, iterations) - Verify determinism
EvidenceChainAssert.ComputeSha256(content) - Compute SHA-256 hash
EvidenceChainAssert.RequirementLinked(requirementId) - Marker assertion
EvidenceChainAssert.TraceabilityComplete(reqId, testId, artifactId) - Verify chain
EvidenceChainReporter.AddAssembly(assembly) - Add assembly to scan
EvidenceChainReporter.GenerateReport() - Generate traceability report
EvidenceChainReport.ToMarkdown() - Markdown output
EvidenceChainReport.ToJson() - JSON output
EvidenceTraceabilityException - Thrown when evidence assertions fail

11. Test Categories

Standardized trait constants for CI lane filtering:

using StellaOps.TestKit;

[Fact, Trait("Category", TestCategories.Unit)]
public void FastUnitTest() { }

[Fact, Trait("Category", TestCategories.Integration)]
public async Task SlowIntegrationTest() { }

[Fact, Trait("Category", TestCategories.Live)]
public async Task RequiresExternalServices() { }

CI Lane Filtering:

# Run only unit tests (fast, no dependencies)
dotnet test --filter "Category=Unit"

# Run all tests except Live
dotnet test --filter "Category!=Live"

# Run Integration + Contract tests
dotnet test --filter "Category=Integration|Category=Contract"

Available Categories:

Unit - Fast, in-memory, no external dependencies
Property - FsCheck/generative testing
Snapshot - Golden master regression
Integration - Testcontainers (PostgreSQL, Valkey)
Contract - API/WebService contract tests
Security - Cryptographic validation
Performance - Benchmarking, load tests
Live - Requires external services (disabled in CI by default)
PostIncident - Tests derived from production incidents (Turn #6)
EvidenceChain - Requirement traceability tests (Turn #6)
Longevity - Time-extended stability tests (Turn #6)
Interop - Cross-version compatibility tests (Turn #6)

12. Post-Incident Testing (Turn #6)

Generate regression tests from production incidents:

Generate Test Scaffold from Incident:

using StellaOps.TestKit.Incident;

// Create incident metadata
var metadata = new IncidentMetadata
{
    IncidentId = "INC-2026-001",
    OccurredAt = DateTimeOffset.Parse("2026-01-15T10:30:00Z"),
    RootCause = "Race condition in concurrent bundle creation",
    AffectedModules = ["EvidenceLocker", "Policy"],
    Severity = IncidentSeverity.P1,
    Title = "Evidence bundle duplication"
};

// Generate test scaffold from replay manifest
var generator = new IncidentTestGenerator();
var scaffold = generator.GenerateFromManifestJson(manifestJson, metadata);

// Output generated test code
var code = scaffold.GenerateTestCode();
File.WriteAllText($"Tests/{scaffold.TestClassName}.cs", code);

Generated Test Structure:

[Trait("Category", TestCategories.PostIncident)]
[Trait("Incident", "INC-2026-001")]
[Trait("Severity", "P1")]
public sealed class Incident_INC_2026_001_Tests
{
    [Fact]
    public async Task Validates_RaceCondition_Fix()
    {
        // Arrange - fixtures from replay manifest
        // Act - execute the incident scenario
        // Assert - verify fix prevents recurrence
    }
}

Filter Post-Incident Tests:

# Run all post-incident tests
dotnet test --filter "Category=PostIncident"

# Run only P1/P2 tests (release-gating)
dotnet test --filter "Category=PostIncident&(Severity=P1|Severity=P2)"

# Run tests for a specific incident
dotnet test --filter "Incident=INC-2026-001"

API Reference:

IncidentMetadata - Incident context (ID, severity, root cause, modules)
IncidentSeverity - P1 (critical) through P4 (low impact)
IncidentTestGenerator.GenerateFromManifestJson(json, metadata) - Generate scaffold
TestScaffold.GenerateTestCode() - Output C# test code
TestScaffold.ToJson() / FromJson() - Serialize/deserialize scaffold
IncidentTestGenerator.GenerateReport() - Summary of registered incident tests

See Post-Incident Testing Guide for complete documentation.

Best Practices

1. Always Use TestCategories

Tag every test with the appropriate category:

[Fact, Trait("Category", TestCategories.Unit)]
public void MyUnitTest() { }

This enables CI lane filtering and improves test discoverability.

2. Prefer Deterministic Primitives

Avoid DateTime.UtcNow, Guid.NewGuid(), Random in tests. Use TestKit alternatives:

// ❌ Flaky test (time-dependent)
var expiration = DateTime.UtcNow.AddHours(1);

// ✅ Deterministic test
using var time = new DeterministicTime(DateTime.UtcNow);
var expiration = time.UtcNow.AddHours(1);

3. Use Snapshot Tests for Complex Outputs

For large JSON outputs (SBOM, VEX, attestations), snapshot testing is more maintainable than manual assertions:

// ❌ Brittle manual assertions
Assert.Equal("SPDX-3.0.1", sbom.SpdxVersion);
Assert.Equal(42, sbom.Packages.Count);
// ...hundreds of assertions...

// ✅ Snapshot testing
SnapshotAssert.MatchesSnapshot(sbom, "MySbomSnapshot");

4. Isolate Integration Tests

Use TestCategories to separate fast unit tests from slow integration tests:

[Fact, Trait("Category", TestCategories.Unit)]
public void FastTest() { /* no external dependencies */ }

[Fact, Trait("Category", TestCategories.Integration)]
public async Task SlowTest() { /* uses PostgresFixture */ }

In CI, run Unit tests first for fast feedback, then Integration tests in parallel.

5. Document Snapshot Baselines

When updating snapshots (UPDATE_SNAPSHOTS=1), add a commit message explaining why:

git commit -m "Update SBOM snapshot: added new package metadata fields"

This helps reviewers understand intentional vs. accidental changes.

Troubleshooting

Snapshot Mismatch

Error: Snapshot 'MySbomSnapshot' does not match expected.

Solution:

Review diff manually (check Snapshots/MySbomSnapshot.json)
If change is intentional: UPDATE_SNAPSHOTS=1 dotnet test
Commit updated snapshot with explanation

Testcontainers Failure

Error: Docker daemon not running

Solution:

Ensure Docker Desktop is running
Verify docker ps works in terminal
Check Testcontainers logs: TESTCONTAINERS_DEBUG=1 dotnet test

Determinism Failure

Error: CanonicalJsonAssert.IsDeterministic failed: byte arrays differ

Root Cause: Non-deterministic data in serialization (e.g., random GUIDs, timestamps)

Solution:

Use DeterministicTime and DeterministicRandom
Ensure all data is seeded or mocked
Check for DateTime.UtcNow or Guid.NewGuid() calls

Migration Guide (Existing Tests)

Step 1: Add TestKit Reference

<ProjectReference Include="../../../__Libraries/StellaOps.TestKit/StellaOps.TestKit.csproj" />

Step 2: Replace Time-Dependent Code

Before:

var now = DateTime.UtcNow;

After:

using var time = new DeterministicTime(DateTime.UtcNow);
var now = time.UtcNow;

Step 3: Add Test Categories

[Fact] // Old
[Fact, Trait("Category", TestCategories.Unit)] // New

Step 4: Adopt Snapshot Testing (Optional)

For complex JSON assertions, replace manual checks with snapshots:

// Old
Assert.Equal(expected.SpdxVersion, actual.SpdxVersion);
// ...

// New
SnapshotAssert.MatchesSnapshot(actual, "TestName");

CI Integration

Example `.gitea/workflows/test.yml`

name: Test Suite
on: [push, pull_request]

jobs:
  unit:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-dotnet@v4
        with:
          dotnet-version: '10.0.x'
      - name: Unit Tests (Fast)
        run: dotnet test --filter "Category=Unit" --logger "trx;LogFileName=unit-results.trx"
      - name: Upload Results
        uses: actions/upload-artifact@v4
        with:
          name: unit-test-results
          path: '**/unit-results.trx'

  integration:
    runs-on: ubuntu-latest
    services:
      docker:
        image: docker:dind
    steps:
      - uses: actions/checkout@v4
      - name: Integration Tests
        run: dotnet test --filter "Category=Integration" --logger "trx;LogFileName=integration-results.trx"

Support and Feedback

Issues: Report bugs in sprint tracking files under docs/implplan/
Questions: Contact Platform Guild
Documentation: src/__Libraries/StellaOps.TestKit/README.md

Changelog

v1.0 (2025-12-23)

Initial release: DeterministicTime, DeterministicRandom
CanonicalJsonAssert, SnapshotAssert
PostgresFixture, ValkeyFixture, HttpFixtureServer
OtelCapture for OpenTelemetry traces
TestCategories for CI lane filtering
Pilot adoption in Scanner.Core.Tests

24 KiB Raw Permalink Blame History

StellaOps.TestKit Usage Guide

Overview

Key Features

Installation

Quick Start Examples

1. Deterministic Time

2. Deterministic Random

3. Canonical JSON Assertions

4. Snapshot Testing

5. PostgreSQL Fixture

6. Valkey Fixture

7. HTTP Fixture Server

8. OpenTelemetry Capture

9. Observability Contract Testing (Turn #6)

10. Evidence Chain Traceability (Turn #6)

11. Test Categories

12. Post-Incident Testing (Turn #6)

Best Practices

1. Always Use TestCategories

2. Prefer Deterministic Primitives

3. Use Snapshot Tests for Complex Outputs

4. Isolate Integration Tests

5. Document Snapshot Baselines

Troubleshooting

Snapshot Mismatch

Testcontainers Failure

Determinism Failure

Migration Guide (Existing Tests)

Step 1: Add TestKit Reference

Step 2: Replace Time-Dependent Code

Step 3: Add Test Categories

Step 4: Adopt Snapshot Testing (Optional)

CI Integration

Example .gitea/workflows/test.yml

Support and Feedback

Changelog

v1.0 (2025-12-23)

24 KiB

Raw Permalink Blame History

Example `.gitea/workflows/test.yml`