# StellaOps.TestKit Usage Guide

**Version:** 1.0
**Status:** Pilot Release (Wave 4 Complete)
**Audience:** StellaOps developers writing unit, integration, and contract tests

---

## Overview

`StellaOps.TestKit` provides deterministic testing infrastructure for StellaOps modules. It eliminates flaky tests, provides reproducible test primitives, and standardizes fixtures for integration testing.

### Key Features

- **Deterministic Time**: Freeze and advance time for reproducible tests
- **Deterministic Random**: Seeded random number generation
- **Canonical JSON Assertions**: SHA-256 hash verification for determinism
- **Snapshot Testing**: Golden master regression testing
- **PostgreSQL Fixture**: Testcontainers-based PostgreSQL 16 for integration tests
- **Valkey Fixture**: Redis-compatible caching tests
- **HTTP Fixture**: In-memory API contract testing
- **OpenTelemetry Capture**: Trace and span assertion helpers
- **Test Categories**: Standardized trait constants for CI filtering

---

## Installation

Add `StellaOps.TestKit` as a project reference to your test project:

```xml
<ItemGroup>
  <ProjectReference Include="../../../__Libraries/StellaOps.TestKit/StellaOps.TestKit.csproj" />
</ItemGroup>
```

---

## Quick Start Examples

### 1. Deterministic Time

Eliminate flaky tests caused by time-dependent logic:

```csharp
using StellaOps.TestKit.Deterministic;
using Xunit;

[Fact]
public void Test_ExpirationLogic()
{
    // Arrange: Fix time at a known UTC timestamp
    using var time = new DeterministicTime(new DateTime(2026, 1, 15, 10, 30, 0, DateTimeKind.Utc));

    var expiresAt = time.UtcNow.AddHours(24);

    // Act: Advance time to just before expiration
    time.Advance(TimeSpan.FromHours(23));
    Assert.False(time.UtcNow > expiresAt);

    // Advance past expiration
    time.Advance(TimeSpan.FromHours(2));
    Assert.True(time.UtcNow > expiresAt);
}
```

**API Reference:**
- `DeterministicTime(DateTime initialUtc)` - Create with fixed start time
- `UtcNow` - Get current deterministic time
- `Advance(TimeSpan duration)` - Move time forward
- `SetTo(DateTime newUtc)` - Jump to specific time

---

### 2. Deterministic Random

Reproducible random sequences for property tests and fuzzing:

```csharp
using StellaOps.TestKit.Deterministic;

[Fact]
public void Test_RandomIdGeneration()
{
    // Arrange: Same seed produces same sequence
    var random1 = new DeterministicRandom(seed: 42);
    var random2 = new DeterministicRandom(seed: 42);

    // Act
    var guid1 = random1.NextGuid();
    var guid2 = random2.NextGuid();

    // Assert: Reproducible GUIDs
    Assert.Equal(guid1, guid2);
}

[Fact]
public void Test_Shuffling()
{
    var random = new DeterministicRandom(seed: 100);
    var array = new[] { 1, 2, 3, 4, 5 };

    random.Shuffle(array);

    // Deterministic shuffle order
    Assert.NotEqual(new[] { 1, 2, 3, 4, 5 }, array);
}
```

**API Reference:**
- `DeterministicRandom(int seed)` - Create with seed
- `NextGuid()` - Generate deterministic GUID
- `NextString(int length)` - Generate alphanumeric string
- `NextInt(int min, int max)` - Generate integer in range
- `Shuffle<T>(T[] array)` - Fisher-Yates shuffle

---

### 3. Canonical JSON Assertions

Verify JSON determinism for SBOM, VEX, and attestation outputs:

```csharp
using StellaOps.TestKit.Assertions;

[Fact]
public void Test_SbomDeterminism()
{
    var sbom = new
    {
        SpdxVersion = "SPDX-3.0.1",
        Name = "MySbom",
        Packages = new[] { new { Name = "Pkg1", Version = "1.0" } }
    };

    // Verify deterministic serialization
    CanonicalJsonAssert.IsDeterministic(sbom, iterations: 100);

    // Verify expected hash (golden master)
    var expectedHash = "abc123..."; // Precomputed SHA-256
    CanonicalJsonAssert.HasExpectedHash(sbom, expectedHash);
}

[Fact]
public void Test_JsonPropertyExists()
{
    var vex = new
    {
        Document = new { Id = "VEX-2026-001" },
        Statements = new[] { new { Vulnerability = "CVE-2026-1234" } }
    };

    // Deep property verification
    CanonicalJsonAssert.ContainsProperty(vex, "Document.Id", "VEX-2026-001");
    CanonicalJsonAssert.ContainsProperty(vex, "Statements[0].Vulnerability", "CVE-2026-1234");
}
```

**API Reference:**
- `IsDeterministic<T>(T value, int iterations)` - Verify N serializations match
- `HasExpectedHash<T>(T value, string expectedSha256Hex)` - Verify SHA-256 hash
- `ComputeCanonicalHash<T>(T value)` - Compute hash for golden master
- `AreCanonicallyEqual<T>(T expected, T actual)` - Compare canonical JSON
- `ContainsProperty<T>(T value, string propertyPath, object expectedValue)` - Deep search

---

### 4. Snapshot Testing

Golden master regression testing for complex outputs:

```csharp
using StellaOps.TestKit.Assertions;

[Fact, Trait("Category", TestCategories.Snapshot)]
public void Test_SbomGeneration()
{
    var sbom = GenerateSbom(); // Your SBOM generation logic

    // Snapshot will be stored in Snapshots/TestSbomGeneration.json
    SnapshotAssert.MatchesSnapshot(sbom, "TestSbomGeneration");
}

// Update snapshots when intentional changes occur:
// UPDATE_SNAPSHOTS=1 dotnet test
```

**Text and Binary Snapshots:**

```csharp
[Fact]
public void Test_LicenseText()
{
    var licenseText = GenerateLicenseNotice();
    SnapshotAssert.MatchesTextSnapshot(licenseText, "LicenseNotice");
}

[Fact]
public void Test_SignatureBytes()
{
    var signature = SignDocument(document);
    SnapshotAssert.MatchesBinarySnapshot(signature, "DocumentSignature");
}
```

**API Reference:**
- `MatchesSnapshot<T>(T value, string snapshotName)` - JSON snapshot
- `MatchesTextSnapshot(string value, string snapshotName)` - Text snapshot
- `MatchesBinarySnapshot(byte[] value, string snapshotName)` - Binary snapshot
- Environment variable: `UPDATE_SNAPSHOTS=1` to update baselines

---

### 5. PostgreSQL Fixture

Testcontainers-based PostgreSQL 16 for integration tests:

```csharp
using StellaOps.TestKit.Fixtures;
using Xunit;

public class DatabaseTests : IClassFixture<PostgresFixture>
{
    private readonly PostgresFixture _fixture;

    public DatabaseTests(PostgresFixture fixture)
    {
        _fixture = fixture;
    }

    [Fact, Trait("Category", TestCategories.Integration)]
    public async Task Test_DatabaseOperations()
    {
        // Use _fixture.ConnectionString to connect
        using var connection = new NpgsqlConnection(_fixture.ConnectionString);
        await connection.OpenAsync();

        // Run migrations
        await _fixture.RunMigrationsAsync(connection);

        // Test database operations
        var result = await connection.QueryAsync("SELECT version()");
        Assert.NotEmpty(result);
    }
}
```

**API Reference:**
- `PostgresFixture` - xUnit class fixture
- `ConnectionString` - PostgreSQL connection string
- `RunMigrationsAsync(DbConnection)` - Apply migrations
- Requires Docker running locally

---

### 6. Valkey Fixture

Redis-compatible caching for integration tests:

```csharp
using StellaOps.TestKit.Fixtures;

public class CacheTests : IClassFixture<ValkeyFixture>
{
    private readonly ValkeyFixture _fixture;

    [Fact, Trait("Category", TestCategories.Integration)]
    public async Task Test_CachingLogic()
    {
        var connection = await ConnectionMultiplexer.Connect(_fixture.ConnectionString);
        var db = connection.GetDatabase();

        await db.StringSetAsync("key", "value");
        var result = await db.StringGetAsync("key");

        Assert.Equal("value", result.ToString());
    }
}
```

**API Reference:**
- `ValkeyFixture` - xUnit class fixture
- `ConnectionString` - Redis connection string (host:port)
- `Host`, `Port` - Connection details
- Uses `redis:7-alpine` image (Valkey-compatible)

---

### 7. HTTP Fixture Server

In-memory API contract testing:

```csharp
using StellaOps.TestKit.Fixtures;

public class ApiTests : IClassFixture<HttpFixtureServer<Program>>
{
    private readonly HttpClient _client;

    public ApiTests(HttpFixtureServer<Program> fixture)
    {
        _client = fixture.CreateClient();
    }

    [Fact, Trait("Category", TestCategories.Contract)]
    public async Task Test_HealthEndpoint()
    {
        var response = await _client.GetAsync("/health");
        response.EnsureSuccessStatusCode();

        var body = await response.Content.ReadAsStringAsync();
        Assert.Contains("healthy", body);
    }
}
```

**HTTP Message Handler Stub (Hermetic Tests):**

```csharp
[Fact]
public async Task Test_ExternalApiCall()
{
    var handler = new HttpMessageHandlerStub()
        .WhenRequest("https://api.example.com/data", HttpStatusCode.OK, "{\"status\":\"ok\"}");

    var httpClient = new HttpClient(handler);
    var response = await httpClient.GetAsync("https://api.example.com/data");

    Assert.Equal(HttpStatusCode.OK, response.StatusCode);
}
```

**API Reference:**
- `HttpFixtureServer<TProgram>` - WebApplicationFactory wrapper
- `CreateClient()` - Get HttpClient for test server
- `HttpMessageHandlerStub` - Stub external HTTP dependencies
- `WhenRequest(url, statusCode, content)` - Configure stub responses

---

### 8. OpenTelemetry Capture

Trace and span assertion helpers:

```csharp
using StellaOps.TestKit.Observability;

[Fact]
public async Task Test_TracingBehavior()
{
    using var capture = new OtelCapture();

    // Execute code that emits traces
    await MyService.DoWorkAsync();

    // Assert traces
    capture.AssertHasSpan("MyService.DoWork");
    capture.AssertHasTag("user_id", "123");
    capture.AssertSpanCount(expectedCount: 3);

    // Verify parent-child hierarchy
    capture.AssertHierarchy("ParentSpan", "ChildSpan");
}
```

**API Reference:**
- `OtelCapture(string? activitySourceName = null)` - Create capture
- `AssertHasSpan(string spanName)` - Verify span exists
- `AssertHasTag(string tagKey, string expectedValue)` - Verify tag
- `AssertSpanCount(int expectedCount)` - Verify span count
- `AssertHierarchy(string parentSpanName, string childSpanName)` - Verify parent-child
- `CapturedActivities` - Get all captured spans

---

### 9. Test Categories

Standardized trait constants for CI lane filtering:

```csharp
using StellaOps.TestKit;

[Fact, Trait("Category", TestCategories.Unit)]
public void FastUnitTest() { }

[Fact, Trait("Category", TestCategories.Integration)]
public async Task SlowIntegrationTest() { }

[Fact, Trait("Category", TestCategories.Live)]
public async Task RequiresExternalServices() { }
```

**CI Lane Filtering:**

```bash
# Run only unit tests (fast, no dependencies)
dotnet test --filter "Category=Unit"

# Run all tests except Live
dotnet test --filter "Category!=Live"

# Run Integration + Contract tests
dotnet test --filter "Category=Integration|Category=Contract"
```

**Available Categories:**
- `Unit` - Fast, in-memory, no external dependencies
- `Property` - FsCheck/generative testing
- `Snapshot` - Golden master regression
- `Integration` - Testcontainers (PostgreSQL, Valkey)
- `Contract` - API/WebService contract tests
- `Security` - Cryptographic validation
- `Performance` - Benchmarking, load tests
- `Live` - Requires external services (disabled in CI by default)

---

## Best Practices

### 1. Always Use TestCategories

Tag every test with the appropriate category:

```csharp
[Fact, Trait("Category", TestCategories.Unit)]
public void MyUnitTest() { }
```

This enables CI lane filtering and improves test discoverability.

### 2. Prefer Deterministic Primitives

Avoid `DateTime.UtcNow`, `Guid.NewGuid()`, `Random` in tests. Use TestKit alternatives:

```csharp
// ❌ Flaky test (time-dependent)
var expiration = DateTime.UtcNow.AddHours(1);

// ✅ Deterministic test
using var time = new DeterministicTime(DateTime.UtcNow);
var expiration = time.UtcNow.AddHours(1);
```

### 3. Use Snapshot Tests for Complex Outputs

For large JSON outputs (SBOM, VEX, attestations), snapshot testing is more maintainable than manual assertions:

```csharp
// ❌ Brittle manual assertions
Assert.Equal("SPDX-3.0.1", sbom.SpdxVersion);
Assert.Equal(42, sbom.Packages.Count);
// ...hundreds of assertions...

// ✅ Snapshot testing
SnapshotAssert.MatchesSnapshot(sbom, "MySbomSnapshot");
```

### 4. Isolate Integration Tests

Use TestCategories to separate fast unit tests from slow integration tests:

```csharp
[Fact, Trait("Category", TestCategories.Unit)]
public void FastTest() { /* no external dependencies */ }

[Fact, Trait("Category", TestCategories.Integration)]
public async Task SlowTest() { /* uses PostgresFixture */ }
```

In CI, run Unit tests first for fast feedback, then Integration tests in parallel.

### 5. Document Snapshot Baselines

When updating snapshots (`UPDATE_SNAPSHOTS=1`), add a commit message explaining why:

```bash
git commit -m "Update SBOM snapshot: added new package metadata fields"
```

This helps reviewers understand intentional vs. accidental changes.

---

## Troubleshooting

### Snapshot Mismatch

**Error:** `Snapshot 'MySbomSnapshot' does not match expected.`

**Solution:**
1. Review diff manually (check `Snapshots/MySbomSnapshot.json`)
2. If change is intentional: `UPDATE_SNAPSHOTS=1 dotnet test`
3. Commit updated snapshot with explanation

### Testcontainers Failure

**Error:** `Docker daemon not running`

**Solution:**
- Ensure Docker Desktop is running
- Verify `docker ps` works in terminal
- Check Testcontainers logs: `TESTCONTAINERS_DEBUG=1 dotnet test`

### Determinism Failure

**Error:** `CanonicalJsonAssert.IsDeterministic failed: byte arrays differ`

**Root Cause:** Non-deterministic data in serialization (e.g., random GUIDs, timestamps)

**Solution:**
- Use `DeterministicTime` and `DeterministicRandom`
- Ensure all data is seeded or mocked
- Check for `DateTime.UtcNow` or `Guid.NewGuid()` calls

---

## Migration Guide (Existing Tests)

### Step 1: Add TestKit Reference

```xml
<ProjectReference Include="../../../__Libraries/StellaOps.TestKit/StellaOps.TestKit.csproj" />
```

### Step 2: Replace Time-Dependent Code

**Before:**
```csharp
var now = DateTime.UtcNow;
```

**After:**
```csharp
using var time = new DeterministicTime(DateTime.UtcNow);
var now = time.UtcNow;
```

### Step 3: Add Test Categories

```csharp
[Fact] // Old
[Fact, Trait("Category", TestCategories.Unit)] // New
```

### Step 4: Adopt Snapshot Testing (Optional)

For complex JSON assertions, replace manual checks with snapshots:

```csharp
// Old
Assert.Equal(expected.SpdxVersion, actual.SpdxVersion);
// ...

// New
SnapshotAssert.MatchesSnapshot(actual, "TestName");
```

---

## CI Integration

### Example `.gitea/workflows/test.yml`

```yaml
name: Test Suite
on: [push, pull_request]

jobs:
  unit:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-dotnet@v4
        with:
          dotnet-version: '10.0.x'
      - name: Unit Tests (Fast)
        run: dotnet test --filter "Category=Unit" --logger "trx;LogFileName=unit-results.trx"
      - name: Upload Results
        uses: actions/upload-artifact@v4
        with:
          name: unit-test-results
          path: '**/unit-results.trx'

  integration:
    runs-on: ubuntu-latest
    services:
      docker:
        image: docker:dind
    steps:
      - uses: actions/checkout@v4
      - name: Integration Tests
        run: dotnet test --filter "Category=Integration" --logger "trx;LogFileName=integration-results.trx"
```

---

## Support and Feedback

- **Issues:** Report bugs in sprint tracking files under `docs/implplan/`
- **Questions:** Contact Platform Guild
- **Documentation:** `src/__Libraries/StellaOps.TestKit/README.md`

---

## Changelog

### v1.0 (2025-12-23)
- Initial release: DeterministicTime, DeterministicRandom
- CanonicalJsonAssert, SnapshotAssert
- PostgresFixture, ValkeyFixture, HttpFixtureServer
- OtelCapture for OpenTelemetry traces
- TestCategories for CI lane filtering
- Pilot adoption in Scanner.Core.Tests