Add tests for SBOM generation determinism across multiple formats

- Created `StellaOps.TestKit.Tests` project for unit tests related to determinism.
- Implemented `DeterminismManifestTests` to validate deterministic output for canonical bytes and strings, file read/write operations, and error handling for invalid schema versions.
- Added `SbomDeterminismTests` to ensure identical inputs produce consistent SBOMs across SPDX 3.0.1 and CycloneDX 1.6/1.7 formats, including parallel execution tests.
- Updated project references in `StellaOps.Integration.Determinism` to include the new determinism testing library.
This commit is contained in:
master
2025-12-23 18:56:12 +02:00
committed by StellaOps Bot
parent 7ac70ece71
commit 491e883653
409 changed files with 23797 additions and 17779 deletions

View File

@@ -1,174 +1,28 @@
# StellaOps.TestKit
Test infrastructure and fixtures for StellaOps projects. Provides deterministic time/random, canonical JSON assertions, snapshot testing, database fixtures, and OpenTelemetry capture.
Testing infrastructure for StellaOps - deterministic helpers, fixtures, and assertions.
## Features
## Quick Start
### Deterministic Time
```csharp
using StellaOps.TestKit.Time;
// Create a clock at a fixed time
var clock = new DeterministicClock();
var now = clock.UtcNow; // 2025-01-01T00:00:00Z
// Advance time
clock.Advance(TimeSpan.FromMinutes(5));
// Or use helpers
var clock2 = DeterministicClockExtensions.AtTestEpoch();
var clock3 = DeterministicClockExtensions.At("2025-06-15T10:30:00Z");
```
### Deterministic Random
```csharp
using StellaOps.TestKit.Random;
// Create deterministic RNG with standard test seed (42)
var rng = DeterministicRandomExtensions.WithTestSeed();
// Generate reproducible values
var number = rng.Next(1, 100);
var text = rng.NextString(10);
var item = rng.PickOne(new[] { "a", "b", "c" });
```
### Canonical JSON Assertions
```csharp
using StellaOps.TestKit.Json;
// Assert JSON equality (ignores formatting)
CanonicalJsonAssert.Equal(expectedJson, actualJson);
// Assert object equivalence
CanonicalJsonAssert.EquivalentObjects(expectedObj, actualObj);
// Hash-based regression testing
var hash = CanonicalJsonAssert.ComputeHash(json);
CanonicalJsonAssert.MatchesHash("abc123...", json);
using var time = new DeterministicTime(new DateTime(2026, 1, 15, 10, 30, 0, DateTimeKind.Utc));
var timestamp = time.UtcNow; // Always 2026-01-15T10:30:00Z
```
### Snapshot Testing
```csharp
using StellaOps.TestKit.Snapshots;
public class MyTests
{
[Fact]
public void TestOutput()
{
var output = GenerateSomeOutput();
// Compare against __snapshots__/test_output.txt
var snapshotPath = SnapshotHelper.GetSnapshotPath("test_output");
SnapshotHelper.VerifySnapshot(output, snapshotPath);
}
[Fact]
public void TestJsonOutput()
{
var obj = new { Name = "test", Value = 42 };
// Compare JSON serialization
var snapshotPath = SnapshotHelper.GetSnapshotPath("test_json", ".json");
SnapshotHelper.VerifyJsonSnapshot(obj, snapshotPath);
}
}
// Update snapshots: set environment variable UPDATE_SNAPSHOTS=1
SnapshotAssert.MatchesSnapshot(sbom, "TestSbom");
// Update: UPDATE_SNAPSHOTS=1 dotnet test
```
### PostgreSQL Fixture
### PostgreSQL Integration
```csharp
using StellaOps.TestKit.Fixtures;
using Xunit;
[Collection("Postgres")]
public class DatabaseTests
public class Tests : IClassFixture<PostgresFixture>
{
private readonly PostgresFixture _postgres;
public DatabaseTests(PostgresFixture postgres)
{
_postgres = postgres;
}
[Fact]
public async Task TestQuery()
{
// Use connection string
await using var conn = new Npgsql.NpgsqlConnection(_postgres.ConnectionString);
await conn.OpenAsync();
// Execute SQL
await _postgres.ExecuteSqlAsync("CREATE TABLE test (id INT)");
// Create additional databases
await _postgres.CreateDatabaseAsync("otherdb");
}
public async Task TestDb() { /* use _fixture.ConnectionString */ }
}
```
### Valkey/Redis Fixture
```csharp
using StellaOps.TestKit.Fixtures;
using Xunit;
[Collection("Valkey")]
public class CacheTests
{
private readonly ValkeyFixture _valkey;
public CacheTests(ValkeyFixture valkey)
{
_valkey = valkey;
}
[Fact]
public void TestCache()
{
var connectionString = _valkey.ConnectionString;
// Use with your Redis/Valkey client
}
}
```
### OpenTelemetry Capture
```csharp
using StellaOps.TestKit.Telemetry;
[Fact]
public void TestTracing()
{
using var otel = new OTelCapture("my-service");
// Code that emits traces
using (var activity = otel.ActivitySource.StartActivity("operation"))
{
activity?.SetTag("key", "value");
}
// Assert traces
otel.AssertActivityExists("operation");
otel.AssertActivityHasTag("operation", "key", "value");
// Get summary for debugging
Console.WriteLine(otel.GetTraceSummary());
}
```
## Usage in Tests
Add to your test project:
```xml
<ItemGroup>
<ProjectReference Include="..\..\__Libraries\StellaOps.TestKit\StellaOps.TestKit.csproj" />
</ItemGroup>
```
## Design Principles
- **Determinism**: All utilities produce reproducible results
- **Offline-first**: No network dependencies (uses Testcontainers for local infrastructure)
- **Minimal dependencies**: Only essential packages
- **xUnit-friendly**: Works seamlessly with xUnit fixtures and collections
See full documentation in this README.