stella-ops.org/git.stella-ops.org
1
0
Fork 0
Code Issues Pull Requests Actions 2 Releases Wiki Activity
Files
70fdbfcf25908e1c3697ae41b931b8875858d4ea
git.stella-ops.org/docs/features/checked/tests/determinism-property-based-testing.md
master e9aeadc040 save checkpoint
2026-02-14 09:11:48 +02:00

3.5 KiB
Raw Blame History

Determinism Property-Based Testing

Module

__Tests

Status

VERIFIED

Description

Comprehensive determinism property-based tests covering unicode normalization, SBOM/VEX ordering, floating-point stability, digest computation, and canonical JSON to ensure reproducible verdicts.

Implementation Details

  • Unicode Normalization Properties: src/__Tests/__Libraries/StellaOps.Testing.Determinism.Properties/UnicodeNormalizationDeterminismProperties.cs -- FsCheck property-based tests verifying Unicode normalization produces identical output for equivalent Unicode representations.
  • SBOM/VEX Ordering Properties: src/__Tests/__Libraries/StellaOps.Testing.Determinism.Properties/SbomVexOrderingDeterminismProperties.cs -- verifies SBOM and VEX document processing produces identical results regardless of element ordering.
  • Floating-Point Stability Properties: src/__Tests/__Libraries/StellaOps.Testing.Determinism.Properties/FloatingPointStabilityProperties.cs -- verifies floating-point computations (scores, percentages) produce identical results across platforms and evaluation orders.
  • Digest Computation Properties: src/__Tests/__Libraries/StellaOps.Testing.Determinism.Properties/DigestComputationDeterminismProperties.cs -- verifies SHA-256 digest computations are deterministic for identical inputs.
  • Canonical JSON Properties: src/__Tests/__Libraries/StellaOps.Testing.Determinism.Properties/CanonicalJsonDeterminismProperties.cs -- verifies RFC 8785 canonical JSON serialization produces identical byte output for semantically equivalent JSON documents.
  • JSON Object Arbitraries: src/__Tests/__Libraries/StellaOps.Testing.Determinism.Properties/JsonObjectArbitraries.cs -- FsCheck arbitrary generators for producing random JSON structures for property-based testing.
  • Determinism Gate: src/__Tests/__Libraries/StellaOps.Testing.Determinism/Determinism/DeterminismGate.cs -- CI gate that fails the build if determinism properties are violated.
  • Determinism Manifest: src/__Tests/__Libraries/StellaOps.Testing.Determinism/Determinism/DeterminismManifest.cs -- captures determinism verification results as a test artifact.
  • Determinism Baseline Store: src/__Tests/__Libraries/StellaOps.Testing.Determinism/Determinism/DeterminismBaselineStore.cs -- stores determinism baselines for comparison across test runs.

E2E Test Plan

  • Run the Unicode normalization property tests with 1000 randomly generated Unicode strings and verify all produce identical normalized output
  • Run the SBOM ordering property tests with randomly shuffled SBOM components and verify the digest is identical regardless of input order
  • Run the floating-point stability properties and verify score computations produce identical results when operands are reordered
  • Run the canonical JSON properties with randomly generated JSON objects and verify RFC 8785 canonicalization produces identical output for equivalent inputs
  • Verify the determinism gate: introduce a non-deterministic computation and confirm the gate blocks the build
  • Verify determinism manifest: run the full property suite and confirm the manifest captures all property results with pass/fail status

Verification

  • Verified on 2026-02-13 via run-001.
  • Tier 0: Source files confirmed present on disk.
  • Tier 1: dotnet build passed (0 errors); 266/266 tests passed across Chaos.Tests, Evidence.Tests, Replay.Tests, FixtureTests.
  • Tier 2d: docs/qa/feature-checks/runs/tests/determinism-property-based-testing/run-001/tier2-integration-check.json