30 lines
3.2 KiB
Markdown
30 lines
3.2 KiB
Markdown
# Determinism Property-Based Testing
|
|
|
|
## Module
|
|
__Tests
|
|
|
|
## Status
|
|
IMPLEMENTED
|
|
|
|
## Description
|
|
Comprehensive determinism property-based tests covering unicode normalization, SBOM/VEX ordering, floating-point stability, digest computation, and canonical JSON to ensure reproducible verdicts.
|
|
|
|
## Implementation Details
|
|
- **Unicode Normalization Properties**: `src/__Tests/__Libraries/StellaOps.Testing.Determinism.Properties/UnicodeNormalizationDeterminismProperties.cs` -- FsCheck property-based tests verifying Unicode normalization produces identical output for equivalent Unicode representations.
|
|
- **SBOM/VEX Ordering Properties**: `src/__Tests/__Libraries/StellaOps.Testing.Determinism.Properties/SbomVexOrderingDeterminismProperties.cs` -- verifies SBOM and VEX document processing produces identical results regardless of element ordering.
|
|
- **Floating-Point Stability Properties**: `src/__Tests/__Libraries/StellaOps.Testing.Determinism.Properties/FloatingPointStabilityProperties.cs` -- verifies floating-point computations (scores, percentages) produce identical results across platforms and evaluation orders.
|
|
- **Digest Computation Properties**: `src/__Tests/__Libraries/StellaOps.Testing.Determinism.Properties/DigestComputationDeterminismProperties.cs` -- verifies SHA-256 digest computations are deterministic for identical inputs.
|
|
- **Canonical JSON Properties**: `src/__Tests/__Libraries/StellaOps.Testing.Determinism.Properties/CanonicalJsonDeterminismProperties.cs` -- verifies RFC 8785 canonical JSON serialization produces identical byte output for semantically equivalent JSON documents.
|
|
- **JSON Object Arbitraries**: `src/__Tests/__Libraries/StellaOps.Testing.Determinism.Properties/JsonObjectArbitraries.cs` -- FsCheck arbitrary generators for producing random JSON structures for property-based testing.
|
|
- **Determinism Gate**: `src/__Tests/__Libraries/StellaOps.Testing.Determinism/Determinism/DeterminismGate.cs` -- CI gate that fails the build if determinism properties are violated.
|
|
- **Determinism Manifest**: `src/__Tests/__Libraries/StellaOps.Testing.Determinism/Determinism/DeterminismManifest.cs` -- captures determinism verification results as a test artifact.
|
|
- **Determinism Baseline Store**: `src/__Tests/__Libraries/StellaOps.Testing.Determinism/Determinism/DeterminismBaselineStore.cs` -- stores determinism baselines for comparison across test runs.
|
|
|
|
## E2E Test Plan
|
|
- [ ] Run the Unicode normalization property tests with 1000 randomly generated Unicode strings and verify all produce identical normalized output
|
|
- [ ] Run the SBOM ordering property tests with randomly shuffled SBOM components and verify the digest is identical regardless of input order
|
|
- [ ] Run the floating-point stability properties and verify score computations produce identical results when operands are reordered
|
|
- [ ] Run the canonical JSON properties with randomly generated JSON objects and verify RFC 8785 canonicalization produces identical output for equivalent inputs
|
|
- [ ] Verify the determinism gate: introduce a non-deterministic computation and confirm the gate blocks the build
|
|
- [ ] Verify determinism manifest: run the full property suite and confirm the manifest captures all property results with pass/fail status
|