Add Canonical JSON serialization library with tests and documentation
- Implemented CanonJson class for deterministic JSON serialization and hashing. - Added unit tests for CanonJson functionality, covering various scenarios including key sorting, handling of nested objects, arrays, and special characters. - Created project files for the Canonical JSON library and its tests, including necessary package references. - Added README.md for library usage and API reference. - Introduced RabbitMqIntegrationFactAttribute for conditional RabbitMQ integration tests.
This commit is contained in:
129
bench/determinism/README.md
Normal file
129
bench/determinism/README.md
Normal file
@@ -0,0 +1,129 @@
|
||||
# Determinism Benchmark Suite
|
||||
|
||||
> **Purpose:** Verify that StellaOps produces bit-identical results across replays.
|
||||
> **Status:** Active
|
||||
> **Sprint:** SPRINT_3850_0001_0001 (Competitive Gap Closure)
|
||||
|
||||
## Overview
|
||||
|
||||
Determinism is a core differentiator for StellaOps:
|
||||
- Same inputs → same outputs (bit-identical)
|
||||
- Replay manifests enable audit verification
|
||||
- No hidden state or environment leakage
|
||||
|
||||
## What Gets Tested
|
||||
|
||||
### Canonical JSON
|
||||
- Object key ordering (alphabetical)
|
||||
- Number formatting consistency
|
||||
- UTF-8 encoding without BOM
|
||||
- No whitespace variation
|
||||
|
||||
### Scan Manifests
|
||||
- Same artifact + same feeds → same manifest hash
|
||||
- Seed values propagate correctly
|
||||
- Timestamp handling (fixed UTC)
|
||||
|
||||
### Proof Bundles
|
||||
- Root hash computation
|
||||
- DSSE envelope determinism
|
||||
- ProofLedger node ordering
|
||||
|
||||
### Score Computation
|
||||
- Same manifest → same score
|
||||
- Lattice merge is associative/commutative
|
||||
- Policy rule ordering doesn't affect outcome
|
||||
|
||||
## Test Cases
|
||||
|
||||
### TC-001: Canonical JSON Determinism
|
||||
|
||||
```bash
|
||||
# Run same object through CanonJson 100 times
|
||||
# All hashes must match
|
||||
```
|
||||
|
||||
### TC-002: Manifest Hash Stability
|
||||
|
||||
```bash
|
||||
# Create manifest with identical inputs
|
||||
# Verify ComputeHash() returns same value
|
||||
```
|
||||
|
||||
### TC-003: Cross-Platform Determinism
|
||||
|
||||
```bash
|
||||
# Run on Linux, Windows, macOS
|
||||
# Compare output hashes
|
||||
```
|
||||
|
||||
### TC-004: Feed Snapshot Determinism
|
||||
|
||||
```bash
|
||||
# Same feed snapshot hash → same scan results
|
||||
```
|
||||
|
||||
## Fixtures
|
||||
|
||||
```
|
||||
fixtures/
|
||||
├── sample-manifest.json
|
||||
├── sample-ledger.json
|
||||
├── expected-hashes.json
|
||||
└── cross-platform/
|
||||
├── linux-x64.hashes.json
|
||||
├── windows-x64.hashes.json
|
||||
└── macos-arm64.hashes.json
|
||||
```
|
||||
|
||||
## Running the Suite
|
||||
|
||||
```bash
|
||||
# Run determinism tests
|
||||
dotnet test tests/StellaOps.Determinism.Tests
|
||||
|
||||
# Run replay verification
|
||||
./run-replay.sh --manifest fixtures/sample-manifest.json --runs 10
|
||||
|
||||
# Cross-platform verification (requires CI matrix)
|
||||
./verify-cross-platform.sh
|
||||
```
|
||||
|
||||
## Metrics
|
||||
|
||||
| Metric | Target | Description |
|
||||
|--------|--------|-------------|
|
||||
| Hash stability | 100% | All runs produce identical hash |
|
||||
| Replay success | 100% | All replays match original |
|
||||
| Cross-platform parity | 100% | Same hash across OS/arch |
|
||||
|
||||
## Integration with CI
|
||||
|
||||
```yaml
|
||||
# .gitea/workflows/bench-determinism.yaml
|
||||
name: Determinism Benchmark
|
||||
on:
|
||||
push:
|
||||
paths:
|
||||
- 'src/__Libraries/StellaOps.Canonical.Json/**'
|
||||
- 'src/Scanner/__Libraries/StellaOps.Scanner.Core/**'
|
||||
- 'bench/determinism/**'
|
||||
|
||||
jobs:
|
||||
determinism:
|
||||
strategy:
|
||||
matrix:
|
||||
os: [ubuntu-latest, windows-latest, macos-latest]
|
||||
runs-on: ${{ matrix.os }}
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- name: Run Determinism Tests
|
||||
run: dotnet test tests/StellaOps.Determinism.Tests
|
||||
- name: Capture Hashes
|
||||
run: ./bench/determinism/capture-hashes.sh
|
||||
- name: Upload Hashes
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: hashes-${{ matrix.os }}
|
||||
path: bench/determinism/results/
|
||||
```
|
||||
Reference in New Issue
Block a user