stella-ops.org/git.stella-ops.org
1
0
Fork 0
Code Issues Pull Requests Actions 2 Releases Wiki Activity
Files
c52ca826528b63f38386a0adddecff0eaacfcf30
git.stella-ops.org/docs/features/checked/tests/deterministic-run-manifest.md
master e9aeadc040 save checkpoint
2026-02-14 09:11:48 +02:00

3.0 KiB
Raw Blame History

Deterministic Run Manifest (Replay Key)

Module

__Tests

Status

VERIFIED

Description

Run manifest as a first-class test artifact capturing all inputs (artifact digests, feed snapshots, policy versions, tool versions) needed for byte-identical verdict replay.

Implementation Details

  • Run Manifest Model: src/__Tests/__Libraries/StellaOps.Testing.Manifests/Models/RunManifest.cs -- data model capturing all inputs needed for replay: artifact digests, feed snapshot versions, policy rule versions, tool versions, and environment metadata.
  • Manifest Capture Service: src/__Tests/__Libraries/StellaOps.Testing.Manifests/Services/ManifestCaptureService.cs -- captures runtime state during test execution and serializes it into a RunManifest for replay.
  • Run Manifest Serializer: src/__Tests/__Libraries/StellaOps.Testing.Manifests/Serialization/RunManifestSerializer.cs -- canonical serializer for run manifests ensuring deterministic byte output.
  • Run Manifest Validator: src/__Tests/__Libraries/StellaOps.Testing.Manifests/Validation/RunManifestValidator.cs -- validates manifest completeness and integrity before replay.
  • Schema Loader: src/__Tests/__Libraries/StellaOps.Testing.Manifests/Validation/SchemaLoader.cs -- loads JSON schema for manifest validation.
  • Test Run Attestation Generator: src/__Tests/__Libraries/StellaOps.Testing.Manifests/Attestation/TestRunAttestationGenerator.cs -- generates DSSE attestations for test runs, binding the run manifest to a cryptographic signature.
  • Test Run Evidence: src/__Tests/__Libraries/StellaOps.Testing.Manifests/Attestation/TestRunEvidence.cs -- evidence model for attested test runs.
  • Test Run Attestation Models: src/__Tests/__Libraries/StellaOps.Testing.Manifests/Attestation/TestRunAttestationModels.cs -- DTOs for test run attestation metadata.

E2E Test Plan

  • Capture a run manifest during a verdict execution via ManifestCaptureService and verify it includes all required fields (artifact digests, feed versions, policy versions, tool versions)
  • Serialize the manifest via RunManifestSerializer and verify the output is deterministic (same manifest produces identical bytes on re-serialization)
  • Validate the manifest via RunManifestValidator and verify it passes schema validation
  • Use the captured manifest to replay the verdict and verify the replayed output is byte-identical to the original
  • Generate a test run attestation via TestRunAttestationGenerator and verify the DSSE envelope contains the manifest digest and a valid signature
  • Verify incomplete manifest detection: remove a required field from the manifest and confirm RunManifestValidator rejects it with a descriptive error

Verification

  • Verified on 2026-02-13 via run-001.
  • Tier 0: Source files confirmed present on disk.
  • Tier 1: dotnet build passed (0 errors); 266/266 tests passed across Chaos.Tests, Evidence.Tests, Replay.Tests, FixtureTests.
  • Tier 2d: docs/qa/feature-checks/runs/tests/deterministic-run-manifest/run-001/tier2-integration-check.json