stella-ops.org/git.stella-ops.org
1
0
Fork 0
Code Issues Pull Requests Actions 2 Releases Wiki Activity
Files
4bdc298ec1fc821ee0d9dda6d2d4643dcedf6b35
git.stella-ops.org/docs/features/unchecked/tests/deterministic-run-manifest.md

2.7 KiB
Raw Blame History

Deterministic Run Manifest (Replay Key)

Module

__Tests

Status

IMPLEMENTED

Description

Run manifest as a first-class test artifact capturing all inputs (artifact digests, feed snapshots, policy versions, tool versions) needed for byte-identical verdict replay.

Implementation Details

  • Run Manifest Model: src/__Tests/__Libraries/StellaOps.Testing.Manifests/Models/RunManifest.cs -- data model capturing all inputs needed for replay: artifact digests, feed snapshot versions, policy rule versions, tool versions, and environment metadata.
  • Manifest Capture Service: src/__Tests/__Libraries/StellaOps.Testing.Manifests/Services/ManifestCaptureService.cs -- captures runtime state during test execution and serializes it into a RunManifest for replay.
  • Run Manifest Serializer: src/__Tests/__Libraries/StellaOps.Testing.Manifests/Serialization/RunManifestSerializer.cs -- canonical serializer for run manifests ensuring deterministic byte output.
  • Run Manifest Validator: src/__Tests/__Libraries/StellaOps.Testing.Manifests/Validation/RunManifestValidator.cs -- validates manifest completeness and integrity before replay.
  • Schema Loader: src/__Tests/__Libraries/StellaOps.Testing.Manifests/Validation/SchemaLoader.cs -- loads JSON schema for manifest validation.
  • Test Run Attestation Generator: src/__Tests/__Libraries/StellaOps.Testing.Manifests/Attestation/TestRunAttestationGenerator.cs -- generates DSSE attestations for test runs, binding the run manifest to a cryptographic signature.
  • Test Run Evidence: src/__Tests/__Libraries/StellaOps.Testing.Manifests/Attestation/TestRunEvidence.cs -- evidence model for attested test runs.
  • Test Run Attestation Models: src/__Tests/__Libraries/StellaOps.Testing.Manifests/Attestation/TestRunAttestationModels.cs -- DTOs for test run attestation metadata.

E2E Test Plan

  • Capture a run manifest during a verdict execution via ManifestCaptureService and verify it includes all required fields (artifact digests, feed versions, policy versions, tool versions)
  • Serialize the manifest via RunManifestSerializer and verify the output is deterministic (same manifest produces identical bytes on re-serialization)
  • Validate the manifest via RunManifestValidator and verify it passes schema validation
  • Use the captured manifest to replay the verdict and verify the replayed output is byte-identical to the original
  • Generate a test run attestation via TestRunAttestationGenerator and verify the DSSE envelope contains the manifest digest and a valid signature
  • Verify incomplete manifest detection: remove a required field from the manifest and confirm RunManifestValidator rejects it with a descriptive error