35 lines
3.0 KiB
Markdown
35 lines
3.0 KiB
Markdown
# Deterministic Run Manifest (Replay Key)
|
|
|
|
## Module
|
|
__Tests
|
|
|
|
## Status
|
|
VERIFIED
|
|
|
|
## Description
|
|
Run manifest as a first-class test artifact capturing all inputs (artifact digests, feed snapshots, policy versions, tool versions) needed for byte-identical verdict replay.
|
|
|
|
## Implementation Details
|
|
- **Run Manifest Model**: `src/__Tests/__Libraries/StellaOps.Testing.Manifests/Models/RunManifest.cs` -- data model capturing all inputs needed for replay: artifact digests, feed snapshot versions, policy rule versions, tool versions, and environment metadata.
|
|
- **Manifest Capture Service**: `src/__Tests/__Libraries/StellaOps.Testing.Manifests/Services/ManifestCaptureService.cs` -- captures runtime state during test execution and serializes it into a `RunManifest` for replay.
|
|
- **Run Manifest Serializer**: `src/__Tests/__Libraries/StellaOps.Testing.Manifests/Serialization/RunManifestSerializer.cs` -- canonical serializer for run manifests ensuring deterministic byte output.
|
|
- **Run Manifest Validator**: `src/__Tests/__Libraries/StellaOps.Testing.Manifests/Validation/RunManifestValidator.cs` -- validates manifest completeness and integrity before replay.
|
|
- **Schema Loader**: `src/__Tests/__Libraries/StellaOps.Testing.Manifests/Validation/SchemaLoader.cs` -- loads JSON schema for manifest validation.
|
|
- **Test Run Attestation Generator**: `src/__Tests/__Libraries/StellaOps.Testing.Manifests/Attestation/TestRunAttestationGenerator.cs` -- generates DSSE attestations for test runs, binding the run manifest to a cryptographic signature.
|
|
- **Test Run Evidence**: `src/__Tests/__Libraries/StellaOps.Testing.Manifests/Attestation/TestRunEvidence.cs` -- evidence model for attested test runs.
|
|
- **Test Run Attestation Models**: `src/__Tests/__Libraries/StellaOps.Testing.Manifests/Attestation/TestRunAttestationModels.cs` -- DTOs for test run attestation metadata.
|
|
|
|
## E2E Test Plan
|
|
- [ ] Capture a run manifest during a verdict execution via `ManifestCaptureService` and verify it includes all required fields (artifact digests, feed versions, policy versions, tool versions)
|
|
- [ ] Serialize the manifest via `RunManifestSerializer` and verify the output is deterministic (same manifest produces identical bytes on re-serialization)
|
|
- [ ] Validate the manifest via `RunManifestValidator` and verify it passes schema validation
|
|
- [ ] Use the captured manifest to replay the verdict and verify the replayed output is byte-identical to the original
|
|
- [ ] Generate a test run attestation via `TestRunAttestationGenerator` and verify the DSSE envelope contains the manifest digest and a valid signature
|
|
- [ ] Verify incomplete manifest detection: remove a required field from the manifest and confirm `RunManifestValidator` rejects it with a descriptive error
|
|
|
|
## Verification
|
|
- Verified on 2026-02-13 via `run-001`.
|
|
- Tier 0: Source files confirmed present on disk.
|
|
- Tier 1: `dotnet build` passed (0 errors); 266/266 tests passed across Chaos.Tests, Evidence.Tests, Replay.Tests, FixtureTests.
|
|
- Tier 2d: `docs/qa/feature-checks/runs/tests/deterministic-run-manifest/run-001/tier2-integration-check.json`
|