29 lines
3.1 KiB
Markdown
29 lines
3.1 KiB
Markdown
# Replay Infrastructure (Manifest, Determinism Verifier, Verdict Engine, Drift Detection)
|
|
|
|
## Module
|
|
Replay
|
|
|
|
## Status
|
|
IMPLEMENTED
|
|
|
|
## Description
|
|
Full replay infrastructure: DeterminismVerifier re-hydrates exact inputs from manifest and verifies bit-for-bit verdict reproduction. Run manifest model capturing pipeline state (feeds, rules, versions). DeterministicResolver with feed snapshots, bundle export, and web service. Verdict replay with divergence detection and input drift testing.
|
|
|
|
## Implementation Details
|
|
- **Determinism Verifier**: `src/Replay/__Libraries/StellaOps.Replay.Core/DeterminismVerifier.cs` -- re-hydrates exact inputs from a run manifest and verifies bit-for-bit verdict reproduction; reports divergences between original and replayed verdicts.
|
|
- **Input Manifest Resolver**: `src/Replay/__Libraries/StellaOps.Replay.Core/InputManifestResolver.cs` -- resolves and snapshots all inputs (feed versions, policy rules, SBOM state) needed to reproduce a verdict, creating a self-contained input manifest.
|
|
- **Replay Executor**: `src/Replay/__Libraries/StellaOps.Replay.Core/ReplayExecutor.cs` -- orchestrates verdict replay by loading the input manifest, executing the policy evaluation pipeline, and comparing outputs against the original verdict.
|
|
- **Replay Job Queue**: `src/Replay/__Libraries/StellaOps.Replay.Core/ReplayJobQueue.cs` -- background job queue for scheduling and processing replay verification requests.
|
|
- **Policy Simulation Input Lock**: `src/Replay/__Libraries/StellaOps.Replay.Core/PolicySimulationInputLock.cs` -- locks input state during replay to prevent concurrent modifications from affecting determinism verification.
|
|
- **Verdict Replay Endpoints**: `src/Replay/StellaOps.Replay.WebService/VerdictReplayEndpoints.cs` -- REST endpoints for submitting replay requests and querying replay results.
|
|
- **Web Service Entry Point**: `src/Replay/StellaOps.Replay.WebService/Program.cs` -- ASP.NET Core web service hosting replay endpoints.
|
|
- **Tests**: `src/Replay/__Tests/StellaOps.Replay.Core.Tests/Unit/DeterminismVerifierTests.cs`, `Unit/InputManifestResolverTests.cs`, `VerdictReplayIntegrationTests.cs`, `VerdictReplayEndpointsTests.cs`, `PolicySimulationInputLockValidatorTests.cs`
|
|
|
|
## E2E Test Plan
|
|
- [ ] Submit a verdict for replay via the REST endpoint and verify `DeterminismVerifier` produces a bit-for-bit identical verdict when given the same input manifest
|
|
- [ ] Modify a policy rule between the original verdict and the replay and verify the verifier detects the divergence and reports the specific rule that changed
|
|
- [ ] Verify input manifest completeness: resolve a manifest for a verdict, then replay using only the manifest contents (no external lookups) and confirm successful reproduction
|
|
- [ ] Verify input lock: start a replay and concurrently attempt to modify the locked inputs; confirm the modification is blocked until the replay completes
|
|
- [ ] Submit multiple replay requests to the job queue and verify they execute in order with correct results
|
|
- [ ] Verify drift detection: replay a verdict with a different feed snapshot version and confirm the verifier reports input drift
|