stella-ops.org/git.stella-ops.org
1
0
Fork 0
Code Issues Pull Requests Actions 3 Releases Wiki Activity

save checkpoint

Browse Source
This commit is contained in:
master
2026-02-14 09:11:48 +02:00
parent 9ca2de05df
commit e9aeadc040
1512 changed files with 30863 additions and 4728 deletions
Download Patch File Download Diff File Expand all files Collapse all files

42
docs/features/checked/policy/proof-replay-deterministic-verdict-replay.md Normal file
View File

@@ -0,0 +1,42 @@
# Proof Replay / Deterministic Verdict Replay
## Module
Policy
## Status
IMPLEMENTED
## Description
Full replay service with a dedicated module, determinism verifier, run manifests, and extensive E2E tests that verify byte-identical verdict replay across runs.
## Implementation Details
- **ReplayEngine**: `src/Policy/__Libraries/StellaOps.Policy/Replay/ReplayEngine.cs` (sealed class implements `IReplayEngine`)
- `ReplayAsync(ReplayRequest)` replays policy evaluation with frozen snapshot inputs
- Pipeline: load snapshot -> resolve frozen inputs -> execute with frozen inputs -> compare with original -> generate delta report
- Uses `ISnapshotService` for snapshot loading, `IKnowledgeSourceResolver` for input resolution, `IVerdictComparer` for comparison
- Returns `ReplayResult` with MatchStatus, ReplayedVerdict, OriginalVerdict, DeltaReport, Duration
- **ReplayRequest**: `src/Policy/__Libraries/StellaOps.Policy/Replay/ReplayRequest.cs` (sealed record)
- ArtifactDigest, SnapshotId, OriginalVerdictId (optional, for comparison)
- ReplayOptions: CompareWithOriginal (default true), AllowNetworkFetch (default false), GenerateDetailedReport (default true), ScoreTolerance (default 0.001)
- **ReplayResult**: `src/Policy/__Libraries/StellaOps.Policy/Replay/ReplayResult.cs` (sealed record)
- ReplayMatchStatus: ExactMatch, MatchWithinTolerance, Mismatch, NoComparison, ReplayFailed
- ReplayedVerdict: VerdictId, ArtifactDigest, Decision (Pass/Fail/PassWithExceptions/Indeterminate), Score, FindingIds, KnowledgeSnapshotId
- ReplayDeltaReport: Summary, FieldDeltas (FieldName, OriginalValue, ReplayedValue), FindingDeltas (FindingId, DeltaType, Description), SuspectedCauses
- **VerdictComparer**: `src/Policy/__Libraries/StellaOps.Policy/Replay/VerdictComparer.cs` -- compares replayed vs original verdicts with tolerance
- **ReplayReport**: `src/Policy/__Libraries/StellaOps.Policy/Replay/ReplayReport.cs` -- detailed replay report generation
- **KnowledgeSourceResolver**: `src/Policy/__Libraries/StellaOps.Policy/Replay/KnowledgeSourceResolver.cs` -- resolves snapshot sources to frozen data
- **KnowledgeSnapshotManifest**: `src/Policy/__Libraries/StellaOps.Policy/Snapshots/KnowledgeSnapshotManifest.cs` -- content-addressed snapshot used as replay input
- **SnapshotAwarePolicyEvaluator**: `src/Policy/__Libraries/StellaOps.Policy/Snapshots/SnapshotAwarePolicyEvaluator.cs` -- evaluates using pinned snapshot inputs
## E2E Test Plan
- [ ] Replay verdict with same snapshot; verify MatchStatus=ExactMatch (deterministic evaluation)
- [ ] Replay verdict with OriginalVerdictId; verify OriginalVerdict is loaded and compared
- [ ] Replay with CompareWithOriginal=false; verify MatchStatus=NoComparison, no OriginalVerdict
- [ ] Replay with missing snapshot; verify MatchStatus=ReplayFailed, DeltaReport contains error
- [ ] Replay with incomplete snapshot (missing source); verify ReplayFailed with missing source names
- [ ] Replay with score difference within ScoreTolerance=0.001; verify MatchStatus=MatchWithinTolerance
- [ ] Replay with score difference exceeding tolerance; verify MatchStatus=Mismatch with FieldDeltas
- [ ] Replay with finding difference; verify DeltaReport.FindingDeltas contains Added/Removed/Modified entries
- [ ] Replay with GenerateDetailedReport=true; verify DeltaReport.SuspectedCauses is populated
- [ ] Verify AllowNetworkFetch=false prevents network access for missing sources
- [ ] Verify replay Duration is recorded in result