stella-ops.org/git.stella-ops.org
1
0
Fork 0
Code Issues Pull Requests Actions 2 Releases Wiki Activity
Files
fa4823f46c4bc090294580a9136fb6b0e11e50f9
git.stella-ops.org/docs/features/unchecked/attestor/replay-fidelity-verification.md

2.4 KiB
Raw Blame History

Replay Fidelity Verification

Module

Attestor

Status

IMPLEMENTED

Description

Replay result and verification models, AI artifact replayer interface, SBOM replay verification service, and CLI replay commands for deterministic replay verification.

Implementation Details

  • AI Artifact Replay Manifest: src/Attestor/__Libraries/StellaOps.Attestor.ProofChain/Replay/AIArtifactReplayManifest.cs -- manifest capturing all inputs (prompts, evidence, model parameters) for deterministic replay.
  • Replay Input Artifact: Replay/ReplayInputArtifact.cs -- individual input artifact with content hash.
  • Replay Prompt Template: Replay/ReplayPromptTemplate.cs -- exact prompt template with parameter bindings.
  • Replay Result: Replay/ReplayResult.cs -- result of a replay attempt with fidelity metrics (semantic similarity, structural match, hash match).
  • Replay Status: Replay/ReplayStatus.cs -- enum tracking replay outcome (Identical, Semantically Equivalent, Divergent, Failed).
  • Replay Verification Result: Replay/ReplayVerificationResult.cs -- verification comparing replayed output to original output.
  • AI Artifact Replayer: Replay/IAIArtifactReplayer.cs -- interface for executing replays.
  • Verification Replay Log: __Libraries/StellaOps.Attestor.EvidencePack/Models/VerificationReplayLog.cs -- log of replay verification steps.
  • Verification Replay Log Builder: __Libraries/StellaOps.Attestor.EvidencePack/Services/VerificationReplayLogBuilder.cs -- builds replay logs.
  • Tests: __Tests/StellaOps.Attestor.ProofChain.Tests/ReplayManifestTests.cs

E2E Test Plan

  • Create an AIArtifactReplayManifest with input artifacts, prompt template, and model parameters; verify all inputs are captured
  • Execute a replay via IAIArtifactReplayer and verify ReplayResult contains fidelity metrics
  • Verify ReplayStatus.Identical when the replayed output exactly matches the original
  • Verify ReplayStatus.SemanticallyEquivalent when outputs differ in formatting but are semantically equivalent
  • Verify ReplayStatus.Divergent when the replayed output significantly differs from the original
  • Compare original and replayed outputs via ReplayVerificationResult and verify detailed comparison metrics
  • Build a VerificationReplayLog capturing all replay steps and verify the log is complete
  • Verify replay determinism: replay the same manifest twice and confirm identical ReplayResult