stella-ops.org/git.stella-ops.org
1
0
Fork 0
Code Issues Pull Requests Actions 2 Releases Wiki Activity
Files
cf5b72974f923126944f4dcd47e85864ebb31a3b
git.stella-ops.org/docs/features/unchecked/attestor/ai-assisted-explanation-and-classification.md

3.2 KiB
Raw Blame History

AI-Assisted Explanation and Classification

Module

Attestor

Status

IMPLEMENTED

Description

AI authority classifier with explanation scoring, citation references, explanation types, and model identifiers. AI artifact verification step integrates into the verification pipeline.

Implementation Details

  • AIAuthorityClassifier: src/Attestor/__Libraries/StellaOps.Attestor.ProofChain/Predicates/AI/AIAuthorityClassifier.cs (with .Explanation, .ExplanationScore, .PolicyDraft, .PolicyDraftScore, .Remediation, .RemediationScore, .VexDraft, .VexDraftScore partials) -- classifies AI outputs into Suggestion, EvidenceBacked, or AuthorityThreshold based on citation rate, verified rate, and confidence score.
  • AIAuthorityThresholds: AIAuthorityThresholds.cs -- configurable thresholds: MinCitationRate (default 0.8), MinConfidenceScore (default 0.7), MinVerifiedCitationRate (default 0.9), AuthorityThresholdScore (default 0.95).
  • AIArtifactAuthority enum: AIArtifactAuthority.cs -- three levels: Suggestion (no evidence), EvidenceBacked (citations verified), AuthorityThreshold (meets auto-processing score).
  • AIExplanationPredicate: AIExplanationPredicate.cs -- record extending AIArtifactBasePredicate with ExplanationType, Content, Citations, ConfidenceScore, CitationRate, Subject, ContextScope.
  • AIExplanationCitation: AIExplanationCitation.cs -- links claims to evidence with ClaimIndex, ClaimText, EvidenceId (sha256 format), EvidenceType, Verified flag.
  • AIExplanationType enum: AIExplanationType.cs -- Exploitability, CodePath, PolicyDecision, RiskFactors, RemediationOptions, PlainLanguageSummary, EvidenceChain.
  • AIModelIdentifier: AIModelIdentifier.cs -- tracks provider/model/version with optional WeightsDigest for local models.
  • Verification Step: src/Attestor/__Libraries/StellaOps.Attestor.ProofChain/Verification/AIArtifactVerificationStep.cs (with .Execute, .Classify, .Helpers, .Summary, .VerifyParse, .VerifyValidation partials) -- integrates into VerificationPipeline to verify AI artifacts in proof bundles.
  • Tests: __Libraries/StellaOps.Attestor.ProofChain.Tests/AI/AIAuthorityClassifierTests.cs

E2E Test Plan

  • Create an AIExplanationPredicate with citation rate >= 0.8, verified rate >= 0.9, and confidence >= 0.7, classify via AIAuthorityClassifier, and verify it returns EvidenceBacked
  • Create an explanation with citation rate < 0.8 and verify classifier returns Suggestion with appropriate reason messages
  • Create an explanation with quality score >= 0.95 and verify classifier returns AuthorityThreshold
  • Submit a proof bundle containing AI artifacts through AIArtifactVerificationStep.ExecuteAsync and verify all artifacts are validated (parse, schema, classification)
  • Submit a proof bundle with invalid AI artifacts (malformed predicate) and verify the verification step returns Passed = false with error details
  • Create an AIExplanationCitation with Verified = false and verify it lowers the verified rate below the threshold, causing the classifier to return Suggestion
  • Verify AIModelIdentifier.ToString() produces the canonical provider:model:version format