stella-ops.org/git.stella-ops.org
1
0
Fork 0
Code Issues Pull Requests Actions 2 Releases Wiki Activity
Files
9ca2de05dff9c7db11eb49df6d4e47bbfe7a99f6
git.stella-ops.org/docs/features/checked/attestor/ai-authority-classification-engine.md
master 5bca406787 save checkpoint: save features
2026-02-12 10:27:23 +02:00

3.7 KiB
Raw Blame History

AI Authority Classification Engine

Module

Attestor

Status

VERIFIED

Description

Authority classification engine that determines whether AI outputs are evidence-backed (authoritative) or suggestion-only, with configurable thresholds and scoring across multiple artifact types.

Implementation Details

  • Core Classifier: src/Attestor/__Libraries/StellaOps.Attestor.ProofChain/Predicates/AI/AIAuthorityClassifier.cs -- partial class with DetermineAuthority() method that evaluates citationRate, verifiedRate, confidenceScore, and qualityScore against configurable thresholds.
  • Artifact-Specific Scoring: Partial files implement scoring for each artifact type:
    • AIAuthorityClassifier.Explanation.cs / AIAuthorityClassifier.ExplanationScore.cs -- explanation classification and scoring
    • AIAuthorityClassifier.PolicyDraft.cs / AIAuthorityClassifier.PolicyDraftScore.cs -- policy draft classification
    • AIAuthorityClassifier.Remediation.cs / AIAuthorityClassifier.RemediationScore.cs -- remediation plan classification
    • AIAuthorityClassifier.VexDraft.cs / AIAuthorityClassifier.VexDraftScore.cs -- VEX statement draft classification
  • Authority Levels: AIArtifactAuthority.cs -- Suggestion (no evidence backing), EvidenceBacked (citations verified, evidence resolvable), AuthorityThreshold (auto-processing eligible)
  • Thresholds Config: AIAuthorityThresholds.cs -- MinCitationRate (0.8), MinConfidenceScore (0.7), MinVerifiedCitationRate (0.9), AuthorityThresholdScore (0.95), RequireResolvableEvidence (true)
  • Classification Result: AIAuthorityClassificationResult.cs -- captures authority level, reasons, and individual scores
  • Evidence Resolution: Constructor accepts optional Func<string, bool> evidence resolver to verify that cited evidence IDs are resolvable
  • Tests: src/Attestor/__Tests/StellaOps.Attestor.ProofChain.Tests/AI/AIAuthorityClassifierTests.cs

E2E Test Plan

  • Classify an explanation with all metrics above thresholds and verify EvidenceBacked result with three reason entries (citation rate, verified rate, confidence)
  • Classify a policy draft with qualityScore >= 0.95 and verify AuthorityThreshold result regardless of other metrics
  • Classify a remediation plan with citationRate = 0.5 and verify Suggestion result with reason mentioning citation rate below threshold
  • Classify a VEX draft with an evidence resolver that returns false for some evidence IDs and verify the verified rate drops below threshold
  • Override AIAuthorityThresholds with stricter values (e.g., MinCitationRate = 0.95) and verify classification changes accordingly
  • Verify all four artifact-type classifiers (Explanation, PolicyDraft, Remediation, VexDraft) produce correct AIAuthorityClassificationResult with type-specific scoring

Verification

  • Verified on 2026-02-11 with run-002.
  • Tier 0 source checks passed for classifier core, thresholds, authority enums, scoring partials, and classification-result model.
  • Tier 1 build passed; scoped xUnit class run passed (11/11) including policy-threshold and VEX evidence-resolvability downgrade scenarios.
  • Tier 2 behavioral checks passed across Explanation, PolicyDraft, Remediation, and VEX draft classification paths.
  • Full ProofChain suite remains red on unrelated baseline failures (35 failed / 780 total); evidence captured for traceability.
  • Evidence:
    • docs/qa/feature-checks/runs/attestor/ai-authority-classification-engine/run-002/tier0-source-check.json
    • docs/qa/feature-checks/runs/attestor/ai-authority-classification-engine/run-002/tier1-build-check.json
    • docs/qa/feature-checks/runs/attestor/ai-authority-classification-engine/run-002/tier2-integration-check.json