stella-ops.org/git.stella-ops.org
1
0
Fork 0
Code Issues Pull Requests Actions 2 Releases Wiki Activity
Files
cf5b72974f923126944f4dcd47e85864ebb31a3b
git.stella-ops.org/docs/features/unchecked/policy/proof-studio-ux.md

4.8 KiB
Raw Blame History

Proof Studio UX (Explainable Confidence Scoring)

Module

Policy

Status

IMPLEMENTED

Description

Backend confidence calculation, verdict rationale rendering, and counterfactual engine exist. The advisory identified frontend proof studio UI as a remaining gap.

What's Implemented

  • VerdictRationaleRenderer: src/Policy/__Libraries/StellaOps.Policy.Explainability/VerdictRationaleRenderer.cs
    • 4-line rationale template: Evidence, Policy Clause, Attestations, Decision
    • Multi-format output: RenderPlainText, RenderMarkdown, RenderJson (RFC 8785 canonical)
    • Content-addressed RationaleId: rat:sha256:{hash}
  • VerdictRationale model: src/Policy/__Libraries/StellaOps.Policy.Explainability/VerdictRationale.cs
    • Structured rationale: VerdictReference, RationaleEvidence (ComponentIdentity, ReachabilityDetail), RationalePolicyClause, RationaleAttestations (AttestationReference list), RationaleDecision (MitigationGuidance)
    • RationaleInputDigests for deterministic replay
  • IVerdictRationaleRenderer: src/Policy/__Libraries/StellaOps.Policy.Explainability/IVerdictRationaleRenderer.cs
    • Interface + VerdictRationaleInput record
  • CounterfactualEngine: src/Policy/__Libraries/StellaOps.Policy/Counterfactuals/CounterfactualEngine.cs
    • ComputeAsync(finding, verdict, document, scoringConfig, options) -> CounterfactualResult
    • 5 counterfactual path types: VexStatus, Exception, Reachability, VersionUpgrade, CompensatingControl
    • Each path: type, description, conditions (field/current/required), estimated effort (1-5), actor, action URI
    • Policy simulation: creates simulated findings with modified VEX/reachability tags and re-evaluates via PolicyEvaluation.EvaluateFinding()
    • Effort estimation: severity-based for exceptions (Critical=5, High=4, Medium=3, Low=2)
  • CounterfactualResult: src/Policy/__Libraries/StellaOps.Policy/Counterfactuals/CounterfactualResult.cs
    • AlreadyPassing / Blocked factory methods
    • RecommendedPath (lowest effort), HasPaths
    • 7 CounterfactualTypes: VexStatus, Exception, Reachability, VersionUpgrade, PolicyChange, ComponentRemoval, CompensatingControl
    • CounterfactualCondition: Field, CurrentValue, RequiredValue, IsMet
  • ScoreExplanation: src/Policy/__Libraries/StellaOps.Policy/Scoring/ScoreExplanation.cs
    • Per-factor explanation: Factor, Value (0-100), Reason, ContributingDigests
    • ScoreExplainBuilder: AddReachability, AddEvidence, AddProvenance, AddBaseSeverity
    • Deterministic output (sorted by factor name + digest)
  • DecayedConfidenceCalculator: src/Policy/__Libraries/StellaOps.Policy.Determinization/Scoring/DecayedConfidenceCalculator.cs
    • Exponential confidence decay for time-based scoring

What's Missing

  • Proof graph visualization: No visual representation of the full evidence graph (ProofGraphNode/Edge/Path) in the UI -- the proof-studio has confidence breakdown but not the graph
  • Interactive counterfactual explorer: CounterfactualEngine exists in backend and what-if-slider component exists in proof-studio, but the full interactive "toggle what-if scenarios" UX may not be fully wired to the backend
  • Score breakdown dashboard: ScoreExplanation data exists but no dashboard visualizing per-factor contributions with charts
  • Confidence timeline: DecayedConfidenceCalculator computes decay but no UI showing confidence over time

Additional Implementation Found

  • Proof Studio Container: src/Web/StellaOps.Web/src/app/features/proof-studio/components/proof-studio-container/proof-studio-container.component.ts -- main container component
  • Confidence Breakdown: src/Web/StellaOps.Web/src/app/features/proof-studio/components/confidence-breakdown/confidence-breakdown.component.ts -- per-factor confidence visualization
  • Confidence Factor Chip: src/Web/StellaOps.Web/src/app/features/proof-studio/components/confidence-factor-chip/confidence-factor-chip.component.ts
  • What-If Slider: src/Web/StellaOps.Web/src/app/features/proof-studio/components/what-if-slider/what-if-slider.component.ts -- counterfactual slider control
  • Proof Studio Service: src/Web/StellaOps.Web/src/app/features/proof-studio/services/proof-studio.service.ts -- API service
  • Proof Trace Model: src/Web/StellaOps.Web/src/app/features/proof-studio/models/proof-trace.model.ts

Implementation Plan

  • Wire what-if-slider to CounterfactualEngine backend API
  • Add proof graph visualization using D3.js or similar for evidence graph rendering
  • Add confidence timeline chart using DecayedConfidenceCalculator data
  • Verify proof-studio-container is fully wired to VerdictRationale API endpoint

Related Documentation

  • Explainability library: src/Policy/__Libraries/StellaOps.Policy.Explainability/
  • Counterfactuals: src/Policy/__Libraries/StellaOps.Policy/Counterfactuals/
  • Score explanations: src/Policy/__Libraries/StellaOps.Policy/Scoring/ScoreExplanation.cs