stella-ops.org/git.stella-ops.org
1
0
Fork 0
Code Issues Pull Requests Actions 2 Releases Wiki Activity
Files
1bf6bbf3955ecbd8d20f5eeb767da5c7f8324b57
git.stella-ops.org/docs/features/unimplemented/policy/proof-studio-ux.md

4.9 KiB
Raw Blame History

Proof Studio UX (Explainable Confidence Scoring)

Module

Policy

Status

PARTIALLY_IMPLEMENTED

Description

Backend confidence calculation, verdict rationale rendering, and counterfactual engine exist. The advisory identified frontend proof studio UI as a remaining gap.

What's Implemented

  • VerdictRationaleRenderer: src/Policy/__Libraries/StellaOps.Policy.Explainability/VerdictRationaleRenderer.cs
    • 4-line rationale template: Evidence, Policy Clause, Attestations, Decision
    • Multi-format output: RenderPlainText, RenderMarkdown, RenderJson (RFC 8785 canonical)
    • Content-addressed RationaleId: rat:sha256:{hash}
  • VerdictRationale model: src/Policy/__Libraries/StellaOps.Policy.Explainability/VerdictRationale.cs
    • Structured rationale: VerdictReference, RationaleEvidence (ComponentIdentity, ReachabilityDetail), RationalePolicyClause, RationaleAttestations (AttestationReference list), RationaleDecision (MitigationGuidance)
    • RationaleInputDigests for deterministic replay
  • IVerdictRationaleRenderer: src/Policy/__Libraries/StellaOps.Policy.Explainability/IVerdictRationaleRenderer.cs
    • Interface + VerdictRationaleInput record
  • CounterfactualEngine: src/Policy/__Libraries/StellaOps.Policy/Counterfactuals/CounterfactualEngine.cs
    • ComputeAsync(finding, verdict, document, scoringConfig, options) -> CounterfactualResult
    • 5 counterfactual path types: VexStatus, Exception, Reachability, VersionUpgrade, CompensatingControl
    • Each path: type, description, conditions (field/current/required), estimated effort (1-5), actor, action URI
    • Policy simulation: creates simulated findings with modified VEX/reachability tags and re-evaluates via PolicyEvaluation.EvaluateFinding()
    • Effort estimation: severity-based for exceptions (Critical=5, High=4, Medium=3, Low=2)
  • CounterfactualResult: src/Policy/__Libraries/StellaOps.Policy/Counterfactuals/CounterfactualResult.cs
    • AlreadyPassing / Blocked factory methods
    • RecommendedPath (lowest effort), HasPaths
    • 7 CounterfactualTypes: VexStatus, Exception, Reachability, VersionUpgrade, PolicyChange, ComponentRemoval, CompensatingControl
    • CounterfactualCondition: Field, CurrentValue, RequiredValue, IsMet
  • ScoreExplanation: src/Policy/__Libraries/StellaOps.Policy/Scoring/ScoreExplanation.cs
    • Per-factor explanation: Factor, Value (0-100), Reason, ContributingDigests
    • ScoreExplainBuilder: AddReachability, AddEvidence, AddProvenance, AddBaseSeverity
    • Deterministic output (sorted by factor name + digest)
  • DecayedConfidenceCalculator: src/Policy/__Libraries/StellaOps.Policy.Determinization/Scoring/DecayedConfidenceCalculator.cs
    • Exponential confidence decay for time-based scoring

What's Missing

  • Proof graph visualization: No visual representation of the full evidence graph (ProofGraphNode/Edge/Path) in the UI -- the proof-studio has confidence breakdown but not the graph
  • Interactive counterfactual explorer: CounterfactualEngine exists in backend and what-if-slider component exists in proof-studio, but the full interactive "toggle what-if scenarios" UX may not be fully wired to the backend
  • Score breakdown dashboard: ScoreExplanation data exists but no dashboard visualizing per-factor contributions with charts
  • Confidence timeline: DecayedConfidenceCalculator computes decay but no UI showing confidence over time

Additional Implementation Found

  • Proof Studio Container: src/Web/StellaOps.Web/src/app/features/proof-studio/components/proof-studio-container/proof-studio-container.component.ts -- main container component
  • Confidence Breakdown: src/Web/StellaOps.Web/src/app/features/proof-studio/components/confidence-breakdown/confidence-breakdown.component.ts -- per-factor confidence visualization
  • Confidence Factor Chip: src/Web/StellaOps.Web/src/app/features/proof-studio/components/confidence-factor-chip/confidence-factor-chip.component.ts
  • What-If Slider: src/Web/StellaOps.Web/src/app/features/proof-studio/components/what-if-slider/what-if-slider.component.ts -- counterfactual slider control
  • Proof Studio Service: src/Web/StellaOps.Web/src/app/features/proof-studio/services/proof-studio.service.ts -- API service
  • Proof Trace Model: src/Web/StellaOps.Web/src/app/features/proof-studio/models/proof-trace.model.ts

Implementation Plan

  • Wire what-if-slider to CounterfactualEngine backend API
  • Add proof graph visualization using D3.js or similar for evidence graph rendering
  • Add confidence timeline chart using DecayedConfidenceCalculator data
  • Verify proof-studio-container is fully wired to VerdictRationale API endpoint

Related Documentation

  • Explainability library: src/Policy/__Libraries/StellaOps.Policy.Explainability/
  • Counterfactuals: src/Policy/__Libraries/StellaOps.Policy/Counterfactuals/
  • Score explanations: src/Policy/__Libraries/StellaOps.Policy/Scoring/ScoreExplanation.cs