64 lines
4.8 KiB
Markdown
64 lines
4.8 KiB
Markdown
# Proof Studio UX (Explainable Confidence Scoring)
|
|
|
|
## Module
|
|
Policy
|
|
|
|
## Status
|
|
IMPLEMENTED
|
|
|
|
## Description
|
|
Backend confidence calculation, verdict rationale rendering, and counterfactual engine exist. The advisory identified frontend proof studio UI as a remaining gap.
|
|
|
|
## What's Implemented
|
|
- **VerdictRationaleRenderer**: `src/Policy/__Libraries/StellaOps.Policy.Explainability/VerdictRationaleRenderer.cs`
|
|
- 4-line rationale template: Evidence, Policy Clause, Attestations, Decision
|
|
- Multi-format output: RenderPlainText, RenderMarkdown, RenderJson (RFC 8785 canonical)
|
|
- Content-addressed RationaleId: `rat:sha256:{hash}`
|
|
- **VerdictRationale model**: `src/Policy/__Libraries/StellaOps.Policy.Explainability/VerdictRationale.cs`
|
|
- Structured rationale: VerdictReference, RationaleEvidence (ComponentIdentity, ReachabilityDetail), RationalePolicyClause, RationaleAttestations (AttestationReference list), RationaleDecision (MitigationGuidance)
|
|
- RationaleInputDigests for deterministic replay
|
|
- **IVerdictRationaleRenderer**: `src/Policy/__Libraries/StellaOps.Policy.Explainability/IVerdictRationaleRenderer.cs`
|
|
- Interface + VerdictRationaleInput record
|
|
- **CounterfactualEngine**: `src/Policy/__Libraries/StellaOps.Policy/Counterfactuals/CounterfactualEngine.cs`
|
|
- `ComputeAsync(finding, verdict, document, scoringConfig, options)` -> CounterfactualResult
|
|
- 5 counterfactual path types: VexStatus, Exception, Reachability, VersionUpgrade, CompensatingControl
|
|
- Each path: type, description, conditions (field/current/required), estimated effort (1-5), actor, action URI
|
|
- Policy simulation: creates simulated findings with modified VEX/reachability tags and re-evaluates via `PolicyEvaluation.EvaluateFinding()`
|
|
- Effort estimation: severity-based for exceptions (Critical=5, High=4, Medium=3, Low=2)
|
|
- **CounterfactualResult**: `src/Policy/__Libraries/StellaOps.Policy/Counterfactuals/CounterfactualResult.cs`
|
|
- AlreadyPassing / Blocked factory methods
|
|
- RecommendedPath (lowest effort), HasPaths
|
|
- 7 CounterfactualTypes: VexStatus, Exception, Reachability, VersionUpgrade, PolicyChange, ComponentRemoval, CompensatingControl
|
|
- CounterfactualCondition: Field, CurrentValue, RequiredValue, IsMet
|
|
- **ScoreExplanation**: `src/Policy/__Libraries/StellaOps.Policy/Scoring/ScoreExplanation.cs`
|
|
- Per-factor explanation: Factor, Value (0-100), Reason, ContributingDigests
|
|
- ScoreExplainBuilder: AddReachability, AddEvidence, AddProvenance, AddBaseSeverity
|
|
- Deterministic output (sorted by factor name + digest)
|
|
- **DecayedConfidenceCalculator**: `src/Policy/__Libraries/StellaOps.Policy.Determinization/Scoring/DecayedConfidenceCalculator.cs`
|
|
- Exponential confidence decay for time-based scoring
|
|
|
|
## What's Missing
|
|
- **Proof graph visualization**: No visual representation of the full evidence graph (ProofGraphNode/Edge/Path) in the UI -- the proof-studio has confidence breakdown but not the graph
|
|
- **Interactive counterfactual explorer**: CounterfactualEngine exists in backend and `what-if-slider` component exists in proof-studio, but the full interactive "toggle what-if scenarios" UX may not be fully wired to the backend
|
|
- **Score breakdown dashboard**: ScoreExplanation data exists but no dashboard visualizing per-factor contributions with charts
|
|
- **Confidence timeline**: DecayedConfidenceCalculator computes decay but no UI showing confidence over time
|
|
|
|
## Additional Implementation Found
|
|
- **Proof Studio Container**: `src/Web/StellaOps.Web/src/app/features/proof-studio/components/proof-studio-container/proof-studio-container.component.ts` -- main container component
|
|
- **Confidence Breakdown**: `src/Web/StellaOps.Web/src/app/features/proof-studio/components/confidence-breakdown/confidence-breakdown.component.ts` -- per-factor confidence visualization
|
|
- **Confidence Factor Chip**: `src/Web/StellaOps.Web/src/app/features/proof-studio/components/confidence-factor-chip/confidence-factor-chip.component.ts`
|
|
- **What-If Slider**: `src/Web/StellaOps.Web/src/app/features/proof-studio/components/what-if-slider/what-if-slider.component.ts` -- counterfactual slider control
|
|
- **Proof Studio Service**: `src/Web/StellaOps.Web/src/app/features/proof-studio/services/proof-studio.service.ts` -- API service
|
|
- **Proof Trace Model**: `src/Web/StellaOps.Web/src/app/features/proof-studio/models/proof-trace.model.ts`
|
|
|
|
## Implementation Plan
|
|
- Wire what-if-slider to CounterfactualEngine backend API
|
|
- Add proof graph visualization using D3.js or similar for evidence graph rendering
|
|
- Add confidence timeline chart using DecayedConfidenceCalculator data
|
|
- Verify proof-studio-container is fully wired to VerdictRationale API endpoint
|
|
|
|
## Related Documentation
|
|
- Explainability library: `src/Policy/__Libraries/StellaOps.Policy.Explainability/`
|
|
- Counterfactuals: `src/Policy/__Libraries/StellaOps.Policy/Counterfactuals/`
|
|
- Score explanations: `src/Policy/__Libraries/StellaOps.Policy/Scoring/ScoreExplanation.cs`
|