# Explainability Testing Framework ## Module Policy ## Status IMPLEMENTED ## Description Explainability testing framework with assertion helpers and verdict rationale rendering, ensuring decisions can be traced back to evidence and assumptions. ## Implementation Details - **VerdictRationaleRenderer**: `src/Policy/__Libraries/StellaOps.Policy.Explainability/VerdictRationaleRenderer.cs` (sealed class implements `IVerdictRationaleRenderer`) - `Render(VerdictRationaleInput)` produces structured 4-line rationale with content-addressed RationaleId - `RenderPlainText(rationale)` produces plain text output (4 lines) - `RenderMarkdown(rationale)` produces Markdown with headers (Evidence, Policy Clause, Attestations, Decision) - `RenderJson(rationale)` produces canonical JSON (RFC 8785) via `CanonJson.Serialize` - Content-addressed ID: `rat:sha256:{hash}` computed from SHA256 of canonical JSON (with empty RationaleId for self-referential hashing) - **VerdictRationale model**: `src/Policy/__Libraries/StellaOps.Policy.Explainability/VerdictRationale.cs` - 4-line template structure: - Line 1 (`RationaleEvidence`): CVE, Component (PURL, name, version, ecosystem), Reachability (vulnerable function, entry point, path summary) - Line 2 (`RationalePolicyClause`): ClauseId, RuleDescription, Conditions - Line 3 (`RationaleAttestations`): PathWitness, VexStatements, Provenance (each as AttestationReference with Id, Type, Digest, Summary) - Line 4 (`RationaleDecision`): Verdict, Score, Recommendation, Mitigation (Action, Details) - `RationaleInputDigests` for reproducibility: VerdictDigest, PolicyDigest, EvidenceDigest - SchemaVersion: "1.0" - **VerdictRationaleInput**: `src/Policy/__Libraries/StellaOps.Policy.Explainability/IVerdictRationaleRenderer.cs` - Full input record with VerdictRef, Cve, Component, Reachability, PolicyClauseId, PolicyRuleDescription, PolicyConditions, attestation references, Verdict, Score, Recommendation, Mitigation, GeneratedAt, digest fields - **Property-based tests**: `src/Policy/__Tests/StellaOps.Policy.Determinization.Tests/PropertyTests/DeterminismPropertyTests.cs` -- tests determinism of rationale rendering - **Explainability integration tests**: verify that `Render` followed by `RenderPlainText`, `RenderMarkdown`, and `RenderJson` all produce deterministic output for identical inputs ## E2E Test Plan - [ ] Render rationale with CVE, component, reachability; verify Evidence.FormattedText contains CVE ID, component name, vulnerable function, entry point - [ ] Render rationale with policy clause; verify PolicyClause.FormattedText contains ClauseId and conditions - [ ] Render rationale with path witness and VEX statements; verify Attestations.FormattedText includes both references - [ ] Render rationale without attestations; verify FormattedText is "No attestations available." - [ ] Render rationale with score and mitigation; verify Decision.FormattedText includes score value and mitigation action - [ ] Render same input twice; verify RationaleId is identical (content-addressed determinism) - [ ] Render different inputs; verify RationaleIds differ - [ ] RenderMarkdown produces valid Markdown with ## headers for each section - [ ] RenderJson produces valid JSON parseable by standard JSON parser - [ ] Verify RationaleId format matches `rat:sha256:{64 hex chars}`