# SPRINT_7000_0001_0003 - Explainability with Assumptions & Falsifiability ## Sprint Metadata | Field | Value | |-------|-------| | **Sprint ID** | 7000.0001.0003 | | **Topic** | Explainability with Assumptions & Falsifiability | | **Duration** | 2 weeks | | **Priority** | HIGH | | **Status** | DOING | | **Owner** | Scanner Team + Policy Team | | **Working Directory** | `src/Scanner/__Libraries/StellaOps.Scanner.Explainability/`, `src/Policy/__Libraries/StellaOps.Policy.Explainability/` | --- ## Objective Implement auditor-grade explainability that answers four non-negotiable questions for every finding: 1. What exact evidence triggered this finding? 2. What code or binary path makes it reachable? 3. What assumptions are being made? 4. **What would falsify this conclusion?** This addresses the advisory gap: "No existing scanner answers #4." --- ## Prerequisites - [ ] Sprint 3500 (Score Proofs) complete - [ ] `StellaOps.Scanner.EntryTrace.Risk` module available - [ ] DSSE predicate schemas accessible --- ## Delivery Tracker | ID | Task | Status | Assignee | Notes | |----|------|--------|----------|-------| | 7000.0003.01 | Design assumption-set model (compiler flags, runtime config, feature gates) | DONE | Agent | Assumption.cs with enums | | 7000.0003.02 | Implement `AssumptionSet` record in findings | DONE | Agent | AssumptionSet.cs, IAssumptionCollector.cs | | 7000.0003.03 | Design falsifiability criteria model | DONE | Agent | FalsifiabilityCriteria.cs with enums | | 7000.0003.04 | Add "what would disprove this?" to `RiskExplainer` output | DONE | Agent | FalsifiabilityGenerator.cs, RiskReport.cs | | 7000.0003.05 | Implement evidence-density confidence scorer | DONE | Agent | EvidenceDensityScorer.cs with 8 factors | | 7000.0003.06 | Add assumption-set to DSSE predicate schema | DONE | Agent | finding-explainability-predicate.schema.json + ExplainabilityPredicateSerializer | | 7000.0003.07 | UI: Explainability widget with assumption drill-down | TODO | | Deferred - Angular | --- ## Task Details ### 7000.0003.01: Assumption-Set Model Design **Description**: Design the data model for tracking assumptions made during analysis. **Deliverables**: - `Assumption` domain model: ```csharp public record Assumption( AssumptionCategory Category, string Key, string AssumedValue, string? ObservedValue, AssumptionSource Source, ConfidenceLevel Confidence ); public enum AssumptionCategory { CompilerFlag, // -fstack-protector, -D_FORTIFY_SOURCE RuntimeConfig, // Environment variables, config files FeatureGate, // Feature flags, build variants LoaderBehavior, // LD_PRELOAD, RPATH, symbol versioning NetworkExposure, // Port bindings, firewall rules ProcessPrivilege // Capabilities, seccomp, AppArmor } public enum AssumptionSource { Static, Dynamic, Inferred, Default } ``` - `AssumptionSet` aggregate: ```csharp public record AssumptionSet( ImmutableArray Assumptions, int TotalCount, int VerifiedCount, int InferredCount, double AssumptionRisk // Higher = more unverified assumptions ); ``` **Acceptance Criteria**: - [ ] All assumption categories covered - [ ] Confidence levels defined - [ ] Risk score derivable from assumptions --- ### 7000.0003.02: AssumptionSet in Findings **Description**: Integrate assumption tracking into finding records. **Deliverables**: - Update `VulnerabilityFinding` to include `AssumptionSet` - Assumption collector during scan: ```csharp public interface IAssumptionCollector { void RecordAssumption(Assumption assumption); AssumptionSet Build(); } ``` - Wire into Scanner Worker pipeline **Acceptance Criteria**: - [ ] Every finding has AssumptionSet - [ ] Assumptions collected during analysis - [ ] Deterministic ordering --- ### 7000.0003.03: Falsifiability Criteria Model **Description**: Design model for expressing what would disprove a finding. **Deliverables**: - `FalsifiabilityCriteria` model: ```csharp public record FalsifiabilityCriteria( ImmutableArray Conditions, string HumanReadable ); public record FalsificationCondition( FalsificationCategory Category, string Description, string? VerificationCommand, // CLI command to verify string? VerificationQuery // API query to verify ); public enum FalsificationCategory { CodeRemoved, // "Vulnerable function call removed" PackageUpgraded, // "Package upgraded past fix version" ConfigDisabled, // "Vulnerable feature disabled via config" PathUnreachable, // "Call path no longer reachable from entrypoint" RuntimeGuarded, // "Runtime check prevents exploitation" SymbolUnresolved // "Vulnerable symbol not linked" } ``` - Falsifiability generator per finding type **Acceptance Criteria**: - [ ] Every finding has falsifiability criteria - [ ] Human-readable description - [ ] Verification command where applicable --- ### 7000.0003.04: RiskExplainer Enhancement **Description**: Extend `RiskExplainer` to output falsifiability and assumptions. **Deliverables**: - Update `RiskReport` to include: ```csharp public record RiskReport( RiskAssessment Assessment, string Explanation, ImmutableArray Recommendations, AssumptionSet Assumptions, // NEW FalsifiabilityCriteria Falsifiability // NEW ); ``` - Natural language generation for: - "This finding assumes..." - "To disprove this finding, verify that..." **Acceptance Criteria**: - [ ] Explanation includes assumptions - [ ] Explanation includes falsifiability - [ ] Language is auditor-appropriate --- ### 7000.0003.05: Evidence-Density Confidence Scorer **Description**: Implement confidence scoring based on evidence density, not CVSS. **Deliverables**: - `EvidenceDensityScorer`: ```csharp public interface IEvidenceDensityScorer { ConfidenceScore Score(EvidenceBundle evidence, AssumptionSet assumptions); } public record ConfidenceScore( double Value, // 0.0 - 1.0 ConfidenceTier Tier, // Confirmed, High, Medium, Low, Speculative ImmutableArray Factors // What contributed to score ); public enum ConfidenceTier { Confirmed, High, Medium, Low, Speculative } ``` - Scoring factors: - Evidence count - Evidence diversity (static + dynamic + runtime) - Assumption penalty (more unverified = lower confidence) - Corroboration bonus (multiple sources agree) **Acceptance Criteria**: - [ ] Confidence derived from evidence, not CVSS - [ ] Deterministic scoring - [ ] Factors explainable --- ### 7000.0003.06: DSSE Predicate Schema Update **Description**: Add assumption-set and falsifiability to DSSE predicate. **Deliverables**: - Schema: `stellaops.dev/predicates/finding@v2` ```json { "$schema": "...", "type": "object", "properties": { "finding": { "$ref": "#/definitions/Finding" }, "assumptions": { "type": "array", "items": { "$ref": "#/definitions/Assumption" } }, "falsifiability": { "type": "object", "properties": { "conditions": { "type": "array" }, "humanReadable": { "type": "string" } } }, "evidenceConfidence": { "type": "object", "properties": { "value": { "type": "number" }, "tier": { "type": "string" }, "factors": { "type": "array" } } } } } ``` - Migration path from v1 predicates **Acceptance Criteria**: - [ ] Schema validates - [ ] Backward compatible - [ ] Registered in predicate registry --- ### 7000.0003.07: UI Explainability Widget **Description**: Angular component for assumption and falsifiability drill-down. **Deliverables**: - `` component - Tabs: Evidence | Assumptions | "How to Disprove" - Assumption table with confidence indicators - Falsifiability checklist with verification commands - Copy-to-clipboard for verification commands **Acceptance Criteria**: - [ ] Renders for all finding types - [ ] Assumptions sortable/filterable - [ ] Verification commands copyable - [ ] Accessible (WCAG 2.1 AA) --- ## Testing Requirements | Test Type | Location | Coverage | |-----------|----------|----------| | Unit tests | `StellaOps.Scanner.Explainability.Tests/` | Models, scorers | | Integration tests | `StellaOps.Scanner.WebService.Tests/Explainability/` | API endpoints | | UI tests | `src/Web/StellaOps.Web/tests/explainability/` | Component tests | | Golden fixtures | `src/Scanner/__Tests/Fixtures/Explainability/` | Deterministic output | --- ## Documentation Updates | Document | Update Required | |----------|-----------------| | `docs/explainability/assumption-model.md` | CREATE - Assumption-set design | | `docs/explainability/falsifiability.md` | CREATE - Falsifiability guide | | `docs/schemas/finding-predicate-v2.md` | CREATE - Schema documentation | | `docs/api/scanner-findings-api.md` | UPDATE - Explainability fields | --- ## Decisions & Risks | ID | Decision/Risk | Status | Resolution | |----|---------------|--------|------------| | D1 | How to handle assumptions for legacy findings? | OPEN | Propose: empty set with "legacy" flag | | D2 | Falsifiability verification commands: shell or API? | OPEN | Propose: both where applicable | | R1 | Performance impact of assumption collection | OPEN | Profile and optimize | --- ## Execution Log | Date (UTC) | Update | Owner | |------------|--------|-------| | 2025-12-22 | Sprint created from advisory gap analysis | Agent | | 2025-12-22 | Tasks 1-6 complete: Assumption models, AssumptionCollector, Falsifiability models, FalsifiabilityGenerator, EvidenceDensityScorer, RiskReport, DSSE predicate schema with serializer. 93 tests passing. Task 7 (Angular UI) deferred. | Agent | --- ## Required Reading - `src/Scanner/__Libraries/StellaOps.Scanner.EntryTrace/Risk/AGENTS.md` - `docs/modules/scanner/architecture.md` - `docs/product-advisories/archived/*/19-Dec-2025 - Benchmarking Container Scanners Against Stella Ops.md` (Section 4: Explainability)