git.stella-ops.org/docs/features/unchecked/policy/counterfactual-engine.md

Counterfactual Engine (Policy Diff Analysis)

Module

Policy

Status

IMPLEMENTED

Description

Counterfactual engine that computes the difference between current and proposed policy configurations to show what would change.

Implementation Details

• CounterfactualEngine: src/Policy/__Libraries/StellaOps.Policy/Counterfactuals/CounterfactualEngine.cs -- CounterfactualEngine (sealed class implements ICounterfactualEngine)
  • ComputeAsync(finding, verdict, document, scoringConfig, options?, ct) computes counterfactual paths for a blocked finding
  • Returns CounterfactualResult with paths to pass when finding is currently blocked
  • Returns AlreadyPassing result when finding already has Pass verdict
  • Five counterfactual path types:
    • VEX path: simulates finding with not_affected VEX status; skipped if already not_affected
    • Exception path: computes exception effort based on severity (Critical=5, High=4, Medium=3, Low=2)
    • Reachability path: simulates finding with reachability:no; effort varies (2 if unknown, 4 if currently reachable)
    • Version upgrade path: uses FixedVersionLookup delegate to find fixed version; extracts current version from PURL
    • Compensating control path: suggests compensating controls (effort=4)
  • VEX and reachability paths use PolicyEvaluation.EvaluateFinding to simulate what-if verdicts
  • Tag-based signals: reads vex: and reachability: prefixed tags from finding
• CounterfactualOptions: src/Policy/__Libraries/StellaOps.Policy/Counterfactuals/CounterfactualEngine.cs -- options record
  • IncludeVexPaths, IncludeExceptionPaths, IncludeReachabilityPaths, IncludeVersionUpgradePaths, IncludeCompensatingControlPaths (all default true)
  • PolicyAllowsExceptions, PolicyConsidersReachability, PolicyAllowsCompensatingControls (all default true)
  • FixedVersionLookup -- async delegate (cve, purl, ct) => fixedVersion? for version upgrade lookup
• CounterfactualResult: src/Policy/__Libraries/StellaOps.Policy/Counterfactuals/CounterfactualResult.cs -- result model with finding ID and list of CounterfactualPath
• CounterfactualPath: paths with type, description, and effort rating
  • Factory methods: Vex(currentStatus, cve, effort), Exception(cve, effort), Reachability(current, findingId, effort), VersionUpgrade(current, fixed, purl, effort), CompensatingControl(findingId, effort)

E2E Test Plan

• Compute counterfactuals for a blocked finding with VEX status=affected; verify VEX path suggests not_affected and simulated verdict would pass
• Compute counterfactuals for a finding already passing; verify AlreadyPassing result with no paths
• Compute counterfactuals with IncludeVexPaths=false; verify no VEX path in result
• Compute counterfactuals for a finding with reachability=unknown; verify reachability path with effort=2
• Compute counterfactuals for a finding with reachability=yes; verify reachability path with effort=4
• Compute counterfactuals with FixedVersionLookup providing a fixed version; verify version upgrade path with current and fixed versions
• Compute counterfactuals with FixedVersionLookup returning null; verify no version upgrade path
• Verify exception path effort: Critical finding has effort=5, Low finding has effort=2
• Compute counterfactuals with PolicyAllowsExceptions=false; verify no exception path
• Verify all five path types are present when all options are enabled and applicable