git.stella-ops.org/docs/features/checked/policy/batch-simulation-orchestration.md

Batch Simulation Orchestration

Module

Policy

Status

VERIFIED

Description

Batch simulation orchestration for running multiple policy simulations in parallel with a dedicated simulation service in the policy registry.

Implementation Details

• RiskSimulationService: src/Policy/StellaOps.Policy.Engine/Simulation/RiskSimulationService.cs -- RiskSimulationService (sealed class)
  • Simulate(RiskSimulationRequest) runs a single risk simulation with finding scores, distributions, top movers, and aggregate metrics
  • SimulateWithBreakdown(RiskSimulationRequest, RiskSimulationBreakdownOptions?) runs with detailed breakdown analytics per POLICY-RISK-67-003
  • CompareProfilesWithBreakdown(baseProfileId, compareProfileId, findings) runs comparison simulation between two risk profiles with trend analysis
  • GenerateBreakdown(result, findings) generates standalone breakdown for existing simulation results
  • Score formula: signal values * weights -> normalized to 0-100 range
  • Signal types: Boolean (0/1), Numeric (direct), Categorical (mapped: none=0.0, low=0.3, medium=0.6, critical=1.0)
  • Severity thresholds: Critical >= 90, High >= 70, Medium >= 40, Low >= 10, else Informational
  • Actions: Critical/High -> Deny, Medium -> Review, Low/Info -> Allow
  • Supports severity and decision overrides via predicate matching on signal values
  • Produces distribution with 10 buckets and percentiles (p25, p50, p75, p90, p95, p99)
  • Top 10 movers identified by score with primary driver signals
• RiskSimulationBreakdownService: src/Policy/StellaOps.Policy.Engine/Simulation/RiskSimulationBreakdownService.cs -- generates detailed breakdown analytics and comparison breakdowns
• SimulationAnalyticsService: src/Policy/StellaOps.Policy.Engine/Simulation/SimulationAnalyticsService.cs -- simulation analytics with delta summary computation
• WhatIfSimulationService: src/Policy/StellaOps.Policy.Engine/WhatIfSimulation/WhatIfSimulationService.cs -- WhatIfSimulationService (internal sealed class)
  • SimulateAsync(WhatIfSimulationRequest) executes what-if simulation without persisting results
  • Supports hypothetical SBOM diffs (add/remove/upgrade/downgrade operations) and draft policy evaluation
  • Computes decision changes between baseline and simulated: status_changed, severity_changed, new, removed
  • Impact analysis: risk delta (increased/decreased/unchanged), blocked/warning deltas, recommendations
  • VEX override handling: not_affected overrides deny to allow; unreachable downgrades deny to warn
  • Uses EffectiveDecisionMap, PolicyCompilationService, PolicyPackRepository
• Simulation Endpoints: src/Policy/StellaOps.Policy.Engine/Endpoints/RiskSimulationEndpoints.cs -- REST API for simulation runs
• Overlay Simulation: src/Policy/StellaOps.Policy.Engine/Endpoints/OverlaySimulationEndpoint.cs -- overlay-based simulation
• Console Simulation: src/Policy/StellaOps.Policy.Engine/Endpoints/ConsoleSimulationEndpoint.cs -- console surface simulation
• Path Scope Simulation: src/Policy/StellaOps.Policy.Engine/Endpoints/PathScopeSimulationEndpoint.cs -- path-scoped simulation
• Batch Evaluation: src/Policy/StellaOps.Policy.Engine/BatchEvaluation/BatchEvaluationModels.cs -- models for batch evaluation runs
• Telemetry: RiskSimulationsRun counter via PolicyEngineTelemetry, activity tracing with risk_simulation.run and policy.whatif.simulate

E2E Test Plan

• POST to risk simulation endpoint with a profile ID and list of findings; verify response contains simulation ID, finding scores, distributions, and aggregate metrics
• POST with IncludeContributions=true and verify signal contribution percentages sum to ~100% and override chain is returned
• POST with IncludeDistribution=true and verify 10 buckets, 6 percentile levels, severity breakdown covering all 5 levels
• Run SimulateWithBreakdown and verify breakdown analytics are present alongside the base simulation result
• Run CompareProfilesWithBreakdown with two profile IDs and verify baseline vs compare results with trend analysis
• POST what-if simulation with SBOM diff (add component with advisory) and verify new deny decision appears
• POST what-if simulation with SBOM diff (remove component) and verify allow decision and removed change type
• POST what-if simulation with draft policy YAML and verify simulated policy reference includes computed digest
• Verify simulation IDs are deterministic (same inputs produce same rsim-* prefix ID)
• Verify simulation with 0 findings returns zeroed aggregate metrics