stella-ops.org/git.stella-ops.org
1
0
Fork 0
Code Issues Pull Requests Actions 2 Releases Wiki Activity
Files
main
git.stella-ops.org/docs/features/checked/policy/policy-simulation-engine.md
master 9ca2de05df more features checks. setup improvements
2026-02-13 02:04:55 +02:00

3.1 KiB
Raw Permalink Blame History

Policy Simulation Engine (Shadow Runs / What-If Analysis)

Module

Policy

Status

IMPLEMENTED

Description

Full policy simulation engine with risk simulation, what-if analysis, simulation analytics, and breakdown services. Multiple simulation endpoints exist (RiskSimulationEndpoints, PathScopeSimulationEndpoint, OverlaySimulationEndpoint, ConsoleSimulationEndpoint).

Implementation Details

  • RiskSimulationService: src/Policy/StellaOps.Policy.Engine/Simulation/RiskSimulationService.cs (sealed class)
    • Simulate(items) computes aggregate risk scores for a set of scored items
    • SimulateWithBreakdown(items) adds distribution (10 buckets, percentiles p25/p50/p75/p90/p95/p99), severity breakdown (Critical/High/Medium/Low), top 10 movers
    • CompareProfilesWithBreakdown(profileA, profileB) side-by-side profile comparison with delta analysis
    • Signal scoring: Boolean (0/1), Numeric (direct), Categorical (mapped weight)
    • Severity mapping: Critical>=90, High>=70, Medium>=40, Low>=10
  • WhatIfSimulationService: src/Policy/StellaOps.Policy.Engine/WhatIfSimulation/WhatIfSimulationService.cs (sealed class)
    • SimulateAsync() computes baseline vs simulated decisions with SBOM diffs
    • Diff operations: add, remove, upgrade, downgrade with advisory/VEX/reachability
    • Decision changes: status_changed, severity_changed, new, removed
    • Impact summary: risk delta, blocked/warning deltas, recommendations
  • ConsoleSimulationDiffService: src/Policy/StellaOps.Policy.Engine/Console/ConsoleSimulationDiffService.cs
    • Schema version: console-policy-23-001
    • Deterministic before/after severity breakdowns, delta, rule impact, explain samples
  • Endpoints:
    • RiskSimulationEndpoints.cs: src/Policy/StellaOps.Policy.Engine/Endpoints/RiskSimulationEndpoints.cs
    • PathScopeSimulationEndpoint.cs: src/Policy/StellaOps.Policy.Engine/Endpoints/PathScopeSimulationEndpoint.cs
    • OverlaySimulationEndpoint.cs: src/Policy/StellaOps.Policy.Engine/Endpoints/OverlaySimulationEndpoint.cs
    • ConsoleSimulationEndpoint.cs: src/Policy/StellaOps.Policy.Engine/Endpoints/ConsoleSimulationEndpoint.cs

E2E Test Plan

  • POST risk simulation with 10 scored items; verify response includes aggregate score, distribution (10 buckets), severity breakdown, top movers
  • POST what-if simulation with add operation (add new component); verify new findings appear in delta
  • POST what-if simulation with upgrade operation; verify severity_changed findings in delta
  • POST path-scope simulation targeting specific artifact; verify results scoped to that artifact
  • POST overlay simulation with policy rule change; verify delta reflects rule impact
  • Compare two scoring profiles; verify CompareProfilesWithBreakdown returns per-profile breakdowns with deltas
  • Verify console simulation diff produces deterministic output (same input -> same output)
  • POST what-if simulation with remove operation; verify removed findings in delta
  • Verify distribution percentiles: p25 <= p50 <= p75 <= p90 <= p95 <= p99
  • Verify top movers are sorted by normalized score descending (top 10 limit)