44 lines
3.1 KiB
Markdown
44 lines
3.1 KiB
Markdown
# Policy Simulation Engine (Shadow Runs / What-If Analysis)
|
|
|
|
## Module
|
|
Policy
|
|
|
|
## Status
|
|
IMPLEMENTED
|
|
|
|
## Description
|
|
Full policy simulation engine with risk simulation, what-if analysis, simulation analytics, and breakdown services. Multiple simulation endpoints exist (RiskSimulationEndpoints, PathScopeSimulationEndpoint, OverlaySimulationEndpoint, ConsoleSimulationEndpoint).
|
|
|
|
## Implementation Details
|
|
- **RiskSimulationService**: `src/Policy/StellaOps.Policy.Engine/Simulation/RiskSimulationService.cs` (sealed class)
|
|
- `Simulate(items)` computes aggregate risk scores for a set of scored items
|
|
- `SimulateWithBreakdown(items)` adds distribution (10 buckets, percentiles p25/p50/p75/p90/p95/p99), severity breakdown (Critical/High/Medium/Low), top 10 movers
|
|
- `CompareProfilesWithBreakdown(profileA, profileB)` side-by-side profile comparison with delta analysis
|
|
- Signal scoring: Boolean (0/1), Numeric (direct), Categorical (mapped weight)
|
|
- Severity mapping: Critical>=90, High>=70, Medium>=40, Low>=10
|
|
- **WhatIfSimulationService**: `src/Policy/StellaOps.Policy.Engine/WhatIfSimulation/WhatIfSimulationService.cs` (sealed class)
|
|
- `SimulateAsync()` computes baseline vs simulated decisions with SBOM diffs
|
|
- Diff operations: add, remove, upgrade, downgrade with advisory/VEX/reachability
|
|
- Decision changes: status_changed, severity_changed, new, removed
|
|
- Impact summary: risk delta, blocked/warning deltas, recommendations
|
|
- **ConsoleSimulationDiffService**: `src/Policy/StellaOps.Policy.Engine/Console/ConsoleSimulationDiffService.cs`
|
|
- Schema version: console-policy-23-001
|
|
- Deterministic before/after severity breakdowns, delta, rule impact, explain samples
|
|
- **Endpoints**:
|
|
- `RiskSimulationEndpoints.cs`: `src/Policy/StellaOps.Policy.Engine/Endpoints/RiskSimulationEndpoints.cs`
|
|
- `PathScopeSimulationEndpoint.cs`: `src/Policy/StellaOps.Policy.Engine/Endpoints/PathScopeSimulationEndpoint.cs`
|
|
- `OverlaySimulationEndpoint.cs`: `src/Policy/StellaOps.Policy.Engine/Endpoints/OverlaySimulationEndpoint.cs`
|
|
- `ConsoleSimulationEndpoint.cs`: `src/Policy/StellaOps.Policy.Engine/Endpoints/ConsoleSimulationEndpoint.cs`
|
|
|
|
## E2E Test Plan
|
|
- [ ] POST risk simulation with 10 scored items; verify response includes aggregate score, distribution (10 buckets), severity breakdown, top movers
|
|
- [ ] POST what-if simulation with add operation (add new component); verify new findings appear in delta
|
|
- [ ] POST what-if simulation with upgrade operation; verify severity_changed findings in delta
|
|
- [ ] POST path-scope simulation targeting specific artifact; verify results scoped to that artifact
|
|
- [ ] POST overlay simulation with policy rule change; verify delta reflects rule impact
|
|
- [ ] Compare two scoring profiles; verify CompareProfilesWithBreakdown returns per-profile breakdowns with deltas
|
|
- [ ] Verify console simulation diff produces deterministic output (same input -> same output)
|
|
- [ ] POST what-if simulation with remove operation; verify removed findings in delta
|
|
- [ ] Verify distribution percentiles: p25 <= p50 <= p75 <= p90 <= p95 <= p99
|
|
- [ ] Verify top movers are sorted by normalized score descending (top 10 limit)
|