49 lines
3.5 KiB
Markdown
49 lines
3.5 KiB
Markdown
# Console Simulation Diff (Shadow Gate Visual Output)
|
|
|
|
## Module
|
|
Policy
|
|
|
|
## Status
|
|
VERIFIED
|
|
|
|
## Description
|
|
Console-based simulation diff output for visual comparison of policy simulation results.
|
|
|
|
## Implementation Details
|
|
- **ConsoleSimulationDiffService**: `src/Policy/StellaOps.Policy.Engine/Console/ConsoleSimulationDiffService.cs` -- `ConsoleSimulationDiffService` (internal sealed class)
|
|
- Schema version: `console-policy-23-001` (POLICY-CONSOLE-23-002)
|
|
- `Compute(ConsoleSimulationDiffRequest)` generates deterministic before/after comparison
|
|
- Produces severity breakdown (critical/high/medium/low/unknown) for baseline and candidate policy versions
|
|
- Delta summary: added, removed, and regressed (escalated severity) finding counts
|
|
- Rule impact analysis: per-rule added/removed counts and severity shift tracking (e.g., "medium->high")
|
|
- Explain samples: deterministic trace IDs for drill-down investigation
|
|
- Budget caps: `MaxFindings` (1-50,000) and `MaxExplainSamples` (0-200) via `ConsoleDiffBudget`
|
|
- Deterministic ID generation using SHA-256 hashing of policy version + artifact digest
|
|
- All ordering is lexicographic by Ordinal for determinism
|
|
- **ConsoleSimulationDiffModels**: `src/Policy/StellaOps.Policy.Engine/Console/ConsoleSimulationDiffModels.cs` -- request/response DTOs
|
|
- `ConsoleSimulationDiffRequest`: BaselinePolicyVersion, CandidatePolicyVersion, ArtifactScope, Budget, EvaluationTimestamp
|
|
- `ConsoleSimulationDiffResponse`: SchemaVersion, Summary (Before/After/Delta), RuleImpact, Samples, Provenance
|
|
- `ConsoleArtifactScope`: ArtifactDigest, Purl, AdvisoryId
|
|
- `ConsoleDiffDelta`: Added, Removed, Regressed
|
|
- `ConsoleRuleImpact`: RuleId, Added, Removed, SeverityShifts
|
|
- `ConsoleDiffProvenance`: BaselineVersion, CandidateVersion, EvaluationTimestamp
|
|
- **SimulationAnalyticsService Integration**: Uses `SimulationAnalyticsService.ComputeDeltaSummary` for severity change detection (escalated counts)
|
|
- **Console Simulation Endpoint**: `src/Policy/StellaOps.Policy.Engine/Endpoints/ConsoleSimulationEndpoint.cs` -- REST API for triggering console simulation diffs
|
|
|
|
## E2E Test Plan
|
|
- [x] POST to console simulation endpoint with baseline and candidate policy versions; verify response contains schema version, summary, rule impact, and samples
|
|
- [x] Verify severity breakdown: before and after both contain counts for all 5 severity levels (critical/high/medium/low/unknown)
|
|
- [x] Verify delta: added count equals findings in candidate but not baseline; removed count is the inverse
|
|
- [x] Verify rule impact: each rule entry shows added, removed, and severity shift details
|
|
- [x] Verify samples: explain trace IDs are deterministic (same inputs produce same trace IDs)
|
|
- [x] POST with MaxFindings=1; verify only 1 finding per policy version in the output
|
|
- [x] POST with MaxExplainSamples=0; verify samples section contains empty arrays
|
|
- [x] POST same request twice; verify identical response (deterministic output)
|
|
- [x] Verify provenance section contains both policy versions and evaluation timestamp
|
|
- [x] POST with multiple artifact scopes; verify findings are ordered by ArtifactDigest (ordinal)
|
|
|
|
## Verification
|
|
- **Run ID**: run-001
|
|
- **Date**: 2026-02-12
|
|
- **Result**: PASS - 708/708 tests pass. ConsoleSimulationDiffServiceTests verifies determinism (JSON equality across repeated calls), schema version 'console-policy-23-001', Before/After severity totals, RuleImpact presence, budget enforcement (samples <= MaxFindings), provenance with evaluation timestamp.
|