stella-ops.org/git.stella-ops.org
1
0
Fork 0
Code Issues Pull Requests Actions 2 Releases Wiki Activity
Files
c52ca826528b63f38386a0adddecff0eaacfcf30
git.stella-ops.org/docs/features/checked/policy/console-simulation-diff.md
master 9ca2de05df more features checks. setup improvements
2026-02-13 02:04:55 +02:00

3.5 KiB
Raw Blame History

Console Simulation Diff (Shadow Gate Visual Output)

Module

Policy

Status

VERIFIED

Description

Console-based simulation diff output for visual comparison of policy simulation results.

Implementation Details

  • ConsoleSimulationDiffService: src/Policy/StellaOps.Policy.Engine/Console/ConsoleSimulationDiffService.cs -- ConsoleSimulationDiffService (internal sealed class)
    • Schema version: console-policy-23-001 (POLICY-CONSOLE-23-002)
    • Compute(ConsoleSimulationDiffRequest) generates deterministic before/after comparison
    • Produces severity breakdown (critical/high/medium/low/unknown) for baseline and candidate policy versions
    • Delta summary: added, removed, and regressed (escalated severity) finding counts
    • Rule impact analysis: per-rule added/removed counts and severity shift tracking (e.g., "medium->high")
    • Explain samples: deterministic trace IDs for drill-down investigation
    • Budget caps: MaxFindings (1-50,000) and MaxExplainSamples (0-200) via ConsoleDiffBudget
    • Deterministic ID generation using SHA-256 hashing of policy version + artifact digest
    • All ordering is lexicographic by Ordinal for determinism
  • ConsoleSimulationDiffModels: src/Policy/StellaOps.Policy.Engine/Console/ConsoleSimulationDiffModels.cs -- request/response DTOs
    • ConsoleSimulationDiffRequest: BaselinePolicyVersion, CandidatePolicyVersion, ArtifactScope, Budget, EvaluationTimestamp
    • ConsoleSimulationDiffResponse: SchemaVersion, Summary (Before/After/Delta), RuleImpact, Samples, Provenance
    • ConsoleArtifactScope: ArtifactDigest, Purl, AdvisoryId
    • ConsoleDiffDelta: Added, Removed, Regressed
    • ConsoleRuleImpact: RuleId, Added, Removed, SeverityShifts
    • ConsoleDiffProvenance: BaselineVersion, CandidateVersion, EvaluationTimestamp
  • SimulationAnalyticsService Integration: Uses SimulationAnalyticsService.ComputeDeltaSummary for severity change detection (escalated counts)
  • Console Simulation Endpoint: src/Policy/StellaOps.Policy.Engine/Endpoints/ConsoleSimulationEndpoint.cs -- REST API for triggering console simulation diffs

E2E Test Plan

  • POST to console simulation endpoint with baseline and candidate policy versions; verify response contains schema version, summary, rule impact, and samples
  • Verify severity breakdown: before and after both contain counts for all 5 severity levels (critical/high/medium/low/unknown)
  • Verify delta: added count equals findings in candidate but not baseline; removed count is the inverse
  • Verify rule impact: each rule entry shows added, removed, and severity shift details
  • Verify samples: explain trace IDs are deterministic (same inputs produce same trace IDs)
  • POST with MaxFindings=1; verify only 1 finding per policy version in the output
  • POST with MaxExplainSamples=0; verify samples section contains empty arrays
  • POST same request twice; verify identical response (deterministic output)
  • Verify provenance section contains both policy versions and evaluation timestamp
  • POST with multiple artifact scopes; verify findings are ordered by ArtifactDigest (ordinal)

Verification

  • Run ID: run-001
  • Date: 2026-02-12
  • Result: PASS - 708/708 tests pass. ConsoleSimulationDiffServiceTests verifies determinism (JSON equality across repeated calls), schema version 'console-policy-23-001', Before/After severity totals, RuleImpact presence, budget enforcement (samples <= MaxFindings), provenance with evaluation timestamp.