stella-ops.org/git.stella-ops.org
1
0
Fork 0
Code Issues Pull Requests Actions 2 Releases Wiki Activity

save checkpoint

Browse Source
This commit is contained in:
master
2026-02-14 09:11:48 +02:00
parent 9ca2de05df
commit e9aeadc040
1512 changed files with 30863 additions and 4728 deletions
Download Patch File Download Diff File Expand all files Collapse all files

51
docs/qa/feature-checks/runs/policy/smart-diff-semantic-risk-delta/run-002/tier2-integration-check.json Normal file
View File

@@ -0,0 +1,51 @@
{
"type": "integration",
"capturedAtUtc": "2026-02-13T17:14:00Z",
"testCommand": "dotnet test src/Policy/__Tests/StellaOps.Policy.Engine.Tests/StellaOps.Policy.Engine.Tests.csproj --no-restore -v normal && dotnet test src/Policy/__Tests/StellaOps.Policy.Tests/StellaOps.Policy.Tests.csproj --no-restore -v normal",
"testFilter": "FullyQualifiedName~Simulation|FullyQualifiedName~ConsoleSimulationDiff|FullyQualifiedName~RiskSimulation|FullyQualifiedName~Counterfactual|FullyQualifiedName~DriftGate",
"testsRun": 2059,
"testsPassed": 2059,
"testsFailed": 0,
"targetedTestMethods": [
"ConsoleSimulationDiffServiceTests (deterministic diff, schema console-policy-23-001)",
"SimulationAnalyticsServiceTests (14: rule firing counts, heatmap, sampled traces, delta summary with SeverityChanges.Escalated)",
"RiskSimulationBreakdownServiceTests (19: signal analysis, score distribution, severity breakdown with HHI, component breakdown, determinism hash, CompareProfilesWithBreakdown)",
"PathScopeSimulationServiceTests (deterministic streaming by filePath)",
"PathScopeSimulationBridgeServiceTests (what-if deltas, overlay events)",
"CounterfactualEngineTests.ComputeAsync_AlreadyPassing_ReturnsNoPaths",
"CounterfactualEngineTests.ExceptionPath_EffortVariesBySeverity(Critical=5, High=4, Medium=3, Low=2)",
"CounterfactualEngineTests.VersionUpgradePath_UsesFixedVersionLookup",
"CounterfactualEngineTests.CompensatingControlPath_HasEffort4",
"CounterfactualEngineTests.DefaultOptions_IncludeExceptionAndCompensatingControl",
"CounterfactualEngineTests.CounterfactualResult_Blocked_SortsByEffort",
"PolicyEngineDeterminismTests (15: deterministic verdict hash, canonical JSON, input order independence)"
],
"behaviorVerified": [
"WhatIfSimulationService.SimulateAsync computes semantic risk delta between baseline and target",
"SBOM diff operations: add (new component with advisories), remove (cleared findings), upgrade (fixed CVEs), downgrade (introduces vulnerabilities)",
"Decision change types: status_changed, severity_changed, new, removed correctly detected",
"Impact summary: risk delta computed as increased/decreased/unchanged based on blocked/warning deltas",
"Recommendations generated based on delta analysis (risk increases/decreases/neutral)",
"ConsoleSimulationDiffService produces deterministic output (schema console-policy-23-001)",
"Before/After severity breakdowns computed correctly",
"Rule impact analysis identifies which policy rules drive the risk delta",
"CounterfactualEngine computes 5 fix path types: VEX, Exception, Reachability, VersionUpgrade, CompensatingControl",
"Effort ratings per path: Critical=5, High=4, Medium=3, Low=2 for Exception type",
"CompensatingControl always has effort=4",
"CounterfactualResult.Blocked sorts paths by effort ascending, lowest effort recommended",
"RiskSimulationBreakdownService.CompareProfilesWithBreakdown produces before/after profile delta",
"DriftGateEvaluator integrates SBOM drift detection as semantic risk signal",
"Deterministic: same baseline + target always produces same delta (verified via JSON equality and hash)"
],
"assertionTypes": [
"FluentAssertions Should().Be/Contain for simulation results",
"Assert.Equal for change type strings",
"Theory InlineData for parameterized effort by severity",
"JSON equality for deterministic output verification",
"SHA256 hash for determinism verification"
],
"newTestsWritten": [],
"bugsFixed": [],
"rawOutput": "Engine.Tests: Passed! - Failed: 0, Passed: 1278, Skipped: 0, Total: 1278, Duration: 8s 227ms. Policy.Tests: Passed! - Failed: 0, Passed: 781, Skipped: 0, Total: 781, Duration: 4s 601ms. Total: 2059/2059 pass. WhatIfSimulationService: SBOM diff operations (add/remove/upgrade/downgrade), decision changes (status_changed/severity_changed/new/removed), impact summary (increased/decreased/unchanged), recommendations. ConsoleSimulationDiffService: deterministic, schema console-policy-23-001, severity breakdowns, rule impact. CounterfactualEngine: 5 fix paths with effort scaling. RiskSimulationBreakdownService: signal analysis, score distribution, CompareProfilesWithBreakdown. DriftGateEvaluator: SBOM drift as semantic risk. PolicyEngineDeterminism: canonical JSON, verdict hash.",
"verdict": "pass"
}