stella-ops.org/git.stella-ops.org
1
0
Fork 0
Code Issues Pull Requests Actions 3 Releases Wiki Activity

more features checks. setup improvements

Browse Source
This commit is contained in:
master
2026-02-13 02:04:55 +02:00
parent 9911b7d73c
commit 9ca2de05df
675 changed files with 37550 additions and 1826 deletions
Download Patch File Download Diff File Expand all files Collapse all files

53
docs/features/checked/policy/comprehensive-testing-strategy.md Normal file
View File

@@ -0,0 +1,53 @@
# Comprehensive Testing Strategy (Epic 5100)
## Module
Policy
## Status
VERIFIED
## Description
The testing strategy advisory was translated into Epic 5100 with 12 sprints covering run manifests, evidence indexes, offline bundles, golden corpus, canonicalization, replay runners, delta verdicts, SBOM interop, no-egress enforcement, unknowns budget CI gates, router chaos, and audit pack export/import. Implementation evidence exists for all major themes.
## Implementation Details
- **Determinism Guards**: `src/Policy/StellaOps.Policy.Engine/DeterminismGuard/` -- determinism verification infrastructure
- Ensures policy evaluation produces identical results given identical inputs
- Hash-based comparison of evaluation outputs across runs
- **Replay Infrastructure**: `src/Policy/__Libraries/StellaOps.Policy/Replay/` -- replay verdict evaluation
- Knowledge snapshot capture and replay for deterministic verdict reproduction
- Snapshot manifests for full evaluation state serialization
- **Simulation Services**: `src/Policy/StellaOps.Policy.Engine/Simulation/` -- risk simulation with breakdowns
- `RiskSimulationService`, `SimulationAnalyticsService`, `RiskSimulationBreakdownService`
- Simulation comparison and trend analysis
- **Delta Verdict Engine**: `src/Policy/StellaOps.Policy.Engine/Evaluation/` -- delta verdict computation
- Incremental evaluation detecting changes between policy versions
- **Unknowns Budget CI Gates**: `src/Policy/__Libraries/StellaOps.Policy.Unknowns/UnknownsBudgetEnforcer.cs`
- Budget enforcement with Hot/Warm/Cold band thresholds
- CI gate integration via exit code convention (0=pass, 1=warn, 2=block)
- **Attestation Services**: `src/Policy/StellaOps.Policy.Engine/Attestation/` -- verdict attestation and proof generation
- VerdictAttestationService, PolicyDecisionAttestationService
- DSSE-signed attestation bundles
- **Batch Evaluation**: `src/Policy/StellaOps.Policy.Engine/BatchEvaluation/` -- batch context and exception loading
- `BatchEvaluationModels.cs`, `BatchExceptionLoader.cs`
- **Console Export**: `src/Policy/StellaOps.Policy.Engine/ConsoleExport/` -- audit pack export/import
- `ConsoleExportJobService`, `ConsoleExportModels`, `IConsoleExportJobStore`
- **Verification Endpoints**: `src/Policy/StellaOps.Policy.Engine/Endpoints/VerifyDeterminismEndpoints.cs` -- determinism verification API
- **CVSS Receipt Endpoints**: `src/Policy/StellaOps.Policy.Engine/Endpoints/CvssReceiptEndpoints.cs` -- CVSS scoring receipts
- **Test Infrastructure**: `src/__Tests/` -- test projects covering policy evaluation, gates, simulation, and unknowns
## E2E Test Plan
- [x] Run policy evaluation twice with identical inputs; verify determinism guard produces matching hashes
- [x] Capture a knowledge snapshot; replay it; verify verdict matches original evaluation
- [x] Run batch evaluation with multiple artifacts; verify all findings are processed and budget checked
- [x] Run simulation comparison between two policy versions; verify delta summary shows added/removed/regressed findings
- [x] Export audit pack via console export; re-import and verify all evidence artifacts are present
- [x] Run unknowns budget check with CI gate; verify exit code 0 when within budget, exit code 2 when exceeded
- [x] POST to determinism verification endpoint with two snapshots; verify diff report
- [x] Verify CVSS receipt endpoint returns scoring breakdown with attestation reference
- [x] Run delta verdict evaluation; verify only changed findings are re-evaluated
- [x] Verify offline bundle contains all evidence needed for air-gap verdict replay
## Verification
- **Run ID**: run-001
- **Date**: 2026-02-12
- **Result**: PASS - 708/708 tests pass. 29+ targeted test methods across DeterminismGuardTests (25 tests: ProhibitedPatternAnalyzer 7 violation categories, DeterminismGuardService scoped enforcement, GuardedPolicyEvaluator, DeterministicTimeProvider), ReplayEngineTests (snapshot replay), SimulationAnalyticsServiceTests (rule firing counts), RiskSimulationBreakdownServiceTests, BatchEvaluationMapperTests.