save checkpoint
This commit is contained in:
39
docs/features/checked/policy/prohibitedpatternanalyzer.md
Normal file
39
docs/features/checked/policy/prohibitedpatternanalyzer.md
Normal file
@@ -0,0 +1,39 @@
|
||||
# ProhibitedPatternAnalyzer (Static Purity Analysis)
|
||||
|
||||
## Module
|
||||
Policy
|
||||
|
||||
## Status
|
||||
IMPLEMENTED
|
||||
|
||||
## Description
|
||||
Static purity analysis detecting prohibited patterns (ambient IO, clock access, etc.) in evaluation code.
|
||||
|
||||
## Implementation Details
|
||||
- **ProhibitedPatternAnalyzer**: `src/Policy/StellaOps.Policy.Engine/DeterminismGuard/ProhibitedPatternAnalyzer.cs`
|
||||
- Regex-based detection of non-deterministic patterns in source code
|
||||
- Prohibited pattern categories:
|
||||
- Wall-clock access: `DateTime.Now`, `DateTime.UtcNow`, `DateTimeOffset.Now`, `DateTimeOffset.UtcNow`
|
||||
- Random number generation: `Random`, `RandomNumberGenerator`
|
||||
- Network access: `HttpClient`, `WebRequest`, `TcpClient`, `UdpClient`
|
||||
- Filesystem access: `File.`, `Directory.`, `Path.GetTempPath`
|
||||
- Line-by-line scanning with comment line skipping (lines starting with `//` or `///`)
|
||||
- Returns list of `ProhibitedPatternMatch` with line number, pattern name, matched text
|
||||
- **DeterminismGuardService**: `src/Policy/StellaOps.Policy.Engine/DeterminismGuard/DeterminismGuardService.cs`
|
||||
- `AnalyzeSource(sourceCode)` invokes ProhibitedPatternAnalyzer to find violations
|
||||
- `CreateScope()` creates a determinism guard scope for runtime monitoring
|
||||
- `ValidateContext<T>()` validates evaluation context for determinism
|
||||
- Combines ProhibitedPatternAnalyzer (static) and RuntimeDeterminismMonitor (runtime)
|
||||
- **RuntimeDeterminismMonitor**: `src/Policy/StellaOps.Policy.Engine/DeterminismGuard/RuntimeDeterminismMonitor.cs` -- runtime monitoring companion
|
||||
- **GuardedPolicyEvaluator**: `src/Policy/StellaOps.Policy.Engine/Gates/Determinization/DeterminizationGate.cs` -- gate that uses determinism guards in evaluation pipeline
|
||||
|
||||
## E2E Test Plan
|
||||
- [ ] Analyze source code containing `DateTime.Now`; verify prohibited pattern detected with correct line number
|
||||
- [ ] Analyze source code containing `new Random()`; verify prohibited pattern detected
|
||||
- [ ] Analyze source code containing `HttpClient`; verify network access pattern detected
|
||||
- [ ] Analyze source code containing `File.ReadAllText`; verify filesystem pattern detected
|
||||
- [ ] Analyze source code with prohibited pattern in a comment line (`// DateTime.Now`); verify NOT detected (comment skipped)
|
||||
- [ ] Analyze clean source code with no prohibited patterns; verify empty results
|
||||
- [ ] Analyze source code with multiple violations on different lines; verify all detected with correct line numbers
|
||||
- [ ] Verify DeterminismGuardService.AnalyzeSource returns results from ProhibitedPatternAnalyzer
|
||||
- [ ] Create determinism guard scope; use TimeProvider instead of DateTime.Now; verify no violations
|
||||
@@ -0,0 +1,42 @@
|
||||
# Proof Replay / Deterministic Verdict Replay
|
||||
|
||||
## Module
|
||||
Policy
|
||||
|
||||
## Status
|
||||
IMPLEMENTED
|
||||
|
||||
## Description
|
||||
Full replay service with a dedicated module, determinism verifier, run manifests, and extensive E2E tests that verify byte-identical verdict replay across runs.
|
||||
|
||||
## Implementation Details
|
||||
- **ReplayEngine**: `src/Policy/__Libraries/StellaOps.Policy/Replay/ReplayEngine.cs` (sealed class implements `IReplayEngine`)
|
||||
- `ReplayAsync(ReplayRequest)` replays policy evaluation with frozen snapshot inputs
|
||||
- Pipeline: load snapshot -> resolve frozen inputs -> execute with frozen inputs -> compare with original -> generate delta report
|
||||
- Uses `ISnapshotService` for snapshot loading, `IKnowledgeSourceResolver` for input resolution, `IVerdictComparer` for comparison
|
||||
- Returns `ReplayResult` with MatchStatus, ReplayedVerdict, OriginalVerdict, DeltaReport, Duration
|
||||
- **ReplayRequest**: `src/Policy/__Libraries/StellaOps.Policy/Replay/ReplayRequest.cs` (sealed record)
|
||||
- ArtifactDigest, SnapshotId, OriginalVerdictId (optional, for comparison)
|
||||
- ReplayOptions: CompareWithOriginal (default true), AllowNetworkFetch (default false), GenerateDetailedReport (default true), ScoreTolerance (default 0.001)
|
||||
- **ReplayResult**: `src/Policy/__Libraries/StellaOps.Policy/Replay/ReplayResult.cs` (sealed record)
|
||||
- ReplayMatchStatus: ExactMatch, MatchWithinTolerance, Mismatch, NoComparison, ReplayFailed
|
||||
- ReplayedVerdict: VerdictId, ArtifactDigest, Decision (Pass/Fail/PassWithExceptions/Indeterminate), Score, FindingIds, KnowledgeSnapshotId
|
||||
- ReplayDeltaReport: Summary, FieldDeltas (FieldName, OriginalValue, ReplayedValue), FindingDeltas (FindingId, DeltaType, Description), SuspectedCauses
|
||||
- **VerdictComparer**: `src/Policy/__Libraries/StellaOps.Policy/Replay/VerdictComparer.cs` -- compares replayed vs original verdicts with tolerance
|
||||
- **ReplayReport**: `src/Policy/__Libraries/StellaOps.Policy/Replay/ReplayReport.cs` -- detailed replay report generation
|
||||
- **KnowledgeSourceResolver**: `src/Policy/__Libraries/StellaOps.Policy/Replay/KnowledgeSourceResolver.cs` -- resolves snapshot sources to frozen data
|
||||
- **KnowledgeSnapshotManifest**: `src/Policy/__Libraries/StellaOps.Policy/Snapshots/KnowledgeSnapshotManifest.cs` -- content-addressed snapshot used as replay input
|
||||
- **SnapshotAwarePolicyEvaluator**: `src/Policy/__Libraries/StellaOps.Policy/Snapshots/SnapshotAwarePolicyEvaluator.cs` -- evaluates using pinned snapshot inputs
|
||||
|
||||
## E2E Test Plan
|
||||
- [ ] Replay verdict with same snapshot; verify MatchStatus=ExactMatch (deterministic evaluation)
|
||||
- [ ] Replay verdict with OriginalVerdictId; verify OriginalVerdict is loaded and compared
|
||||
- [ ] Replay with CompareWithOriginal=false; verify MatchStatus=NoComparison, no OriginalVerdict
|
||||
- [ ] Replay with missing snapshot; verify MatchStatus=ReplayFailed, DeltaReport contains error
|
||||
- [ ] Replay with incomplete snapshot (missing source); verify ReplayFailed with missing source names
|
||||
- [ ] Replay with score difference within ScoreTolerance=0.001; verify MatchStatus=MatchWithinTolerance
|
||||
- [ ] Replay with score difference exceeding tolerance; verify MatchStatus=Mismatch with FieldDeltas
|
||||
- [ ] Replay with finding difference; verify DeltaReport.FindingDeltas contains Added/Removed/Modified entries
|
||||
- [ ] Replay with GenerateDetailedReport=true; verify DeltaReport.SuspectedCauses is populated
|
||||
- [ ] Verify AllowNetworkFetch=false prevents network access for missing sources
|
||||
- [ ] Verify replay Duration is recorded in result
|
||||
63
docs/features/checked/policy/proof-studio-ux.md
Normal file
63
docs/features/checked/policy/proof-studio-ux.md
Normal file
@@ -0,0 +1,63 @@
|
||||
# Proof Studio UX (Explainable Confidence Scoring)
|
||||
|
||||
## Module
|
||||
Policy
|
||||
|
||||
## Status
|
||||
IMPLEMENTED
|
||||
|
||||
## Description
|
||||
Backend confidence calculation, verdict rationale rendering, and counterfactual engine exist. The advisory identified frontend proof studio UI as a remaining gap.
|
||||
|
||||
## What's Implemented
|
||||
- **VerdictRationaleRenderer**: `src/Policy/__Libraries/StellaOps.Policy.Explainability/VerdictRationaleRenderer.cs`
|
||||
- 4-line rationale template: Evidence, Policy Clause, Attestations, Decision
|
||||
- Multi-format output: RenderPlainText, RenderMarkdown, RenderJson (RFC 8785 canonical)
|
||||
- Content-addressed RationaleId: `rat:sha256:{hash}`
|
||||
- **VerdictRationale model**: `src/Policy/__Libraries/StellaOps.Policy.Explainability/VerdictRationale.cs`
|
||||
- Structured rationale: VerdictReference, RationaleEvidence (ComponentIdentity, ReachabilityDetail), RationalePolicyClause, RationaleAttestations (AttestationReference list), RationaleDecision (MitigationGuidance)
|
||||
- RationaleInputDigests for deterministic replay
|
||||
- **IVerdictRationaleRenderer**: `src/Policy/__Libraries/StellaOps.Policy.Explainability/IVerdictRationaleRenderer.cs`
|
||||
- Interface + VerdictRationaleInput record
|
||||
- **CounterfactualEngine**: `src/Policy/__Libraries/StellaOps.Policy/Counterfactuals/CounterfactualEngine.cs`
|
||||
- `ComputeAsync(finding, verdict, document, scoringConfig, options)` -> CounterfactualResult
|
||||
- 5 counterfactual path types: VexStatus, Exception, Reachability, VersionUpgrade, CompensatingControl
|
||||
- Each path: type, description, conditions (field/current/required), estimated effort (1-5), actor, action URI
|
||||
- Policy simulation: creates simulated findings with modified VEX/reachability tags and re-evaluates via `PolicyEvaluation.EvaluateFinding()`
|
||||
- Effort estimation: severity-based for exceptions (Critical=5, High=4, Medium=3, Low=2)
|
||||
- **CounterfactualResult**: `src/Policy/__Libraries/StellaOps.Policy/Counterfactuals/CounterfactualResult.cs`
|
||||
- AlreadyPassing / Blocked factory methods
|
||||
- RecommendedPath (lowest effort), HasPaths
|
||||
- 7 CounterfactualTypes: VexStatus, Exception, Reachability, VersionUpgrade, PolicyChange, ComponentRemoval, CompensatingControl
|
||||
- CounterfactualCondition: Field, CurrentValue, RequiredValue, IsMet
|
||||
- **ScoreExplanation**: `src/Policy/__Libraries/StellaOps.Policy/Scoring/ScoreExplanation.cs`
|
||||
- Per-factor explanation: Factor, Value (0-100), Reason, ContributingDigests
|
||||
- ScoreExplainBuilder: AddReachability, AddEvidence, AddProvenance, AddBaseSeverity
|
||||
- Deterministic output (sorted by factor name + digest)
|
||||
- **DecayedConfidenceCalculator**: `src/Policy/__Libraries/StellaOps.Policy.Determinization/Scoring/DecayedConfidenceCalculator.cs`
|
||||
- Exponential confidence decay for time-based scoring
|
||||
|
||||
## What's Missing
|
||||
- **Proof graph visualization**: No visual representation of the full evidence graph (ProofGraphNode/Edge/Path) in the UI -- the proof-studio has confidence breakdown but not the graph
|
||||
- **Interactive counterfactual explorer**: CounterfactualEngine exists in backend and `what-if-slider` component exists in proof-studio, but the full interactive "toggle what-if scenarios" UX may not be fully wired to the backend
|
||||
- **Score breakdown dashboard**: ScoreExplanation data exists but no dashboard visualizing per-factor contributions with charts
|
||||
- **Confidence timeline**: DecayedConfidenceCalculator computes decay but no UI showing confidence over time
|
||||
|
||||
## Additional Implementation Found
|
||||
- **Proof Studio Container**: `src/Web/StellaOps.Web/src/app/features/proof-studio/components/proof-studio-container/proof-studio-container.component.ts` -- main container component
|
||||
- **Confidence Breakdown**: `src/Web/StellaOps.Web/src/app/features/proof-studio/components/confidence-breakdown/confidence-breakdown.component.ts` -- per-factor confidence visualization
|
||||
- **Confidence Factor Chip**: `src/Web/StellaOps.Web/src/app/features/proof-studio/components/confidence-factor-chip/confidence-factor-chip.component.ts`
|
||||
- **What-If Slider**: `src/Web/StellaOps.Web/src/app/features/proof-studio/components/what-if-slider/what-if-slider.component.ts` -- counterfactual slider control
|
||||
- **Proof Studio Service**: `src/Web/StellaOps.Web/src/app/features/proof-studio/services/proof-studio.service.ts` -- API service
|
||||
- **Proof Trace Model**: `src/Web/StellaOps.Web/src/app/features/proof-studio/models/proof-trace.model.ts`
|
||||
|
||||
## Implementation Plan
|
||||
- Wire what-if-slider to CounterfactualEngine backend API
|
||||
- Add proof graph visualization using D3.js or similar for evidence graph rendering
|
||||
- Add confidence timeline chart using DecayedConfidenceCalculator data
|
||||
- Verify proof-studio-container is fully wired to VerdictRationale API endpoint
|
||||
|
||||
## Related Documentation
|
||||
- Explainability library: `src/Policy/__Libraries/StellaOps.Policy.Explainability/`
|
||||
- Counterfactuals: `src/Policy/__Libraries/StellaOps.Policy/Counterfactuals/`
|
||||
- Score explanations: `src/Policy/__Libraries/StellaOps.Policy/Scoring/ScoreExplanation.cs`
|
||||
39
docs/features/checked/policy/property-based-tests.md
Normal file
39
docs/features/checked/policy/property-based-tests.md
Normal file
@@ -0,0 +1,39 @@
|
||||
# Property-Based Tests (FsCheck)
|
||||
|
||||
## Module
|
||||
Policy
|
||||
|
||||
## Status
|
||||
IMPLEMENTED
|
||||
|
||||
## Description
|
||||
Property-based tests using FsCheck for canonical JSON determinism, SBOM/VEX ordering invariants, floating-point stability, digest computation determinism, smart-diff properties, and VEX lattice merge commutativity.
|
||||
|
||||
## Implementation Details
|
||||
- **DecayPropertyTests**: `src/Policy/__Tests/StellaOps.Policy.Determinization.Tests/PropertyTests/DecayPropertyTests.cs`
|
||||
- FsCheck properties for DecayedConfidenceCalculator: monotonicity, bounds, floor enforcement, half-life correctness
|
||||
- Verifies: decay factor always in [0, 1], result always >= floor, older observations have lower scores
|
||||
- **DeterminismPropertyTests**: `src/Policy/__Tests/StellaOps.Policy.Determinization.Tests/PropertyTests/DeterminismPropertyTests.cs`
|
||||
- FsCheck properties for deterministic evaluation: same inputs produce same outputs
|
||||
- Verifies: ObservationDecay CalculateDecay is deterministic, content-addressed ID generation is stable
|
||||
- **ObservationDecayTests**: `src/Policy/__Tests/StellaOps.Policy.Determinization.Tests/Models/ObservationDecayTests.cs`
|
||||
- Unit tests for ObservationDecay model: Create, Fresh, WithSettings, CalculateDecay, CheckIsStale
|
||||
- **DeterminizationResultTests**: `src/Policy/__Tests/StellaOps.Policy.Determinization.Tests/Models/DeterminizationResultTests.cs`
|
||||
- Unit tests for DeterminizationResult model
|
||||
- **DecayedConfidenceCalculatorTests**: `src/Policy/__Tests/StellaOps.Policy.Determinization.Tests/DecayedConfidenceCalculatorTests.cs`
|
||||
- Unit tests for DecayedConfidenceCalculator: boundary values, half-life verification, floor enforcement
|
||||
- **DeterminizationGateTests**: `src/Policy/__Tests/StellaOps.Policy.Engine.Tests/Gates/Determinization/DeterminizationGateTests.cs`
|
||||
- Integration tests for DeterminizationGate in gate pipeline
|
||||
- **DeterminizationPolicyTests**: `src/Policy/__Tests/StellaOps.Policy.Engine.Tests/Policies/DeterminizationPolicyTests.cs`
|
||||
- Policy-level tests for determinization enforcement
|
||||
|
||||
## E2E Test Plan
|
||||
- [ ] Run DecayPropertyTests; verify all FsCheck properties pass (monotonicity, bounds, floor, half-life)
|
||||
- [ ] Run DeterminismPropertyTests; verify deterministic evaluation property holds for 1000+ random inputs
|
||||
- [ ] Verify DecayedConfidenceCalculator boundary: baseConfidence=0.0 always returns floor
|
||||
- [ ] Verify DecayedConfidenceCalculator boundary: ageDays=0 always returns baseConfidence
|
||||
- [ ] Verify ObservationDecay.Fresh(now).CheckIsStale(now) == false
|
||||
- [ ] Verify decay monotonicity: older age always produces lower or equal score
|
||||
- [ ] Verify content-addressed ID generation produces same ID for identical inputs (1000 iterations)
|
||||
- [ ] Verify DeterminizationGate integration with DecayedConfidenceCalculator in gate pipeline
|
||||
- [ ] Verify all test suites pass in CI with frozen test clock (no time-dependent flakiness)
|
||||
39
docs/features/checked/policy/release-gate-levels.md
Normal file
39
docs/features/checked/policy/release-gate-levels.md
Normal file
@@ -0,0 +1,39 @@
|
||||
# Release Gate Levels (G0-G4)
|
||||
|
||||
## Module
|
||||
Policy
|
||||
|
||||
## Status
|
||||
IMPLEMENTED
|
||||
|
||||
## Description
|
||||
Five gate levels (G0 through G4) with escalating requirements. GateSelector computes RRS, maps to gate level, and applies budget modifiers (Yellow/Red/Exhausted escalations). Each gate level has defined requirements matching the advisory specification.
|
||||
|
||||
## Implementation Details
|
||||
- **PolicyGateEvaluator**: `src/Policy/StellaOps.Policy.Engine/Gates/PolicyGateEvaluator.cs`
|
||||
- Multi-gate evaluation with escalating strictness per gate level
|
||||
- Gate levels mapped from Risk Readiness Score (RRS) and budget status
|
||||
- 5 sequential gates with configurable thresholds per level
|
||||
- Lattice states drive gate decisions: U, SR, SU, RO, RU, CR, CU, X
|
||||
- Uncertainty tiers: T1 (High) -> strictest, T4 (Negligible) -> most permissive
|
||||
- **PolicyGateOptions**: `src/Policy/StellaOps.Policy.Engine/Gates/PolicyGateOptions.cs` -- per-level gate configuration
|
||||
- **PolicyGateDecision**: `src/Policy/StellaOps.Policy.Engine/Gates/PolicyGateDecision.cs` -- decision model with per-gate results
|
||||
- **RiskSimulationService**: `src/Policy/StellaOps.Policy.Engine/Simulation/RiskSimulationService.cs`
|
||||
- Signal-based RRS computation with severity mapping
|
||||
- Severity: Critical>=90, High>=70, Medium>=40, Low>=10
|
||||
- Distribution and percentile calculation for gate level determination
|
||||
- **Budget integration**: Budget status (Yellow/Red/Exhausted) escalates gate level
|
||||
- `BudgetEndpoints.cs`: `src/Policy/StellaOps.Policy.Engine/Endpoints/BudgetEndpoints.cs`
|
||||
- `RiskBudgetEndpoints.cs`: `src/Policy/StellaOps.Policy.Engine/Endpoints/RiskBudgetEndpoints.cs`
|
||||
- **Ledger**: `src/Policy/StellaOps.Policy.Engine/Ledger/` -- tracks gate decisions for compliance
|
||||
|
||||
## E2E Test Plan
|
||||
- [ ] Evaluate artifact with low RRS (minimal risk); verify gate level maps to G0 (minimal requirements)
|
||||
- [ ] Evaluate artifact with high RRS (many critical findings); verify gate level escalates to G3 or G4
|
||||
- [ ] Evaluate with budget status Yellow; verify gate level escalates by one level
|
||||
- [ ] Evaluate with budget status Exhausted; verify gate level escalates to maximum (G4)
|
||||
- [ ] Evaluate at G0: verify only basic evidence completeness is checked
|
||||
- [ ] Evaluate at G4: verify all gates apply strictest thresholds (lattice state, VEX trust, uncertainty, confidence)
|
||||
- [ ] Verify each gate level (G0-G4) has progressively stricter thresholds
|
||||
- [ ] Verify gate decision includes per-gate Pass/Warn/Block results for audit trail
|
||||
- [ ] Verify ledger records gate level and decision for compliance
|
||||
@@ -0,0 +1,40 @@
|
||||
# Replayable Verdict Evaluation
|
||||
|
||||
## Module
|
||||
Policy
|
||||
|
||||
## Status
|
||||
IMPLEMENTED
|
||||
|
||||
## Description
|
||||
Full replay engine that re-evaluates verdicts using stored snapshot inputs, producing match/mismatch reports with delta explanations when results differ. Exposed via API endpoints.
|
||||
|
||||
## Implementation Details
|
||||
- **ReplayEngine**: `src/Policy/__Libraries/StellaOps.Policy/Replay/ReplayEngine.cs` (sealed class implements `IReplayEngine`)
|
||||
- `ReplayAsync(ReplayRequest)` -- full replay pipeline: load snapshot, resolve inputs, execute, compare, report
|
||||
- Uses frozen inputs from KnowledgeSnapshotManifest to ensure deterministic re-evaluation
|
||||
- Comparison with VerdictComparer produces ExactMatch, MatchWithinTolerance, or Mismatch
|
||||
- Delta report with FieldDeltas, FindingDeltas, and SuspectedCauses
|
||||
- **ReplayRequest**: `src/Policy/__Libraries/StellaOps.Policy/Replay/ReplayRequest.cs`
|
||||
- ArtifactDigest, SnapshotId, OriginalVerdictId
|
||||
- ReplayOptions: CompareWithOriginal, AllowNetworkFetch=false, GenerateDetailedReport, ScoreTolerance=0.001
|
||||
- **ReplayResult**: `src/Policy/__Libraries/StellaOps.Policy/Replay/ReplayResult.cs`
|
||||
- MatchStatus: ExactMatch, MatchWithinTolerance, Mismatch, NoComparison, ReplayFailed
|
||||
- ReplayedVerdict with Decision (Pass/Fail/PassWithExceptions/Indeterminate), Score, FindingIds
|
||||
- Duration tracking for performance monitoring
|
||||
- **VerdictComparer**: `src/Policy/__Libraries/StellaOps.Policy/Replay/VerdictComparer.cs` -- deterministic comparison with configurable tolerance
|
||||
- **KnowledgeSourceResolver**: `src/Policy/__Libraries/StellaOps.Policy/Replay/KnowledgeSourceResolver.cs` -- resolves snapshot source descriptors to data
|
||||
- **SnapshotAwarePolicyEvaluator**: `src/Policy/__Libraries/StellaOps.Policy/Snapshots/SnapshotAwarePolicyEvaluator.cs` -- evaluation with pinned inputs
|
||||
- **KnowledgeSnapshotManifest**: `src/Policy/__Libraries/StellaOps.Policy/Snapshots/KnowledgeSnapshotManifest.cs` -- content-addressed snapshot manifest
|
||||
|
||||
## E2E Test Plan
|
||||
- [ ] Replay verdict with same snapshot and inputs; verify ExactMatch status
|
||||
- [ ] Replay verdict after source data changed; verify Mismatch with SuspectedCauses listing changed source
|
||||
- [ ] Replay with ScoreTolerance=0.01; introduce 0.005 score drift; verify MatchWithinTolerance
|
||||
- [ ] Replay with ScoreTolerance=0.001; introduce 0.005 score drift; verify Mismatch with FieldDelta showing score difference
|
||||
- [ ] Verify ReplayDeltaReport.FindingDeltas lists Added findings (present in replay, absent in original)
|
||||
- [ ] Verify ReplayDeltaReport.FindingDeltas lists Removed findings (absent in replay, present in original)
|
||||
- [ ] Replay with AllowNetworkFetch=false and missing bundled source; verify ReplayFailed
|
||||
- [ ] Replay with CompareWithOriginal=false; verify NoComparison status, no DeltaReport
|
||||
- [ ] Verify replay Duration is non-zero and reasonable
|
||||
- [ ] POST replay endpoint; verify JSON response includes all ReplayResult fields
|
||||
34
docs/features/checked/policy/risk-budget-api-endpoints.md
Normal file
34
docs/features/checked/policy/risk-budget-api-endpoints.md
Normal file
@@ -0,0 +1,34 @@
|
||||
# Risk Budget API Endpoints
|
||||
|
||||
## Module
|
||||
Policy
|
||||
|
||||
## Status
|
||||
IMPLEMENTED
|
||||
|
||||
## Description
|
||||
API endpoints for risk budget management and enforcement with integration-level testing of budget enforcement.
|
||||
|
||||
## Implementation Details
|
||||
- **BudgetEndpoints**: `src/Policy/StellaOps.Policy.Engine/Endpoints/BudgetEndpoints.cs` -- CRUD endpoints for budget management
|
||||
- **RiskBudgetEndpoints**: `src/Policy/StellaOps.Policy.Engine/Endpoints/RiskBudgetEndpoints.cs` -- risk budget evaluation and status endpoints
|
||||
- **RiskProfileEndpoints**: `src/Policy/StellaOps.Policy.Engine/Endpoints/RiskProfileEndpoints.cs` -- risk profile configuration endpoints
|
||||
- **RiskProfileSchemaEndpoints**: `src/Policy/StellaOps.Policy.Engine/Endpoints/RiskProfileSchemaEndpoints.cs` -- schema validation for risk profiles
|
||||
- **RiskProfileAirGapEndpoints**: `src/Policy/StellaOps.Policy.Engine/Endpoints/RiskProfileAirGapEndpoints.cs` -- air-gap compatible risk profile endpoints
|
||||
- **LedgerExportService**: `src/Policy/StellaOps.Policy.Engine/Ledger/LedgerExportService.cs` -- budget ledger export for compliance
|
||||
- **LedgerModels**: `src/Policy/StellaOps.Policy.Engine/Ledger/LedgerModels.cs` -- ledger data models
|
||||
- **LedgerExportStore**: `src/Policy/StellaOps.Policy.Engine/Ledger/LedgerExportStore.cs` -- persistence for ledger exports
|
||||
- **UnknownBudgetService** / **UnknownsBudgetEnforcer**: `src/Policy/__Libraries/StellaOps.Policy.Unknowns/` -- budget enforcement for unknowns
|
||||
- **PolicyGateEvaluator**: `src/Policy/StellaOps.Policy.Engine/Gates/PolicyGateEvaluator.cs` -- budget status affects gate level selection
|
||||
|
||||
## E2E Test Plan
|
||||
- [ ] GET budget status endpoint; verify response includes current consumption, limits, and status (Green/Yellow/Red/Exhausted)
|
||||
- [ ] POST create budget with critical/high/medium limits; verify budget created with correct thresholds
|
||||
- [ ] POST evaluate risk budget for artifact; verify consumption is calculated and compared against limits
|
||||
- [ ] Consume budget beyond Yellow threshold; verify status changes to Yellow
|
||||
- [ ] Consume budget beyond Red threshold; verify status changes to Red
|
||||
- [ ] Consume budget beyond limit; verify status changes to Exhausted and gate level escalates
|
||||
- [ ] GET risk profile endpoint; verify profile includes budget configuration and scoring weights
|
||||
- [ ] POST risk profile schema validation; verify invalid profile returns validation errors
|
||||
- [ ] GET ledger export; verify budget transactions are exported with timestamps and actor IDs
|
||||
- [ ] GET air-gap risk profile endpoint; verify offline-compatible response without external dependencies
|
||||
33
docs/features/checked/policy/risk-budget-management.md
Normal file
33
docs/features/checked/policy/risk-budget-management.md
Normal file
@@ -0,0 +1,33 @@
|
||||
# Risk Budget Management
|
||||
|
||||
## Module
|
||||
Policy
|
||||
|
||||
## Status
|
||||
IMPLEMENTED
|
||||
|
||||
## Description
|
||||
Per-service risk budget management with budget ledger (RP consumed per release, remaining, trendline), constraint enforcement, threshold notifications, and earned capacity replenishment.
|
||||
|
||||
## Implementation Details
|
||||
- **BudgetEndpoints**: `src/Policy/StellaOps.Policy.Engine/Endpoints/BudgetEndpoints.cs` -- CRUD API for budget definitions and status queries
|
||||
- **RiskBudgetEndpoints**: `src/Policy/StellaOps.Policy.Engine/Endpoints/RiskBudgetEndpoints.cs` -- risk budget evaluation, consumption tracking, and status
|
||||
- **LedgerExportService**: `src/Policy/StellaOps.Policy.Engine/Ledger/LedgerExportService.cs` -- budget ledger tracking RP consumed per release, remaining capacity, trendline
|
||||
- **LedgerModels**: `src/Policy/StellaOps.Policy.Engine/Ledger/LedgerModels.cs` -- data models for ledger entries (release ID, RP consumed, timestamp, actor)
|
||||
- **LedgerExportStore**: `src/Policy/StellaOps.Policy.Engine/Ledger/LedgerExportStore.cs` -- persistence for ledger export
|
||||
- **UnknownBudgetService**: `src/Policy/__Libraries/StellaOps.Policy.Unknowns/Services/UnknownBudgetService.cs` -- budget management for unknowns
|
||||
- **UnknownsBudgetEnforcer**: `src/Policy/__Libraries/StellaOps.Policy.Unknowns/Services/UnknownsBudgetEnforcer.cs` -- constraint enforcement (Green/Yellow/Red/Exhausted thresholds)
|
||||
- **PolicyGateEvaluator budget integration**: `src/Policy/StellaOps.Policy.Engine/Gates/PolicyGateEvaluator.cs` -- budget status (Yellow/Red/Exhausted) escalates gate levels
|
||||
- **RiskSimulationService**: `src/Policy/StellaOps.Policy.Engine/Simulation/RiskSimulationService.cs` -- simulates budget impact of policy changes
|
||||
|
||||
## E2E Test Plan
|
||||
- [ ] Create budget with critical=10, high=20, medium=50 limits; verify budget status is Green
|
||||
- [ ] Consume 15 critical RP; verify budget status transitions to Yellow for critical
|
||||
- [ ] Consume 8 more critical RP (total 23 > limit 10); verify budget status transitions to Red/Exhausted
|
||||
- [ ] Verify ledger records each consumption with release ID, RP amount, timestamp, and actor
|
||||
- [ ] Query budget trendline; verify declining remaining capacity across releases
|
||||
- [ ] Trigger threshold notification at Yellow; verify notification includes budget name, threshold, and current consumption
|
||||
- [ ] Verify earned capacity replenishment: resolve findings to restore budget capacity
|
||||
- [ ] Export ledger; verify all entries are included with compliance-ready format
|
||||
- [ ] Verify budget constraint enforcement blocks release when Exhausted
|
||||
- [ ] Verify gate level escalation when budget is in Red status
|
||||
41
docs/features/checked/policy/risk-budget-model.md
Normal file
41
docs/features/checked/policy/risk-budget-model.md
Normal file
@@ -0,0 +1,41 @@
|
||||
# Risk Budget Model (Service Tiers + Risk Points)
|
||||
|
||||
## Module
|
||||
Policy
|
||||
|
||||
## Status
|
||||
IMPLEMENTED
|
||||
|
||||
## Description
|
||||
Complete risk budget system with service tier-based scoring, risk point computation, budget ledger tracking, constraint enforcement, threshold notifications, capacity replenishment, and persistence. Includes API endpoints and property-based tests for monotonicity.
|
||||
|
||||
## Implementation Details
|
||||
- **RiskSimulationService**: `src/Policy/StellaOps.Policy.Engine/Simulation/RiskSimulationService.cs`
|
||||
- Signal-based risk point computation: Boolean (0/1), Numeric (direct), Categorical (mapped weight)
|
||||
- Severity mapping: Critical>=90, High>=70, Medium>=40, Low>=10
|
||||
- Distribution calculation with 10 buckets, 6 percentiles (p25/p50/p75/p90/p95/p99)
|
||||
- Aggregate metrics: total, mean, median, stddev, severity breakdown
|
||||
- **UnknownRanker**: `src/Policy/__Libraries/StellaOps.Policy.Unknowns/Services/UnknownRanker.cs`
|
||||
- Two-factor risk score: Score = (Uncertainty * 50) + (ExploitPressure * 50)
|
||||
- Band assignment: Hot >= 75, Warm >= 50, Cold >= 25, Negligible < 25
|
||||
- Containment reduction capped at 40% (Seccomp 10%, FsRO 10%, Isolated 15%, etc.)
|
||||
- Decay buckets for time-based scoring adjustments
|
||||
- **UnknownBudgetService** / **UnknownsBudgetEnforcer**: `src/Policy/__Libraries/StellaOps.Policy.Unknowns/`
|
||||
- Per-service budget constraints with Green/Yellow/Red/Exhausted thresholds
|
||||
- Budget enforcement blocks releases when Exhausted
|
||||
- **LedgerExportService**: `src/Policy/StellaOps.Policy.Engine/Ledger/LedgerExportService.cs` -- budget ledger persistence
|
||||
- **BudgetEndpoints** / **RiskBudgetEndpoints**: `src/Policy/StellaOps.Policy.Engine/Endpoints/` -- API for budget CRUD and evaluation
|
||||
- **Scoring engines**: `src/Policy/StellaOps.Policy.Engine/Scoring/Engines/` -- SimpleScoringEngine, AdvancedScoringEngine, ProofAwareScoringEngine
|
||||
- **ScoringProfileService**: `src/Policy/StellaOps.Policy.Engine/Scoring/ScoringProfileService.cs` -- configurable scoring profiles with weighted signals
|
||||
|
||||
## E2E Test Plan
|
||||
- [ ] Compute risk points for finding with Critical severity (CVSS>=9.0, EPSS>=0.90); verify score >= 90
|
||||
- [ ] Compute risk points for finding with Low severity (CVSS=3.0); verify score maps to Low range (10-39)
|
||||
- [ ] Compute risk points with containment reduction (Seccomp + FsRO); verify 20% reduction applied
|
||||
- [ ] Verify score monotonicity: higher CVSS/EPSS always produces higher or equal score
|
||||
- [ ] Compute aggregate risk for 10 findings; verify distribution has 10 buckets, valid percentiles
|
||||
- [ ] Verify severity breakdown: Critical count + High count + Medium count + Low count = total findings
|
||||
- [ ] Create budget with tier-based limits; consume RP; verify threshold transitions (Green -> Yellow -> Red -> Exhausted)
|
||||
- [ ] Verify budget capacity replenishment: resolve 5 findings; verify remaining capacity increases
|
||||
- [ ] Verify ProofAwareScoringEngine includes proof references in scored output
|
||||
- [ ] Verify scoring profile weights CVSS, EPSS, reachability contributions correctly
|
||||
42
docs/features/checked/policy/risk-point-scoring.md
Normal file
42
docs/features/checked/policy/risk-point-scoring.md
Normal file
@@ -0,0 +1,42 @@
|
||||
# Risk Point Scoring
|
||||
|
||||
## Module
|
||||
Policy
|
||||
|
||||
## Status
|
||||
IMPLEMENTED
|
||||
|
||||
## Description
|
||||
Risk Point (RP) scoring model computing Release Risk Score from base criticality, diff risk, operational context, and mitigations with monotonicity guarantees.
|
||||
|
||||
## Implementation Details
|
||||
- **RiskSimulationService**: `src/Policy/StellaOps.Policy.Engine/Simulation/RiskSimulationService.cs`
|
||||
- `Simulate(items)` computes aggregate risk scores
|
||||
- `SimulateWithBreakdown(items)` returns full distribution, severity breakdown, top movers
|
||||
- Signal-based scoring: Boolean (0/1), Numeric (direct value), Categorical (mapped weight)
|
||||
- Severity mapping: Critical>=90, High>=70, Medium>=40, Low>=10
|
||||
- Normalization ensures scores in valid range
|
||||
- **Scoring engines**: `src/Policy/StellaOps.Policy.Engine/Scoring/Engines/`
|
||||
- `SimpleScoringEngine.cs` -- basic risk point computation
|
||||
- `AdvancedScoringEngine.cs` -- multi-factor scoring with mitigations
|
||||
- `ProofAwareScoringEngine.cs` -- proof-linked scoring with attestation references
|
||||
- **IScoringEngine interface**: `src/Policy/StellaOps.Policy.Engine/Scoring/IScoringEngine.cs`
|
||||
- **ScoringProfileService**: `src/Policy/StellaOps.Policy.Engine/Scoring/ScoringProfileService.cs` -- profile-based scoring with configurable weights
|
||||
- **ScorePolicyService**: `src/Policy/StellaOps.Policy.Engine/Scoring/ScorePolicyService.cs` -- policy-aware scoring
|
||||
- **EvidenceWeightedScore**: `src/Policy/StellaOps.Policy.Engine/Scoring/EvidenceWeightedScore/` -- evidence-weighted scoring enricher
|
||||
- **UnknownRanker**: `src/Policy/__Libraries/StellaOps.Policy.Unknowns/Services/UnknownRanker.cs`
|
||||
- Two-factor formula: Score = (Uncertainty * 50) + (ExploitPressure * 50)
|
||||
- Uncertainty: Missing VEX +0.40, Missing reachability +0.30, Conflicting +0.20, Stale +0.10
|
||||
- Exploit pressure: KEV +0.50, EPSS>=0.90 +0.30, EPSS>=0.50 +0.15, CVSS>=9.0 +0.05
|
||||
|
||||
## E2E Test Plan
|
||||
- [ ] Score finding with base criticality=Critical (CVSS 9.5); verify RP score >= 90
|
||||
- [ ] Score finding with diff risk (new finding in delta); verify RP includes diff risk component
|
||||
- [ ] Score finding with operational context (production, internet-facing); verify RP adjusted upward
|
||||
- [ ] Score finding with mitigation (compensating control verified); verify RP reduced
|
||||
- [ ] Verify monotonicity: increasing CVSS from 7.0 to 9.0 always increases RP score
|
||||
- [ ] Verify monotonicity: adding KEV flag always increases RP score
|
||||
- [ ] Score with SimpleScoringEngine; verify basic RP computation
|
||||
- [ ] Score with AdvancedScoringEngine; verify multi-factor computation includes mitigations
|
||||
- [ ] Score with ProofAwareScoringEngine; verify attestation references included
|
||||
- [ ] Verify EvidenceWeightedScore enricher adjusts score based on evidence freshness/completeness
|
||||
@@ -0,0 +1,41 @@
|
||||
# Risk Verdict Attestation (RVA) Contract
|
||||
|
||||
## Module
|
||||
Policy
|
||||
|
||||
## Status
|
||||
IMPLEMENTED
|
||||
|
||||
## Description
|
||||
Structured Risk Verdict Attestation with PASS/FAIL/PASS_WITH_EXCEPTIONS/INDETERMINATE verdicts, policy references, knowledge snapshot bindings, evidence references, and reason codes as a first-class product artifact.
|
||||
|
||||
## Implementation Details
|
||||
- **VerdictAttestationService**: `src/Policy/StellaOps.Policy.Engine/Attestation/VerdictAttestationService.cs`
|
||||
- Generates DSSE-signed attestations for policy verdicts
|
||||
- Verdict types: PASS, FAIL, PASS_WITH_EXCEPTIONS, INDETERMINATE
|
||||
- Policy reference binding: PolicyBundleDigest links attestation to specific policy version
|
||||
- Knowledge snapshot binding: SnapshotId links to frozen evaluation inputs
|
||||
- Evidence references: content-addressed digests for all evidence used
|
||||
- Reason codes for verdict justification
|
||||
- **PolicyDecisionAttestationService**: `src/Policy/StellaOps.Policy.Engine/Attestation/PolicyDecisionAttestationService.cs`
|
||||
- Creates attestations for individual policy decisions within a verdict
|
||||
- **RvaService**: `src/Policy/StellaOps.Policy.Engine/Attestation/RvaService.cs` -- Risk Verdict Attestation service
|
||||
- **ScoringDeterminismVerifier**: `src/Policy/StellaOps.Policy.Engine/Attestation/ScoringDeterminismVerifier.cs` -- verifies scoring determinism before attestation
|
||||
- **ReplayedVerdict model**: `src/Policy/__Libraries/StellaOps.Policy/Replay/ReplayResult.cs`
|
||||
- ReplayDecision enum: Unknown, Pass, Fail, PassWithExceptions, Indeterminate
|
||||
- Verdict includes Score, FindingIds, KnowledgeSnapshotId
|
||||
- **KnowledgeSnapshotManifest**: `src/Policy/__Libraries/StellaOps.Policy/Snapshots/KnowledgeSnapshotManifest.cs`
|
||||
- Content-addressed snapshot binding: SnapshotId (ksm:sha256:{hash})
|
||||
- **Attestation directory**: `src/Policy/StellaOps.Policy.Engine/Attestation/` -- 28 files for attestation generation, verification, and management
|
||||
|
||||
## E2E Test Plan
|
||||
- [ ] Generate RVA for artifact with all gates passing; verify verdict=PASS with reason codes
|
||||
- [ ] Generate RVA for artifact with blocked gate; verify verdict=FAIL with blocking gate in reason
|
||||
- [ ] Generate RVA for artifact with exception applied; verify verdict=PASS_WITH_EXCEPTIONS
|
||||
- [ ] Generate RVA with indeterminate state (missing evidence); verify verdict=INDETERMINATE
|
||||
- [ ] Verify RVA includes PolicyBundleDigest matching the policy used for evaluation
|
||||
- [ ] Verify RVA includes SnapshotId matching the KnowledgeSnapshotManifest
|
||||
- [ ] Verify RVA includes evidence references (content-addressed digests)
|
||||
- [ ] Verify DSSE signature on RVA is valid and covers all verdict fields
|
||||
- [ ] Verify ScoringDeterminismVerifier passes before attestation generation
|
||||
- [ ] Parse RVA JSON; verify all required fields are present (verdict, policy_ref, snapshot_id, evidence_refs, reason_codes, generated_at)
|
||||
@@ -0,0 +1,36 @@
|
||||
# Runtime Containment Signals for Unknowns Scoring
|
||||
|
||||
## Module
|
||||
Policy
|
||||
|
||||
## Status
|
||||
IMPLEMENTED
|
||||
|
||||
## Description
|
||||
Incorporates runtime isolation posture (Seccomp enforcement, read-only filesystem, network isolation) as risk reduction factors in unknowns scoring. Enforced Seccomp gives 10% reduction, read-only FS gives 10%, network isolation gives 5%. Total containment reduction capped at 40%.
|
||||
|
||||
## Implementation Details
|
||||
- **UnknownRanker**: `src/Policy/__Libraries/StellaOps.Policy.Unknowns/Services/UnknownRanker.cs`
|
||||
- Containment reduction applied after raw score computation
|
||||
- Signal flags: Isolated (15%), NotNetFacing (5%), NonRoot (5%), Seccomp (10%), FsRO (10%), NetworkIsolated (5%)
|
||||
- Total containment reduction capped at 40%
|
||||
- Computation: `containmentBps = min(Sum(signal_bps), 4000)`, then `score *= (10000 - containmentBps) / 10000`
|
||||
- Containment signals come from `UnknownRankInput.ContainmentSignals`
|
||||
- **BlastRadius model**: `src/Policy/__Libraries/StellaOps.Policy.Unknowns/Models/BlastRadius.cs`
|
||||
- Dependents (int) -- number of dependent services
|
||||
- NetFacing (bool) -- whether the service is internet-facing
|
||||
- Privilege (string?) -- privilege level of the service process
|
||||
- **UnknownRankerOptions**: `src/Policy/__Libraries/StellaOps.Policy.Unknowns/Services/UnknownRankerOptions.cs` -- configurable containment signal weights
|
||||
- **Band assignment**: After containment reduction, score maps to bands: Hot >= 75, Warm >= 50, Cold >= 25, Negligible < 25
|
||||
|
||||
## E2E Test Plan
|
||||
- [ ] Score unknown with no containment signals; verify full score (no reduction)
|
||||
- [ ] Score unknown with Seccomp=true; verify 10% reduction from raw score
|
||||
- [ ] Score unknown with FsRO=true; verify 10% reduction from raw score
|
||||
- [ ] Score unknown with Isolated=true; verify 15% reduction from raw score
|
||||
- [ ] Score unknown with all containment signals (Isolated+NotNetFacing+NonRoot+Seccomp+FsRO+NetworkIsolated); verify reduction capped at 40%
|
||||
- [ ] Score unknown with Seccomp+FsRO (20% reduction); verify score reduced by exactly 20%
|
||||
- [ ] Score unknown with containment reducing score from Hot (76) to Warm range; verify band changes from Hot to Warm
|
||||
- [ ] Verify containment does not reduce score below 0
|
||||
- [ ] Verify containment signals from BlastRadius.NetFacing=false maps to NotNetFacing signal
|
||||
- [ ] Verify configurable weights in UnknownRankerOptions are applied correctly
|
||||
36
docs/features/checked/policy/sbom-presence-policy-gate.md
Normal file
36
docs/features/checked/policy/sbom-presence-policy-gate.md
Normal file
@@ -0,0 +1,36 @@
|
||||
# SBOM Presence Policy Gate (SbomPresenceGate)
|
||||
|
||||
## Module
|
||||
Policy
|
||||
|
||||
## Status
|
||||
IMPLEMENTED
|
||||
|
||||
## Description
|
||||
Policy gate that blocks releases lacking a valid SBOM document, with configurable format requirements (CycloneDX/SPDX), minimum component count thresholds, and freshness checks.
|
||||
|
||||
## Implementation Details
|
||||
- **PolicyGateEvaluator Evidence Completeness gate**: `src/Policy/StellaOps.Policy.Engine/Gates/PolicyGateEvaluator.cs`
|
||||
- Evidence Completeness gate (first in 5-gate pipeline) checks for SBOM presence
|
||||
- Missing SBOM triggers Block or Warn based on gate configuration
|
||||
- Evaluates SBOM format, component count, and freshness as part of evidence checks
|
||||
- **DriftGateEvaluator**: `src/Policy/StellaOps.Policy.Engine/Gates/DriftGateEvaluator.cs`
|
||||
- Evaluates SBOM drift between baseline and target
|
||||
- SBOM format validation (CycloneDX/SPDX) as part of drift analysis
|
||||
- **DriftGateOptions**: `src/Policy/StellaOps.Policy.Engine/Gates/DriftGateOptions.cs` -- configurable SBOM requirements
|
||||
- **EvidenceTtlEnforcer**: `src/Policy/__Libraries/StellaOps.Policy/Freshness/EvidenceTtlEnforcer.cs`
|
||||
- SBOM/Provenance freshness: checks BuildTime against TTL
|
||||
- Freshness statuses: Fresh, Warning, Stale
|
||||
- **WhatIfSimulationService**: `src/Policy/StellaOps.Policy.Engine/WhatIfSimulation/WhatIfSimulationService.cs`
|
||||
- SBOM diff operations verify SBOM presence before simulation
|
||||
|
||||
## E2E Test Plan
|
||||
- [ ] Evaluate artifact without SBOM; verify Evidence Completeness gate blocks
|
||||
- [ ] Evaluate artifact with valid CycloneDX SBOM; verify gate passes
|
||||
- [ ] Evaluate artifact with valid SPDX SBOM; verify gate passes
|
||||
- [ ] Configure minimum component count threshold=10; provide SBOM with 5 components; verify gate warns/blocks
|
||||
- [ ] Configure minimum component count threshold=10; provide SBOM with 15 components; verify gate passes
|
||||
- [ ] Evaluate artifact with stale SBOM (BuildTime exceeds TTL); verify freshness check warns
|
||||
- [ ] Evaluate artifact with fresh SBOM (BuildTime within TTL); verify freshness check passes
|
||||
- [ ] Verify gate result message indicates SBOM format and component count when present
|
||||
- [ ] Verify DriftGateEvaluator detects missing SBOM in drift analysis
|
||||
@@ -0,0 +1,41 @@
|
||||
# Score Attestation and Proof Ledger
|
||||
|
||||
## Module
|
||||
Policy
|
||||
|
||||
## Status
|
||||
IMPLEMENTED
|
||||
|
||||
## Description
|
||||
Score attestation statements linked to proof nodes in a proof ledger for auditable scoring decisions.
|
||||
|
||||
## Implementation Details
|
||||
- **VerdictAttestationService**: `src/Policy/StellaOps.Policy.Engine/Attestation/VerdictAttestationService.cs`
|
||||
- DSSE-signed attestations for scoring decisions
|
||||
- Links score to evidence references and policy bundle digest
|
||||
- Content-addressed attestation IDs for proof chain linking
|
||||
- **PolicyDecisionAttestationService**: `src/Policy/StellaOps.Policy.Engine/Attestation/PolicyDecisionAttestationService.cs`
|
||||
- Per-decision attestations within a verdict
|
||||
- **ScoringDeterminismVerifier**: `src/Policy/StellaOps.Policy.Engine/Attestation/ScoringDeterminismVerifier.cs`
|
||||
- Verifies scoring is deterministic before creating attestation
|
||||
- Prevents non-deterministic scores from entering proof ledger
|
||||
- **LedgerExportService**: `src/Policy/StellaOps.Policy.Engine/Ledger/LedgerExportService.cs`
|
||||
- Proof ledger: records score attestations with timestamps, actors, and references
|
||||
- Export for compliance and audit review
|
||||
- **LedgerModels**: `src/Policy/StellaOps.Policy.Engine/Ledger/LedgerModels.cs` -- data models for ledger entries
|
||||
- **LedgerExportStore**: `src/Policy/StellaOps.Policy.Engine/Ledger/LedgerExportStore.cs` -- persistence layer
|
||||
- **ProofAwareScoringEngine**: `src/Policy/StellaOps.Policy.Engine/Scoring/Engines/ProofAwareScoringEngine.cs`
|
||||
- Scoring engine that embeds proof references in scored output
|
||||
- Links each score to the evidence and policy rules that produced it
|
||||
- **EvidenceWeightedScore**: `src/Policy/StellaOps.Policy.Engine/Scoring/EvidenceWeightedScore/` -- evidence-weighted scoring with attestation links
|
||||
|
||||
## E2E Test Plan
|
||||
- [ ] Score finding with ProofAwareScoringEngine; verify score includes proof references
|
||||
- [ ] Generate score attestation; verify DSSE signature covers score, evidence refs, and policy bundle
|
||||
- [ ] Verify ScoringDeterminismVerifier passes for deterministic scoring inputs
|
||||
- [ ] Record score attestation in proof ledger; verify ledger entry includes attestation ID, score, and timestamp
|
||||
- [ ] Export proof ledger; verify all score attestations are included in chronological order
|
||||
- [ ] Score same finding twice with same inputs; verify attestation IDs match (content-addressed)
|
||||
- [ ] Score finding with different evidence; verify attestation IDs differ
|
||||
- [ ] Verify proof ledger links score attestation to parent verdict attestation
|
||||
- [ ] Query ledger by time range; verify filtered results include only entries in range
|
||||
40
docs/features/checked/policy/score-v1-policy-format.md
Normal file
40
docs/features/checked/policy/score-v1-policy-format.md
Normal file
@@ -0,0 +1,40 @@
|
||||
# Score.v1 Policy Format
|
||||
|
||||
## Status
|
||||
IMPLEMENTED
|
||||
|
||||
## Description
|
||||
No Score.v1 policy format was found in the codebase. Scoring is embedded in the TrustVerdict and SmartDiff modules without a standalone schema.
|
||||
|
||||
## Why Marked as Dropped (Correction)
|
||||
**FINDING: The Score.v1 policy format IS implemented.** The following exist:
|
||||
- `src/Policy/__Libraries/StellaOps.Policy/Schemas/score-policy.v1.schema.json` -- the v1 JSON schema itself
|
||||
- `src/Policy/__Libraries/StellaOps.Policy/Scoring/ScorePolicyValidator.cs` -- validates policies against the v1 schema
|
||||
- `src/Policy/__Libraries/StellaOps.Policy/Scoring/ScorePolicyModels.cs` -- data models for score policies
|
||||
- `src/Policy/__Libraries/StellaOps.Policy/Scoring/ScorePolicyLoader.cs` -- loads score policy definitions
|
||||
- `src/Policy/__Libraries/StellaOps.Policy/Scoring/ScoreAttestationStatement.cs` -- score attestation statements
|
||||
- `src/Policy/StellaOps.Policy.Engine/Scoring/ScorePolicyService.cs` -- runtime score policy evaluation service
|
||||
- `src/Policy/StellaOps.Policy.Engine/Scoring/ScoringProfileService.cs` -- scoring profile management
|
||||
- Policy interop: `src/Policy/__Libraries/StellaOps.Policy.Interop/Import/JsonPolicyImporter.cs`, `FormatDetector.cs`
|
||||
- Tests: `src/Policy/__Tests/StellaOps.Policy.Engine.Tests/Scoring/ScorePolicyServiceCachingTests.cs`, `ScorePolicyDigestReplayIntegrationTests.cs`, `ScoreBasedRuleTests.cs`
|
||||
|
||||
## Implementation Details
|
||||
- Schema: `src/Policy/__Libraries/StellaOps.Policy/Schemas/score-policy.v1.schema.json`
|
||||
- Validator: `src/Policy/__Libraries/StellaOps.Policy/Scoring/ScorePolicyValidator.cs`
|
||||
- Models: `src/Policy/__Libraries/StellaOps.Policy/Scoring/ScorePolicyModels.cs`
|
||||
- Loader: `src/Policy/__Libraries/StellaOps.Policy/Scoring/ScorePolicyLoader.cs`
|
||||
- Engine service: `src/Policy/StellaOps.Policy.Engine/Scoring/ScorePolicyService.cs`
|
||||
|
||||
## E2E Test Plan
|
||||
- Validate score policy schema validation against valid/invalid policies
|
||||
- Test score policy loading and evaluation
|
||||
- Verify score-based rules produce deterministic decisions
|
||||
- Test digest replay for score policies
|
||||
|
||||
## Source
|
||||
- Feature matrix scan
|
||||
|
||||
## Notes
|
||||
- Module: Policy
|
||||
- Modules referenced: `src/Policy/__Libraries/StellaOps.Policy/Scoring/`, `src/Policy/StellaOps.Policy.Engine/Scoring/`
|
||||
- **Status should be reclassified from NOT_FOUND to IMPLEMENTED**
|
||||
41
docs/features/checked/policy/security-state-delta.md
Normal file
41
docs/features/checked/policy/security-state-delta.md
Normal file
@@ -0,0 +1,41 @@
|
||||
# Security State Delta (Diff Engine)
|
||||
|
||||
## Module
|
||||
Policy
|
||||
|
||||
## Status
|
||||
IMPLEMENTED
|
||||
|
||||
## Description
|
||||
A diff engine that takes baseline and target snapshot digests and produces structured delta objects with baseline selection methods (previous build, last approved, last deployed).
|
||||
|
||||
## Implementation Details
|
||||
- **WhatIfSimulationService**: `src/Policy/StellaOps.Policy.Engine/WhatIfSimulation/WhatIfSimulationService.cs`
|
||||
- `SimulateAsync()` computes baseline vs target deltas
|
||||
- Baseline selection: current artifact state as baseline, simulated changes as target
|
||||
- Delta objects: decision changes (status_changed, severity_changed, new, removed)
|
||||
- Impact summary: risk delta (increased/decreased/unchanged), blocked/warning deltas
|
||||
- **ConsoleSimulationDiffService**: `src/Policy/StellaOps.Policy.Engine/Console/ConsoleSimulationDiffService.cs`
|
||||
- Schema version: console-policy-23-001
|
||||
- Structured before/after delta with severity breakdowns
|
||||
- Rule impact analysis: which policy rules drove the delta
|
||||
- Deterministic output for same inputs
|
||||
- **DriftGateEvaluator**: `src/Policy/StellaOps.Policy.Engine/Gates/DriftGateEvaluator.cs`
|
||||
- SBOM drift detection between baseline and target snapshots
|
||||
- Produces structured drift delta with component additions/removals/upgrades
|
||||
- **DriftGateContext**: `src/Policy/StellaOps.Policy.Engine/Gates/DriftGateContext.cs` -- context for drift evaluation with baseline/target digests
|
||||
- **KnowledgeSnapshotManifest**: `src/Policy/__Libraries/StellaOps.Policy/Snapshots/KnowledgeSnapshotManifest.cs`
|
||||
- Content-addressed snapshots enable diff between any two evaluation states
|
||||
- Baseline selection via SnapshotId comparison
|
||||
|
||||
## E2E Test Plan
|
||||
- [ ] Compute delta between baseline and target with 2 new critical findings; verify delta shows 2 new findings with severity=Critical
|
||||
- [ ] Compute delta between baseline and target with 1 resolved finding; verify delta shows 1 removed finding
|
||||
- [ ] Compute delta with severity change (High->Critical); verify delta shows severity_changed
|
||||
- [ ] Compute delta with status change (Warn->Block); verify delta shows status_changed
|
||||
- [ ] Select baseline as "previous build"; verify correct baseline snapshot used
|
||||
- [ ] Select baseline as "last approved"; verify correct baseline snapshot used
|
||||
- [ ] Verify delta includes risk delta (increased/decreased/unchanged) summary
|
||||
- [ ] Verify ConsoleSimulationDiffService produces deterministic delta for same inputs
|
||||
- [ ] Verify DriftGateEvaluator detects component additions in SBOM drift
|
||||
- [ ] Verify delta is empty when baseline and target are identical
|
||||
@@ -0,0 +1,41 @@
|
||||
# Signature Required Policy Gate (SignatureRequiredGate)
|
||||
|
||||
## Module
|
||||
Policy
|
||||
|
||||
## Status
|
||||
IMPLEMENTED
|
||||
|
||||
## Description
|
||||
Policy gate requiring valid cryptographic signatures on release artifacts before promotion, with configurable signing key allowlists, certificate chain validation, and Rekor inclusion proof requirements.
|
||||
|
||||
## Implementation Details
|
||||
- **PolicyGateEvaluator**: `src/Policy/StellaOps.Policy.Engine/Gates/PolicyGateEvaluator.cs`
|
||||
- Evidence Completeness gate (first in pipeline) verifies signature presence
|
||||
- Signature requirements configurable per environment
|
||||
- Gate result types: Pass (valid signature), Block (missing/invalid signature)
|
||||
- **VexTrustGate**: `src/Policy/StellaOps.Policy.Engine/Gates/VexTrustGate.cs`
|
||||
- `RequireIssuerVerified` per-environment: production=true, staging=true, development=false
|
||||
- Issuer signature verification as part of VEX trust evaluation
|
||||
- **VexTrustGateOptions**: `src/Policy/StellaOps.Policy.Engine/Gates/VexTrustGateOptions.cs`
|
||||
- Per-environment signing requirements (RequireIssuerVerified flag)
|
||||
- FailureAction: Warn or Block when signature verification fails
|
||||
- **EvidenceRequirementValidator**: `src/Policy/__Libraries/StellaOps.Policy.Exceptions/Services/EvidenceRequirementValidator.cs`
|
||||
- DSSE signature verification for evidence attestations
|
||||
- Validates signed evidence meets trust requirements
|
||||
- **VerdictAttestationService**: `src/Policy/StellaOps.Policy.Engine/Attestation/VerdictAttestationService.cs`
|
||||
- DSSE-signed verdict attestations with certificate chain
|
||||
- **KnowledgeSnapshotManifest**: `src/Policy/__Libraries/StellaOps.Policy/Snapshots/KnowledgeSnapshotManifest.cs`
|
||||
- TrustBundleRef (BundleId, Digest, Uri) for trust anchor set
|
||||
- Signature field on manifest for optional DSSE signing
|
||||
|
||||
## E2E Test Plan
|
||||
- [ ] Evaluate artifact with valid signature from allowed key; verify gate passes
|
||||
- [ ] Evaluate artifact without signature; verify gate blocks with "missing signature" message
|
||||
- [ ] Evaluate artifact with signature from key not in allowlist; verify gate blocks
|
||||
- [ ] Configure environment requiring issuer verification; provide unverified issuer; verify gate blocks
|
||||
- [ ] Configure environment not requiring issuer verification (development); provide unsigned VEX; verify gate passes
|
||||
- [ ] Evaluate artifact with expired certificate; verify gate blocks with certificate validation error
|
||||
- [ ] Verify DSSE envelope structure on verdict attestation includes valid signature
|
||||
- [ ] Verify TrustBundleRef in KnowledgeSnapshotManifest references correct trust anchor set
|
||||
- [ ] Verify EvidenceRequirementValidator validates DSSE signature on evidence attestation
|
||||
@@ -0,0 +1,43 @@
|
||||
# Signed VEX Override Enforcement in Policy Engine
|
||||
|
||||
## Module
|
||||
Policy
|
||||
|
||||
## Status
|
||||
IMPLEMENTED
|
||||
|
||||
## Description
|
||||
Policy engine requires signed VEX override attestations with DSSE/Rekor validation, exposes override_signed and override_rekor_verified signals to DSL, and supports key trust levels and validity period enforcement.
|
||||
|
||||
## Implementation Details
|
||||
- **VexTrustGate**: `src/Policy/StellaOps.Policy.Engine/Gates/VexTrustGate.cs` (implements `IVexTrustGate`)
|
||||
- Evaluates VEX trust including signature verification status
|
||||
- VexTrustStatus with TrustScore and TrustBreakdown (issuer verification, accuracy, freshness)
|
||||
- Per-environment thresholds for signature requirements
|
||||
- **VexTrustGateOptions**: `src/Policy/StellaOps.Policy.Engine/Gates/VexTrustGateOptions.cs`
|
||||
- Production: RequireIssuerVerified=true, MinCompositeScore=0.80, FailureAction=Block
|
||||
- Staging: RequireIssuerVerified=true, FailureAction=Warn
|
||||
- MissingTrustBehavior: Allow/Warn/Block when VEX trust data absent
|
||||
- **TrustLatticeEngine**: `src/Policy/__Libraries/StellaOps.Policy/TrustLattice/TrustLatticeEngine.cs`
|
||||
- VEX normalization pipeline supports DSSE-signed VEX documents
|
||||
- Three normalizers: CycloneDX, OpenVEX, CSAF
|
||||
- Signed VEX claims receive higher trust scores in ClaimScoreMerger
|
||||
- **ClaimScoreMerger**: `src/Policy/__Libraries/StellaOps.Policy/TrustLattice/ClaimScoreMerger.cs`
|
||||
- Signed claims scored higher via specificity and score adjustments
|
||||
- Conflict penalization (0.25) applies to conflicting signed/unsigned claims
|
||||
- **EvidenceRequirementValidator**: `src/Policy/__Libraries/StellaOps.Policy.Exceptions/Services/EvidenceRequirementValidator.cs`
|
||||
- DSSE signature verification on VEX override evidence
|
||||
- Trust score threshold validation for signed evidence
|
||||
- **PolicyGateEvaluator**: `src/Policy/StellaOps.Policy.Engine/Gates/PolicyGateEvaluator.cs`
|
||||
- VEX Trust gate evaluates signed override status as part of multi-gate pipeline
|
||||
|
||||
## E2E Test Plan
|
||||
- [ ] Submit DSSE-signed VEX override; verify VexTrustGate passes with high TrustScore
|
||||
- [ ] Submit unsigned VEX override in production; verify VexTrustGate blocks (RequireIssuerVerified=true)
|
||||
- [ ] Submit unsigned VEX override in development; verify VexTrustGate passes (RequireIssuerVerified=false)
|
||||
- [ ] Submit signed VEX with expired signing key; verify trust score reduced or gate blocks
|
||||
- [ ] Submit signed VEX with Rekor inclusion proof; verify higher trust score than without proof
|
||||
- [ ] Submit conflicting signed and unsigned VEX claims; verify ClaimScoreMerger applies conflict penalty, signed claim wins
|
||||
- [ ] Verify VexTrustStatus includes TrustBreakdown with issuer verification status
|
||||
- [ ] Submit VEX override with trust score below MinCompositeScore; verify gate blocks in production
|
||||
- [ ] Configure MissingTrustBehavior=Block; submit VEX without trust data; verify gate blocks
|
||||
@@ -0,0 +1,44 @@
|
||||
# Smart-Diff Semantic Risk Delta (Moat Score 4)
|
||||
|
||||
## Module
|
||||
Policy
|
||||
|
||||
## Status
|
||||
IMPLEMENTED
|
||||
|
||||
## Description
|
||||
Material risk change detection with delta verdict computation, security state delta analysis, and delta computing.
|
||||
|
||||
## Implementation Details
|
||||
- **WhatIfSimulationService**: `src/Policy/StellaOps.Policy.Engine/WhatIfSimulation/WhatIfSimulationService.cs`
|
||||
- `SimulateAsync()` computes semantic risk delta between baseline and target
|
||||
- SBOM diff operations: add, remove, upgrade, downgrade with advisory/VEX/reachability context
|
||||
- Decision change types: status_changed, severity_changed, new, removed
|
||||
- Impact summary: risk delta (increased/decreased/unchanged), material risk indicators
|
||||
- Recommendations based on delta analysis
|
||||
- **ConsoleSimulationDiffService**: `src/Policy/StellaOps.Policy.Engine/Console/ConsoleSimulationDiffService.cs`
|
||||
- Schema version: console-policy-23-001
|
||||
- Deterministic before/after severity breakdowns
|
||||
- Rule impact analysis: identifies which policy rules drive the risk delta
|
||||
- Explain samples for delta reasoning
|
||||
- **DriftGateEvaluator**: `src/Policy/StellaOps.Policy.Engine/Gates/DriftGateEvaluator.cs`
|
||||
- SBOM drift detection as a semantic risk signal
|
||||
- Component addition/removal/version change tracking
|
||||
- **CounterfactualEngine**: `src/Policy/__Libraries/StellaOps.Policy/Counterfactuals/CounterfactualEngine.cs`
|
||||
- Computes "what would fix this" paths: VEX, Exception, Reachability, VersionUpgrade, CompensatingControl
|
||||
- Effort ratings per path: Critical=5, High=4, Medium=3, Low=2
|
||||
- **RiskSimulationService**: `src/Policy/StellaOps.Policy.Engine/Simulation/RiskSimulationService.cs`
|
||||
- Signal-based risk scoring with distribution and top movers
|
||||
- `CompareProfilesWithBreakdown()` for before/after profile delta
|
||||
|
||||
## E2E Test Plan
|
||||
- [ ] Simulate adding component with known critical CVE; verify risk delta shows "increased" with new critical finding
|
||||
- [ ] Simulate upgrading component that fixes CVE; verify risk delta shows "decreased" with removed finding
|
||||
- [ ] Simulate no changes; verify risk delta shows "unchanged"
|
||||
- [ ] Verify severity_changed detection: finding changes from High to Critical
|
||||
- [ ] Verify new detection: component addition introduces new findings
|
||||
- [ ] Verify removed detection: component removal clears associated findings
|
||||
- [ ] Verify CounterfactualEngine computes fix paths for blocked findings in delta
|
||||
- [ ] Verify console diff output includes rule impact analysis
|
||||
- [ ] Verify deterministic output: same baseline + target always produces same delta
|
||||
- [ ] Verify DriftGateEvaluator integrates semantic drift into gate evaluation
|
||||
43
docs/features/checked/policy/time-travel-replay-engine.md
Normal file
43
docs/features/checked/policy/time-travel-replay-engine.md
Normal file
@@ -0,0 +1,43 @@
|
||||
# Time-Travel Replay Engine
|
||||
|
||||
## Module
|
||||
Policy
|
||||
|
||||
## Status
|
||||
IMPLEMENTED
|
||||
|
||||
## Description
|
||||
Re-evaluation of any historical decision using only snapshot content and recorded execution contract, producing match/mismatch reports with deterministic comparison.
|
||||
|
||||
## Implementation Details
|
||||
- **ReplayEngine**: `src/Policy/__Libraries/StellaOps.Policy/Replay/ReplayEngine.cs` (sealed class implements `IReplayEngine`)
|
||||
- `ReplayAsync(ReplayRequest)` replays historical decisions using frozen snapshot inputs
|
||||
- Pipeline: load snapshot -> verify snapshot integrity -> resolve frozen inputs -> execute evaluation -> compare with original -> generate report
|
||||
- Frozen inputs prevent time-dependent drift (no network fetch, no live data)
|
||||
- Duration tracking for performance analysis
|
||||
- **ReplayRequest**: `src/Policy/__Libraries/StellaOps.Policy/Replay/ReplayRequest.cs`
|
||||
- ArtifactDigest, SnapshotId (content-addressed), OriginalVerdictId
|
||||
- ReplayOptions: CompareWithOriginal=true, AllowNetworkFetch=false, GenerateDetailedReport=true, ScoreTolerance=0.001
|
||||
- **ReplayResult**: `src/Policy/__Libraries/StellaOps.Policy/Replay/ReplayResult.cs`
|
||||
- MatchStatus: ExactMatch, MatchWithinTolerance, Mismatch, NoComparison, ReplayFailed
|
||||
- DeltaReport: Summary, FieldDeltas, FindingDeltas, SuspectedCauses
|
||||
- **VerdictComparer**: `src/Policy/__Libraries/StellaOps.Policy/Replay/VerdictComparer.cs` -- deterministic verdict comparison
|
||||
- **ReplayReport**: `src/Policy/__Libraries/StellaOps.Policy/Replay/ReplayReport.cs` -- detailed report generation
|
||||
- **KnowledgeSourceResolver**: `src/Policy/__Libraries/StellaOps.Policy/Replay/KnowledgeSourceResolver.cs` -- resolves snapshot sources to frozen data
|
||||
- **KnowledgeSnapshotManifest**: `src/Policy/__Libraries/StellaOps.Policy/Snapshots/KnowledgeSnapshotManifest.cs`
|
||||
- Content-addressed snapshots enable time-travel to any historical state
|
||||
- Sources with Epoch and Digest for point-in-time data resolution
|
||||
- **SnapshotAwarePolicyEvaluator**: `src/Policy/__Libraries/StellaOps.Policy/Snapshots/SnapshotAwarePolicyEvaluator.cs`
|
||||
- Evaluates policy using only pinned snapshot inputs (no live data)
|
||||
|
||||
## E2E Test Plan
|
||||
- [ ] Replay decision from 30 days ago using its snapshot; verify ExactMatch (deterministic)
|
||||
- [ ] Replay decision with modified snapshot source; verify Mismatch with SuspectedCauses
|
||||
- [ ] Replay with AllowNetworkFetch=false; verify no external calls made during replay
|
||||
- [ ] Verify ReplayResult.Duration tracks execution time
|
||||
- [ ] Replay with missing snapshot; verify ReplayFailed with descriptive error
|
||||
- [ ] Replay with ScoreTolerance=0.01; introduce minor floating-point drift; verify MatchWithinTolerance
|
||||
- [ ] Verify DeltaReport.FieldDeltas shows exact field-level differences for Mismatch
|
||||
- [ ] Verify DeltaReport.FindingDeltas shows finding-level changes (Added/Removed/Modified)
|
||||
- [ ] Replay multiple historical decisions; verify each produces correct MatchStatus
|
||||
- [ ] Verify snapshot integrity check runs before replay execution
|
||||
@@ -0,0 +1,40 @@
|
||||
# Unknown Budget Policy Enforcement
|
||||
|
||||
## Module
|
||||
Policy
|
||||
|
||||
## Status
|
||||
IMPLEMENTED
|
||||
|
||||
## Description
|
||||
Unknown budget enforcement with environment-aware thresholds, supporting policy evaluation that can fail/warn based on unknown counts by type.
|
||||
|
||||
## Implementation Details
|
||||
- **UnknownsBudgetEnforcer**: `src/Policy/__Libraries/StellaOps.Policy.Unknowns/Services/UnknownsBudgetEnforcer.cs`
|
||||
- Enforces budget constraints based on unknown counts by type
|
||||
- Threshold levels: Green (within limits), Yellow (warning), Red (over budget), Exhausted (blocked)
|
||||
- Environment-aware: different thresholds for production vs staging vs development
|
||||
- **UnknownBudgetService**: `src/Policy/__Libraries/StellaOps.Policy.Unknowns/Services/UnknownBudgetService.cs`
|
||||
- Budget management: create, query, consume, replenish
|
||||
- Per-type budget tracking (reachability unknowns, identity unknowns, VEX gaps, etc.)
|
||||
- **UnknownRanker**: `src/Policy/__Libraries/StellaOps.Policy.Unknowns/Services/UnknownRanker.cs`
|
||||
- Ranks unknowns by two-factor score: (Uncertainty * 50) + (ExploitPressure * 50)
|
||||
- Reason codes: AnalyzerLimit, Reachability, Identity, Provenance, VexConflict, FeedGap, ConfigUnknown
|
||||
- Band assignment: Hot >= 75, Warm >= 50, Cold >= 25, Negligible < 25
|
||||
- **PolicyGateEvaluator Uncertainty Tier gate**: `src/Policy/StellaOps.Policy.Engine/Gates/PolicyGateEvaluator.cs`
|
||||
- Uncertainty Tier gate (4th in pipeline) evaluates unknown counts against thresholds
|
||||
- Tiers: T1 (High uncertainty -> strictest enforcement), T4 (Negligible -> most permissive)
|
||||
- **BudgetEndpoints**: `src/Policy/StellaOps.Policy.Engine/Endpoints/BudgetEndpoints.cs` -- API for budget management
|
||||
- **RiskBudgetEndpoints**: `src/Policy/StellaOps.Policy.Engine/Endpoints/RiskBudgetEndpoints.cs` -- API for budget evaluation
|
||||
|
||||
## E2E Test Plan
|
||||
- [ ] Set budget limit for reachability unknowns=10; report 5 unknowns; verify status Green
|
||||
- [ ] Set budget limit for reachability unknowns=10; report 8 unknowns; verify status Yellow (warning)
|
||||
- [ ] Set budget limit for reachability unknowns=10; report 12 unknowns; verify status Red/Exhausted
|
||||
- [ ] Verify environment-aware thresholds: production has stricter limits than development
|
||||
- [ ] Evaluate through PolicyGateEvaluator with T1 uncertainty tier; verify Uncertainty gate blocks
|
||||
- [ ] Evaluate through PolicyGateEvaluator with T4 uncertainty tier; verify Uncertainty gate passes
|
||||
- [ ] Consume unknowns budget; resolve some unknowns; verify budget capacity restored
|
||||
- [ ] Query budget status via API; verify response includes per-type counts and thresholds
|
||||
- [ ] Verify Hot-band unknowns consume more budget than Cold-band unknowns
|
||||
- [ ] Verify Exhausted budget escalates gate level in PolicyGateEvaluator
|
||||
39
docs/features/checked/policy/unknowns-budget-dashboard.md
Normal file
39
docs/features/checked/policy/unknowns-budget-dashboard.md
Normal file
@@ -0,0 +1,39 @@
|
||||
# Unknowns budget dashboard (budgeted unknowns with policy thresholds)
|
||||
|
||||
## Module
|
||||
Policy
|
||||
|
||||
## Status
|
||||
IMPLEMENTED
|
||||
|
||||
## Description
|
||||
Grey queue, SLA monitoring, unknown budget service, and budget constraint enforcer implement first-class unknowns management with policy thresholds.
|
||||
|
||||
## Implementation Details
|
||||
- **UnknownBudgetService**: `src/Policy/__Libraries/StellaOps.Policy.Unknowns/Services/UnknownBudgetService.cs`
|
||||
- Budget CRUD operations: create, query, consume, replenish, status check
|
||||
- Per-type budget tracking across multiple unknown categories
|
||||
- **UnknownsBudgetEnforcer**: `src/Policy/__Libraries/StellaOps.Policy.Unknowns/Services/UnknownsBudgetEnforcer.cs`
|
||||
- Constraint enforcement with Green/Yellow/Red/Exhausted thresholds
|
||||
- SLA monitoring: tracks unknown resolution against SLA targets
|
||||
- **UnknownRanker**: `src/Policy/__Libraries/StellaOps.Policy.Unknowns/Services/UnknownRanker.cs`
|
||||
- Prioritizes unknowns by HOT/WARM/COLD/Negligible bands
|
||||
- Score = (Uncertainty * 50) + (ExploitPressure * 50)
|
||||
- Reason codes for triage: AnalyzerLimit, Reachability, Identity, Provenance, VexConflict, FeedGap, ConfigUnknown
|
||||
- **Grey queue models**: `src/Policy/__Libraries/StellaOps.Policy.Unknowns/Models/`
|
||||
- UnknownItem with reason code, band assignment, score, and SLA tracking
|
||||
- BlastRadius (Dependents, NetFacing, Privilege)
|
||||
- **BudgetEndpoints**: `src/Policy/StellaOps.Policy.Engine/Endpoints/BudgetEndpoints.cs` -- budget dashboard API
|
||||
- **PolicyGateEvaluator**: `src/Policy/StellaOps.Policy.Engine/Gates/PolicyGateEvaluator.cs` -- uncertainty tier gate enforces budget thresholds
|
||||
|
||||
## E2E Test Plan
|
||||
- [ ] Query budget dashboard; verify response includes per-type unknown counts, thresholds, and status
|
||||
- [ ] Verify HOT band unknowns appear first in priority queue
|
||||
- [ ] Verify SLA tracking: unknown exceeding SLA target flagged as overdue
|
||||
- [ ] Consume budget; verify dashboard updates with new consumption levels
|
||||
- [ ] Resolve unknowns; verify budget capacity replenished and dashboard reflects change
|
||||
- [ ] Verify reason codes in dashboard: AnalyzerLimit, Reachability, Identity, etc.
|
||||
- [ ] Query budget history; verify trendline shows consumption over time
|
||||
- [ ] Verify budget threshold transitions are logged for audit
|
||||
- [ ] Create budget with per-type limits (reachability=10, identity=5); verify independent tracking
|
||||
- [ ] Verify Exhausted status prevents new releases via PolicyGateEvaluator
|
||||
@@ -0,0 +1,49 @@
|
||||
# Unknowns Decay and Triage Queue
|
||||
|
||||
## Module
|
||||
Policy
|
||||
|
||||
## Status
|
||||
IMPLEMENTED
|
||||
|
||||
## Description
|
||||
Unknowns ranking and API endpoints exist. BlastRadius model present with database migration. The full time-based decay algorithm and containment signals ranking were identified as gaps in the archive manifest.
|
||||
|
||||
## What's Implemented
|
||||
- **DecayedConfidenceCalculator**: `src/Policy/__Libraries/StellaOps.Policy.Determinization/Scoring/DecayedConfidenceCalculator.cs`
|
||||
- Exponential decay formula: `max(floor, baseConfidence * exp(-ln(2) * ageDays / halfLifeDays))`
|
||||
- Configurable half-life (default 14 days) and floor
|
||||
- OpenTelemetry histogram: `stellaops_determinization_decay_multiplier`
|
||||
- **ObservationDecay**: `src/Policy/__Libraries/StellaOps.Policy.Determinization/Models/ObservationDecay.cs`
|
||||
- Per-observation decay state: BaseConfidence, ObservedAt, HalfLifeDays=14, Floor=0.35, StalenessThreshold=0.50
|
||||
- `CalculateDecay(now)`: computes current decayed confidence
|
||||
- `CheckIsStale(now)`: returns true when decayed confidence falls below staleness threshold
|
||||
- Factory methods: `Create()`, `Fresh()`, `WithSettings()`
|
||||
- **UnknownRanker** (from unchecked features): two-factor scoring `Uncertainty*50 + ExploitPressure*50`
|
||||
- Containment reduction capped at 40%
|
||||
- Band assignment: Hot>=75, Warm>=50, Cold>=25, Ice<25
|
||||
- **UncertaintyScoreCalculator**: `src/Policy/__Libraries/StellaOps.Policy.Determinization/Scoring/UncertaintyScoreCalculator.cs`
|
||||
- Entropy-based uncertainty from 6 signal dimensions
|
||||
- Signal gap tracking (which signals are missing)
|
||||
- **Risk budget API endpoints** (from unchecked features): budget tracking, ledger, Green/Yellow/Red/Exhausted thresholds
|
||||
- **Unknown budget policy enforcement** (from unchecked features): UncertaintyTierGate (gate 4 in pipeline)
|
||||
|
||||
## What's Missing
|
||||
- **Time-based decay triage queue**: No service that automatically re-queues unknowns for triage when their confidence decays below the staleness threshold
|
||||
- **Triage queue UI**: No frontend triage interface showing unknowns sorted by decay urgency
|
||||
- **Automated re-analysis triggering**: ObservationDecay tracks staleness but no event-driven mechanism triggers re-analysis when an unknown becomes stale
|
||||
- **Containment signal integration**: The advisory describes containment signals (WAF rules, network segmentation, runtime controls) reducing unknown scores -- this is partially in UnknownRanker but not connected to real containment data sources
|
||||
- **Decay notification**: No notification system alerting when high-priority unknowns decay below acceptable confidence
|
||||
- **Historical decay tracking**: DecayedConfidenceCalculator computes point-in-time decay but no ledger tracks confidence decay history over time
|
||||
|
||||
## Implementation Plan
|
||||
- Create `UnknownTriageQueueService` that periodically evaluates ObservationDecay.CheckIsStale() and queues stale unknowns for re-analysis
|
||||
- Add event-driven triggers (e.g., background job or message queue) when confidence drops below threshold
|
||||
- Build triage queue frontend component showing unknowns sorted by urgency (band + decay rate)
|
||||
- Integrate containment signal sources (WAF, network, runtime) as inputs to UnknownRanker
|
||||
- Add decay history ledger for audit trail
|
||||
|
||||
## Related Documentation
|
||||
- Decay calculator: `src/Policy/__Libraries/StellaOps.Policy.Determinization/Scoring/DecayedConfidenceCalculator.cs`
|
||||
- Observation decay model: `src/Policy/__Libraries/StellaOps.Policy.Determinization/Models/ObservationDecay.cs`
|
||||
- Unknowns ranking (unchecked): `docs/features/unchecked/policy/unknowns-ranking-algorithm.md`
|
||||
@@ -0,0 +1,44 @@
|
||||
# Unknowns Grey Queue with Conflict Detection and Reanalysis Fingerprints
|
||||
|
||||
## Module
|
||||
Policy
|
||||
|
||||
## Status
|
||||
IMPLEMENTED
|
||||
|
||||
## Description
|
||||
Deterministic reanalysis fingerprints, conflict detection routing (VEX/reachability contradiction, static/runtime contradiction, VEX status conflict), grey queue with Disputed state and manual adjudication gates, versioned signal event handling.
|
||||
|
||||
## Implementation Details
|
||||
- **UnknownRanker**: `src/Policy/__Libraries/StellaOps.Policy.Unknowns/Services/UnknownRanker.cs`
|
||||
- Reason codes for conflict detection: VexConflict (conflicting VEX claims), Reachability (unknown reachability), Identity (unknown identity), Provenance (unknown provenance)
|
||||
- Uncertainty factor: Conflicting signals +0.20 to uncertainty score
|
||||
- Stale evidence +0.10 to uncertainty score
|
||||
- **K4Lattice conflict detection**: `src/Policy/__Libraries/StellaOps.Policy/TrustLattice/K4Lattice.cs`
|
||||
- K4Value.Conflict (value=3) when True join False occurs
|
||||
- Conflict detection in lattice evaluation routes to grey queue
|
||||
- **TrustLatticeEngine**: `src/Policy/__Libraries/StellaOps.Policy/TrustLattice/TrustLatticeEngine.cs`
|
||||
- Pipeline detects conflicts during VEX normalization and claim ingestion
|
||||
- Conflicting claims flagged for manual adjudication
|
||||
- **ClaimScoreMerger**: `src/Policy/__Libraries/StellaOps.Policy/TrustLattice/ClaimScoreMerger.cs`
|
||||
- ConflictPenalizer applies 0.25 penalty to conflicting claims
|
||||
- RequiresReplayProof flag set when conflicts detected
|
||||
- Deterministic merge ordering ensures reproducible conflict resolution
|
||||
- **DeterminizationGate**: `src/Policy/StellaOps.Policy.Engine/Gates/Determinization/DeterminizationGate.cs`
|
||||
- Gates using DecayedConfidenceCalculator for stale signal detection
|
||||
- Reanalysis triggering based on confidence decay below threshold
|
||||
- **ObservationDecay**: `src/Policy/__Libraries/StellaOps.Policy.Determinization/Models/ObservationDecay.cs`
|
||||
- Reanalysis fingerprint: ObservedAt + RefreshedAt + HalfLifeDays + DecayedMultiplier
|
||||
- CheckIsStale(now) triggers reanalysis when decay below StalenessThreshold
|
||||
|
||||
## E2E Test Plan
|
||||
- [ ] Ingest conflicting VEX claims (True + False for same subject); verify K4 Conflict state detected
|
||||
- [ ] Verify conflicting claims route to grey queue with Disputed state
|
||||
- [ ] Verify ConflictPenalizer applies 0.25 penalty to conflicting claims
|
||||
- [ ] Verify RequiresReplayProof=true when conflicts detected
|
||||
- [ ] Verify UnknownRanker adds +0.20 uncertainty for VexConflict reason code
|
||||
- [ ] Detect stale reachability evidence; verify reanalysis triggered (ObservationDecay.CheckIsStale)
|
||||
- [ ] Verify reanalysis fingerprint is deterministic: same inputs produce same fingerprint
|
||||
- [ ] Verify versioned signal events: newer signal supersedes older signal
|
||||
- [ ] Verify manual adjudication gate prevents automated resolution of Disputed unknowns
|
||||
- [ ] Verify stale evidence adds +0.10 to uncertainty in UnknownRanker
|
||||
45
docs/features/checked/policy/unknowns-ranking-algorithm.md
Normal file
45
docs/features/checked/policy/unknowns-ranking-algorithm.md
Normal file
@@ -0,0 +1,45 @@
|
||||
# Unknowns Ranking Algorithm (HOT/WARM/COLD bands)
|
||||
|
||||
## Module
|
||||
Policy
|
||||
|
||||
## Status
|
||||
IMPLEMENTED
|
||||
|
||||
## Description
|
||||
Unknown ranker with weighted scoring (popularity, exploit potential, uncertainty density, centrality, staleness), HOT/WARM/COLD band assignment, and BlastRadius model. Database migration for blast radius/containment exists.
|
||||
|
||||
## Implementation Details
|
||||
- **UnknownRanker**: `src/Policy/__Libraries/StellaOps.Policy.Unknowns/Services/UnknownRanker.cs` (sealed class)
|
||||
- Two-factor scoring: Score = (Uncertainty * 50) + (ExploitPressure * 50)
|
||||
- **Uncertainty factors** (sum capped at 1.0):
|
||||
- Missing VEX: +0.40
|
||||
- Missing reachability: +0.30
|
||||
- Conflicting signals: +0.20
|
||||
- Stale evidence: +0.10
|
||||
- **Exploit pressure factors** (sum capped at 1.0):
|
||||
- KEV flagged: +0.50
|
||||
- EPSS >= 0.90: +0.30
|
||||
- EPSS >= 0.50: +0.15
|
||||
- CVSS >= 9.0: +0.05
|
||||
- **Containment reduction** (capped at 40%):
|
||||
- Isolated: 15%, NotNetFacing: 5%, NonRoot: 5%, Seccomp: 10%, FsRO: 10%, NetworkIsolated: 5%
|
||||
- **Time decay**: 7d=100%, 30d=90%, 90d=75%, 180d=60%, 365d=40%, >365d=20%
|
||||
- **Band assignment**: Hot >= 75, Warm >= 50, Cold >= 25, Negligible < 25
|
||||
- **Reason codes**: AnalyzerLimit, Reachability, Identity, Provenance, VexConflict, FeedGap, ConfigUnknown
|
||||
- **BlastRadius model**: `src/Policy/__Libraries/StellaOps.Policy.Unknowns/Models/BlastRadius.cs`
|
||||
- Dependents (int), NetFacing (bool), Privilege (string?)
|
||||
- **UnknownRankerOptions**: `src/Policy/__Libraries/StellaOps.Policy.Unknowns/Services/UnknownRankerOptions.cs` -- configurable weights and thresholds
|
||||
- **UnknownRankInput**: includes EpssScore, CvssScore, IsKev, ContainmentSignals, LastEvaluatedAt, ReasonCodes
|
||||
|
||||
## E2E Test Plan
|
||||
- [ ] Rank unknown with missing VEX + KEV flagged; verify score = (0.40*50) + (0.50*50) = 45; band=Warm
|
||||
- [ ] Rank unknown with missing VEX + missing reachability + KEV + EPSS=0.95; verify score = (0.70*50) + (0.80*50) = 75; band=Hot
|
||||
- [ ] Rank unknown with no uncertainty + no exploit pressure; verify score=0; band=Negligible
|
||||
- [ ] Rank unknown with Isolated containment; verify 15% reduction from raw score
|
||||
- [ ] Rank unknown with all containment signals; verify exactly 40% reduction (cap)
|
||||
- [ ] Rank unknown last evaluated 90 days ago; verify decay factor=0.75 applied
|
||||
- [ ] Rank unknown last evaluated 365 days ago; verify decay factor=0.40 applied
|
||||
- [ ] Verify band assignment: score 80 -> Hot, score 55 -> Warm, score 30 -> Cold, score 10 -> Negligible
|
||||
- [ ] Verify reason codes: unknown with VexConflict produces reason=VexConflict
|
||||
- [ ] Verify deterministic ranking: same inputs always produce same score and band
|
||||
@@ -0,0 +1,42 @@
|
||||
# Verdict Explainability / Rationale Renderer
|
||||
|
||||
## Module
|
||||
Policy
|
||||
|
||||
## Status
|
||||
IMPLEMENTED
|
||||
|
||||
## Description
|
||||
Verdict rationale renderer and rationale model in Policy Explainability library. Testing infrastructure includes explainability assertions, IExplainableDecision interface, and explainability models.
|
||||
|
||||
## Implementation Details
|
||||
- **VerdictRationaleRenderer**: `src/Policy/__Libraries/StellaOps.Policy.Explainability/VerdictRationaleRenderer.cs` (sealed class implements `IVerdictRationaleRenderer`)
|
||||
- `Render(VerdictRationaleInput)` produces structured 4-line rationale
|
||||
- `RenderPlainText(rationale)` produces 4-line plain text output
|
||||
- `RenderMarkdown(rationale)` produces Markdown with ## headers (Evidence, Policy Clause, Attestations, Decision)
|
||||
- `RenderJson(rationale)` produces canonical JSON (RFC 8785) via `CanonJson.Serialize`
|
||||
- Content-addressed RationaleId: `rat:sha256:{hash}` computed from SHA256 of canonical JSON
|
||||
- Evidence rendering: CVE ID, component PURL/name/version, reachability (vulnerable function, entry point, path summary)
|
||||
- Policy clause rendering: ClauseId, RuleDescription, Conditions
|
||||
- Attestation rendering: path witness, VEX statements, provenance references
|
||||
- Decision rendering: verdict, score, recommendation, mitigation (action, details)
|
||||
- **VerdictRationale model**: `src/Policy/__Libraries/StellaOps.Policy.Explainability/VerdictRationale.cs`
|
||||
- SchemaVersion: "1.0"
|
||||
- 4-line template: RationaleEvidence, RationalePolicyClause, RationaleAttestations, RationaleDecision
|
||||
- RationaleInputDigests: VerdictDigest, PolicyDigest, EvidenceDigest for reproducibility
|
||||
- Supporting records: ComponentIdentity, ReachabilityDetail, AttestationReference, MitigationGuidance
|
||||
- **IVerdictRationaleRenderer**: `src/Policy/__Libraries/StellaOps.Policy.Explainability/IVerdictRationaleRenderer.cs`
|
||||
- Interface with Render, RenderPlainText, RenderMarkdown, RenderJson methods
|
||||
- VerdictRationaleInput record with full input specification
|
||||
|
||||
## E2E Test Plan
|
||||
- [ ] Render rationale for CVE-2024-1234 in lodash@4.17.21 with reachability; verify Evidence.FormattedText contains CVE, component, vulnerable function
|
||||
- [ ] Render rationale with policy clause "require-vex-for-critical"; verify PolicyClause.FormattedText includes clause ID and conditions
|
||||
- [ ] Render rationale with 2 VEX attestation references; verify Attestations.FormattedText includes both
|
||||
- [ ] Render rationale without attestations; verify FormattedText says "No attestations available."
|
||||
- [ ] Render same input twice; verify RationaleId is identical (content-addressed determinism)
|
||||
- [ ] Render with score=0.85 and mitigation; verify Decision.FormattedText includes "score 0.85" and mitigation action
|
||||
- [ ] RenderPlainText produces 4-line output (evidence, clause, attestations, decision)
|
||||
- [ ] RenderMarkdown produces valid Markdown with ## headers
|
||||
- [ ] RenderJson produces valid JSON parseable by standard parser
|
||||
- [ ] Verify RationaleId matches format `rat:sha256:{64 hex chars}`
|
||||
58
docs/features/checked/policy/versioned-weight-manifests.md
Normal file
58
docs/features/checked/policy/versioned-weight-manifests.md
Normal file
@@ -0,0 +1,58 @@
|
||||
# Versioned Weight Manifests
|
||||
|
||||
## Module
|
||||
Policy
|
||||
|
||||
## Status
|
||||
IMPLEMENTED
|
||||
|
||||
## Description
|
||||
Initial weight manifest file exists, but the weight manifest infrastructure (loading, versioning, hashing, CLI management) is marked TODO in the sprint (TSF-001).
|
||||
|
||||
## What's Implemented
|
||||
- **Weight manifest file**: `etc/weights/v2026-01-22.weights.json`
|
||||
- Schema: `https://stella-ops.org/schemas/weight-manifest/v1.0.0`
|
||||
- Schema version: 1.0.0, version: v2026-01-22, profile: production
|
||||
- Legacy 6-dimension weights: RCH=0.30, RTS=0.25, BKP=0.15, XPL=0.15, SRC=0.10, MIT=0.10
|
||||
- Advisory 5-dimension weights: CVSS=0.25, EPSS=0.30, Reachability=0.20, ExploitMaturity=0.10, PatchProof=0.15
|
||||
- Dimension names mapping (human-readable)
|
||||
- Subtractive dimensions: MIT, patchProof
|
||||
- Guardrails: notAffectedCap (maxScore=15, requires BKP>=1.0 and RTS<=0.6), runtimeFloor (minScore=60, requires RTS>=0.8), speculativeCap (maxScore=45, requires RCH<=0.0 and RTS<=0.0)
|
||||
- Priority buckets: actNowMin=90, scheduleNextMin=70, investigateMin=40
|
||||
- Determinization thresholds: manualReviewEntropy=0.60, refreshEntropy=0.40
|
||||
- Signal weights for entropy: VEX=0.25, Reachability=0.25, EPSS=0.15, Runtime=0.15, Backport=0.10, SBOMLineage=0.10
|
||||
- Content hash: `sha256:auto` (placeholder for computed hash)
|
||||
- Metadata: changelog, creation date, notes
|
||||
- **SignalWeights record**: `src/Policy/__Libraries/StellaOps.Policy.Determinization/Scoring/SignalWeights.cs`
|
||||
- Matches the signalWeightsForEntropy values from the manifest
|
||||
- **ScoringRulesSnapshot**: `src/Policy/__Libraries/StellaOps.Policy/Scoring/ScoringRulesSnapshot.cs`
|
||||
- Content-addressed snapshots with SHA256 digest
|
||||
- Builder pattern with WithWeights, WithThresholds, WithSeverityMultipliers, etc.
|
||||
- `IScoringRulesSnapshotService` interface for CRUD operations
|
||||
- **ScorePolicyLoader**: `src/Policy/__Libraries/StellaOps.Policy/Scoring/ScorePolicyLoader.cs`
|
||||
- YAML policy loading with version and weight sum validation
|
||||
- **ScorePolicyValidator**: `src/Policy/__Libraries/StellaOps.Policy/Scoring/ScorePolicyValidator.cs`
|
||||
- JSON Schema validation for score policies
|
||||
|
||||
## Additional Implementation Found
|
||||
- **FileBasedWeightManifestLoader**: `src/Signals/StellaOps.Signals/EvidenceWeightedScore/FileBasedWeightManifestLoader.cs` -- loads manifests from `etc/weights/*.json` files, implements `IWeightManifestLoader`
|
||||
- **ScoringManifestVersioner**: `src/__Libraries/StellaOps.DeltaVerdict/Manifest/ScoringManifestVersioner.cs` (with `.Compare.cs`, `.Compare.Helpers.cs`) -- manifest versioning with compare, bump, and generate-next-version capabilities
|
||||
- **ScoringManifestSigningService**: `src/__Libraries/StellaOps.DeltaVerdict/` -- manifest signing with KMS integration and Rekor anchoring
|
||||
- **Extensive tests**: `src/__Libraries/__Tests/StellaOps.DeltaVerdict.Tests/Manifest/` -- 7 test files covering versioning, comparison, bumping, signing
|
||||
|
||||
## What's Missing
|
||||
- **CLI management commands**: No `stella weights list`, `stella weights validate`, `stella weights diff`, or `stella weights activate` CLI commands wrapping the existing loader/versioner
|
||||
- **Content hash auto-compute at build**: Manifest has `"contentHash": "sha256:auto"` placeholder -- no build step replaces it with actual computed hash
|
||||
- **Unified binding**: FileBasedWeightManifestLoader is in Signals, ScoringManifestVersioner is in DeltaVerdict; no unified service in the Policy module that binds manifest loading, versioning, signing, and runtime configuration together
|
||||
|
||||
## Implementation Plan
|
||||
- Create `WeightManifestLoader` service that discovers manifests in `etc/weights/`, validates schema, computes/verifies content hash, and selects by `effectiveFrom` date
|
||||
- Add build step to compute content hash and replace `sha256:auto` placeholder
|
||||
- Create CLI commands for manifest lifecycle management
|
||||
- Build manifest-to-runtime binding that configures SignalWeights and ScoringRulesSnapshot from the active manifest
|
||||
- Add manifest diff utility for comparing versions
|
||||
|
||||
## Related Documentation
|
||||
- Weight manifest: `etc/weights/v2026-01-22.weights.json`
|
||||
- Signal weights: `src/Policy/__Libraries/StellaOps.Policy.Determinization/Scoring/SignalWeights.cs`
|
||||
- Scoring rules snapshot: `src/Policy/__Libraries/StellaOps.Policy/Scoring/ScoringRulesSnapshot.cs`
|
||||
45
docs/features/checked/policy/vex-decisioning-engine.md
Normal file
45
docs/features/checked/policy/vex-decisioning-engine.md
Normal file
@@ -0,0 +1,45 @@
|
||||
# VEX Decisioning Engine (Not Just Ingestion) (Moat Score 4)
|
||||
|
||||
## Module
|
||||
Policy
|
||||
|
||||
## Status
|
||||
IMPLEMENTED
|
||||
|
||||
## Description
|
||||
Full VEX decisioning with consensus engine, trust scoring, OpenVEX and CSAF normalization, and trust lattice conflict resolution.
|
||||
|
||||
## Implementation Details
|
||||
- **TrustLatticeEngine**: `src/Policy/__Libraries/StellaOps.Policy/TrustLattice/TrustLatticeEngine.cs`
|
||||
- Full VEX decisioning pipeline: VEX normalization -> claim ingestion -> K4 evaluation -> disposition selection -> proof bundle
|
||||
- Three VEX format normalizers: CycloneDX, OpenVEX, CSAF
|
||||
- Fluent ClaimBuilder: Assert, Present, Applies, Reachable, Mitigated, Fixed, Misattributed
|
||||
- `Evaluate()` with optional SubjectFilter and proof bundle generation
|
||||
- Disposition selection from K4 lattice values: True -> not_affected, False -> affected, Conflict -> disputed, Unknown -> under_investigation
|
||||
- **K4Lattice**: `src/Policy/__Libraries/StellaOps.Policy/TrustLattice/K4Lattice.cs`
|
||||
- Belnap four-valued logic: Unknown=0, True=1, False=2, Conflict=3
|
||||
- Join (consensus): T join F = Conflict; commutative, idempotent
|
||||
- Meet (agreement): T meet F = Unknown
|
||||
- FromSupport(): maps evidence support to K4 value
|
||||
- **ClaimScoreMerger**: `src/Policy/__Libraries/StellaOps.Policy/TrustLattice/ClaimScoreMerger.cs`
|
||||
- Deterministic claim merging with conflict penalization (0.25 penalty)
|
||||
- Merge ordering: adjusted score -> specificity -> original score -> source ID -> index
|
||||
- MergePolicy: ConflictPenalty, PreferSpecificity, RequireReplayProofOnConflict
|
||||
- Returns MergeResult: winning claim, conflicts, RequiresReplayProof flag
|
||||
- **VexTrustGate**: `src/Policy/StellaOps.Policy.Engine/Gates/VexTrustGate.cs`
|
||||
- Gates policy decisions based on VEX trust scores
|
||||
- Per-environment thresholds with FailureAction (Warn/Block)
|
||||
- **PolicyGateEvaluator**: `src/Policy/StellaOps.Policy.Engine/Gates/PolicyGateEvaluator.cs`
|
||||
- VEX Trust gate (3rd in 5-gate pipeline) evaluates trust lattice results
|
||||
|
||||
## E2E Test Plan
|
||||
- [ ] Ingest CycloneDX VEX with not_affected status; verify K4 value=True after normalization
|
||||
- [ ] Ingest OpenVEX with affected status; verify K4 value=False after normalization
|
||||
- [ ] Ingest CSAF VEX with fixed status; verify K4 value=True after normalization
|
||||
- [ ] Ingest conflicting claims (CycloneDX not_affected + OpenVEX affected); verify K4 Conflict, disposition=disputed
|
||||
- [ ] Verify ClaimScoreMerger: signed claim (score 0.9) wins over unsigned claim (score 0.95) when PreferSpecificity enabled
|
||||
- [ ] Verify conflict penalization: conflicting claims receive 0.25 penalty
|
||||
- [ ] Evaluate with VexTrustGate in production (threshold 0.80); provide trust score 0.85; verify gate passes
|
||||
- [ ] Evaluate with VexTrustGate; provide trust score 0.60 in production; verify gate blocks
|
||||
- [ ] Build claim via fluent API: Assert("CVE-2024-1234").Present("pkg:npm/lodash@4.17.21").Mitigated(); verify claim correctly formed
|
||||
- [ ] Verify proof bundle includes all claims, scores, and K4 evaluations for audit
|
||||
40
docs/features/checked/policy/vex-format-normalization.md
Normal file
40
docs/features/checked/policy/vex-format-normalization.md
Normal file
@@ -0,0 +1,40 @@
|
||||
# VEX Format Normalization (CycloneDX, OpenVEX, CSAF)
|
||||
|
||||
## Module
|
||||
Policy
|
||||
|
||||
## Status
|
||||
IMPLEMENTED
|
||||
|
||||
## Description
|
||||
Normalizers for CSAF and OpenVEX formats to convert heterogeneous VEX statements into the unified trust lattice representation.
|
||||
|
||||
## Implementation Details
|
||||
- **TrustLatticeEngine**: `src/Policy/__Libraries/StellaOps.Policy/TrustLattice/TrustLatticeEngine.cs`
|
||||
- Three VEX format normalizers integrated into evaluation pipeline:
|
||||
- CycloneDX normalizer: converts CycloneDX VEX analysis states to K4 claims
|
||||
- OpenVEX normalizer: converts OpenVEX status to K4 claims
|
||||
- CSAF normalizer: converts CSAF product status to K4 claims
|
||||
- All normalizers produce unified claim objects for K4 lattice evaluation
|
||||
- Format-specific metadata preserved in claim provenance
|
||||
- **K4Lattice**: `src/Policy/__Libraries/StellaOps.Policy/TrustLattice/K4Lattice.cs`
|
||||
- Unified representation: Unknown=0, True=1, False=2, Conflict=3
|
||||
- `FromSupport()` maps normalized evidence to K4 values
|
||||
- **ClaimBuilder**: `src/Policy/__Libraries/StellaOps.Policy/TrustLattice/TrustLatticeEngine.cs`
|
||||
- Fluent API for building claims from any format:
|
||||
- Assert(cve).Present(component).Mitigated() -> K4 True
|
||||
- Assert(cve).Present(component).Applies() -> K4 False (affected)
|
||||
- Assert(cve).Present(component).Fixed() -> K4 True (fixed version)
|
||||
- Assert(cve).Present(component).Misattributed() -> K4 True (not applicable)
|
||||
- **Trust lattice directory**: `src/Policy/__Libraries/StellaOps.Policy/TrustLattice/` (15 files total)
|
||||
|
||||
## E2E Test Plan
|
||||
- [ ] Normalize CycloneDX VEX with status "not_affected" and justification "code_not_reachable"; verify K4 True claim with correct provenance
|
||||
- [ ] Normalize OpenVEX with status "affected"; verify K4 False claim
|
||||
- [ ] Normalize CSAF with status "known_affected" and remediation "vendor_fix"; verify K4 claim reflects affected + fix available
|
||||
- [ ] Normalize CycloneDX VEX with status "fixed"; verify K4 True claim (vulnerability fixed)
|
||||
- [ ] Normalize all 3 formats for same CVE; merge via ClaimScoreMerger; verify deterministic result
|
||||
- [ ] Normalize VEX with invalid format; verify error handling (parse failure does not crash pipeline)
|
||||
- [ ] Verify format-specific metadata preserved: CycloneDX justification, OpenVEX statement, CSAF product_status
|
||||
- [ ] Normalize VEX from unknown format; verify treated as Unknown K4 value
|
||||
- [ ] Verify all normalizers produce claims compatible with K4Lattice.Join() and Meet()
|
||||
40
docs/features/checked/policy/vex-status-promotion-gate.md
Normal file
40
docs/features/checked/policy/vex-status-promotion-gate.md
Normal file
@@ -0,0 +1,40 @@
|
||||
# VEX Status Promotion Gate
|
||||
|
||||
## Module
|
||||
Policy
|
||||
|
||||
## Status
|
||||
IMPLEMENTED
|
||||
|
||||
## Description
|
||||
Promotion gate that blocks environment promotions based on VEX status thresholds, ensuring only properly triaged artifacts can advance.
|
||||
|
||||
## Implementation Details
|
||||
- **VexTrustGate**: `src/Policy/StellaOps.Policy.Engine/Gates/VexTrustGate.cs` (implements `IVexTrustGate`)
|
||||
- `EvaluateAsync(VexTrustGateRequest)` evaluates VEX trust for status transitions
|
||||
- VexTrustGateRequest: RequestedStatus, Environment, VexTrustStatus, TenantId
|
||||
- VexTrustStatus: TrustScore (0.0-1.0), PolicyTrustThreshold, MeetsPolicyThreshold, TrustBreakdown
|
||||
- Per-environment evaluation: production requires highest trust, development most permissive
|
||||
- **VexTrustGateOptions**: `src/Policy/StellaOps.Policy.Engine/Gates/VexTrustGateOptions.cs`
|
||||
- ApplyToStatuses: ["not_affected", "fixed"] -- statuses requiring trust verification
|
||||
- Per-environment thresholds:
|
||||
- Production: MinCompositeScore=0.80, RequireIssuerVerified=true, MinAccuracyRate=0.85, AcceptableFreshness=["fresh"], FailureAction=Block
|
||||
- Staging: MinCompositeScore=0.60, RequireIssuerVerified=true, AcceptableFreshness=["fresh","stale"], FailureAction=Warn
|
||||
- Development: MinCompositeScore=0.40, RequireIssuerVerified=false, AcceptableFreshness=["fresh","stale","superseded"], FailureAction=Warn
|
||||
- MissingTrustBehavior: Allow, Warn, Block
|
||||
- TenantOverrides for tenant-specific thresholds
|
||||
- **PolicyGateEvaluator**: `src/Policy/StellaOps.Policy.Engine/Gates/PolicyGateEvaluator.cs`
|
||||
- VEX Trust gate (3rd in pipeline) blocks promotion when trust insufficient
|
||||
- VEX trust evaluation integrated with lattice state and uncertainty tier gates
|
||||
|
||||
## E2E Test Plan
|
||||
- [ ] Request promotion to production with trust score 0.85; verify gate passes
|
||||
- [ ] Request promotion to production with trust score 0.70; verify gate blocks (threshold 0.80)
|
||||
- [ ] Request promotion to staging with trust score 0.65; verify gate passes (threshold 0.60)
|
||||
- [ ] Request promotion with stale VEX in production; verify gate blocks (only "fresh" acceptable)
|
||||
- [ ] Request promotion with stale VEX in staging; verify gate passes (stale acceptable)
|
||||
- [ ] Request promotion with unverified issuer in production; verify gate blocks
|
||||
- [ ] Request promotion with unverified issuer in development; verify gate passes
|
||||
- [ ] Request promotion with MissingTrustBehavior=Block and no VEX data; verify gate blocks
|
||||
- [ ] Verify tenant-specific overrides apply when TenantId matches
|
||||
- [ ] Verify gate returns descriptive message identifying which threshold was not met
|
||||
@@ -0,0 +1,47 @@
|
||||
# VEX Trust Lattice with Provenance/Coverage/Replayability Scoring
|
||||
|
||||
## Module
|
||||
Policy
|
||||
|
||||
## Status
|
||||
IMPLEMENTED
|
||||
|
||||
## Description
|
||||
Full trust lattice engine with claim score merging, conflict penalization, trust labels, and configurable trust source weights per the advisory's P/C/R model.
|
||||
|
||||
## Implementation Details
|
||||
- **TrustLatticeEngine**: `src/Policy/__Libraries/StellaOps.Policy/TrustLattice/TrustLatticeEngine.cs`
|
||||
- P/C/R scoring model: Provenance, Coverage, Replayability
|
||||
- Pipeline: VEX normalization -> claim ingestion -> K4 evaluation -> disposition selection -> proof bundle
|
||||
- Trust labels derived from K4 lattice values and claim scores
|
||||
- Configurable trust source weights per claim source
|
||||
- **K4Lattice**: `src/Policy/__Libraries/StellaOps.Policy/TrustLattice/K4Lattice.cs`
|
||||
- Four-valued trust algebra: Unknown=0, True=1, False=2, Conflict=3
|
||||
- Join: T join F = Conflict (contradictory evidence produces conflict)
|
||||
- Meet: T meet F = Unknown (no agreement reduces to unknown)
|
||||
- LessOrEqual: lattice ordering for trust comparisons
|
||||
- **ClaimScoreMerger**: `src/Policy/__Libraries/StellaOps.Policy/TrustLattice/ClaimScoreMerger.cs`
|
||||
- Deterministic merge with configurable MergePolicy
|
||||
- ConflictPenalizer: applies 0.25 penalty to conflicting claims
|
||||
- PreferSpecificity: more specific claims (exact PURL vs wildcard) preferred
|
||||
- RequireReplayProofOnConflict: flags conflicts requiring replay verification
|
||||
- Merge ordering: adjusted score -> specificity -> original score -> source ID -> index
|
||||
- Returns MergeResult: winning claim, conflicts list, RequiresReplayProof flag
|
||||
- **VexTrustGateOptions**: `src/Policy/StellaOps.Policy.Engine/Gates/VexTrustGateOptions.cs`
|
||||
- Per-environment trust thresholds with P/C/R scoring integration
|
||||
- Production: MinCompositeScore=0.80 (requires high P/C/R)
|
||||
- Development: MinCompositeScore=0.40 (accepts lower P/C/R)
|
||||
- **Trust lattice files** (15 files in `src/Policy/__Libraries/StellaOps.Policy/TrustLattice/`):
|
||||
- Normalizers, claim builders, score mergers, lattice algebra
|
||||
|
||||
## E2E Test Plan
|
||||
- [ ] Evaluate claim with high Provenance (signed, verified issuer) + high Coverage (exact CVE+PURL match) + high Replayability (deterministic); verify high composite trust score
|
||||
- [ ] Evaluate claim with low Provenance (unsigned) + low Coverage (wildcard match); verify low composite score
|
||||
- [ ] Merge 2 claims: one with high P/C/R, one with low P/C/R; verify high-scoring claim wins
|
||||
- [ ] Merge conflicting claims; verify ConflictPenalizer reduces both scores by 0.25
|
||||
- [ ] Verify PreferSpecificity: exact PURL claim beats wildcard PURL claim even with lower raw score
|
||||
- [ ] Verify RequiresReplayProof flag is set when conflicting claims are merged
|
||||
- [ ] Verify deterministic merge: same inputs always produce same MergeResult
|
||||
- [ ] Evaluate trust lattice with Join(True, False); verify K4 Conflict
|
||||
- [ ] Evaluate trust lattice with Meet(True, False); verify K4 Unknown
|
||||
- [ ] Verify trust labels derived from K4 values: True -> "trusted", False -> "untrusted", Conflict -> "disputed", Unknown -> "unverified"
|
||||
@@ -0,0 +1,43 @@
|
||||
# VexTrustGate Policy Integration
|
||||
|
||||
## Module
|
||||
Policy
|
||||
|
||||
## Status
|
||||
IMPLEMENTED
|
||||
|
||||
## Description
|
||||
Integrates VEX trust evaluation as a named policy gate in the policy evaluation chain. VexTrustGate validates VEX statement trust levels against configurable thresholds before accepting VEX-based risk reductions. Registered in the GateSelector alongside existing gates. Distinct from known "VEX Trust Scoring" (which computes scores) -- this gates policy decisions based on those scores.
|
||||
|
||||
## Implementation Details
|
||||
- **VexTrustGate**: `src/Policy/StellaOps.Policy.Engine/Gates/VexTrustGate.cs` (implements `IVexTrustGate`)
|
||||
- `EvaluateAsync(VexTrustGateRequest)` evaluates trust score against thresholds
|
||||
- Returns VexTrustGateResult with decision (Pass/Warn/Block), details, and recommendations
|
||||
- Checks: composite score >= threshold, issuer verified, accuracy rate, freshness
|
||||
- **VexTrustGateOptions**: `src/Policy/StellaOps.Policy.Engine/Gates/VexTrustGateOptions.cs`
|
||||
- Section key: `"Policy:Gates:VexTrust"`
|
||||
- Enabled flag (default true)
|
||||
- ApplyToStatuses: ["not_affected", "fixed"]
|
||||
- Per-environment thresholds (production/staging/development/default)
|
||||
- MissingTrustBehavior: Allow, Warn, Block
|
||||
- EmitMetrics: true (OpenTelemetry)
|
||||
- TenantOverrides for multi-tenant deployments
|
||||
- **PolicyGateEvaluator integration**: `src/Policy/StellaOps.Policy.Engine/Gates/PolicyGateEvaluator.cs`
|
||||
- VEX Trust gate is 3rd in the 5-gate sequential pipeline
|
||||
- Evaluates after Evidence Completeness and Lattice State gates
|
||||
- Per-environment thresholds with MissingTrustBehavior fallback
|
||||
- **VexTrustGateMetrics**: `src/Policy/StellaOps.Policy.Engine/Gates/VexTrustGateMetrics.cs`
|
||||
- OpenTelemetry metrics: gate decisions, trust scores, evaluation duration
|
||||
- Tags: environment, decision, trust_score_bucket
|
||||
|
||||
## E2E Test Plan
|
||||
- [ ] Evaluate VEX trust in production with score 0.85 (above 0.80 threshold); verify gate passes
|
||||
- [ ] Evaluate VEX trust in production with score 0.75 (below 0.80 threshold); verify gate blocks
|
||||
- [ ] Evaluate VEX trust in staging with score 0.55 (below 0.60 threshold); verify gate warns (FailureAction=Warn)
|
||||
- [ ] Evaluate VEX trust in development with score 0.35 (below 0.40 threshold); verify gate warns
|
||||
- [ ] Evaluate without VEX trust data, MissingTrustBehavior=Warn; verify gate warns with descriptive message
|
||||
- [ ] Evaluate without VEX trust data, MissingTrustBehavior=Block; verify gate blocks
|
||||
- [ ] Evaluate without VEX trust data, MissingTrustBehavior=Allow; verify gate passes
|
||||
- [ ] Configure TenantOverrides for tenant-A with custom staging threshold 0.70; evaluate in staging for tenant-A; verify custom threshold used
|
||||
- [ ] Verify VexTrustGateMetrics emits gate decision metric with environment and decision tags
|
||||
- [ ] Disable VexTrustGate (Enabled=false); evaluate; verify gate is skipped in pipeline
|
||||
Reference in New Issue
Block a user