up

docs/implplan/SPRINT_124_policy_reasoning.md

@@ -1,6 +1,6 @@
 # Sprint 124 - Policy & Reasoning
 
-_Last updated: November 8, 2025. Implementation order is DOING → TODO → BLOCKED._
+_Last updated: November 28, 2025. Implementation order is DOING → TODO → BLOCKED._
 
 Focus areas below were split out of the previous combined sprint; execute sections in order unless noted.
 
@@ -18,8 +18,8 @@ Focus: Policy & Reasoning focus on Policy (phase II).
 | 5 | POLICY-ENGINE-20-005 | DONE (2025-11-27) | Determinism guard implemented in `src/Policy/StellaOps.Policy.Engine/DeterminismGuard/` with static analyzer (`ProhibitedPatternAnalyzer`), runtime sandbox (`DeterminismGuardService`, `EvaluationScope`), and guarded evaluator integration (`GuardedPolicyEvaluator`) | Policy Guild, Security Engineering / src/Policy/StellaOps.Policy.Engine |
 | 6 | POLICY-ENGINE-20-006 | DONE (2025-11-27) | Incremental orchestrator implemented in `src/Policy/StellaOps.Policy.Engine/IncrementalOrchestrator/` with `PolicyChangeEvent` models (advisory/VEX/SBOM change types), `IncrementalPolicyOrchestrator` (batching, deduplication, retry logic), and `IncrementalOrchestratorBackgroundService` (continuous processing, metrics) | Policy Guild, Scheduler Worker Guild / src/Policy/StellaOps.Policy.Engine |
 | 7 | POLICY-ENGINE-20-007 | DONE (2025-11-27) | Structured traces implemented in `src/Policy/StellaOps.Policy.Engine/Telemetry/` with `RuleHitTrace.cs` (trace models, statistics), `RuleHitTraceCollector.cs` (sampling controls, exporters), and `ExplainTraceExport.cs` (JSON/NDJSON/Text/Markdown export formats) | Policy Guild, Observability Guild / src/Policy/StellaOps.Policy.Engine |
-| 8 | POLICY-ENGINE-20-008 | TODO | Add unit/property/golden/perf suites covering policy compilation, evaluation correctness, determinism, and SLA targets (Deps: POLICY-ENGINE-20-007) | Policy Guild, QA Guild / src/Policy/StellaOps.Policy.Engine |
-| 9 | POLICY-ENGINE-20-009 | TODO | Define Mongo schemas/indexes for `policies`, `policy_runs`, and `effective_finding_*`; implement migrations and tenant enforcement (Deps: POLICY-ENGINE-20-008) | Policy Guild, Storage Guild / src/Policy/StellaOps.Policy.Engine |
+| 8 | POLICY-ENGINE-20-008 | DONE (2025-11-28) | Unit test suites added in `src/Policy/__Tests/StellaOps.Policy.Engine.Tests/` for DeterminismGuard, SelectionJoin, IncrementalOrchestrator, Materialization, and Telemetry components (99 tests passing) | Policy Guild, QA Guild / src/Policy/StellaOps.Policy.Engine |
+| 9 | POLICY-ENGINE-20-009 | DONE (2025-11-28) | MongoDB schemas implemented in `src/Policy/StellaOps.Policy.Engine/Storage/Mongo/` with document classes (`PolicyDocuments.cs`, `PolicyRunDocument.cs`, `EffectiveFindingDocument.cs`, `PolicyAuditDocument.cs`), options (`PolicyEngineMongoOptions.cs`), context (`PolicyEngineMongoContext.cs`), migrations (`EnsurePolicyCollectionsMigration.cs`, `EnsurePolicyIndexesMigration.cs`, `EffectiveFindingCollectionInitializer.cs`), migration runner, and tenant enforcement (`TenantFilterBuilder.cs`) | Policy Guild, Storage Guild / src/Policy/StellaOps.Policy.Engine |
 | 10 | POLICY-ENGINE-27-001 | TODO | Extend compile outputs to include rule coverage metadata, symbol table, inline documentation, and rule index for editor autocomplete; persist deterministic hashes (Deps: POLICY-ENGINE-20-009) | Policy Guild / src/Policy/StellaOps.Policy.Engine |
 | 11 | POLICY-ENGINE-27-002 | TODO | Enhance simulate endpoints to emit rule firing counts, heatmap aggregates, sampled explain traces with deterministic ordering, and delta summaries for quick/batch sims (Deps: POLICY-ENGINE-27-001) | Policy Guild, Observability Guild / src/Policy/StellaOps.Policy.Engine |
 | 12 | POLICY-ENGINE-29-001 | TODO | Implement batch evaluation endpoint (`POST /policy/eval/batch`) returning determinations + rationale chain for sets of `(artifact,purl,version,advisory)` tuples; support pagination and cost budgets (Deps: POLICY-ENGINE-27-004) | Policy Guild / src/Policy/StellaOps.Policy.Engine |
@@ -29,6 +29,9 @@ Focus: Policy & Reasoning focus on Policy (phase II).
 ## Execution Log
 | Date (UTC) | Update | Owner |
 | --- | --- | --- |
+| 2025-11-28 | POLICY-ENGINE-20-009: Completed MongoDB storage layer - document schemas for policies, policy_revisions, policy_bundles, policy_runs, effective_finding_*, effective_finding_history_*, and policy_audit collections. Created `PolicyEngineMongoOptions.cs` (connection/collection configuration with TTL settings), `PolicyEngineMongoContext.cs` (database access with read/write concerns), migration infrastructure (`IPolicyEngineMongoMigration`, `PolicyEngineMigrationRunner`, `PolicyEngineMongoInitializer`), `EnsurePolicyCollectionsMigration.cs` (creates base collections), `EnsurePolicyIndexesMigration.cs` (indexes for policies, revisions, bundles, runs, audit), `EffectiveFindingCollectionInitializer.cs` (dynamic per-policy collection creation with indexes), `TenantFilterBuilder.cs` (tenant enforcement utilities), and `ServiceCollectionExtensions.cs` (DI registration). Status → DONE. | Implementer |
+| 2025-11-28 | POLICY-ENGINE-20-008: Completed unit test suites - `DeterminismGuardTests.cs` (static analyzer, runtime sandbox, guarded evaluator), `SelectionJoinTests.cs` (PURL equivalence, tuple resolution, VEX overlay), `IncrementalOrchestratorTests.cs` (event processing, deduplication, priority batching), `MaterializationTests.cs` (deterministic IDs, content hashing), `TelemetryTests.cs` (trace factory, statistics, sampling). 99 tests passing. Status → DONE. | Implementer |
+| 2025-11-27 | POLICY-ENGINE-20-007: Completed structured traces - `RuleHitTrace.cs` (trace models, factory, statistics aggregation), `RuleHitTraceCollector.cs` (sampling controls with VEX/severity-aware rates, incident mode, exporters), `ExplainTraceExport.cs` (JSON/NDJSON/Text/Markdown formats, builder pattern). Status → DONE. | Implementer |
 | 2025-11-27 | POLICY-ENGINE-20-006: Completed incremental orchestrator - `PolicyChangeEvent.cs` (change event models with factory for advisory/VEX/SBOM changes, deterministic content hashing, batching), `IncrementalPolicyOrchestrator.cs` (event processing with idempotency, retry logic, priority-based batching), `IncrementalOrchestratorBackgroundService.cs` (continuous processing with metrics). Status → DONE. | Implementer |
 | 2025-11-27 | POLICY-ENGINE-20-005: Completed determinism guard - `DeterminismViolation.cs` (violation models/options), `ProhibitedPatternAnalyzer.cs` (static analysis with regex patterns for DateTime.Now, Random, Guid.NewGuid, HttpClient, File.Read, etc.), `DeterminismGuardService.cs` (runtime sandbox with EvaluationScope, DeterministicTimeProvider), `GuardedPolicyEvaluator.cs` (integration layer). Status → DONE. | Implementer |
 | 2025-11-27 | POLICY-ENGINE-20-004: Completed materialization writer - `EffectiveFindingModels.cs` (document schema), `EffectiveFindingWriter.cs` (upsert + append-only history). Tenant-scoped collections, trace references, content hash deduplication. Status → DONE. | Implementer |