git.stella-ops.org/docs/implplan/SPRINT_126_policy_reasoning.md

Sprint 126 - Policy & Reasoning

Superseded by docs/implplan/SPRINT_0126_0001_0001_policy_reasoning.md; maintained for historical context only.

Last updated: November 8, 2025. Implementation order is DOING → TODO → BLOCKED.

Focus areas below were split out of the previous combined sprint; execute sections in order unless noted.

Policy.IV

Dependency: Sprint 120.C - Policy.III (must land before this track). Focus: Policy & Reasoning focus on Policy (phase IV).

#	Task ID & handle	State	Key dependency / next step	Owners
1	POLICY-ENGINE-40-003	DONE	Provide API/SDK utilities for consumers (Web Scanner, Graph Explorer) to request policy decisions with source evidence summaries (top severity sources, conflict counts) (Deps: POLICY-ENGINE-40-002)	Policy Guild, Web Scanner Guild / src/Policy/StellaOps.Policy.Engine
2	POLICY-ENGINE-50-001	DONE	Implement SPL compiler: validate YAML, canonicalize, produce signed bundle, store artifact in object storage, write policy_revisions with AOC metadata (Deps: POLICY-ENGINE-40-003)	Policy Guild, Platform Security / src/Policy/StellaOps.Policy.Engine
3	POLICY-ENGINE-50-002	DONE	Build runtime evaluator executing compiled plans over advisory/vex linksets + SBOM asset metadata with deterministic caching (Redis) and fallback path (Deps: POLICY-ENGINE-50-001)	Policy Guild, Runtime Guild / src/Policy/StellaOps.Policy.Engine
4	POLICY-ENGINE-50-003	DONE	Implement evaluation/compilation metrics, tracing, and structured logs (policy_eval_seconds, policy_compiles_total, explanation sampling) (Deps: POLICY-ENGINE-50-002)	Policy Guild, Observability Guild / src/Policy/StellaOps.Policy.Engine
5	POLICY-ENGINE-50-004	DONE	Build event pipeline: subscribe to linkset/SBOM updates, schedule re-eval jobs, emit policy.effective.updated events with diff metadata (Deps: POLICY-ENGINE-50-003)	Policy Guild, Platform Events Guild / src/Policy/StellaOps.Policy.Engine
6	POLICY-ENGINE-50-005	DONE	Design and implement policy_packs, policy_revisions, policy_runs, policy_artifacts collections with indexes, TTL, and tenant scoping (Deps: POLICY-ENGINE-50-004)	Policy Guild, Storage Guild / src/Policy/StellaOps.Policy.Engine
7	POLICY-ENGINE-50-006	DONE	Implement explainer persistence + retrieval APIs linking decisions to explanation tree and AOC chain (Deps: POLICY-ENGINE-50-005)	Policy Guild, QA Guild / src/Policy/StellaOps.Policy.Engine
8	POLICY-ENGINE-50-007	DONE	Provide evaluation worker host/DI wiring and job orchestration hooks for batch re-evaluations after policy activation (Deps: POLICY-ENGINE-50-006)	Policy Guild, Scheduler Worker Guild / src/Policy/StellaOps.Policy.Engine
9	POLICY-ENGINE-60-001	DONE	Maintain Redis effective decision maps per asset/snapshot for Graph overlays; implement versioning and eviction strategy (Deps: POLICY-ENGINE-50-007)	Policy Guild, SBOM Service Guild / src/Policy/StellaOps.Policy.Engine
10	POLICY-ENGINE-60-002	DONE	Expose simulation bridge for Graph What-if APIs, supporting hypothetical SBOM diffs and draft policies without persisting results (Deps: POLICY-ENGINE-60-001)	Policy Guild, BE-Base Platform Guild / src/Policy/StellaOps.Policy.Engine
11	POLICY-ENGINE-70-002	DONE	Design and create Mongo collections (exceptions, exception_reviews, exception_bindings) with indexes and migrations; expose repository APIs (Deps: POLICY-ENGINE-60-002)	Policy Guild, Storage Guild / src/Policy/StellaOps.Policy.Engine
12	POLICY-ENGINE-70-003	DONE	Build Redis exception decision cache (exceptions_effective_map) with warm/invalidation logic reacting to exception.* events (Deps: POLICY-ENGINE-70-002)	Policy Guild, Runtime Guild / src/Policy/StellaOps.Policy.Engine
13	POLICY-ENGINE-70-004	TODO	Extend metrics/tracing/logging for exception application (latency, counts, expiring events) and include AOC references in logs (Deps: POLICY-ENGINE-70-003)	Policy Guild, Observability Guild / src/Policy/StellaOps.Policy.Engine
14	POLICY-ENGINE-70-005	TODO	Provide APIs/workers hook for exception activation/expiry (auto start/end) and event emission (exception.activated/expired) (Deps: POLICY-ENGINE-70-004)	Policy Guild, Scheduler Worker Guild / src/Policy/StellaOps.Policy.Engine
15	POLICY-ENGINE-80-001	TODO	Integrate reachability/exploitability inputs into evaluation pipeline (state/score/confidence) with caching and explain support (Deps: POLICY-ENGINE-70-005)	Policy Guild, Signals Guild / src/Policy/StellaOps.Policy.Engine
16	POLICY-RISK-90-001	TODO	Ingest entropy penalty inputs from Scanner (entropy.report.json, layer_summary.json), extend trust algebra with configurable weights/caps, and expose explanations/metrics for opaque ratio penalties (docs/modules/scanner/entropy.md).	Policy Guild, Scanner Guild / src/Policy/StellaOps.Policy.Engine

Notes & Risks (2025-11-27)

• POLICY-ENGINE-40-003 implementation complete: Added PolicyDecisionModels.cs, PolicyDecisionService.cs, PolicyDecisionEndpoint.cs, and PolicyDecisionServiceTests.cs. Service registered in Program.cs. All 9 tests pass.
• POLICY-ENGINE-50-001 implementation complete: Extended SPL compiler with AOC (Attestation of Compliance) metadata support:
  • Added PolicyAocMetadata, PolicyProvenance, PolicyAttestationRef records to PolicyPackRecord.cs
  • Added PolicyProvenanceInput, PolicyAocMetadataResponse to PolicyBundleModels.cs
  • Updated PolicyBundleService to capture compilation ID, source/artifact digests, complexity metrics, provenance
  • Added 4 new tests for AOC metadata in PolicyBundleServiceTests.cs (all pass)
  • Existing YAML validation via PolicyBinder, canonicalization via PolicyCompilationService, signed bundles via PolicyBundleService, storage via IPolicyPackRepository all integrate with new AOC metadata
• Pre-existing test issue: EvidenceSummaryServiceTests.Summarize_BuildsDeterministicSummary fails due to date derivation mismatch (unrelated to current changes)
• Pre-existing build issues resolved:
  • StellaOps.Telemetry.Core: Fixed TelemetryContext API (added CorrelationId/TraceId aliases, Current/Context property aliases), added Grpc.AspNetCore package, removed duplicate FrameworkReference.
  • StellaOps.Policy.RiskProfile: Fixed JsonSchema.Net v5 API changes (ValidationResults → EvaluationResults), JsonDocument.Parse signature.
  • StellaOps.Policy.Engine: Fixed OpenTelemetry Meter API changes (observeValues parameter, nullable returns), SamplingResult API changes, parameter casing fixes.
  • Test project: Added Microsoft.Extensions.TimeProvider.Testing package, fixed using directives, fixed parameter casing.

Execution Log

Date (UTC)	Update	Owner
2025-11-27	Started POLICY-ENGINE-40-003; implemented PolicyDecisionService, PolicyDecisionEndpoint, PolicyDecisionModels, tests. Blocked by pre-existing build issues in Telemetry.Core and RiskProfile projects.	Implementer
2025-11-27	Fixed pre-existing build issues (TelemetryContext API mismatch, JsonSchema.Net v5 API changes, OpenTelemetry Meter API changes, test project missing packages/namespaces). All 9 PolicyDecisionServiceTests pass. POLICY-ENGINE-40-003 marked DONE.	Implementer
2025-11-27	Implemented POLICY-ENGINE-50-001: Extended SPL compiler with AOC metadata support. Added PolicyAocMetadata, PolicyProvenance, PolicyAttestationRef models. Updated PolicyBundleService to capture compilation metadata, source/artifact digests, complexity metrics, provenance info. Added 4 new tests (all pass). POLICY-ENGINE-50-001 marked DONE.	Implementer
2025-11-28	Implemented POLICY-ENGINE-50-002: Built runtime evaluator with deterministic caching. Created IPolicyEvaluationCache interface, InMemoryPolicyEvaluationCache implementation with TTL/eviction, PolicyRuntimeEvaluationService with batch evaluation support, cache key generation using SHA256 digests (policy, subject, context). Extended PolicyBundleRecord to store compiled PolicyIrDocument. Added 8 tests (all pass). POLICY-ENGINE-50-002 marked DONE.	Implementer
2025-11-28	Implemented POLICY-ENGINE-50-003: Integrated telemetry into PolicyCompilationService and PolicyRuntimeEvaluationService. Added OpenTelemetry Activity tracing for policy.compile, policy.evaluate, and policy.evaluate_batch operations. Integrated existing metrics (RecordCompilation, RecordEvaluation, RecordEvaluationLatency, RecordRuleFired, RecordError, RecordEvaluationFailure). Added structured logging with context (duration, rule counts, complexity, cache hits). All 23 core tests pass. POLICY-ENGINE-50-003 marked DONE.	Implementer
2025-11-28	Implemented POLICY-ENGINE-50-004: Built event pipeline for linkset/SBOM updates. Created PolicyEffectiveEventModels.cs with event types (policy.effective.updated, policy.effective.added, policy.effective.removed, policy.effective.batch_completed), PolicyDecisionDiff for diff metadata. Created PolicyEventProcessor.cs with IPolicyEffectiveEventPublisher, IReEvaluationJobScheduler interfaces. Processor handles PolicyChangeEvents, schedules re-evaluation jobs, and emits effective events with diffs. Added 3 new telemetry counters. Build succeeds. POLICY-ENGINE-50-004 marked DONE.	Implementer
2025-11-28	Implemented POLICY-ENGINE-50-005: MongoDB collections with tenant scoping and indexes. Pre-existing infrastructure includes: PolicyDocument, PolicyRevisionDocument, PolicyBundleDocument, PolicyRunDocument classes in Documents folder; EnsurePolicyIndexesMigration with TTL indexes for policy_runs collection; PolicyEngineMongoOptions for configuration. Created MongoPolicyPackRepository.cs implementing IPolicyPackRepository with tenant-scoped CRUD operations for policy packs, revisions, bundles; approval workflow; activation tracking. Fixed pre-existing bug in PolicyMetadataExtractor.cs (string comparisons for enum operators). All 11 core tests pass. POLICY-ENGINE-50-005 marked DONE.	Implementer
2025-11-28	Implemented POLICY-ENGINE-50-006: Explainer persistence and retrieval APIs with AOC chain linkage. Created PolicyExplainDocument.cs with MongoDB documents for explain traces including ExplainInputContextDocument, ExplainRuleStepDocument, ExplainVexEvidenceDocument, ExplainStatisticsDocument, ExplainAocChainDocument. Created PolicyExplainerService.cs with IExplainTraceRepository interface, StoredExplainTrace/ExplainAocChain records, ExplainQueryOptions for filtering/pagination, AocChainValidationResult for verifying attestation chain integrity. Service links explain traces to policy bundle AOC metadata (compilation ID, source/artifact digests, attestation references). Added policy_explain_traces_stored_total telemetry counter. Added PolicyExplainsCollection and ExplainTraceRetention to options. Added indexes for policy_explains collection (tenant_runId, tenant_policy_evaluatedAt_desc, tenant_subjectHash, aocChain_compilationId, expiresAt_ttl). All 11 core tests pass. POLICY-ENGINE-50-006 marked DONE.	Implementer
2025-11-28	Implemented POLICY-ENGINE-50-007: Evaluation worker host and DI wiring for job orchestration. Created PolicyEvaluationWorkerService.cs integrating with existing PolicyEventProcessor.DequeueJob() for job scheduling, with EvaluationJobResult record for tracking job outcomes. Created PolicyEvaluationWorkerHost.cs as BackgroundService with configurable concurrency from PolicyEngineWorkerOptions. Created PolicyEngineServiceCollectionExtensions.cs with AddPolicyEngineCore(), AddPolicyEngineEventPipeline(), AddPolicyEngineWorker(), AddPolicyEngineExplainer(), and combined AddPolicyEngine() extension methods. Worker integrates with existing IPolicyEffectiveEventPublisher and IReEvaluationJobScheduler interfaces. Added ScheduleActivationReEvalAsync() hook for triggering re-evaluations after policy activation. All 182 tests pass. POLICY-ENGINE-50-007 marked DONE.	Implementer
2025-11-28	Implemented POLICY-ENGINE-60-001: Redis effective decision maps for Graph overlays. Added StackExchange.Redis package. Created EffectiveDecisionMap/EffectiveDecisionModels.cs with EffectiveDecisionEntry, EffectiveDecisionQueryResult, EffectiveDecisionSummary, EffectiveDecisionFilter records for storing/querying policy decisions per asset/snapshot. Created EffectiveDecisionMap/IEffectiveDecisionMap.cs interface with Set/Get/Query/Invalidate operations plus versioning (GetVersionAsync, IncrementVersionAsync). Created EffectiveDecisionMap/RedisEffectiveDecisionMap.cs with TTL-based eviction using Redis key structure stellaops:edm:{tenant}:{snapshot}:e:{asset} for entries, :idx sorted sets for indexing, :v for version counters. Added EffectiveDecisionMapOptions to PolicyEngineOptions. Added policy_effective_decision_map_operations_total telemetry counter. Added AddEffectiveDecisionMap() and AddPolicyEngineRedis() DI extensions. All 182 tests pass. POLICY-ENGINE-60-001 marked DONE.	Implementer
2025-11-28	Implemented POLICY-ENGINE-60-002: What-If simulation bridge for Graph APIs. Created WhatIfSimulation/WhatIfSimulationModels.cs with comprehensive request/response models (WhatIfSimulationRequest, WhatIfSimulationResponse, WhatIfDraftPolicy, WhatIfSbomDiff, WhatIfDecisionChange, WhatIfDecision, WhatIfExplanation, WhatIfSummary, WhatIfImpact, WhatIfPolicyRef). Created WhatIfSimulation/WhatIfSimulationService.cs supporting: hypothetical SBOM diffs (add/remove/upgrade/downgrade operations), draft policy comparison, baseline decision lookup from effective decision map, simulated decision computation considering VEX status and reachability, change detection and diff computation, impact assessment with risk delta recommendations. Service integrates with IEffectiveDecisionMap for baseline lookups, IPolicyPackRepository for policy retrieval, PolicyCompilationService for potential on-the-fly compilation. Added AddWhatIfSimulation() DI extension. Telemetry via existing RecordSimulation() counter. All 181 core tests pass. POLICY-ENGINE-60-002 marked DONE.	Implementer
2025-11-28	Implemented POLICY-ENGINE-70-002: MongoDB collections for policy exceptions with indexes and repository APIs. Created Storage/Mongo/Documents/PolicyExceptionDocuments.cs with PolicyExceptionDocument (exceptions with scope, risk assessment, compensating controls, workflow states), ExceptionScopeDocument (advisory/CVE/PURL/asset targeting), ExceptionRiskAssessmentDocument (risk levels, justification), ExceptionReviewDocument (multi-reviewer approval workflow), ReviewDecisionDocument (individual decisions with conditions), ExceptionBindingDocument (asset-specific bindings with time ranges). Created Storage/Mongo/Repositories/IExceptionRepository.cs interface with CRUD operations for exceptions, reviews, and bindings; query options for filtering/pagination; methods for finding applicable exceptions, pending activations, expiring exceptions. Created Storage/Mongo/Repositories/MongoExceptionRepository.cs MongoDB implementation with tenant scoping. Added collection names to PolicyEngineMongoOptions (exceptions, exception_reviews, exception_bindings). Created Storage/Mongo/Migrations/EnsureExceptionIndexesMigration.cs with comprehensive indexes: tenant+status, tenant+type+status, tenant+created, tenant+tags, scope.advisoryIds, scope.assetIds, scope.cveIds, expiry tracking, reviewer queues, binding lookups. Added policy_exception_operations_total telemetry counter with RecordExceptionOperation() method. Registered migration and repository in ServiceCollectionExtensions. All 196 core tests pass. POLICY-ENGINE-70-002 marked DONE.	Implementer
2025-11-28	Implemented POLICY-ENGINE-70-003: Redis exception decision cache with warm/invalidation logic. Created ExceptionCache/ExceptionCacheModels.cs with ExceptionCacheEntry (cached exception for fast lookup with priority, decision override, expiry), ExceptionCacheQueryResult (query results with cache metadata), ExceptionCacheSummary (tenant summary with counts by type/decision), ExceptionCacheOptions (TTL, auto-warm, max entries), ExceptionCacheStats (hit/miss counts, memory usage). Created ExceptionCache/IExceptionEffectiveCache.cs interface with GetForAssetAsync, GetBatchAsync, SetAsync, SetBatchAsync, InvalidateExceptionAsync, InvalidateAssetAsync, InvalidateTenantAsync, WarmAsync, HandleExceptionEventAsync for event-driven invalidation; ExceptionEvent record for exception lifecycle events (activated, expired, revoked, updated, created, deleted). Created ExceptionCache/RedisExceptionEffectiveCache.cs Redis implementation with key structure: stellaops:exc:{tenant}:a:{asset}:{advisory} for asset entries, stellaops:exc:{tenant}:idx:e:{exceptionId} for exception-to-asset index, stellaops:exc:{tenant}:v for version counter. Warm logic loads from IExceptionRepository for active/pending exceptions. Invalidation reacts to exception events. Added ExceptionCacheOptions to PolicyEngineOptions. Added policy_exception_cache_operations_total telemetry counter with RecordExceptionCacheOperation() method. Added AddExceptionEffectiveCache() DI extension. All 197 core tests pass. POLICY-ENGINE-70-003 marked DONE.	Implementer