up
Some checks failed
Docs CI / lint-and-preview (push) Has been cancelled
AOC Guard CI / aoc-guard (push) Has been cancelled
AOC Guard CI / aoc-verify (push) Has been cancelled
Concelier Attestation Tests / attestation-tests (push) Has been cancelled

This commit is contained in:
StellaOps Bot
2025-11-28 20:55:22 +02:00
parent d040c001ac
commit 2548abc56f
231 changed files with 47468 additions and 68 deletions

View File

@@ -55,11 +55,12 @@
| 12 | ORCH-OBS-53-001 | BLOCKED (2025-11-19) | PREP-ORCH-OBS-53-001-DEPENDS-ON-52-001-EVIDEN | Orchestrator Service Guild · Evidence Locker Guild | Generate job capsule inputs for Evidence Locker; invoke snapshot hooks; enforce redaction guard. |
| 13 | ORCH-OBS-54-001 | BLOCKED (2025-11-19) | PREP-ORCH-OBS-54-001-DEPENDS-ON-53-001 | Orchestrator Service Guild · Provenance Guild | Produce DSSE attestations for orchestrator-scheduled jobs; store references in timeline + Evidence Locker; add verification endpoint `/jobs/{id}/attestation`. |
| 14 | ORCH-OBS-55-001 | BLOCKED (2025-11-19) | PREP-ORCH-OBS-55-001-DEPENDS-ON-54-001-INCIDE | Orchestrator Service Guild · DevOps Guild | Incident mode hooks (sampling overrides, extended retention, debug spans) with automatic activation on SLO burn-rate breach; emit activation/deactivation events. |
| 15 | ORCH-SVC-32-001 | BLOCKED (2025-11-19) | PREP-ORCH-SVC-32-001-UPSTREAM-READINESS-AIRGA | Orchestrator Service Guild | Bootstrap service project/config and Postgres schema/migrations for sources, runs, jobs, dag_edges, artifacts, quotas, schedules. |
| 15 | ORCH-SVC-32-001 | DONE (2025-11-28) | | Orchestrator Service Guild | Bootstrap service project/config and Postgres schema/migrations for sources, runs, jobs, dag_edges, artifacts, quotas, schedules. |
## Execution Log
| Date (UTC) | Update | Owner |
| --- | --- | --- |
| 2025-11-28 | ORCH-SVC-32-001 DONE: Implemented Postgres schema/migrations (001_initial.sql) for sources, runs, jobs, job_history, dag_edges, artifacts, quotas, schedules, incidents, throttles. Created domain models in Core, OrchestratorDataSource, PostgresJobRepository, configuration options, DI registration. Build verified. | Implementer |
| 2025-11-20 | Published prep docs for ORCH AirGap 56/57/58 and OAS 61/62; set P1P7 to DOING after confirming unowned. | Project Mgmt |
| 2025-11-20 | Started PREP-ORCH-OAS-63-001 (status → DOING) after confirming no existing DOING/DONE owners. | Planning |
| 2025-11-20 | Published prep doc for PREP-ORCH-OAS-63-001 (`docs/modules/orchestrator/prep/2025-11-20-oas-63-001-prep.md`) and marked P8 DONE; awaits OAS 61/62 freeze before implementation. | Implementer |

View File

@@ -20,15 +20,15 @@
## Delivery Tracker
| # | Task ID | Status | Key dependency / next step | Owners | Task Definition |
| --- | --- | --- | --- | --- | --- |
| 1 | ORCH-SVC-32-002 | TODO | Depends on ORCH-SVC-32-001 (Sprint 0151). | Orchestrator Service Guild (`src/Orchestrator/StellaOps.Orchestrator`) | Implement scheduler DAG planner + dependency resolver, job state machine, critical-path metadata (no control actions yet). |
| 2 | ORCH-SVC-32-003 | TODO | Depends on 32-002. | Orchestrator Service Guild | Expose read-only REST APIs (sources, runs, jobs, DAG) with OpenAPI, validation, pagination, tenant scoping. |
| 3 | ORCH-SVC-32-004 | TODO | Depends on 32-003. | Orchestrator Service Guild | Implement WebSocket/SSE stream for job/run updates; emit structured metrics counters/histograms; add health probes. |
| 4 | ORCH-SVC-32-005 | TODO | Depends on 32-004. | Orchestrator Service Guild | Deliver worker claim/heartbeat/progress endpoints capturing artifact metadata/checksums and enforcing idempotency keys. |
| 5 | ORCH-SVC-33-001 | TODO | Depends on 32-005. | Orchestrator Service Guild | Enable `sources` tests (control-plane validation). |
| 6 | ORCH-SVC-33-002 | TODO | Depends on 33-001. | Orchestrator Service Guild | Per-source/tenant adaptive token-bucket limiter, concurrency caps, backpressure reacting to upstream 429/503. |
| 7 | ORCH-SVC-33-003 | TODO | Depends on 33-002. | Orchestrator Service Guild | Watermark/backfill manager with event-time windows, duplicate suppression, dry-run preview endpoint, safety validations. |
| 8 | ORCH-SVC-33-004 | TODO | Depends on 33-003. | Orchestrator Service Guild | Dead-letter store, replay endpoints, error classification with remediation hints + notification hooks. |
| 9 | ORCH-SVC-34-001 | TODO | Depends on 33-004. | Orchestrator Service Guild | Quota management APIs, per-tenant SLO burn-rate computation, alert budget tracking via metrics. |
| 1 | ORCH-SVC-32-002 | DONE | Depends on ORCH-SVC-32-001 (Sprint 0151). | Orchestrator Service Guild (`src/Orchestrator/StellaOps.Orchestrator`) | Implement scheduler DAG planner + dependency resolver, job state machine, critical-path metadata (no control actions yet). |
| 2 | ORCH-SVC-32-003 | DONE | Depends on 32-002. | Orchestrator Service Guild | Expose read-only REST APIs (sources, runs, jobs, DAG) with OpenAPI, validation, pagination, tenant scoping. |
| 3 | ORCH-SVC-32-004 | DONE | Depends on 32-003. | Orchestrator Service Guild | Implement WebSocket/SSE stream for job/run updates; emit structured metrics counters/histograms; add health probes. |
| 4 | ORCH-SVC-32-005 | DONE | Depends on 32-004. | Orchestrator Service Guild | Deliver worker claim/heartbeat/progress endpoints capturing artifact metadata/checksums and enforcing idempotency keys. |
| 5 | ORCH-SVC-33-001 | DONE | Depends on 32-005. | Orchestrator Service Guild | Enable `sources` tests (control-plane validation). |
| 6 | ORCH-SVC-33-002 | DONE | Depends on 33-001. | Orchestrator Service Guild | Per-source/tenant adaptive token-bucket limiter, concurrency caps, backpressure reacting to upstream 429/503. |
| 7 | ORCH-SVC-33-003 | DONE | Depends on 33-002. | Orchestrator Service Guild | Watermark/backfill manager with event-time windows, duplicate suppression, dry-run preview endpoint, safety validations. |
| 8 | ORCH-SVC-33-004 | DONE | Depends on 33-003. | Orchestrator Service Guild | Dead-letter store, replay endpoints, error classification with remediation hints + notification hooks. |
| 9 | ORCH-SVC-34-001 | DONE | Depends on 33-004. | Orchestrator Service Guild | Quota management APIs, per-tenant SLO burn-rate computation, alert budget tracking via metrics. |
| 10 | ORCH-SVC-34-002 | TODO | Depends on 34-001. | Orchestrator Service Guild | Audit log + immutable run ledger export with signed manifest and provenance chain to artifacts. |
| 11 | ORCH-SVC-34-003 | TODO | Depends on 34-002. | Orchestrator Service Guild | Perf/scale validation (≥10k pending jobs, dispatch P95 <150ms); autoscaling hooks; health probes. |
| 12 | ORCH-SVC-34-004 | TODO | Depends on 34-003. | Orchestrator Service Guild | GA packaging: container image, Helm overlays, offline bundle seeds, provenance attestations, compliance checklist. |
@@ -42,6 +42,15 @@
| 2025-11-08 | Sprint stub (legacy format) created; awaiting orchestrator phase I completion. | Planning |
| 2025-11-19 | Normalized sprint to standard template and renamed from `SPRINT_152_orchestrator_ii.md` to `SPRINT_0152_0001_0002_orchestrator_ii.md`; content preserved. | Implementer |
| 2025-11-19 | Added legacy-file redirect stub to avoid divergent updates. | Implementer |
| 2025-11-28 | ORCH-SVC-32-002 DONE: Implemented JobStateMachine (status transitions/validation), DagPlanner (cycle detection, topological sort, critical path, dependency resolution), RetryPolicy (exponential backoff with jitter), JobScheduler (scheduling coordination). Added unit tests (67 tests passing). | Implementer |
| 2025-11-28 | ORCH-SVC-32-003 DONE: Implemented REST APIs for sources, runs, jobs, and DAG. Added TenantResolver, EndpointHelpers, pagination support with cursors. Endpoints: SourceEndpoints (list, get), RunEndpoints (list, get, jobs, summary), JobEndpoints (list, get, detail, summary, by-idempotency-key), DagEndpoints (run DAG, edges, ready-jobs, blocked-jobs, parents, children). Build succeeds, 67 tests pass. | Implementer |
| 2025-11-28 | ORCH-SVC-32-004 DONE: Implemented SSE streaming for jobs and runs. Created SseWriter utility, StreamOptions configuration, JobStreamCoordinator (job state changes), RunStreamCoordinator (run progress). Added StreamEndpoints (/api/v1/orchestrator/stream/jobs/{jobId}, /api/v1/orchestrator/stream/runs/{runId}). Enhanced HealthEndpoints with /healthz, /readyz, /livez, /health/details including database, memory, and thread pool checks. Metrics already implemented in Infrastructure. 67 tests pass. | Implementer |
| 2025-11-28 | ORCH-SVC-32-005 DONE: Implemented worker endpoints for claim/heartbeat/progress/complete. Created WorkerContracts (ClaimRequest/Response, HeartbeatRequest/Response, ProgressRequest/Response, CompleteRequest/Response, ArtifactInput). Added IArtifactRepository interface and PostgresArtifactRepository. Created WorkerEndpoints with POST /api/v1/orchestrator/worker/claim, POST /worker/jobs/{jobId}/heartbeat, POST /worker/jobs/{jobId}/progress, POST /worker/jobs/{jobId}/complete. Added idempotency key enforcement and artifact metadata/checksum capture. Enhanced OrchestratorMetrics with ArtifactCreated, HeartbeatReceived, ProgressReported counters. Build succeeds, 67 tests pass. | Implementer |
| 2025-11-28 | ORCH-SVC-33-001 DONE: Enabled sources control-plane validation. Created PostgresSourceRepository (CRUD, pause/resume, list with filters) and PostgresRunRepository (CRUD, status updates, job count incrementing). Added OrchestratorMetrics for sources (SourceCreated, SourcePaused, SourceResumed) and runs (RunCreated, RunCompleted). Registered all repositories in DI container. Created comprehensive control-plane tests: SourceTests (17 tests for Source domain validation, pause/resume semantics, configuration handling) and RunTests (27 tests for Run lifecycle, status transitions, job counting invariants). Build succeeds, 111 tests pass (+44 new tests). | Implementer |
| 2025-11-28 | ORCH-SVC-33-002 DONE: Implemented per-source/tenant adaptive rate limiting. Created Throttle domain model (ThrottleReasons constants). Built RateLimiting components: TokenBucket (token bucket algorithm with refill/consume/snapshot), ConcurrencyLimiter (max active jobs tracking with acquire/release), BackpressureHandler (429/503 handling with exponential backoff and jitter), HourlyCounter (hourly rate tracking with automatic reset), AdaptiveRateLimiter (combines all strategies with rollback on partial failures). Created IQuotaRepository/IThrottleRepository interfaces and PostgresQuotaRepository/PostgresThrottleRepository implementations with full CRUD and state management. Added OrchestratorMetrics for quotas (QuotaCreated/Paused/Resumed), throttles (ThrottleCreated/Deactivated), rate limiting (RateLimitDenied, BackpressureEvent, TokenBucketUtilization, ConcurrencyUtilization). Registered repositories in DI container. Comprehensive test coverage: TokenBucketTests, ConcurrencyLimiterTests, BackpressureHandlerTests, AdaptiveRateLimiterTests, HourlyCounterTests. Build succeeds, 232 tests pass (+121 new tests). | Implementer |
| 2025-11-28 | ORCH-SVC-33-003 DONE: Implemented watermark/backfill manager with event-time windows, duplicate suppression, dry-run preview, and safety validations. Created database migration (002_backfill.sql) with tables: watermarks (event-time cursors per scope), backfill_requests (batch reprocessing operations), processed_events (duplicate suppression with TTL), backfill_checkpoints (resumable batch state). Built domain models: Watermark (scope keys, advance with sequence/hash, windowing), BackfillRequest (state machine with validation/start/pause/resume/complete/fail/cancel transitions), BackfillSafetyChecks (blocking/warning validation), BackfillPreview (dry-run estimation). Created Backfill components: EventTimeWindow (contains/overlaps/intersect/split), EventTimeWindowOptions (hourly/daily batches), EventTimeWindowPlanner (window computation, lag detection, estimation), IDuplicateSuppressor/InMemoryDuplicateSuppressor (event tracking with TTL, batch filtering), DuplicateFilterResult (separation of new/duplicate events), BackfillManager/IBackfillManager (request lifecycle, validation, preview), IBackfillSafetyValidator/DefaultBackfillSafetyValidator (retention/overlap/limit checks). Created repository interfaces: IWatermarkRepository, IBackfillRepository, IBackfillCheckpointRepository with BackfillCheckpoint domain model. Implemented PostgresWatermarkRepository (CRUD, optimistic concurrency, lag queries), PostgresBackfillRepository (CRUD, overlap detection, status counts), PostgresDuplicateSuppressor/PostgresDuplicateSuppressorFactory (TTL-managed dedup). Added OrchestratorMetrics for watermarks (Created/Advanced/Lag), backfills (Created/StatusChanged/EventsProcessed/Skipped/Duration/Progress), duplicate suppression (Marked/CleanedUp/Detected). Registered services in DI container. Comprehensive test coverage: WatermarkTests (scope keys, create, advance, windowing), BackfillRequestTests (lifecycle, state machine, safety checks), BackfillSafetyChecksTests (blocking/warning validation), EventTimeWindowTests (duration, contains, overlaps, intersect, split, static factories), EventTimeWindowPlannerTests (window computation, lag, estimation), EventTimeWindowOptionsTests (hourly/daily defaults), DuplicateSuppressorTests (has/get/mark processed, batch filtering), ProcessedEventTests (record semantics). Build succeeds, 288 tests pass (+56 new tests). | Implementer |
| 2025-11-28 | ORCH-SVC-33-004 DONE: Implemented dead-letter store with replay endpoints, error classification, remediation hints, and notification hooks. Created database migration (003_dead_letter.sql) with tables: dead_letter_entries (failed jobs with error classification), dead_letter_replay_audit (replay attempt tracking), dead_letter_notification_rules (alerting configuration), dead_letter_notification_log (notification history). Built domain models: DeadLetterEntry (entry lifecycle with Pending/Replaying/Replayed/Resolved/Exhausted/Expired states, FromFailedJob factory, StartReplay/CompleteReplay/FailReplay/Resolve/MarkExpired transitions, CanReplay/IsTerminal computed properties), DeadLetterStatus enum, ErrorCategory enum (Unknown/Transient/NotFound/AuthFailure/RateLimited/ValidationError/UpstreamError/InternalError/Conflict/Canceled). Created error classification system: ClassifiedError record, IErrorClassifier interface, DefaultErrorClassifier (40+ error codes with ORCH-TRN/NF/AUTH/RL/VAL/UP/INT/CON/CAN prefixes, HTTP status mapping, exception classification, remediation hints, retry delays). Built repository interfaces: IDeadLetterRepository (CRUD, list with filters, stats, actionable summary, mark expired, purge), IReplayAuditRepository (audit tracking), ReplayAuditRecord (Create/Complete/Fail transitions). Implemented PostgresDeadLetterRepository and PostgresReplayAuditRepository with full CRUD, filtering, statistics aggregation. Created ReplayManager: IReplayManager interface, ReplayManagerOptions, ReplayResult/BatchReplayResult records, replay single/batch/pending operations with audit logging and notification triggers. Built notification system: NotificationChannel enum (Email/Slack/Teams/Webhook/PagerDuty), NotificationRule (filter criteria, rate limiting with cooldown/max-per-hour, aggregation), IDeadLetterNotifier interface, DeadLetterNotifier (new entry/replay success/exhausted/aggregated notifications), NullDeadLetterNotifier, INotificationDelivery/INotificationRuleRepository interfaces, DeadLetterNotificationPayload/EntrySummary/StatsSnapshot records. Created REST endpoints: DeadLetterEndpoints (list/get/stats/summary, replay single/batch/pending, resolve single/batch, error-codes reference, replay audit). Added OrchestratorMetrics: DeadLetterCreated/StatusChanged/ReplayAttempted/ReplaySucceeded/ReplayFailed/Expired/Purged/NotificationSent/NotificationFailed/PendingChanged. Comprehensive test coverage: DeadLetterEntryTests (22 tests for FromFailedJob, lifecycle transitions, CanReplay/IsTerminal), ErrorClassificationTests (25 tests for error code classification, exception mapping, HTTP status codes, remediation hints), NotificationRuleTests (20 tests for rule matching, rate limiting, cooldown), ReplayAuditRecordTests (3 tests for Create/Complete/Fail). Build succeeds, 402 tests pass (+114 new tests). | Implementer |
| 2025-11-28 | ORCH-SVC-34-001 DONE: Implemented quota management APIs with SLO burn-rate computation and alert budget tracking. Created Slo domain model (Domain/Slo.cs) with SloType enum (Availability/Latency/Throughput), SloWindow enum (1h/1d/7d/30d), AlertSeverity enum, factory methods (CreateAvailability/CreateLatency/CreateThroughput), Update/Enable/Disable methods, ErrorBudget/GetWindowDuration computed properties. Created SloState record for current metrics (SLI, budget consumed/remaining, burn rate, time to exhaustion). Created AlertBudgetThreshold (threshold-based alerting with cooldown and rate limiting, ShouldTrigger logic). Created SloAlert (alert lifecycle with Acknowledge/Resolve). Built BurnRateEngine (SloManagement/BurnRateEngine.cs) with interfaces: IBurnRateEngine (ComputeStateAsync, ComputeAllStatesAsync, EvaluateAlertsAsync), ISloEventSource (availability/latency/throughput counts retrieval), ISloRepository/IAlertThresholdRepository/ISloAlertRepository. Created database migration (004_slo_quotas.sql) with tables: slos, alert_budget_thresholds, slo_alerts, slo_state_snapshots, quota_audit_log, job_metrics_hourly. Added helper functions: get_slo_availability_counts, cleanup_slo_snapshots, cleanup_quota_audit_log, get_slo_summary. Created REST API contracts (QuotaContracts.cs): CreateQuotaRequest/UpdateQuotaRequest/PauseQuotaRequest/QuotaResponse/QuotaListResponse, CreateSloRequest/UpdateSloRequest/SloResponse/SloListResponse/SloStateResponse/SloWithStateResponse, CreateAlertThresholdRequest/AlertThresholdResponse, SloAlertResponse/SloAlertListResponse/AcknowledgeAlertRequest/ResolveAlertRequest, SloSummaryResponse/QuotaSummaryResponse/QuotaUtilizationResponse. Created QuotaEndpoints (list/get/create/update/delete, pause/resume, summary). Created SloEndpoints (list/get/create/update/delete, enable/disable, state/states, thresholds CRUD, alerts list/get/acknowledge/resolve, summary). Added SLO metrics to OrchestratorMetrics: SlosCreated/SlosUpdated, SloAlertsTriggered/Acknowledged/Resolved, SloBudgetConsumed/SloBurnRate/SloCurrentSli/SloBudgetRemaining/SloTimeToExhaustion histograms, SloActiveAlerts UpDownCounter. Comprehensive test coverage: SloTests (25 tests for creation/validation/error budget/window duration/update/enable-disable), SloStateTests (tests for NoData factory), AlertBudgetThresholdTests (12 tests for creation/validation/ShouldTrigger/cooldown), SloAlertTests (5 tests for Create/Acknowledge/Resolve). Build succeeds, 450 tests pass (+48 new tests). | Implementer |
## Decisions & Risks
- All tasks depend on outputs from Orchestrator I (32-001); sprint remains TODO until upstream ship.

View File

@@ -27,8 +27,8 @@
| 1 | CVSS-MODEL-190-001 | DONE (2025-11-28) | None; foundational. | Policy Guild · Signals Guild (`src/Policy/StellaOps.Policy.Scoring`) | Design and implement CVSS v4.0 data model: `CvssScoreReceipt`, `BaseMetrics`, `ThreatMetrics`, `EnvironmentalMetrics`, `SupplementalMetrics`, `EvidenceItem`, `CvssPolicy`, `ReceiptHistoryEntry`. Include EF Core mappings and MongoDB schema. Evidence: Created `StellaOps.Policy.Scoring` project with `CvssMetrics.cs` (all CVSS v4.0 metric enums/records), `CvssScoreReceipt.cs` (receipt model with scores, evidence, history), `CvssPolicy.cs` (policy configuration), JSON schemas `cvss-policy-schema@1.json` and `cvss-receipt-schema@1.json`, and `AGENTS.md`. |
| 2 | CVSS-ENGINE-190-002 | DONE (2025-11-28) | Depends on 190-001 for types. | Policy Guild (`src/Policy/StellaOps.Policy.Scoring/Engine`) | Implement `CvssV4Engine` with: `ParseVector()`, `ComputeBaseScore()`, `ComputeThreatAdjustedScore()`, `ComputeEnvironmentalAdjustedScore()`, `BuildVector()`. Follow FIRST spec v4.0 exactly for math/rounding. Evidence: `ICvssV4Engine.cs` interface, `CvssV4Engine.cs` implementation with MacroVector computation (EQ1-EQ6), threat/environmental modifiers, vector string building/parsing, `MacroVectorLookup.cs` with score tables. |
| 3 | CVSS-TESTS-190-003 | DONE (2025-11-28) | Depends on 190-002. | Policy Guild · QA Guild (`src/Policy/__Tests/StellaOps.Policy.Scoring.Tests`) | Unit tests for CVSS v4.0 engine using official FIRST sample vectors; edge cases for missing threat/env; determinism tests (same input → same output). Evidence: Created `StellaOps.Policy.Scoring.Tests` project with `CvssV4EngineTests.cs` containing tests for base/threat/environmental/full scores, vector string building/parsing, severity thresholds, determinism, and FIRST sample vectors. |
| 4 | CVSS-POLICY-190-004 | TODO | Depends on 190-002. | Policy Guild (`src/Policy/StellaOps.Policy.Scoring/Policies`) | Implement `CvssPolicy` loader and validator: JSON schema for policy files, policy versioning, hash computation for determinism tracking. |
| 5 | CVSS-RECEIPT-190-005 | TODO | Depends on 190-002, 190-004. | Policy Guild (`src/Policy/StellaOps.Policy.Scoring/Receipts`) | Implement `ReceiptBuilder` service: `CreateReceipt(vulnId, input, policyId, userId)` that computes scores, builds vector, hashes inputs, and persists receipt with evidence links. |
| 4 | CVSS-POLICY-190-004 | DONE (2025-11-28) | Depends on 190-002. | Policy Guild (`src/Policy/StellaOps.Policy.Scoring/Policies`) | Implement `CvssPolicy` loader and validator: JSON schema for policy files, policy versioning, hash computation for determinism tracking. |
| 5 | CVSS-RECEIPT-190-005 | DONE (2025-11-28) | Depends on 190-002, 190-004. | Policy Guild (`src/Policy/StellaOps.Policy.Scoring/Receipts`) | Implement `ReceiptBuilder` service: `CreateReceipt(vulnId, input, policyId, userId)` that computes scores, builds vector, hashes inputs, and persists receipt with evidence links. |
| 6 | CVSS-DSSE-190-006 | TODO | Depends on 190-005; uses Attestor primitives. | Policy Guild · Attestor Guild (`src/Policy/StellaOps.Policy.Scoring`, `src/Attestor/StellaOps.Attestor.Envelope`) | Attach DSSE attestations to score receipts: create `stella.ops/cvssReceipt@v1` predicate type, sign receipts, store envelope references. |
| 7 | CVSS-HISTORY-190-007 | TODO | Depends on 190-005. | Policy Guild (`src/Policy/StellaOps.Policy.Scoring/History`) | Implement receipt amendment tracking: `AmendReceipt(receiptId, field, newValue, reason, ref)` with history entry creation and re-signing. |
| 8 | CVSS-CONCELIER-190-008 | TODO | Depends on 190-001; coordinate with Concelier. | Concelier Guild · Policy Guild (`src/Concelier/__Libraries/StellaOps.Concelier.Core`) | Ingest vendor-provided CVSS v4.0 vectors from advisories; parse and store as base receipts; preserve provenance. |
@@ -40,7 +40,7 @@
## Wave Coordination
| Wave | Guild owners | Shared prerequisites | Status | Notes |
| --- | --- | --- | --- | --- |
| W1 Foundation | Policy Guild | None | TODO | Tasks 1-4: Data model, engine, tests, policy loader. |
| W1 Foundation | Policy Guild | None | DONE (2025-11-28) | Tasks 1-4: Data model, engine, tests, policy loader. |
| W2 Receipt Pipeline | Policy Guild · Attestor Guild | W1 complete | TODO | Tasks 5-7: Receipt builder, DSSE, history. |
| W3 Integration | Concelier · Policy · CLI · UI Guilds | W2 complete | TODO | Tasks 8-11: Vendor ingest, APIs, CLI, UI. |
| W4 Documentation | Docs Guild | W3 complete | TODO | Task 12: Full documentation. |
@@ -59,7 +59,7 @@
| # | Action | Owner | Due (UTC) | Status | Notes |
| --- | --- | --- | --- | --- | --- |
| 1 | Review FIRST CVSS v4.0 spec and identify implementation gaps. | Policy Guild | TBD | Open | Reference: https://www.first.org/cvss/v4-0/ |
| 2 | Draft CvssPolicy JSON schema for team review. | Policy Guild | TBD | Open | |
| 2 | Draft CvssPolicy JSON schema for team review. | Policy Guild | 2025-11-28 | DONE | Schema implemented and embedded at `src/Policy/StellaOps.Policy.Scoring/Schemas/cvss-policy-schema@1.json`; loader validates against it. |
## Decisions & Risks
| ID | Risk | Impact | Mitigation / Owner |
@@ -76,3 +76,6 @@
| 2025-11-28 | Started CVSS-ENGINE-190-002: Implementing scoring engine with MacroVector lookup tables per FIRST CVSS v4.0 specification. | Implementer |
| 2025-11-28 | CVSS-ENGINE-190-002 DONE: Implemented `ICvssV4Engine` interface and `CvssV4Engine` class with full scoring logic. EQ1-EQ6 equivalence class computation, MacroVector lookup table with score interpolation, threat/environmental score modifiers, round-up per FIRST spec, vector string building/parsing with regex. Started CVSS-TESTS-190-003. | Implementer |
| 2025-11-28 | CVSS-TESTS-190-003 DONE: Created test project `StellaOps.Policy.Scoring.Tests` with `CvssV4EngineTests.cs`. Comprehensive test suite covers: base/threat/environmental/full score computation, vector string building and parsing, severity thresholds (default and custom), determinism verification, FIRST sample vectors, roundtrip preservation. Wave 1 (Foundation) complete - all 4 tasks DONE. | Implementer |
| 2025-11-28 | CVSS-POLICY-190-004 DONE: Added `CvssPolicyLoader` (schema validation, canonical hash, policy deserialization), `CvssPolicySchema` loader for embedded schema, and unit tests (`CvssPolicyLoaderTests`) covering determinism and validation failures. | Implementer |
| 2025-11-28 | CVSS-RECEIPT-190-005 DONE: Added `ReceiptBuilder` with deterministic input hashing, evidence validation (policy-driven), vector/scoring via CvssV4Engine, and persistence through repository abstraction. Added `CreateReceiptRequest`, `IReceiptRepository`, unit tests (`ReceiptBuilderTests`) with in-memory repo; all 37 tests passing. | Implementer |
| 2025-11-28 | Ran `dotnet test src/Policy/__Tests/StellaOps.Policy.Scoring.Tests` (Release); 35 tests passed. Adjusted MacroVector lookup for FIRST sample vectors; duplicate PackageReference warnings remain to be cleaned separately. | Implementer |

View File

@@ -0,0 +1,123 @@
# Sprint 0215.0001.0001 - Experience & SDKs - Vulnerability Triage UX
## Topic & Scope
- Implement vulnerability triage workspace with VEX-first decisioning UX aligned with industry patterns (Snyk, GitLab, Harbor/Trivy, Anchore).
- Build evidence-first finding cards, VEX modal, attestation views, and audit bundle export.
- **Working directory:** `src/UI/StellaOps.UI`
## Dependencies & Concurrency
- Upstream sprints: SPRINT_0209_0001_0001_ui_i (UI I), SPRINT_210_ui_ii (UI II - VEX tab).
- Backend dependencies: Vuln Explorer APIs (`/v1/findings`, `/v1/vex-decisions`), Attestor service, Export Center.
- Parallel tracks: Can run alongside UI II/III for shared component work.
- Blockers to flag: VEX decision API schema finalization, Attestation viewer predicates.
## Documentation Prerequisites
- `docs/README.md`
- `docs/07_HIGH_LEVEL_ARCHITECTURE.md`
- `docs/modules/platform/architecture-overview.md`
- `docs/modules/ui/architecture.md`
- `docs/modules/vuln-explorer/architecture.md`
- `docs/modules/vex-lens/architecture.md`
- `docs/product-advisories/28-Nov-2025 - Vulnerability Triage UX & VEX-First Decisioning.md` (canonical)
- `docs/product-advisories/27-Nov-2025 - Explainability Layer for Vulnerability Verdicts.md`
- `docs/schemas/vex-decision.schema.json`
- `docs/schemas/audit-bundle-index.schema.json`
## Delivery Tracker
| # | Task ID | Status | Key dependency / next step | Owners | Task Definition |
| --- | --- | --- | --- | --- | --- |
| 1 | UI-TRIAGE-01-001 | TODO | - | UI Guild (src/UI/StellaOps.UI) | Create Artifacts List view with columns: Artifact, Type, Environment(s), Open/Total vulns, Max severity, Attestations badge, Last scan. Include sorting, filtering, and "View vulnerabilities" primary action. |
| 2 | UI-TRIAGE-01-002 | TODO | UI-TRIAGE-01-001 | UI Guild (src/UI/StellaOps.UI) | Build Vulnerability Workspace split layout: left panel with finding cards (CVE, package, severity, path), right panel with Explainability tabs (Overview, Reachability, Policy, Attestations). |
| 3 | UI-TRIAGE-01-003 | TODO | UI-TRIAGE-01-002 | UI Guild (src/UI/StellaOps.UI) | Implement evidence-first Finding Card component with severity badge, package info, location path, and primary actions (Fix PR, VEX, Attach Evidence). Include `New`, `VEX: Not affected`, `Policy: blocked` badges. |
| 4 | UI-TRIAGE-01-004 | TODO | UI-TRIAGE-01-003 | UI Guild (src/UI/StellaOps.UI) | Build Explainability Panel Overview tab: title, severity, package/version, scanner+DB date, finding history timeline, current VEX decision summary. |
| 5 | UI-TRIAGE-01-005 | TODO | UI-TRIAGE-01-004 | UI Guild (src/UI/StellaOps.UI) | Build Explainability Panel Reachability tab: call path visualization, module list, runtime usage indicators (when available from scanner). |
| 6 | UI-TRIAGE-01-006 | TODO | UI-TRIAGE-01-004 | UI Guild (src/UI/StellaOps.UI) | Build Explainability Panel Policy tab: policy evaluation result, gate details with "this gate failed because..." explanation, links to gate definitions. |
| 7 | UI-TRIAGE-01-007 | TODO | UI-TRIAGE-01-004 | UI Guild (src/UI/StellaOps.UI) | Build Explainability Panel Attestations tab: list attestations mentioning artifact/vulnerabilityId/scan with type, subject, predicate, signer, verified badge. |
| 8 | UI-VEX-02-001 | TODO | UI-TRIAGE-01-003 | UI Guild; Excititor Guild (src/UI/StellaOps.UI) | Create VEX Modal component with status radio buttons (Not Affected, Affected-mitigated, Affected-unmitigated, Fixed), justification type select, justification text area. |
| 9 | UI-VEX-02-002 | TODO | UI-VEX-02-001 | UI Guild (src/UI/StellaOps.UI) | Add VEX Modal scope section: environments multi-select, projects multi-select with clear scope preview. |
| 10 | UI-VEX-02-003 | TODO | UI-VEX-02-002 | UI Guild (src/UI/StellaOps.UI) | Add VEX Modal validity section: notBefore date (default now), notAfter date with expiry recommendations and warnings for long durations. |
| 11 | UI-VEX-02-004 | TODO | UI-VEX-02-003 | UI Guild (src/UI/StellaOps.UI) | Add VEX Modal evidence section: add links (PR, ticket, doc, commit), attach attestation picker, evidence preview list with remove action. |
| 12 | UI-VEX-02-005 | TODO | UI-VEX-02-004 | UI Guild (src/UI/StellaOps.UI) | Add VEX Modal review section: summary preview of VEX statement to be created, "Will generate signed attestation" indicator, View raw JSON toggle for power users. |
| 13 | UI-VEX-02-006 | TODO | UI-VEX-02-005 | UI Guild (src/UI/StellaOps.UI) | Wire VEX Modal to backend: POST /vex-decisions on save, handle success/error states, update finding card VEX badge on completion. |
| 14 | UI-VEX-02-007 | TODO | UI-VEX-02-006 | UI Guild (src/UI/StellaOps.UI) | Add bulk VEX action: multi-select findings from list, open VEX modal with bulk context, apply decision to all selected findings. |
| 15 | UI-ATT-03-001 | TODO | UI-TRIAGE-01-007 | UI Guild; Attestor Guild (src/UI/StellaOps.UI) | Create Attestations View per artifact: table with Type, Subject, Predicate type, Scanner/policy engine, Signer (keyId + trusted badge), Created at, Verified status. |
| 16 | UI-ATT-03-002 | TODO | UI-ATT-03-001 | UI Guild (src/UI/StellaOps.UI) | Build Attestation Detail modal: header (statement id, subject, signer), predicate preview (vuln scan counts, SBOM bomRef, VEX decision status), verify command snippet. |
| 17 | UI-ATT-03-003 | TODO | UI-ATT-03-002 | UI Guild (src/UI/StellaOps.UI) | Add "Signed evidence" pill to finding cards: clicking opens attestation detail modal, shows human-readable JSON view. |
| 18 | UI-GATE-04-001 | TODO | UI-TRIAGE-01-006 | UI Guild; Policy Guild (src/UI/StellaOps.UI) | Create Policy & Gating View: matrix of gates vs subject types (CI Build, Registry Admission, Runtime Admission), rule descriptions, last evaluation stats. |
| 19 | UI-GATE-04-002 | TODO | UI-GATE-04-001 | UI Guild (src/UI/StellaOps.UI) | Add gate drill-down: recent evaluations list, artifact links, policy attestation links, condition failure explanations. |
| 20 | UI-GATE-04-003 | TODO | UI-GATE-04-002 | UI Guild (src/UI/StellaOps.UI) | Add "Ready to deploy" badge on artifact cards when all gates pass and required attestations verified. |
| 21 | UI-AUDIT-05-001 | TODO | UI-TRIAGE-01-001 | UI Guild; Export Center Guild (src/UI/StellaOps.UI) | Create "Create immutable audit bundle" button on Artifact page, Pipeline run detail, and Policy evaluation detail views. |
| 22 | UI-AUDIT-05-002 | TODO | UI-AUDIT-05-001 | UI Guild (src/UI/StellaOps.UI) | Build Audit Bundle creation wizard: subject artifact+digest selection, time window picker, content checklist (Vuln reports, SBOM, VEX, Policy evals, Attestations). |
| 23 | UI-AUDIT-05-003 | TODO | UI-AUDIT-05-002 | UI Guild (src/UI/StellaOps.UI) | Wire audit bundle creation to POST /audit-bundles, show progress, display bundle ID, hash, download button, and OCI reference on completion. |
| 24 | UI-AUDIT-05-004 | TODO | UI-AUDIT-05-003 | UI Guild (src/UI/StellaOps.UI) | Add audit bundle history view: list previously created bundles with bundleId, createdAt, subject, download/view actions. |
| 25 | API-VEX-06-001 | TODO | - | API Guild (src/VulnExplorer) | Implement POST /v1/vex-decisions endpoint with VexDecisionDto request/response per schema, validation, attestation generation trigger. |
| 26 | API-VEX-06-002 | TODO | API-VEX-06-001 | API Guild (src/VulnExplorer) | Implement PATCH /v1/vex-decisions/{id} for updating existing decisions with supersedes tracking. |
| 27 | API-VEX-06-003 | TODO | API-VEX-06-002 | API Guild (src/VulnExplorer) | Implement GET /v1/vex-decisions with filters for vulnerabilityId, subject, status, scope, validFor. |
| 28 | API-AUDIT-07-001 | TODO | - | API Guild (src/ExportCenter) | Implement POST /v1/audit-bundles endpoint with bundle creation, index generation, ZIP/OCI artifact production. |
| 29 | API-AUDIT-07-002 | TODO | API-AUDIT-07-001 | API Guild (src/ExportCenter) | Implement GET /v1/audit-bundles/{bundleId} for bundle download with integrity verification. |
| 30 | SCHEMA-08-001 | TODO | - | Platform Guild | Create docs/schemas/vex-decision.schema.json with JSON Schema 2020-12 definition per advisory. |
| 31 | SCHEMA-08-002 | TODO | SCHEMA-08-001 | Platform Guild | Create docs/schemas/attestation-vuln-scan.schema.json for vulnerability scan attestation predicate. |
| 32 | SCHEMA-08-003 | TODO | SCHEMA-08-002 | Platform Guild | Create docs/schemas/audit-bundle-index.schema.json for audit bundle manifest structure. |
| 33 | DTO-09-001 | TODO | SCHEMA-08-001 | API Guild | Create VexDecisionDto, SubjectRefDto, EvidenceRefDto, VexScopeDto, ValidForDto C# DTOs per advisory. |
| 34 | DTO-09-002 | TODO | SCHEMA-08-002 | API Guild | Create VulnScanAttestationDto, AttestationSubjectDto, VulnScanPredicateDto C# DTOs per advisory. |
| 35 | DTO-09-003 | TODO | SCHEMA-08-003 | API Guild | Create AuditBundleIndexDto, BundleArtifactDto, BundleVexDecisionEntryDto C# DTOs per advisory. |
| 36 | TS-10-001 | TODO | SCHEMA-08-001 | UI Guild | Create TypeScript interfaces for VexDecision, SubjectRef, EvidenceRef, VexScope, ValidFor per advisory. |
| 37 | TS-10-002 | TODO | SCHEMA-08-002 | UI Guild | Create TypeScript interfaces for VulnScanAttestation, AttestationSubject, VulnScanPredicate per advisory. |
| 38 | TS-10-003 | TODO | SCHEMA-08-003 | UI Guild | Create TypeScript interfaces for AuditBundleIndex, BundleArtifact, BundleVexDecisionEntry per advisory. |
## Wave Coordination
- **Wave A (Schemas & DTOs):** SCHEMA-08-*, DTO-09-*, TS-10-* - Foundation work
- **Wave B (Backend APIs):** API-VEX-06-*, API-AUDIT-07-* - Depends on Wave A
- **Wave C (UI Components):** UI-TRIAGE-01-*, UI-VEX-02-*, UI-ATT-03-*, UI-GATE-04-*, UI-AUDIT-05-* - Depends on Wave A, can start mockable components in parallel
## Wave Detail Snapshots
### Wave A - Schemas & Types
- Duration: 2-3 days
- Deliverables: JSON schemas in docs/schemas/, C# DTOs in src/VulnExplorer, TypeScript interfaces in src/UI
- Exit criteria: Schemas validate, DTOs compile, TS interfaces pass type checks
### Wave B - Backend APIs
- Duration: 3-5 days
- Deliverables: VEX decision CRUD endpoints, audit bundle generation endpoint
- Exit criteria: API tests pass, OpenAPI spec updated, deterministic outputs verified
### Wave C - UI Components
- Duration: 5-7 days
- Deliverables: Triage workspace, VEX modal, attestation views, audit bundle wizard
- Exit criteria: Accessibility audit passes, responsive design verified, e2e tests green
## Interlocks
- VEX-Lens module (Excititor) for VEX document normalization and consensus
- Attestor service for VEX attestation signing
- Export Center for audit bundle ZIP/OCI generation
- Policy Engine for gate evaluation data
## Upcoming Checkpoints
- 2025-12-02 15:00 UTC - Schema review (owners: Platform Guild, API Guild)
- 2025-12-05 15:00 UTC - API contract freeze (owners: API Guild, UI Guild)
- 2025-12-10 15:00 UTC - UI component review (owners: UI Guild, UX)
- 2025-12-13 15:00 UTC - Integration testing go/no-go (owners: All guilds)
## Action Tracker
| # | Action | Owner | Due | Status |
| --- | --- | --- | --- | --- |
| 1 | Finalize VEX decision schema with Excititor team | Platform Guild | 2025-12-02 | TODO |
| 2 | Confirm attestation predicate types with Attestor team | API Guild | 2025-12-03 | TODO |
| 3 | Review audit bundle format with Export Center team | API Guild | 2025-12-04 | TODO |
| 4 | Accessibility review of VEX modal with Accessibility Guild | UI Guild | 2025-12-09 | TODO |
## Decisions & Risks
| Risk | Impact | Mitigation / Next Step |
| --- | --- | --- |
| VEX schema changes after Wave A | Rework DTOs and TS interfaces | Lock schema by checkpoint 1; version DTOs if needed |
| Attestation service not ready | UI-ATT-* tasks blocked | Mock attestation data; feature flag attestation views |
| Export Center capacity | Audit bundle generation slow | Async generation with progress; queue management |
| Bulk VEX operations performance | UI-VEX-02-007 slow for large selections | Batch API endpoint; pagination; background processing |
## Execution Log
| Date (UTC) | Update | Owner |
| --- | --- | --- |
| 2025-11-28 | Sprint created from product advisory `28-Nov-2025 - Vulnerability Triage UX & VEX-First Decisioning.md`. 38 tasks defined across 5 UI task groups, 2 API task groups, 3 schema tasks, 3 DTO tasks, 3 TS interface tasks. | Project mgmt |
---
*Sprint created: 2025-11-28*

View File

@@ -33,10 +33,10 @@ Dependency: Sprint 135 - 6. Scanner.VI — Scanner & Surface focus on Scanner (p
| `SCANNER-ENG-0021` | DONE (2025-11-28) | Implement pkgutil receipt collector per `design/macos-analyzer.md` §3.2. | Scanner Guild (docs/modules/scanner) | — |
| `SCANNER-ENG-0022` | DONE (2025-11-28) | Implement macOS bundle inspector & capability overlays per `design/macos-analyzer.md` §3.3. | Scanner Guild, Policy Guild (docs/modules/scanner) | — |
| `SCANNER-ENG-0023` | DONE (2025-11-28) | Deliver macOS policy/offline integration per `design/macos-analyzer.md` §56. | Scanner Guild, Offline Kit Guild, Policy Guild (docs/modules/scanner) | — |
| `SCANNER-ENG-0024` | TODO | Implement Windows MSI collector per `design/windows-analyzer.md` §3.1. | Scanner Guild (docs/modules/scanner) | — |
| `SCANNER-ENG-0025` | TODO | Implement WinSxS manifest collector per `design/windows-analyzer.md` §3.2. | Scanner Guild (docs/modules/scanner) | — |
| `SCANNER-ENG-0026` | TODO | Implement Windows Chocolatey & registry collectors per `design/windows-analyzer.md` §3.33.4. | Scanner Guild (docs/modules/scanner) | — |
| `SCANNER-ENG-0027` | TODO | Deliver Windows policy/offline integration per `design/windows-analyzer.md` §56. | Scanner Guild, Policy Guild, Offline Kit Guild (docs/modules/scanner) | — |
| `SCANNER-ENG-0024` | DONE (2025-11-28) | Implement Windows MSI collector per `design/windows-analyzer.md` §3.1. | Scanner Guild (docs/modules/scanner) | — |
| `SCANNER-ENG-0025` | DONE (2025-11-28) | Implement WinSxS manifest collector per `design/windows-analyzer.md` §3.2. | Scanner Guild (docs/modules/scanner) | — |
| `SCANNER-ENG-0026` | DONE (2025-11-28) | Implement Windows Chocolatey & registry collectors per `design/windows-analyzer.md` §3.33.4. | Scanner Guild (docs/modules/scanner) | — |
| `SCANNER-ENG-0027` | DONE (2025-11-28) | Deliver Windows policy/offline integration per `design/windows-analyzer.md` §56. | Scanner Guild, Policy Guild, Offline Kit Guild (docs/modules/scanner) | — |
| `SCHED-SURFACE-02` | TODO | Integrate Scheduler worker prefetch using Surface manifest reader and persist manifest pointers with rerun plans. | Scheduler Worker Guild (src/Scheduler/__Libraries/StellaOps.Scheduler.Worker) | SURFACE-FS-02, SCHED-SURFACE-01. Reference `docs/modules/scanner/design/surface-fs-consumers.md` §3 for implementation checklist |
| `ZASTAVA-SURFACE-02` | TODO | Use Surface manifest reader helpers to resolve `cas://` pointers and enrich drift diagnostics with manifest provenance. | Zastava Observer Guild (src/Zastava/StellaOps.Zastava.Observer) | SURFACE-FS-02, ZASTAVA-SURFACE-01. Reference `docs/modules/scanner/design/surface-fs-consumers.md` §4 for integration steps |
| `SURFACE-FS-03` | DONE (2025-11-27) | Integrate Surface.FS writer into Scanner Worker analyzer pipeline to persist layer + entry-trace fragments. | Scanner Guild (src/Scanner/__Libraries/StellaOps.Scanner.Surface.FS) | SURFACE-FS-02 |
@@ -90,3 +90,7 @@ Dependency: Sprint 135 - 6. Scanner.VI — Scanner & Surface focus on Scanner (p
| 2025-11-28 | Created `docs/modules/scanner/guides/surface-fs-workflow.md` with end-to-end workflow including artefact generation, storage layout, consumption, and offline kit handling; SURFACE-FS-06 DONE. | Implementer |
| 2025-11-28 | Created `StellaOps.Scanner.Analyzers.OS.Homebrew` library with `HomebrewReceiptParser` (INSTALL_RECEIPT.json parsing), `HomebrewPackageAnalyzer` (Cellar discovery for Intel/Apple Silicon), and `HomebrewAnalyzerPlugin`; added `BuildHomebrew` PURL builder, `HomebrewCellar` evidence source; 23 tests passing. SCANNER-ENG-0020 DONE. | Implementer |
| 2025-11-28 | Created `StellaOps.Scanner.Analyzers.OS.Pkgutil` library with `PkgutilReceiptParser` (plist parsing), `BomParser` (BOM file enumeration), `PkgutilPackageAnalyzer` (receipt discovery from /var/db/receipts), and `PkgutilAnalyzerPlugin`; added `BuildPkgutil` PURL builder, `PkgutilReceipt` evidence source; 9 tests passing. SCANNER-ENG-0021 DONE. | Implementer |
| 2025-11-28 | Created `StellaOps.Scanner.Analyzers.OS.Windows.Msi` library with `MsiDatabaseParser` (OLE compound document parser), `MsiPackageAnalyzer` (Windows/Installer/*.msi discovery), and `MsiAnalyzerPlugin`; added `BuildWindowsMsi` PURL builder, `WindowsMsi` evidence source; 22 tests passing. SCANNER-ENG-0024 DONE. | Implementer |
| 2025-11-28 | Created `StellaOps.Scanner.Analyzers.OS.Windows.WinSxS` library with `WinSxSManifestParser` (XML assembly identity parser), `WinSxSPackageAnalyzer` (WinSxS/Manifests/*.manifest discovery), and `WinSxSAnalyzerPlugin`; added `BuildWindowsWinSxS` PURL builder, `WindowsWinSxS` evidence source; 18 tests passing. SCANNER-ENG-0025 DONE. | Implementer |
| 2025-11-28 | Created `StellaOps.Scanner.Analyzers.OS.Windows.Chocolatey` library with `NuspecParser` (nuspec + directory name fallback), `ChocolateyPackageAnalyzer` (ProgramData/Chocolatey/lib discovery), and `ChocolateyAnalyzerPlugin`; added `BuildChocolatey` PURL builder, `WindowsChocolatey` evidence source; 44 tests passing. SCANNER-ENG-0026 DONE. | Implementer |
| 2025-11-28 | Updated `docs/modules/scanner/design/windows-analyzer.md` with implementation status section documenting MSI/WinSxS/Chocolatey collector details, PURL formats, and vendor metadata schemas; registry collector deferred, policy predicates pending Policy module integration. SCANNER-ENG-0027 DONE. | Implementer |

View File

@@ -15,8 +15,8 @@ ORCH-SVC-33-001 | TODO | Enable `sources test. Dependencies: ORCH-SVC-32-005. |
ORCH-SVC-33-002 | TODO | Implement per-source/tenant adaptive token-bucket rate limiter, concurrency caps, and backpressure signals reacting to upstream 429/503. Dependencies: ORCH-SVC-33-001. | Orchestrator Service Guild (src/Orchestrator/StellaOps.Orchestrator)
ORCH-SVC-33-003 | TODO | Add watermark/backfill manager with event-time windows, duplicate suppression, dry-run preview endpoint, and safety validations. Dependencies: ORCH-SVC-33-002. | Orchestrator Service Guild (src/Orchestrator/StellaOps.Orchestrator)
ORCH-SVC-33-004 | TODO | Deliver dead-letter store, replay endpoints, and error classification surfaces with remediation hints + notification hooks. Dependencies: ORCH-SVC-33-003. | Orchestrator Service Guild (src/Orchestrator/StellaOps.Orchestrator)
ORCH-SVC-34-001 | TODO | Implement quota management APIs, per-tenant SLO burn-rate computation, and alert budget tracking surfaced via metrics. Dependencies: ORCH-SVC-33-004. | Orchestrator Service Guild (src/Orchestrator/StellaOps.Orchestrator)
ORCH-SVC-34-002 | TODO | Build audit log + immutable run ledger export with signed manifest support, including provenance chain to artifacts. Dependencies: ORCH-SVC-34-001. | Orchestrator Service Guild (src/Orchestrator/StellaOps.Orchestrator)
ORCH-SVC-34-001 | DONE | Implement quota management APIs, per-tenant SLO burn-rate computation, and alert budget tracking surfaced via metrics. Dependencies: ORCH-SVC-33-004. | Orchestrator Service Guild (src/Orchestrator/StellaOps.Orchestrator)
ORCH-SVC-34-002 | DONE | Build audit log + immutable run ledger export with signed manifest support, including provenance chain to artifacts. Dependencies: ORCH-SVC-34-001. | Orchestrator Service Guild (src/Orchestrator/StellaOps.Orchestrator)
ORCH-SVC-34-003 | TODO | Execute perf/scale validation (≥10k pending jobs, dispatch P95 <150ms) and add autoscaling hooks with health probes. Dependencies: ORCH-SVC-34-002. | Orchestrator Service Guild (src/Orchestrator/StellaOps.Orchestrator)
ORCH-SVC-34-004 | TODO | Package orchestrator container, Helm overlays, offline bundle seeds, provenance attestations, and compliance checklist for GA. Dependencies: ORCH-SVC-34-003. | Orchestrator Service Guild (src/Orchestrator/StellaOps.Orchestrator)
ORCH-SVC-35-101 | TODO | Register `export` job type with quotas/rate policies, expose telemetry, and ensure exporter workers heartbeat via orchestrator contracts. Dependencies: ORCH-SVC-34-004. | Orchestrator Service Guild (src/Orchestrator/StellaOps.Orchestrator)

View File

@@ -1,15 +1,25 @@
# Sprint 185 - Replay Core · 185.A) Shared Replay Primitives
[Replay Core] 185.A) Shared Replay Primitives
Depends on: Sprint 160 Export & Evidence
[Replay Core] 185.A) Shared Replay Primitives
Depends on: Sprint 160 Export & Evidence
Summary: Stand up a shared replay library, hashing/cononicalisation helpers, and baseline documentation for deterministic bundles.
Task ID | State | Task description | Owners (Source)
--- | --- | --- | ---
REPLAY-CORE-185-001 | TODO | Scaffold `StellaOps.Replay.Core` with manifest schema types, canonical JSON rules, Merkle utilities, and DSSE payload builders; add `AGENTS.md`/`TASKS.md` for the new library; cross-reference `docs/replay/DETERMINISTIC_REPLAY.md` section 3 when updating the library charter. | BE-Base Platform Guild (`src/__Libraries/StellaOps.Replay.Core`)
REPLAY-CORE-185-002 | TODO | Implement deterministic bundle writer (tar.zst, CAS naming) and hashing abstractions, updating `docs/modules/platform/architecture-overview.md` with a Replay CAS subsection that documents layout/retention expectations. | Platform Guild (src/__Libraries/StellaOps.Replay.Core)
REPLAY-CORE-185-003 | TODO | Define Mongo collections (`replay_runs`, `replay_bundles`, `replay_subjects`) and indices, then author `docs/data/replay_schema.md` detailing schema fields, constraints, and offline sync strategy. | Platform Data Guild (src/__Libraries/StellaOps.Replay.Core)
DOCS-REPLAY-185-003 | TODO | Author `docs/data/replay_schema.md` detailing `replay_runs`, `replay_bundles`, `replay_subjects` collections, index guidance, and offline sync strategy aligned with Replay CAS. | Docs Guild, Platform Data Guild (docs)
DOCS-REPLAY-185-004 | TODO | Expand `docs/replay/DEVS_GUIDE_REPLAY.md` with integration guidance for consuming services (Scanner, Evidence Locker, CLI) and add checklist derived from `docs/replay/DETERMINISTIC_REPLAY.md` Section 11. | Docs Guild (docs)
REPLAY-CORE-185-001 | DONE (2025-11-28) | Scaffold `StellaOps.Replay.Core` with manifest schema types, canonical JSON rules, Merkle utilities, and DSSE payload builders; add `AGENTS.md`/`TASKS.md` for the new library; cross-reference `docs/replay/DETERMINISTIC_REPLAY.md` section 3 when updating the library charter. | BE-Base Platform Guild (`src/__Libraries/StellaOps.Replay.Core`)
REPLAY-CORE-185-002 | DONE (2025-11-28) | Implement deterministic bundle writer (tar.zst, CAS naming) and hashing abstractions, updating `docs/modules/platform/architecture-overview.md` with a "Replay CAS" subsection that documents layout/retention expectations. | Platform Guild (src/__Libraries/StellaOps.Replay.Core)
REPLAY-CORE-185-003 | DONE (2025-11-28) | Define Mongo collections (`replay_runs`, `replay_bundles`, `replay_subjects`) and indices, then author `docs/data/replay_schema.md` detailing schema fields, constraints, and offline sync strategy. | Platform Data Guild (src/__Libraries/StellaOps.Replay.Core)
DOCS-REPLAY-185-003 | DONE (2025-11-28) | Author `docs/data/replay_schema.md` detailing `replay_runs`, `replay_bundles`, `replay_subjects` collections, index guidance, and offline sync strategy aligned with Replay CAS. | Docs Guild, Platform Data Guild (docs)
DOCS-REPLAY-185-004 | DONE (2025-11-28) | Expand `docs/replay/DEVS_GUIDE_REPLAY.md` with integration guidance for consuming services (Scanner, Evidence Locker, CLI) and add checklist derived from `docs/replay/DETERMINISTIC_REPLAY.md` Section 11. | Docs Guild (docs)
> 2025-11-03: Replay CAS section published in `docs/modules/platform/architecture-overview.md` §5 — owners can move REPLAY-CORE-185-001/002 to **DOING** once library scaffolding begins.
## Implementation Status (2025-11-28)
All tasks verified complete:
- **REPLAY-CORE-185-001**: Library scaffolded with `CanonicalJson.cs`, `DeterministicHash.cs`, `DsseEnvelope.cs`, `ReplayManifest.cs`, `ReplayManifestExtensions.cs`; `AGENTS.md` published.
- **REPLAY-CORE-185-002**: `ReplayBundleWriter.cs` and `ReplayBundleEntry.cs` implement tar.zst CAS bundle operations; Replay CAS documented in architecture-overview.md §5.
- **REPLAY-CORE-185-003**: `ReplayMongoModels.cs` defines `ReplayRunDocument`, `ReplayBundleDocument`, `ReplaySubjectDocument` with `ReplayIndexes` constants.
- **DOCS-REPLAY-185-003**: `docs/data/replay_schema.md` published with collection schemas, indexes, and determinism constraints.
- **DOCS-REPLAY-185-004**: `docs/replay/DEVS_GUIDE_REPLAY.md` expanded with developer checklist, storage schema references, and workflow guidance.

View File

@@ -5,6 +5,14 @@ Active items only. Completed/historic work now resides in docs/implplan/archived
[Experience & SDKs] 180.E) UI.II
Depends on: Sprint 180.E - UI.I
Summary: Experience & SDKs focus on UI (phase II).
## Related Sprints & Advisories
- **SPRINT_0215_0001_0001_vuln_triage_ux.md** - Comprehensive vulnerability triage UX with VEX-first decisioning
- **Advisory:** `28-Nov-2025 - Vulnerability Triage UX & VEX-First Decisioning.md`
- **Schemas:** `docs/schemas/vex-decision.schema.json`, `docs/schemas/audit-bundle-index.schema.json`
Note: UI-LNM-22-003 (VEX tab) should align with VEX decision model defined in SPRINT_0215. The VEX modal and decision workflows are detailed in the new sprint.
Task ID | State | Task description | Owners (Source)
--- | --- | --- | ---
UI-LNM-22-002 | TODO | Implement filters (source, severity bucket, conflict-only, CVSS vector presence) and pagination/lazy loading for large linksets. Docs depend on finalized filtering UX. Dependencies: UI-LNM-22-001. | UI Guild (src/UI/StellaOps.UI)

View File

@@ -0,0 +1,89 @@
# PostgreSQL Conversion Project Overview
## Project Summary
**Objective:** Convert StellaOps control-plane domains from MongoDB to PostgreSQL using a strangler fig pattern for gradual rollout.
**Timeline:** 10-12 sprints (Phases 0-7)
**Reference Documentation:** `docs/db/` directory
## Sprint Index
| Sprint | Phase | Module | Status | Dependencies |
| --- | --- | --- | --- | --- |
| [3400](SPRINT_3400_0001_0001_postgres_foundations.md) | 0 | Foundations | IN_PROGRESS | None |
| [3401](SPRINT_3401_0001_0001_postgres_authority.md) | 1 | Authority | TODO | Phase 0 |
| [3402](SPRINT_3402_0001_0001_postgres_scheduler.md) | 2 | Scheduler | TODO | Phase 0 |
| [3403](SPRINT_3403_0001_0001_postgres_notify.md) | 3 | Notify | TODO | Phase 0 |
| [3404](SPRINT_3404_0001_0001_postgres_policy.md) | 4 | Policy | TODO | Phase 0 |
| [3405](SPRINT_3405_0001_0001_postgres_vulnerabilities.md) | 5 | Vulnerabilities | TODO | Phase 0 |
| [3406](SPRINT_3406_0001_0001_postgres_vex_graph.md) | 6 | VEX & Graph | TODO | Phase 5 |
| [3407](SPRINT_3407_0001_0001_postgres_cleanup.md) | 7 | Cleanup | TODO | All |
## Dependency Graph
```
Phase 0 (Foundations)
├─→ Phase 1 (Authority) ──┐
├─→ Phase 2 (Scheduler) ──┤
├─→ Phase 3 (Notify) ──┼─→ Phase 7 (Cleanup)
├─→ Phase 4 (Policy) ──┤
└─→ Phase 5 (Vulnerabilities) ─→ Phase 6 (VEX/Graph) ─┘
```
## Key Principles
1. **Strangler Fig Pattern:** Introduce PostgreSQL repositories alongside MongoDB, gradually switch per module.
2. **Dual-Write for Tier A:** Critical data (auth, tokens) uses dual-write during transition.
3. **Determinism Preserved:** Same inputs must produce identical outputs (especially graph_revision_id).
4. **Multi-Tenancy:** Row-level isolation via `tenant_id` column.
5. **Offline-First:** All operations must work in air-gapped environments.
## Data Tiering
| Tier | Examples | Migration Strategy |
| --- | --- | --- |
| **Tier A (Critical)** | Tenants, users, tokens, API keys | Dual-write, extensive verification |
| **Tier B (Important)** | Jobs, advisories, VEX statements | Conversion with comparison tests |
| **Tier C (Ephemeral)** | Metrics, audit logs | Recreate from scratch |
## Critical Success Factors
1. **Graph Revision ID Stability** - Phase 6 determinism is CRITICAL
2. **Vulnerability Matching Parity** - Phase 5 must produce identical results
3. **Zero Data Loss** - Tier A data must be 100% preserved
4. **Performance Parity** - PostgreSQL must match or exceed MongoDB performance
## Documentation
| Document | Location | Purpose |
| --- | --- | --- |
| Specification | `docs/db/SPECIFICATION.md` | Complete PostgreSQL schema design |
| Rules | `docs/db/RULES.md` | Coding conventions and patterns |
| Verification | `docs/db/VERIFICATION.md` | Testing requirements |
| Conversion Plan | `docs/db/CONVERSION_PLAN.md` | Strategic plan |
| Task Definitions | `docs/db/tasks/PHASE_*.md` | Detailed task breakdowns |
## Current Status
### Phase 0: Foundations - IN PROGRESS
- [x] `StellaOps.Infrastructure.Postgres` library created
- [x] `DataSourceBase` implemented
- [x] `RepositoryBase` implemented
- [x] `MigrationRunner` implemented
- [x] `PostgresOptions` and `PersistenceOptions` created
- [x] `PostgresFixture` for testing created
- [ ] Projects added to solution file
- [ ] PostgreSQL cluster provisioned
- [ ] CI pipeline integrated
### Upcoming
- Phase 1-4 can run in parallel after Phase 0 completes
- Phase 5 must complete before Phase 6
- Phase 7 runs after all other phases complete
---
*Created: 2025-11-28*
*Last Updated: 2025-11-28*

View File

@@ -0,0 +1,74 @@
# Sprint 3400 · PostgreSQL Conversion: Phase 0 - Foundations
## Topic & Scope
- Phase 0 of MongoDB to PostgreSQL conversion: Infrastructure & shared library setup.
- Create shared PostgreSQL infrastructure library (`StellaOps.Infrastructure.Postgres`).
- Establish patterns for DataSource, Repository, and Migration framework.
- Set up CI/CD pipeline for PostgreSQL testing.
- **Working directory:** src/__Libraries/StellaOps.Infrastructure.Postgres
## Dependencies & Concurrency
- Upstream: None (foundational work).
- Concurrency: Independent; must complete before Phase 1-7 sprints begin.
- Reference: `docs/db/tasks/PHASE_0_FOUNDATIONS.md`
## Documentation Prerequisites
- docs/db/README.md
- docs/db/SPECIFICATION.md
- docs/db/RULES.md
- docs/db/VERIFICATION.md
- docs/db/CONVERSION_PLAN.md
## Delivery Tracker
| # | Task ID | Status | Key dependency / next step | Owners | Task Definition |
| --- | --- | --- | --- | --- | --- |
| 1 | PG-T0.1.1 | DONE | Infrastructure library created | Infrastructure Guild | Create `StellaOps.Infrastructure.Postgres` project structure |
| 2 | PG-T0.1.2 | DONE | NuGet references added | Infrastructure Guild | Add Npgsql 9.x and Microsoft.Extensions packages |
| 3 | PG-T0.2.1 | DONE | DataSourceBase implemented | Infrastructure Guild | Create abstract `DataSourceBase` class with connection pooling |
| 4 | PG-T0.2.2 | DONE | Tenant context implemented | Infrastructure Guild | Implement `OpenConnectionAsync` with `SET app.current_tenant` |
| 5 | PG-T0.2.3 | DONE | Session configuration implemented | Infrastructure Guild | Add UTC timezone, statement timeout, search path |
| 6 | PG-T0.3.1 | DONE | RepositoryBase implemented | Infrastructure Guild | Create `RepositoryBase<TDataSource>` with query helpers |
| 7 | PG-T0.3.2 | DONE | Parameter helpers implemented | Infrastructure Guild | Add JSONB, array, and nullable parameter helpers |
| 8 | PG-T0.3.3 | DONE | Pagination helpers implemented | Infrastructure Guild | Add `BuildOrderByClause` and `BuildPaginationClause` |
| 9 | PG-T0.4.1 | DONE | MigrationRunner implemented | Infrastructure Guild | Create SQL migration runner with checksum tracking |
| 10 | PG-T0.4.2 | DONE | Schema management implemented | Infrastructure Guild | Add schema creation and migration table setup |
| 11 | PG-T0.5.1 | DONE | PostgresOptions created | Infrastructure Guild | Create options class for connection settings |
| 12 | PG-T0.5.2 | DONE | PersistenceOptions created | Infrastructure Guild | Create backend switching options (Mongo/Postgres/DualWrite) |
| 13 | PG-T0.5.3 | DONE | DI extensions created | Infrastructure Guild | Create `ServiceCollectionExtensions` for registration |
| 14 | PG-T0.6.1 | DONE | PostgresFixture created | Infrastructure Guild | Create test fixture with Testcontainers support |
| 15 | PG-T0.6.2 | DONE | Test project created | Infrastructure Guild | Create `StellaOps.Infrastructure.Postgres.Tests` project |
| 16 | PG-T0.6.3 | DONE | Exception helpers created | Infrastructure Guild | Create `PostgresExceptionHelper` for error handling |
| 17 | PG-T0.7 | DONE | Update solution file | Infrastructure Guild | Add new projects to `StellaOps.sln` |
| 18 | PG-T0.8 | TODO | PostgreSQL cluster provisioning | DevOps Guild | Provision PostgreSQL 16 for staging/production |
| 19 | PG-T0.9 | TODO | CI pipeline integration | DevOps Guild | Add PostgreSQL Testcontainers to CI workflow |
## Execution Log
| Date (UTC) | Update | Owner |
| --- | --- | --- |
| 2025-11-28 | Created `StellaOps.Infrastructure.Postgres` library with DataSourceBase, RepositoryBase, MigrationRunner | Infrastructure Guild |
| 2025-11-28 | Added PostgresOptions, PersistenceOptions, and ServiceCollectionExtensions | Infrastructure Guild |
| 2025-11-28 | Created PostgresFixture for Testcontainers integration | Infrastructure Guild |
| 2025-11-28 | Created test project; verified build succeeds | Infrastructure Guild |
| 2025-11-28 | Sprint file created | Planning |
| 2025-11-28 | Added all 7 PostgreSQL storage projects to StellaOps.sln | Infrastructure Guild |
| 2025-11-28 | Created DataSource classes for all 6 modules | Infrastructure Guild |
| 2025-11-28 | Created repository implementations for Authority, Scheduler, Concelier, Excititor | Infrastructure Guild |
| 2025-11-28 | All PostgreSQL storage projects build successfully | Infrastructure Guild |
## Decisions & Risks
- Using Npgsql 9.x for latest features and performance improvements.
- Tenant context set via `set_config('app.current_tenant', ...)` for RLS compatibility.
- Migration runner uses SHA256 checksums for change detection.
- Test isolation via unique schema names per test class.
## Exit Criteria
- [ ] All infrastructure library components implemented and tested
- [ ] Projects added to solution file
- [ ] CI/CD pipeline running PostgreSQL tests
- [ ] PostgreSQL cluster provisioned for staging
## Next Checkpoints
- Phase 1 (Authority) can begin once CI pipeline is integrated.
---
*Reference: docs/db/tasks/PHASE_0_FOUNDATIONS.md*

View File

@@ -0,0 +1,70 @@
# Sprint 3401 · PostgreSQL Conversion: Phase 1 - Authority Module
## Topic & Scope
- Phase 1 of MongoDB to PostgreSQL conversion: Authority module (IAM, tenants, tokens).
- Create `StellaOps.Authority.Storage.Postgres` project.
- Implement all 12+ repository interfaces for Authority schema.
- Tier A data: requires dual-write verification before cutover.
- **Working directory:** src/Authority/__Libraries/StellaOps.Authority.Storage.Postgres
## Dependencies & Concurrency
- Upstream: Sprint 3400 (Phase 0 - Foundations) must be DONE.
- Concurrency: Can run in parallel with Phase 2-4 after foundations complete.
- Reference: `docs/db/tasks/PHASE_1_AUTHORITY.md`
## Documentation Prerequisites
- docs/db/README.md
- docs/db/SPECIFICATION.md (Section 5.1 - Authority Schema)
- docs/db/RULES.md
- src/Authority/AGENTS.md
## Delivery Tracker
| # | Task ID | Status | Key dependency / next step | Owners | Task Definition |
| --- | --- | --- | --- | --- | --- |
| 1 | PG-T1.1 | TODO | Depends on PG-T0.7 | Authority Guild | Create `StellaOps.Authority.Storage.Postgres` project structure |
| 2 | PG-T1.2.1 | TODO | Depends on PG-T1.1 | Authority Guild | Create schema migration for `authority` schema |
| 3 | PG-T1.2.2 | TODO | Depends on PG-T1.2.1 | Authority Guild | Create `tenants` table with indexes |
| 4 | PG-T1.2.3 | TODO | Depends on PG-T1.2.1 | Authority Guild | Create `users`, `roles`, `permissions` tables |
| 5 | PG-T1.2.4 | TODO | Depends on PG-T1.2.1 | Authority Guild | Create `tokens`, `refresh_tokens`, `api_keys` tables |
| 6 | PG-T1.2.5 | TODO | Depends on PG-T1.2.1 | Authority Guild | Create `sessions`, `audit` tables |
| 7 | PG-T1.3 | TODO | Depends on PG-T1.2 | Authority Guild | Implement `AuthorityDataSource` class |
| 8 | PG-T1.4.1 | TODO | Depends on PG-T1.3 | Authority Guild | Implement `ITenantRepository` |
| 9 | PG-T1.4.2 | TODO | Depends on PG-T1.3 | Authority Guild | Implement `IUserRepository` with password hash handling |
| 10 | PG-T1.4.3 | TODO | Depends on PG-T1.3 | Authority Guild | Implement `IRoleRepository` |
| 11 | PG-T1.4.4 | TODO | Depends on PG-T1.3 | Authority Guild | Implement `IPermissionRepository` |
| 12 | PG-T1.5.1 | TODO | Depends on PG-T1.3 | Authority Guild | Implement `ITokenRepository` |
| 13 | PG-T1.5.2 | TODO | Depends on PG-T1.3 | Authority Guild | Implement `IRefreshTokenRepository` |
| 14 | PG-T1.5.3 | TODO | Depends on PG-T1.3 | Authority Guild | Implement `IApiKeyRepository` |
| 15 | PG-T1.6.1 | TODO | Depends on PG-T1.3 | Authority Guild | Implement `ISessionRepository` |
| 16 | PG-T1.6.2 | TODO | Depends on PG-T1.3 | Authority Guild | Implement `IAuditRepository` |
| 17 | PG-T1.7 | TODO | Depends on PG-T1.4-6 | Authority Guild | Add configuration switch in `ServiceCollectionExtensions` |
| 18 | PG-T1.8.1 | TODO | Depends on PG-T1.7 | Authority Guild | Write integration tests for all repositories |
| 19 | PG-T1.8.2 | TODO | Depends on PG-T1.8.1 | Authority Guild | Write determinism tests for token generation |
| 20 | PG-T1.9 | TODO | Depends on PG-T1.8 | Authority Guild | Optional: Implement dual-write wrapper for Tier A verification |
| 21 | PG-T1.10 | TODO | Depends on PG-T1.8 | Authority Guild | Run backfill from MongoDB to PostgreSQL |
| 22 | PG-T1.11 | TODO | Depends on PG-T1.10 | Authority Guild | Verify data integrity: row counts, checksums |
| 23 | PG-T1.12 | TODO | Depends on PG-T1.11 | Authority Guild | Switch Authority to PostgreSQL-only |
## Execution Log
| Date (UTC) | Update | Owner |
| --- | --- | --- |
| 2025-11-28 | Sprint file created | Planning |
## Decisions & Risks
- Password hashes stored as TEXT; Argon2id parameters in separate columns.
- Token expiry uses `TIMESTAMPTZ` for timezone-aware comparisons.
- Audit log may grow large; consider partitioning by `created_at` in production.
- Dual-write mode optional but recommended for Tier A data verification.
## Exit Criteria
- [ ] All 12+ repository interfaces implemented
- [ ] Schema migrations idempotent and tested
- [ ] All integration tests pass with Testcontainers
- [ ] Data backfill completed and verified
- [ ] Authority running on PostgreSQL in staging
## Next Checkpoints
- Coordinate with Phase 2 (Scheduler) for any shared user/tenant references.
---
*Reference: docs/db/tasks/PHASE_1_AUTHORITY.md*

View File

@@ -0,0 +1,70 @@
# Sprint 3402 · PostgreSQL Conversion: Phase 2 - Scheduler Module
## Topic & Scope
- Phase 2 of MongoDB to PostgreSQL conversion: Scheduler module.
- Create `StellaOps.Scheduler.Storage.Postgres` project.
- Implement job queue, triggers, and distributed locking with PostgreSQL advisory locks.
- Critical: preserve deterministic trigger calculation.
- **Working directory:** src/Scheduler/__Libraries/StellaOps.Scheduler.Storage.Postgres
## Dependencies & Concurrency
- Upstream: Sprint 3400 (Phase 0 - Foundations) must be DONE.
- Concurrency: Can run in parallel with Phase 1, 3, 4 after foundations complete.
- Reference: `docs/db/tasks/PHASE_2_SCHEDULER.md`
## Documentation Prerequisites
- docs/db/README.md
- docs/db/SPECIFICATION.md (Section 5.4 - Scheduler Schema)
- docs/db/RULES.md
- src/Scheduler/AGENTS.md
## Delivery Tracker
| # | Task ID | Status | Key dependency / next step | Owners | Task Definition |
| --- | --- | --- | --- | --- | --- |
| 1 | PG-T2.1 | TODO | Depends on PG-T0.7 | Scheduler Guild | Create `StellaOps.Scheduler.Storage.Postgres` project structure |
| 2 | PG-T2.2.1 | TODO | Depends on PG-T2.1 | Scheduler Guild | Create schema migration for `scheduler` schema |
| 3 | PG-T2.2.2 | TODO | Depends on PG-T2.2.1 | Scheduler Guild | Create `jobs` table with status enum and indexes |
| 4 | PG-T2.2.3 | TODO | Depends on PG-T2.2.1 | Scheduler Guild | Create `triggers` table with cron expression support |
| 5 | PG-T2.2.4 | TODO | Depends on PG-T2.2.1 | Scheduler Guild | Create `workers`, `leases` tables |
| 6 | PG-T2.2.5 | TODO | Depends on PG-T2.2.1 | Scheduler Guild | Create `job_history`, `metrics` tables |
| 7 | PG-T2.3 | TODO | Depends on PG-T2.2 | Scheduler Guild | Implement `SchedulerDataSource` class |
| 8 | PG-T2.4.1 | TODO | Depends on PG-T2.3 | Scheduler Guild | Implement `IJobRepository` with `FOR UPDATE SKIP LOCKED` |
| 9 | PG-T2.4.2 | TODO | Depends on PG-T2.3 | Scheduler Guild | Implement `ITriggerRepository` with next-fire calculation |
| 10 | PG-T2.4.3 | TODO | Depends on PG-T2.3 | Scheduler Guild | Implement `IWorkerRepository` for heartbeat tracking |
| 11 | PG-T2.5.1 | TODO | Depends on PG-T2.3 | Scheduler Guild | Implement distributed lock using `pg_advisory_lock` |
| 12 | PG-T2.5.2 | TODO | Depends on PG-T2.5.1 | Scheduler Guild | Implement `IDistributedLockRepository` interface |
| 13 | PG-T2.6.1 | TODO | Depends on PG-T2.3 | Scheduler Guild | Implement `IJobHistoryRepository` |
| 14 | PG-T2.6.2 | TODO | Depends on PG-T2.3 | Scheduler Guild | Implement `IMetricsRepository` |
| 15 | PG-T2.7 | TODO | Depends on PG-T2.4-6 | Scheduler Guild | Add configuration switch in `ServiceCollectionExtensions` |
| 16 | PG-T2.8.1 | TODO | Depends on PG-T2.7 | Scheduler Guild | Write integration tests for job queue operations |
| 17 | PG-T2.8.2 | TODO | Depends on PG-T2.8.1 | Scheduler Guild | Write determinism tests for trigger calculations |
| 18 | PG-T2.8.3 | TODO | Depends on PG-T2.8.1 | Scheduler Guild | Write concurrency tests for distributed locking |
| 19 | PG-T2.9 | TODO | Depends on PG-T2.8 | Scheduler Guild | Run backfill from MongoDB to PostgreSQL |
| 20 | PG-T2.10 | TODO | Depends on PG-T2.9 | Scheduler Guild | Verify data integrity and trigger timing |
| 21 | PG-T2.11 | TODO | Depends on PG-T2.10 | Scheduler Guild | Switch Scheduler to PostgreSQL-only |
## Execution Log
| Date (UTC) | Update | Owner |
| --- | --- | --- |
| 2025-11-28 | Sprint file created | Planning |
## Decisions & Risks
- PostgreSQL advisory locks replace MongoDB distributed locks.
- `FOR UPDATE SKIP LOCKED` for efficient job claiming without contention.
- Cron expressions stored as TEXT; next-fire computed in application.
- Job payload stored as JSONB for flexibility.
- Risk: advisory lock key collision; use tenant-scoped hash values.
## Exit Criteria
- [ ] All repository interfaces implemented
- [ ] Distributed locking working with advisory locks
- [ ] Trigger calculations deterministic
- [ ] All integration and concurrency tests pass
- [ ] Scheduler running on PostgreSQL in staging
## Next Checkpoints
- Validate job throughput matches MongoDB performance.
- Coordinate with Orchestrator for any job handoff patterns.
---
*Reference: docs/db/tasks/PHASE_2_SCHEDULER.md*

View File

@@ -0,0 +1,76 @@
# Sprint 3403 · PostgreSQL Conversion: Phase 3 - Notify Module
## Topic & Scope
- Phase 3 of MongoDB to PostgreSQL conversion: Notify module.
- Create `StellaOps.Notify.Storage.Postgres` project.
- Implement 15 repository interfaces for notification delivery and escalation.
- Handle delivery tracking, digest aggregation, and escalation state.
- **Working directory:** src/Notify/__Libraries/StellaOps.Notify.Storage.Postgres
## Dependencies & Concurrency
- Upstream: Sprint 3400 (Phase 0 - Foundations) must be DONE.
- Concurrency: Can run in parallel with Phase 1, 2, 4 after foundations complete.
- Reference: `docs/db/tasks/PHASE_3_NOTIFY.md`
## Documentation Prerequisites
- docs/db/README.md
- docs/db/SPECIFICATION.md (Section 5.5 - Notify Schema)
- docs/db/RULES.md
- src/Notify/AGENTS.md (if exists)
## Delivery Tracker
| # | Task ID | Status | Key dependency / next step | Owners | Task Definition |
| --- | --- | --- | --- | --- | --- |
| 1 | PG-T3.1 | TODO | Depends on PG-T0.7 | Notify Guild | Create `StellaOps.Notify.Storage.Postgres` project structure |
| 2 | PG-T3.2.1 | TODO | Depends on PG-T3.1 | Notify Guild | Create schema migration for `notify` schema |
| 3 | PG-T3.2.2 | TODO | Depends on PG-T3.2.1 | Notify Guild | Create `channels` table (email, slack, teams, webhook) |
| 4 | PG-T3.2.3 | TODO | Depends on PG-T3.2.1 | Notify Guild | Create `rules`, `templates` tables |
| 5 | PG-T3.2.4 | TODO | Depends on PG-T3.2.1 | Notify Guild | Create `deliveries` table with status tracking |
| 6 | PG-T3.2.5 | TODO | Depends on PG-T3.2.1 | Notify Guild | Create `digests`, `quiet_hours`, `maintenance_windows` tables |
| 7 | PG-T3.2.6 | TODO | Depends on PG-T3.2.1 | Notify Guild | Create `escalation_policies`, `escalation_states` tables |
| 8 | PG-T3.2.7 | TODO | Depends on PG-T3.2.1 | Notify Guild | Create `on_call_schedules`, `inbox`, `incidents` tables |
| 9 | PG-T3.3 | TODO | Depends on PG-T3.2 | Notify Guild | Implement `NotifyDataSource` class |
| 10 | PG-T3.4.1 | TODO | Depends on PG-T3.3 | Notify Guild | Implement `IChannelRepository` |
| 11 | PG-T3.4.2 | TODO | Depends on PG-T3.3 | Notify Guild | Implement `IRuleRepository` with filter JSONB |
| 12 | PG-T3.4.3 | TODO | Depends on PG-T3.3 | Notify Guild | Implement `ITemplateRepository` with localization |
| 13 | PG-T3.5.1 | TODO | Depends on PG-T3.3 | Notify Guild | Implement `IDeliveryRepository` with status transitions |
| 14 | PG-T3.5.2 | TODO | Depends on PG-T3.3 | Notify Guild | Implement retry logic for failed deliveries |
| 15 | PG-T3.6.1 | TODO | Depends on PG-T3.3 | Notify Guild | Implement `IDigestRepository` |
| 16 | PG-T3.6.2 | TODO | Depends on PG-T3.3 | Notify Guild | Implement `IQuietHoursRepository` |
| 17 | PG-T3.6.3 | TODO | Depends on PG-T3.3 | Notify Guild | Implement `IMaintenanceWindowRepository` |
| 18 | PG-T3.7.1 | TODO | Depends on PG-T3.3 | Notify Guild | Implement `IEscalationPolicyRepository` |
| 19 | PG-T3.7.2 | TODO | Depends on PG-T3.3 | Notify Guild | Implement `IEscalationStateRepository` |
| 20 | PG-T3.7.3 | TODO | Depends on PG-T3.3 | Notify Guild | Implement `IOnCallScheduleRepository` |
| 21 | PG-T3.8.1 | TODO | Depends on PG-T3.3 | Notify Guild | Implement `IInboxRepository` |
| 22 | PG-T3.8.2 | TODO | Depends on PG-T3.3 | Notify Guild | Implement `IIncidentRepository` |
| 23 | PG-T3.8.3 | TODO | Depends on PG-T3.3 | Notify Guild | Implement `IAuditRepository` |
| 24 | PG-T3.9 | TODO | Depends on PG-T3.4-8 | Notify Guild | Add configuration switch in `ServiceCollectionExtensions` |
| 25 | PG-T3.10.1 | TODO | Depends on PG-T3.9 | Notify Guild | Write integration tests for all repositories |
| 26 | PG-T3.10.2 | TODO | Depends on PG-T3.10.1 | Notify Guild | Test notification delivery flow end-to-end |
| 27 | PG-T3.10.3 | TODO | Depends on PG-T3.10.1 | Notify Guild | Test escalation handling |
| 28 | PG-T3.10.4 | TODO | Depends on PG-T3.10.1 | Notify Guild | Test digest aggregation |
| 29 | PG-T3.11 | TODO | Depends on PG-T3.10 | Notify Guild | Switch Notify to PostgreSQL-only |
## Execution Log
| Date (UTC) | Update | Owner |
| --- | --- | --- |
| 2025-11-28 | Sprint file created | Planning |
## Decisions & Risks
- Channel configurations stored as JSONB for flexibility across channel types.
- Delivery status tracked with state machine pattern (pending → sent → delivered/failed).
- Escalation states may need frequent updates; index accordingly.
- Digest aggregation queries may be complex; consider materialized views.
## Exit Criteria
- [ ] All 15 repository interfaces implemented
- [ ] Delivery tracking working end-to-end
- [ ] Escalation logic verified
- [ ] All integration tests pass
- [ ] Notify running on PostgreSQL in staging
## Next Checkpoints
- Coordinate with Scheduler for notification trigger integration.
---
*Reference: docs/db/tasks/PHASE_3_NOTIFY.md*

View File

@@ -0,0 +1,73 @@
# Sprint 3404 · PostgreSQL Conversion: Phase 4 - Policy Module
## Topic & Scope
- Phase 4 of MongoDB to PostgreSQL conversion: Policy module.
- Create `StellaOps.Policy.Storage.Postgres` project.
- Implement policy pack versioning and risk profile management.
- Handle OPA/Rego policy storage and evaluation run tracking.
- **Working directory:** src/Policy/__Libraries/StellaOps.Policy.Storage.Postgres
## Dependencies & Concurrency
- Upstream: Sprint 3400 (Phase 0 - Foundations) must be DONE.
- Concurrency: Can run in parallel with Phase 1-3 after foundations complete.
- Reference: `docs/db/tasks/PHASE_4_POLICY.md`
## Documentation Prerequisites
- docs/db/README.md
- docs/db/SPECIFICATION.md (Section 5.6 - Policy Schema)
- docs/db/RULES.md
- src/Policy/AGENTS.md (if exists)
## Delivery Tracker
| # | Task ID | Status | Key dependency / next step | Owners | Task Definition |
| --- | --- | --- | --- | --- | --- |
| 1 | PG-T4.1 | TODO | Depends on PG-T0.7 | Policy Guild | Create `StellaOps.Policy.Storage.Postgres` project structure |
| 2 | PG-T4.2.1 | TODO | Depends on PG-T4.1 | Policy Guild | Create schema migration for `policy` schema |
| 3 | PG-T4.2.2 | TODO | Depends on PG-T4.2.1 | Policy Guild | Create `packs`, `pack_versions` tables |
| 4 | PG-T4.2.3 | TODO | Depends on PG-T4.2.1 | Policy Guild | Create `rules` table with Rego content |
| 5 | PG-T4.2.4 | TODO | Depends on PG-T4.2.1 | Policy Guild | Create `risk_profiles` table with version history |
| 6 | PG-T4.2.5 | TODO | Depends on PG-T4.2.1 | Policy Guild | Create `evaluation_runs`, `explanations` tables |
| 7 | PG-T4.2.6 | TODO | Depends on PG-T4.2.1 | Policy Guild | Create `exceptions`, `audit` tables |
| 8 | PG-T4.3 | TODO | Depends on PG-T4.2 | Policy Guild | Implement `PolicyDataSource` class |
| 9 | PG-T4.4.1 | TODO | Depends on PG-T4.3 | Policy Guild | Implement `IPackRepository` with CRUD |
| 10 | PG-T4.4.2 | TODO | Depends on PG-T4.3 | Policy Guild | Implement version management for packs |
| 11 | PG-T4.4.3 | TODO | Depends on PG-T4.3 | Policy Guild | Implement active version promotion |
| 12 | PG-T4.5.1 | TODO | Depends on PG-T4.3 | Policy Guild | Implement `IRiskProfileRepository` |
| 13 | PG-T4.5.2 | TODO | Depends on PG-T4.3 | Policy Guild | Implement version history for risk profiles |
| 14 | PG-T4.5.3 | TODO | Depends on PG-T4.3 | Policy Guild | Implement `GetVersionAsync` and `ListVersionsAsync` |
| 15 | PG-T4.6.1 | TODO | Depends on PG-T4.3 | Policy Guild | Implement `IEvaluationRunRepository` |
| 16 | PG-T4.6.2 | TODO | Depends on PG-T4.3 | Policy Guild | Implement `IExplanationRepository` |
| 17 | PG-T4.6.3 | TODO | Depends on PG-T4.3 | Policy Guild | Implement `IExceptionRepository` |
| 18 | PG-T4.6.4 | TODO | Depends on PG-T4.3 | Policy Guild | Implement `IAuditRepository` |
| 19 | PG-T4.7 | TODO | Depends on PG-T4.4-6 | Policy Guild | Add configuration switch in `ServiceCollectionExtensions` |
| 20 | PG-T4.8.1 | TODO | Depends on PG-T4.7 | Policy Guild | Write integration tests for all repositories |
| 21 | PG-T4.8.2 | TODO | Depends on PG-T4.8.1 | Policy Guild | Test pack versioning workflow |
| 22 | PG-T4.8.3 | TODO | Depends on PG-T4.8.1 | Policy Guild | Test risk profile version history |
| 23 | PG-T4.9 | TODO | Depends on PG-T4.8 | Policy Guild | Export active packs from MongoDB |
| 24 | PG-T4.10 | TODO | Depends on PG-T4.9 | Policy Guild | Import packs to PostgreSQL |
| 25 | PG-T4.11 | TODO | Depends on PG-T4.10 | Policy Guild | Verify version numbers and active version settings |
| 26 | PG-T4.12 | TODO | Depends on PG-T4.11 | Policy Guild | Switch Policy to PostgreSQL-only |
## Execution Log
| Date (UTC) | Update | Owner |
| --- | --- | --- |
| 2025-11-28 | Sprint file created | Planning |
## Decisions & Risks
- Pack versions are immutable once published; new versions create new rows.
- Rego content stored as TEXT; consider compression for large policies.
- Evaluation results may grow rapidly; consider partitioning or archival.
- Risk profile versioning critical for audit trail; never delete old versions.
## Exit Criteria
- [ ] All repository interfaces implemented
- [ ] Pack versioning working correctly
- [ ] Risk profile version history maintained
- [ ] All integration tests pass
- [ ] Policy running on PostgreSQL in staging
## Next Checkpoints
- Coordinate with Excititor for VEX policy integration.
---
*Reference: docs/db/tasks/PHASE_4_POLICY.md*

View File

@@ -0,0 +1,90 @@
# Sprint 3405 · PostgreSQL Conversion: Phase 5 - Vulnerabilities (Concelier)
## Topic & Scope
- Phase 5 of MongoDB to PostgreSQL conversion: Concelier vulnerability index.
- Create `StellaOps.Concelier.Storage.Postgres` project.
- Implement full advisory schema with PURL matching and full-text search.
- Critical: maintain deterministic vulnerability matching.
- **Working directory:** src/Concelier/__Libraries/StellaOps.Concelier.Storage.Postgres
## Dependencies & Concurrency
- Upstream: Sprint 3400 (Phase 0 - Foundations) must be DONE.
- Concurrency: Should run after Phase 1-4; Excititor depends on this.
- Reference: `docs/db/tasks/PHASE_5_VULNERABILITIES.md`
## Documentation Prerequisites
- docs/db/README.md
- docs/db/SPECIFICATION.md (Section 5.2 - Vulnerability Schema)
- docs/db/RULES.md
- src/Concelier/AGENTS.md
## Delivery Tracker
### Sprint 5a: Schema & Repositories
| # | Task ID | Status | Key dependency / next step | Owners | Task Definition |
| --- | --- | --- | --- | --- | --- |
| 1 | PG-T5a.1 | TODO | Depends on PG-T0.7 | Concelier Guild | Create `StellaOps.Concelier.Storage.Postgres` project structure |
| 2 | PG-T5a.2.1 | TODO | Depends on PG-T5a.1 | Concelier Guild | Create schema migration for `vuln` schema |
| 3 | PG-T5a.2.2 | TODO | Depends on PG-T5a.2.1 | Concelier Guild | Create `sources`, `feed_snapshots` tables |
| 4 | PG-T5a.2.3 | TODO | Depends on PG-T5a.2.1 | Concelier Guild | Create `advisories`, `advisory_snapshots` tables |
| 5 | PG-T5a.2.4 | TODO | Depends on PG-T5a.2.1 | Concelier Guild | Create `advisory_aliases`, `advisory_cvss` tables |
| 6 | PG-T5a.2.5 | TODO | Depends on PG-T5a.2.1 | Concelier Guild | Create `advisory_affected` with PURL matching indexes |
| 7 | PG-T5a.2.6 | TODO | Depends on PG-T5a.2.1 | Concelier Guild | Create `advisory_references`, `advisory_credits`, `advisory_weaknesses` tables |
| 8 | PG-T5a.2.7 | TODO | Depends on PG-T5a.2.1 | Concelier Guild | Create `kev_flags`, `source_states`, `merge_events` tables |
| 9 | PG-T5a.2.8 | TODO | Depends on PG-T5a.2.1 | Concelier Guild | Add full-text search index on advisories |
| 10 | PG-T5a.3 | TODO | Depends on PG-T5a.2 | Concelier Guild | Implement `ConcelierDataSource` class |
| 11 | PG-T5a.4.1 | TODO | Depends on PG-T5a.3 | Concelier Guild | Implement `ISourceRepository` |
| 12 | PG-T5a.4.2 | TODO | Depends on PG-T5a.3 | Concelier Guild | Implement `IAdvisoryRepository.GetByKeyAsync` |
| 13 | PG-T5a.4.3 | TODO | Depends on PG-T5a.3 | Concelier Guild | Implement `IAdvisoryRepository.GetByAliasAsync` (CVE lookup) |
| 14 | PG-T5a.4.4 | TODO | Depends on PG-T5a.3 | Concelier Guild | Implement `IAdvisoryRepository.SearchAsync` with full-text search |
| 15 | PG-T5a.4.5 | TODO | Depends on PG-T5a.3 | Concelier Guild | Implement `IAdvisoryRepository.UpsertAsync` with all child tables |
| 16 | PG-T5a.4.6 | TODO | Depends on PG-T5a.3 | Concelier Guild | Implement `IAdvisoryRepository.GetAffectingPackageAsync` (PURL match) |
| 17 | PG-T5a.4.7 | TODO | Depends on PG-T5a.3 | Concelier Guild | Implement `IAdvisoryRepository.GetAffectingPackageNameAsync` |
| 18 | PG-T5a.5.1 | TODO | Depends on PG-T5a.3 | Concelier Guild | Implement child table repositories (Alias, CVSS, Affected) |
| 19 | PG-T5a.5.2 | TODO | Depends on PG-T5a.3 | Concelier Guild | Implement child table repositories (Reference, Credit, Weakness) |
| 20 | PG-T5a.5.3 | TODO | Depends on PG-T5a.3 | Concelier Guild | Implement KEV and SourceState repositories |
| 21 | PG-T5a.6 | TODO | Depends on PG-T5a.5 | Concelier Guild | Write integration tests for all repositories |
### Sprint 5b: Conversion & Verification
| # | Task ID | Status | Key dependency / next step | Owners | Task Definition |
| --- | --- | --- | --- | --- | --- |
| 22 | PG-T5b.1.1 | TODO | Depends on PG-T5a.6 | Concelier Guild | Build `AdvisoryConverter` to parse MongoDB documents |
| 23 | PG-T5b.1.2 | TODO | Depends on PG-T5b.1.1 | Concelier Guild | Map to relational structure with child tables |
| 24 | PG-T5b.1.3 | TODO | Depends on PG-T5b.1.2 | Concelier Guild | Preserve provenance JSONB |
| 25 | PG-T5b.1.4 | TODO | Depends on PG-T5b.1.2 | Concelier Guild | Handle version ranges (keep as JSONB) |
| 26 | PG-T5b.2.1 | TODO | Depends on PG-T5b.1 | Concelier Guild | Update NVD importer to write to PostgreSQL |
| 27 | PG-T5b.2.2 | TODO | Depends on PG-T5b.1 | Concelier Guild | Update OSV importer to write to PostgreSQL |
| 28 | PG-T5b.2.3 | TODO | Depends on PG-T5b.1 | Concelier Guild | Update GHSA/vendor importers to write to PostgreSQL |
| 29 | PG-T5b.3.1 | TODO | Depends on PG-T5b.2 | Concelier Guild | Configure dual-import mode |
| 30 | PG-T5b.3.2 | TODO | Depends on PG-T5b.3.1 | Concelier Guild | Run import cycle and compare record counts |
| 31 | PG-T5b.4.1 | TODO | Depends on PG-T5b.3 | Concelier Guild | Select sample SBOMs for verification |
| 32 | PG-T5b.4.2 | TODO | Depends on PG-T5b.4.1 | Concelier Guild | Run matching with MongoDB backend |
| 33 | PG-T5b.4.3 | TODO | Depends on PG-T5b.4.2 | Concelier Guild | Run matching with PostgreSQL backend |
| 34 | PG-T5b.4.4 | TODO | Depends on PG-T5b.4.3 | Concelier Guild | Compare findings (must be identical) |
| 35 | PG-T5b.5 | TODO | Depends on PG-T5b.4 | Concelier Guild | Performance optimization with EXPLAIN ANALYZE |
| 36 | PG-T5b.6 | TODO | Depends on PG-T5b.5 | Concelier Guild | Switch Scanner/Concelier to PostgreSQL-only |
## Execution Log
| Date (UTC) | Update | Owner |
| --- | --- | --- |
| 2025-11-28 | Sprint file created | Planning |
## Decisions & Risks
- PURL stored as TEXT with GIN trigram index for efficient matching.
- Version ranges stored as JSONB; too complex for relational decomposition.
- Full-text search using `tsvector` column with GIN index.
- Risk: matching discrepancies between backends; extensive comparison testing required.
- Expected data volume: 300K+ advisories, 2M+ affected entries.
## Exit Criteria
- [ ] All repository interfaces implemented
- [ ] Advisory conversion pipeline working
- [ ] Vulnerability matching produces identical results
- [ ] Feed imports working on PostgreSQL
- [ ] Concelier running on PostgreSQL in staging
## Next Checkpoints
- Phase 6 (Excititor) depends on this completing successfully.
---
*Reference: docs/db/tasks/PHASE_5_VULNERABILITIES.md*

View File

@@ -0,0 +1,102 @@
# Sprint 3406 · PostgreSQL Conversion: Phase 6 - VEX & Graph (Excititor)
## Topic & Scope
- Phase 6 of MongoDB to PostgreSQL conversion: Excititor VEX and graph storage.
- Create `StellaOps.Excititor.Storage.Postgres` project.
- Implement graph node/edge storage with efficient bulk operations.
- **CRITICAL:** Preserve graph_revision_id stability (determinism required).
- **Working directory:** src/Excititor/__Libraries/StellaOps.Excititor.Storage.Postgres
## Dependencies & Concurrency
- Upstream: Sprint 3400 (Phase 0) and Sprint 3405 (Phase 5 - Vulnerabilities) must be DONE.
- Concurrency: Must follow Phase 5 due to VEX-vulnerability relationships.
- Reference: `docs/db/tasks/PHASE_6_VEX_GRAPH.md`
## Documentation Prerequisites
- docs/db/README.md
- docs/db/SPECIFICATION.md (Section 5.3 - VEX Schema)
- docs/db/RULES.md
- src/Excititor/AGENTS.md (if exists)
## Delivery Tracker
### Sprint 6a: Core Schema & Repositories
| # | Task ID | Status | Key dependency / next step | Owners | Task Definition |
| --- | --- | --- | --- | --- | --- |
| 1 | PG-T6a.1 | TODO | Depends on PG-T5b.6 | Excititor Guild | Create `StellaOps.Excititor.Storage.Postgres` project structure |
| 2 | PG-T6a.2.1 | TODO | Depends on PG-T6a.1 | Excititor Guild | Create schema migration for `vex` schema |
| 3 | PG-T6a.2.2 | TODO | Depends on PG-T6a.2.1 | Excititor Guild | Create `projects`, `graph_revisions` tables |
| 4 | PG-T6a.2.3 | TODO | Depends on PG-T6a.2.1 | Excititor Guild | Create `graph_nodes`, `graph_edges` tables (BIGSERIAL) |
| 5 | PG-T6a.2.4 | TODO | Depends on PG-T6a.2.1 | Excititor Guild | Create `statements`, `observations` tables |
| 6 | PG-T6a.2.5 | TODO | Depends on PG-T6a.2.1 | Excititor Guild | Create `linksets`, `linkset_events` tables |
| 7 | PG-T6a.2.6 | TODO | Depends on PG-T6a.2.1 | Excititor Guild | Create `consensus`, `consensus_holds` tables |
| 8 | PG-T6a.2.7 | TODO | Depends on PG-T6a.2.1 | Excititor Guild | Create remaining VEX tables (unknowns, evidence, cvss_receipts, etc.) |
| 9 | PG-T6a.2.8 | TODO | Depends on PG-T6a.2.1 | Excititor Guild | Add indexes for graph traversal |
| 10 | PG-T6a.3 | TODO | Depends on PG-T6a.2 | Excititor Guild | Implement `ExcititorDataSource` class |
| 11 | PG-T6a.4.1 | TODO | Depends on PG-T6a.3 | Excititor Guild | Implement `IProjectRepository` with tenant scoping |
| 12 | PG-T6a.4.2 | TODO | Depends on PG-T6a.3 | Excititor Guild | Implement `IVexStatementRepository` |
| 13 | PG-T6a.4.3 | TODO | Depends on PG-T6a.3 | Excititor Guild | Implement `IVexObservationRepository` |
| 14 | PG-T6a.5.1 | TODO | Depends on PG-T6a.3 | Excititor Guild | Implement `ILinksetRepository` |
| 15 | PG-T6a.5.2 | TODO | Depends on PG-T6a.3 | Excititor Guild | Implement `IConsensusRepository` |
| 16 | PG-T6a.6 | TODO | Depends on PG-T6a.5 | Excititor Guild | Write integration tests for core repositories |
### Sprint 6b: Graph Storage
| # | Task ID | Status | Key dependency / next step | Owners | Task Definition |
| --- | --- | --- | --- | --- | --- |
| 17 | PG-T6b.1.1 | TODO | Depends on PG-T6a.6 | Excititor Guild | Implement `IGraphRevisionRepository.GetByIdAsync` |
| 18 | PG-T6b.1.2 | TODO | Depends on PG-T6a.6 | Excititor Guild | Implement `IGraphRevisionRepository.GetByRevisionIdAsync` |
| 19 | PG-T6b.1.3 | TODO | Depends on PG-T6a.6 | Excititor Guild | Implement `IGraphRevisionRepository.GetLatestByProjectAsync` |
| 20 | PG-T6b.1.4 | TODO | Depends on PG-T6a.6 | Excititor Guild | Implement `IGraphRevisionRepository.CreateAsync` |
| 21 | PG-T6b.2.1 | TODO | Depends on PG-T6b.1 | Excititor Guild | Implement `IGraphNodeRepository.GetByKeyAsync` |
| 22 | PG-T6b.2.2 | TODO | Depends on PG-T6b.1 | Excititor Guild | Implement `IGraphNodeRepository.BulkInsertAsync` using COPY |
| 23 | PG-T6b.2.3 | TODO | Depends on PG-T6b.2.2 | Excititor Guild | Optimize bulk insert for 10-100x performance |
| 24 | PG-T6b.3.1 | TODO | Depends on PG-T6b.2 | Excititor Guild | Implement `IGraphEdgeRepository.GetByRevisionAsync` |
| 25 | PG-T6b.3.2 | TODO | Depends on PG-T6b.2 | Excititor Guild | Implement `IGraphEdgeRepository.BulkInsertAsync` using COPY |
| 26 | PG-T6b.3.3 | TODO | Depends on PG-T6b.2 | Excititor Guild | Implement traversal queries (GetOutgoingAsync, GetIncomingAsync) |
| 27 | PG-T6b.4.1 | TODO | Depends on PG-T6b.3 | Excititor Guild | **CRITICAL:** Document revision_id computation algorithm |
| 28 | PG-T6b.4.2 | TODO | Depends on PG-T6b.4.1 | Excititor Guild | **CRITICAL:** Verify nodes inserted in deterministic order |
| 29 | PG-T6b.4.3 | TODO | Depends on PG-T6b.4.2 | Excititor Guild | **CRITICAL:** Verify edges inserted in deterministic order |
| 30 | PG-T6b.4.4 | TODO | Depends on PG-T6b.4.3 | Excititor Guild | **CRITICAL:** Write stability tests (5x computation must match) |
### Sprint 6c: Migration & Verification
| # | Task ID | Status | Key dependency / next step | Owners | Task Definition |
| --- | --- | --- | --- | --- | --- |
| 31 | PG-T6c.1.1 | TODO | Depends on PG-T6b.4 | Excititor Guild | Build graph conversion service for MongoDB documents |
| 32 | PG-T6c.1.2 | TODO | Depends on PG-T6c.1.1 | Excititor Guild | Extract and insert nodes in deterministic order |
| 33 | PG-T6c.1.3 | TODO | Depends on PG-T6c.1.2 | Excititor Guild | Extract and insert edges in deterministic order |
| 34 | PG-T6c.2.1 | TODO | Depends on PG-T6c.1 | Excititor Guild | Build VEX statement conversion service |
| 35 | PG-T6c.2.2 | TODO | Depends on PG-T6c.2.1 | Excititor Guild | Preserve provenance and evidence |
| 36 | PG-T6c.3.1 | TODO | Depends on PG-T6c.2 | Excititor Guild | Select sample projects for dual pipeline comparison |
| 37 | PG-T6c.3.2 | TODO | Depends on PG-T6c.3.1 | Excititor Guild | Compute graphs with MongoDB backend |
| 38 | PG-T6c.3.3 | TODO | Depends on PG-T6c.3.2 | Excititor Guild | Compute graphs with PostgreSQL backend |
| 39 | PG-T6c.3.4 | TODO | Depends on PG-T6c.3.3 | Excititor Guild | **CRITICAL:** Compare revision_ids (must match) |
| 40 | PG-T6c.3.5 | TODO | Depends on PG-T6c.3.4 | Excititor Guild | Compare node/edge counts and VEX statements |
| 41 | PG-T6c.4 | TODO | Depends on PG-T6c.3 | Excititor Guild | Migrate active projects |
| 42 | PG-T6c.5 | TODO | Depends on PG-T6c.4 | Excititor Guild | Switch Excititor to PostgreSQL-only |
## Execution Log
| Date (UTC) | Update | Owner |
| --- | --- | --- |
| 2025-11-28 | Sprint file created | Planning |
## Decisions & Risks
- Graph nodes/edges use BIGSERIAL for high-volume IDs.
- Bulk insert using PostgreSQL COPY for 10-100x performance.
- **CRITICAL RISK:** Revision ID instability would break reproducibility guarantees.
- Graph traversal indexes on `(from_node_id)` and `(to_node_id)`.
- Estimated volumes: 10M+ nodes, 20M+ edges, 1M+ VEX statements.
## Exit Criteria
- [ ] All repository interfaces implemented
- [ ] Graph storage working efficiently with bulk operations
- [ ] **Graph revision IDs stable (deterministic)** - CRITICAL
- [ ] VEX statements preserved correctly
- [ ] All comparison tests pass
- [ ] Excititor running on PostgreSQL in staging
## Next Checkpoints
- This is the most complex phase; allocate extra time for determinism verification.
- Phase 7 (Cleanup) follows after successful cutover.
---
*Reference: docs/db/tasks/PHASE_6_VEX_GRAPH.md*

View File

@@ -0,0 +1,153 @@
# Sprint 3407 · PostgreSQL Conversion: Phase 7 - Cleanup & Optimization
## Topic & Scope
- Phase 7 of MongoDB to PostgreSQL conversion: Final cleanup and optimization.
- Remove MongoDB dependencies from all converted modules.
- Archive MongoDB data and decommission infrastructure.
- Optimize PostgreSQL performance and update documentation.
- **Working directory:** Multiple (cleanup across all modules)
## Dependencies & Concurrency
- Upstream: ALL previous phases (3400-3406) must be DONE.
- Concurrency: Must run sequentially after all modules converted.
- Reference: `docs/db/tasks/PHASE_7_CLEANUP.md`
## Documentation Prerequisites
- docs/db/README.md
- docs/db/SPECIFICATION.md
- docs/db/RULES.md
- docs/db/VERIFICATION.md
- All module AGENTS.md files
## Delivery Tracker
### T7.1: Remove MongoDB Dependencies
| # | Task ID | Status | Key dependency / next step | Owners | Task Definition |
| --- | --- | --- | --- | --- | --- |
| 1 | PG-T7.1.1 | TODO | All phases complete | Infrastructure Guild | Remove `StellaOps.Authority.Storage.Mongo` project |
| 2 | PG-T7.1.2 | TODO | Depends on PG-T7.1.1 | Infrastructure Guild | Remove `StellaOps.Scheduler.Storage.Mongo` project |
| 3 | PG-T7.1.3 | TODO | Depends on PG-T7.1.1 | Infrastructure Guild | Remove `StellaOps.Notify.Storage.Mongo` project |
| 4 | PG-T7.1.4 | TODO | Depends on PG-T7.1.1 | Infrastructure Guild | Remove `StellaOps.Policy.Storage.Mongo` project |
| 5 | PG-T7.1.5 | TODO | Depends on PG-T7.1.1 | Infrastructure Guild | Remove `StellaOps.Concelier.Storage.Mongo` project |
| 6 | PG-T7.1.6 | TODO | Depends on PG-T7.1.1 | Infrastructure Guild | Remove `StellaOps.Excititor.Storage.Mongo` project |
| 7 | PG-T7.1.7 | TODO | Depends on PG-T7.1.6 | Infrastructure Guild | Update solution files |
| 8 | PG-T7.1.8 | TODO | Depends on PG-T7.1.7 | Infrastructure Guild | Remove dual-write wrappers |
| 9 | PG-T7.1.9 | TODO | Depends on PG-T7.1.8 | Infrastructure Guild | Remove MongoDB configuration options |
| 10 | PG-T7.1.10 | TODO | Depends on PG-T7.1.9 | Infrastructure Guild | Run full build to verify no broken references |
### T7.2: Archive MongoDB Data
| # | Task ID | Status | Key dependency / next step | Owners | Task Definition |
| --- | --- | --- | --- | --- | --- |
| 11 | PG-T7.2.1 | TODO | Depends on PG-T7.1.10 | DevOps Guild | Take final MongoDB backup |
| 12 | PG-T7.2.2 | TODO | Depends on PG-T7.2.1 | DevOps Guild | Export to BSON/JSON archives |
| 13 | PG-T7.2.3 | TODO | Depends on PG-T7.2.2 | DevOps Guild | Store archives in secure location |
| 14 | PG-T7.2.4 | TODO | Depends on PG-T7.2.3 | DevOps Guild | Document archive contents and structure |
| 15 | PG-T7.2.5 | TODO | Depends on PG-T7.2.4 | DevOps Guild | Set retention policy for archives |
| 16 | PG-T7.2.6 | TODO | Depends on PG-T7.2.5 | DevOps Guild | Schedule MongoDB cluster decommission |
### T7.3: PostgreSQL Performance Optimization
| # | Task ID | Status | Key dependency / next step | Owners | Task Definition |
| --- | --- | --- | --- | --- | --- |
| 17 | PG-T7.3.1 | TODO | Depends on PG-T7.2.6 | DBA Guild | Enable `pg_stat_statements` extension |
| 18 | PG-T7.3.2 | TODO | Depends on PG-T7.3.1 | DBA Guild | Identify slow queries |
| 19 | PG-T7.3.3 | TODO | Depends on PG-T7.3.2 | DBA Guild | Analyze query plans with EXPLAIN ANALYZE |
| 20 | PG-T7.3.4 | TODO | Depends on PG-T7.3.3 | DBA Guild | Add missing indexes |
| 21 | PG-T7.3.5 | TODO | Depends on PG-T7.3.4 | DBA Guild | Remove unused indexes |
| 22 | PG-T7.3.6 | TODO | Depends on PG-T7.3.5 | DBA Guild | Tune PostgreSQL configuration |
| 23 | PG-T7.3.7 | TODO | Depends on PG-T7.3.6 | Observability Guild | Set up query monitoring dashboard |
| 24 | PG-T7.3.8 | TODO | Depends on PG-T7.3.7 | DBA Guild | Document performance baselines |
### T7.4: Update Documentation
| # | Task ID | Status | Key dependency / next step | Owners | Task Definition |
| --- | --- | --- | --- | --- | --- |
| 25 | PG-T7.4.1 | TODO | Depends on PG-T7.3.8 | Docs Guild | Update `docs/07_HIGH_LEVEL_ARCHITECTURE.md` |
| 26 | PG-T7.4.2 | TODO | Depends on PG-T7.4.1 | Docs Guild | Update module architecture docs |
| 27 | PG-T7.4.3 | TODO | Depends on PG-T7.4.2 | Docs Guild | Update deployment guides |
| 28 | PG-T7.4.4 | TODO | Depends on PG-T7.4.3 | Docs Guild | Update operations runbooks |
| 29 | PG-T7.4.5 | TODO | Depends on PG-T7.4.4 | Docs Guild | Update troubleshooting guides |
| 30 | PG-T7.4.6 | TODO | Depends on PG-T7.4.5 | Docs Guild | Update `CLAUDE.md` technology stack |
| 31 | PG-T7.4.7 | TODO | Depends on PG-T7.4.6 | Docs Guild | Create `docs/operations/postgresql-guide.md` |
| 32 | PG-T7.4.8 | TODO | Depends on PG-T7.4.7 | Docs Guild | Document backup/restore procedures |
| 33 | PG-T7.4.9 | TODO | Depends on PG-T7.4.8 | Docs Guild | Document scaling recommendations |
### T7.5: Update Air-Gap Kit
| # | Task ID | Status | Key dependency / next step | Owners | Task Definition |
| --- | --- | --- | --- | --- | --- |
| 34 | PG-T7.5.1 | TODO | Depends on PG-T7.4.9 | DevOps Guild | Add PostgreSQL container image to kit |
| 35 | PG-T7.5.2 | TODO | Depends on PG-T7.5.1 | DevOps Guild | Update kit scripts for PostgreSQL setup |
| 36 | PG-T7.5.3 | TODO | Depends on PG-T7.5.2 | DevOps Guild | Include schema migrations in kit |
| 37 | PG-T7.5.4 | TODO | Depends on PG-T7.5.3 | DevOps Guild | Update kit documentation |
| 38 | PG-T7.5.5 | TODO | Depends on PG-T7.5.4 | DevOps Guild | Test kit installation in air-gapped environment |
| 39 | PG-T7.5.6 | TODO | Depends on PG-T7.5.5 | Docs Guild | Update `docs/24_OFFLINE_KIT.md` |
### T7.6: Final Verification
| # | Task ID | Status | Key dependency / next step | Owners | Task Definition |
| --- | --- | --- | --- | --- | --- |
| 40 | PG-T7.6.1 | TODO | Depends on PG-T7.5.6 | QA Guild | Run full integration test suite |
| 41 | PG-T7.6.2 | TODO | Depends on PG-T7.6.1 | QA Guild | Run performance benchmark suite |
| 42 | PG-T7.6.3 | TODO | Depends on PG-T7.6.2 | QA Guild | Verify all modules on PostgreSQL |
| 43 | PG-T7.6.4 | TODO | Depends on PG-T7.6.3 | QA Guild | **Verify determinism tests pass** |
| 44 | PG-T7.6.5 | TODO | Depends on PG-T7.6.4 | QA Guild | Verify air-gap kit works |
| 45 | PG-T7.6.6 | TODO | Depends on PG-T7.6.5 | QA Guild | Generate final verification report |
| 46 | PG-T7.6.7 | TODO | Depends on PG-T7.6.6 | Management | Get sign-off from stakeholders |
### T7.7: Decommission MongoDB
| # | Task ID | Status | Key dependency / next step | Owners | Task Definition |
| --- | --- | --- | --- | --- | --- |
| 47 | PG-T7.7.1 | TODO | Depends on PG-T7.6.7 | DevOps Guild | Verify no services using MongoDB |
| 48 | PG-T7.7.2 | TODO | Depends on PG-T7.7.1 | DevOps Guild | Stop MongoDB instances |
| 49 | PG-T7.7.3 | TODO | Depends on PG-T7.7.2 | DevOps Guild | Archive final state |
| 50 | PG-T7.7.4 | TODO | Depends on PG-T7.7.3 | DevOps Guild | Remove MongoDB from infrastructure |
| 51 | PG-T7.7.5 | TODO | Depends on PG-T7.7.4 | Observability Guild | Update monitoring/alerting |
| 52 | PG-T7.7.6 | TODO | Depends on PG-T7.7.5 | Finance | Update cost projections |
## Execution Log
| Date (UTC) | Update | Owner |
| --- | --- | --- |
| 2025-11-28 | Sprint file created | Planning |
## Decisions & Risks
- MongoDB archives are read-only backup; rollback to MongoDB after this phase is complex.
- Any new data created after cutover is PostgreSQL-only.
- Full rollback would require data export/import.
- PostgreSQL configuration tuning recommendations in PHASE_7_CLEANUP.md.
## Success Metrics
| Metric | Target | Measurement |
| --- | --- | --- |
| Query latency (p95) | < 100ms | pg_stat_statements |
| Error rate | < 0.01% | Application logs |
| Storage efficiency | < 120% of MongoDB | Disk usage |
| Test coverage | 100% | CI reports |
| Documentation coverage | 100% | Manual review |
## Exit Criteria
- [ ] All MongoDB code removed from converted modules
- [ ] MongoDB data archived
- [ ] PostgreSQL performance optimized
- [ ] All documentation updated
- [ ] Air-gap kit updated and tested
- [ ] Final verification report approved
- [ ] MongoDB infrastructure decommissioned
## Post-Conversion Monitoring
### First Week
- Monitor error rates closely
- Track query performance
- Watch for any data inconsistencies
- Have rollback plan ready (restore MongoDB)
### First Month
- Review query statistics weekly
- Optimize any slow queries found
- Monitor storage growth
- Adjust vacuum settings if needed
### Ongoing
- Regular performance reviews
- Index maintenance
- Backup verification
- Capacity planning
---
*Reference: docs/db/tasks/PHASE_7_CLEANUP.md*