stella-ops.org/git.stella-ops.org
1
0
Fork 0
Code Issues Pull Requests Actions 2 Releases Wiki Activity

feat: Document completed tasks for KMS, Cryptography, and Plugin Libraries

Browse Source 
- Added detailed task completion records for KMS interface implementation and CLI support for file-based keys.
- Documented security enhancements including Argon2id password hashing, audit event contracts, and rate limiting configurations.
- Included scoped service support and integration updates for the Plugin platform, ensuring proper DI handling and testing coverage.
This commit is contained in:
master
2025-10-31 14:37:45 +02:00
parent 240e8ff25d
commit 15b4a1de6a
312 changed files with 6399 additions and 3319 deletions
Download Patch File Download Diff File Expand all files Collapse all files

7
src/Scheduler/__Libraries/StellaOps.Scheduler.ImpactIndex/TASKS.completed.md Normal file
View File

@@ -0,0 +1,7 @@
# Completed Tasks
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| SCHED-IMPACT-16-300 | DONE (2025-10-20) | Scheduler ImpactIndex Guild | SAMPLES-10-001 | **STUB** ingest/query using fixtures to unblock Scheduler planning (remove by SP16 end). | Stub merges fixture BOM-Index, query API returns deterministic results, removal note tracked. |
| SCHED-IMPACT-16-301 | DONE (2025-10-26) | Scheduler ImpactIndex Guild | SCANNER-EMIT-10-605 | Implement ingestion of per-image BOM-Index sidecars into roaring bitmap store (contains/usedBy). | Ingestion tests process sample SBOM index; bitmaps persisted; deterministic IDs assigned. |
| SCHED-IMPACT-16-302 | DONE (2025-10-26) | Scheduler ImpactIndex Guild | SCHED-IMPACT-16-301 | Provide query APIs (ResolveByPurls, ResolveByVulns, ResolveAll, selectors) with tenant/namespace filters. | Query functions tested; performance benchmarks documented; selectors enforce filters. |

3
src/Scheduler/__Libraries/StellaOps.Scheduler.ImpactIndex/TASKS.md
View File

@@ -2,9 +2,6 @@
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| SCHED-IMPACT-16-300 | DONE (2025-10-20) | Scheduler ImpactIndex Guild | SAMPLES-10-001 | **STUB** ingest/query using fixtures to unblock Scheduler planning (remove by SP16 end). | Stub merges fixture BOM-Index, query API returns deterministic results, removal note tracked. |
| SCHED-IMPACT-16-301 | DONE (2025-10-26) | Scheduler ImpactIndex Guild | SCANNER-EMIT-10-605 | Implement ingestion of per-image BOM-Index sidecars into roaring bitmap store (contains/usedBy). | Ingestion tests process sample SBOM index; bitmaps persisted; deterministic IDs assigned. |
| SCHED-IMPACT-16-302 | DONE (2025-10-26) | Scheduler ImpactIndex Guild | SCHED-IMPACT-16-301 | Provide query APIs (ResolveByPurls, ResolveByVulns, ResolveAll, selectors) with tenant/namespace filters. | Query functions tested; performance benchmarks documented; selectors enforce filters. |
| SCHED-IMPACT-16-303 | TODO | Scheduler ImpactIndex Guild | SCHED-IMPACT-16-301 | Snapshot/compaction + invalidation for removed images; persistence to RocksDB/Redis per architecture. | Snapshot routine implemented; invalidation tests pass; docs describe recovery. |
> Removal tracking note: see `src/Scheduler/__Libraries/StellaOps.Scheduler.ImpactIndex/REMOVAL_NOTE.md` for follow-up actions once the roaring bitmap implementation lands.

11
src/Scheduler/__Libraries/StellaOps.Scheduler.Models/TASKS.completed.md Normal file
View File

@@ -0,0 +1,11 @@
# Completed Tasks
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| SCHED-MODELS-16-101 | DONE (2025-10-19) | Scheduler Models Guild | — | Define DTOs (Schedule, Run, ImpactSet, Selector, DeltaSummary, AuditRecord) with validation + canonical JSON. | DTOs merged with tests; documentation snippet added; serialization deterministic. |
| SCHED-MODELS-16-102 | DONE (2025-10-19) | Scheduler Models Guild | SCHED-MODELS-16-101 | Publish schema docs & sample payloads for UI/Notify integration. | Samples committed; docs referenced; contract tests pass. |
| SCHED-MODELS-16-103 | DONE (2025-10-20) | Scheduler Models Guild | SCHED-MODELS-16-101 | Versioning/migration helpers (schedule evolution, run state transitions). | Migration helpers implemented; tests cover upgrade/downgrade; guidelines documented. |
| SCHED-MODELS-20-001 | DONE (2025-10-26) | Scheduler Models Guild, Policy Guild | POLICY-ENGINE-20-000 | Define DTOs/schemas for policy runs, diffs, and explain traces (`PolicyRunRequest`, `PolicyRunStatus`, `PolicyDiffSummary`). | DTOs serialize deterministically; schema samples committed; validation helpers added. |
| SCHED-MODELS-20-002 | DONE (2025-10-29) | Scheduler Models Guild | SCHED-MODELS-20-001 | Extend scheduler schema docs to include policy run lifecycle, environment metadata, and diff payloads. | Docs updated with compliance checklist; samples validated against JSON schema; consumers notified. |
| SCHED-MODELS-21-001 | DONE (2025-10-26) | Scheduler Models Guild, Cartographer Guild | CARTO-GRAPH-21-007 | Define job DTOs for graph builds/overlay refresh (`GraphBuildJob`, `GraphOverlayJob`) with deterministic serialization and status enums. | DTOs serialized deterministically; schema snippets documented in `src/Scheduler/__Libraries/StellaOps.Scheduler.Models/docs/SCHED-MODELS-21-001-GRAPH-JOBS.md`; tests cover transitions. |
| SCHED-MODELS-21-002 | DONE (2025-10-26) | Scheduler Models Guild | SCHED-MODELS-21-001 | Publish schema docs/sample payloads for graph jobs and overlay events for downstream workers/UI. | Docs updated with compliance checklist; samples validated; notifications sent to guilds. |

7
src/Scheduler/__Libraries/StellaOps.Scheduler.Models/TASKS.md
View File

@@ -2,21 +2,14 @@
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| SCHED-MODELS-16-101 | DONE (2025-10-19) | Scheduler Models Guild | — | Define DTOs (Schedule, Run, ImpactSet, Selector, DeltaSummary, AuditRecord) with validation + canonical JSON. | DTOs merged with tests; documentation snippet added; serialization deterministic. |
| SCHED-MODELS-16-102 | DONE (2025-10-19) | Scheduler Models Guild | SCHED-MODELS-16-101 | Publish schema docs & sample payloads for UI/Notify integration. | Samples committed; docs referenced; contract tests pass. |
| SCHED-MODELS-16-103 | DONE (2025-10-20) | Scheduler Models Guild | SCHED-MODELS-16-101 | Versioning/migration helpers (schedule evolution, run state transitions). | Migration helpers implemented; tests cover upgrade/downgrade; guidelines documented. |
## Policy Engine v2 (Sprint 20)
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| SCHED-MODELS-20-001 | DONE (2025-10-26) | Scheduler Models Guild, Policy Guild | POLICY-ENGINE-20-000 | Define DTOs/schemas for policy runs, diffs, and explain traces (`PolicyRunRequest`, `PolicyRunStatus`, `PolicyDiffSummary`). | DTOs serialize deterministically; schema samples committed; validation helpers added. |
| SCHED-MODELS-20-002 | DONE (2025-10-29) | Scheduler Models Guild | SCHED-MODELS-20-001 | Extend scheduler schema docs to include policy run lifecycle, environment metadata, and diff payloads. | Docs updated with compliance checklist; samples validated against JSON schema; consumers notified. |
> 2025-10-29: Added lifecycle table, environment metadata section, and diff payload breakdown to `SCHED-MODELS-20-001-POLICY-RUNS.md`; compliance checklist extended to cover new documentation.
## Graph Explorer v1 (Sprint 21)
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| SCHED-MODELS-21-001 | DONE (2025-10-26) | Scheduler Models Guild, Cartographer Guild | CARTO-GRAPH-21-007 | Define job DTOs for graph builds/overlay refresh (`GraphBuildJob`, `GraphOverlayJob`) with deterministic serialization and status enums. | DTOs serialized deterministically; schema snippets documented in `src/Scheduler/__Libraries/StellaOps.Scheduler.Models/docs/SCHED-MODELS-21-001-GRAPH-JOBS.md`; tests cover transitions. |
| SCHED-MODELS-21-002 | DONE (2025-10-26) | Scheduler Models Guild | SCHED-MODELS-21-001 | Publish schema docs/sample payloads for graph jobs and overlay events for downstream workers/UI. | Docs updated with compliance checklist; samples validated; notifications sent to guilds. |

7
src/Scheduler/__Libraries/StellaOps.Scheduler.Queue/TASKS.completed.md Normal file
View File

@@ -0,0 +1,7 @@
# Completed Tasks
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| SCHED-QUEUE-16-401 | DONE (2025-10-20) | Scheduler Queue Guild | SCHED-MODELS-16-101 | Implement queue abstraction + Redis Streams adapter (planner inputs, runner segments) with ack/lease semantics. | Integration tests cover enqueue/dequeue/ack; lease renewal implemented; ordering preserved. |
| SCHED-QUEUE-16-402 | DONE (2025-10-20) | Scheduler Queue Guild | SCHED-QUEUE-16-401 | Add NATS JetStream adapter with configuration binding, health probes, failover. | Health endpoints verified; failover documented; adapter tested. |
| SCHED-QUEUE-16-403 | DONE (2025-10-20) | Scheduler Queue Guild | SCHED-QUEUE-16-401 | Dead-letter handling + metrics (queue depth, retry counts), configuration toggles. | Dead-letter policy tested; metrics exported; docs updated. |

15
src/Scheduler/__Libraries/StellaOps.Scheduler.Queue/TASKS.md
View File

@@ -1,9 +1,6 @@
# Scheduler Queue Task Board (Sprint 16)
> **Status note (2025-10-19):** Scheduler DTOs and sample payloads are now available (SCHED-MODELS-16-102). Queue tasks remain pending on this board.
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| SCHED-QUEUE-16-401 | DONE (2025-10-20) | Scheduler Queue Guild | SCHED-MODELS-16-101 | Implement queue abstraction + Redis Streams adapter (planner inputs, runner segments) with ack/lease semantics. | Integration tests cover enqueue/dequeue/ack; lease renewal implemented; ordering preserved. |
| SCHED-QUEUE-16-402 | DONE (2025-10-20) | Scheduler Queue Guild | SCHED-QUEUE-16-401 | Add NATS JetStream adapter with configuration binding, health probes, failover. | Health endpoints verified; failover documented; adapter tested. |
| SCHED-QUEUE-16-403 | DONE (2025-10-20) | Scheduler Queue Guild | SCHED-QUEUE-16-401 | Dead-letter handling + metrics (queue depth, retry counts), configuration toggles. | Dead-letter policy tested; metrics exported; docs updated. |
# Scheduler Queue Task Board (Sprint 16)
> **Status note (2025-10-19):** Scheduler DTOs and sample payloads are now available (SCHED-MODELS-16-102). Queue tasks remain pending on this board.
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|

7
src/Scheduler/__Libraries/StellaOps.Scheduler.Storage.Mongo/TASKS.completed.md Normal file
View File

@@ -0,0 +1,7 @@
# Completed Tasks
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| SCHED-STORAGE-16-201 | DONE (2025-10-19) | Scheduler Storage Guild | SCHED-MODELS-16-101 | Create Mongo collections (schedules, runs, impact_cursors, locks, audit) with indexes/migrations per architecture. | Migration scripts and indexes implemented; integration tests cover CRUD paths. |
| SCHED-STORAGE-16-202 | DONE (2025-10-26) | Scheduler Storage Guild | SCHED-STORAGE-16-201 | Implement repositories/services with tenant scoping, soft delete, TTL for completed runs, and causal consistency options. | Unit tests pass; TTL/soft delete validated; documentation updated. |
| SCHED-STORAGE-16-203 | DONE (2025-10-26) | Scheduler Storage Guild | SCHED-STORAGE-16-201 | Audit/logging pipeline + run stats materialized views for UI. | Audit entries persisted; stats queries efficient; docs capture usage. |

15
src/Scheduler/__Libraries/StellaOps.Scheduler.Storage.Mongo/TASKS.md
View File

@@ -1,9 +1,6 @@
# Scheduler Storage Task Board (Sprint 16)
> **Status note (2025-10-19):** Scheduler models/samples delivered in SCHED-MODELS-16-102. Tasks below remain pending for the Storage guild.
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| SCHED-STORAGE-16-201 | DONE (2025-10-19) | Scheduler Storage Guild | SCHED-MODELS-16-101 | Create Mongo collections (schedules, runs, impact_cursors, locks, audit) with indexes/migrations per architecture. | Migration scripts and indexes implemented; integration tests cover CRUD paths. |
| SCHED-STORAGE-16-202 | DONE (2025-10-26) | Scheduler Storage Guild | SCHED-STORAGE-16-201 | Implement repositories/services with tenant scoping, soft delete, TTL for completed runs, and causal consistency options. | Unit tests pass; TTL/soft delete validated; documentation updated. |
| SCHED-STORAGE-16-203 | DONE (2025-10-26) | Scheduler Storage Guild | SCHED-STORAGE-16-201 | Audit/logging pipeline + run stats materialized views for UI. | Audit entries persisted; stats queries efficient; docs capture usage. |
# Scheduler Storage Task Board (Sprint 16)
> **Status note (2025-10-19):** Scheduler models/samples delivered in SCHED-MODELS-16-102. Tasks below remain pending for the Storage guild.
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|

14
src/Scheduler/__Libraries/StellaOps.Scheduler.Worker/TASKS.completed.md Normal file
View File

@@ -0,0 +1,14 @@
# Completed Tasks
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| SCHED-WORKER-16-201 | DONE (2025-10-28) | Scheduler Worker Guild | SCHED-QUEUE-16-401 | Planner loop (cron + event triggers) with lease management, fairness, and rate limiting (§6). | Planner integration tests cover cron/event triggers; rate limits enforced; logs include run IDs. |
| SCHED-WORKER-16-202 | DONE (2025-10-27) | Scheduler Worker Guild | SCHED-IMPACT-16-301 | Wire ImpactIndex targeting (ResolveByPurls/vulns), dedupe, shard planning. | Targeting tests confirm correct image selection; dedupe documented; shards evenly distributed. |
| SCHED-WORKER-16-203 | DONE (2025-10-27) | Scheduler Worker Guild | SCHED-WORKER-16-202 | Runner execution: call Scanner `/reports` (analysis-only) or `/scans` when configured; collect deltas; handle retries. | Runner tests stub Scanner; retries/backoff validated; deltas aggregated deterministically. |
| SCHED-WORKER-16-204 | DONE (2025-10-27) | Scheduler Worker Guild | SCHED-WORKER-16-203 | Emit events (`scheduler.rescan.delta`, `scanner.report.ready`) for Notify/UI with summaries. | Events published to queue; payload schema documented; integration tests verify consumption. |
| SCHED-WORKER-16-205 | DONE (2025-10-27) | Scheduler Worker Guild | SCHED-WORKER-16-201 | Metrics/telemetry: run stats, queue depth, planner latency, delta counts. | Metrics exported per spec; dashboards updated; alerts configured. |
| SCHED-WORKER-20-301 | DONE (2025-10-28) | Scheduler Worker Guild, Policy Guild | SCHED-WORKER-16-201, POLICY-ENGINE-20-000 | Extend scheduler worker to trigger policy runs (full/incremental/simulate) via Policy Engine API, with idempotent job tracking and tenant scoping. | Worker schedules policy jobs deterministically; job records persisted; integration tests cover modes + cancellation. |
| SCHED-WORKER-21-201 | DONE (2025-10-29) | Scheduler Worker Guild, Cartographer Guild | SCHED-MODELS-21-001 | Implement graph build worker that dequeues SBOM graph jobs, invokes Cartographer build APIs, and records status with retries/backoff. | Worker processes fixtures; retries handled; logs include `graph_id`; integration tests pass. |
| SCHED-WORKER-20-302 | DONE (2025-10-29) | Scheduler Worker Guild | SCHED-WORKER-20-301, POLICY-ENGINE-20-006 | Implement policy delta targeting to re-evaluate only affected SBOM sets based on change streams and policy metadata. | Targeting reduces workload per design; tests simulate advisory/vex updates; metrics show delta counts. |
| SCHED-WORKER-20-303 | DONE (2025-10-29) | Scheduler Worker Guild, Observability Guild | SCHED-WORKER-20-301 | Expose metrics (`policy_runs_scheduled`, `policy_runs_failed`, planner latency) and structured logs with policy/run identifiers. | Metrics registered; dashboards updated; logs validated in integration tests. |
| SCHED-WORKER-21-202 | DONE (2025-10-29) | Scheduler Worker Guild | SCHED-WORKER-21-201, CARTO-GRAPH-21-007 | Overlay refresh worker subscribing to policy/SBOM change events, batching affected graph overlays, and enforcing <2 min SLA. | Overlay jobs scheduled deterministically; lag metrics < 2 min in tests; alerts configured. |

10
src/Scheduler/__Libraries/StellaOps.Scheduler.Worker/TASKS.md
View File

@@ -2,11 +2,6 @@
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| SCHED-WORKER-16-201 | DONE (2025-10-28) | Scheduler Worker Guild | SCHED-QUEUE-16-401 | Planner loop (cron + event triggers) with lease management, fairness, and rate limiting (§6). | Planner integration tests cover cron/event triggers; rate limits enforced; logs include run IDs. |
| SCHED-WORKER-16-202 | DONE (2025-10-27) | Scheduler Worker Guild | SCHED-IMPACT-16-301 | Wire ImpactIndex targeting (ResolveByPurls/vulns), dedupe, shard planning. | Targeting tests confirm correct image selection; dedupe documented; shards evenly distributed. |
| SCHED-WORKER-16-203 | DONE (2025-10-27) | Scheduler Worker Guild | SCHED-WORKER-16-202 | Runner execution: call Scanner `/reports` (analysis-only) or `/scans` when configured; collect deltas; handle retries. | Runner tests stub Scanner; retries/backoff validated; deltas aggregated deterministically. |
| SCHED-WORKER-16-204 | DONE (2025-10-27) | Scheduler Worker Guild | SCHED-WORKER-16-203 | Emit events (`scheduler.rescan.delta`, `scanner.report.ready`) for Notify/UI with summaries. | Events published to queue; payload schema documented; integration tests verify consumption. |
| SCHED-WORKER-16-205 | DONE (2025-10-27) | Scheduler Worker Guild | SCHED-WORKER-16-201 | Metrics/telemetry: run stats, queue depth, planner latency, delta counts. | Metrics exported per spec; dashboards updated; alerts configured. |
> 2025-10-27: Impact targeting sanitizes selector-constrained results, dedupes digests, and documents shard planning in `docs/SCHED-WORKER-16-202-IMPACT-TARGETING.md`.
@@ -17,20 +12,15 @@
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| SCHED-WORKER-20-301 | DONE (2025-10-28) | Scheduler Worker Guild, Policy Guild | SCHED-WORKER-16-201, POLICY-ENGINE-20-000 | Extend scheduler worker to trigger policy runs (full/incremental/simulate) via Policy Engine API, with idempotent job tracking and tenant scoping. | Worker schedules policy jobs deterministically; job records persisted; integration tests cover modes + cancellation. |
> 2025-10-26: Align worker serializers with `PolicyRunRequest/Status/DiffSummary` contracts from `src/Scheduler/__Libraries/StellaOps.Scheduler.Models`. Reference fixtures in `samples/api/scheduler/` for expected payload ordering.
| SCHED-WORKER-20-302 | DONE (2025-10-29) | Scheduler Worker Guild | SCHED-WORKER-20-301, POLICY-ENGINE-20-006 | Implement policy delta targeting to re-evaluate only affected SBOM sets based on change streams and policy metadata. | Targeting reduces workload per design; tests simulate advisory/vex updates; metrics show delta counts. |
> 2025-10-29: `PolicyRunTargetingService` translates change-stream metadata into SBOM sets, marks no-work jobs completed, and surfaces targeting options (`Policy.Targeting`). See `docs/SCHED-WORKER-20-302-POLICY-DELTA-TARGETING.md` for supported metadata keys and behaviour.
| SCHED-WORKER-20-303 | DONE (2025-10-29) | Scheduler Worker Guild, Observability Guild | SCHED-WORKER-20-301 | Expose metrics (`policy_runs_scheduled`, `policy_runs_failed`, planner latency) and structured logs with policy/run identifiers. | Metrics registered; dashboards updated; logs validated in integration tests. |
> 2025-10-29: Added `scheduler_policy_run_events_total` + latency histogram, instrumented policy dispatch success/retry/failure/cancel paths, and upgraded structured logs with tenant/policy/run identifiers. Docs updated in `docs/SCHED-WORKER-20-301-POLICY-RUNS.md` Observability section.
## Graph Explorer v1 (Sprint 21)
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| SCHED-WORKER-21-201 | DONE (2025-10-29) | Scheduler Worker Guild, Cartographer Guild | SCHED-MODELS-21-001 | Implement graph build worker that dequeues SBOM graph jobs, invokes Cartographer build APIs, and records status with retries/backoff. | Worker processes fixtures; retries handled; logs include `graph_id`; integration tests pass. |
> 2025-10-29: Graph build worker background service + execution pipeline added. Cartographer + Scheduler API options documented in `docs/SCHED-WORKER-21-201-GRAPH-BUILD.md`; unit tests cover success/retry/failure paths.
| SCHED-WORKER-21-202 | DONE (2025-10-29) | Scheduler Worker Guild | SCHED-WORKER-21-201, CARTO-GRAPH-21-007 | Overlay refresh worker subscribing to policy/SBOM change events, batching affected graph overlays, and enforcing <2 min SLA. | Overlay jobs scheduled deterministically; lag metrics < 2 min in tests; alerts configured. |
> 2025-10-29: Overlay worker now polls pending jobs, posts to Cartographer overlay endpoint, and reports completion via Scheduler webhook. Config + behaviour documented in `docs/SCHED-WORKER-21-202-GRAPH-OVERLAY.md`.
| SCHED-WORKER-21-203 | TODO | Scheduler Worker Guild, Observability Guild | SCHED-WORKER-21-201 | Export metrics (`graph_build_seconds`, `graph_jobs_inflight`, `overlay_lag_seconds`) and structured logs with tenant/graph identifiers. | Metrics/traces exposed; dashboards updated; integration tests verify metrics emission. |