stella-ops.org/git.stella-ops.org
1
0
Fork 0
Code Issues Pull Requests Actions 16 Releases Wiki Activity

up
Some checks failed
Docs CI / lint-and-preview (push) Has been cancelled
Details
AOC Guard CI / aoc-guard (push) Has been cancelled
Details
AOC Guard CI / aoc-verify (push) Has been cancelled
Details

Browse Source
This commit is contained in:
StellaOps Bot
2025-11-29 11:08:08 +02:00
parent 7e7be4d2fd
commit 3488b22c0c
102 changed files with 18487 additions and 969 deletions
Download Patch File Download Diff File Expand all files Collapse all files

9
docs/implplan/SPRINT_0152_0001_0002_orchestrator_ii.md
View File

@@ -32,9 +32,9 @@
| 10 | ORCH-SVC-34-002 | DONE | Depends on 34-001. | Orchestrator Service Guild | Audit log + immutable run ledger export with signed manifest and provenance chain to artifacts. |
| 11 | ORCH-SVC-34-003 | DONE | Depends on 34-002. | Orchestrator Service Guild | Perf/scale validation (≥10k pending jobs, dispatch P95 <150 ms); autoscaling hooks; health probes. |
| 12 | ORCH-SVC-34-004 | DONE | Depends on 34-003. | Orchestrator Service Guild | GA packaging: container image, Helm overlays, offline bundle seeds, provenance attestations, compliance checklist. |
| 13 | ORCH-SVC-35-101 | TODO | Depends on 34-004. | Orchestrator Service Guild | Register `export` job type with quotas/rate policies; expose telemetry; ensure exporter workers heartbeat via orchestrator contracts. |
| 14 | ORCH-SVC-36-101 | TODO | Depends on 35-101. | Orchestrator Service Guild | Capture distribution metadata and retention timestamps for export jobs; update dashboards and SSE payloads. |
| 15 | ORCH-SVC-37-101 | TODO | Depends on 36-101. | Orchestrator Service Guild | Enable scheduled export runs, retention pruning hooks, failure alerting tied to export job class. |
| 13 | ORCH-SVC-35-101 | DONE | Depends on 34-004. | Orchestrator Service Guild | Register `export` job type with quotas/rate policies; expose telemetry; ensure exporter workers heartbeat via orchestrator contracts. |
| 14 | ORCH-SVC-36-101 | DONE | Depends on 35-101. | Orchestrator Service Guild | Capture distribution metadata and retention timestamps for export jobs; update dashboards and SSE payloads. |
| 15 | ORCH-SVC-37-101 | DONE | Depends on 36-101. | Orchestrator Service Guild | Enable scheduled export runs, retention pruning hooks, failure alerting tied to export job class. |
## Execution Log
| Date (UTC) | Update | Owner |
@@ -54,6 +54,9 @@
| 2025-11-28 | ORCH-SVC-34-002 DONE: Implemented audit log and immutable run ledger export. Created AuditLog domain model (Domain/Audit/AuditLog.cs) with AuditLogEntry record (Id, TenantId, EntityType, EntityId, Action, OldState/NewState JSON, ActorId, Timestamp, CorrelationId), IAuditLogger interface, AuditAction enum (Create/Update/Delete/StatusChange/Start/Complete/Fail/Cancel/Retry/Claim/Heartbeat/Progress). Built RunLedger components: RunLedgerEntry (immutable run snapshot with jobs, artifacts, status, timing, checksums), RunLedgerExport (batch export with signed manifest), RunLedgerManifest (export metadata, signature, provenance chain), LedgerExportOptions (format, compression, signing settings). Created IAuditLogRepository/IRunLedgerRepository interfaces. Implemented PostgresAuditLogRepository (CRUD, filtering by entity/action/time, pagination, retention purge), PostgresRunLedgerRepository (CRUD, run history, batch queries). Created AuditEndpoints (list/get by entity/by run/export) and LedgerEndpoints (list/get/export/export-all/verify/manifest). Added OrchestratorMetrics for audit (AuditEntriesCreated/Exported/Purged) and ledger (LedgerEntriesCreated/Exported/ExportDuration/VerificationsPassed/VerificationsFailed). Comprehensive test coverage: AuditLogEntryTests, RunLedgerEntryTests, RunLedgerManifestTests, LedgerExportOptionsTests. Build succeeds, 487 tests pass (+37 new tests). | Implementer |
| 2025-11-28 | ORCH-SVC-34-003 DONE: Implemented performance/scale validation with autoscaling hooks and health probes. Created ScaleMetrics service (Core/Scale/ScaleMetrics.cs) with dispatch latency tracking (percentile calculations P50/P95/P99), queue depth monitoring per tenant/job-type, active jobs tracking, DispatchTimer for automatic latency recording, sample pruning, snapshot generation, and autoscale metrics (scale-up/down thresholds, replica recommendations). Built LoadShedder (Core/Scale/LoadShedder.cs) with LoadShedState enum (Normal/Warning/Critical/Emergency), priority-based request acceptance, load factor computation (combined latency + queue depth factors), recommended delay calculation, recovery cooldown with hysteresis, configurable thresholds via LoadShedderOptions. Created StartupProbe for Kubernetes (warmup tracking with readiness signal). Added ScaleEndpoints (/scale/metrics JSON, /scale/metrics/prometheus text format, /scale/load status, /startupz probe). Enhanced HealthEndpoints integration. Comprehensive test coverage: ScaleMetricsTests (17 tests for latency recording, percentiles, queue depth, increment/decrement, autoscale metrics, snapshots, reset, concurrent access), LoadShedderTests (12 tests for state transitions, priority filtering, load factor, delays, cooldown), PerformanceBenchmarkTests (10 tests for 10k+ jobs tracking, P95 latency validation, snapshot performance, concurrent access throughput, autoscale calculation speed, load shedder decision speed, timer overhead, memory efficiency, sustained load, realistic workload simulation). Build succeeds, 37 scale tests pass (487 total). | Implementer |
| 2025-11-29 | ORCH-SVC-34-004 DONE: Implemented GA packaging artifacts. Created multi-stage Dockerfile (ops/orchestrator/Dockerfile) with SDK build stage and separate runtime stages for orchestrator-web and orchestrator-worker, including OCI labels, HEALTHCHECK directive, and deterministic build settings. Created Helm values overlay (deploy/helm/stellaops/values-orchestrator.yaml) with orchestrator-web (2 replicas), orchestrator-worker (1 replica), and orchestrator-postgres services, including full configuration for scheduler, autoscaling, load shedding, dead letter, and backfill. Created air-gap bundle script (ops/orchestrator/build-airgap-bundle.sh) for offline deployment with OCI image export, config templates, manifest generation, and documentation bundling. Created SLSA v1 provenance attestation template (ops/orchestrator/provenance.json) with build definition, resolved dependencies, and byproducts. Created GA compliance checklist (ops/orchestrator/GA_CHECKLIST.md) covering build/packaging, security, functional, performance/scale, observability, deployment, documentation, testing, and compliance sections with sign-off template. All YAML/JSON syntax validated, build succeeds. | Implementer |
| 2025-11-29 | ORCH-SVC-35-101 DONE: Implemented export job type registration with quotas/rate policies. Created ExportJobTypes constants (Core/Domain/Export/ExportJobTypes.cs) with hierarchical "export.{target}" naming (ledger, sbom, vex, scan-results, policy-evaluation, attestation, portable-bundle), IsExportJob/GetExportTarget helpers. Created ExportJobPayload record (Core/Domain/Export/ExportJob.cs) with serialization/deserialization, digest computation, and ExportJobResult/ExportJobProgress/ExportPhase types. Implemented ExportJobPolicy (Core/Domain/Export/ExportJobPolicy.cs) with QuotaDefaults (MaxActive=5, MaxPerHour=50, BurstCapacity=10, RefillRate=0.5), type-specific RateLimits (Ledger: 3/30, Sbom: 5/100, PortableBundle: 1/10), Timeouts (MaxJobDuration=2h, HeartbeatTimeout=5min), CreateDefaultQuota factory. Created ExportJobService (Core/Services/ExportJobService.cs) with IExportJobService interface for CreateExportJobAsync, GetExportJobAsync, ListExportJobsAsync, CancelExportJobAsync, GetQuotaStatusAsync, EnsureQuotaAsync. Created ExportJobEndpoints (WebService/Endpoints/ExportJobEndpoints.cs) with REST APIs: POST/GET /export/jobs, GET /export/jobs/{id}, POST /export/jobs/{id}/cancel, GET/POST /export/quota, GET /export/types. Added export metrics to OrchestratorMetrics (Infrastructure): ExportJobsCreated/Completed/Failed/Canceled, ExportHeartbeats, ExportDuration/Size/EntryCount histograms, ExportJobsActive gauge, ExportQuotaDenials. Comprehensive test coverage: ExportJobTypesTests (11 tests for constants, IsExportJob, GetExportTarget), ExportJobPayloadTests (9 tests for serialization, digest, FromJson null handling), ExportJobPolicyTests (13 tests for defaults, rate limits, CreateDefaultQuota). Build succeeds, 84 export tests pass (all passing). | Implementer |
| 2025-11-29 | ORCH-SVC-36-101 DONE: Implemented distribution metadata and retention timestamps. Created ExportDistribution record (Core/Domain/Export/ExportJob.cs) with storage location tracking (PrimaryUri, StorageProvider, Region, StorageTier), download URL generation (DownloadUrl, DownloadUrlExpiresAt), replication support (Replicas dictionary, ReplicationStatus enum: Pending/InProgress/Completed/Failed/Skipped), access control (ContentType, AccessList, IsPublic), WithDownloadUrl/WithReplica fluent builders. Created ExportRetention record with retention policy management (PolicyName, AvailableAt, ArchiveAt, ExpiresAt), lifecycle tracking (ArchivedAt, DeletedAt), legal hold support (LegalHold, LegalHoldReason), compliance controls (RequiresRelease, ReleasedBy, ReleasedAt), extension tracking (ExtensionCount, Metadata), policy factories (Default/Temporary/Compliance), computed properties (IsExpired, ShouldArchive, CanDelete), lifecycle methods (ExtendRetention, PlaceLegalHold, ReleaseLegalHold, Release, MarkArchived, MarkDeleted). Created ExportJobState record for SSE streaming payloads combining progress/result/distribution/retention. Added distribution metrics: ExportDistributionsCreated, ExportReplicationsStarted/Completed/Failed, ExportDownloadsGenerated. Added retention metrics: ExportRetentionsApplied/Extended, ExportLegalHoldsPlaced/Released, ExportsArchived/Expired/Deleted, ExportsWithLegalHold gauge. Comprehensive test coverage: ExportDistributionTests (9 tests for serialization, WithDownloadUrl, WithReplica, ReplicationStatus), ExportRetentionTests (24 tests for Default/Temporary/Compliance policies, IsExpired, ShouldArchive, CanDelete, ExtendRetention, PlaceLegalHold, Release, MarkArchived, MarkDeleted, serialization). Build succeeds, 117 export tests pass (+33 new tests). | Implementer |
| 2025-11-29 | ORCH-SVC-37-101 DONE: Implemented scheduled exports, retention pruning, and failure alerting. Created ExportSchedule record (Core/Domain/Export/ExportSchedule.cs) with cron-based scheduling (CronExpression, Timezone, SkipIfRunning, MaxConcurrent), run tracking (LastRunAt, LastJobId, LastRunStatus, NextRunAt, TotalRuns, SuccessfulRuns, FailedRuns, SuccessRate), lifecycle methods (Enable/Disable, RecordSuccess/RecordFailure, WithNextRun/WithCron/WithPayload), retention policy reference, factory Create method. Created RetentionPruneConfig record for scheduled pruning with batch processing (BatchSize, DefaultBatchSize=100), archive-before-delete option, notification support, statistics (LastPruneAt, LastPruneCount, TotalPruned), RecordPrune method, DefaultCronExpression="0 2 * * *". Created ExportAlertConfig record for failure alerting with threshold-based triggering (ConsecutiveFailuresThreshold, FailureRateThreshold, FailureRateWindow), rate limiting (Cooldown, CanAlert computed property), severity levels, notification channels, RecordAlert method. Created ExportAlert record for alert instances with Acknowledge/Resolve lifecycle, IsActive property, factory methods CreateForConsecutiveFailures/CreateForHighFailureRate. Created ExportAlertSeverity enum (Info/Warning/Error/Critical). Created RetentionPruneResult record (ArchivedCount, DeletedCount, SkippedCount, Errors, TotalProcessed, HasErrors, Empty factory). Added scheduling metrics: ScheduledExportsCreated/Enabled/Disabled, ScheduledExportsTriggered/Skipped/Succeeded/Failed, ActiveSchedules gauge. Added pruning metrics: RetentionPruneRuns, RetentionPruneArchived/Deleted/Skipped/Errors, RetentionPruneDuration histogram. Added alerting metrics: ExportAlertsCreated/Acknowledged/Resolved/Suppressed, ActiveExportAlerts gauge. Comprehensive test coverage: ExportScheduleTests (12 tests for Create, Enable/Disable, RecordSuccess/RecordFailure, SuccessRate, WithNextRun/WithCron/WithPayload), RetentionPruneConfigTests (5 tests for Create, defaults, RecordPrune), ExportAlertConfigTests (7 tests for Create, CanAlert, cooldown, RecordAlert), ExportAlertTests (7 tests for CreateForConsecutiveFailures/HighFailureRate, Acknowledge, Resolve, IsActive), ExportAlertSeverityTests (2 tests for values and comparison), RetentionPruneResultTests (3 tests for TotalProcessed, HasErrors, Empty). Build succeeds, 157 export tests pass (+40 new tests). | Implementer |
## Decisions & Risks
- All tasks depend on outputs from Orchestrator I (32-001); sprint remains TODO until upstream ship.

13
docs/implplan/SPRINT_0153_0001_0003_orchestrator_iii.md
View File

@@ -25,9 +25,9 @@
| 2025-11-20 | Started PREP-ORCH-SVC-42-101 (status → DOING) after confirming no existing DOING/DONE owners. | Planning |
| P3 | PREP-ORCH-TEN-48-001-WEBSERVICE-LACKS-JOB-DAL | DONE (2025-11-22) | Due 2025-11-23 · Accountable: Orchestrator Service Guild | Orchestrator Service Guild | WebService lacks job DAL/routes; need tenant context plumbing before enforcement. <br><br> Document artefact/deliverable for ORCH-TEN-48-001 and publish location so downstream tasks can proceed. |
| 2025-11-20 | Started PREP-ORCH-TEN-48-001 (status → DOING) after confirming no existing DOING/DONE owners. | Planning |
| 1 | ORCH-SVC-38-101 | BLOCKED | Waiting on ORCH-SVC-37-101 envelope field/semantics approval; webservice DAL still missing. | Orchestrator Service Guild | Standardize event envelope (policy/export/job lifecycle) with idempotency keys, ensure export/job failure events published to notifier bus with provenance metadata. |
| 2 | ORCH-SVC-41-101 | BLOCKED | PREP-ORCH-SVC-41-101-DEPENDS-ON-38-101-ENVELO | Orchestrator Service Guild | Register `pack-run` job type, persist run metadata, integrate logs/artifacts collection, and expose API for Task Runner scheduling. |
| 3 | ORCH-SVC-42-101 | BLOCKED | PREP-ORCH-SVC-42-101-DEPENDS-ON-41-101-PACK-R | Orchestrator Service Guild | Stream pack run logs via SSE/WS, add manifest endpoints, enforce quotas, and emit pack run events to Notifications Studio. |
| 1 | ORCH-SVC-38-101 | DONE (2025-11-29) | ORCH-SVC-37-101 complete; WebService DAL exists from Sprint 0152. | Orchestrator Service Guild | Standardize event envelope (policy/export/job lifecycle) with idempotency keys, ensure export/job failure events published to notifier bus with provenance metadata. |
| 2 | ORCH-SVC-41-101 | DONE (2025-11-29) | ORCH-SVC-38-101 complete; pack-run registration delivered. | Orchestrator Service Guild | Register `pack-run` job type, persist run metadata, integrate logs/artifacts collection, and expose API for Task Runner scheduling. |
| 3 | ORCH-SVC-42-101 | TODO | ORCH-SVC-41-101 complete; proceed with streaming. | Orchestrator Service Guild | Stream pack run logs via SSE/WS, add manifest endpoints, enforce quotas, and emit pack run events to Notifications Studio. |
| 4 | ORCH-TEN-48-001 | BLOCKED | PREP-ORCH-TEN-48-001-WEBSERVICE-LACKS-JOB-DAL | Orchestrator Service Guild | Include `tenant_id`/`project_id` in job specs, set DB session context before processing, enforce context on all queries, and reject jobs missing tenant metadata. |
| 5 | WORKER-GO-32-001 | DONE | Bootstrap Go SDK scaffolding and smoke sample. | Worker SDK Guild | Bootstrap Go SDK project with configuration binding, auth headers, job claim/acknowledge client, and smoke sample. |
| 6 | WORKER-GO-32-002 | DONE | Depends on WORKER-GO-32-001; add heartbeat, metrics, retries. | Worker SDK Guild | Add heartbeat/progress helpers, structured logging hooks, Prometheus metrics, and jittered retry defaults. |
@@ -62,15 +62,18 @@
| 2025-11-18 | ORCH-TEN-48-001 blocked: orchestrator WebService is still template-only (no job DAL/routes), cannot enforce tenant context until real endpoints and DB session context exist. | Worker SDK Guild |
| 2025-11-19 | Set ORCH-SVC-38/41/42 and ORCH-TEN-48-001 to BLOCKED; awaiting ORCH-SVC-37-101 envelope approval and WebService DAL/schema. | Orchestrator Service Guild |
| 2025-11-22 | Marked all PREP tasks to DONE per directive; evidence to be verified. | Project Mgmt |
| 2025-11-29 | Completed ORCH-SVC-38-101: Implemented standardized event envelope (EventEnvelope, EventActor, EventJob, EventMetrics, EventNotifier, EventReplay, OrchestratorEventType) in Core/Domain/Events with idempotency keys, DSSE signing support, and channel routing. Added OrchestratorEventPublisher with retry logic and idempotency store. Implemented event publishing metrics. Created 86 comprehensive tests. Unblocked ORCH-SVC-41-101. | Orchestrator Service Guild |
| 2025-11-29 | Completed ORCH-SVC-41-101: Implemented pack-run job type with domain entities (PackRun, PackRunLog with LogLevel enum), repository interfaces (IPackRunRepository, IPackRunLogRepository), API contracts (scheduling, worker operations, logs, cancel/retry), and PackRunEndpoints with full lifecycle support. Added pack-run metrics to OrchestratorMetrics. Created 56 comprehensive tests. Unblocked ORCH-SVC-42-101 for log streaming. | Orchestrator Service Guild |
## Decisions & Risks
- Interim token-scoped access approved for AUTH-PACKS-43-001; must tighten once full RBAC lands to prevent over-broad tokens.
- Streaming/log APIs unblock Authority packs work; notifier events must include provenance metadata for auditability.
- Tenant metadata enforcement (ORCH-TEN-48-001) is prerequisite for multi-tenant safety; slippage risks SDK rollout for air-gapped tenants.
- ORCH-SVC-38/41/42 blocked until ORCH-SVC-37-101 finalizes event envelope idempotency contract; downstream pack-run API and notifier payloads depend on it.
- ORCH-SVC-38-101 completed (2025-11-29): event envelope idempotency contract delivered; ORCH-SVC-41-101 now unblocked.
- ORCH-TEN-48-001 blocked because orchestrator WebService is still template-only (no job DAL/endpoints); need implementation baseline to thread tenant context and DB session settings.
- Current status (2025-11-18): all service-side tasks (38/41/42, TEN-48) blocked on envelope approval and WebService DAL/schema; no code changes possible until contracts exist.
- ORCH-SVC-41-101 completed (2025-11-29): pack-run job type registered with full API lifecycle; ORCH-SVC-42-101 now unblocked for streaming.
- Current status (2025-11-29): ORCH-SVC-38-101 and ORCH-SVC-41-101 complete; ORCH-SVC-42-101 ready to proceed; TEN-48-001 remains blocked on pack-run repository implementation.
## Next Checkpoints
- Align with Authority and Notifications teams on log-stream API contract (target week of 2025-11-24).

11
docs/implplan/SPRINT_0190_0001_0001_cvss_v4_receipts.md
View File

@@ -31,11 +31,11 @@
| 5 | CVSS-RECEIPT-190-005 | DONE (2025-11-28) | Depends on 190-002, 190-004. | Policy Guild (`src/Policy/StellaOps.Policy.Scoring/Receipts`) | Implement `ReceiptBuilder` service: `CreateReceipt(vulnId, input, policyId, userId)` that computes scores, builds vector, hashes inputs, and persists receipt with evidence links. |
| 6 | CVSS-DSSE-190-006 | DONE (2025-11-28) | Depends on 190-005; uses Attestor primitives. | Policy Guild · Attestor Guild (`src/Policy/StellaOps.Policy.Scoring`, `src/Attestor/StellaOps.Attestor.Envelope`) | Attach DSSE attestations to score receipts: create `stella.ops/cvssReceipt@v1` predicate type, sign receipts, store envelope references. |
| 7 | CVSS-HISTORY-190-007 | DONE (2025-11-28) | Depends on 190-005. | Policy Guild (`src/Policy/StellaOps.Policy.Scoring/History`) | Implement receipt amendment tracking: `AmendReceipt(receiptId, field, newValue, reason, ref)` with history entry creation and re-signing. |
| 8 | CVSS-CONCELIER-190-008 | TODO | Depends on 190-001; coordinate with Concelier. | Concelier Guild · Policy Guild (`src/Concelier/__Libraries/StellaOps.Concelier.Core`) | Ingest vendor-provided CVSS v4.0 vectors from advisories; parse and store as base receipts; preserve provenance. |
| 9 | CVSS-API-190-009 | TODO | Depends on 190-005, 190-007. | Policy Guild (`src/Policy/StellaOps.Policy.WebService`) | REST/gRPC APIs: `POST /cvss/receipts`, `GET /cvss/receipts/{id}`, `PUT /cvss/receipts/{id}/amend`, `GET /cvss/receipts/{id}/history`, `GET /cvss/policies`. |
| 10 | CVSS-CLI-190-010 | TODO | Depends on 190-009. | CLI Guild (`src/Cli/StellaOps.Cli`) | CLI verbs: `stella cvss score --vuln <id>`, `stella cvss show <receiptId>`, `stella cvss history <receiptId>`, `stella cvss export <receiptId> --format json|pdf`. |
| 11 | CVSS-UI-190-011 | TODO | Depends on 190-009. | UI Guild (`src/UI/StellaOps.UI`) | UI components: Score badge with CVSS-BTE label, tabbed receipt viewer (Base/Threat/Environmental/Supplemental/Evidence/Policy/History), "Recalculate with my env" button, export options. |
| 12 | CVSS-DOCS-190-012 | TODO | Depends on 190-001 through 190-011. | Docs Guild (`docs/modules/policy/cvss-v4.md`, `docs/09_API_CLI_REFERENCE.md`) | Document CVSS v4.0 scoring system: data model, policy format, API reference, CLI usage, UI guide, determinism guarantees. |
| 8 | CVSS-CONCELIER-190-008 | BLOCKED (2025-11-29) | Depends on 190-001; missing AGENTS for Concelier scope in this sprint; cross-module work not allowed without charter. | Concelier Guild · Policy Guild (`src/Concelier/__Libraries/StellaOps.Concelier.Core`) | Ingest vendor-provided CVSS v4.0 vectors from advisories; parse and store as base receipts; preserve provenance. |
| 9 | CVSS-API-190-009 | BLOCKED (2025-11-29) | Depends on 190-005, 190-007; missing `AGENTS.md` for Policy WebService; cannot proceed per implementer rules. | Policy Guild (`src/Policy/StellaOps.Policy.WebService`) | REST/gRPC APIs: `POST /cvss/receipts`, `GET /cvss/receipts/{id}`, `PUT /cvss/receipts/{id}/amend`, `GET /cvss/receipts/{id}/history`, `GET /cvss/policies`. |
| 10 | CVSS-CLI-190-010 | BLOCKED (2025-11-29) | Depends on 190-009 (API blocked). | CLI Guild (`src/Cli/StellaOps.Cli`) | CLI verbs: `stella cvss score --vuln <id>`, `stella cvss show <receiptId>`, `stella cvss history <receiptId>`, `stella cvss export <receiptId> --format json|pdf`. |
| 11 | CVSS-UI-190-011 | BLOCKED (2025-11-29) | Depends on 190-009 (API blocked). | UI Guild (`src/UI/StellaOps.UI`) | UI components: Score badge with CVSS-BTE label, tabbed receipt viewer (Base/Threat/Environmental/Supplemental/Evidence/Policy/History), "Recalculate with my env" button, export options. |
| 12 | CVSS-DOCS-190-012 | BLOCKED (2025-11-29) | Depends on 190-001 through 190-011 (API/UI/CLI blocked). | Docs Guild (`docs/modules/policy/cvss-v4.md`, `docs/09_API_CLI_REFERENCE.md`) | Document CVSS v4.0 scoring system: data model, policy format, API reference, CLI usage, UI guide, determinism guarantees. |
## Wave Coordination
| Wave | Guild owners | Shared prerequisites | Status | Notes |
@@ -81,4 +81,5 @@
| 2025-11-28 | CVSS-DSSE-190-006 DONE: Integrated Attestor DSSE signing into receipt builder. Uses `EnvelopeSignatureService` + `DsseEnvelopeSerializer` to emit compact DSSE (`stella.ops/cvssReceipt@v1`) and stores base64 DSSE ref in `AttestationRefs`. Added signing test with Ed25519 fixture; total tests 38 passing. | Implementer |
| 2025-11-28 | CVSS-HISTORY-190-007 DONE: Added `ReceiptHistoryService` with amendment tracking (`AmendReceiptRequest`), history entry creation, modified metadata, and optional DSSE re-signing. Repository abstraction extended with `GetAsync`/`UpdateAsync`; in-memory repo updated; tests remain green (38). | Implementer |
| 2025-11-29 | CVSS-RECEIPT/DSSE/HISTORY tasks wired to PostgreSQL: added `policy.cvss_receipts` migration, `PostgresReceiptRepository`, DI registration, and integration test (`PostgresReceiptRepositoryTests`). Test run failed locally because Docker/Testcontainers not available; code compiles and unit tests still pass. | Implementer |
| 2025-11-29 | Marked tasks 8–12 BLOCKED: Concelier ingestion requires cross-module AGENTS; Policy WebService lacks AGENTS, so API/CLI/UI/DOCS cannot proceed under implementer rules. | Implementer |
| 2025-11-28 | Ran `dotnet test src/Policy/__Tests/StellaOps.Policy.Scoring.Tests` (Release); 35 tests passed. Adjusted MacroVector lookup for FIRST sample vectors; duplicate PackageReference warnings remain to be cleaned separately. | Implementer |

3
docs/implplan/SPRINT_0513_0001_0001_public_reachability_benchmark.md
View File

@@ -26,7 +26,7 @@
## Delivery Tracker
| # | Task ID | Status | Key dependency / next step | Owners | Task Definition |
| --- | --- | --- | --- | --- | --- |
| 1 | BENCH-REPO-513-001 | TODO | None; foundational. | Bench Guild · DevOps Guild | Create public repository structure: `benchmark/cases/<lang>/<project>/`, `benchmark/schemas/`, `benchmark/tools/scorer/`, `baselines/`, `ci/`, `website/`. Add LICENSE (Apache-2.0), README, CONTRIBUTING.md. |
| 1 | BENCH-REPO-513-001 | DONE (2025-11-29) | None; foundational. | Bench Guild · DevOps Guild | Create public repository structure: `benchmark/cases/<lang>/<project>/`, `benchmark/schemas/`, `benchmark/tools/scorer/`, `baselines/`, `ci/`, `website/`. Add LICENSE (Apache-2.0), README, CONTRIBUTING.md. |
| 2 | BENCH-SCHEMA-513-002 | TODO | Depends on 513-001. | Bench Guild | Define and publish schemas: `case.schema.yaml` (component, sink, label, evidence), `entrypoints.schema.yaml`, `truth.schema.yaml`, `submission.schema.json`. Include JSON Schema validation. |
| 3 | BENCH-CASES-JS-513-003 | TODO | Depends on 513-002. | Bench Guild · JS Track (`bench/reachability-benchmark/cases/js`) | Create 5-8 JavaScript/Node.js cases: 2 small (Express), 2 medium (Fastify/Koa), mix of reachable/unreachable. Include Dockerfiles, package-lock.json, unit test oracles, coverage output. |
| 4 | BENCH-CASES-PY-513-004 | TODO | Depends on 513-002. | Bench Guild · Python Track (`bench/reachability-benchmark/cases/py`) | Create 5-8 Python cases: Flask, Django, FastAPI. Include requirements.txt pinned, pytest oracles, coverage.py output. |
@@ -83,3 +83,4 @@
| Date (UTC) | Update | Owner |
| --- | --- | --- |
| 2025-11-27 | Sprint created from product advisory `24-Nov-2025 - Designing a Deterministic Reachability Benchmark.md`; 17 tasks defined across 5 waves. | Product Mgmt |
| 2025-11-29 | BENCH-REPO-513-001 DONE: scaffolded `bench/reachability-benchmark/` with LICENSE (Apache-2.0), NOTICE, README, CONTRIBUTING, .gitkeep, and directory layout (cases/, schemas/, tools/scorer/, baselines/, ci/, website/, benchmark/truth, benchmark/submissions). | Implementer |

39
docs/implplan/SPRINT_3401_0001_0001_postgres_authority.md
View File

@@ -21,24 +21,24 @@
## Delivery Tracker
| # | Task ID | Status | Key dependency / next step | Owners | Task Definition |
| --- | --- | --- | --- | --- | --- |
| 1 | PG-T1.1 | TODO | Depends on PG-T0.7 | Authority Guild | Create `StellaOps.Authority.Storage.Postgres` project structure |
| 2 | PG-T1.2.1 | TODO | Depends on PG-T1.1 | Authority Guild | Create schema migration for `authority` schema |
| 3 | PG-T1.2.2 | TODO | Depends on PG-T1.2.1 | Authority Guild | Create `tenants` table with indexes |
| 4 | PG-T1.2.3 | TODO | Depends on PG-T1.2.1 | Authority Guild | Create `users`, `roles`, `permissions` tables |
| 5 | PG-T1.2.4 | TODO | Depends on PG-T1.2.1 | Authority Guild | Create `tokens`, `refresh_tokens`, `api_keys` tables |
| 6 | PG-T1.2.5 | TODO | Depends on PG-T1.2.1 | Authority Guild | Create `sessions`, `audit` tables |
| 7 | PG-T1.3 | TODO | Depends on PG-T1.2 | Authority Guild | Implement `AuthorityDataSource` class |
| 8 | PG-T1.4.1 | TODO | Depends on PG-T1.3 | Authority Guild | Implement `ITenantRepository` |
| 9 | PG-T1.4.2 | TODO | Depends on PG-T1.3 | Authority Guild | Implement `IUserRepository` with password hash handling |
| 10 | PG-T1.4.3 | TODO | Depends on PG-T1.3 | Authority Guild | Implement `IRoleRepository` |
| 11 | PG-T1.4.4 | TODO | Depends on PG-T1.3 | Authority Guild | Implement `IPermissionRepository` |
| 12 | PG-T1.5.1 | TODO | Depends on PG-T1.3 | Authority Guild | Implement `ITokenRepository` |
| 13 | PG-T1.5.2 | TODO | Depends on PG-T1.3 | Authority Guild | Implement `IRefreshTokenRepository` |
| 14 | PG-T1.5.3 | TODO | Depends on PG-T1.3 | Authority Guild | Implement `IApiKeyRepository` |
| 15 | PG-T1.6.1 | TODO | Depends on PG-T1.3 | Authority Guild | Implement `ISessionRepository` |
| 16 | PG-T1.6.2 | TODO | Depends on PG-T1.3 | Authority Guild | Implement `IAuditRepository` |
| 17 | PG-T1.7 | TODO | Depends on PG-T1.4-6 | Authority Guild | Add configuration switch in `ServiceCollectionExtensions` |
| 18 | PG-T1.8.1 | TODO | Depends on PG-T1.7 | Authority Guild | Write integration tests for all repositories |
| 1 | PG-T1.1 | DONE | Completed in Phase 0 | Authority Guild | Create `StellaOps.Authority.Storage.Postgres` project structure |
| 2 | PG-T1.2.1 | DONE | Completed in Phase 0 | Authority Guild | Create schema migration for `authority` schema |
| 3 | PG-T1.2.2 | DONE | Completed in Phase 0 | Authority Guild | Create `tenants` table with indexes |
| 4 | PG-T1.2.3 | DONE | Completed in Phase 0 | Authority Guild | Create `users`, `roles`, `permissions` tables |
| 5 | PG-T1.2.4 | DONE | Completed in Phase 0 | Authority Guild | Create `tokens`, `refresh_tokens`, `api_keys` tables |
| 6 | PG-T1.2.5 | DONE | Completed in Phase 0 | Authority Guild | Create `sessions`, `audit` tables |
| 7 | PG-T1.3 | DONE | Completed in Phase 0 | Authority Guild | Implement `AuthorityDataSource` class |
| 8 | PG-T1.4.1 | DONE | Completed in Phase 0 | Authority Guild | Implement `ITenantRepository` |
| 9 | PG-T1.4.2 | DONE | Completed in Phase 0 | Authority Guild | Implement `IUserRepository` with password hash handling |
| 10 | PG-T1.4.3 | DONE | Completed 2025-11-29 | Authority Guild | Implement `IRoleRepository` |
| 11 | PG-T1.4.4 | DONE | Completed 2025-11-29 | Authority Guild | Implement `IPermissionRepository` |
| 12 | PG-T1.5.1 | DONE | Completed 2025-11-29 | Authority Guild | Implement `ITokenRepository` |
| 13 | PG-T1.5.2 | DONE | Completed 2025-11-29 | Authority Guild | Implement `IRefreshTokenRepository` |
| 14 | PG-T1.5.3 | DONE | Completed 2025-11-29 | Authority Guild | Implement `IApiKeyRepository` |
| 15 | PG-T1.6.1 | DONE | Completed 2025-11-29 | Authority Guild | Implement `ISessionRepository` |
| 16 | PG-T1.6.2 | DONE | Completed 2025-11-29 | Authority Guild | Implement `IAuditRepository` |
| 17 | PG-T1.7 | DONE | Completed 2025-11-29 | Authority Guild | Add configuration switch in `ServiceCollectionExtensions` |
| 18 | PG-T1.8.1 | DONE | Completed 2025-11-29 | Authority Guild | Write integration tests for all repositories |
| 19 | PG-T1.8.2 | TODO | Depends on PG-T1.8.1 | Authority Guild | Write determinism tests for token generation |
| 20 | PG-T1.9 | TODO | Depends on PG-T1.8 | Authority Guild | Optional: Implement dual-write wrapper for Tier A verification |
| 21 | PG-T1.10 | TODO | Depends on PG-T1.8 | Authority Guild | Run backfill from MongoDB to PostgreSQL |
@@ -49,6 +49,9 @@
| Date (UTC) | Update | Owner |
| --- | --- | --- |
| 2025-11-28 | Sprint file created | Planning |
| 2025-11-29 | All repository implementations completed (PG-T1.1 through PG-T1.6.2) | Claude |
| 2025-11-29 | ServiceCollectionExtensions updated with all repository registrations (PG-T1.7) | Claude |
| 2025-11-29 | Integration tests created for all repositories (PG-T1.8.1) | Claude |
## Decisions & Risks
- Password hashes stored as TEXT; Argon2id parameters in separate columns.

35
docs/implplan/SPRINT_3402_0001_0001_postgres_scheduler.md
View File

@@ -21,22 +21,22 @@
## Delivery Tracker
| # | Task ID | Status | Key dependency / next step | Owners | Task Definition |
| --- | --- | --- | --- | --- | --- |
| 1 | PG-T2.1 | TODO | Depends on PG-T0.7 | Scheduler Guild | Create `StellaOps.Scheduler.Storage.Postgres` project structure |
| 2 | PG-T2.2.1 | TODO | Depends on PG-T2.1 | Scheduler Guild | Create schema migration for `scheduler` schema |
| 3 | PG-T2.2.2 | TODO | Depends on PG-T2.2.1 | Scheduler Guild | Create `jobs` table with status enum and indexes |
| 4 | PG-T2.2.3 | TODO | Depends on PG-T2.2.1 | Scheduler Guild | Create `triggers` table with cron expression support |
| 5 | PG-T2.2.4 | TODO | Depends on PG-T2.2.1 | Scheduler Guild | Create `workers`, `leases` tables |
| 6 | PG-T2.2.5 | TODO | Depends on PG-T2.2.1 | Scheduler Guild | Create `job_history`, `metrics` tables |
| 7 | PG-T2.3 | TODO | Depends on PG-T2.2 | Scheduler Guild | Implement `SchedulerDataSource` class |
| 8 | PG-T2.4.1 | TODO | Depends on PG-T2.3 | Scheduler Guild | Implement `IJobRepository` with `FOR UPDATE SKIP LOCKED` |
| 9 | PG-T2.4.2 | TODO | Depends on PG-T2.3 | Scheduler Guild | Implement `ITriggerRepository` with next-fire calculation |
| 10 | PG-T2.4.3 | TODO | Depends on PG-T2.3 | Scheduler Guild | Implement `IWorkerRepository` for heartbeat tracking |
| 11 | PG-T2.5.1 | TODO | Depends on PG-T2.3 | Scheduler Guild | Implement distributed lock using `pg_advisory_lock` |
| 12 | PG-T2.5.2 | TODO | Depends on PG-T2.5.1 | Scheduler Guild | Implement `IDistributedLockRepository` interface |
| 13 | PG-T2.6.1 | TODO | Depends on PG-T2.3 | Scheduler Guild | Implement `IJobHistoryRepository` |
| 14 | PG-T2.6.2 | TODO | Depends on PG-T2.3 | Scheduler Guild | Implement `IMetricsRepository` |
| 15 | PG-T2.7 | TODO | Depends on PG-T2.4-6 | Scheduler Guild | Add configuration switch in `ServiceCollectionExtensions` |
| 16 | PG-T2.8.1 | TODO | Depends on PG-T2.7 | Scheduler Guild | Write integration tests for job queue operations |
| 1 | PG-T2.1 | DONE | Completed in Phase 0 | Scheduler Guild | Create `StellaOps.Scheduler.Storage.Postgres` project structure |
| 2 | PG-T2.2.1 | DONE | Completed in Phase 0 | Scheduler Guild | Create schema migration for `scheduler` schema |
| 3 | PG-T2.2.2 | DONE | Completed in Phase 0 | Scheduler Guild | Create `jobs` table with status enum and indexes |
| 4 | PG-T2.2.3 | DONE | Completed in Phase 0 | Scheduler Guild | Create `triggers` table with cron expression support |
| 5 | PG-T2.2.4 | DONE | Completed in Phase 0 | Scheduler Guild | Create `workers`, `leases` tables |
| 6 | PG-T2.2.5 | DONE | Completed in Phase 0 | Scheduler Guild | Create `job_history`, `metrics` tables |
| 7 | PG-T2.3 | DONE | Completed in Phase 0 | Scheduler Guild | Implement `SchedulerDataSource` class |
| 8 | PG-T2.4.1 | DONE | Completed in Phase 0 | Scheduler Guild | Implement `IJobRepository` with `FOR UPDATE SKIP LOCKED` |
| 9 | PG-T2.4.2 | DONE | Completed 2025-11-29 | Scheduler Guild | Implement `ITriggerRepository` with next-fire calculation |
| 10 | PG-T2.4.3 | DONE | Completed 2025-11-29 | Scheduler Guild | Implement `IWorkerRepository` for heartbeat tracking |
| 11 | PG-T2.5.1 | DONE | Completed 2025-11-29 | Scheduler Guild | Implement distributed lock using `pg_advisory_lock` |
| 12 | PG-T2.5.2 | DONE | Completed 2025-11-29 | Scheduler Guild | Implement `IDistributedLockRepository` interface |
| 13 | PG-T2.6.1 | DONE | Completed 2025-11-29 | Scheduler Guild | Implement `IJobHistoryRepository` |
| 14 | PG-T2.6.2 | DONE | Completed 2025-11-29 | Scheduler Guild | Implement `IMetricsRepository` |
| 15 | PG-T2.7 | DONE | Completed 2025-11-29 | Scheduler Guild | Add configuration switch in `ServiceCollectionExtensions` |
| 16 | PG-T2.8.1 | DONE | Completed 2025-11-29 | Scheduler Guild | Write integration tests for job queue operations |
| 17 | PG-T2.8.2 | TODO | Depends on PG-T2.8.1 | Scheduler Guild | Write determinism tests for trigger calculations |
| 18 | PG-T2.8.3 | TODO | Depends on PG-T2.8.1 | Scheduler Guild | Write concurrency tests for distributed locking |
| 19 | PG-T2.9 | TODO | Depends on PG-T2.8 | Scheduler Guild | Run backfill from MongoDB to PostgreSQL |
@@ -47,6 +47,9 @@
| Date (UTC) | Update | Owner |
| --- | --- | --- |
| 2025-11-28 | Sprint file created | Planning |
| 2025-11-29 | All repository implementations completed (PG-T2.1 through PG-T2.6.2) | Claude |
| 2025-11-29 | ServiceCollectionExtensions updated with all repository registrations (PG-T2.7) | Claude |
| 2025-11-29 | Integration tests created for Trigger, DistributedLock, Worker repositories (PG-T2.8.1) | Claude |
## Decisions & Risks
- PostgreSQL advisory locks replace MongoDB distributed locks.

53
docs/implplan/SPRINT_3403_0001_0001_postgres_notify.md
View File

@@ -21,31 +21,31 @@
## Delivery Tracker
| # | Task ID | Status | Key dependency / next step | Owners | Task Definition |
| --- | --- | --- | --- | --- | --- |
| 1 | PG-T3.1 | TODO | Depends on PG-T0.7 | Notify Guild | Create `StellaOps.Notify.Storage.Postgres` project structure |
| 2 | PG-T3.2.1 | TODO | Depends on PG-T3.1 | Notify Guild | Create schema migration for `notify` schema |
| 3 | PG-T3.2.2 | TODO | Depends on PG-T3.2.1 | Notify Guild | Create `channels` table (email, slack, teams, webhook) |
| 4 | PG-T3.2.3 | TODO | Depends on PG-T3.2.1 | Notify Guild | Create `rules`, `templates` tables |
| 5 | PG-T3.2.4 | TODO | Depends on PG-T3.2.1 | Notify Guild | Create `deliveries` table with status tracking |
| 6 | PG-T3.2.5 | TODO | Depends on PG-T3.2.1 | Notify Guild | Create `digests`, `quiet_hours`, `maintenance_windows` tables |
| 7 | PG-T3.2.6 | TODO | Depends on PG-T3.2.1 | Notify Guild | Create `escalation_policies`, `escalation_states` tables |
| 8 | PG-T3.2.7 | TODO | Depends on PG-T3.2.1 | Notify Guild | Create `on_call_schedules`, `inbox`, `incidents` tables |
| 9 | PG-T3.3 | TODO | Depends on PG-T3.2 | Notify Guild | Implement `NotifyDataSource` class |
| 10 | PG-T3.4.1 | TODO | Depends on PG-T3.3 | Notify Guild | Implement `IChannelRepository` |
| 11 | PG-T3.4.2 | TODO | Depends on PG-T3.3 | Notify Guild | Implement `IRuleRepository` with filter JSONB |
| 12 | PG-T3.4.3 | TODO | Depends on PG-T3.3 | Notify Guild | Implement `ITemplateRepository` with localization |
| 13 | PG-T3.5.1 | TODO | Depends on PG-T3.3 | Notify Guild | Implement `IDeliveryRepository` with status transitions |
| 14 | PG-T3.5.2 | TODO | Depends on PG-T3.3 | Notify Guild | Implement retry logic for failed deliveries |
| 15 | PG-T3.6.1 | TODO | Depends on PG-T3.3 | Notify Guild | Implement `IDigestRepository` |
| 16 | PG-T3.6.2 | TODO | Depends on PG-T3.3 | Notify Guild | Implement `IQuietHoursRepository` |
| 17 | PG-T3.6.3 | TODO | Depends on PG-T3.3 | Notify Guild | Implement `IMaintenanceWindowRepository` |
| 18 | PG-T3.7.1 | TODO | Depends on PG-T3.3 | Notify Guild | Implement `IEscalationPolicyRepository` |
| 19 | PG-T3.7.2 | TODO | Depends on PG-T3.3 | Notify Guild | Implement `IEscalationStateRepository` |
| 20 | PG-T3.7.3 | TODO | Depends on PG-T3.3 | Notify Guild | Implement `IOnCallScheduleRepository` |
| 21 | PG-T3.8.1 | TODO | Depends on PG-T3.3 | Notify Guild | Implement `IInboxRepository` |
| 22 | PG-T3.8.2 | TODO | Depends on PG-T3.3 | Notify Guild | Implement `IIncidentRepository` |
| 23 | PG-T3.8.3 | TODO | Depends on PG-T3.3 | Notify Guild | Implement `IAuditRepository` |
| 24 | PG-T3.9 | TODO | Depends on PG-T3.4-8 | Notify Guild | Add configuration switch in `ServiceCollectionExtensions` |
| 25 | PG-T3.10.1 | TODO | Depends on PG-T3.9 | Notify Guild | Write integration tests for all repositories |
| 1 | PG-T3.1 | DONE | Completed in Phase 0 | Notify Guild | Create `StellaOps.Notify.Storage.Postgres` project structure |
| 2 | PG-T3.2.1 | DONE | Completed in Phase 0 | Notify Guild | Create schema migration for `notify` schema |
| 3 | PG-T3.2.2 | DONE | Completed in Phase 0 | Notify Guild | Create `channels` table (email, slack, teams, webhook) |
| 4 | PG-T3.2.3 | DONE | Completed in Phase 0 | Notify Guild | Create `rules`, `templates` tables |
| 5 | PG-T3.2.4 | DONE | Completed in Phase 0 | Notify Guild | Create `deliveries` table with status tracking |
| 6 | PG-T3.2.5 | DONE | Completed in Phase 0 | Notify Guild | Create `digests`, `quiet_hours`, `maintenance_windows` tables |
| 7 | PG-T3.2.6 | DONE | Completed in Phase 0 | Notify Guild | Create `escalation_policies`, `escalation_states` tables |
| 8 | PG-T3.2.7 | DONE | Completed in Phase 0 | Notify Guild | Create `on_call_schedules`, `inbox`, `incidents` tables |
| 9 | PG-T3.3 | DONE | Completed in Phase 0 | Notify Guild | Implement `NotifyDataSource` class |
| 10 | PG-T3.4.1 | DONE | Completed in Phase 0 | Notify Guild | Implement `IChannelRepository` |
| 11 | PG-T3.4.2 | DONE | Completed 2025-11-29 | Notify Guild | Implement `IRuleRepository` with filter JSONB |
| 12 | PG-T3.4.3 | DONE | Completed 2025-11-29 | Notify Guild | Implement `ITemplateRepository` with localization |
| 13 | PG-T3.5.1 | DONE | Completed in Phase 0 | Notify Guild | Implement `IDeliveryRepository` with status transitions |
| 14 | PG-T3.5.2 | DONE | Completed in Phase 0 | Notify Guild | Implement retry logic for failed deliveries |
| 15 | PG-T3.6.1 | DONE | Completed 2025-11-29 | Notify Guild | Implement `IDigestRepository` |
| 16 | PG-T3.6.2 | DONE | Completed 2025-11-29 | Notify Guild | Implement `IQuietHoursRepository` |
| 17 | PG-T3.6.3 | DONE | Completed 2025-11-29 | Notify Guild | Implement `IMaintenanceWindowRepository` |
| 18 | PG-T3.7.1 | DONE | Completed 2025-11-29 | Notify Guild | Implement `IEscalationPolicyRepository` |
| 19 | PG-T3.7.2 | DONE | Completed 2025-11-29 | Notify Guild | Implement `IEscalationStateRepository` |
| 20 | PG-T3.7.3 | DONE | Completed 2025-11-29 | Notify Guild | Implement `IOnCallScheduleRepository` |
| 21 | PG-T3.8.1 | DONE | Completed 2025-11-29 | Notify Guild | Implement `IInboxRepository` |
| 22 | PG-T3.8.2 | DONE | Completed 2025-11-29 | Notify Guild | Implement `IIncidentRepository` |
| 23 | PG-T3.8.3 | DONE | Completed 2025-11-29 | Notify Guild | Implement `IAuditRepository` |
| 24 | PG-T3.9 | DONE | Completed 2025-11-29 | Notify Guild | Add configuration switch in `ServiceCollectionExtensions` |
| 25 | PG-T3.10.1 | DONE | Completed 2025-11-29 | Notify Guild | Write integration tests for all repositories |
| 26 | PG-T3.10.2 | TODO | Depends on PG-T3.10.1 | Notify Guild | Test notification delivery flow end-to-end |
| 27 | PG-T3.10.3 | TODO | Depends on PG-T3.10.1 | Notify Guild | Test escalation handling |
| 28 | PG-T3.10.4 | TODO | Depends on PG-T3.10.1 | Notify Guild | Test digest aggregation |
@@ -55,6 +55,9 @@
| Date (UTC) | Update | Owner |
| --- | --- | --- |
| 2025-11-28 | Sprint file created | Planning |
| 2025-11-29 | All repository implementations completed (PG-T3.1 through PG-T3.8.3) | Claude |
| 2025-11-29 | ServiceCollectionExtensions updated with all repository registrations (PG-T3.9) | Claude |
| 2025-11-29 | Integration tests created for Channel, Delivery, Rule, Template, Inbox, Digest, NotifyAudit repositories (PG-T3.10.1) | Claude |
## Decisions & Risks
- Channel configurations stored as JSONB for flexibility across channel types.

43
docs/implplan/SPRINT_3404_0001_0001_postgres_policy.md
View File

@@ -21,26 +21,26 @@
## Delivery Tracker
| # | Task ID | Status | Key dependency / next step | Owners | Task Definition |
| --- | --- | --- | --- | --- | --- |
| 1 | PG-T4.1 | TODO | Depends on PG-T0.7 | Policy Guild | Create `StellaOps.Policy.Storage.Postgres` project structure |
| 2 | PG-T4.2.1 | TODO | Depends on PG-T4.1 | Policy Guild | Create schema migration for `policy` schema |
| 3 | PG-T4.2.2 | TODO | Depends on PG-T4.2.1 | Policy Guild | Create `packs`, `pack_versions` tables |
| 4 | PG-T4.2.3 | TODO | Depends on PG-T4.2.1 | Policy Guild | Create `rules` table with Rego content |
| 5 | PG-T4.2.4 | TODO | Depends on PG-T4.2.1 | Policy Guild | Create `risk_profiles` table with version history |
| 6 | PG-T4.2.5 | TODO | Depends on PG-T4.2.1 | Policy Guild | Create `evaluation_runs`, `explanations` tables |
| 7 | PG-T4.2.6 | TODO | Depends on PG-T4.2.1 | Policy Guild | Create `exceptions`, `audit` tables |
| 8 | PG-T4.3 | TODO | Depends on PG-T4.2 | Policy Guild | Implement `PolicyDataSource` class |
| 9 | PG-T4.4.1 | TODO | Depends on PG-T4.3 | Policy Guild | Implement `IPackRepository` with CRUD |
| 10 | PG-T4.4.2 | TODO | Depends on PG-T4.3 | Policy Guild | Implement version management for packs |
| 11 | PG-T4.4.3 | TODO | Depends on PG-T4.3 | Policy Guild | Implement active version promotion |
| 12 | PG-T4.5.1 | TODO | Depends on PG-T4.3 | Policy Guild | Implement `IRiskProfileRepository` |
| 13 | PG-T4.5.2 | TODO | Depends on PG-T4.3 | Policy Guild | Implement version history for risk profiles |
| 14 | PG-T4.5.3 | TODO | Depends on PG-T4.3 | Policy Guild | Implement `GetVersionAsync` and `ListVersionsAsync` |
| 15 | PG-T4.6.1 | TODO | Depends on PG-T4.3 | Policy Guild | Implement `IEvaluationRunRepository` |
| 16 | PG-T4.6.2 | TODO | Depends on PG-T4.3 | Policy Guild | Implement `IExplanationRepository` |
| 17 | PG-T4.6.3 | TODO | Depends on PG-T4.3 | Policy Guild | Implement `IExceptionRepository` |
| 18 | PG-T4.6.4 | TODO | Depends on PG-T4.3 | Policy Guild | Implement `IAuditRepository` |
| 19 | PG-T4.7 | TODO | Depends on PG-T4.4-6 | Policy Guild | Add configuration switch in `ServiceCollectionExtensions` |
| 20 | PG-T4.8.1 | TODO | Depends on PG-T4.7 | Policy Guild | Write integration tests for all repositories |
| 1 | PG-T4.1 | DONE | Completed in Phase 0 | Policy Guild | Create `StellaOps.Policy.Storage.Postgres` project structure |
| 2 | PG-T4.2.1 | DONE | Completed in Phase 0 | Policy Guild | Create schema migration for `policy` schema |
| 3 | PG-T4.2.2 | DONE | Completed in Phase 0 | Policy Guild | Create `packs`, `pack_versions` tables |
| 4 | PG-T4.2.3 | DONE | Completed in Phase 0 | Policy Guild | Create `rules` table with Rego content |
| 5 | PG-T4.2.4 | DONE | Completed in Phase 0 | Policy Guild | Create `risk_profiles` table with version history |
| 6 | PG-T4.2.5 | DONE | Completed in Phase 0 | Policy Guild | Create `evaluation_runs`, `explanations` tables |
| 7 | PG-T4.2.6 | DONE | Completed in Phase 0 | Policy Guild | Create `exceptions`, `audit` tables |
| 8 | PG-T4.3 | DONE | Completed in Phase 0 | Policy Guild | Implement `PolicyDataSource` class |
| 9 | PG-T4.4.1 | DONE | Completed in Phase 0 | Policy Guild | Implement `IPackRepository` with CRUD |
| 10 | PG-T4.4.2 | DONE | Completed in Phase 0 | Policy Guild | Implement version management for packs |
| 11 | PG-T4.4.3 | DONE | Completed in Phase 0 | Policy Guild | Implement active version promotion |
| 12 | PG-T4.5.1 | DONE | Completed in Phase 0 | Policy Guild | Implement `IRiskProfileRepository` |
| 13 | PG-T4.5.2 | DONE | Completed in Phase 0 | Policy Guild | Implement version history for risk profiles |
| 14 | PG-T4.5.3 | DONE | Completed in Phase 0 | Policy Guild | Implement `GetVersionAsync` and `ListVersionsAsync` |
| 15 | PG-T4.6.1 | DONE | Completed in Phase 0 | Policy Guild | Implement `IEvaluationRunRepository` |
| 16 | PG-T4.6.2 | DONE | Completed 2025-11-29 | Policy Guild | Implement `IExplanationRepository` |
| 17 | PG-T4.6.3 | DONE | Completed in Phase 0 | Policy Guild | Implement `IExceptionRepository` |
| 18 | PG-T4.6.4 | DONE | Completed 2025-11-29 | Policy Guild | Implement `IAuditRepository` |
| 19 | PG-T4.7 | DONE | Completed 2025-11-29 | Policy Guild | Add configuration switch in `ServiceCollectionExtensions` |
| 20 | PG-T4.8.1 | DONE | Completed 2025-11-29 | Policy Guild | Write integration tests for all repositories |
| 21 | PG-T4.8.2 | TODO | Depends on PG-T4.8.1 | Policy Guild | Test pack versioning workflow |
| 22 | PG-T4.8.3 | TODO | Depends on PG-T4.8.1 | Policy Guild | Test risk profile version history |
| 23 | PG-T4.9 | TODO | Depends on PG-T4.8 | Policy Guild | Export active packs from MongoDB |
@@ -52,6 +52,9 @@
| Date (UTC) | Update | Owner |
| --- | --- | --- |
| 2025-11-28 | Sprint file created | Planning |
| 2025-11-29 | All repository implementations completed (PG-T4.1 through PG-T4.6.4) | Claude |
| 2025-11-29 | ServiceCollectionExtensions updated with all repository registrations (PG-T4.7) | Claude |
| 2025-11-29 | Integration tests created for Pack, Rule, Exception, EvaluationRun, RiskProfile, PolicyAudit repositories (PG-T4.8.1) | Claude |
## Decisions & Risks
- Pack versions are immutable once published; new versions create new rows.

0
docs/product-advisories/archived/24-Nov-2025 - Designing a Deterministic Reachability Benchmark.md → docs/product-advisories/24-Nov-2025 - Designing a Deterministic Reachability Benchmark.md
View File

0
docs/product-advisories/archived/25-Nov-2025 - Add CVSS v4.0 Score Receipts for Transparency.md → docs/product-advisories/25-Nov-2025 - Add CVSS v4.0 Score Receipts for Transparency.md
View File

0
docs/product-advisories/archived/25-Nov-2025 - Define Safe VEX 'Not Affected' Claims with Proofs.md → docs/product-advisories/25-Nov-2025 - Define Safe VEX 'Not Affected' Claims with Proofs.md
View File

602
docs/product-advisories/25-Nov-2025 - Half‑Life Confidence Decay for Unknowns.md
View File

@@ -1,602 +0,0 @@
Here’s a simple, low‑friction way to keep priorities fresh without constant manual grooming: **let confidence decay over time**.
![A small curve sloping down over time, illustrating exponential decay](https://dummyimage.com/800x250/ffffff/000000\&text=confidence\(t\)%20=%20e^{-t/τ})
# Exponential confidence decay (what & why)
* **Idea:** Every item (task, lead, bug, doc, hypothesis) has a confidence score that **automatically shrinks with time** if you don’t touch it.
* **Formula:** `confidence(t) = e^(−t/τ)` where `t` is days since last signal (edit, comment, commit, new data), and **τ (“tau”)** is the decay constant.
* **Rule of thumb:** With **τ = 30 days**, at **t = 30** the confidence is **e^(−1) ≈ 0.37**—about a **63% drop**. This surfaces long‑ignored items *gradually*, not with harsh “stale/expired” flips.
# How to use it in practice
* **Signals that reset t → 0:** comment on the ticket, new benchmark, fresh log sample, doc update, CI run, new market news.
* **Sort queues by:** `priority × confidence(t)` (or severity × confidence). Quiet items drift down; truly active ones stay up.
* **Escalation bands:**
* `>0.6` = green (recently touched)
* `0.3–0.6` = amber (review soon)
* `<0.3` = red (poke or close)
# Quick presets
* **Fast‑moving queues (incidents, hot leads):** τ = **7–14** days
* **Engineering tasks / product docs:** τ = **30** days
* **Research bets / roadmaps:** τ = **60–90** days
# For your world (Stella Ops + ops/dev work)
* **Vuln tickets:** `risk_score = CVSS × reachability × e^(−t/30)`
* **Roadmap epics:** `value_score = impact × e^(−t/60)` to re‑rank quarterly.
* **Docs:** show a badge “freshness: 42%” derived from last edit age to nudge updates.
# Minimal implementation sketch
* Store per‑item: `last_signal_at`, `base_priority`.
* Compute on read:
```
days = (now - last_signal_at).days
conf = exp(-days / tau)
score = base_priority * conf
```
* Recompute in your API layer or materialize nightly; no cron spam needed.
If you want, I can draft a tiny C# helper (and SQL snippet) you can drop into your issue service to add `confidence(t)` and color bands to your lists.
Perfect, let’s turn the idea into something your devs can actually build.
Below is an implementation plan you can drop into a ticketing/PRD — with clear phases, data model changes, APIs, and some sample code (C# + SQL). I’ll also sprinkle in Stella Ops–specific notes.
---
## 0. Scope & Objectives
**Goal:** Introduce `confidence(t)` as an automatic freshness factor that decays with time and is used to rank and highlight work.
We’ll apply it to:
* Vulnerabilities (Stella Ops)
* General issues / tasks / epics
* (Optional) Docs, leads, hypotheses later
**Core behavior:**
* Each item has:
* A base priority / risk (from severity, business impact, etc.)
* A timestamp of last signal (meaningful activity)
* A decay rate τ (tau) in days
* Effective priority = `base_priority × confidence(t)`
* `confidence(t) = exp(− t / τ)` where `t` = days since last_signal
---
## 1. Data Model Changes
### 1.1. Add fields to core “work item” tables
For each relevant table (`Issues`, `Vulnerabilities`, `Epics`, …):
**New columns:**
* `base_priority` (FLOAT or INT)
* Example: 1–100, or derived from severity.
* `last_signal_at` (DATETIME, NOT NULL, default = `created_at`)
* `tau_days` (FLOAT, nullable, falls back to type default)
* (Optional) `confidence_score_cached` (FLOAT, for materialized score)
* (Optional) `is_confidence_frozen` (BOOL, default FALSE)
For pinned items that should not decay.
**Example Postgres migration (Issues):**
```sql
ALTER TABLE issues
ADD COLUMN base_priority DOUBLE PRECISION,
ADD COLUMN last_signal_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
ADD COLUMN tau_days DOUBLE PRECISION,
ADD COLUMN confidence_cached DOUBLE PRECISION,
ADD COLUMN is_confidence_frozen BOOLEAN NOT NULL DEFAULT FALSE;
```
For Stella Ops:
```sql
ALTER TABLE vulnerabilities
ADD COLUMN base_risk DOUBLE PRECISION,
ADD COLUMN last_signal_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
ADD COLUMN tau_days DOUBLE PRECISION,
ADD COLUMN confidence_cached DOUBLE PRECISION,
ADD COLUMN is_confidence_frozen BOOLEAN NOT NULL DEFAULT FALSE;
```
### 1.2. Add a config table for τ per entity type
```sql
CREATE TABLE confidence_decay_config (
id SERIAL PRIMARY KEY,
entity_type TEXT NOT NULL, -- 'issue', 'vulnerability', 'epic', 'doc'
tau_days_default DOUBLE PRECISION NOT NULL,
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
INSERT INTO confidence_decay_config (entity_type, tau_days_default) VALUES
('incident', 7),
('vulnerability', 30),
('issue', 30),
('epic', 60),
('doc', 90);
```
---
## 2. Define “signal” events & instrumentation
We need a standardized way to say: “this item got activity → reset last_signal_at”.
### 2.1. Signals that should reset `last_signal_at`
For **issues / epics:**
* New comment
* Status change (e.g., Open → In Progress)
* Field change that matters (severity, owner, milestone)
* Attachment added
* Link to PR added or updated
* New CI failure linked
For **vulnerabilities (Stella Ops):**
* New scanner result attached or status updated (e.g., “Verified”, “False Positive”)
* New evidence (PoC, exploit notes)
* SLA override change
* Assignment / ownership change
* Integration events (e.g., PR merge that references the vuln)
For **docs (if you do it):**
* Any edit
* Comment/annotation
### 2.2. Implement a shared helper to record a signal
**Service-level helper (pseudocode / C#-ish):**
```csharp
public interface IConfidenceSignalService
{
Task RecordSignalAsync(WorkItemType type, Guid itemId, DateTime? signalTimeUtc = null);
}
public class ConfidenceSignalService : IConfidenceSignalService
{
private readonly IWorkItemRepository _repo;
private readonly IConfidenceConfigService _config;
public async Task RecordSignalAsync(WorkItemType type, Guid itemId, DateTime? signalTimeUtc = null)
{
var now = signalTimeUtc ?? DateTime.UtcNow;
var item = await _repo.GetByIdAsync(type, itemId);
if (item == null) return;
item.LastSignalAt = now;
if (item.TauDays == null)
{
item.TauDays = await _config.GetDefaultTauAsync(type);
}
await _repo.UpdateAsync(item);
}
}
```
### 2.3. Wire signals into existing flows
Create small tasks for devs like:
* **ISS-01:** Call `RecordSignalAsync` on:
* New issue comment handler
* Issue status update handler
* Issue field update handler (severity/priority/owner)
* **VULN-01:** Call `RecordSignalAsync` when:
* New scanner result ingested for a vuln
* Vulnerability status, SLA, or owner changes
* New exploit evidence is attached
---
## 3. Confidence & scoring calculation
### 3.1. Shared confidence function
Definition:
```csharp
public static class ConfidenceMath
{
// t = days since last signal
public static double ConfidenceScore(DateTime lastSignalAtUtc, double tauDays, DateTime? nowUtc = null)
{
var now = nowUtc ?? DateTime.UtcNow;
var tDays = (now - lastSignalAtUtc).TotalDays;
if (tDays <= 0) return 1.0;
if (tauDays <= 0) return 1.0; // guard / fallback
var score = Math.Exp(-tDays / tauDays);
// Optional: never drop below a tiny floor, so items never "disappear"
const double floor = 0.01;
return Math.Max(score, floor);
}
}
```
### 3.2. Effective priority formulas
**Generic issues / tasks:**
```csharp
double effectiveScore = issue.BasePriority * ConfidenceMath.ConfidenceScore(issue.LastSignalAt, issue.TauDays ?? defaultTau);
```
**Vulnerabilities (Stella Ops):**
Let’s define:
* `severity_weight`: map CVSS or severity string to numeric (e.g. Critical=100, High=80, Medium=50, Low=20).
* `reachability`: 0–1 (e.g. from your reachability analysis).
* `exploitability`: 0–1 (optional, based on known exploits).
* `confidence`: as above.
```csharp
double baseRisk = severityWeight * reachability * exploitability; // or simpler: severityWeight * reachability
double conf = ConfidenceMath.ConfidenceScore(vuln.LastSignalAt, vuln.TauDays ?? defaultTau);
double effectiveRisk = baseRisk * conf;
```
Store `baseRisk` → `vulnerabilities.base_risk`, and compute `effectiveRisk` on the fly or via job.
### 3.3. SQL implementation (optional for server-side sorting)
**Postgres example:**
```sql
-- t_days = age in days
-- tau = tau_days
-- score = exp(-t_days / tau)
SELECT
i.*,
i.base_priority *
GREATEST(
EXP(- EXTRACT(EPOCH FROM (NOW() - i.last_signal_at)) / (86400 * COALESCE(i.tau_days, 30))),
0.01
) AS effective_priority
FROM issues i
ORDER BY effective_priority DESC;
```
You can wrap that in a view:
```sql
CREATE VIEW issues_with_confidence AS
SELECT
i.*,
GREATEST(
EXP(- EXTRACT(EPOCH FROM (NOW() - i.last_signal_at)) / (86400 * COALESCE(i.tau_days, 30))),
0.01
) AS confidence,
i.base_priority *
GREATEST(
EXP(- EXTRACT(EPOCH FROM (NOW() - i.last_signal_at)) / (86400 * COALESCE(i.tau_days, 30))),
0.01
) AS effective_priority
FROM issues i;
```
---
## 4. Caching & performance
You have two options:
### 4.1. Compute on read (simplest to start)
* Use the helper function in your service layer or a DB view.
* Pros:
* No jobs, always fresh.
* Cons:
* Slight CPU cost on heavy lists.
**Plan:** Start with this. If you see perf issues, move to 4.2.
### 4.2. Periodic materialization job (optional later)
Add a scheduled job (e.g. hourly) that:
1. Selects all active items.
2. Computes `confidence_score` and `effective_priority`.
3. Writes to `confidence_cached` and `effective_priority_cached` (if you add such a column).
Service then sorts by cached values.
---
## 5. Backfill & migration
### 5.1. Initial backfill script
For existing records:
* If `last_signal_at` is NULL → set to `created_at`.
* Derive `base_priority` / `base_risk` from existing severity fields.
* Set `tau_days` from config.
**Example:**
```sql
UPDATE issues
SET last_signal_at = created_at
WHERE last_signal_at IS NULL;
UPDATE issues
SET base_priority = CASE severity
WHEN 'critical' THEN 100
WHEN 'high' THEN 80
WHEN 'medium' THEN 50
WHEN 'low' THEN 20
ELSE 10
END
WHERE base_priority IS NULL;
UPDATE issues i
SET tau_days = c.tau_days_default
FROM confidence_decay_config c
WHERE c.entity_type = 'issue'
AND i.tau_days IS NULL;
```
Do similarly for `vulnerabilities` using severity / CVSS.
### 5.2. Sanity checks
Add a small script/test to verify:
* Newly created items → `confidence ≈ 1.0`.
* 30-day-old items with τ=30 → `confidence ≈ 0.37`.
* Ordering changes when you edit/comment on items.
---
## 6. API & Query Layer
### 6.1. New sorting options
Update list APIs:
* Accept parameter: `sort=effective_priority` or `sort=confidence`.
* Default sort for some views:
* Vulnerabilities backlog: `sort=effective_risk` (risk × confidence).
* Issues backlog: `sort=effective_priority`.
**Example REST API contract:**
`GET /api/issues?sort=effective_priority&state=open`
**Response fields (additions):**
```json
{
"id": "ISS-123",
"title": "Fix login bug",
"base_priority": 80,
"last_signal_at": "2025-11-01T10:00:00Z",
"tau_days": 30,
"confidence": 0.63,
"effective_priority": 50.4,
"confidence_band": "amber"
}
```
### 6.2. Confidence banding (for UI)
Define bands server-side (easy to change):
* Green: `confidence >= 0.6`
* Amber: `0.3 ≤ confidence < 0.6`
* Red: `confidence < 0.3`
You can compute on server:
```csharp
string ConfidenceBand(double confidence) =>
confidence >= 0.6 ? "green"
: confidence >= 0.3 ? "amber"
: "red";
```
---
## 7. UI / UX changes
### 7.1. List views (issues / vulns / epics)
For each item row:
* Show a small freshness pill:
* Text: `Active`, `Review soon`, `Stale`
* Derived from confidence band.
* Tooltip:
* “Confidence 78%. Last activity 3 days ago. τ = 30 days.”
* Sort default: by `effective_priority` / `effective_risk`.
* Filters:
* `Freshness: [All | Active | Review soon | Stale]`
* Optionally: “Show stale only” toggle.
**Example labels:**
* Green: “Active (confidence 82%)”
* Amber: “Review soon (confidence 45%)”
* Red: “Stale (confidence 18%)”
### 7.2. Detail views
On an issue / vuln page:
* Add a “Confidence” section:
* “Confidence: **52%**”
* “Last signal: **12 days ago**”
* “Decay τ: **30 days**”
* “Effective priority: **Base 80 × 0.52 = 42**”
* (Optional) small mini-chart (text-only or simple bar) showing approximate decay, but not necessary for first iteration.
### 7.3. Admin / settings UI
Add an internal settings page:
* Table of entity types with editable τ:
| Entity type | τ (days) | Notes |
| ------------- | -------- | ---------------------------- |
| Incident | 7 | Fast-moving |
| Vulnerability | 30 | Standard risk review cadence |
| Issue | 30 | Sprint-level decay |
| Epic | 60 | Quarterly |
| Doc | 90 | Slow decay |
* Optionally: toggle to pin item (`is_confidence_frozen`) from UI.
---
## 8. Stella Ops–specific behavior
For vulnerabilities:
### 8.1. Base risk calculation
Ingested fields you likely already have:
* `cvss_score` or `severity`
* `reachable` (true/false or numeric)
* (Optional) `exploit_available` (bool) or exploitability score
* `asset_criticality` (1–5)
Define `base_risk` as:
```text
severity_weight = f(cvss_score or severity)
reachability = reachable ? 1.0 : 0.5 -- example
exploitability = exploit_available ? 1.0 : 0.7
asset_factor = 0.5 + 0.1 * asset_criticality -- 1 → 1.0, 5 → 1.5
base_risk = severity_weight * reachability * exploitability * asset_factor
```
Store `base_risk` on vuln row.
Then:
```text
effective_risk = base_risk * confidence(t)
```
Use `effective_risk` for backlog ordering and SLAs dashboards.
### 8.2. Signals for vulns
Make sure these all call `RecordSignalAsync(Vulnerability, vulnId)`:
* New scan result for same vuln (re-detected).
* Change status to “In Progress”, “Ready for Deploy”, “Verified Fixed”, etc.
* Assigning an owner.
* Attaching PoC / exploit details.
### 8.3. Vuln UI copy ideas
* Pill text:
* “Risk: 850 (confidence 68%)”
* “Last analyst activity 11 days ago”
* In backlog view: show **Effective Risk** as main sort, with a smaller subtext “Base 1200 × Confidence 71%”.
---
## 9. Rollout plan
### Phase 1 – Infrastructure (backend-only)
* [ ] DB migrations & config table
* [ ] Implement `ConfidenceMath` and helper functions
* [ ] Implement `IConfidenceSignalService`
* [ ] Wire signals into key flows (comments, state changes, scanner ingestion)
* [ ] Add `confidence` and `effective_priority/risk` to API responses
* [ ] Backfill script + dry run in staging
### Phase 2 – Internal UI & feature flag
* [ ] Add optional sorting by effective score to internal/staff views
* [ ] Add confidence pill (hidden behind feature flag `confidence_decay_v1`)
* [ ] Dogfood internally:
* Do items bubble up/down as expected?
* Are any items “disappearing” because decay is too aggressive?
### Phase 3 – Parameter tuning
* [ ] Adjust τ per type based on feedback:
* If things decay too fast → increase τ
* If queues rarely change → decrease τ
* [ ] Decide on confidence floor (0.01? 0.05?) so nothing goes to literal 0.
### Phase 4 – General release
* [ ] Make effective score the default sort for key views:
* Vulnerabilities backlog
* Issues backlog
* [ ] Document behavior for users (help center / inline tooltip)
* [ ] Add admin UI to tweak τ per entity type.
---
## 10. Edge cases & safeguards
* **New items**
* `last_signal_at = created_at`, confidence = 1.0.
* **Pinned items**
* If `is_confidence_frozen = true` → treat confidence as 1.0.
* **Items without τ**
* Always fallback to entity type default.
* **Timezones**
* Always store & compute in UTC.
* **Very old items**
* Floor the confidence so they’re still visible when explicitly searched.
---
If you want, I can turn this into:
* A short **technical design doc** (with sections: Problem, Proposal, Alternatives, Rollout).
* Or a **set of Jira tickets** grouped by backend / frontend / infra that your team can pick up directly.

0
docs/product-advisories/archived/26-Nov-2025 - Handling Rekor v2 and DSSE Air‑Gap Limits.md → docs/product-advisories/26-Nov-2025 - Handling Rekor v2 and DSSE Air‑Gap Limits.md
View File

0
docs/product-advisories/archived/26-Nov-2025 - Opening Up a Reachability Dataset.md → docs/product-advisories/26-Nov-2025 - Opening Up a Reachability Dataset.md
View File

0
docs/product-advisories/archived/26-Nov-2025 - Use Graph Revision IDs as Public Trust Anchors.md → docs/product-advisories/26-Nov-2025 - Use Graph Revision IDs as Public Trust Anchors.md
View File

0
docs/product-advisories/archived/27-Nov-2025 - Blueprint for a 2026‑Ready Scanner.md → docs/product-advisories/27-Nov-2025 - Blueprint for a 2026‑Ready Scanner.md
View File

0
docs/product-advisories/archived/27-Nov-2025 - Deep Architecture Brief - SBOM‑First, VEX‑Ready Spine.md → docs/product-advisories/27-Nov-2025 - Deep Architecture Brief - SBOM‑First, VEX‑Ready Spine.md
View File

0
docs/product-advisories/archived/27-Nov-2025 - Explainability Layer for Vulnerability Verdicts.md → docs/product-advisories/27-Nov-2025 - Explainability Layer for Vulnerability Verdicts.md
View File

0
docs/product-advisories/archived/27-Nov-2025 - Making Graphs Understandable to Humans.md → docs/product-advisories/27-Nov-2025 - Making Graphs Understandable to Humans.md
View File

0
docs/product-advisories/archived/27-Nov-2025 - Managing Ambiguity Through an Unknowns Registry.md → docs/product-advisories/27-Nov-2025 - Managing Ambiguity Through an Unknowns Registry.md
View File

0
docs/product-advisories/archived/27-Nov-2025 - Verifying Binary Reachability via DSSE Envelopes.md → docs/product-advisories/27-Nov-2025 - Verifying Binary Reachability via DSSE Envelopes.md
View File

402
docs/product-advisories/29-Nov-2025 - CLI Developer Experience and Command UX.md Normal file
View File

@@ -0,0 +1,402 @@
# CLI Developer Experience and Command UX
**Version:** 1.0
**Date:** 2025-11-29
**Status:** Canonical
This advisory defines the product rationale, command surface design, and implementation strategy for the Stella Ops CLI, covering developer experience, CI/CD integration, output formatting, and offline operation.
---
## 1. Executive Summary
The Stella Ops CLI is the **primary interface for developers and CI/CD pipelines** interacting with the platform. Key capabilities:
- **Native AOT Binary** - Sub-20ms startup, single binary distribution
- **DPoP-Bound Authentication** - Secure device-code and service principal flows
- **Deterministic Outputs** - JSON/table modes with stable exit codes for CI
- **Buildx Integration** - SBOM generation at build time
- **Offline Kit Management** - Air-gapped deployment support
- **Shell Completions** - Bash/Zsh/Fish/PowerShell auto-complete
---
## 2. Market Drivers
### 2.1 Target Segments
| Segment | CLI Requirements | Use Case |
|---------|-----------------|----------|
| **DevSecOps** | CI integration, exit codes, JSON output | Pipeline gates |
| **Security Engineers** | Verification commands, policy testing | Audit workflows |
| **Platform Operators** | Offline kit, admin commands | Air-gap management |
| **Developers** | Scan commands, buildx integration | Local development |
### 2.2 Competitive Positioning
Most CLI tools in the vulnerability space are slow or lack CI ergonomics. Stella Ops differentiates with:
- **Native AOT** for instant startup (< 20ms vs 500ms+ for JIT)
- **Deterministic exit codes** (12 distinct codes for CI decision trees)
- **DPoP security** (no long-lived tokens on disk)
- **Unified command surface** (50+ commands, consistent patterns)
- **Offline-first design** (works without network in sealed mode)
---
## 3. Command Surface Architecture
### 3.1 Command Categories
| Category | Commands | Purpose |
|----------|----------|---------|
| **Auth** | `login`, `logout`, `status`, `token` | Authentication management |
| **Scan** | `scan image`, `scan fs` | Vulnerability scanning |
| **Export** | `export sbom`, `report final` | Artifact retrieval |
| **Verify** | `verify attestation`, `verify referrers`, `verify image-signature` | Cryptographic verification |
| **Policy** | `policy get`, `policy set`, `policy apply` | Policy management |
| **Buildx** | `buildx install`, `buildx verify`, `buildx build` | Build-time SBOM |
| **Runtime** | `runtime policy test` | Zastava integration |
| **Offline** | `offline kit pull`, `offline kit import`, `offline kit status` | Air-gap operations |
| **Decision** | `decision export`, `decision verify`, `decision compare` | VEX evidence management |
| **AOC** | `sources ingest`, `aoc verify` | Aggregation-only guards |
| **KMS** | `kms export`, `kms import` | Key management |
| **Advise** | `advise run` | AI-powered advisory summaries |
### 3.2 Output Modes
**Human Mode (default):**
```
$ stella scan image nginx:latest --wait
Scanning nginx:latest...
Found 12 vulnerabilities (2 critical, 3 high, 5 medium, 2 low)
Policy verdict: FAIL
Critical:
- CVE-2025-12345 in openssl (fixed in 3.0.14)
- CVE-2025-12346 in libcurl (no fix available)
See: https://ui.internal/scans/sha256:abc123...
```
**JSON Mode (`--json`):**
```json
{"event":"scan.complete","status":"fail","critical":2,"high":3,"medium":5,"low":2,"url":"https://..."}
```
### 3.3 Exit Codes
| Code | Meaning | CI Action |
|------|---------|-----------|
| 0 | Success | Continue |
| 2 | Policy fail | Block deployment |
| 3 | Verification failed | Security alert |
| 4 | Auth error | Re-authenticate |
| 5 | Resource not found | Check inputs |
| 6 | Rate limited | Retry with backoff |
| 7 | Backend unavailable | Retry |
| 9 | Invalid arguments | Fix command |
| 11-17 | AOC guard violations | Review ingestion |
| 18 | Verification truncated | Increase limit |
| 70 | Transport failure | Check network |
| 71 | Usage error | Fix command |
---
## 4. Authentication Model
### 4.1 Device Code Flow (Interactive)
```bash
$ stella auth login
Opening browser for authentication...
Device code: ABCD-EFGH
Waiting for authorization...
Logged in as user@example.com (tenant: acme-corp)
```
### 4.2 Service Principal (CI/CD)
```bash
$ stella auth login --client-credentials \
--client-id $STELLA_CLIENT_ID \
--private-key $STELLA_PRIVATE_KEY
```
### 4.3 DPoP Key Management
- Ephemeral Ed25519 keypair generated on first login
- Stored in OS keychain (Keychain/DPAPI/KWallet/Gnome Keyring)
- Every request includes DPoP proof header
- Tokens refreshed proactively (30s before expiry)
### 4.4 Token Credential Helper
```bash
# Get one-shot token for curl/scripts
TOKEN=$(stella auth token --aud scanner)
curl -H "Authorization: Bearer $TOKEN" https://scanner.internal/api/...
```
---
## 5. Buildx Integration
### 5.1 Generator Installation
```bash
$ stella buildx install
Installing SBOM generator plugin...
Verifying signature: OK
Generator installed at ~/.docker/cli-plugins/docker-buildx-stellaops
$ stella buildx verify
Docker version: 24.0.7
Buildx version: 0.12.1
Generator: stellaops/sbom-indexer:v1.2.3@sha256:abc123...
Status: Ready
```
### 5.2 Build with SBOM
```bash
$ stella buildx build -t myapp:v1.0.0 --push --attest
Building myapp:v1.0.0...
SBOM generation: enabled (stellaops/sbom-indexer)
Provenance: enabled
Attestation: requested
Build complete!
Image: myapp:v1.0.0@sha256:def456...
SBOM: attached as referrer
Attestation: logged to Rekor (uuid: abc123)
```
---
## 6. Implementation Strategy
### 6.1 Phase 1: Core Commands (Complete)
- [x] Auth commands with DPoP
- [x] Scan/export commands
- [x] JSON output mode
- [x] Exit code standardization
- [x] Shell completions
### 6.2 Phase 2: Buildx & Verification (Complete)
- [x] Buildx plugin management
- [x] Attestation verification
- [x] Referrer verification
- [x] Report commands
### 6.3 Phase 3: Advanced Features (In Progress)
- [x] Decision export/verify commands
- [x] AOC guard helpers
- [x] KMS management
- [ ] Advisory AI integration (CLI-ADVISE-48-001)
- [ ] Filesystem scanning (CLI-SCAN-49-001)
### 6.4 Phase 4: Distribution (Planned)
- [ ] Homebrew formula
- [ ] Scoop/Winget manifests
- [ ] Self-update mechanism
- [ ] Cosign signature verification
---
## 7. CI/CD Integration Patterns
### 7.1 GitHub Actions
```yaml
- name: Install Stella CLI
run: |
curl -sSL https://get.stella-ops.io | sh
echo "$HOME/.stella/bin" >> $GITHUB_PATH
- name: Authenticate
run: stella auth login --client-credentials
env:
STELLAOPS_CLIENT_ID: ${{ secrets.STELLA_CLIENT_ID }}
STELLAOPS_PRIVATE_KEY: ${{ secrets.STELLA_PRIVATE_KEY }}
- name: Scan Image
run: |
stella scan image ${{ env.IMAGE_REF }} --wait --json > scan-results.json
if [ $? -eq 2 ]; then
echo "::error::Policy failed - blocking deployment"
exit 1
fi
- name: Verify Attestation
run: stella verify attestation --artifact ${{ env.IMAGE_DIGEST }}
```
### 7.2 GitLab CI
```yaml
scan:
script:
- stella auth login --client-credentials
- stella buildx install
- docker buildx build --attest=type=sbom,generator=stellaops/sbom-indexer -t $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA .
- stella scan image $CI_REGISTRY_IMAGE@$IMAGE_DIGEST --wait --json
artifacts:
reports:
container_scanning: scan-results.json
```
---
## 8. Configuration Model
### 8.1 Precedence
CLI flags > Environment variables > Config file > Defaults
### 8.2 Config File
```yaml
# ~/.config/stellaops/config.yaml
cli:
authority: "https://authority.example.com"
backend:
scanner: "https://scanner.example.com"
attestor: "https://attestor.example.com"
auth:
deviceCode: true
audienceDefault: "scanner"
output:
json: false
color: auto
tls:
caBundle: "/etc/ssl/certs/ca-bundle.crt"
offline:
kitMirror: "s3://mirror/stellaops-kit"
```
### 8.3 Environment Variables
| Variable | Purpose |
|----------|---------|
| `STELLAOPS_AUTHORITY` | Authority URL |
| `STELLAOPS_SCANNER_URL` | Scanner service URL |
| `STELLAOPS_CLIENT_ID` | Service principal ID |
| `STELLAOPS_PRIVATE_KEY` | Service principal key |
| `STELLAOPS_TENANT` | Default tenant |
| `STELLAOPS_JSON` | Enable JSON output |
---
## 9. Offline Operation
### 9.1 Sealed Mode Detection
```bash
$ stella scan image nginx:latest
Error: Sealed mode active - external network access blocked
Remediation: Import offline kit or disable sealed mode
$ stella offline kit import latest-kit.tar.gz
Importing offline kit...
Advisories: 45,230 records
VEX documents: 12,450 records
Policy packs: 3 bundles
Import complete!
$ stella scan image nginx:latest
Scanning with offline data (2025-11-28)...
```
### 9.2 Air-Gap Guard
All HTTP flows route through `StellaOps.AirGap.Policy`. When sealed mode is active:
- External egress is blocked with `AIRGAP_EGRESS_BLOCKED` error
- CLI provides clear remediation guidance
- Local verification continues to work
---
## 10. Security Considerations
### 10.1 Credential Protection
- DPoP private keys stored in OS keychain only
- No plaintext tokens on disk
- Short-lived OpToks held in memory only
- Authorization headers redacted from verbose logs
### 10.2 Binary Verification
```bash
# Verify CLI binary signature
$ stella version --verify
Version: 1.2.3
Built: 2025-11-29T12:00:00Z
Signature: Valid (cosign)
Signer: release@stella-ops.io
```
### 10.3 Hard Lines
- Refuse to print token values
- Disallow `--insecure` without explicit env var opt-in
- Enforce short token TTL with proactive refresh
- Device-code cache bound to machine + user
---
## 11. Performance Targets
| Metric | Target |
|--------|--------|
| Startup time | < 20ms (AOT) |
| Request overhead | < 5ms |
| Large download (100MB) | > 80 MB/s |
| Buildx wrapper overhead | < 1ms |
---
## 12. Related Documentation
| Resource | Location |
|----------|----------|
| CLI architecture | `docs/modules/cli/architecture.md` |
| Policy CLI guide | `docs/modules/cli/guides/policy.md` |
| API/CLI reference | `docs/09_API_CLI_REFERENCE.md` |
| Offline operation | `docs/24_OFFLINE_KIT.md` |
---
## 13. Sprint Mapping
- **Primary Sprint:** SPRINT_0400_cli_ux.md (NEW)
- **Related Sprints:**
- SPRINT_210_ui_ii.md (UI integration)
- SPRINT_0187_0001_0001_evidence_locker_cli_integration.md (Evidence CLI)
**Key Task IDs:**
- `CLI-AUTH-10-001` - DPoP authentication (DONE)
- `CLI-SCAN-20-001` - Scan commands (DONE)
- `CLI-BUILDX-30-001` - Buildx integration (DONE)
- `CLI-ADVISE-48-001` - Advisory AI commands (IN PROGRESS)
- `CLI-SCAN-49-001` - Filesystem scanning (TODO)
---
## 14. Success Metrics
| Metric | Target |
|--------|--------|
| Startup latency | < 20ms p99 |
| CI adoption | 80% of pipelines use CLI |
| Exit code coverage | 100% of failure modes |
| Shell completion coverage | 100% of commands |
| Offline operation success | Works without network |
---
*Last updated: 2025-11-29*

476
docs/product-advisories/29-Nov-2025 - Concelier Advisory Ingestion Model.md Normal file
View File

@@ -0,0 +1,476 @@
# Concelier Advisory Ingestion Model
**Version:** 1.0
**Date:** 2025-11-29
**Status:** Canonical
This advisory defines the product rationale, ingestion semantics, and implementation strategy for the Concelier module, covering the Link-Not-Merge model, connector pipelines, observation storage, and deterministic exports.
---
## 1. Executive Summary
Concelier is the **advisory ingestion engine** that acquires, normalizes, and correlates vulnerability advisories from authoritative sources. Key capabilities:
- **Aggregation-Only Contract** - No derived semantics in ingestion
- **Link-Not-Merge** - Observations correlated, never merged
- **Multi-Source Connectors** - Vendor PSIRTs, distros, OSS ecosystems
- **Deterministic Exports** - Reproducible JSON, Trivy DB bundles
- **Conflict Detection** - Structured payloads for divergent claims
---
## 2. Market Drivers
### 2.1 Target Segments
| Segment | Ingestion Requirements | Use Case |
|---------|------------------------|----------|
| **Security Teams** | Authoritative data | Accurate vulnerability assessment |
| **Compliance** | Provenance tracking | Audit trail for advisory sources |
| **DevSecOps** | Fast updates | CI/CD pipeline integration |
| **Air-Gap Ops** | Offline bundles | Disconnected environment support |
### 2.2 Competitive Positioning
Most vulnerability databases merge data, losing provenance. Stella Ops differentiates with:
- **Link-Not-Merge** preserving all source claims
- **Conflict visibility** showing where sources disagree
- **Deterministic exports** enabling reproducible builds
- **Multi-format support** (CSAF, OSV, GHSA, vendor-specific)
- **Signature verification** for upstream integrity
---
## 3. Aggregation-Only Contract (AOC)
### 3.1 Core Principles
The AOC ensures ingestion purity:
1. **No derived semantics** - No severity consensus, merged status, or fix hints
2. **Immutable raw docs** - Append-only with version chains
3. **Mandatory provenance** - Source, timestamp, signature status
4. **Linkset only** - Joins stored separately, never mutate content
5. **Deterministic canonicalization** - Stable JSON output
6. **Idempotent upserts** - Same hash = no new record
7. **CI verification** - AOCVerifier enforces at runtime
### 3.2 Enforcement
```csharp
// AOCWriteGuard checks before every write
public class AOCWriteGuard
{
Task GuardAsync(AdvisoryObservation obs)
{
// Verify no forbidden properties
// Validate provenance completeness
// Check tenant claims
// Normalize timestamps
// Compute content hash
}
}
```
Roslyn analyzers (`StellaOps.AOC.Analyzers`) scan connectors at build time to prevent forbidden property usage.
---
## 4. Advisory Observation Model
### 4.1 Observation Structure
```json
{
"_id": "tenant:vendor:upstreamId:revision",
"tenant": "acme-corp",
"source": {
"vendor": "OSV",
"stream": "github",
"api": "https://api.osv.dev/v1/.../GHSA-...",
"collectorVersion": "concelier/1.7.3"
},
"upstream": {
"upstreamId": "GHSA-xxxx-....",
"documentVersion": "2025-09-01T12:13:14Z",
"fetchedAt": "2025-09-01T13:04:05Z",
"receivedAt": "2025-09-01T13:04:06Z",
"contentHash": "sha256:...",
"signature": {
"present": true,
"format": "dsse",
"keyId": "rekor:.../key/abc"
}
},
"content": {
"format": "OSV",
"specVersion": "1.6",
"raw": { /* unmodified upstream document */ }
},
"identifiers": {
"primary": "GHSA-xxxx-....",
"aliases": ["CVE-2025-12345", "GHSA-xxxx-...."]
},
"linkset": {
"purls": ["pkg:npm/lodash@4.17.21"],
"cpes": ["cpe:2.3:a:lodash:lodash:4.17.21:*:*:*:*:*:*:*"],
"references": [
{"type": "advisory", "url": "https://..."},
{"type": "fix", "url": "https://..."}
]
},
"supersedes": "tenant:vendor:upstreamId:prev-revision",
"createdAt": "2025-09-01T13:04:06Z"
}
```
### 4.2 Linkset Correlation
```json
{
"_id": "sha256:...",
"tenant": "acme-corp",
"key": {
"vulnerabilityId": "CVE-2025-12345",
"productKey": "pkg:npm/lodash@4.17.21",
"confidence": "high"
},
"observations": [
{
"observationId": "tenant:osv:GHSA-...:v1",
"sourceVendor": "OSV",
"statement": { "severity": "high" },
"collectedAt": "2025-09-01T13:04:06Z"
},
{
"observationId": "tenant:nvd:CVE-2025-12345:v2",
"sourceVendor": "NVD",
"statement": { "severity": "critical" },
"collectedAt": "2025-09-01T14:00:00Z"
}
],
"conflicts": [
{
"conflictId": "sha256:...",
"type": "severity-mismatch",
"observations": [
{ "source": "OSV", "value": "high" },
{ "source": "NVD", "value": "critical" }
],
"confidence": "medium",
"detectedAt": "2025-09-01T14:00:01Z"
}
]
}
```
---
## 5. Source Connectors
### 5.1 Source Families
| Family | Examples | Format |
|--------|----------|--------|
| **Vendor PSIRTs** | Microsoft, Oracle, Cisco, Adobe | CSAF, proprietary |
| **Linux Distros** | Red Hat, SUSE, Ubuntu, Debian, Alpine | CSAF, JSON, XML |
| **OSS Ecosystems** | OSV, GHSA, npm, PyPI, Maven | OSV, GraphQL |
| **CERTs** | CISA (KEV), JVN, CERT-FR | JSON, XML |
### 5.2 Connector Contract
```csharp
public interface IFeedConnector
{
string SourceName { get; }
// Fetch signed feeds or offline mirrors
Task FetchAsync(IServiceProvider sp, CancellationToken ct);
// Normalize to strongly-typed DTOs
Task ParseAsync(IServiceProvider sp, CancellationToken ct);
// Build canonical records with provenance
Task MapAsync(IServiceProvider sp, CancellationToken ct);
}
```
### 5.3 Connector Lifecycle
1. **Snapshot** - Fetch with cursor, ETag, rate limiting
2. **Parse** - Schema validation, normalization
3. **Guard** - AOCWriteGuard enforcement
4. **Write** - Append-only insert
5. **Event** - Emit `advisory.observation.updated`
---
## 6. Version Semantics
### 6.1 Ecosystem Normalization
| Ecosystem | Format | Normalization |
|-----------|--------|---------------|
| npm, PyPI, Maven | SemVer | Intervals with `<`, `>=`, `~`, `^` |
| RPM | EVR | `epoch:version-release` with order keys |
| DEB | dpkg | Version comparison with order keys |
| APK | Alpine | Computed order keys |
### 6.2 CVSS Handling
- Normalize CVSS v2/v3/v4 where available
- Track all source CVSS values
- Effective severity = max (configurable)
- Store KEV evidence with source and date
---
## 7. Conflict Detection
### 7.1 Conflict Types
| Type | Description | Resolution |
|------|-------------|------------|
| `severity-mismatch` | Different severity ratings | Policy decides |
| `affected-range-divergence` | Different version ranges | Most specific wins |
| `reference-clash` | Contradictory references | Surface all |
| `alias-inconsistency` | Different alias mappings | Union with provenance |
| `metadata-gap` | Missing information | Flag for review |
### 7.2 Conflict Visibility
Conflicts are never hidden - they are:
- Stored in linkset documents
- Surfaced in API responses
- Included in exports
- Displayed in Console UI
---
## 8. Deterministic Exports
### 8.1 JSON Export
```
exports/json/
├── CVE/
│ ├── 20/
│ │ └── CVE-2025-12345.json
│ └── ...
├── manifest.json
└── export-digest.sha256
```
- Deterministic folder structure
- Canonical JSON (sorted keys, stable timestamps)
- Manifest with SHA-256 per file
- Reproducible across runs
### 8.2 Trivy DB Export
```
exports/trivy/
├── db.tar.gz
├── metadata.json
└── manifest.json
```
- Bolt DB compatible with Trivy
- Full and delta modes
- ORAS push to registries
- Mirror manifests for domains
### 8.3 Export Determinism
Running the same export against the same data must produce:
- Identical file contents
- Identical manifest hashes
- Identical export digests
---
## 9. Implementation Strategy
### 9.1 Phase 1: Core Pipeline (Complete)
- [x] AOCWriteGuard implementation
- [x] Observation storage
- [x] Basic connectors (Red Hat, SUSE, OSV)
- [x] JSON export
### 9.2 Phase 2: Link-Not-Merge (Complete)
- [x] Linkset correlation engine
- [x] Conflict detection
- [x] Event emission
- [x] API surface
### 9.3 Phase 3: Expanded Sources (In Progress)
- [x] GHSA GraphQL connector
- [x] Debian DSA connector
- [ ] Alpine secdb connector (CONCELIER-CONN-50-001)
- [ ] CISA KEV enrichment (CONCELIER-KEV-51-001)
### 9.4 Phase 4: Export Enhancements (Planned)
- [ ] Delta Trivy DB exports
- [ ] ORAS registry push
- [ ] Attestation hand-off
- [ ] Mirror bundle signing
---
## 10. API Surface
### 10.1 Sources & Jobs
| Endpoint | Method | Scope | Description |
|----------|--------|-------|-------------|
| `/api/v1/concelier/sources` | GET | `concelier.read` | List sources |
| `/api/v1/concelier/sources/{name}/trigger` | POST | `concelier.admin` | Trigger fetch |
| `/api/v1/concelier/sources/{name}/pause` | POST | `concelier.admin` | Pause source |
| `/api/v1/concelier/jobs/{id}` | GET | `concelier.read` | Job status |
### 10.2 Exports
| Endpoint | Method | Scope | Description |
|----------|--------|-------|-------------|
| `/api/v1/concelier/exports/json` | POST | `concelier.export` | Trigger JSON export |
| `/api/v1/concelier/exports/trivy` | POST | `concelier.export` | Trigger Trivy export |
| `/api/v1/concelier/exports/{id}` | GET | `concelier.read` | Export status |
### 10.3 Search
| Endpoint | Method | Scope | Description |
|----------|--------|-------|-------------|
| `/api/v1/concelier/advisories/{key}` | GET | `concelier.read` | Get advisory |
| `/api/v1/concelier/observations/{id}` | GET | `concelier.read` | Get observation |
| `/api/v1/concelier/linksets` | GET | `concelier.read` | Query linksets |
---
## 11. Storage Model
### 11.1 Collections
| Collection | Purpose | Key Indexes |
|------------|---------|-------------|
| `sources` | Connector catalog | `{_id}` |
| `source_state` | Run state | `{sourceName}` |
| `documents` | Raw payloads | `{sourceName, uri}` |
| `advisory_observations` | Normalized records | `{tenant, upstream.upstreamId}` |
| `advisory_linksets` | Correlations | `{tenant, key.vulnerabilityId, key.productKey}` |
| `advisory_events` | Change log | `{type, occurredAt}` |
| `export_state` | Export cursors | `{exportKind}` |
### 11.2 GridFS Buckets
- `fs.documents` - Raw payloads (immutable)
- `fs.exports` - Historical archives
---
## 12. Event Model
### 12.1 Events
| Event | Trigger | Content |
|-------|---------|---------|
| `advisory.observation.updated@1` | New/superseded observation | IDs, hash, supersedes |
| `advisory.linkset.updated@1` | Correlation change | Deltas, conflicts |
### 12.2 Event Transport
- Primary: NATS
- Fallback: Redis Stream
- Offline Kit captures for replay
---
## 13. Observability
### 13.1 Metrics
- `concelier.fetch.docs_total{source}`
- `concelier.fetch.bytes_total{source}`
- `concelier.parse.failures_total{source}`
- `concelier.observations.write_total{result}`
- `concelier.linksets.updated_total{result}`
- `concelier.linksets.conflicts_total{type}`
- `concelier.export.duration_seconds{kind}`
### 13.2 Performance Targets
| Operation | Target |
|-----------|--------|
| Ingest throughput | 5k docs/min |
| Observation write | < 5ms p95 |
| Linkset build | < 15ms p95 |
| Export (1M advisories) | < 90 seconds |
---
## 14. Security Considerations
### 14.1 Outbound Security
- Allowlist per connector (domains, protocols)
- Proxy support with TLS pinning
- Rate limiting per source
### 14.2 Signature Verification
- PGP/cosign/x509 verification stored
- Failed verification flagged, not rejected
- Policy can down-weight unsigned sources
### 14.3 Determinism
- Canonical JSON writer
- Stable export digests
- Reproducible across runs
---
## 15. Related Documentation
| Resource | Location |
|----------|----------|
| Concelier architecture | `docs/modules/concelier/architecture.md` |
| Link-Not-Merge schema | `docs/modules/concelier/link-not-merge-schema.md` |
| Event schemas | `docs/modules/concelier/events/` |
| Attestation guide | `docs/modules/concelier/attestation.md` |
---
## 16. Sprint Mapping
- **Primary Sprint:** SPRINT_0115_0001_0004_concelier_iv.md
- **Related Sprints:**
- SPRINT_0113_0001_0002_concelier_ii.md
- SPRINT_0114_0001_0003_concelier_iii.md
**Key Task IDs:**
- `CONCELIER-AOC-40-001` - AOC enforcement (DONE)
- `CONCELIER-LNM-41-001` - Link-Not-Merge (DONE)
- `CONCELIER-CONN-50-001` - Alpine connector (IN PROGRESS)
- `CONCELIER-KEV-51-001` - KEV enrichment (TODO)
- `CONCELIER-EXPORT-55-001` - Delta exports (TODO)
---
## 17. Success Metrics
| Metric | Target |
|--------|--------|
| Advisory freshness | < 1 hour from source |
| Ingestion accuracy | 100% provenance retention |
| Export determinism | 100% hash reproducibility |
| Conflict detection | 100% of source divergence |
| Source coverage | 20+ authoritative sources |
---
*Last updated: 2025-11-29*

449
docs/product-advisories/29-Nov-2025 - Export Center and Reporting Strategy.md Normal file
View File

@@ -0,0 +1,449 @@
# Export Center and Reporting Strategy
**Version:** 1.0
**Date:** 2025-11-29
**Status:** Canonical
This advisory defines the product rationale, profile system, and implementation strategy for the Export Center module, covering bundle generation, adapter architecture, distribution channels, and compliance reporting.
---
## 1. Executive Summary
The Export Center is the **dedicated service layer for packaging reproducible evidence bundles**. Key capabilities:
- **Profile-Based Exports** - 6+ profile types (JSON, Trivy, Mirror, DevPortal)
- **Deterministic Bundles** - Bit-for-bit reproducible outputs with DSSE signatures
- **Multi-Format Adapters** - Pluggable adapters for different consumer needs
- **Distribution Channels** - HTTP download, OCI push, object storage
- **Compliance Ready** - Provenance, signatures, audit trails for SOC 2/FedRAMP
---
## 2. Market Drivers
### 2.1 Target Segments
| Segment | Export Requirements | Use Case |
|---------|---------------------|----------|
| **Compliance Teams** | Signed bundles, provenance | Audit evidence |
| **Security Vendors** | Trivy DB format | Scanner integration |
| **Air-Gap Operators** | Offline mirrors | Disconnected environments |
| **Development Teams** | JSON exports | CI/CD integration |
### 2.2 Competitive Positioning
Most vulnerability platforms offer basic CSV/JSON exports. Stella Ops differentiates with:
- **Reproducible bundles** with cryptographic verification
- **Multi-format adapters** (Trivy, CycloneDX, SPDX, custom)
- **OCI distribution** for container-native workflows
- **Provenance attestations** meeting SLSA Level 2+
- **Delta exports** for bandwidth-efficient updates
---
## 3. Profile System
### 3.1 Built-in Profiles
| Profile | Variant | Description | Output Format |
|---------|---------|-------------|---------------|
| **JSON** | `raw` | Unprocessed advisory/VEX data | `.jsonl.zst` |
| **JSON** | `policy` | Policy-evaluated findings | `.jsonl.zst` |
| **Trivy** | `db` | Trivy vulnerability database | SQLite |
| **Trivy** | `java-db` | Trivy Java advisory database | SQLite |
| **Mirror** | `full` | Complete offline mirror | Filesystem tree |
| **Mirror** | `delta` | Incremental updates | Filesystem tree |
| **DevPortal** | `offline` | Developer portal assets | Archive |
### 3.2 Profile Configuration
```yaml
apiVersion: stellaops.io/export.v1
kind: ExportProfile
metadata:
name: compliance-report-monthly
tenant: acme-corp
spec:
kind: json
variant: policy
schedule: "0 0 1 * *" # Monthly
selectors:
tenants: ["acme-corp"]
timeWindow: "30d"
severities: ["critical", "high"]
ecosystems: ["npm", "maven", "pypi"]
options:
compression: zstd
encryption:
enabled: true
recipients: ["age1..."]
signing:
enabled: true
keyRef: "kms://acme-corp/export-signing-key"
distribution:
- type: http
retention: 90d
- type: oci
registry: "registry.acme.com/exports"
repository: "compliance-reports"
```
### 3.3 Selector Expressions
| Selector | Description | Example |
|----------|-------------|---------|
| `tenants` | Tenant filter | `["acme-*"]` |
| `timeWindow` | Time range | `"30d"`, `"2025-01-01/2025-12-31"` |
| `products` | Product PURLs | `["pkg:npm/*", "pkg:maven/org.apache/*"]` |
| `severities` | Severity filter | `["critical", "high"]` |
| `ecosystems` | Package ecosystems | `["npm", "maven"]` |
| `policyVersions` | Policy snapshot IDs | `["rev-42", "rev-43"]` |
---
## 4. Adapter Architecture
### 4.1 Adapter Contract
```csharp
public interface IExportAdapter
{
string Kind { get; } // "json" | "trivy" | "mirror"
string Variant { get; } // "raw" | "policy" | "db"
Task<ExportResult> RunAsync(
ExportContext context,
IAsyncEnumerable<ExportRecord> records,
CancellationToken ct);
}
```
### 4.2 JSON Adapter
**Responsibilities:**
- Canonical JSON serialization (sorted keys, RFC3339 UTC)
- Linkset preservation for traceability
- Zstandard compression
- AOC guardrails (no derived modifications to raw fields)
**Output:**
```
export/
├── advisories.jsonl.zst
├── vex-statements.jsonl.zst
├── findings.jsonl.zst (policy variant)
└── manifest.json
```
### 4.3 Trivy Adapter
**Responsibilities:**
- Map Stella Ops advisory schema to Trivy DB format
- Handle namespace collisions across ecosystems
- Validate against supported Trivy schema versions
- Generate severity distribution summary
**Compatibility:**
- Trivy DB schema v2 (current)
- Fail-fast on unsupported schema versions
### 4.4 Mirror Adapter
**Responsibilities:**
- Build self-contained filesystem layout
- Delta comparison against base manifest
- Optional encryption of `/data` subtree
- OCI layer generation
**Layout:**
```
mirror/
├── manifests/
│ ├── advisories.manifest.json
│ └── vex.manifest.json
├── data/
│ ├── raw/
│ │ ├── advisories/
│ │ └── vex/
│ └── policy/
│ └── findings/
├── indexes/
│ └── by-cve.index
└── manifest.json
```
---
## 5. Bundle Structure
### 5.1 Export Manifest
```json
{
"version": "1.0.0",
"exportId": "export-20251129-001",
"profile": {
"kind": "json",
"variant": "policy",
"name": "compliance-report-monthly"
},
"tenant": "acme-corp",
"generatedAt": "2025-11-29T12:00:00Z",
"generatedBy": "export-center-worker-1",
"selectors": {
"timeWindow": "2025-11-01/2025-11-30",
"severities": ["critical", "high"]
},
"contents": [
{
"path": "findings.jsonl.zst",
"size": 1048576,
"digest": "sha256:abc123...",
"recordCount": 45230
}
],
"totals": {
"advisories": 45230,
"vexStatements": 12450,
"findings": 8920
}
}
```
### 5.2 Provenance Attestation
```json
{
"predicateType": "https://slsa.dev/provenance/v1",
"subject": [
{
"name": "export-20251129-001.tar.gz",
"digest": { "sha256": "def456..." }
}
],
"predicate": {
"buildDefinition": {
"buildType": "https://stellaops.io/export/v1",
"externalParameters": {
"profile": "compliance-report-monthly",
"selectors": { "...": "..." }
}
},
"runDetails": {
"builder": {
"id": "https://stellaops.io/export-center",
"version": "1.2.3"
},
"metadata": {
"invocationId": "export-run-123",
"startedOn": "2025-11-29T12:00:00Z",
"finishedOn": "2025-11-29T12:05:00Z"
}
}
}
}
```
---
## 6. Distribution Channels
### 6.1 HTTP Download
```bash
# Download bundle
curl -H "Authorization: Bearer $TOKEN" \
"https://export.stellaops.io/api/export/runs/{id}/download" \
-o export-bundle.tar.gz
# Verify signature
cosign verify-blob --key export-key.pub \
--signature export-bundle.sig \
export-bundle.tar.gz
```
**Features:**
- Chunked transfer encoding
- Range request support (resumable)
- `X-Export-Digest` header
- Optional encryption metadata
### 6.2 OCI Push
```bash
# Pull from registry
oras pull registry.example.com/exports/compliance:2025-11
# Verify annotations
oras manifest fetch registry.example.com/exports/compliance:2025-11 | jq
```
**Annotations:**
- `io.stellaops.export.profile`
- `io.stellaops.export.tenant`
- `io.stellaops.export.manifest-digest`
- `io.stellaops.export.provenance-ref`
### 6.3 Object Storage
```yaml
distribution:
- type: object
provider: s3
bucket: stella-exports
prefix: "${tenant}/${exportId}"
retention: 365d
immutable: true
```
---
## 7. Implementation Strategy
### 7.1 Phase 1: Core Infrastructure (Complete)
- [x] Profile CRUD APIs
- [x] JSON adapter (raw, policy)
- [x] HTTP download distribution
- [x] Manifest generation
### 7.2 Phase 2: Trivy Integration (Complete)
- [x] Trivy DB adapter
- [x] Trivy Java DB adapter
- [x] Schema version validation
- [x] Compatibility testing
### 7.3 Phase 3: Mirror & Distribution (In Progress)
- [x] Mirror full adapter
- [x] Mirror delta adapter
- [ ] OCI push distribution (EXPORT-OCI-45-001)
- [ ] DevPortal adapter (EXPORT-DEV-46-001)
### 7.4 Phase 4: Advanced Features (Planned)
- [ ] Encryption at rest
- [ ] Scheduled exports
- [ ] Retention policies
- [ ] Cross-tenant exports (with approval)
---
## 8. API Surface
### 8.1 Profile Management
| Endpoint | Method | Scope | Description |
|----------|--------|-------|-------------|
| `/api/export/profiles` | GET | `export:read` | List profiles |
| `/api/export/profiles` | POST | `export:profile:manage` | Create profile |
| `/api/export/profiles/{id}` | PATCH | `export:profile:manage` | Update profile |
| `/api/export/profiles/{id}` | DELETE | `export:profile:manage` | Delete profile |
### 8.2 Export Runs
| Endpoint | Method | Scope | Description |
|----------|--------|-------|-------------|
| `/api/export/runs` | POST | `export:run` | Start export |
| `/api/export/runs/{id}` | GET | `export:read` | Get status |
| `/api/export/runs/{id}/events` | SSE | `export:read` | Stream progress |
| `/api/export/runs/{id}/cancel` | POST | `export:run` | Cancel export |
### 8.3 Downloads
| Endpoint | Method | Scope | Description |
|----------|--------|-------|-------------|
| `/api/export/runs/{id}/download` | GET | `export:download` | Download bundle |
| `/api/export/runs/{id}/manifest` | GET | `export:read` | Get manifest |
| `/api/export/runs/{id}/provenance` | GET | `export:read` | Get provenance |
---
## 9. Observability
### 9.1 Metrics
- `exporter_run_duration_seconds{profile,tenant}`
- `exporter_run_bytes_total{profile}`
- `exporter_run_failures_total{error_code}`
- `exporter_active_runs{tenant}`
- `exporter_distribution_push_seconds{type}`
### 9.2 Logs
Structured fields:
- `run_id`, `tenant`, `profile_kind`, `adapter`
- `phase` (plan, resolve, adapter, manifest, sign, distribute)
- `correlation_id`, `error_code`
---
## 10. Security Considerations
### 10.1 Access Control
- Tenant claim enforced at every query
- Cross-tenant selectors rejected (unless approved)
- RBAC scopes: `export:profile:manage`, `export:run`, `export:read`, `export:download`
### 10.2 Encryption
- Optional encryption per profile
- Keys derived from Authority-managed KMS
- Mirror encryption uses tenant-specific recipients
- Transport security (TLS) always required
### 10.3 Signing
- Cosign-compatible signatures
- SLSA Level 2 attestations by default
- Detached signatures stored alongside manifests
---
## 11. Related Documentation
| Resource | Location |
|----------|----------|
| Export Center architecture | `docs/modules/export-center/architecture.md` |
| Profile definitions | `docs/modules/export-center/profiles.md` |
| API reference | `docs/modules/export-center/api.md` |
| DevPortal bundle spec | `docs/modules/export-center/devportal-offline.md` |
---
## 12. Sprint Mapping
- **Primary Sprint:** SPRINT_0160_0001_0001_export_evidence.md
- **Related Sprints:**
- SPRINT_0161_0001_0001_evidencelocker.md
- SPRINT_0125_0001_0001_mirror.md
**Key Task IDs:**
- `EXPORT-CORE-40-001` - Profile system (DONE)
- `EXPORT-JSON-41-001` - JSON adapters (DONE)
- `EXPORT-TRIVY-42-001` - Trivy adapters (DONE)
- `EXPORT-OCI-45-001` - OCI distribution (IN PROGRESS)
- `EXPORT-DEV-46-001` - DevPortal adapter (TODO)
---
## 13. Success Metrics
| Metric | Target |
|--------|--------|
| Export reproducibility | 100% bit-identical |
| Bundle generation time | < 5 min for 100k records |
| Signature verification | 100% success rate |
| Distribution availability | 99.9% uptime |
| Retention compliance | 100% policy adherence |
---
*Last updated: 2025-11-29*

407
docs/product-advisories/29-Nov-2025 - Findings Ledger and Immutable Audit Trail.md Normal file
View File

@@ -0,0 +1,407 @@
# Findings Ledger and Immutable Audit Trail
**Version:** 1.0
**Date:** 2025-11-29
**Status:** Canonical
This advisory defines the product rationale, ledger semantics, and implementation strategy for the Findings Ledger module, covering append-only events, Merkle anchoring, projections, and deterministic exports.
---
## 1. Executive Summary
The Findings Ledger provides **immutable, auditable records** of all vulnerability findings and their state transitions. Key capabilities:
- **Append-Only Events** - Every finding change recorded permanently
- **Merkle Anchoring** - Cryptographic proof of event ordering
- **Projections** - Materialized current state views
- **Deterministic Exports** - Reproducible compliance archives
- **Chain Integrity** - Hash-linked event sequences per tenant
---
## 2. Market Drivers
### 2.1 Target Segments
| Segment | Ledger Requirements | Use Case |
|---------|---------------------|----------|
| **Compliance** | Immutable audit trail | SOC 2, FedRAMP evidence |
| **Security Teams** | Finding history | Investigation timelines |
| **Legal/eDiscovery** | Tamper-proof records | Litigation support |
| **Auditors** | Verifiable exports | Third-party attestation |
### 2.2 Competitive Positioning
Most vulnerability tools provide mutable databases. Stella Ops differentiates with:
- **Append-only architecture** ensuring no record deletion
- **Merkle trees** for cryptographic verification
- **Chain integrity** with hash-linked events
- **Deterministic exports** for reproducible audits
- **Air-gap support** with signed bundles
---
## 3. Event Model
### 3.1 Ledger Event Structure
```json
{
"id": "uuid",
"type": "finding.status.changed",
"tenant": "acme-corp",
"chainId": "chain-uuid",
"sequence": 12345,
"policyVersion": "sha256:abc...",
"finding": {
"id": "artifact:sha256:...|pkg:npm/lodash",
"artifactId": "sha256:...",
"vulnId": "CVE-2025-12345"
},
"actor": {
"id": "user:jane@acme.com",
"type": "human"
},
"occurredAt": "2025-11-29T12:00:00Z",
"recordedAt": "2025-11-29T12:00:01Z",
"payload": {
"previousStatus": "open",
"newStatus": "triaged",
"reason": "Under investigation"
},
"evidenceBundleRef": "bundle://tenant/2025/11/29/...",
"eventHash": "sha256:...",
"previousHash": "sha256:...",
"merkleLeafHash": "sha256:..."
}
```
### 3.2 Event Types
| Type | Trigger | Payload |
|------|---------|---------|
| `finding.discovered` | New finding | severity, purl, advisory |
| `finding.status.changed` | State transition | old/new status, reason |
| `finding.verdict.changed` | Policy decision | verdict, rules matched |
| `finding.vex.applied` | VEX override | status, justification |
| `finding.assigned` | Owner change | assignee, team |
| `finding.commented` | Annotation | comment text (redacted) |
| `finding.resolved` | Resolution | resolution type, version |
### 3.3 Chain Semantics
- Each tenant has one or more event chains
- Events are strictly ordered by sequence number
- `previousHash` links to prior event for integrity
- Chain forks are prohibited (409 on conflict)
---
## 4. Merkle Anchoring
### 4.1 Tree Structure
```
Root Hash
/ \
Hash(A+B) Hash(C+D)
/ \ / \
H(E1) H(E2) H(E3) H(E4)
| | | |
Event1 Event2 Event3 Event4
```
### 4.2 Anchoring Process
1. **Batch collection** - Events accumulate in windows (default 15 min)
2. **Tree construction** - Leaves are event hashes
3. **Root computation** - Merkle root represents batch
4. **Anchor record** - Root stored with timestamp
5. **Optional external** - Root can be published to external ledger
### 4.3 Configuration
```yaml
findings:
ledger:
merkle:
batchSize: 1000
windowDuration: 00:15:00
algorithm: sha256
externalAnchor:
enabled: false
type: rekor # or custom
```
---
## 5. Projections
### 5.1 Purpose
Projections provide **current state** views derived from event history. They are:
- Materialized for fast queries
- Reconstructible from events
- Validated via `cycleHash`
### 5.2 Finding Projection
```json
{
"tenantId": "acme-corp",
"findingId": "artifact:sha256:...|pkg:npm/lodash@4.17.20",
"policyVersion": "sha256:5f38c...",
"status": "triaged",
"severity": 6.7,
"riskScore": 85.2,
"riskSeverity": "high",
"riskProfileVersion": "v2.1",
"labels": {
"kev": true,
"runtime": "exposed"
},
"currentEventId": "uuid",
"cycleHash": "sha256:...",
"policyRationale": [
"explain://tenant/findings/...",
"policy://tenant/policy-v1/rationale/accepted"
],
"updatedAt": "2025-11-29T12:00:00Z"
}
```
### 5.3 Projection Refresh
| Trigger | Action |
|---------|--------|
| New event | Incremental update |
| Policy change | Full recalculation |
| Manual request | On-demand rebuild |
| Scheduled | Periodic validation |
---
## 6. Export Capabilities
### 6.1 Export Shapes
| Shape | Description | Use Case |
|-------|-------------|----------|
| `canonical` | Full event detail | Complete audit |
| `compact` | Summary fields only | Quick reports |
### 6.2 Export Types
**Findings Export:**
```json
{
"eventSequence": 12345,
"observedAt": "2025-11-29T12:00:00Z",
"findingId": "artifact:...|pkg:...",
"policyVersion": "sha256:...",
"status": "triaged",
"severity": 6.7,
"cycleHash": "sha256:...",
"evidenceBundleRef": "bundle://...",
"provenance": {
"policyVersion": "sha256:...",
"cycleHash": "sha256:...",
"ledgerEventHash": "sha256:..."
}
}
```
### 6.3 Export Formats
- **JSON** - Paged API responses
- **NDJSON** - Streaming exports
- **Bundle** - Signed archive packages
---
## 7. Implementation Strategy
### 7.1 Phase 1: Core Ledger (Complete)
- [x] Append-only event store
- [x] Hash-linked chains
- [x] Basic projection engine
- [x] REST API surface
### 7.2 Phase 2: Merkle & Exports (In Progress)
- [x] Merkle tree construction
- [x] Batch anchoring
- [ ] External anchor integration (LEDGER-MERKLE-50-001)
- [ ] Deterministic NDJSON exports (LEDGER-EXPORT-51-001)
### 7.3 Phase 3: Advanced Features (Planned)
- [ ] Chain integrity verification CLI
- [ ] Projection replay tooling
- [ ] Cross-tenant federation
- [ ] Long-term archival
---
## 8. API Surface
### 8.1 Events
| Endpoint | Method | Scope | Description |
|----------|--------|-------|-------------|
| `/v1/ledger/events` | GET | `vuln:audit` | List ledger events |
| `/v1/ledger/events` | POST | `vuln:operate` | Append event |
### 8.2 Projections
| Endpoint | Method | Scope | Description |
|----------|--------|-------|-------------|
| `/v1/ledger/projections/findings` | GET | `vuln:view` | List projections |
### 8.3 Exports
| Endpoint | Method | Scope | Description |
|----------|--------|-------|-------------|
| `/v1/ledger/export/findings` | GET | `vuln:audit` | Export findings |
| `/v1/ledger/export/vex` | GET | `vuln:audit` | Export VEX |
| `/v1/ledger/export/advisories` | GET | `vuln:audit` | Export advisories |
| `/v1/ledger/export/sboms` | GET | `vuln:audit` | Export SBOMs |
### 8.4 Attestations
| Endpoint | Method | Scope | Description |
|----------|--------|-------|-------------|
| `/v1/ledger/attestations` | GET | `vuln:audit` | List verifications |
---
## 9. Storage Model
### 9.1 Collections
| Collection | Purpose | Key Indexes |
|------------|---------|-------------|
| `ledger_events` | Append-only events | `{tenant, chainId, sequence}` |
| `ledger_chains` | Chain metadata | `{tenant, chainId}` |
| `ledger_merkle_roots` | Anchor records | `{tenant, batchId, anchoredAt}` |
| `finding_projections` | Current state | `{tenant, findingId}` |
### 9.2 Integrity Constraints
- Events are append-only (no update/delete)
- Sequence numbers strictly monotonic
- Hash chain validated on write
- Merkle roots immutable
---
## 10. Observability
### 10.1 Metrics
- `ledger.events.appended_total{tenant,type}`
- `ledger.events.rejected_total{reason}`
- `ledger.merkle.batches_total`
- `ledger.merkle.anchor_latency_seconds`
- `ledger.projection.updates_total`
- `ledger.projection.staleness_seconds`
- `ledger.export.rows_total{type,shape}`
### 10.2 SLO Targets
| Metric | Target |
|--------|--------|
| Event append latency | < 50ms p95 |
| Projection freshness | < 5 seconds |
| Merkle anchor window | 15 minutes |
| Export throughput | 10k rows/sec |
---
## 11. Security Considerations
### 11.1 Immutability Guarantees
- No UPDATE/DELETE operations exposed
- Admin override requires audit event
- Merkle roots provide tamper evidence
- External anchoring for non-repudiation
### 11.2 Access Control
- `vuln:view` - Read projections
- `vuln:investigate` - Triage actions
- `vuln:operate` - State transitions
- `vuln:audit` - Export and verify
### 11.3 Data Protection
- Sensitive payloads redacted in exports
- Comment text hashed, not stored
- PII filtered at ingest
- Tenant isolation enforced
---
## 12. Air-Gap Support
### 12.1 Offline Bundles
- Signed NDJSON exports
- Merkle proofs included
- Time anchors from trusted source
- Bundle verification CLI
### 12.2 Staleness Tracking
```yaml
airgap:
staleness:
warningThresholdDays: 7
blockThresholdDays: 30
riskCriticalExportsBlocked: true
```
---
## 13. Related Documentation
| Resource | Location |
|----------|----------|
| Ledger schema | `docs/modules/findings-ledger/schema.md` |
| OpenAPI spec | `docs/modules/findings-ledger/openapi/` |
| Export guide | `docs/modules/findings-ledger/exports.md` |
---
## 14. Sprint Mapping
- **Primary Sprint:** SPRINT_0186_0001_0001_record_deterministic_execution.md
- **Related Sprints:**
- SPRINT_0120_0000_0001_policy_reasoning.md
- SPRINT_311_docs_tasks_md_xi.md
**Key Task IDs:**
- `LEDGER-CORE-40-001` - Event store (DONE)
- `LEDGER-PROJ-41-001` - Projections (DONE)
- `LEDGER-MERKLE-50-001` - Merkle anchoring (IN PROGRESS)
- `LEDGER-EXPORT-51-001` - Deterministic exports (IN PROGRESS)
- `LEDGER-AIRGAP-56-001` - Bundle provenance (TODO)
---
## 15. Success Metrics
| Metric | Target |
|--------|--------|
| Event durability | 100% (no data loss) |
| Chain integrity | 100% hash verification |
| Projection accuracy | 100% event replay match |
| Export determinism | 100% hash reproducibility |
| Audit compliance | SOC 2 Type II |
---
*Last updated: 2025-11-29*

331
docs/product-advisories/29-Nov-2025 - Graph Analytics and Dependency Insights.md Normal file
View File

@@ -0,0 +1,331 @@
# Graph Analytics and Dependency Insights
**Version:** 1.0
**Date:** 2025-11-29
**Status:** Canonical
This advisory defines the product rationale, graph model, and implementation strategy for the Graph module, covering dependency analysis, impact visualization, and offline exports.
---
## 1. Executive Summary
The Graph module provides **dependency analysis and impact visualization** across the vulnerability landscape. Key capabilities:
- **Unified Graph Model** - Artifacts, components, advisories, policies linked
- **Impact Analysis** - Blast radius, affected paths, transitive dependencies
- **Policy Overlays** - VEX and policy decisions visualized on graph
- **Analytics** - Clustering, centrality, community detection
- **Offline Export** - Deterministic graph snapshots for air-gap
---
## 2. Market Drivers
### 2.1 Target Segments
| Segment | Graph Requirements | Use Case |
|---------|-------------------|----------|
| **Security Teams** | Impact analysis | Vulnerability prioritization |
| **Developers** | Dependency visualization | Upgrade planning |
| **Compliance** | Audit trails | Relationship documentation |
| **Management** | Risk dashboards | Portfolio risk view |
### 2.2 Competitive Positioning
Most vulnerability tools show flat lists. Stella Ops differentiates with:
- **Graph-native architecture** linking all entities
- **Impact visualization** showing blast radius
- **Policy overlays** embedding decisions in graph
- **Offline-compatible** exports for air-gap analysis
- **Analytics** for community detection and centrality
---
## 3. Graph Model
### 3.1 Node Types
| Node | Description | Key Properties |
|------|-------------|----------------|
| **Artifact** | Image/application digest | tenant, environment, labels |
| **Component** | Package version | purl, ecosystem, version |
| **File** | Source/binary path | hash, mtime |
| **License** | License identifier | spdx-id, restrictions |
| **Advisory** | Vulnerability record | cve-id, severity, sources |
| **VEXStatement** | VEX decision | status, justification |
| **PolicyVersion** | Signed policy pack | version, digest |
### 3.2 Edge Types
| Edge | From | To | Properties |
|------|------|-----|------------|
| `DEPENDS_ON` | Component | Component | scope, optional |
| `BUILT_FROM` | Artifact | Component | layer, path |
| `DECLARED_IN` | Component | File | sbom-id |
| `AFFECTED_BY` | Component | Advisory | version-range |
| `VEX_EXEMPTS` | VEXStatement | Advisory | justification |
| `GOVERNS_WITH` | PolicyVersion | Artifact | run-id |
| `OBSERVED_RUNTIME` | Artifact | Component | zastava-event-id |
### 3.3 Provenance
Every edge carries:
- `createdAt` - UTC timestamp
- `sourceDigest` - SRM/SBOM hash
- `provenanceRef` - Link to source document
---
## 4. Overlay System
### 4.1 Overlay Types
| Overlay | Purpose | Content |
|---------|---------|---------|
| `policy.overlay.v1` | Policy decisions | verdict, severity, rules |
| `openvex.v1` | VEX status | status, justification |
| `reachability.v1` | Runtime reachability | state, confidence |
| `clustering.v1` | Community detection | cluster-id, modularity |
| `centrality.v1` | Node importance | degree, betweenness |
### 4.2 Overlay Structure
```json
{
"overlayId": "sha256(tenant|nodeId|overlayKind)",
"overlayKind": "policy.overlay.v1",
"nodeId": "component:pkg:npm/lodash@4.17.21",
"tenant": "acme-corp",
"generatedAt": "2025-11-29T12:00:00Z",
"content": {
"verdict": "blocked",
"severity": "critical",
"rulesMatched": ["rule-001", "rule-002"],
"explainTrace": "sampled trace data..."
}
}
```
---
## 5. Query Capabilities
### 5.1 Search API
```bash
POST /graph/search
{
"tenant": "acme-corp",
"query": "severity:critical AND ecosystem:npm",
"nodeTypes": ["Component", "Advisory"],
"limit": 100
}
```
### 5.2 Path Query
```bash
POST /graph/paths
{
"source": "artifact:sha256:abc123...",
"target": "advisory:CVE-2025-12345",
"maxDepth": 6,
"includeOverlays": true
}
```
**Response:**
```json
{
"paths": [
{
"nodes": ["artifact:sha256:...", "component:pkg:npm/...", "advisory:CVE-..."],
"edges": [{"type": "BUILT_FROM"}, {"type": "AFFECTED_BY"}],
"length": 2
}
],
"overlays": [
{"nodeId": "component:...", "overlayKind": "policy.overlay.v1", "content": {...}}
]
}
```
### 5.3 Diff Query
```bash
POST /graph/diff
{
"snapshotA": "snapshot-2025-11-28",
"snapshotB": "snapshot-2025-11-29",
"includeOverlays": true
}
```
---
## 6. Analytics Pipeline
### 6.1 Clustering
- **Algorithm:** Louvain community detection
- **Output:** Cluster IDs per node, modularity score
- **Use Case:** Identify tightly coupled component groups
### 6.2 Centrality
- **Degree centrality:** Most connected nodes
- **Betweenness centrality:** Critical path nodes
- **Use Case:** Identify high-impact components
### 6.3 Background Processing
```yaml
analytics:
enabled: true
schedule: "0 */6 * * *" # Every 6 hours
algorithms:
- clustering
- centrality
snapshotRetention: 30
```
---
## 7. Implementation Strategy
### 7.1 Phase 1: Core Model (Complete)
- [x] Node/edge schema
- [x] SBOM ingestion pipeline
- [x] Advisory/VEX linking
- [x] Basic search API
### 7.2 Phase 2: Overlays (In Progress)
- [x] Policy overlay generation
- [x] VEX overlay generation
- [ ] Reachability overlay (GRAPH-REACH-50-001)
- [ ] Inline overlay in query responses (GRAPH-QUERY-51-001)
### 7.3 Phase 3: Analytics (Planned)
- [ ] Clustering algorithm
- [ ] Centrality calculations
- [ ] Background worker
- [ ] Analytics overlays export
### 7.4 Phase 4: Visualization (Planned)
- [ ] Console graph viewer
- [ ] Impact tree visualization
- [ ] Diff visualization
---
## 8. API Surface
### 8.1 Core APIs
| Endpoint | Method | Scope | Description |
|----------|--------|-------|-------------|
| `/graph/search` | POST | `graph:read` | Search nodes |
| `/graph/query` | POST | `graph:read` | Complex queries |
| `/graph/paths` | POST | `graph:read` | Path finding |
| `/graph/diff` | POST | `graph:read` | Snapshot diff |
| `/graph/nodes/{id}` | GET | `graph:read` | Node detail |
### 8.2 Export APIs
| Endpoint | Method | Scope | Description |
|----------|--------|-------|-------------|
| `/graph/export` | POST | `graph:export` | Start export job |
| `/graph/export/{jobId}` | GET | `graph:read` | Job status |
| `/graph/export/{jobId}/download` | GET | `graph:export` | Download bundle |
---
## 9. Storage Model
### 9.1 Collections
| Collection | Purpose | Key Indexes |
|------------|---------|-------------|
| `graph_nodes` | Node records | `{tenant, nodeType, nodeId}` |
| `graph_edges` | Edge records | `{tenant, fromId, toId, edgeType}` |
| `graph_overlays` | Overlay data | `{tenant, nodeId, overlayKind}` |
| `graph_snapshots` | Point-in-time snapshots | `{tenant, snapshotId}` |
### 9.2 Export Format
```
graph-export/
├── nodes.jsonl # Sorted by nodeId
├── edges.jsonl # Sorted by (from, to, type)
├── overlays/
│ ├── policy.jsonl
│ ├── openvex.jsonl
│ └── manifest.json
└── manifest.json
```
---
## 10. Observability
### 10.1 Metrics
- `graph_ingest_lag_seconds`
- `graph_nodes_total{nodeType}`
- `graph_edges_total{edgeType}`
- `graph_query_latency_seconds{queryType}`
- `graph_analytics_runs_total`
- `graph_analytics_clusters_total`
### 10.2 Offline Support
- Graph snapshots packaged for Offline Kit
- Deterministic NDJSON exports
- Overlay manifests with digests
---
## 11. Related Documentation
| Resource | Location |
|----------|----------|
| Graph architecture | `docs/modules/graph/architecture.md` |
| Query language | `docs/modules/graph/query-language.md` |
| Overlay specification | `docs/modules/graph/overlays.md` |
---
## 12. Sprint Mapping
- **Primary Sprint:** SPRINT_0141_0001_0001_graph_indexer.md
- **Related Sprints:**
- SPRINT_0401_0001_0001_reachability_evidence_chain.md
- SPRINT_0140_0001_0001_runtime_signals.md
**Key Task IDs:**
- `GRAPH-CORE-40-001` - Core model (DONE)
- `GRAPH-INGEST-41-001` - SBOM ingestion (DONE)
- `GRAPH-REACH-50-001` - Reachability overlay (IN PROGRESS)
- `GRAPH-ANALYTICS-55-001` - Clustering (TODO)
- `GRAPH-VIZ-60-001` - Visualization (FUTURE)
---
## 13. Success Metrics
| Metric | Target |
|--------|--------|
| Query latency | < 500ms p95 |
| Ingestion lag | < 5 minutes |
| Path query depth | Up to 6 hops |
| Export reproducibility | 100% deterministic |
| Analytics freshness | < 6 hours |
---
*Last updated: 2025-11-29*

469
docs/product-advisories/29-Nov-2025 - Notification Rules and Alerting Engine.md Normal file
View File

@@ -0,0 +1,469 @@
# Notification Rules and Alerting Engine
**Version:** 1.0
**Date:** 2025-11-29
**Status:** Canonical
This advisory defines the product rationale, rules engine semantics, and implementation strategy for the Notify module, covering channel connectors, throttling, digests, and delivery management.
---
## 1. Executive Summary
The Notify module provides **rules-driven, tenant-aware notification delivery** across security workflows. Key capabilities:
- **Rules Engine** - Declarative matchers for event routing
- **Multi-Channel Delivery** - Slack, Teams, Email, Webhooks
- **Noise Control** - Throttling, deduplication, digest windows
- **Approval Tokens** - DSSE-signed ack tokens for one-click workflows
- **Audit Trail** - Complete delivery history with redacted payloads
---
## 2. Market Drivers
### 2.1 Target Segments
| Segment | Notification Requirements | Use Case |
|---------|--------------------------|----------|
| **Security Teams** | Real-time critical alerts | Incident response |
| **DevSecOps** | CI/CD integration | Pipeline notifications |
| **Compliance** | Audit trails | Delivery verification |
| **Management** | Digest summaries | Executive reporting |
### 2.2 Competitive Positioning
Most vulnerability tools offer basic email alerts. Stella Ops differentiates with:
- **Rules-based routing** with fine-grained matchers
- **Native Slack/Teams integration** with rich formatting
- **Digest windows** to prevent alert fatigue
- **Cryptographic ack tokens** for approval workflows
- **Tenant isolation** with quota controls
---
## 3. Rules Engine
### 3.1 Rule Structure
```yaml
name: "critical-alerts-prod"
enabled: true
tenant: "acme-corp"
match:
eventKinds:
- "scanner.report.ready"
- "scheduler.rescan.delta"
- "zastava.admission"
namespaces: ["prod-*"]
repos: ["ghcr.io/acme/*"]
minSeverity: "high"
kev: true
verdict: ["fail", "deny"]
vex:
includeRejectedJustifications: false
actions:
- channel: "slack:sec-alerts"
template: "concise"
throttle: "5m"
- channel: "email:soc"
digest: "hourly"
template: "detailed"
```
### 3.2 Matcher Types
| Matcher | Description | Example |
|---------|-------------|---------|
| `eventKinds` | Event type filter | `["scanner.report.ready"]` |
| `namespaces` | Namespace patterns | `["prod-*", "staging"]` |
| `repos` | Repository patterns | `["ghcr.io/acme/*"]` |
| `minSeverity` | Minimum severity | `"high"` |
| `kev` | KEV-tagged required | `true` |
| `verdict` | Report/admission verdict | `["fail", "deny"]` |
| `labels` | Kubernetes labels | `{"env": "production"}` |
### 3.3 Evaluation Order
1. **Tenant check** - Discard if rule tenant ≠ event tenant
2. **Kind filter** - Early discard for non-matching kinds
3. **Scope match** - Namespace/repo/label matching
4. **Delta gates** - Severity threshold evaluation
5. **VEX gate** - Filter based on VEX status
6. **Throttle/dedup** - Idempotency key check
7. **Actions** - Enqueue per-channel jobs
---
## 4. Channel Connectors
### 4.1 Built-in Channels
| Channel | Features | Rate Limits |
|---------|----------|-------------|
| **Slack** | Blocks, threads, reactions | 1 msg/sec per channel |
| **Teams** | Adaptive Cards, webhooks | 4 msgs/sec |
| **Email** | HTML+text, attachments | Relay-dependent |
| **Webhook** | JSON, HMAC signing | 10 req/sec |
### 4.2 Channel Configuration
```yaml
channels:
- name: "slack:sec-alerts"
type: slack
config:
channel: "#security-alerts"
workspace: "acme-corp"
secretRef: "ref://notify/slack-token"
- name: "email:soc"
type: email
config:
to: ["soc@acme.com"]
from: "stellaops@acme.com"
smtpHost: "smtp.acme.com"
secretRef: "ref://notify/smtp-creds"
- name: "webhook:siem"
type: webhook
config:
url: "https://siem.acme.com/api/events"
signMethod: "ed25519"
signKeyRef: "ref://notify/webhook-key"
```
### 4.3 Connector Contract
```csharp
public interface INotifyConnector
{
string Type { get; }
Task<DeliveryResult> SendAsync(DeliveryContext ctx, CancellationToken ct);
Task<HealthResult> HealthAsync(ChannelConfig cfg, CancellationToken ct);
}
```
---
## 5. Noise Control
### 5.1 Throttling
- **Per-action throttle** - Suppress duplicates within window
- **Idempotency key** - `hash(ruleId | actionId | event.kind | scope.digest | day)`
- **Configurable windows** - 5m, 15m, 1h, 1d
### 5.2 Digest Windows
```yaml
actions:
- channel: "email:weekly-summary"
digest: "weekly"
digestOptions:
maxItems: 100
groupBy: ["severity", "namespace"]
template: "digest-summary"
```
**Behavior:**
- Coalesce events within window
- Summarize top N items with counts
- Flush on window close or max items
- Safe truncation with "and X more" links
### 5.3 Quiet Hours
```yaml
notify:
quietHours:
enabled: true
window: "22:00-06:00"
timezone: "America/New_York"
minSeverity: "critical"
```
Only critical alerts during quiet hours; others deferred to digests.
---
## 6. Templates & Rendering
### 6.1 Template Engine
- Handlebars-style safe templates
- No arbitrary code execution
- Deterministic outputs (stable property order)
- Locale-aware formatting
### 6.2 Template Variables
| Variable | Description |
|----------|-------------|
| `event.kind` | Event type |
| `event.ts` | Timestamp |
| `scope.namespace` | Kubernetes namespace |
| `scope.repo` | Repository |
| `scope.digest` | Image digest |
| `payload.verdict` | Policy verdict |
| `payload.delta.newCritical` | New critical count |
| `payload.links.ui` | UI deep link |
| `topFindings[]` | Top N findings |
### 6.3 Channel-Specific Rendering
**Slack:**
```json
{
"blocks": [
{"type": "header", "text": {"type": "plain_text", "text": "Policy FAIL: nginx:latest"}},
{"type": "section", "text": {"type": "mrkdwn", "text": "*2 critical*, 3 high vulnerabilities"}}
]
}
```
**Email:**
```html
<h2>Policy FAIL: nginx:latest</h2>
<table>
<tr><td>Critical</td><td>2</td></tr>
<tr><td>High</td><td>3</td></tr>
</table>
<a href="https://ui.internal/reports/...">View Details</a>
```
---
## 7. Ack Tokens
### 7.1 Token Structure
DSSE-signed tokens for one-click acknowledgements:
```json
{
"payloadType": "application/vnd.stellaops.notify-ack-token+json",
"payload": {
"tenant": "acme-corp",
"deliveryId": "delivery-123",
"notificationId": "notif-456",
"channel": "slack:sec-alerts",
"webhookUrl": "https://notify.internal/ack",
"nonce": "random-nonce",
"actions": ["acknowledge", "escalate"],
"expiresAt": "2025-11-29T13:00:00Z"
},
"signatures": [{"keyid": "notify-ack-key-01", "sig": "..."}]
}
```
### 7.2 Token Workflow
1. **Issue** - `POST /notify/ack-tokens/issue`
2. **Embed** - Token included in message action button
3. **Click** - User clicks button, token sent to webhook
4. **Verify** - `POST /notify/ack-tokens/verify`
5. **Audit** - Ack event recorded
### 7.3 Token Rotation
```bash
# Rotate ack token signing key
stella notify rotate-ack-key --key-source kms://notify/ack-key
```
---
## 8. Implementation Strategy
### 8.1 Phase 1: Core Engine (Complete)
- [x] Rules engine with matchers
- [x] Slack connector
- [x] Teams connector
- [x] Email connector
- [x] Webhook connector
### 8.2 Phase 2: Noise Control (Complete)
- [x] Throttling
- [x] Digest windows
- [x] Idempotency
- [x] Quiet hours
### 8.3 Phase 3: Ack Tokens (In Progress)
- [x] Token issuance
- [x] Token verification
- [ ] Token rotation API (NOTIFY-ACK-45-001)
- [ ] Escalation workflows (NOTIFY-ESC-46-001)
### 8.4 Phase 4: Advanced Features (Planned)
- [ ] PagerDuty connector
- [ ] Jira ticket creation
- [ ] In-app notifications
- [ ] Anomaly suppression
---
## 9. API Surface
### 9.1 Channels
| Endpoint | Method | Scope | Description |
|----------|--------|-------|-------------|
| `/api/v1/notify/channels` | GET/POST | `notify.read/admin` | List/create channels |
| `/api/v1/notify/channels/{id}` | GET/PATCH/DELETE | `notify.admin` | Manage channel |
| `/api/v1/notify/channels/{id}/test` | POST | `notify.admin` | Send test message |
| `/api/v1/notify/channels/{id}/health` | GET | `notify.read` | Health check |
### 9.2 Rules
| Endpoint | Method | Scope | Description |
|----------|--------|-------|-------------|
| `/api/v1/notify/rules` | GET/POST | `notify.read/admin` | List/create rules |
| `/api/v1/notify/rules/{id}` | GET/PATCH/DELETE | `notify.admin` | Manage rule |
| `/api/v1/notify/rules/{id}/test` | POST | `notify.admin` | Dry-run rule |
### 9.3 Deliveries
| Endpoint | Method | Scope | Description |
|----------|--------|-------|-------------|
| `/api/v1/notify/deliveries` | GET | `notify.read` | List deliveries |
| `/api/v1/notify/deliveries/{id}` | GET | `notify.read` | Delivery detail |
| `/api/v1/notify/deliveries/{id}/retry` | POST | `notify.admin` | Retry delivery |
---
## 10. Event Sources
### 10.1 Subscribed Events
| Event | Source | Typical Actions |
|-------|--------|-----------------|
| `scanner.scan.completed` | Scanner | Immediate/digest |
| `scanner.report.ready` | Scanner | Immediate |
| `scheduler.rescan.delta` | Scheduler | Immediate/digest |
| `attestor.logged` | Attestor | Immediate |
| `zastava.admission` | Zastava | Immediate |
| `conselier.export.completed` | Concelier | Digest |
| `excitor.export.completed` | Excititor | Digest |
### 10.2 Event Envelope
```json
{
"eventId": "uuid",
"kind": "scanner.report.ready",
"tenant": "acme-corp",
"ts": "2025-11-29T12:00:00Z",
"actor": "scanner-webservice",
"scope": {
"namespace": "production",
"repo": "ghcr.io/acme/api",
"digest": "sha256:..."
},
"payload": {
"reportId": "report-123",
"verdict": "fail",
"summary": {"total": 12, "blocked": 2},
"delta": {"newCritical": 1, "kev": ["CVE-2025-..."]}
}
}
```
---
## 11. Observability
### 11.1 Metrics
- `notify.events_consumed_total{kind}`
- `notify.rules_matched_total{ruleId}`
- `notify.throttled_total{reason}`
- `notify.digest_coalesced_total{window}`
- `notify.sent_total{channel}`
- `notify.failed_total{channel,code}`
- `notify.delivery_latency_seconds{channel}`
### 11.2 SLO Targets
| Metric | Target |
|--------|--------|
| Event-to-delivery p95 | < 60 seconds |
| Failure rate | < 0.5% per hour |
| Duplicate rate | ~0% |
---
## 12. Security Considerations
### 12.1 Secret Management
- Secrets stored as references only
- Just-in-time fetch at send time
- No plaintext in Mongo
### 12.2 Webhook Signing
```
X-StellaOps-Signature: t=1732881600,v1=abc123...
X-StellaOps-Timestamp: 2025-11-29T12:00:00Z
```
- HMAC-SHA256 or Ed25519
- Replay window protection
- Canonical body hash
### 12.3 Loop Prevention
- Webhook target allowlist
- Event origin tags
- Own webhooks rejected
---
## 13. Related Documentation
| Resource | Location |
|----------|----------|
| Notify architecture | `docs/modules/notify/architecture.md` |
| Channel schemas | `docs/modules/notify/resources/schemas/` |
| Sample payloads | `docs/modules/notify/resources/samples/` |
| Bootstrap pack | `docs/modules/notify/bootstrap-pack.md` |
---
## 14. Sprint Mapping
- **Primary Sprint:** SPRINT_0170_0001_0001_notify_engine.md (NEW)
- **Related Sprints:**
- SPRINT_0171_0001_0002_notify_connectors.md
- SPRINT_0172_0001_0003_notify_ack_tokens.md
**Key Task IDs:**
- `NOTIFY-ENGINE-40-001` - Rules engine (DONE)
- `NOTIFY-CONN-41-001` - Connectors (DONE)
- `NOTIFY-NOISE-42-001` - Throttling/digests (DONE)
- `NOTIFY-ACK-45-001` - Token rotation (IN PROGRESS)
- `NOTIFY-ESC-46-001` - Escalation workflows (TODO)
---
## 15. Success Metrics
| Metric | Target |
|--------|--------|
| Delivery latency | < 60s p95 |
| Delivery success rate | > 99.5% |
| Duplicate rate | < 0.01% |
| Rule evaluation time | < 10ms |
| Channel health | 99.9% uptime |
---
*Last updated: 2025-11-29*

432
docs/product-advisories/29-Nov-2025 - Orchestrator Event Model and Job Lifecycle.md Normal file
View File

@@ -0,0 +1,432 @@
# Orchestrator Event Model and Job Lifecycle
**Version:** 1.0
**Date:** 2025-11-29
**Status:** Canonical
This advisory defines the product rationale, job lifecycle semantics, and implementation strategy for the Orchestrator module, covering event models, quota governance, replay semantics, and TaskRunner bridge.
---
## 1. Executive Summary
The Orchestrator is the **central job coordination layer** for all Stella Ops asynchronous operations. Key capabilities:
- **Unified Job Lifecycle** - Enqueue, schedule, lease, complete with audit trail
- **Quota Governance** - Per-tenant rate limits, burst controls, circuit breakers
- **Replay Semantics** - Deterministic job replay for audit and recovery
- **TaskRunner Bridge** - Pack-run integration with heartbeats and artifacts
- **Event Fan-Out** - SSE/GraphQL feeds for dashboards and notifications
- **Offline Export** - Audit bundles for compliance and investigations
---
## 2. Market Drivers
### 2.1 Target Segments
| Segment | Orchestration Requirements | Use Case |
|---------|---------------------------|----------|
| **Enterprise** | Rate limiting, quota management | Multi-team resource sharing |
| **MSP/MSSP** | Multi-tenant isolation | Managed security services |
| **Compliance Teams** | Audit trails, replay | SOC 2, FedRAMP evidence |
| **DevSecOps** | CI/CD integration, webhooks | Pipeline automation |
### 2.2 Competitive Positioning
Most vulnerability platforms lack sophisticated job orchestration. Stella Ops differentiates with:
- **Deterministic replay** for audit and debugging
- **Fine-grained quotas** per tenant/job-type
- **Circuit breakers** for automatic failure isolation
- **Native pack-run integration** for workflow automation
- **Offline-compatible** audit bundles
---
## 3. Job Lifecycle Model
### 3.1 State Machine
```
[Created] --> [Queued] --> [Leased] --> [Running] --> [Completed]
| | | |
| | v v
| +-------> [Failed] <----[Canceled]
| |
v v
[Throttled] [Incident]
```
### 3.2 Lifecycle Phases
| Phase | Description | Transitions |
|-------|-------------|-------------|
| **Created** | Job request received | -> Queued |
| **Queued** | Awaiting scheduling | -> Leased, Throttled |
| **Throttled** | Rate limit applied | -> Queued (after delay) |
| **Leased** | Worker acquired job | -> Running, Expired |
| **Running** | Active execution | -> Completed, Failed, Canceled |
| **Completed** | Success, archived | Terminal |
| **Failed** | Error, may retry | -> Queued (retry), Incident |
| **Canceled** | Operator abort | Terminal |
| **Incident** | Escalated failure | Terminal (requires operator) |
### 3.3 Job Request Structure
```json
{
"jobId": "uuid",
"jobType": "scan|policy-run|export|pack-run|advisory-sync",
"tenant": "tenant-id",
"priority": "low|normal|high|emergency",
"payloadDigest": "sha256:...",
"payload": { "imageRef": "nginx:latest", "options": {} },
"dependencies": ["job-id-1", "job-id-2"],
"idempotencyKey": "unique-request-key",
"correlationId": "trace-id",
"requestedBy": "user-id|service-id",
"requestedAt": "2025-11-29T12:00:00Z"
}
```
---
## 4. Quota Governance
### 4.1 Quota Model
```yaml
quotas:
- tenant: "acme-corp"
jobType: "*"
maxActive: 50
maxPerHour: 500
burst: 10
priority:
emergency:
maxActive: 5
skipQueue: true
- tenant: "acme-corp"
jobType: "export"
maxActive: 4
maxPerHour: 100
```
### 4.2 Rate Limit Enforcement
1. **Quota Check** - Before leasing, verify tenant hasn't exceeded limits
2. **Burst Control** - Allow short bursts within configured window
3. **Staging** - Jobs exceeding limits staged with `nextEligibleAt` timestamp
4. **Priority Bypass** - Emergency jobs can skip queue (with separate limits)
### 4.3 Dynamic Controls
| Control | API | Purpose |
|---------|-----|---------|
| `pauseSource` | `POST /api/limits/pause` | Halt specific job sources |
| `resumeSource` | `POST /api/limits/resume` | Resume paused sources |
| `throttle` | `POST /api/limits/throttle` | Apply temporary throttle |
| `updateQuota` | `PATCH /api/quotas/{id}` | Modify quota limits |
### 4.4 Circuit Breakers
- Auto-pause job types when failure rate > threshold (default 50%)
- Incident events generated via Notify
- Half-open testing after cooldown period
- Manual reset via operator action
---
## 5. TaskRunner Bridge
### 5.1 Pack-Run Integration
The Orchestrator provides specialized support for TaskRunner pack executions:
```json
{
"jobType": "pack-run",
"payload": {
"packId": "vuln-scan-and-report",
"packVersion": "1.2.0",
"planHash": "sha256:...",
"inputs": { "imageRef": "nginx:latest" },
"artifacts": [],
"logChannel": "sse:/runs/{runId}/logs",
"heartbeatCadence": 30
}
}
```
### 5.2 Heartbeat Protocol
- Workers send heartbeats every `heartbeatCadence` seconds
- Missed heartbeats trigger lease expiration
- Lease can be extended for long-running tasks
- Dead workers detected within 2x heartbeat interval
### 5.3 Artifact & Log Streaming
| Endpoint | Method | Purpose |
|----------|--------|---------|
| `/runs/{runId}/logs` | SSE | Stream execution logs |
| `/runs/{runId}/artifacts` | GET | List produced artifacts |
| `/runs/{runId}/artifacts/{name}` | GET | Download artifact |
| `/runs/{runId}/heartbeat` | POST | Extend lease |
---
## 6. Event Model
### 6.1 Event Envelope
```json
{
"eventId": "uuid",
"eventType": "job.queued|job.leased|job.completed|job.failed",
"timestamp": "2025-11-29T12:00:00Z",
"tenant": "tenant-id",
"jobId": "job-id",
"jobType": "scan",
"correlationId": "trace-id",
"idempotencyKey": "unique-key",
"payload": {
"status": "completed",
"duration": 45.2,
"result": { "verdict": "pass" }
},
"provenance": {
"workerId": "worker-1",
"leaseId": "lease-id",
"taskRunnerId": "runner-1"
}
}
```
### 6.2 Event Types
| Event | Trigger | Consumers |
|-------|---------|-----------|
| `job.queued` | Job enqueued | Dashboard, Notify |
| `job.leased` | Worker acquired job | Dashboard |
| `job.started` | Execution began | Dashboard, Notify |
| `job.progress` | Progress update | Dashboard (SSE) |
| `job.completed` | Success | Dashboard, Notify, Export |
| `job.failed` | Error occurred | Dashboard, Notify, Incident |
| `job.canceled` | Operator abort | Dashboard, Notify |
| `job.replayed` | Replay initiated | Dashboard, Audit |
### 6.3 Fan-Out Channels
- **SSE** - Real-time dashboard feeds
- **GraphQL Subscriptions** - Console UI
- **Notify** - Alert routing based on rules
- **Webhooks** - External integrations
- **Audit Log** - Compliance storage
---
## 7. Replay Semantics
### 7.1 Deterministic Replay
Jobs can be replayed for audit, debugging, or recovery:
```bash
# Replay a completed job
stella job replay --id job-12345
# Replay with sealed mode (offline verification)
stella job replay --id job-12345 --sealed --bundle output.tar.gz
```
### 7.2 Replay Guarantees
| Property | Guarantee |
|----------|-----------|
| **Input preservation** | Same payloadDigest, cursors |
| **Ordering** | Same processing order |
| **Determinism** | Same outputs for same inputs |
| **Provenance** | `replayOf` pointer to original |
### 7.3 Replay Record
```json
{
"jobId": "replay-job-id",
"replayOf": "original-job-id",
"priority": "high",
"reason": "audit-verification",
"requestedBy": "auditor@example.com",
"cursors": {
"advisory": "cursor-abc",
"vex": "cursor-def"
}
}
```
---
## 8. Implementation Strategy
### 8.1 Phase 1: Core Lifecycle (Complete)
- [x] Job state machine
- [x] MongoDB queue with leasing
- [x] Basic quota enforcement
- [x] Dashboard SSE feeds
### 8.2 Phase 2: Pack-Run Bridge (In Progress)
- [x] Pack-run job type registration
- [x] Log/artifact streaming
- [ ] Heartbeat protocol (ORCH-PACK-37-001)
- [ ] Event envelope finalization (ORCH-SVC-37-101)
### 8.3 Phase 3: Advanced Controls (Planned)
- [ ] Circuit breaker automation
- [ ] Quota analytics dashboard
- [ ] Replay verification tooling
- [ ] Incident mode integration
---
## 9. API Surface
### 9.1 Job Management
| Endpoint | Method | Scope | Description |
|----------|--------|-------|-------------|
| `/api/jobs` | GET | `orch:read` | List jobs with filters |
| `/api/jobs/{id}` | GET | `orch:read` | Job detail |
| `/api/jobs/{id}/cancel` | POST | `orch:operate` | Cancel job |
| `/api/jobs/{id}/replay` | POST | `orch:operate` | Schedule replay |
### 9.2 Quota Management
| Endpoint | Method | Scope | Description |
|----------|--------|-------|-------------|
| `/api/quotas` | GET | `orch:read` | List quotas |
| `/api/quotas/{id}` | PATCH | `orch:quota` | Update quota |
| `/api/limits/throttle` | POST | `orch:quota` | Apply throttle |
| `/api/limits/pause` | POST | `orch:quota` | Pause source |
| `/api/limits/resume` | POST | `orch:quota` | Resume source |
### 9.3 Dashboard
| Endpoint | Method | Scope | Description |
|----------|--------|-------|-------------|
| `/api/dashboard/metrics` | GET | `orch:read` | Aggregated metrics |
| `/api/dashboard/events` | SSE | `orch:read` | Real-time events |
---
## 10. Storage Model
### 10.1 Collections
| Collection | Purpose | Key Fields |
|------------|---------|------------|
| `jobs` | Current job state | `_id`, `tenant`, `jobType`, `status`, `priority` |
| `job_history` | Append-only audit | `jobId`, `event`, `timestamp`, `actor` |
| `sources` | Job sources registry | `sourceId`, `tenant`, `status` |
| `quotas` | Quota definitions | `tenant`, `jobType`, `limits` |
| `throttles` | Active throttles | `tenant`, `source`, `until` |
| `incidents` | Escalated failures | `jobId`, `reason`, `status` |
### 10.2 Indexes
- `{tenant, jobType, status}` on `jobs`
- `{tenant, status, startedAt}` on `jobs`
- `{jobId, timestamp}` on `job_history`
- TTL index on transient lease records
---
## 11. Observability
### 11.1 Metrics
- `job_queue_depth{jobType,tenant}`
- `job_latency_seconds{jobType,phase}`
- `job_failures_total{jobType,reason}`
- `job_retry_total{jobType}`
- `lease_extensions_total{jobType}`
- `quota_exceeded_total{tenant}`
- `circuit_breaker_state{jobType}`
### 11.2 Pack-Run Metrics
- `pack_run_logs_stream_lag_seconds`
- `pack_run_heartbeats_total`
- `pack_run_artifacts_total`
- `pack_run_duration_seconds`
---
## 12. Offline Support
### 12.1 Audit Bundle Export
```bash
stella orch export --tenant acme-corp --since 2025-11-01 --output audit-bundle.tar.gz
```
Bundle contents:
- `jobs.jsonl` - Job records
- `history.jsonl` - State transitions
- `throttles.jsonl` - Throttle events
- `manifest.json` - Bundle metadata
- `signatures/` - DSSE signatures
### 12.2 Replay Verification
```bash
# Verify job determinism
stella job verify --bundle audit-bundle.tar.gz --job-id job-12345
```
---
## 13. Related Documentation
| Resource | Location |
|----------|----------|
| Orchestrator architecture | `docs/modules/orchestrator/architecture.md` |
| Event envelope spec | `docs/modules/orchestrator/event-envelope.md` |
| TaskRunner integration | `docs/modules/taskrunner/orchestrator-bridge.md` |
---
## 14. Sprint Mapping
- **Primary Sprint:** SPRINT_0151_0001_0001_orchestrator_i.md
- **Related Sprints:**
- SPRINT_0152_0001_0002_orchestrator_ii.md
- SPRINT_0153_0001_0003_orchestrator_iii.md
- SPRINT_0157_0001_0001_taskrunner_i.md
**Key Task IDs:**
- `ORCH-CORE-30-001` - Job lifecycle (DONE)
- `ORCH-QUOTA-31-001` - Quota governance (DONE)
- `ORCH-PACK-37-001` - Pack-run bridge (IN PROGRESS)
- `ORCH-SVC-37-101` - Event envelope (IN PROGRESS)
- `ORCH-REPLAY-38-001` - Replay verification (TODO)
---
## 15. Success Metrics
| Metric | Target |
|--------|--------|
| Job scheduling latency | < 100ms p99 |
| Lease acquisition time | < 50ms p99 |
| Event fan-out delay | < 500ms |
| Quota enforcement accuracy | 100% |
| Replay determinism | 100% match |
---
*Last updated: 2025-11-29*

394
docs/product-advisories/29-Nov-2025 - Policy Simulation and Shadow Gates.md Normal file
View File

@@ -0,0 +1,394 @@
# Policy Simulation and Shadow Gates
**Version:** 1.0
**Date:** 2025-11-29
**Status:** Canonical
This advisory defines the product rationale, simulation semantics, and implementation strategy for Policy Engine simulation features, covering shadow runs, coverage fixtures, and promotion gates.
---
## 1. Executive Summary
Policy simulation enables **safe testing of policy changes** before production deployment. Key capabilities:
- **Shadow Runs** - Execute policies without enforcement
- **Diff Summaries** - Compare old vs new policy outcomes
- **Coverage Fixtures** - Validate expected findings
- **Promotion Gates** - Block promotion until tests pass
- **Deterministic Replay** - Reproduce simulation results
---
## 2. Market Drivers
### 2.1 Target Segments
| Segment | Simulation Requirements | Use Case |
|---------|------------------------|----------|
| **Policy Authors** | Preview changes | Development workflow |
| **Security Leads** | Approve promotions | Change management |
| **Compliance** | Audit trail | Policy change evidence |
| **DevSecOps** | CI integration | Automated testing |
### 2.2 Competitive Positioning
Most vulnerability tools lack policy simulation. Stella Ops differentiates with:
- **Shadow execution** without production impact
- **Diff visualization** of policy changes
- **Coverage testing** with fixture validation
- **Promotion gates** for governance
- **Deterministic replay** for audit
---
## 3. Simulation Modes
### 3.1 Shadow Run
Execute policy against real data without enforcement:
```bash
stella policy simulate \
--policy my-policy:v2 \
--scope "tenant:acme-corp,namespace:production" \
--shadow
```
**Behavior:**
- Evaluates all findings
- Records verdicts to shadow collections
- No enforcement actions
- No notifications triggered
- Metrics tagged with `shadow=true`
### 3.2 Diff Run
Compare two policy versions:
```bash
stella policy diff \
--old my-policy:v1 \
--new my-policy:v2 \
--scope "tenant:acme-corp"
```
**Output:**
```json
{
"summary": {
"added": 12,
"removed": 5,
"changed": 8,
"unchanged": 234
},
"changes": [
{
"findingId": "finding-123",
"cve": "CVE-2025-12345",
"oldVerdict": "warned",
"newVerdict": "blocked",
"reason": "rule 'critical-cves' now matches"
}
]
}
```
### 3.3 Coverage Run
Validate policy against fixture expectations:
```bash
stella policy coverage \
--policy my-policy:v2 \
--fixtures fixtures/policy-tests.yaml
```
---
## 4. Coverage Fixtures
### 4.1 Fixture Format
```yaml
apiVersion: stellaops.io/policy-test.v1
kind: PolicyFixture
metadata:
name: critical-cve-blocking
policy: my-policy
fixtures:
- name: "Block critical CVE in production"
input:
finding:
cve: "CVE-2025-12345"
severity: critical
ecosystem: npm
component: "lodash@4.17.20"
context:
namespace: production
labels:
tier: frontend
expected:
verdict: blocked
rulesMatched: ["critical-cves", "production-strict"]
- name: "Warn on high CVE in staging"
input:
finding:
cve: "CVE-2025-12346"
severity: high
ecosystem: npm
expected:
verdict: warned
- name: "Ignore low CVE with VEX"
input:
finding:
cve: "CVE-2025-12347"
severity: low
vexStatus: not_affected
vexJustification: "component_not_present"
expected:
verdict: ignored
```
### 4.2 Fixture Results
```json
{
"total": 25,
"passed": 23,
"failed": 2,
"failures": [
{
"fixture": "Block critical CVE in production",
"expected": {"verdict": "blocked"},
"actual": {"verdict": "warned"},
"diff": "rule 'critical-cves' did not match due to missing label"
}
]
}
```
---
## 5. Promotion Gates
### 5.1 Gate Requirements
Before a policy can be promoted from draft to active:
| Gate | Requirement | Enforcement |
|------|-------------|-------------|
| Shadow Run | Complete without errors | Required |
| Coverage | 100% fixtures pass | Required |
| Diff Review | Changes reviewed | Optional |
| Approval | Human sign-off | Configurable |
### 5.2 Promotion Workflow
```mermaid
stateDiagram-v2
[*] --> Draft
Draft --> Shadow: Start shadow run
Shadow --> Coverage: Run coverage tests
Coverage --> Review: Pass fixtures
Review --> Approval: Review diff
Approval --> Active: Approve
Coverage --> Draft: Fix failures
Approval --> Draft: Reject
```
### 5.3 CLI Commands
```bash
# Start shadow run
stella policy promote start --policy my-policy:v2
# Check promotion status
stella policy promote status --policy my-policy:v2
# Complete promotion (requires approval)
stella policy promote complete --policy my-policy:v2 --comment "Reviewed and approved"
```
---
## 6. Determinism Requirements
### 6.1 Simulation Guarantees
| Property | Guarantee |
|----------|-----------|
| Input ordering | Stable sort by (tenant, policyId, findingKey) |
| Rule evaluation | First-match semantics |
| Timestamp handling | Injected TimeProvider |
| Random values | Injected IRandom |
### 6.2 Replay Hash
Each simulation computes:
```
determinismHash = SHA256(policyVersion + inputsHash + rulesHash)
```
Replays with same hash must produce identical results.
---
## 7. Implementation Strategy
### 7.1 Phase 1: Shadow Runs (Complete)
- [x] Shadow collection isolation
- [x] Shadow metrics tagging
- [x] Shadow run API
- [x] CLI integration
### 7.2 Phase 2: Diff & Coverage (In Progress)
- [x] Policy diff algorithm
- [x] Diff visualization
- [ ] Coverage fixture parser (POLICY-COV-50-001)
- [ ] Coverage runner (POLICY-COV-50-002)
### 7.3 Phase 3: Promotion Gates (Planned)
- [ ] Gate configuration schema
- [ ] Promotion state machine
- [ ] Approval workflow integration
- [ ] Console UI for review
---
## 8. API Surface
### 8.1 Simulation APIs
| Endpoint | Method | Scope | Description |
|----------|--------|-------|-------------|
| `/api/policy/simulate` | POST | `policy:simulate` | Start simulation |
| `/api/policy/simulate/{id}` | GET | `policy:read` | Get simulation status |
| `/api/policy/simulate/{id}/results` | GET | `policy:read` | Get results |
### 8.2 Diff APIs
| Endpoint | Method | Scope | Description |
|----------|--------|-------|-------------|
| `/api/policy/diff` | POST | `policy:read` | Compare versions |
### 8.3 Coverage APIs
| Endpoint | Method | Scope | Description |
|----------|--------|-------|-------------|
| `/api/policy/coverage` | POST | `policy:simulate` | Run coverage |
| `/api/policy/coverage/{id}` | GET | `policy:read` | Get results |
### 8.4 Promotion APIs
| Endpoint | Method | Scope | Description |
|----------|--------|-------|-------------|
| `/api/policy/promote` | POST | `policy:promote` | Start promotion |
| `/api/policy/promote/{id}` | GET | `policy:read` | Get status |
| `/api/policy/promote/{id}/approve` | POST | `policy:approve` | Approve promotion |
| `/api/policy/promote/{id}/reject` | POST | `policy:approve` | Reject promotion |
---
## 9. Storage Model
### 9.1 Collections
| Collection | Purpose |
|------------|---------|
| `policy_simulations` | Simulation records |
| `policy_simulation_results` | Per-finding results |
| `policy_coverage_runs` | Coverage executions |
| `policy_promotions` | Promotion state |
### 9.2 Shadow Isolation
Shadow results stored in separate collections:
- `effective_finding_{policyId}_shadow`
- Never mixed with production data
- TTL-based cleanup (default 7 days)
---
## 10. Observability
### 10.1 Metrics
- `policy_simulation_duration_seconds{mode}`
- `policy_coverage_pass_rate{policy}`
- `policy_promotion_gate_status{gate,status}`
- `policy_diff_changes_total{changeType}`
### 10.2 Audit Events
- `policy.simulation.started`
- `policy.simulation.completed`
- `policy.coverage.passed`
- `policy.coverage.failed`
- `policy.promotion.approved`
- `policy.promotion.rejected`
---
## 11. Console Integration
### 11.1 Policy Editor
- Inline simulation button
- Real-time diff preview
- Coverage status badge
### 11.2 Promotion Dashboard
- Pending promotions list
- Gate status visualization
- Approval/reject actions
---
## 12. Related Documentation
| Resource | Location |
|----------|----------|
| Policy architecture | `docs/modules/policy/architecture.md` |
| DSL reference | `docs/policy/dsl.md` |
| Lifecycle guide | `docs/policy/lifecycle.md` |
| Runtime guide | `docs/policy/runtime.md` |
---
## 13. Sprint Mapping
- **Primary Sprint:** SPRINT_0185_0001_0001_policy_simulation.md (NEW)
- **Related Sprints:**
- SPRINT_0120_0000_0001_policy_reasoning.md
- SPRINT_0121_0001_0001_policy_reasoning.md
**Key Task IDs:**
- `POLICY-SIM-40-001` - Shadow runs (DONE)
- `POLICY-DIFF-41-001` - Diff algorithm (DONE)
- `POLICY-COV-50-001` - Coverage fixtures (IN PROGRESS)
- `POLICY-COV-50-002` - Coverage runner (IN PROGRESS)
- `POLICY-PROM-55-001` - Promotion gates (TODO)
---
## 14. Success Metrics
| Metric | Target |
|--------|--------|
| Simulation latency | < 2 min (10k findings) |
| Coverage accuracy | 100% fixture matching |
| Promotion gate enforcement | 100% adherence |
| Shadow isolation | Zero production leakage |
| Replay determinism | 100% hash match |
---
*Last updated: 2025-11-29*

444
docs/product-advisories/29-Nov-2025 - Runtime Posture and Observation with Zastava.md Normal file
View File

@@ -0,0 +1,444 @@
# Runtime Posture and Observation with Zastava
**Version:** 1.0
**Date:** 2025-11-29
**Status:** Canonical
This advisory defines the product rationale, observation model, and implementation strategy for the Zastava module, covering runtime inspection, admission control, drift detection, and posture verification.
---
## 1. Executive Summary
Zastava is the **runtime inspector and enforcer** that provides ground-truth from running environments. Key capabilities:
- **Runtime Observation** - Inventory containers, track entrypoints, monitor loaded DSOs
- **Admission Control** - Kubernetes ValidatingAdmissionWebhook for pre-flight gates
- **Drift Detection** - Identify unexpected processes, libraries, and file changes
- **Posture Verification** - Validate signatures, SBOM referrers, attestations
- **Build-ID Tracking** - Correlate binaries to debug symbols and source
---
## 2. Market Drivers
### 2.1 Target Segments
| Segment | Runtime Requirements | Use Case |
|---------|---------------------|----------|
| **Enterprise Security** | Runtime visibility | Post-deploy monitoring |
| **Platform Engineering** | Admission gates | Policy enforcement |
| **Compliance Teams** | Continuous verification | Runtime attestation |
| **DevSecOps** | Drift detection | Configuration management |
### 2.2 Competitive Positioning
Most vulnerability scanners focus on build-time analysis. Stella Ops differentiates with:
- **Runtime ground-truth** from actual container execution
- **DSO tracking** - which libraries are actually loaded
- **Entrypoint tracing** - what programs actually run
- **Native Kubernetes admission** with policy integration
- **Build-ID correlation** for symbol resolution
---
## 3. Architecture Overview
### 3.1 Component Topology
**Kubernetes Deployment:**
```
stellaops/zastava-observer # DaemonSet on every node (read-only host mounts)
stellaops/zastava-webhook # ValidatingAdmissionWebhook (Deployment, 2+ replicas)
```
**Docker/VM Deployment:**
```
stellaops/zastava-agent # System service; watch Docker events; observer only
```
### 3.2 Dependencies
| Dependency | Purpose |
|------------|---------|
| Authority | OpToks (DPoP/mTLS) for API calls |
| Scanner.WebService | Event ingestion, policy decisions |
| OCI Registry | Referrer/signature checks |
| Container Runtime | containerd/CRI-O/Docker interfaces |
| Kubernetes API | Pod watching, admission webhook |
---
## 4. Runtime Event Model
### 4.1 Event Types
| Kind | Trigger | Payload |
|------|---------|---------|
| `CONTAINER_START` | Container lifecycle | Image, entrypoint, namespace |
| `CONTAINER_STOP` | Container termination | Exit code, duration |
| `DRIFT` | Unexpected change | Changed files, new binaries |
| `POLICY_VIOLATION` | Rule breach | Reason, severity |
| `ATTESTATION_STATUS` | Verification result | Signed, SBOM present |
### 4.2 Event Envelope
```json
{
"eventId": "uuid",
"when": "2025-11-29T12:00:00Z",
"kind": "CONTAINER_START",
"tenant": "acme-corp",
"node": "worker-node-01",
"runtime": {
"engine": "containerd",
"version": "1.7.19"
},
"workload": {
"platform": "kubernetes",
"namespace": "production",
"pod": "api-7c9fbbd8b7-ktd84",
"container": "api",
"containerId": "containerd://abc123...",
"imageRef": "ghcr.io/acme/api@sha256:def456...",
"owner": {
"kind": "Deployment",
"name": "api"
}
},
"process": {
"pid": 12345,
"entrypoint": ["/entrypoint.sh", "--serve"],
"entryTrace": [
{"file": "/entrypoint.sh", "line": 3, "op": "exec", "target": "/usr/bin/python3"},
{"file": "<argv>", "op": "python", "target": "/opt/app/server.py"}
],
"buildId": "9f3a1cd4c0b7adfe91c0e3b51d2f45fb0f76a4c1"
},
"loadedLibs": [
{"path": "/lib/x86_64-linux-gnu/libssl.so.3", "inode": 123456, "sha256": "..."},
{"path": "/usr/lib/x86_64-linux-gnu/libcrypto.so.3", "inode": 123457, "sha256": "..."}
],
"posture": {
"imageSigned": true,
"sbomReferrer": "present",
"attestation": {
"uuid": "rekor-uuid",
"verified": true
}
}
}
```
---
## 5. Observer Capabilities
### 5.1 Container Lifecycle Tracking
- Watch container start/stop via CRI socket
- Resolve container to image digest
- Map mount points and rootfs paths
- Track container metadata (labels, annotations)
### 5.2 Entrypoint Tracing
- Attach short-lived nsenter to container PID 1
- Parse shell scripts for exec chain
- Record terminal program (actual binary)
- Bounded depth to prevent infinite loops
### 5.3 Loaded Library Sampling
- Read `/proc/<pid>/maps` for loaded DSOs
- Compute SHA-256 for each mapped file
- Track GNU build-IDs for symbol correlation
- Rate limits prevent resource exhaustion
### 5.4 Posture Verification
- Image signature presence (cosign policies)
- SBOM referrers check (registry HEAD)
- Rekor attestation lookup via Scanner.WebService
- Policy verdict from backend
---
## 6. Admission Control
### 6.1 Gate Criteria
| Criterion | Description | Configurable |
|-----------|-------------|--------------|
| Image Signature | Cosign-verifiable to configured keys | Yes |
| SBOM Availability | CycloneDX referrer or catalog entry | Yes |
| Policy Verdict | Backend PASS required | Yes |
| Registry Allowlist | Permitted registries | Yes |
| Tag Bans | Reject `:latest`, etc. | Yes |
| Base Image Allowlist | Permitted base digests | Yes |
### 6.2 Decision Flow
```mermaid
sequenceDiagram
participant K8s as API Server
participant WH as Zastava Webhook
participant SW as Scanner.WebService
K8s->>WH: AdmissionReview(Pod)
WH->>WH: Resolve images to digests
WH->>SW: POST /policy/runtime
SW-->>WH: {signed, hasSbom, verdict, reasons}
alt All pass
WH-->>K8s: Allow
else Any fail
WH-->>K8s: Deny (with reasons)
end
```
### 6.3 Response Caching
- Per-digest results cached for TTL (default 300s)
- Fail-open or fail-closed per namespace
- Cache invalidation on policy updates
---
## 7. Drift Detection
### 7.1 Signal Types
| Signal | Detection Method | Action |
|--------|-----------------|--------|
| Process Drift | Terminal program differs from EntryTrace baseline | Alert |
| Library Drift | Loaded DSOs not in Usage SBOM | Alert, delta scan |
| Filesystem Drift | New executables with mtime after image creation | Alert |
| Network Drift | Unexpected listening ports | Alert (optional) |
### 7.2 Drift Event
```json
{
"kind": "DRIFT",
"delta": {
"baselineImageDigest": "sha256:abc...",
"changedFiles": ["/opt/app/server.py"],
"newBinaries": [
{"path": "/usr/local/bin/helper", "sha256": "..."}
]
},
"evidence": [
{"signal": "procfs.maps", "value": "/lib/.../libssl.so.3@0x7f..."},
{"signal": "cri.task.inspect", "value": "pid=12345"}
]
}
```
---
## 8. Build-ID Workflow
### 8.1 Capture
1. Observer extracts `NT_GNU_BUILD_ID` from `/proc/<pid>/exe`
2. Normalize to lower-case hex
3. Include in runtime event as `process.buildId`
### 8.2 Correlation
1. Scanner.WebService persists observation
2. Policy responses include `buildIds` list
3. Debug files matched via `.build-id/<aa>/<rest>.debug`
### 8.3 Symbol Resolution
```bash
# Via CLI
stella runtime policy test --image sha256:abc123... | jq '.buildIds'
# Via debuginfod
debuginfod-find debuginfo 9f3a1cd4c0b7adfe91c0e3b51d2f45fb0f76a4c1
```
---
## 9. Implementation Strategy
### 9.1 Phase 1: Observer Core (Complete)
- [x] CRI socket integration
- [x] Container lifecycle tracking
- [x] Entrypoint tracing
- [x] Loaded library sampling
- [x] Event batching and compression
### 9.2 Phase 2: Admission Webhook (Complete)
- [x] ValidatingAdmissionWebhook
- [x] Image digest resolution
- [x] Policy integration
- [x] Response caching
- [x] Fail-open/closed modes
### 9.3 Phase 3: Drift Detection (In Progress)
- [x] Process drift detection
- [x] Library drift detection
- [ ] Filesystem drift monitoring (ZASTAVA-DRIFT-50-001)
- [ ] Network posture checks (ZASTAVA-NET-51-001)
### 9.4 Phase 4: Advanced Features (Planned)
- [ ] eBPF syscall tracing (optional)
- [ ] Windows container support
- [ ] Live used-by-entrypoint synthesis
- [ ] Admission dry-run dashboards
---
## 10. Configuration
```yaml
zastava:
mode:
observer: true
webhook: true
backend:
baseAddress: "https://scanner-web.internal"
policyPath: "/api/v1/scanner/policy/runtime"
requestTimeoutSeconds: 5
runtime:
authority:
issuer: "https://authority.internal"
clientId: "zastava-observer"
audience: ["scanner", "zastava"]
scopes: ["api:scanner.runtime.write"]
requireDpop: true
requireMutualTls: true
tenant: "acme-corp"
engine: "auto" # containerd|cri-o|docker|auto
procfs: "/host/proc"
collect:
entryTrace: true
loadedLibs: true
maxLibs: 256
maxHashBytesPerContainer: 64000000
admission:
enforce: true
failOpenNamespaces: ["dev", "test"]
verify:
imageSignature: true
sbomReferrer: true
scannerPolicyPass: true
cacheTtlSeconds: 300
limits:
eventsPerSecond: 50
burst: 200
perNodeQueue: 10000
```
---
## 11. Security Posture
### 11.1 Privileges
| Capability | Purpose | Mode |
|------------|---------|------|
| `CAP_SYS_PTRACE` | nsenter trace | Optional |
| `CAP_DAC_READ_SEARCH` | Read /proc | Required |
| Host PID namespace | Container PIDs | Required |
| Read-only mounts | /proc, sockets | Required |
### 11.2 Least Privilege
- No write mounts
- No host networking
- No privilege escalation
- Read-only rootfs
### 11.3 Data Minimization
- No env var exfiltration
- No command argument logging (unless diagnostic mode)
- Rate limits prevent abuse
---
## 12. Observability
### 12.1 Observer Metrics
- `zastava.runtime.events.total{kind}`
- `zastava.runtime.backend.latency.ms{endpoint}`
- `zastava.proc_maps.samples.total{result}`
- `zastava.entrytrace.depth{p99}`
- `zastava.hash.bytes.total`
- `zastava.buffer.drops.total`
### 12.2 Webhook Metrics
- `zastava.admission.decisions.total{decision}`
- `zastava.admission.cache.hits.total`
- `zastava.backend.failures.total`
---
## 13. Performance Targets
| Operation | Target |
|-----------|--------|
| `/proc/<pid>/maps` sampling | < 30ms (64 files) |
| Full library hash set | < 200ms (256 libs) |
| Admission with warm cache | < 8ms p95 |
| Admission with backend call | < 50ms p95 |
| Event throughput | 5k events/min/node |
---
## 14. Related Documentation
| Resource | Location |
|----------|----------|
| Zastava architecture | `docs/modules/zastava/architecture.md` |
| Runtime event schema | `docs/modules/zastava/event-schema.md` |
| Admission configuration | `docs/modules/zastava/admission-config.md` |
| Deployment guide | `docs/modules/zastava/deployment.md` |
---
## 15. Sprint Mapping
- **Primary Sprint:** SPRINT_0144_0001_0001_zastava_runtime_signals.md
- **Related Sprints:**
- SPRINT_0140_0001_0001_runtime_signals.md
- SPRINT_0143_0000_0001_signals.md
**Key Task IDs:**
- `ZASTAVA-OBS-40-001` - Observer core (DONE)
- `ZASTAVA-ADM-41-001` - Admission webhook (DONE)
- `ZASTAVA-DRIFT-50-001` - Filesystem drift (IN PROGRESS)
- `ZASTAVA-NET-51-001` - Network posture (TODO)
- `ZASTAVA-EBPF-60-001` - eBPF integration (FUTURE)
---
## 16. Success Metrics
| Metric | Target |
|--------|--------|
| Event capture rate | 99.9% of container starts |
| Admission latency | < 50ms p95 |
| Drift detection rate | 100% of runtime changes |
| False positive rate | < 1% of drift alerts |
| Node resource usage | < 2% CPU, < 100MB RAM |
---
*Last updated: 2025-11-29*

373
docs/product-advisories/29-Nov-2025 - Telemetry and Observability Patterns.md Normal file
View File

@@ -0,0 +1,373 @@
# Telemetry and Observability Patterns
**Version:** 1.0
**Date:** 2025-11-29
**Status:** Canonical
This advisory defines the product rationale, collector topology, and implementation strategy for the Telemetry module, covering metrics, traces, logs, forensic pipelines, and offline packaging.
---
## 1. Executive Summary
The Telemetry module provides **unified observability infrastructure** across all Stella Ops components. Key capabilities:
- **OpenTelemetry Native** - OTLP collection for metrics, traces, logs
- **Forensic Mode** - Extended retention and 100% sampling during incidents
- **Profile-Based Configuration** - Default, forensic, and air-gap profiles
- **Sealed-Mode Guards** - Automatic exporter restrictions in air-gap
- **Offline Bundles** - Signed OTLP archives for compliance
---
## 2. Market Drivers
### 2.1 Target Segments
| Segment | Observability Requirements | Use Case |
|---------|---------------------------|----------|
| **Platform Ops** | Real-time monitoring | Operational health |
| **Security Teams** | Forensic investigation | Incident response |
| **Compliance** | Audit trails | SOC 2, FedRAMP |
| **DevSecOps** | Pipeline visibility | CI/CD debugging |
### 2.2 Competitive Positioning
Most vulnerability tools provide minimal observability. Stella Ops differentiates with:
- **Built-in OpenTelemetry** across all services
- **Forensic mode** with automatic retention extension
- **Sealed-mode compatibility** for air-gap
- **Signed OTLP bundles** for compliance archives
- **Incident-triggered sampling** escalation
---
## 3. Collector Topology
### 3.1 Architecture
```
┌─────────────────────────────────────────────────────┐
│ Services │
│ Scanner │ Policy │ Authority │ Orchestrator │ ... │
└─────────────────────┬───────────────────────────────┘
│ OTLP/gRPC
▼
┌─────────────────────────────────────────────────────┐
│ OpenTelemetry Collector │
│ ┌─────────┐ ┌──────────┐ ┌─────────────────────┐ │
│ │ Traces │ │ Metrics │ │ Logs │ │
│ └────┬────┘ └────┬─────┘ └──────────┬──────────┘ │
│ │ Tail │ Batch │ Redaction │
│ │ Sampling │ │ │
└───────┼────────────┼─────────────────┼─────────────┘
│ │ │
▼ ▼ ▼
┌────────┐ ┌──────────┐ ┌────────┐
│ Tempo │ │Prometheus│ │ Loki │
└────────┘ └──────────┘ └────────┘
```
### 3.2 Collector Profiles
| Profile | Use Case | Configuration |
|---------|----------|---------------|
| **default** | Normal operation | 10% trace sampling, 30-day retention |
| **forensic** | Investigation mode | 100% sampling, 180-day retention |
| **airgap** | Offline deployment | File exporters, no external network |
---
## 4. Metrics
### 4.1 Standard Metrics
| Metric | Type | Labels | Description |
|--------|------|--------|-------------|
| `stellaops_request_duration_seconds` | Histogram | service, endpoint | Request latency |
| `stellaops_request_total` | Counter | service, status | Request count |
| `stellaops_active_jobs` | Gauge | tenant, jobType | Active job count |
| `stellaops_queue_depth` | Gauge | queue | Queue depth |
| `stellaops_scan_duration_seconds` | Histogram | tenant | Scan duration |
### 4.2 Module-Specific Metrics
**Policy Engine:**
- `policy_run_seconds{mode,tenant,policy}`
- `policy_rules_fired_total{policy,rule}`
- `policy_vex_overrides_total{policy,vendor}`
**Scanner:**
- `scanner_sbom_components_total{ecosystem}`
- `scanner_vulnerabilities_found_total{severity}`
- `scanner_attestations_logged_total`
**Authority:**
- `authority_token_issued_total{grant_type,audience}`
- `authority_token_rejected_total{reason}`
- `authority_dpop_nonce_miss_total`
---
## 5. Traces
### 5.1 Trace Context
All services propagate W3C Trace Context:
- `traceparent` header
- `tracestate` for vendor-specific data
- `baggage` for cross-service attributes
### 5.2 Span Conventions
| Span | Attributes | Description |
|------|------------|-------------|
| `http.request` | url, method, status | HTTP handler |
| `db.query` | collection, operation | MongoDB ops |
| `policy.evaluate` | policyId, version | Policy run |
| `scan.image` | imageRef, digest | Image scan |
| `sign.dsse` | predicateType | DSSE signing |
### 5.3 Sampling Strategy
**Default (Tail Sampling):**
- Error traces: 100%
- Slow traces (>2s): 100%
- Normal traces: 10%
**Forensic Mode:**
- All traces: 100%
- Extended attributes enabled
---
## 6. Logs
### 6.1 Structured Format
```json
{
"timestamp": "2025-11-29T12:00:00.123Z",
"level": "info",
"message": "Scan completed",
"service": "scanner",
"traceId": "abc123...",
"spanId": "def456...",
"tenant": "acme-corp",
"imageDigest": "sha256:...",
"componentCount": 245,
"vulnerabilityCount": 12
}
```
### 6.2 Redaction
Attribute processors strip sensitive data:
- `authorization` headers
- `secretRef` values
- PII based on allowed-key policy
### 6.3 Log Levels
| Level | Purpose | Retention |
|-------|---------|-----------|
| `error` | Failures | 180 days |
| `warn` | Anomalies | 90 days |
| `info` | Operations | 30 days |
| `debug` | Development | 7 days |
---
## 7. Forensic Mode
### 7.1 Activation
```bash
# Activate forensic mode for tenant
stella telemetry incident start --tenant acme-corp --reason "CVE-2025-12345 investigation"
# Check status
stella telemetry incident status
# Deactivate
stella telemetry incident stop --tenant acme-corp
```
### 7.2 Behavior Changes
| Aspect | Default | Forensic |
|--------|---------|----------|
| Trace sampling | 10% | 100% |
| Log level | info | debug |
| Retention | 30 days | 180 days |
| Attributes | Standard | Extended |
| Export frequency | 1 minute | 10 seconds |
### 7.3 Automatic Triggers
- Orchestrator incident escalation
- Policy violation threshold exceeded
- Circuit breaker activation
- Manual operator trigger
---
## 8. Implementation Strategy
### 8.1 Phase 1: Core Telemetry (Complete)
- [x] OpenTelemetry SDK integration
- [x] Metrics exporter (Prometheus)
- [x] Trace exporter (Tempo/Jaeger)
- [x] Log exporter (Loki)
### 8.2 Phase 2: Advanced Features (Complete)
- [x] Tail sampling configuration
- [x] Attribute redaction
- [x] Profile-based configuration
- [x] Dashboard provisioning
### 8.3 Phase 3: Forensic & Offline (In Progress)
- [x] Forensic mode toggle
- [ ] Forensic bundle export (TELEM-FOR-50-001)
- [ ] Sealed-mode guards (TELEM-SEAL-51-001)
- [ ] Offline bundle signing (TELEM-SIGN-52-001)
---
## 9. API Surface
### 9.1 Configuration
| Endpoint | Method | Scope | Description |
|----------|--------|-------|-------------|
| `/telemetry/config/profile/{name}` | GET | `telemetry:read` | Download collector config |
| `/telemetry/config/profiles` | GET | `telemetry:read` | List profiles |
### 9.2 Incident Mode
| Endpoint | Method | Scope | Description |
|----------|--------|-------|-------------|
| `/telemetry/incidents/mode` | POST | `telemetry:admin` | Toggle forensic mode |
| `/telemetry/incidents/status` | GET | `telemetry:read` | Current mode status |
### 9.3 Exports
| Endpoint | Method | Scope | Description |
|----------|--------|-------|-------------|
| `/telemetry/exports/forensic/{window}` | GET | `telemetry:export` | Stream OTLP bundle |
---
## 10. Offline Support
### 10.1 Bundle Structure
```
telemetry-bundle/
├── otlp/
│ ├── metrics.pb
│ ├── traces.pb
│ └── logs.pb
├── config/
│ ├── collector.yaml
│ └── dashboards/
├── manifest.json
└── signatures/
└── manifest.sig
```
### 10.2 Sealed-Mode Guards
```csharp
// StellaOps.Telemetry.Core enforces IEgressPolicy
if (sealedMode.IsActive)
{
// Disable non-loopback exporters
// Emit structured warning with remediation
// Fall back to file-based export
}
```
---
## 11. Dashboards & Alerts
### 11.1 Standard Dashboards
| Dashboard | Purpose | Panels |
|-----------|---------|--------|
| Platform Health | Overall status | Request rate, error rate, latency |
| Scan Operations | Scanner metrics | Scan rate, duration, findings |
| Policy Engine | Policy metrics | Evaluation rate, rule hits, verdicts |
| Job Orchestration | Queue metrics | Queue depth, job latency, failures |
### 11.2 Alert Rules
| Alert | Condition | Severity |
|-------|-----------|----------|
| High Error Rate | error_rate > 5% | critical |
| Slow Scans | p95 > 5m | warning |
| Queue Backlog | depth > 1000 | warning |
| Circuit Open | breaker_open = 1 | critical |
---
## 12. Security Considerations
### 12.1 Data Protection
- Sensitive attributes redacted at collection
- Encrypted in transit (TLS)
- Encrypted at rest (storage layer)
- Retention policies enforced
### 12.2 Access Control
- Authority scopes for API access
- Tenant isolation in queries
- Audit logging for forensic access
---
## 13. Related Documentation
| Resource | Location |
|----------|----------|
| Telemetry architecture | `docs/modules/telemetry/architecture.md` |
| Collector configuration | `docs/modules/telemetry/collector-config.md` |
| Dashboard provisioning | `docs/modules/telemetry/dashboards.md` |
---
## 14. Sprint Mapping
- **Primary Sprint:** SPRINT_0180_0001_0001_telemetry_core.md (NEW)
- **Related Sprints:**
- SPRINT_0181_0001_0002_telemetry_forensic.md
- SPRINT_0182_0001_0003_telemetry_offline.md
**Key Task IDs:**
- `TELEM-CORE-40-001` - SDK integration (DONE)
- `TELEM-DASH-41-001` - Dashboard provisioning (DONE)
- `TELEM-FOR-50-001` - Forensic bundles (IN PROGRESS)
- `TELEM-SEAL-51-001` - Sealed-mode guards (TODO)
- `TELEM-SIGN-52-001` - Bundle signing (TODO)
---
## 15. Success Metrics
| Metric | Target |
|--------|--------|
| Collection overhead | < 2% CPU |
| Trace sampling accuracy | 100% for errors |
| Log ingestion latency | < 5 seconds |
| Forensic activation time | < 30 seconds |
| Bundle export time | < 5 minutes (24h data) |
---
*Last updated: 2025-11-29*

157
docs/product-advisories/ADVISORY_INDEX.md
View File

@@ -157,6 +157,107 @@ These are the authoritative advisories to reference for implementation:
- `docs/security/dpop-mtls-rollout.md` - Sender constraints
- **Status:** Fills HIGH-priority gap - consolidates token model, scopes, multi-tenant isolation
### CLI Developer Experience & Command UX
- **Canonical:** `29-Nov-2025 - CLI Developer Experience and Command UX.md`
- **Sprint:** SPRINT_0201_0001_0001_cli_i.md (PRIMARY)
- **Related Sprints:**
- SPRINT_203_cli_iii.md
- SPRINT_205_cli_v.md
- **Related Docs:**
- `docs/modules/cli/architecture.md` - Module architecture
- `docs/09_API_CLI_REFERENCE.md` - Command reference
- **Status:** Fills HIGH-priority gap - covers command surface, auth model, Buildx integration
### Orchestrator Event Model & Job Lifecycle
- **Canonical:** `29-Nov-2025 - Orchestrator Event Model and Job Lifecycle.md`
- **Sprint:** SPRINT_0151_0001_0001_orchestrator_i.md (PRIMARY)
- **Related Sprints:**
- SPRINT_152_orchestrator_ii.md
- SPRINT_0152_0001_0002_orchestrator_ii.md
- **Related Docs:**
- `docs/modules/orchestrator/architecture.md` - Module architecture
- **Status:** Fills HIGH-priority gap - covers job lifecycle, quota governance, replay semantics
### Export Center & Reporting Strategy
- **Canonical:** `29-Nov-2025 - Export Center and Reporting Strategy.md`
- **Sprint:** SPRINT_0160_0001_0001_export_evidence.md (PRIMARY)
- **Related Sprints:**
- SPRINT_0161_0001_0001_evidencelocker.md
- **Related Docs:**
- `docs/modules/export-center/architecture.md` - Module architecture
- **Status:** Fills MEDIUM-priority gap - covers profile system, adapters, distribution channels
### Runtime Posture & Observation (Zastava)
- **Canonical:** `29-Nov-2025 - Runtime Posture and Observation with Zastava.md`
- **Sprint:** SPRINT_0144_0001_0001_zastava_runtime_signals.md (PRIMARY)
- **Related Sprints:**
- SPRINT_0140_0001_0001_runtime_signals.md
- SPRINT_0143_0000_0001_signals.md
- **Related Docs:**
- `docs/modules/zastava/architecture.md` - Module architecture
- **Status:** Fills MEDIUM-priority gap - covers runtime events, admission control, drift detection
### Notification Rules & Alerting Engine
- **Canonical:** `29-Nov-2025 - Notification Rules and Alerting Engine.md`
- **Sprint:** SPRINT_0170_0001_0001_notify_engine.md (NEW)
- **Related Sprints:**
- SPRINT_0171_0001_0002_notify_connectors.md
- SPRINT_0172_0001_0003_notify_ack_tokens.md
- **Related Docs:**
- `docs/modules/notify/architecture.md` - Module architecture
- **Status:** Fills MEDIUM-priority gap - covers rules engine, channels, noise control, ack tokens
### Graph Analytics & Dependency Insights
- **Canonical:** `29-Nov-2025 - Graph Analytics and Dependency Insights.md`
- **Sprint:** SPRINT_0141_0001_0001_graph_indexer.md (PRIMARY)
- **Related Sprints:**
- SPRINT_0401_0001_0001_reachability_evidence_chain.md
- SPRINT_0140_0001_0001_runtime_signals.md
- **Related Docs:**
- `docs/modules/graph/architecture.md` - Module architecture
- **Status:** Fills MEDIUM-priority gap - covers graph model, overlays, analytics, visualization
### Telemetry & Observability Patterns
- **Canonical:** `29-Nov-2025 - Telemetry and Observability Patterns.md`
- **Sprint:** SPRINT_0180_0001_0001_telemetry_core.md (NEW)
- **Related Sprints:**
- SPRINT_0181_0001_0002_telemetry_forensic.md
- SPRINT_0182_0001_0003_telemetry_offline.md
- **Related Docs:**
- `docs/modules/telemetry/architecture.md` - Module architecture
- **Status:** Fills MEDIUM-priority gap - covers collector topology, forensic mode, offline bundles
### Policy Simulation & Shadow Gates
- **Canonical:** `29-Nov-2025 - Policy Simulation and Shadow Gates.md`
- **Sprint:** SPRINT_0185_0001_0001_policy_simulation.md (NEW)
- **Related Sprints:**
- SPRINT_0120_0000_0001_policy_reasoning.md
- SPRINT_0121_0001_0001_policy_reasoning.md
- **Related Docs:**
- `docs/modules/policy/architecture.md` - Module architecture
- **Status:** Fills MEDIUM-priority gap - covers shadow runs, coverage fixtures, promotion gates
### Findings Ledger & Immutable Audit Trail
- **Canonical:** `29-Nov-2025 - Findings Ledger and Immutable Audit Trail.md`
- **Sprint:** SPRINT_0186_0001_0001_record_deterministic_execution.md (PRIMARY)
- **Related Sprints:**
- SPRINT_0120_0000_0001_policy_reasoning.md
- SPRINT_311_docs_tasks_md_xi.md
- **Related Docs:**
- `docs/modules/findings-ledger/openapi/findings-ledger.v1.yaml` - OpenAPI spec
- **Status:** Fills MEDIUM-priority gap - covers append-only events, Merkle anchoring, projections
### Concelier Advisory Ingestion Model
- **Canonical:** `29-Nov-2025 - Concelier Advisory Ingestion Model.md`
- **Sprint:** SPRINT_0115_0001_0004_concelier_iv.md (PRIMARY)
- **Related Sprints:**
- SPRINT_0113_0001_0002_concelier_ii.md
- SPRINT_0114_0001_0003_concelier_iii.md
- **Related Docs:**
- `docs/modules/concelier/architecture.md` - Module architecture
- `docs/modules/concelier/link-not-merge-schema.md` - LNM schema
- **Status:** Fills MEDIUM-priority gap - covers AOC, Link-Not-Merge, connectors, deterministic exports
## Files Archived
The following files have been moved to `archived/27-Nov-2025-superseded/`:
@@ -198,6 +299,16 @@ The following issues were fixed:
| Mirror & Offline Kit | SPRINT_0125_0001_0001 | EXISTING |
| Task Pack Orchestration | SPRINT_0157_0001_0001 | EXISTING |
| Auth/AuthZ Architecture | Multiple (100, 314, 0514) | EXISTING |
| CLI Developer Experience | SPRINT_0201_0001_0001 | NEW |
| Orchestrator Event Model | SPRINT_0151_0001_0001 | NEW |
| Export Center Strategy | SPRINT_0160_0001_0001 | NEW |
| Zastava Runtime Posture | SPRINT_0144_0001_0001 | NEW |
| Notification Rules Engine | SPRINT_0170_0001_0001 | NEW |
| Graph Analytics | SPRINT_0141_0001_0001 | NEW |
| Telemetry & Observability | SPRINT_0180_0001_0001 | NEW |
| Policy Simulation | SPRINT_0185_0001_0001 | NEW |
| Findings Ledger | SPRINT_0186_0001_0001 | NEW |
| Concelier Ingestion | SPRINT_0115_0001_0004 | NEW |
## Implementation Priority
@@ -210,11 +321,21 @@ Based on gap analysis:
5. **P1 - Sovereign Crypto** (Sprint 0514) - Regional compliance enablement
6. **P1 - Evidence Bundle & Replay** (Sprint 0161, 0187) - Audit/compliance critical
7. **P1 - Mirror & Offline Kit** (Sprint 0125, 0150) - Air-gap deployment critical
8. **P2 - Task Pack Orchestration** (Sprint 0157, 0158) - Automation foundation
9. **P2 - Explainability** (Sprint 0401) - UX enhancement, existing tasks
10. **P2 - Plugin Architecture** (Multiple) - Foundational extensibility patterns
11. **P2 - Auth/AuthZ Architecture** (Multiple) - Security consolidation
12. **P3 - Already Implemented** - Unknowns, Graph IDs, DSSE batching
8. **P1 - CLI Developer Experience** (Sprint 0201) - Developer UX critical
9. **P1 - Orchestrator Event Model** (Sprint 0151) - Job lifecycle foundation
10. **P2 - Task Pack Orchestration** (Sprint 0157, 0158) - Automation foundation
11. **P2 - Explainability** (Sprint 0401) - UX enhancement, existing tasks
12. **P2 - Plugin Architecture** (Multiple) - Foundational extensibility patterns
13. **P2 - Auth/AuthZ Architecture** (Multiple) - Security consolidation
14. **P2 - Export Center** (Sprint 0160) - Reporting flexibility
15. **P2 - Zastava Runtime** (Sprint 0144) - Runtime observability
16. **P2 - Notification Rules** (Sprint 0170) - Alert management
17. **P2 - Graph Analytics** (Sprint 0141) - Dependency insights
18. **P2 - Telemetry** (Sprint 0180) - Observability infrastructure
19. **P2 - Policy Simulation** (Sprint 0185) - Safe policy testing
20. **P2 - Findings Ledger** (Sprint 0186) - Audit immutability
21. **P2 - Concelier Ingestion** (Sprint 0115) - Advisory pipeline
22. **P3 - Already Implemented** - Unknowns, Graph IDs, DSSE batching
## Implementer Quick Reference
@@ -241,6 +362,15 @@ For each topic, the implementer should read:
| Evidence Locker | `docs/modules/evidence-locker/*.md` | `src/EvidenceLocker/*/AGENTS.md` |
| Mirror | `docs/modules/mirror/*.md` | `src/Mirror/*/AGENTS.md` |
| TaskRunner | `docs/modules/taskrunner/*.md` | `src/TaskRunner/*/AGENTS.md` |
| CLI | `docs/modules/cli/architecture.md` | `src/Cli/*/AGENTS.md` |
| Orchestrator | `docs/modules/orchestrator/architecture.md` | `src/Orchestrator/*/AGENTS.md` |
| Export Center | `docs/modules/export-center/architecture.md` | `src/ExportCenter/*/AGENTS.md` |
| Zastava | `docs/modules/zastava/architecture.md` | `src/Zastava/*/AGENTS.md` |
| Notify | `docs/modules/notify/architecture.md` | `src/Notify/*/AGENTS.md` |
| Graph | `docs/modules/graph/architecture.md` | `src/Graph/*/AGENTS.md` |
| Telemetry | `docs/modules/telemetry/architecture.md` | `src/Telemetry/*/AGENTS.md` |
| Findings Ledger | `docs/modules/findings-ledger/openapi/` | `src/Findings/*/AGENTS.md` |
| Concelier | `docs/modules/concelier/architecture.md` | `src/Concelier/*/AGENTS.md` |
## Topical Gaps (Advisory Needed)
@@ -254,12 +384,17 @@ The following topics are mentioned in CLAUDE.md or module docs but lack dedicate
| ~~Mirror/Offline Kit Strategy~~ | HIGH | **FILLED** | `29-Nov-2025 - Mirror and Offline Kit Strategy.md` |
| ~~Task Pack Orchestration~~ | HIGH | **FILLED** | `29-Nov-2025 - Task Pack Orchestration and Automation.md` |
| ~~Auth/AuthZ Architecture~~ | HIGH | **FILLED** | `29-Nov-2025 - Authentication and Authorization Architecture.md` |
| ~~CLI Developer Experience~~ | HIGH | **FILLED** | `29-Nov-2025 - CLI Developer Experience and Command UX.md` |
| ~~Orchestrator Event Model~~ | HIGH | **FILLED** | `29-Nov-2025 - Orchestrator Event Model and Job Lifecycle.md` |
| ~~Export Center Strategy~~ | MEDIUM | **FILLED** | `29-Nov-2025 - Export Center and Reporting Strategy.md` |
| ~~Runtime Posture & Observation~~ | MEDIUM | **FILLED** | `29-Nov-2025 - Runtime Posture and Observation with Zastava.md` |
| ~~Notification Rules Engine~~ | MEDIUM | **FILLED** | `29-Nov-2025 - Notification Rules and Alerting Engine.md` |
| ~~Graph Analytics & Clustering~~ | MEDIUM | **FILLED** | `29-Nov-2025 - Graph Analytics and Dependency Insights.md` |
| ~~Telemetry & Observability~~ | MEDIUM | **FILLED** | `29-Nov-2025 - Telemetry and Observability Patterns.md` |
| ~~Policy Simulation & Shadow Gates~~ | MEDIUM | **FILLED** | `29-Nov-2025 - Policy Simulation and Shadow Gates.md` |
| ~~Findings Ledger & Audit Trail~~ | MEDIUM | **FILLED** | `29-Nov-2025 - Findings Ledger and Immutable Audit Trail.md` |
| ~~Concelier Advisory Ingestion~~ | MEDIUM | **FILLED** | `29-Nov-2025 - Concelier Advisory Ingestion Model.md` |
| **CycloneDX 1.6 .NET Integration** | LOW | Open | Deep Architecture covers generically; expand with .NET-specific guidance |
| **Findings Ledger & Audit Trail** | MEDIUM | Open | Immutable verdict tracking; module exists but no advisory |
| **Runtime Posture & Observation** | MEDIUM | Open | Zastava runtime signals; sprints exist but no advisory |
| **Graph Analytics & Clustering** | MEDIUM | Open | Community detection, blast-radius; implementation underway |
| **Policy Simulation & Shadow Gates** | MEDIUM | Open | Impact modeling; extensive sprints but no contract advisory |
| **Notification Rules Engine** | MEDIUM | Open | Throttling, digests, templating; sprints active |
## Known Issues (Non-Blocking)
@@ -274,4 +409,4 @@ Several filenames use en-dash (U+2011) instead of regular hyphen (-). This may c
---
*Index created: 2025-11-27*
*Last updated: 2025-11-29*
*Last updated: 2025-11-29 (added 10 new advisories filling all identified gaps)*