7040984215
- Introduced a new document outlining the inline DSSE provenance for SBOM, VEX, scan, and derived events. - Defined the Mongo schema for event patches, including key fields for provenance and trust verification. - Documented the write path for ingesting provenance metadata and backfilling historical events. - Created CI/CD snippets for uploading DSSE attestations and generating provenance metadata. - Established Mongo indexes for efficient provenance queries and provided query recipes for various use cases. - Outlined policy gates for managing VEX decisions based on provenance verification. - Included UI nudges for displaying provenance information and implementation tasks for future enhancements. --- Implement reachability lattice and scoring model - Developed a comprehensive document detailing the reachability lattice and scoring model. - Defined core types for reachability states, evidence, and mitigations with corresponding C# models. - Established a scoring policy with base score contributions from various evidence classes. - Mapped reachability states to VEX gates and provided a clear overview of evidence sources. - Documented the event graph schema for persisting reachability data in MongoDB. - Outlined the integration of runtime probes for evidence collection and defined a roadmap for future tasks. --- Introduce uncertainty states and entropy scoring - Created a draft document for tracking uncertainty states and their impact on risk scoring. - Defined core uncertainty states with associated entropy values and evidence requirements. - Established a schema for storing uncertainty states alongside findings. - Documented the risk score calculation incorporating uncertainty and its effect on final risk assessments. - Provided policy guidelines for handling uncertainty in decision-making processes. - Outlined UI guidelines for displaying uncertainty information and suggested remediation actions. --- Add Ruby package inventory management - Implemented Ruby package inventory management with corresponding data models and storage mechanisms. - Created C# records for Ruby package inventory, artifacts, provenance, and runtime details. - Developed a repository for managing Ruby package inventory documents in MongoDB. - Implemented a service for storing and retrieving Ruby package inventories. - Added unit tests for the Ruby package inventory store to ensure functionality and data integrity.
19 KiB
19 KiB
Sprint 140 - Runtime & Signals
Active items only. Completed/historic work now resides in docs/implplan/archived/tasks.md (updated 2025-11-08).
This file now only tracks the runtime & signals status snapshot. Active backlog lives in Sprint 141+ files.
Wave coordination
| Wave | Guild owners | Shared prerequisites | Status | Notes |
|---|---|---|---|---|
| 140.A Graph | Graph Indexer Guild · Observability Guild | Sprint 120.A – AirGap; Sprint 130.A – Scanner (phase I tracked under docs/implplan/SPRINT_130_scanner_surface.md) |
TODO | Hold until Scanner surface work emits the analyzer artifacts required for clustering jobs. |
| 140.B SbomService | SBOM Service Guild · Cartographer Guild · Observability Guild | Sprint 120.A – AirGap; Sprint 130.A – Scanner | TODO | Projection schema remains blocked on Concelier outputs; keep AirGap parity requirements in scope. |
| 140.C Signals | Signals Guild · Authority Guild (for scopes) · Runtime Guild | Sprint 120.A – AirGap; Sprint 130.A – Scanner | DOING | API skeleton and callgraph ingestion are active; runtime facts endpoint still depends on the same shared prerequisites. |
| 140.D Zastava | Zastava Observer/Webhook Guilds · Security Guild | Sprint 120.A – AirGap; Sprint 130.A – Scanner | TODO | Surface.FS integration waits on Scanner surface caches; prep sealed-mode env helpers meanwhile. |
Status snapshot (2025-11-12)
- 140.A Graph – GRAPH-INDEX-28-007/008/009/010 remain TODO while Scanner surface artifacts and SBOM projection schemas are outstanding; clustering/backfill/fixture scaffolds are staged but cannot progress until analyzer payloads arrive.
- 140.B SbomService – Advisory AI, console, and orchestrator tracks stay TODO; SBOM-SERVICE-21-001..004 remain BLOCKED waiting for Concelier Link-Not-Merge (
CONCELIER-GRAPH-21-001) plus Cartographer schema (CARTO-GRAPH-21-002), and AirGap parity must be re-validated once schemas land. Teams are refining projection docs so we can flip to DOING as soon as payloads land. - 140.C Signals – SIGNALS-24-001 shipped on 2025-11-09; SIGNALS-24-002 is DOING with callgraph retrieval live but CAS promotion + signed manifest tooling still pending; SIGNALS-24-003 is DOING after JSON/NDJSON ingestion merged, yet provenance/context enrichment and runtime feed reconciliation remain in-flight. Scoring/cache work (SIGNALS-24-004/005) stays BLOCKED until runtime uploads publish consistently and scope propagation validation (post
AUTH-SIG-26-001) completes. - 140.D Zastava – ZASTAVA-ENV/SECRETS/SURFACE tracks remain TODO because Surface.FS cache outputs from Scanner are still unavailable; guilds continue prepping Surface.Env helper adoption and sealed-mode scaffolding.
Wave task tracker (refreshed 2025-11-12)
140.A Graph
| Task ID | State | Notes |
|---|---|---|
| GRAPH-INDEX-28-007 | TODO | Clustering/centrality jobs queued behind Scanner surface analyzer artifacts; design work complete but implementation held. |
| GRAPH-INDEX-28-008 | TODO | Incremental update/backfill pipeline depends on 28-007 artifacts; retry/backoff plumbing sketched but blocked. |
| GRAPH-INDEX-28-009 | TODO | Test/fixture/chaos coverage waits on earlier jobs to exist so determinism checks have data. |
| GRAPH-INDEX-28-010 | TODO | Packaging/offline bundles paused until upstream graph jobs are available to embed. |
140.B SbomService
| Task ID | State | Notes |
|---|---|---|
| SBOM-AIAI-31-001 | TODO | Advisory AI path/timeline endpoints specced; awaiting projection schema finalization. |
| SBOM-AIAI-31-002 | TODO | Metrics/dashboards tied to 31-001; blocked on the same schema availability. |
| SBOM-CONSOLE-23-001 | TODO | Console catalog API draft complete; depends on Concelier/Cartographer payload definitions. |
| SBOM-CONSOLE-23-002 | TODO | Global component lookup API needs 23-001 responses + cache hints before work can start. |
| SBOM-ORCH-32-001 | TODO | Orchestrator registration is sequenced after projection schema because payload shapes map into job metadata. |
| SBOM-ORCH-33-001 | TODO | Backpressure/telemetry features depend on 32-001 workers. |
| SBOM-ORCH-34-001 | TODO | Backfill + watermark logic requires the orchestrator integration from 33-001. |
| SBOM-SERVICE-21-001 | BLOCKED | Normalized SBOM projection schema cannot ship until Concelier (CONCELIER-GRAPH-21-001) delivers Link-Not-Merge definitions. |
| SBOM-SERVICE-21-002 | BLOCKED | Change events hinge on 21-001 response contract; no work underway. |
| SBOM-SERVICE-21-003 | BLOCKED | Entry point/service node management blocked behind 21-002 event outputs. |
| SBOM-SERVICE-21-004 | BLOCKED | Observability wiring follows projection + event pipelines; on hold. |
| SBOM-SERVICE-23-001 | TODO | Asset metadata extensions queued once 21-004 observability baseline exists. |
| SBOM-SERVICE-23-002 | TODO | Asset update events depend on 23-001 schema. |
| SBOM-VULN-29-001 | TODO | Inventory evidence feed deferred until projection schema + runtime align. |
| SBOM-VULN-29-002 | TODO | Resolver feed requires 29-001 event payloads. |
140.C Signals
| Task ID | State | Notes |
|---|---|---|
| SIGNALS-24-001 | DONE (2025-11-09) | Host skeleton, RBAC, sealed-mode readiness, /signals/facts/{subject} retrieval, and readiness probes merged; serves as base for downstream ingestion. |
| SIGNALS-24-002 | DOING (2025-11-07) | Callgraph ingestion + retrieval APIs are live, but CAS promotion and signed manifest publication remain; cannot close until reachability jobs can trust stored graphs. |
| SIGNALS-24-003 | DOING (2025-11-09) | Runtime facts ingestion accepts JSON/NDJSON and gzip streams; provenance/context enrichment and NDJSON-to-AOC wiring still outstanding. |
| SIGNALS-24-004 | BLOCKED (2025-10-27) | Reachability scoring waits on complete ingestion feeds (24-002/003) plus Authority scope validation. |
| SIGNALS-24-005 | BLOCKED (2025-10-27) | Cache + signals.fact.updated events depend on scoring outputs; remains idle until 24-004 unblocks. |
140.D Zastava
| Task ID | State | Notes |
|---|---|---|
| ZASTAVA-ENV-01 | TODO | Observer adoption of Surface.Env helpers paused while Surface.FS cache contract finalizes. |
| ZASTAVA-ENV-02 | TODO | Webhook helper migration follows ENV-01 completion. |
| ZASTAVA-SECRETS-01 | TODO | Surface.Secrets wiring for Observer pending published cache endpoints. |
| ZASTAVA-SECRETS-02 | TODO | Webhook secret retrieval cascades from SECRETS-01 work. |
| ZASTAVA-SURFACE-01 | TODO | Surface.FS client integration blocked on Scanner layer metadata; tests ready once packages mirror offline dependencies. |
| ZASTAVA-SURFACE-02 | TODO | Admission enforcement requires SURFACE-01 so webhook responses can gate on cache freshness. |
In-flight focus (DOING items)
| Task ID | Remaining work | Target date | Owners |
|---|---|---|---|
| SIGNALS-24-002 | Promote callgraph CAS buckets to prod scopes, publish signed manifest metadata, document retention/GC policy, wire alerts for failed graph retrievals. | 2025-11-14 | Signals Guild, Platform Storage Guild |
| SIGNALS-24-003 | Finalize provenance/context enrichment (Authority scopes + runtime metadata), support NDJSON batch provenance, backfill existing facts, and validate AOC contract. | 2025-11-15 | Signals Guild, Runtime Guild, Authority Guild |
Wave readiness checklist (2025-11-12)
| Wave | Entry criteria | Prep status | Next checkpoint |
|---|---|---|---|
| 140.A Graph | Scanner surface analyzer artifacts + SBOM projection schema for clustering inputs. | Job scaffolds and determinism harness drafted; waiting on artifact ETA. | 2025-11-13 cross-guild sync (Scanner ↔ Graph) to lock delivery window. |
| 140.B SbomService | Concelier Link-Not-Merge + Cartographer projection schema, plus AirGap parity review. | Projection doc redlines complete; schema doc ready for Concelier feedback. | 2025-11-14 schema review (Concelier, Cartographer, SBOM). |
| 140.C Signals | CAS promotion approval + runtime provenance contract + AUTH-SIG-26-001 sign-off. | HOST + callgraph retrieval merged; CAS/provenance work tracked in DOING table above. | 2025-11-13 runtime sync to approve CAS rollout + schema freeze. |
| 140.D Zastava | Surface.FS cache availability + Surface.Env helper specs published. | Env/secrets design notes ready; waiting for Scanner cache drop and Surface.FS API stubs. | 2025-11-15 Surface guild office hours to confirm helper adoption plan. |
Signals DOING activity log
| Date | Update | Owners |
|---|---|---|
| 2025-11-12 | Drafted CAS promotion checklist (bucket policies, signer config, GC guardrails) and circulated to Platform Storage for approval; added alert runbooks for failed graph retrievals. | Signals Guild, Platform Storage Guild |
| 2025-11-11 | Completed NDJSON ingestion soak test (JSON/NDJSON + gzip) and documented provenance enrichment mapping required from Authority scopes; open PR wiring AOC metadata pending review. | Signals Guild, Runtime Guild |
| 2025-11-09 | Runtime facts ingestion endpoint + streaming NDJSON support merged with sealed-mode gating; next tasks are provenance enrichment and scoring linkage. | Signals Guild, Runtime Guild |
Dependency status watchlist (2025-11-12)
| Dependency | Status | Latest detail | Owner(s) / follow-up |
|---|---|---|---|
| AUTH-SIG-26-001 (Signals scopes + AOC) | DONE (2025-10-29) | Authority shipped scope + role templates; Signals is validating propagation + provenance enrichment before enabling scoring. | Authority Guild · Runtime Guild · Signals Guild |
| CONCELIER-GRAPH-21-001 (SBOM projection enrichment) | BLOCKED (2025-10-27) | Awaiting Cartographer schema + Link-Not-Merge contract; SBOM/Graph/Zastava work cannot proceed without enriched projections. | Concelier Core · Cartographer Guild |
| CONCELIER-GRAPH-21-002 / CARTO-GRAPH-21-002 (SBOM change events) | BLOCKED (2025-10-27) | Change event contract depends on 21-001; Cartographer has not provided webhook schema yet. | Concelier Core · Cartographer Guild · Platform Events Guild |
| Sprint 130 Scanner surface artifacts | ETA pending | Analyzer artifact publication schedule still outstanding; Graph/Zastava need cache outputs and manifests. | Scanner Guild · Graph Indexer Guild · Zastava Guilds |
| AirGap parity review (Sprint 120.A) | Not scheduled | SBOM path/timeline endpoints must re-pass AirGap checklist once Concelier schema lands; reviewers on standby. | AirGap Guild · SBOM Service Guild |
Upcoming checkpoints
| Date | Session | Goal | Impacted wave(s) | Prep owner(s) |
|---|---|---|---|---|
| 2025-11-13 | Scanner ↔ Graph readiness sync | Lock analyzer artifact ETA + cache publish plan so GRAPH-INDEX-28-007 can start immediately after delivery. | 140.A Graph · 140.D Zastava | Scanner Guild · Graph Indexer Guild |
| 2025-11-13 | Runtime/Signals CAS + provenance review | Approve CAS promotion checklist, freeze provenance schema, and green-light SIGNALS-24-002/003 close-out tasks. | 140.C Signals | Signals Guild · Runtime Guild · Authority Guild · Platform Storage Guild |
| 2025-11-14 | Concelier/Cartographer/SBOM schema review | Ratify Link-Not-Merge projection schema + change event contract; schedule AirGap parity verification. | 140.B SbomService · 140.A Graph · 140.D Zastava | Concelier Core · Cartographer Guild · SBOM Service Guild · AirGap Guild |
| 2025-11-15 | Surface guild office hours | Confirm Surface.Env helper adoption + Surface.FS cache drop timeline for Zastava. | 140.D Zastava | Surface Guild · Zastava Observer/Webhook Guilds |
Meeting prep checklist
| Session | Pre-reads / artifacts | Open questions to resolve | Owners |
|---|---|---|---|
| Scanner ↔ Graph (2025-11-13) | Sprint 130 surface artifact roadmap draft, GRAPH-INDEX-28-007 scaffolds, ZASTAVA-SURFACE dependency list. | Exact drop date for analyzer artifacts? Will caches ship phased or all at once? Need mock payloads if delayed? | Scanner Guild · Graph Indexer Guild · Zastava Guilds |
| Runtime/Signals CAS review (2025-11-13) | CAS promotion checklist, signed manifest PR links, provenance schema draft, NDJSON ingestion soak results. | Storage approval on bucket policies/GC? Authority confirmation on scope propagation + AOC metadata? Backfill approach for existing runtime facts? | Signals Guild · Runtime Guild · Authority Guild · Platform Storage Guild |
| Concelier schema review (2025-11-14) | Link-Not-Merge schema redlines, Cartographer webhook contract, AirGap parity checklist, SBOM-SERVICE-21-001 scaffolding plan. | Final field list for relationships/scopes? Event payload metadata requirements? AirGap review schedule & owners? | Concelier Core · Cartographer Guild · SBOM Service Guild · AirGap Guild |
| Surface guild office hours (2025-11-15) | Surface.Env helper adoption notes, sealed-mode test harness outline, Surface.FS API stub timeline. | Can Surface.FS caches publish before Analyzer drop? Any additional sealed-mode requirements? Who owns Surface.Env rollout in Observer/Webhook repos? | Surface Guild · Zastava Observer/Webhook Guilds |
Target outcomes (through 2025-11-15)
| Deliverable | Target date | Status | Dependencies / notes |
|---|---|---|---|
| SIGNALS-24-002 CAS promotion + signed manifests | 2025-11-14 | DOING | Needs Platform Storage sign-off from 2025-11-13 review; alerts/runbooks drafted. |
| SIGNALS-24-003 provenance enrichment + backfill | 2025-11-15 | DOING | Waiting on Runtime/Authority schema freeze + scope fixtures; NDJSON ingestion landed. |
| Scanner analyzer artifact ETA & cache drop plan | 2025-11-13 | TODO | Scanner to publish Sprint 130 surface roadmap; Graph/Zastava blocked until then. |
| Concelier Link-Not-Merge schema ratified | 2025-11-14 | BLOCKED | Requires CONCELIER-GRAPH-21-001 + CARTO-GRAPH-21-002 agreement; AirGap review scheduled after sign-off. |
| Surface.Env helper adoption checklist | 2025-11-15 | TODO | Zastava guild preparing sealed-mode test harness; depends on Surface guild office hours outcomes. |
Blockers & coordination
- Concelier Link-Not-Merge / Cartographer schemas – SBOM-SERVICE-21-001..004 cannot start until
CONCELIER-GRAPH-21-001andCARTO-GRAPH-21-002deliver the projection payloads. - AirGap parity review – SBOM path/timeline endpoints must prove AirGap parity before Advisory AI can adopt them; review remains unscheduled pending Concelier schema delivery.
- Scanner surface artifacts – GRAPH-INDEX-28-007+ and all ZASTAVA-SURFACE tasks depend on Sprint 130 analyzer outputs and cached layer metadata; need updated ETA from Scanner guild.
- Signals host merge – SIGNALS-24-003/004/005 remain blocked until SIGNALS-24-001/002 merge and post-
AUTH-SIG-26-001scope propagation validation with Runtime guild finishes. - CAS promotion + signed manifests – SIGNALS-24-002 cannot close until Storage guild reviews CAS promotion plan and manifest signing tooling; downstream scoring needs immutable graph IDs.
- Runtime provenance wiring – SIGNALS-24-003 still needs Authority scope propagation and NDJSON provenance mapping before runtime feeds can unblock scoring/cache layers.
Next actions (target: 2025-11-14)
| Owner(s) | Action |
|---|---|
| Graph Indexer Guild | Use 2025-11-13 Scanner sync to lock analyzer artifact ETA; keep clustering/backfill scaffolds staged so GRAPH-INDEX-28-007 can flip to DOING immediately after feeds land. |
| SBOM Service Guild | Circulate redlined projection schema to Concelier/Cartographer ahead of the 2025-11-14 review; scaffold SBOM-SERVICE-21-001 PR so coding can start once schema is approved. |
| Signals Guild | Merge CAS promotion + signed manifest PRs, then pivot to SIGNALS-24-003 provenance enrichment/backfill; prepare scoring/cache kickoff deck for 24-004/005 owners. |
| Runtime & Authority Guilds | Use delivered AUTH-SIG-26-001 scopes to finish propagation validation, freeze provenance schema, and hand off fixtures to Signals before 2025-11-15. |
| Platform Storage Guild | Review CAS bucket policies/GC guardrails from the 2025-11-12 checklist and provide written sign-off before runtime sync on 2025-11-13. |
| Scanner Guild | Publish Sprint 130 surface artifact roadmap + Surface.FS cache drop timeline so Graph/Zastava can schedule start dates; provide mock datasets if slips extend past 2025-11-15. |
| Zastava Guilds | Convert Surface.Env helper adoption notes into a ready-to-execute checklist, align sealed-mode tests, and be prepared to start once Surface.FS caches are announced. |
Downstream dependency rollup (snapshot: 2025-11-12)
| Track | Dependent sprint(s) | Impact if delayed |
|---|---|---|
| 140.A Graph | docs/implplan/SPRINT_141_graph.md (Graph clustering/backfill) and downstream Graph UI overlays |
Graph insights, policy overlays, and runtime clustering views cannot progress without GRAPH-INDEX-28-007+ landing. |
| 140.B SbomService | docs/implplan/SPRINT_142_sbomservice.md, Advisory AI (Sprint 111), Policy/Vuln Explorer feeds |
SBOM projections/events stay unavailable, blocking Advisory AI remedation heuristics, policy joins, and Vuln Explorer candidate generation. |
| 140.C Signals | docs/implplan/SPRINT_143_signals.md plus Runtime/Reachability dashboards |
Reachability scoring, cache/event layers, and runtime facts outputs cannot start until SIGNALS-24-001/002 merge and Scanner runtime data flows. |
| 140.D Zastava | docs/implplan/SPRINT_144_zastava.md, Runtime admission enforcement |
Surface-integrated drift/admission hooks remain stalled; sealed-mode env helpers cannot ship without Surface.FS metadata. |
Risk log
| Risk | Impact | Mitigation / owner |
|---|---|---|
| Concelier Link-Not-Merge schema slips | SBOM-SERVICE-21-001..004 + Advisory AI SBOM endpoints stay blocked | Concelier + Cartographer guilds to publish CARTO-GRAPH-21-002 ETA during next coordination call; SBOM guild to prep schema doc meanwhile. |
| Scanner surface artifact delay | GRAPH-INDEX-28-007+ and ZASTAVA-SURFACE-* cannot even start | Scanner guild to deliver analyzer artifact roadmap; Graph/Zastava teams to prepare mocks/tests in advance. |
| Signals host/callgraph merge misses 2025-11-09 | SIGNALS-24-003/004/005 remain blocked, pushing reachability scoring past sprint goals | Signals + Authority guilds to prioritize AUTH-SIG-26-001 review and merge SIGNALS-24-001/002 before 2025-11-10 standup. |
Authority build regression (PackApprovalFreshAuthWindow) |
Signals test suite cannot run in CI, delaying validation of new endpoints | Coordinate with Authority guild to restore missing constant in StellaOps.Auth.ServerIntegration; rerun Signals tests once fixed. |
| CAS promotion slips past 2025-11-14 | SIGNALS-24-002 cannot close; reachability scoring has no trusted graph artifacts | Signals + Platform Storage to co-own CAS rollout checklist, escalate blockers during 2025-11-13 runtime sync. |
| Runtime provenance schema churn | SIGNALS-24-003 enrichment delays scoring/cache unblock and risks double uploads | Runtime + Authority guilds to freeze schema by 2025-11-14 and publish contract appendix; Signals updates ingestion once finalized. |
Coordination log
| Date | Notes |
|---|---|
| 2025-11-12 | Snapshot + wave tracker refreshed; pending dependencies captured for Graph/SBOM/Signals/Zastava while Signals DOING work progresses on callgraph CAS promotion + runtime ingestion wiring. |
| 2025-11-11 | Runtime + Signals ran NDJSON ingestion soak test; Authority flagged remaining provenance fields for schema freeze ahead of 2025-11-13 sync. |
| 2025-11-09 | Sprint 140 snapshot refreshed; awaiting Scanner surface artifact ETA, Concelier/CARTO schema delivery, and Signals host merge before any wave can advance to DOING. |