feat: Implement Runtime Facts ingestion service and NDJSON reader
Some checks failed
Docs CI / lint-and-preview (push) Has been cancelled
Some checks failed
Docs CI / lint-and-preview (push) Has been cancelled
- Added RuntimeFactsNdjsonReader for reading NDJSON formatted runtime facts. - Introduced IRuntimeFactsIngestionService interface and its implementation. - Enhanced Program.cs to register new services and endpoints for runtime facts. - Updated CallgraphIngestionService to include CAS URI in stored artifacts. - Created RuntimeFactsValidationException for validation errors during ingestion. - Added tests for RuntimeFactsIngestionService and RuntimeFactsNdjsonReader. - Implemented SignalsSealedModeMonitor for compliance checks in sealed mode. - Updated project dependencies for testing utilities.
This commit is contained in:
@@ -6,26 +6,89 @@ Active items only. Completed/historic work now resides in docs/implplan/archived
|
||||
|
||||
| Wave | Guild owners | Shared prerequisites | Status | Notes |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| 110.A AdvisoryAI | Advisory AI Guild · Docs Guild · SBOM Service Guild | Sprint 100.A – Attestor (closed 2025-11-09 per `docs/implplan/archived/SPRINT_100_identity_signing.md`) | DOING | WebService/Worker orchestration, guardrails, and docs are live; continue console/CLI coverage as endpoints land. |
|
||||
| 110.B Concelier | Concelier Core & WebService Guilds · Observability Guild · AirGap Guilds (Importer/Policy/Time) | Sprint 100.A – Attestor | DOING | Telemetry wiring started; mirror/air-gap tasks unlocked with AdvisoryAI evidence dependencies met. |
|
||||
| 110.C Excititor | Excititor WebService/Core Guilds · Observability Guild · Evidence Locker Guild | Sprint 100.A – Attestor | DOING | VEX justification enrichment and provenance metadata are underway; keep Link-Not-Merge blockers tracked. |
|
||||
| 110.D Mirror | Mirror Creator Guild · Exporter Guild · CLI Guild · AirGap Time Guild | Sprint 100.A – Attestor | TODO | Deterministic bundle assembler remains the gating task before DSSE/OCI work can proceed. |
|
||||
| 110.A AdvisoryAI | Advisory AI Guild · Docs Guild · SBOM Service Guild | Sprint 100.A – Attestor (closed 2025-11-09 per `docs/implplan/archived/SPRINT_100_identity_signing.md`) | DOING | Regression/perf suite (AIAI-31-009) and console doc (DOCS-AIAI-31-004) remain DOING; SBOM (SBOM-AIAI-31-001/003), CLI (CLI-VULN-29-001/CLI-VEX-30-001), Policy (POLICY-ENGINE-31-001), and DevOps (DEVOPS-AIAI-31-001) owners owe delivery ETA updates on 2025-11-10 so the CLI/policy/runbook docs can unblock. |
|
||||
| 110.B Concelier | Concelier Core & WebService Guilds · Observability Guild · AirGap Guilds (Importer/Policy/Time) | Sprint 100.A – Attestor | DOING | Paragraph chunk API shipped 2025-11-07; structured field/caching (CONCELIER-AIAI-31-002) is still TODO, telemetry (CONCELIER-AIAI-31-003) DOING, and air-gap/console/attestation tracks remain gated on Link-Not-Merge + Cartographer schema. |
|
||||
| 110.C Excititor | Excititor WebService/Core Guilds · Observability Guild · Evidence Locker Guild | Sprint 100.A – Attestor | DOING | Normalized justification projections (EXCITITOR-AIAI-31-001) are DOING; chunk API, telemetry, docs, attestation, and mirror backlog stay queued behind that work plus Link-Not-Merge / Cartographer prerequisites. |
|
||||
| 110.D Mirror | Mirror Creator Guild · Exporter Guild · CLI Guild · AirGap Time Guild | Sprint 100.A – Attestor | TODO | Wave remains TODO—MIRROR-CRT-56-001 has not started, so DSSE/TUF, OCI/time-anchor, CLI, and scheduling integrations cannot proceed. |
|
||||
|
||||
## Status Snapshot (2025-11-04)
|
||||
## Status Snapshot (2025-11-09)
|
||||
|
||||
- **Advisory AI** – 5 of 11 tasks are DONE (AIAI-31-001, AIAI-31-002, AIAI-31-003, AIAI-31-010, AIAI-31-011); orchestration pipeline (AIAI-31-004) and host wiring (AIAI-31-004A) remain TODO while downstream guardrails, CLI, and observability tracks (AIAI-31-004B/004C and AIAI-31-005 through AIAI-31-009) stay TODO pending cache/guardrail implementation and WebService/Worker hardening.
|
||||
- 2025-11-04: AIAI-31-002 and AIAI-31-003 shipped with deterministic SBOM context client wiring (`AddSbomContext` typed HTTP client) and toolset integration; WebService/Worker now invoke the orchestrator with SBOM-backed simulations and emit initial metrics.
|
||||
- 2025-11-03: AIAI-31-002 landed the configurable HTTP client + DI defaults; retriever now resolves data via `/v1/sbom/context`, retaining a null fallback until SBOM service ships.
|
||||
- 2025-11-03: Follow-up: SBOM guild to deliver base URL/API key and run an Advisory AI smoke retrieval once SBOM-AIAI-31-001 endpoints are live.
|
||||
- 2025-11-08: AIAI-31-009 marked DONE – injection harness + dual golden prompts + plan-cache determinism tests landed; perf memo added to Advisory AI architecture, `dotnet test src/AdvisoryAI/__Tests/StellaOps.AdvisoryAI.Tests/StellaOps.AdvisoryAI.Tests.csproj --no-build` green.
|
||||
- 2025-11-08: AIAI-31-008 DONE – deterministic plan ordering + conflict prompt goldens refreshed, remote inference packaging verified across Compose/Helm/Offline Kit profiles, and Advisory AI test suite re-run.
|
||||
- **Concelier** – CONCELIER-CORE-AOC-19-004 is the only in-flight Concelier item; air-gap, console, attestation, and Link-Not-Merge tasks remain TODO, and several connector upgrades still carry overdue October due dates.
|
||||
- **Excititor** – Excititor WebService, console, policy, and observability tracks are all TODO and hinge on Link-Not-Merge schema delivery plus trust-provenance connectors (SUSE/Ubuntu) progressing in section 110.C.
|
||||
- **Mirror** – Mirror Creator track (MIRROR-CRT-56-001 through MIRROR-CRT-58-002) has not started; DSSE signing, OCI bundle, and scheduling integrations depend on the deterministic bundle assembler landing first.
|
||||
- **Advisory AI (110.A)** – WebService orchestration (AIAI-31-004), typed SBOM client/tooling (AIAI-31-002/003), guardrail pipeline (AIAI-31-005), and overview/API/architecture docs (DOCS-AIAI-31-001/002/003) are DONE; focus now sits on DOCS-AIAI-31-004 and AIAI-31-009 while CLI/policy/SBOM deliverables unblock the remaining docs.
|
||||
- 2025-11-09: AIAI-31-009 remains DOING after converting the guardrail harness into JSON fixtures, expanding property/perf coverage, and validating offline cache seeding; remote inference packaging (AIAI-31-008) is still TODO until the policy knob work in AIAI-31-006..007 completes.
|
||||
- 2025-11-09: DOCS-AIAI-31-004 continues DOING—guardrail/offline sections are drafted, but screenshots plus copy blocks wait on CONSOLE-VULN-29-001, CONSOLE-VEX-30-001, and EXCITITOR-CONSOLE-23-001.
|
||||
- SBOM-AIAI-31-003 and DOCS-AIAI-31-005/006/008/009 remain BLOCKED pending SBOM-AIAI-31-001, CLI-VULN-29-001, CLI-VEX-30-001, POLICY-ENGINE-31-001, and DEVOPS-AIAI-31-001.
|
||||
- **Concelier (110.B)** – `/advisories/{advisoryKey}/chunks` shipped on 2025-11-07 with tenant enforcement, chunk tuning knobs, and regression fixtures; structured field/caching work (CONCELIER-AIAI-31-002) is still TODO while telemetry/guardrail instrumentation (CONCELIER-AIAI-31-003) is DOING.
|
||||
- Air-gap provenance/staleness bundles (`CONCELIER-AIRGAP-56-001` → `CONCELIER-AIRGAP-58-001`), console views/deltas (`CONCELIER-CONSOLE-23-001..003`), and attestation metadata (`CONCELIER-ATTEST-73-001/002`) remain TODO pending Link-Not-Merge plus Cartographer schema delivery.
|
||||
- Connector provenance refreshes `FEEDCONN-ICSCISA-02-012` and `FEEDCONN-KISA-02-008` are still overdue, leaving evidence parity gaps for those feeds.
|
||||
- **Excititor (110.C)** – Normalized VEX justification projections (EXCITITOR-AIAI-31-001) are DOING as of 2025-11-09; the downstream chunk API (EXCITITOR-AIAI-31-002), telemetry/guardrails (EXCITITOR-AIAI-31-003), docs/OpenAPI alignment (EXCITITOR-AIAI-31-004), and attestation payload work (`EXCITITOR-ATTEST-*`) stay TODO until that projection work plus Link-Not-Merge schema land.
|
||||
- Mirror/air-gap backlog (`EXCITITOR-AIRGAP-56-001` .. `EXCITITOR-AIRGAP-58-001`) and connector provenance parity (`EXCITITOR-CONN-TRUST-01-001`) remain unscheduled, so Advisory AI cannot yet hydrate sealed VEX evidence or cite connector signatures.
|
||||
- **Mirror (110.D)** – MIRROR-CRT-56-001 (deterministic bundle assembler) has not kicked off, so DSSE/TUF (MIRROR-CRT-56-002), OCI exports (MIRROR-CRT-57-001), time anchors (MIRROR-CRT-57-002), CLI verbs (MIRROR-CRT-58-001), and Export Center automation (MIRROR-CRT-58-002) are all blocked.
|
||||
|
||||
## Blockers & Overdue Follow-ups
|
||||
|
||||
- Advisory AI customer-facing coverage remains blocked until SBOM-AIAI-31-001 exposes the `/v1/sbom/context` hand-off kit and until CLI-VULN-29-001, CLI-VEX-30-001, POLICY-ENGINE-31-001, and DEVOPS-AIAI-31-001 ship—keeping SBOM-AIAI-31-003 plus DOCS-AIAI-31-005/006/008/009 and the remote inference packaging work (AIAI-31-008) on hold.
|
||||
- `CONCELIER-GRAPH-21-001`, `CONCELIER-GRAPH-21-002`, and `CONCELIER-GRAPH-21-005` remain BLOCKED awaiting `CONCELIER-POLICY-20-002` outputs and Cartographer schema (`CARTO-GRAPH-21-002`), keeping downstream Excititor graph consumers on hold.
|
||||
- `EXCITITOR-GRAPH-21-001`, `EXCITITOR-GRAPH-21-002`, and `EXCITITOR-GRAPH-21-005` stay BLOCKED until the same Cartographer/Link-Not-Merge prerequisites are delivered.
|
||||
- Connector provenance updates `FEEDCONN-ICSCISA-02-012` (due 2025-10-23) and `FEEDCONN-KISA-02-008` (due 2025-10-24) remain past due and need scheduling. FeedMerge coordination tasks have been dropped (no AOC policy/governance backing yet), so capacity shifts to schema/guard deliverables.
|
||||
- Mirror evidence work remains blocked until `MIRROR-CRT-56-001` ships; align Export Center (`EXPORT-OBS-51-001`) and AirGap time anchor (`AIRGAP-TIME-57-001`) owners for kickoff.
|
||||
|
||||
## Immediate actions (target: 2025-11-12)
|
||||
|
||||
- **Advisory AI** – Land AIAI-31-009 test harness updates plus remote inference packaging (AIAI-31-008) once POLICY-ENGINE-31-001 and DEVOPS-AIAI-31-001 expose the required knobs; SBOM guild to deliver SBOM-AIAI-31-001 so SBOM-AIAI-31-003 and the CLI/policy/runbook docs can unblock.
|
||||
- **Concelier** – Finish CONCELIER-AIAI-31-002 structured fields/caching and wire CONCELIER-AIAI-31-003 telemetry before starting air-gap or console endpoints; hold daily sync with Cartographer owners on CONCELIER-LNM-21-201/202 + CARTO-GRAPH-21-002.
|
||||
- **Excititor** – Wrap EXCITITOR-AIAI-31-001 justification projections, then immediately stage EXCITITOR-AIAI-31-002/003 plus EXCITITOR-ATTEST-01-003 to keep Advisory AI evidence feeds parallel to Concelier.
|
||||
- **Mirror** – Schedule MIRROR-CRT-56-001 kickoff with Export Center/AirGap Time guilds, confirm `EXPORT-OBS-51-001` + `AIRGAP-TIME-57-001` owners, and pre-stage DSSE/TUF design notes so MIRROR-CRT-56-002 can start as soon as the assembler lands.
|
||||
- **Downstream prep** – Scanner (Sprint 130) and Policy/Vuln Explorer (Sprint 129) owners should review AIAI-31-009 outputs after 2025-11-10 to ensure schema expectations match; Concelier CONSOLE (23-001..003) and AIRGAP (56/57/58) leads need Link-Not-Merge dates set during the 2025-11-11 checkpoint; Excititor mirror/air-gap teams should stage EXCITITOR-AIRGAP-56/57/58 implementation plans; Mirror CLI/Export Center teams should assemble design notes ahead of MIRROR-CRT-56-002/58-001 once the assembler kickoff happens.
|
||||
|
||||
## Wave detail references (2025-11-09)
|
||||
|
||||
- **110.A AdvisoryAI (docs/implplan/SPRINT_111_advisoryai.md)**
|
||||
DOCS-AIAI-31-004 remains DOING; DOCS-AIAI-31-005/006/008/009 are BLOCKED on CLI/POLICY/SBOM/DevOps dependencies; SBOM-AIAI-31-003 is still TODO awaiting SBOM-AIAI-31-001; AIAI-31-008 is TODO until guardrail knobs land, and AIAI-31-009 stays DOING with the expanded harness/perf coverage work.
|
||||
- **110.B Concelier (docs/implplan/SPRINT_112_concelier_i.md)**
|
||||
CONCELIER-AIAI-31-002 is TODO while CONCELIER-AIAI-31-003 is DOING; all air-gap (`CONCELIER-AIRGAP-56/57/58-*`), attestation (`CONCELIER-ATTEST-73-*`), and console (`CONCELIER-CONSOLE-23-*`) tracks remain TODO pending Link-Not-Merge (`CONCELIER-LNM-21-*`) and Cartographer schema (`CARTO-GRAPH-21-002`) delivery.
|
||||
- **110.C Excititor (docs/implplan/SPRINT_119_excititor_i.md)**
|
||||
EXCITITOR-AIAI-31-001 is DOING; EXCITITOR-AIAI-31-002/003/004, EXCITITOR-ATTEST-01-003/-73-001/-73-002, EXCITITOR-AIRGAP-56/57/58-* and EXCITITOR-CONN-TRUST-01-001 are all TODO awaiting the justification projection output plus Link-Not-Merge contracts.
|
||||
- **110.D Mirror (docs/implplan/SPRINT_125_mirror.md)**
|
||||
Every MIRROR-CRT-56/57/58 task is still TODO; DSSE/TUF, OCI bundle, time-anchor, CLI, and Export Center automation cannot start until the deterministic bundle assembler (MIRROR-CRT-56-001) is underway with EXPORT-OBS-51-001 and AIRGAP-TIME-57-001 owners confirmed.
|
||||
|
||||
## Downstream dependency rollup (snapshot: 2025-11-09)
|
||||
|
||||
| Wave | Dependent sprint(s) (selected) | Impact if 110.* slips |
|
||||
| --- | --- | --- |
|
||||
| 110.A AdvisoryAI | `SPRINT_130_scanner_surface.md`, `SPRINT_129_policy_reasoning.md`, `SPRINT_513_provenance.md`, `SPRINT_514_sovereign_crypto_enablement.md` | Scanner analyzers need AdvisoryAI schemas/feeds, Policy/Vuln Explorer tracks cannot expose advisory reasoning, and provenance/sovereign crypto programs remain paused until evidence contracts land. |
|
||||
| 110.B Concelier | `SPRINT_113_concelier_ii.md`, `SPRINT_114_concelier_iii.md`, `SPRINT_115_concelier_iv.md` | Link-Not-Merge schema + observation APIs gate Concelier graph, telemetry, and orchestrator waves; Console/advisor UIs stay blocked. |
|
||||
| 110.C Excititor | `SPRINT_120_excititor_ii.md` → `SPRINT_124_excititor_vi.md` | VEX chunk/attestation phases cannot progress until Excititor.I ships justification projections/guardrails, delaying Lens, Policy, and Advisory AI parity for VEX evidence. |
|
||||
| 110.D Mirror | `SPRINT_125_mirror.md` | Export Center, CLI, and air-gap bundles rely on MIRROR-CRT-56-001; no downstream mirror automation can begin until the deterministic assembler is complete. |
|
||||
|
||||
## Interlocks & owners
|
||||
|
||||
| Interlock | Participants | Needed artifact(s) | Status / notes (2025-11-09) |
|
||||
| --- | --- | --- | --- |
|
||||
| Advisory AI customer surfaces | Advisory AI Guild · SBOM Service Guild · CLI Guild · Policy Guild · DevOps Guild | `SBOM-AIAI-31-001`, `SBOM-AIAI-31-003`, `CLI-VULN-29-001`, `CLI-VEX-30-001`, `POLICY-ENGINE-31-001`, `DEVOPS-AIAI-31-001` | SBOM hand-off kit + CLI/Policy knobs still pending; DOCS-AIAI-31-005/006/008/009 stay blocked until these artifacts ship. |
|
||||
| Link-Not-Merge contract | Concelier Core/WebService Guilds · Cartographer Guild · Platform Events Guild | `CONCELIER-LNM-21-001`→`21-203`, `CARTO-GRAPH-21-002`, `CONCELIER-GRAPH-21-001/002`, `CONCELIER-CONSOLE-23-001..003` | Schema and observation APIs not started; Cartographer schema delivery remains the gate for CONCELIER-AIAI-31-002/003 and all console/air-gap tracks. |
|
||||
| VEX justification + attestation | Excititor WebService/Core Guilds · Observability Guild · Evidence Locker Guild · Cartographer Guild | `EXCITITOR-AIAI-31-001`→`31-004`, `EXCITITOR-ATTEST-01-003`, `EXCITITOR-ATTEST-73-001/002`, `EXCITITOR-AIRGAP-56/57/58-*`, `EXCITITOR-CONN-TRUST-01-001` | Justification enrichment is DOING; every downstream chunk/telemetry/attestation/mirror task remains TODO pending that output plus Link-Not-Merge contracts. |
|
||||
| Mirror evidence kickoff | Mirror Creator Guild · Exporter Guild · AirGap Time Guild · Security Guild · CLI Guild | `MIRROR-CRT-56-001`→`56-002`, `MIRROR-CRT-57-001/002`, `MIRROR-CRT-58-001/002`, `EXPORT-OBS-51-001`, `EXPORT-OBS-54-001`, `AIRGAP-TIME-57-001`, `CLI-AIRGAP-56-001`, `PROV-OBS-53-001` | No owner meeting yet; assembler (MIRROR-CRT-56-001) is still unscheduled, so DSSE/TUF, OCI, time-anchor, CLI, and Export Center hooks cannot start. |
|
||||
|
||||
### Upcoming checkpoints
|
||||
|
||||
| Date (UTC) | Focus | Agenda / expected exit |
|
||||
| --- | --- | --- |
|
||||
| 2025-11-10 | Advisory AI customer surfaces | Confirm SBOM-AIAI-31-001 delivery slot, align CLI-VULN/CLI-VEX scope owners, and capture POLICY-ENGINE-31-001 + DEVOPS-AIAI-31-001 readiness so DOCS-AIAI-31-005/006/008/009 can resume. |
|
||||
| 2025-11-11 | Link-Not-Merge contract | Cartographer to present CARTO-GRAPH-21-002 schema draft, Concelier to commit dates for CONCELIER-LNM-21-001..003 and CONCELIER-AIAI-31-002/003 telemetry wiring. |
|
||||
| 2025-11-11 | VEX justification + attestation | Walk EXCITITOR-AIAI-31-001 output, sequence EXCITITOR-AIAI-31-002/003, and lock attestation backlog order (`EXCITITOR-ATTEST-01-003`, `-73-001`, `-73-002`). |
|
||||
| 2025-11-12 | Mirror evidence kickoff | Assign MIRROR-CRT-56-001 lead, confirm EXPORT-OBS-51-001/AIRGAP-TIME-57-001 owners, and outline DSSE/TUF design reviews for MIRROR-CRT-56-002. |
|
||||
|
||||
## Coordination log
|
||||
|
||||
| Date | Notes |
|
||||
| --- | --- |
|
||||
| 2025-11-09 | Sprint file refreshed with wave detail references, interlocks, and risk log; waiting on 2025-11-10/11/12 syncs for SBOM/CLI/POLICY/DevOps, Link-Not-Merge, Excititor justification, and Mirror assembler commitments. |
|
||||
|
||||
## Risk log (2025-11-09)
|
||||
|
||||
| Risk | Impact | Mitigation / owner |
|
||||
| --- | --- | --- |
|
||||
| SBOM/CLI/Policy/DevOps deliverables slip past 2025-11-12 | Advisory AI CLI/docs remain blocked; downstream Scanner/Policy/Vuln Explorer sprints cannot validate schema feeds | Capture ETAs during 2025-11-10 interlock; SBOM/CLI/Policy/DevOps guild leads to publish commit dates and update sprint rows immediately |
|
||||
| Link-Not-Merge schema delays (`CONCELIER-LNM-21-*`, `CARTO-GRAPH-21-002`) | Concelier evidence APIs, console views, and Excititor graph consumers cannot progress; Advisory AI loses deterministic Concelier feeds | 2025-11-11 checkpoint to lock schema delivery; Cartographer + Concelier core owners to share migration plan and unblock CONCELIER-AIAI-31-002/003 |
|
||||
| Excititor justification/attestation backlog stalls | Advisory AI cannot cite VEX evidence, Excititor attestation/air-gap tasks remain TODO, Mirror parity slips | Excititor web/core leads to finish EXCITITOR-AIAI-31-001 and schedule EXCITITOR-AIAI-31-002/003 + ATTEST tasks during 2025-11-11 session |
|
||||
| Mirror assembler lacks staffing (`MIRROR-CRT-56-001`) | DSSE/TUF, OCI/time-anchor, CLI, Export Center automations cannot even start, blocking Wave 110.D and Sprint 125 entirely | 2025-11-12 kickoff must assign an owner and confirm EXPORT-OBS/AIRGAP-TIME prerequisites; track progress daily until assembler code is in flight |
|
||||
|
||||
@@ -8,11 +8,11 @@ Execute the tasks below strictly in order; each artifact unblocks the next analy
|
||||
|
||||
| Order | Task ID | State | Summary | Owner / Source | Depends On |
|
||||
| --- | --- | --- | --- | --- | --- |
|
||||
| 1 | `SCANNER-ANALYZERS-DENO-26-001` | TODO | Build the deterministic input normalizer + VFS merger for `deno.json(c)`, import maps, lockfiles, vendor trees, `$DENO_DIR`, and OCI layers so analyzers have a canonical file view. | Deno Analyzer Guild (src/Scanner/StellaOps.Scanner.Analyzers.Lang.Deno) | — |
|
||||
| 2 | `SCANNER-ANALYZERS-DENO-26-002` | TODO | Implement the module graph resolver covering static/dynamic imports, npm bridge, cache lookups, built-ins, WASM/JSON assertions, and annotate edges with their resolution provenance. | Deno Analyzer Guild (src/Scanner/StellaOps.Scanner.Analyzers.Lang.Deno) | SCANNER-ANALYZERS-DENO-26-001 |
|
||||
| 3 | `SCANNER-ANALYZERS-DENO-26-003` | TODO | Ship the npm/node compatibility adapter that maps `npm:` specifiers, evaluates `exports` conditionals, and logs builtin usage for policy overlays. | Deno Analyzer Guild (src/Scanner/StellaOps.Scanner.Analyzers.Lang.Deno) | SCANNER-ANALYZERS-DENO-26-002 |
|
||||
| 4 | `SCANNER-ANALYZERS-DENO-26-004` | TODO | Add the permission/capability analyzer covering FS/net/env/process/crypto/FFI/workers plus dynamic-import + literal fetch heuristics with reason codes. | Deno Analyzer Guild (src/Scanner/StellaOps.Scanner.Analyzers.Lang.Deno) | SCANNER-ANALYZERS-DENO-26-003 |
|
||||
| 5 | `SCANNER-ANALYZERS-DENO-26-005` | TODO | Build bundle/binary inspectors for eszip and `deno compile` executables to recover graphs, configs, embedded resources, and snapshots. | Deno Analyzer Guild (src/Scanner/StellaOps.Scanner.Analyzers.Lang.Deno) | SCANNER-ANALYZERS-DENO-26-004 |
|
||||
| 6 | `SCANNER-ANALYZERS-DENO-26-006` | TODO | Implement the OCI/container adapter that stitches per-layer Deno caches, vendor trees, and compiled binaries back into provenance-aware analyzer inputs. | Deno Analyzer Guild (src/Scanner/StellaOps.Scanner.Analyzers.Lang.Deno) | SCANNER-ANALYZERS-DENO-26-005 |
|
||||
| 7 | `SCANNER-ANALYZERS-DENO-26-007` | TODO | Produce AOC-compliant observation writers (entrypoints, modules, capability edges, workers, warnings, binaries) with deterministic reason codes. | Deno Analyzer Guild (src/Scanner/StellaOps.Scanner.Analyzers.Lang.Deno) | SCANNER-ANALYZERS-DENO-26-006 |
|
||||
| 1 | `SCANNER-ANALYZERS-DENO-26-001` | DONE | Build the deterministic input normalizer + VFS merger for `deno.json(c)`, import maps, lockfiles, vendor trees, `$DENO_DIR`, and OCI layers so analyzers have a canonical file view. | Deno Analyzer Guild (src/Scanner/StellaOps.Scanner.Analyzers.Lang.Deno) | — |
|
||||
| 2 | `SCANNER-ANALYZERS-DENO-26-002` | DONE | Implement the module graph resolver covering static/dynamic imports, npm bridge, cache lookups, built-ins, WASM/JSON assertions, and annotate edges with their resolution provenance. | Deno Analyzer Guild (src/Scanner/StellaOps.Scanner.Analyzers.Lang.Deno) | SCANNER-ANALYZERS-DENO-26-001 |
|
||||
| 3 | `SCANNER-ANALYZERS-DENO-26-003` | DONE | Ship the npm/node compatibility adapter that maps `npm:` specifiers, evaluates `exports` conditionals, and logs builtin usage for policy overlays. | Deno Analyzer Guild (src/Scanner/StellaOps.Scanner.Analyzers.Lang.Deno) | SCANNER-ANALYZERS-DENO-26-002 |
|
||||
| 4 | `SCANNER-ANALYZERS-DENO-26-004` | DONE | Add the permission/capability analyzer covering FS/net/env/process/crypto/FFI/workers plus dynamic-import + literal fetch heuristics with reason codes. | Deno Analyzer Guild (src/Scanner/StellaOps.Scanner.Analyzers.Lang.Deno) | SCANNER-ANALYZERS-DENO-26-003 |
|
||||
| 5 | `SCANNER-ANALYZERS-DENO-26-005` | DONE | Build bundle/binary inspectors for eszip and `deno compile` executables to recover graphs, configs, embedded resources, and snapshots. | Deno Analyzer Guild (src/Scanner/StellaOps.Scanner.Analyzers.Lang.Deno) | SCANNER-ANALYZERS-DENO-26-004 |
|
||||
| 6 | `SCANNER-ANALYZERS-DENO-26-006` | DONE | Implement the OCI/container adapter that stitches per-layer Deno caches, vendor trees, and compiled binaries back into provenance-aware analyzer inputs. | Deno Analyzer Guild (src/Scanner/StellaOps.Scanner.Analyzers.Lang.Deno) | SCANNER-ANALYZERS-DENO-26-005 |
|
||||
| 7 | `SCANNER-ANALYZERS-DENO-26-007` | DOING | Produce AOC-compliant observation writers (entrypoints, modules, capability edges, workers, warnings, binaries) with deterministic reason codes. | Deno Analyzer Guild (src/Scanner/StellaOps.Scanner.Analyzers.Lang.Deno) | SCANNER-ANALYZERS-DENO-26-006 |
|
||||
| 8 | `SCANNER-ANALYZERS-DENO-26-008` | TODO | Finalize fixture + benchmark suite (vendor/npm/FFI/worker/dynamic import/bundle/cache/container cases) validating analyzer determinism and performance. | Deno Analyzer Guild, QA Guild (src/Scanner/StellaOps.Scanner.Analyzers.Lang.Deno) | SCANNER-ANALYZERS-DENO-26-007 |
|
||||
|
||||
@@ -6,9 +6,160 @@
|
||||
|
||||
| Task ID | State | Summary | Owner / Source | Depends On |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| `SCANNER-ENG-0002` | TODO | Design the Node.js lockfile collector + CLI validator per `docs/benchmarks/scanner/scanning-gaps-stella-misses-from-competitors.md`, capturing Surface + policy requirements before implementation. | Scanner Guild, CLI Guild (docs/modules/scanner) | — |
|
||||
| `SCANNER-ENG-0003` | TODO | Design Python lockfile + editable-install parity checks with policy predicates and CLI workflow coverage as outlined in the gap analysis. | Python Analyzer Guild, CLI Guild (docs/modules/scanner) | — |
|
||||
| `SCANNER-ENG-0004` | TODO | Design Java lockfile ingestion/validation (Gradle/SBT collectors, CLI verb, policy hooks) to close comparison gaps. | Java Analyzer Guild, CLI Guild (docs/modules/scanner) | — |
|
||||
| `SCANNER-ENG-0005` | TODO | Enhance Go stripped-binary fallback inference design, including inferred module metadata + policy integration, per the gap analysis. | Go Analyzer Guild (docs/modules/scanner) | — |
|
||||
| `SCANNER-ENG-0006` | TODO | Expand Rust fingerprint coverage design (enriched fingerprint catalogue + policy controls) per the comparison matrix. | Rust Analyzer Guild (docs/modules/scanner) | — |
|
||||
| `SCANNER-ENG-0007` | TODO | Design the deterministic secret leak detection pipeline covering rule packaging, Policy Engine integration, and CLI workflow. | Scanner Guild, Policy Guild (docs/modules/scanner) | — |
|
||||
| `SCANNER-ENG-0002` | DONE (2025-11-09) | Design the Node.js lockfile collector + CLI validator per `docs/benchmarks/scanner/scanning-gaps-stella-misses-from-competitors.md`, capturing Surface + policy requirements before implementation. | Scanner Guild, CLI Guild (docs/modules/scanner) | — |
|
||||
| `SCANNER-ENG-0003` | DONE (2025-11-09) | Design Python lockfile + editable-install parity checks with policy predicates and CLI workflow coverage as outlined in the gap analysis. | Python Analyzer Guild, CLI Guild (docs/modules/scanner) | — |
|
||||
| `SCANNER-ENG-0004` | DONE (2025-11-09) | Design Java lockfile ingestion/validation (Gradle/SBT collectors, CLI verb, policy hooks) to close comparison gaps. | Java Analyzer Guild, CLI Guild (docs/modules/scanner) | — |
|
||||
| `SCANNER-ENG-0005` | DONE (2025-11-09) | Enhance Go stripped-binary fallback inference design, including inferred module metadata + policy integration, per the gap analysis. | Go Analyzer Guild (docs/modules/scanner) | — |
|
||||
| `SCANNER-ENG-0006` | DONE (2025-11-09) | Expand Rust fingerprint coverage design (enriched fingerprint catalogue + policy controls) per the comparison matrix. | Rust Analyzer Guild (docs/modules/scanner) | — |
|
||||
| `SCANNER-ENG-0007` | DONE (2025-11-09) | Design the deterministic secret leak detection pipeline covering rule packaging, Policy Engine integration, and CLI workflow. | Scanner Guild, Policy Guild (docs/modules/scanner) | — |
|
||||
|
||||
> 2025-11-09: The gap designs below capture analyzer, Surface, CLI, and policy contracts for SCANNER-ENG-0002…0007; tasks were taken DOING → DONE after this review.
|
||||
|
||||
## Implementation progress (2025-11-09)
|
||||
|
||||
- Gradle/Maven lock ingestion is now wired into `JavaLanguageAnalyzer`: `JavaLockFileCollector` sorts lock metadata deterministically, merges it with archive findings (`lockConfiguration`, `lockRepository`, `lockResolved`), and emits declared-only components (with `declaredOnly=true`, `lockSource`, `lockLocator`) whenever jars are missing. CLI/Surface telemetry tags were updated to carry per-language declared/missing counters.
|
||||
- `stella java lock-validate` shares the `HandleLanguageLockValidateAsync` helper with Node/Python, has table/JSON output parity, and is documented alongside the scanner README + CLI guide (including the new metric `stellaops.cli.java.lock_validate.count`). Tests now cover the Ruby/Node/Java lock workflows end-to-end via `CommandHandlersTests`.
|
||||
|
||||
## Design outcomes
|
||||
|
||||
### SCANNER-ENG-0002 — Node.js lockfile collector + CLI validator
|
||||
|
||||
**Scope & goals**
|
||||
- Provide deterministic ingestion of `pnpm-lock.yaml`, `package-lock.json`, and `yarn.lock` so declared dependencies are preserved even when `node_modules` is absent.
|
||||
- Offer a CLI validator that runs without scheduling a scan, reusing the same collector and Surface safety rails.
|
||||
|
||||
**Design decisions**
|
||||
- Add `NodeLockfileCollector` under `StellaOps.Scanner.Analyzers.Lang.Node`. The collector normalises manifests into a shared model (`package name`, `version`, `resolved`, `integrity`, `registry`, `workspace path`) and emits `DeclaredOnly = true` components stored beside installed fragments (`LayerComponentFragment.DeclaredSources`).
|
||||
- Reuse `LanguageAnalyzerContext` merge rules so installed packages supersede declared-only entries while retaining discrepancies for policy.
|
||||
- Gate execution through `Surface.Validation` (`scanner.lockfiles.node.*` knobs) that enforce max lockfile size, workspace limits, and registry allowlists; violations fail fast with deterministic error IDs.
|
||||
- Private registries referenced in lockfiles must use `secret://` handles. `Surface.Secrets` resolves these handles before validation and the resolved metadata (never the secret) is attached to the collector context for auditing.
|
||||
- EntryTrace usage hints annotate runtime packages; when a package is used at runtime but missing from the lockfile, the merge step tags it with `UsageWithoutDeclaration`.
|
||||
|
||||
**CLI, policy, docs**
|
||||
- Add `stella node lock-validate [path] --format {auto|pnpm|npm|yarn}` that runs locally, reuses Surface controls, and returns canonical JSON + table summaries. The CLI inherits `--surface-config` so air-gapped configs stay consistent.
|
||||
- Scanner/WebService gains `--node-lockfiles` / `SCANNER__NODE__LOCKFILES__ENABLED` toggles to control ingestion during full scans.
|
||||
- Policy Engine receives predicates: `node.lock.declaredMissing`, `node.lock.registryDisallowed`, `node.lock.declarationOnly`. Templates show how to fail on disallowed registries while only warning on declared-only findings that never reach runtime.
|
||||
- Update `docs/modules/scanner/architecture.md` and policy DSL appendices with the new evidence flags and CLI workflow.
|
||||
|
||||
**Testing, telemetry, rollout**
|
||||
- Golden fixtures for pnpm v8, npm v9, and yarn berry lockfiles live under `tests/Scanner.Analyzers.Node/__fixtures__/lockfiles`. Deterministic snapshots are asserted in both analyzer and CLI tests.
|
||||
- Add integration coverage in `tests/Scanner.Cli.Node` verifying exit codes and explain output for mismatched packages/registries.
|
||||
- Emit counters (`scanner.node.lock.declared`, `scanner.node.lock.mismatch`, `scanner.node.lock.registry_blocked`) plus structured logs keyed by lockfile digest.
|
||||
- Offline Kit ships the parser tables and CLI binary help under `offline/scanner/node-lockfiles/README.md`.
|
||||
|
||||
**Implementation status (2025-11-09)**
|
||||
- Lockfile declarations now emit `DeclaredOnly` components in `StellaOps.Scanner.Analyzers.Lang.Node` with lock source/locator metadata and deterministic evidence for policy use.
|
||||
- CLI verb `stella node lock-validate` inspects lockfiles locally, rendering declared-only/missing-lock summaries and emitting `stellaops.cli.node.lock_validate.count` telemetry.
|
||||
- Node analyzer determinism fixtures updated with declared-only coverage; CLI unit suite exercises the new handler.
|
||||
- Python analyzer ingests `requirements*.txt`, `Pipfile.lock`, and `poetry.lock`, tagging installed distributions with `lockSource` metadata and creating declared-only components. `stella python lock-validate` mirrors the workflow for offline validation and records `stellaops.cli.python.lock_validate.count`.
|
||||
|
||||
### SCANNER-ENG-0003 — Python lockfile + editable-install parity
|
||||
|
||||
**Scope & goals**
|
||||
- Parse Python lockfiles (`poetry.lock`, `Pipfile.lock`, hashed `requirements*.txt`) to capture declared graphs pre-install.
|
||||
- Detect editable installs and local path references so policy can assert parity between lockfiles and runtime contents.
|
||||
|
||||
**Design decisions**
|
||||
- Introduce `PythonLockfileCollector` in `StellaOps.Scanner.Analyzers.Lang.Python`, capable of reading Poetry, Pipenv, pip-tools, and raw requirements syntax (including environment markers, extras, hashes, VCS refs).
|
||||
- Extend the collector with an `EditableResolver` that inspects lockfile entries (`path =`, `editable = true`, `-e ./pkg`) and consults `Surface.FS` to normalise the referenced directory, capturing `EditablePath`, `SourceDigest`, and `VcsRef` metadata.
|
||||
- Merge results with installed `*.dist-info` data using `LanguageAnalyzerContext`. Installed evidence overrides declared-only components; editable packages missing from the artifact layer are tagged `EditableMissing`.
|
||||
- `Surface.Validation` adds knobs `scanner.lockfiles.python.maxBytes`, `scanner.lockfiles.python.allowedIndexes`, and ensures hashes are present when policy mandates repeatable environments. Private index credentials are provided via `Surface.Secrets` and never persisted.
|
||||
|
||||
**CLI, policy, docs**
|
||||
- New CLI verb `stella python lock-validate` mirrors the Node workflow, validates editable references resolve within the checked-out tree, and emits parity diagnostics.
|
||||
- Scanner runs accept `--python-lockfiles` to toggle ingestion per tenant.
|
||||
- Policy predicates: `python.lock.declaredMissing`, `python.lock.editableUnpinned`, `python.lock.indexDisallowed`. Editable packages missing from the filesystem can be set to fail builds or raise waivers.
|
||||
- Document the workflow in `docs/modules/scanner/architecture.md` and the policy cookbook, including guidance on handling build-system backends.
|
||||
|
||||
**Testing, telemetry, rollout**
|
||||
- Fixtures covering Poetry 1.6, Pipenv 2024.x, `requirements.txt` with markers, and mixed editable/VCS entries live beside the analyzer tests.
|
||||
- CLI golden output asserts deterministic ordering and masking of secrets in URLs.
|
||||
- Metrics: `scanner.python.lock.declared`, `scanner.python.lock.editable`, `scanner.python.lock.failures`.
|
||||
- Offline Kit bundles include parser definitions and sample policies to keep air-gapped tenants aligned.
|
||||
|
||||
### SCANNER-ENG-0004 — Java/Gradle/SBT lockfile ingestion & validation
|
||||
|
||||
**Scope & goals**
|
||||
- Capture Gradle, Maven, and SBT dependency locks before artifacts are built, along with repository provenance and configuration scopes.
|
||||
- Provide CLI validation and policy predicates enforcing repository allowlists and declared/runtime parity.
|
||||
|
||||
**Design decisions**
|
||||
- Add collectors: `GradleLockfileCollector` (reads `gradle.lockfile` and `gradle/dependency-locks/*.lock`), `MavenLockfileCollector` (parses `pom.xml`/`pom.lock` + dependencyManagement overrides), and `SbtLockfileCollector` (reads Ivy resolution outputs or `dependencies.lock`).
|
||||
- Each collector emits normalized records keyed by `groupId:artifactId:version` plus config scope (`compileClasspath`, `runtimeClasspath`, etc.), repository URI, checksum, and optional classifier. Records are stored as `DeclaredOnly` fragments associated with their workspace path.
|
||||
- `Surface.Validation` enforces file-size limits, repository allowlists (`scanner.lockfiles.java.allowedRepos`), and optional checksum requirements. Private Maven credentials flow through `Surface.Secrets`.
|
||||
- `JavaLanguageAnalyzer` merges declared entries with installed archives. Runtime usage from EntryTrace is attached so policies can prioritize gaps that reach runtime.
|
||||
|
||||
**CLI, policy, docs**
|
||||
- CLI verb `stella java lock-validate` supports Gradle/Maven/SBT modes, prints mismatched dependencies, and checks repository policy.
|
||||
- Scanner flags `--java-lockfiles` or env `SCANNER__JAVA__LOCKFILES__ENABLED` gate ingestion. Lockfile artifacts are uploaded to Surface.FS for evidence replay.
|
||||
- Policy predicates: `java.lock.declaredMissing`, `java.lock.repoDisallowed`, `java.lock.unpinned` (no checksum). Explain traces cite repository + config scope for each discrepancy.
|
||||
- Docs: update scanner module dossier and policy template library with repository governance examples.
|
||||
|
||||
**Testing, telemetry, rollout**
|
||||
- Fixtures derived from sample Gradle multi-projects, Maven BOM hierarchies, and SBT builds validate parser coverage and CLI messaging.
|
||||
- Metrics `scanner.java.lock.declared`, `scanner.java.lock.missing`, `scanner.java.lock.repo_blocked` feed the observability dashboards.
|
||||
- Offline kits include parser grammars and CLI docs so air-gapped tenants can enforce repo policies without SaaS dependencies.
|
||||
|
||||
### SCANNER-ENG-0005 — Go stripped-binary fallback inference
|
||||
|
||||
**Scope & goals**
|
||||
- Enrich the stripped-binary fallback so Go modules remain explainable even without embedded `buildinfo`, and give Policy Engine knobs to treat inferred evidence differently.
|
||||
|
||||
**Design decisions**
|
||||
- Extend `GoBinaryScanner` with an inference pipeline that, when build info is absent, parses ELF/Mach-O symbol tables and DWARF data using the existing `ElfSharp` bindings. Symbols feed into a new `GoSymbolInferenceEngine` that matches against a signed `GoFingerprintCatalog` under `StellaOps.Scanner.Analyzers.Lang.Go.Fingerprints`.
|
||||
- Inferred results carry `Confidence` (0–1), matched symbol counts, and reasons (`BuildInfoMissing`, `SymbolMatches`, `PkgPathFallback`). Records are emitted as `InferredModule` metadata alongside hashed fallback components.
|
||||
- Update fragment schemas so DSSE-composed BOMs include both the hashed fallback and the inference summary, enabling deterministic replay.
|
||||
- `Surface.Validation` exposes `scanner.analyzers.go.fallback.enabled`, `scanner.analyzers.go.fallback.maxSymbolBytes`, ensuring workloads can opt out or constrain processing time.
|
||||
|
||||
**Policy, CLI, docs**
|
||||
- Policy predicates `go.module.inferenceConfidence` and `go.module.hashOnly` let tenants fail when only hashed provenance exists or warn when inference confidence < threshold.
|
||||
- CLI flag `--go-fallback-detail` (and corresponding API query) prints hashed vs inferred modules, confidence, and remediation hints (e.g., rebuild with `-buildvcs`).
|
||||
- Documentation updates cover inference details, how confidence feeds lattice weights, and how to author waivers.
|
||||
|
||||
**Testing, telemetry, rollout**
|
||||
- Add stripped binary fixtures (Linux, macOS) plus intentionally obfuscated samples. Tests assert deterministic inference and hashing.
|
||||
- Metrics `scanner.go.inference.count`, `scanner.go.inference.confidence_bucket` ensure observability; logs include `imageDigest`, `binaryPath`, `confidence`.
|
||||
- Offline Kit bundles the fingerprint catalog and inference changelog so air-gapped tenants can audit provenance.
|
||||
|
||||
### SCANNER-ENG-0006 — Rust fingerprint coverage expansion
|
||||
|
||||
**Scope & goals**
|
||||
- Improve Rust evidence for stripped binaries by expanding fingerprint sources, symbol parsing, and policy controls over heuristic findings.
|
||||
|
||||
**Design decisions**
|
||||
- Build a new `RustFingerprintCatalog` signed and versioned, fed by Cargo crate metadata, community hash contributions, and curated fingerprints from StellaOps scans. Catalog lives under `StellaOps.Scanner.Analyzers.Lang.Rust.Fingerprints` with deterministic ordering.
|
||||
- Extend `RustAnalyzerCollector` with symbol parsing (DWARF, ELF build IDs) via `SymbolGraphResolver`. Resolver correlates crate sections, monomorphized symbol prefixes, and `#[panic_handler]` markers to infer crate names and versions.
|
||||
- Emit inference metadata (`fingerprintId`, `confidence`, `symbolEvidence[]`) alongside hashed fallbacks. Authoritative Cargo.lock data (when present) still wins in merges.
|
||||
- `Surface.Validation` adds toggles for fingerprint freshness and maximum catalog size per tenant. Offline bundles deliver catalog updates signed via DSSE.
|
||||
|
||||
**Policy, CLI, docs**
|
||||
- Policy predicates: `rust.fingerprint.confidence`, `rust.fingerprint.catalogAgeDays`. Templates show how to warn when only heuristic data exists, or fail if catalog updates are stale.
|
||||
- CLI flag `--rust-fingerprint-detail` prints authoritative vs inferred crates, symbol samples, and guidance.
|
||||
- Documentation (scanner module + policy guide) explains how inference is stored, how catalog publishing works, and how to tune policy weights.
|
||||
|
||||
**Testing, telemetry, rollout**
|
||||
- Add fixtures for stripped Rust binaries across editions (2018–2024) and with/without LTO. Determinism tests compare catalog revisions and inference outputs.
|
||||
- Metrics `scanner.rust.fingerprint.authoritative`, `scanner.rust.fingerprint.inferred`, `scanner.rust.fingerprint.catalog_version` feed dashboards and alerts.
|
||||
- Offline kit updates include catalog packages, verification instructions, and waiver templates tied to predicate names.
|
||||
|
||||
### SCANNER-ENG-0007 — Deterministic secret leak detection pipeline
|
||||
|
||||
**Scope & goals**
|
||||
- Provide first-party secret leak detection that matches competitor capabilities while preserving deterministic, offline-friendly execution and explainability.
|
||||
|
||||
**Design decisions**
|
||||
- Introduce `StellaOps.Scanner.Analyzers.Secrets`, a restart-time plug-in that consumes rule bundles (`ruleset.tgz`) signed with DSSE and versioned (semantic version + hash). Bundles live under `plugins/scanner/secrets/rules/<version>`.
|
||||
- Rule bundles contain deterministic regex/entropy definitions, context windows, and masking directives. A rule index is generated at build time to guarantee deterministic ordering.
|
||||
- Analyzer executes after Surface validation of each file/layer. Files pass through a streaming matcher that outputs `SecretLeakEvidence` (rule id, severity, confidence, file path, byte ranges, masking applied). Findings persist in `ScanAnalysisStore` and align with DSSE exports.
|
||||
- `Surface.Validation` introduces `scanner.secrets.rules.bundle`, `scanner.secrets.maxFileBytes`, and `scanner.secrets.targetGlobs`. `Surface.Secrets` supplies allowlist tokens (e.g., approved test keys) without exposing plaintext to analyzers.
|
||||
- Events/attestations: findings optionally published via the existing Redis events, and Export Center bundles include masked evidence plus rule metadata.
|
||||
|
||||
**CLI, policy, docs**
|
||||
- Add `stella secrets scan [path|image]` plus `--secrets` flag on `stella scan` to run the analyzer inline. CLI output redacts payloads, shows rule IDs, severity, and remediation hints.
|
||||
- Policy Engine ingests `secret.leak` evidence, including `ruleId`, `confidence`, `masking.applied`, enabling predicates like `secret.leak.highConfidence`, `secret.leak.ruleDisabled`. Templates cover severities, approvals, and ticket automation.
|
||||
- Documentation updates: scanner module dossier (new analyzer), policy cookbook (rule management), and Offline Kit guide (bundling rule updates).
|
||||
|
||||
**Testing, telemetry, rollout**
|
||||
- Rule-pack regression tests ensure deterministic matching and masking; analyzer unit tests cover regex + entropy combos, while integration tests run across sample repositories and OCI layers.
|
||||
- Metrics: `scanner.secrets.ruleset.version`, `scanner.secrets.findings.total`, `scanner.secrets.findings.high_confidence`. Logs include rule ID, masked hash, and file digests for auditing.
|
||||
- Offline Kit delivers the signed ruleset catalog, upgrade guide, and policy defaults so fully air-gapped tenants can keep pace without internet access.
|
||||
|
||||
@@ -13,11 +13,16 @@
|
||||
| `SCANNER-ENG-0012` | TODO | Evaluate Dart analyzer requirements (pubspec parsing, AOT artifacts) and split implementation tasks. | Language Analyzer Guild (src/Scanner/StellaOps.Scanner.Analyzers.Lang.Dart) | — |
|
||||
| `SCANNER-ENG-0013` | TODO | Plan Swift Package Manager coverage (Package.resolved, xcframeworks, runtime hints) with policy hooks. | Swift Analyzer Guild (src/Scanner/StellaOps.Scanner.Analyzers.Lang.Swift) | — |
|
||||
| `SCANNER-ENG-0014` | TODO | Align Kubernetes/VM target coverage between Scanner and Zastava per comparison findings; publish joint roadmap. | Runtime Guild, Zastava Guild (docs/modules/scanner) | — |
|
||||
| `SCANNER-ENG-0015` | TODO | Document DSSE/Rekor operator enablement guidance and rollout levers surfaced in the gap analysis. | Export Center Guild, Scanner Guild (docs/modules/scanner) | — |
|
||||
| `SCANNER-ENG-0015` | DOING (2025-11-09) | Document DSSE/Rekor operator enablement guidance and rollout levers surfaced in the gap analysis. | Export Center Guild, Scanner Guild (docs/modules/scanner) | — |
|
||||
| `SCANNER-ENG-0016` | DOING (2025-11-02) | Implement `RubyLockCollector` + vendor cache ingestion per design §4.1–4.3. | Ruby Analyzer Guild (src/Scanner/StellaOps.Scanner.Analyzers.Lang.Ruby) | SCANNER-ENG-0009 |
|
||||
| `SCANNER-ENG-0017` | TODO | Build the runtime require/autoload graph builder with tree-sitter Ruby per design §4.4 and integrate EntryTrace hints. | Ruby Analyzer Guild (src/Scanner/StellaOps.Scanner.Analyzers.Lang.Ruby) | SCANNER-ENG-0016 |
|
||||
| `SCANNER-ENG-0018` | TODO | Emit Ruby capability + framework surface signals as defined in design §4.5 with policy predicate hooks. | Ruby Analyzer Guild (src/Scanner/StellaOps.Scanner.Analyzers.Lang.Ruby) | SCANNER-ENG-0017 |
|
||||
| `SCANNER-ENG-0019` | TODO | Ship Ruby CLI verbs (`stella ruby inspect|resolve`) and Offline Kit packaging per design §4.6. | Ruby Analyzer Guild, CLI Guild (src/Scanner/StellaOps.Scanner.Analyzers.Lang.Ruby) | SCANNER-ENG-0016..0018 |
|
||||
| `SCANNER-ENG-0017` | DONE (2025-11-09) | Build the runtime require/autoload graph builder with tree-sitter Ruby per design §4.4 and integrate EntryTrace hints. | Ruby Analyzer Guild (src/Scanner/StellaOps.Scanner.Analyzers.Lang.Ruby) | SCANNER-ENG-0016 |
|
||||
| `SCANNER-ENG-0018` | DONE (2025-11-09) | Emit Ruby capability + framework surface signals as defined in design §4.5 with policy predicate hooks. | Ruby Analyzer Guild (src/Scanner/StellaOps.Scanner.Analyzers.Lang.Ruby) | SCANNER-ENG-0017 |
|
||||
| `SCANNER-ENG-0019` | DOING (2025-11-10) | Ship Ruby CLI verbs (`stella ruby inspect|resolve`) and Offline Kit packaging per design §4.6. | Ruby Analyzer Guild, CLI Guild (src/Scanner/StellaOps.Scanner.Analyzers.Lang.Ruby) | SCANNER-ENG-0016..0018 |
|
||||
| `SCANNER-LIC-0001` | DOING (2025-11-02) | Vet tree-sitter Ruby licensing + Offline Kit packaging requirements and document SPDX posture. | Scanner Guild, Legal Guild (docs/modules/scanner) | SCANNER-ENG-0016 |
|
||||
| `SCANNER-POLICY-0001` | TODO | Define Policy Engine predicates for Ruby groups/capabilities and align lattice weights. | Policy Guild, Ruby Analyzer Guild (docs/modules/scanner) | SCANNER-ENG-0018 |
|
||||
| `SCANNER-CLI-0001` | TODO | Coordinate CLI UX/help text for new Ruby verbs and update CLI docs/golden outputs. | CLI Guild, Ruby Analyzer Guild (src/Cli/StellaOps.Cli) | SCANNER-ENG-0019 |
|
||||
| `SCANNER-CLI-0001` | DOING (2025-11-09) | Coordinate CLI UX/help text for new Ruby verbs and update CLI docs/golden outputs. | CLI Guild, Ruby Analyzer Guild (src/Cli/StellaOps.Cli) | SCANNER-ENG-0019 |
|
||||
|
||||
### Updates — 2025-11-09
|
||||
|
||||
- `SCANNER-CLI-0001`: Completed Spectre table wrapping fix for runtime/lockfile columns, expanded Ruby resolve JSON assertions, removed ad-hoc debug artifacts, and drafted CLI docs covering `stellaops-cli ruby inspect|resolve`. Pending: final verification + handoff once docs/tests merge.
|
||||
- `SCANNER-CLI-0001`: Wired `stellaops-cli ruby inspect|resolve` into `CommandFactory` so the verbs are available via `System.CommandLine` with the expected `--root`, `--image/--scan-id`, and `--format` options; `dotnet test ... --filter Ruby` passes.
|
||||
|
||||
@@ -13,4 +13,48 @@ This file now only tracks the runtime & signals status snapshot. Active backlog
|
||||
| 140.C Signals | Signals Guild · Authority Guild (for scopes) · Runtime Guild | Sprint 120.A – AirGap; Sprint 130.A – Scanner | DOING | API skeleton and callgraph ingestion are active; runtime facts endpoint still depends on the same shared prerequisites. |
|
||||
| 140.D Zastava | Zastava Observer/Webhook Guilds · Security Guild | Sprint 120.A – AirGap; Sprint 130.A – Scanner | TODO | Surface.FS integration waits on Scanner surface caches; prep sealed-mode env helpers meanwhile. |
|
||||
|
||||
# Status snapshot (2025-11-09)
|
||||
|
||||
- **140.A Graph** – GRAPH-INDEX-28-007/008/009/010 remain TODO while Scanner surface artifacts and SBOM projection schemas are outstanding; no clustering/backfill/fixture work has started.
|
||||
- **140.B SbomService** – Advisory AI, console, and orchestrator tracks stay TODO; SBOM-SERVICE-21-001..004 are BLOCKED until Concelier Link-Not-Merge (`CONCELIER-GRAPH-21-001`) + Cartographer schema (`CARTO-GRAPH-21-002`) land.
|
||||
- **140.C Signals** – SIGNALS-24-001 now complete (host, RBAC, sealed-mode readiness, `/signals/facts/{subject}`); SIGNALS-24-002 added callgraph retrieval APIs but still needs CAS promotion; SIGNALS-24-003 accepts JSON + NDJSON runtime uploads, yet NDJSON provenance/context wiring remains TODO. Scoring/cache work (SIGNALS-24-004/005) is still BLOCKED pending runtime feed availability (target 2025-11-09).
|
||||
- **140.D Zastava** – ZASTAVA-ENV-01/02, ZASTAVA-SECRETS-01/02, and ZASTAVA-SURFACE-01/02 are still TODO because Surface.FS cache outputs from Scanner aren’t published; guilds limited to design/prep.
|
||||
|
||||
# Blockers & coordination
|
||||
|
||||
- **Concelier Link-Not-Merge / Cartographer schemas** – SBOM-SERVICE-21-001..004 cannot start until `CONCELIER-GRAPH-21-001` and `CARTO-GRAPH-21-002` deliver the projection payloads.
|
||||
- **Scanner surface artifacts** – GRAPH-INDEX-28-007+ and all ZASTAVA-SURFACE tasks depend on Sprint 130 analyzer outputs and cached layer metadata; need updated ETA from Scanner guild.
|
||||
- **Signals host merge** – SIGNALS-24-003/004/005 remain blocked until SIGNALS-24-001/002 merge and Authority scope work (`AUTH-SIG-26-001`) is validated with Runtime guild.
|
||||
|
||||
# Next actions (target: 2025-11-12)
|
||||
|
||||
| Owner(s) | Action |
|
||||
| --- | --- |
|
||||
| Graph Indexer Guild | Hold design sync with Scanner Surface + SBOM Service owners to lock artifact delivery dates; prep clustering job scaffolds so work can start once feeds land. |
|
||||
| SBOM Service Guild | Finalize projection schema doc with Concelier/Cartographer, then flip SBOM-SERVICE-21-001 to DOING and align SBOM-AIAI-31-001 with Sprint 111 requirements. |
|
||||
| Signals Guild | Land SIGNALS-24-001/002 PRs, then immediately kick off SIGNALS-24-003; coordinate scoring/cache roadmap with Runtime + Data Science guilds. |
|
||||
| Zastava Guilds | Draft Surface.Env helper adoption plan and ensure Surface.Secrets references are wired so implementation can begin when Surface.FS caches publish. |
|
||||
|
||||
# Downstream dependency rollup (snapshot: 2025-11-09)
|
||||
|
||||
| Track | Dependent sprint(s) | Impact if delayed |
|
||||
| --- | --- | --- |
|
||||
| 140.A Graph | `docs/implplan/SPRINT_141_graph.md` (Graph clustering/backfill) and downstream Graph UI overlays | Graph insights, policy overlays, and runtime clustering views cannot progress without GRAPH-INDEX-28-007+ landing. |
|
||||
| 140.B SbomService | `docs/implplan/SPRINT_142_sbomservice.md`, Advisory AI (Sprint 111), Policy/Vuln Explorer feeds | SBOM projections/events stay unavailable, blocking Advisory AI remedation heuristics, policy joins, and Vuln Explorer candidate generation. |
|
||||
| 140.C Signals | `docs/implplan/SPRINT_143_signals.md` plus Runtime/Reachability dashboards | Reachability scoring, cache/event layers, and runtime facts outputs cannot start until SIGNALS-24-001/002 merge and Scanner runtime data flows. |
|
||||
| 140.D Zastava | `docs/implplan/SPRINT_144_zastava.md`, Runtime admission enforcement | Surface-integrated drift/admission hooks remain stalled; sealed-mode env helpers cannot ship without Surface.FS metadata. |
|
||||
|
||||
# Risk log
|
||||
|
||||
| Risk | Impact | Mitigation / owner |
|
||||
| --- | --- | --- |
|
||||
| Concelier Link-Not-Merge schema slips | SBOM-SERVICE-21-001..004 + Advisory AI SBOM endpoints stay blocked | Concelier + Cartographer guilds to publish CARTO-GRAPH-21-002 ETA during next coordination call; SBOM guild to prep schema doc meanwhile. |
|
||||
| Scanner surface artifact delay | GRAPH-INDEX-28-007+ and ZASTAVA-SURFACE-* cannot even start | Scanner guild to deliver analyzer artifact roadmap; Graph/Zastava teams to prepare mocks/tests in advance. |
|
||||
| Signals host/callgraph merge misses 2025-11-09 | SIGNALS-24-003/004/005 remain blocked, pushing reachability scoring past sprint goals | Signals + Authority guilds to prioritize AUTH-SIG-26-001 review and merge SIGNALS-24-001/002 before 2025-11-10 standup. |
|
||||
|
||||
# Coordination log
|
||||
|
||||
| Date | Notes |
|
||||
| --- | --- |
|
||||
| 2025-11-09 | Sprint 140 snapshot refreshed; awaiting Scanner surface artifact ETA, Concelier/CARTO schema delivery, and Signals host merge before any wave can advance to DOING. |
|
||||
# Sprint 140 - Runtime & Signals
|
||||
|
||||
@@ -10,11 +10,14 @@ Notes:
|
||||
- 2025-10-29: JSON parsers for Java/Node.js/Python/Go implemented; artifacts stored on filesystem with SHA-256 and callgraphs upserted into Mongo.
|
||||
Task ID | State | Task description | Owners (Source)
|
||||
--- | --- | --- | ---
|
||||
SIGNALS-24-001 | DOING (2025-11-07) | Stand up Signals API skeleton with RBAC, sealed-mode config, DPoP/mTLS enforcement, and `/facts` scaffolding so downstream ingestion work can begin. Dependencies: AUTH-SIG-26-001. | Signals Guild, Authority Guild (src/Signals/StellaOps.Signals)
|
||||
SIGNALS-24-001 | DONE (2025-11-09) | Stand up Signals API skeleton with RBAC, sealed-mode config, DPoP/mTLS enforcement, and `/facts` scaffolding so downstream ingestion work can begin. Dependencies: AUTH-SIG-26-001. | Signals Guild, Authority Guild (src/Signals/StellaOps.Signals)
|
||||
> 2025-11-09: Signals host now registers sealed-mode evidence validation, exposes `/readyz`/`/status` indicators, enforces scope policies, and adds `/signals/facts/{subjectKey}` retrieval plus runtime-facts ingestion backing services.
|
||||
SIGNALS-24-002 | DOING (2025-11-07) | Implement callgraph ingestion/normalization (Java/Node/Python/Go) with CAS persistence and retrieval APIs to feed reachability scoring. Dependencies: SIGNALS-24-001. | Signals Guild (src/Signals/StellaOps.Signals)
|
||||
SIGNALS-24-003 | BLOCKED (2025-10-27) | Implement runtime facts ingestion endpoint and normalizer (process, sockets, container metadata) populating `context_facts` with AOC provenance.<br>2025-10-27: Depends on `SIGNALS-24-001` for base API host and authentication plumbing. | Signals Guild, Runtime Guild (src/Signals/StellaOps.Signals)
|
||||
> 2025-11-09: Added `/signals/callgraphs/{id}` retrieval, sealed-mode gating, and CAS-backed artifact metadata responses; remaining work is CAS bucket promotion + signed graph manifests.
|
||||
SIGNALS-24-003 | DOING (2025-11-09) | Implement runtime facts ingestion endpoint and normalizer (process, sockets, container metadata) populating `context_facts` with AOC provenance.<br>2025-11-09: Initial JSON ingestion service + persistence landed; NDJSON/gzip + context enrichment remain TODO. | Signals Guild, Runtime Guild (src/Signals/StellaOps.Signals)
|
||||
> 2025-11-07: Waiting on SIGNALS-24-001 / SIGNALS-24-002 DOING work to land before flipping this to DOING.
|
||||
> 2025-11-07: Upstream SIGNALS-24-001 / SIGNALS-24-002 now DOING; this flips to DOING once host + callgraph ingestion merge.
|
||||
> 2025-11-08: Targeting 2025-11-09 merge for SIGNALS-24-001/002; schema + AOC contract drafted so SIGNALS-24-003 can move to DOING immediately after those PRs land (dependencies confirmed, none missing).
|
||||
> 2025-11-09: Added runtime facts ingestion service + endpoint, aggregated runtime hit storage, and unit tests; next steps are NDJSON/gzip ingestion and provenance metadata wiring.
|
||||
SIGNALS-24-004 | BLOCKED (2025-10-27) | Deliver reachability scoring engine producing states/scores and writing to `reachability_facts`; expose configuration for weights. Dependencies: SIGNALS-24-003.<br>2025-10-27: Upstream ingestion pipelines (`SIGNALS-24-002/003`) blocked; scoring engine cannot proceed. | Signals Guild, Data Science (src/Signals/StellaOps.Signals)
|
||||
SIGNALS-24-005 | BLOCKED (2025-10-27) | Implement Redis caches (`reachability_cache:*`), invalidation on new facts, and publish `signals.fact.updated` events. Dependencies: SIGNALS-24-004.<br>2025-10-27: Awaiting scoring engine and ingestion layers before wiring cache/events. | Signals Guild, Platform Events Guild (src/Signals/StellaOps.Signals)
|
||||
SIGNALS-24-005 | BLOCKED (2025-10-27) | Implement Redis caches (`reachability_cache:*`), invalidation on new facts, and publish `signals.fact.updated` events. Dependencies: SIGNALS-24-004.<br>2025-10-27: Awaiting scoring engine and ingestion layers before wiring cache/events. | Signals Guild, Platform Events Guild (src/Signals/StellaOps.Signals)
|
||||
|
||||
Reference in New Issue
Block a user