stella-ops.org/git.stella-ops.org
1
0
Fork 0
Code Issues Pull Requests Actions 2 Releases Wiki Activity

feat: add stella-callgraph-node for JavaScript/TypeScript call graph extraction

Browse Source 
- Implemented a new tool `stella-callgraph-node` that extracts call graphs from JavaScript/TypeScript projects using Babel AST.
- Added command-line interface with options for JSON output and help.
- Included functionality to analyze project structure, detect functions, and build call graphs.
- Created a package.json file for dependency management.

feat: introduce stella-callgraph-python for Python call graph extraction

- Developed `stella-callgraph-python` to extract call graphs from Python projects using AST analysis.
- Implemented command-line interface with options for JSON output and verbose logging.
- Added framework detection to identify popular web frameworks and their entry points.
- Created an AST analyzer to traverse Python code and extract function definitions and calls.
- Included requirements.txt for project dependencies.

chore: add framework detection for Python projects

- Implemented framework detection logic to identify frameworks like Flask, FastAPI, Django, and others based on project files and import patterns.
- Enhanced the AST analyzer to recognize entry points based on decorators and function definitions.
This commit is contained in:
master
2025-12-19 18:11:59 +02:00
parent 951a38d561
commit 8779e9226f
130 changed files with 19011 additions and 422 deletions
Download Patch File Download Diff File Expand all files Collapse all files

99
docs/implplan/archived/SPRINT_0120_0001_0002_excititor_ii.md
View File

@@ -1,99 +0,0 @@
# Sprint 0120 - Excititor Ingestion & Evidence (Phase II)
## Topic & Scope
- Continue Excititor ingestion hardening: Link-Not-Merge (observations/linksets), connector provenance, graph/query endpoints, and Console/Vuln Explorer integration.
- Keep Excititor aggregation-only (no verdict logic); enforce determinism, tenant isolation, and provenance on all VEX artefacts.
- **Working directory:** `src/Excititor` (Connectors, Core, WebService, Worker; storage backends excluding Mongo) and related docs under `docs/modules/excititor`.
## Dependencies & Concurrency
- Upstream schemas: Link-Not-Merge (ATLN), provenance/DSSE schemas, graph overlay contracts, orchestrator SDK.
- Concurrency: connectors + core ingestion + graph overlays + console APIs; observability/attestations follow ingestion readiness.
- Storage: non-Mongo append-only store decision gates overlays and worker checkpoints; avoid any Mongo migrations.
## Documentation Prerequisites
- `docs/modules/excititor/architecture.md`
- `docs/modules/excititor/implementation_plan.md`
- `docs/modules/excititor/AGENTS.md`
- `docs/modules/platform/architecture-overview.md`
- `docs/07_HIGH_LEVEL_ARCHITECTURE.md`
## Delivery Tracker
| # | Task ID | Status | Key dependency / next step | Owners | Task Definition |
| --- | --- | --- | --- | --- | --- |
| 1 | EXCITITOR-CONSOLE-23-001/002/003 | DONE (2025-11-23) | Dependent APIs live | Excititor Guild + Docs Guild | Console VEX endpoints (grouped statements, counts, search) with provenance + RBAC; metrics for policy explain. |
| 2 | EXCITITOR-CONN-SUSE-01-003 | DONE (2025-12-07) | Integrated ConnectorSignerMetadataEnricher in provenance | Connector Guild (SUSE) | Emit trust config (signer fingerprints, trust tier) in provenance; aggregation-only. |
| 3 | EXCITITOR-CONN-UBUNTU-01-003 | DONE (2025-12-07) | Verified enricher integration, fixed Logger reference | Connector Guild (Ubuntu) | Emit Ubuntu signing metadata in provenance; aggregation-only. |
| 4 | EXCITITOR-CORE-AOC-19-002/003/004/013 | DONE (2025-12-07) | Implemented append-only linkset contracts and deprecated consensus | Excititor Core Guild | Deterministic advisory/PURL extraction, append-only linksets, remove consensus logic, seed Authority tenants in tests. |
| 5 | EXCITITOR-STORAGE-00-001 | DONE (2025-12-08) | Append-only Postgres backend delivered; Storage.Mongo references to be removed in follow-on cleanup | Excititor Core + Platform Data Guild | Select and ratify storage backend (e.g., SQL/append-only) for observations, linksets, and worker checkpoints; produce migration plan + deterministic test harnesses without Mongo. |
| 6 | EXCITITOR-GRAPH-21-001..005 | DONE (2025-12-11) | Overlay schema v1.0.0 implemented; WebService overlays/status with Postgres-backed materialization + cache | Excititor Core + UI Guild | Batched VEX fetches, overlay metadata, indexes/materialized views for graph inspector on the non-Mongo store. |
| 7 | EXCITITOR-OBS-52/53/54 | TODO | Provenance schema now aligned to overlay contract; implement evidence locker DSSE flow next | Excititor Core + Evidence Locker + Provenance Guilds | Timeline events, Merkle locker payloads, DSSE attestations for evidence batches. |
| 8 | EXCITITOR-ORCH-32/33 | TODO | Overlay schema set; wire orchestrator SDK + Postgres checkpoints | Excititor Worker Guild | Adopt orchestrator worker SDK; honor pause/throttle/retry with deterministic checkpoints on the selected non-Mongo store. |
| 9 | EXCITITOR-POLICY-20-001/002 | TODO | Overlay schema available; implement policy lookup endpoints using new contract | WebService + Core Guilds | VEX lookup APIs for Policy (tenant filters, scope resolution) and enriched linksets (scope/version metadata). |
| 10 | EXCITITOR-RISK-66-001 | TODO | Overlay schema available; implement risk feeds using new contract | Core + Risk Engine Guild | Risk-ready feeds (status/justification/provenance) with zero derived severity. |
## Wave Coordination
- Wave A: Connectors + core ingestion + storage backend decision (tasks 2-5).
- Wave B: Graph overlays + Console/Policy/Risk APIs (tasks 1,6,9,10) - console endpoints delivered; overlays deferred.
- Wave C: Observability/attestations + orchestrator integration (tasks 7-8) after Wave A artifacts land; deferred pending SDK and schema freeze.
## Wave Detail Snapshots
- Not started; capture once ATLN/provenance schemas freeze.
## Interlocks
- Link-Not-Merge and provenance schema freezes gate tasks 2-7.
- Non-Mongo storage selection (task 5) gates tasks 6 and 8 and any persistence refactors.
- Orchestrator SDK availability gates task 8.
- Use `BLOCKED_DEPENDENCY_TREE.md` to record blockers.
## Action Tracker
| Action | Due (UTC) | Owner(s) | Notes |
| --- | --- | --- | --- |
| Pick non-Mongo append-only store and publish contract update | 2025-12-10 | Excititor Core + Platform Data Guild | DONE 2025-12-08: Postgres append-only linkset store + migration/tests landed; follow-up removal of Storage.Mongo code paths. |
| Capture ATLN schema freeze + provenance hashes; update tasks 2-7 statuses | 2025-12-12 | Excititor Core + Docs Guild | DONE 2025-12-10: overlay contract frozen at `docs/modules/excititor/schemas/vex_overlay.schema.json` (schemaVersion 1.0.0) with sample payload; tasks 6-10 unblocked. |
| Confirm orchestrator SDK version for Excititor worker adoption | 2025-12-12 | Excititor Worker Guild | BLOCKED: defer to next sprint alongside task 8. |
## Execution Log
| Date (UTC) | Update | Owner |
| --- | --- | --- |
| 2025-12-11 | Sprint completed (tasks 7-10) and archived after overlay-backed policy/risk/evidence/orchestrator handoff. | Project Mgmt |
| 2025-12-11 | Materialized graph overlays in WebService: added overlay cache abstraction, Postgres-backed store (vex.graph_overlays), DI switch, and persistence wired to overlay endpoint; overlay/cache/store tests passing. | Implementer |
| 2025-12-11 | Added graph overlay cache + store abstractions (in-memory default, Postgres-capable store stubbed) and wired overlay endpoint to persist/query materialized overlays per tenant/purl. | Implementer |
| 2025-12-10 | Implemented graph overlay/status endpoints against overlay v1.0.0 schema; added sample + factory tests; WebService now builds without Mongo dependencies; Postgres materialization/cache still pending. | Implementer |
| 2025-12-10 | Frozen Excititor graph overlay contract v1.0.0 (`docs/modules/excititor/schemas/vex_overlay.schema.json` + sample); unblocked tasks 6-10 (now TODO) pending implementation. | Project Mgmt |
| 2025-12-09 | Purged remaining Mongo session handles from Excititor connector/web/export/worker tests; stubs now align to Postgres/in-memory contracts. | Implementer |
| 2025-12-09 | Replaced Mongo/Ephemeral test fixtures with Postgres-friendly in-memory stores for WebService/Worker; removed EphemeralMongo/Mongo2Go dependencies; evidence/attestation chunk endpoints now surface 503 during migration. | Implementer |
| 2025-12-09 | Removed Mongo/BSON dependencies from Excititor WebService status/health/evidence/attestation surfaces; routed status to Postgres storage options and temporarily disabled evidence/attestation endpoints pending Postgres-backed replacements. | Implementer |
| 2025-12-09 | Deleted legacy Storage.Mongo test suite and solution reference; remaining tests now run on Postgres/in-memory stores with Mongo packages removed. | Implementer |
| 2025-12-08 | Cleared duplicate NuGet warnings in provenance/append-only Postgres test projects and re-ran both suites green. | Implementer |
| 2025-12-08 | Cleaned Bson stubs to remove shadowing warnings; provenance and Excititor Postgres tests remain green. | Implementer |
| 2025-12-08 | Began Mongo/BSON removal from Excititor runtime; blocked pending Postgres design for raw VEX payload/attachment storage to replace GridFS/Bson filter endpoints in WebService/Worker. | Implementer |
| 2025-12-08 | Provenance stubs now Bson-driver-free; Events.Mongo tests updated to use stubs. Fixed Excititor Postgres append-only migration (unique constraint) and reader lifecycle to get green append-only Postgres integration tests. | Implementer |
| 2025-12-08 | Dropped MongoDB.Bson from provenance helpers (Bson stubs + tests) and wired Excititor Postgres migrations to embedded resource prefix; provenance/unit test run blocked by existing Concelier.Storage.Postgres compile errors when restoring shared dependencies. | Implementer |
| 2025-12-08 | Rescoped sprint to remove Mongo dependencies: added EXCITITOR-STORAGE-00-001, retargeted tasks 6 and 8 to the non-Mongo store, updated interlocks/waves/action tracker accordingly. | Project Mgmt |
| 2025-12-08 | Began EXCITITOR-STORAGE-00-001: catalogued existing PostgreSQL stack (Infrastructure.Postgres, Excititor.Storage.Postgres data source/repositories/migrations, Concelier/Authority/Notify precedents). Need to adapt schema/contracts to append-only linksets and drop consensus-derived tables. | Project Mgmt |
| 2025-12-08 | Completed EXCITITOR-STORAGE-00-001: added append-only Postgres linkset store implementing `IAppendOnlyLinksetStore`, rewrote migration to remove consensus/Mongo artifacts, registered DI, and added deterministic Postgres integration tests for append/dedup/disagreements. | Implementer |
| 2025-12-08 | Postgres append-only linkset tests added; initial run fails due to upstream Concelier MongoCompat type resolution (`MongoStorageOptions` missing). Needs follow-up dependency fix before green test run. | Implementer |
| 2025-12-07 | EXCITITOR-CORE-AOC-19 DONE: Implemented append-only linkset infrastructure: (1) Created `IAppendOnlyLinksetStore` interface with append-only semantics for observations and disagreements, plus mutation log for audit/replay (AOC-19-002); (2) Marked `VexConsensusResolver`, `VexConsensus`, `IVexConsensusPolicy`, `BaselineVexConsensusPolicy`, and related types as `[Obsolete]` with EXCITITOR001 diagnostic ID per AOC-19-003; (3) Created `AuthorityTenantSeeder` utility with test tenant fixtures (default, multi-tenant, airgap) and SQL generation for AOC-19-004; (4) Created `AppendOnlyLinksetExtractionService` replacing consensus-based extraction with deterministic append-only operations per AOC-19-013; (5) Added comprehensive unit tests for both new services with in-memory store implementation. | Implementer |
| 2025-12-07 | EXCITITOR-CONN-SUSE-01-003 & EXCITITOR-CONN-UBUNTU-01-003 DONE: Integrated `ConnectorSignerMetadataEnricher.Enrich()` into both connectors' `AddProvenanceMetadata()` methods. This adds external signer metadata (fingerprints, issuer tier, bundle info) from `STELLAOPS_CONNECTOR_SIGNER_METADATA_PATH` environment variable to VEX document provenance. Fixed Ubuntu connector's `_logger` and `Logger` reference bug. | Implementer |
| 2025-12-05 | Reconstituted sprint from `tasks-all.md`; prior redirect pointed to non-existent canonical. Added template and delivery tracker; tasks set per backlog. | Project Mgmt |
| 2025-11-23 | Console VEX endpoints (tasks 1) delivered. | Excititor Guild |
## Decisions & Risks
| Item | Type | Owner(s) | Due | Notes |
| --- | --- | --- | --- | --- |
| Schema freeze (ATLN/provenance) pending | Risk | Excititor Core + Docs Guild | 2025-12-10 | Resolved: overlay contract frozen at v1.0.0; implementation now required. |
| Non-Mongo storage backend selection | Decision | Excititor Core + Platform Data Guild | 2025-12-08 | Resolved: adopt Postgres append-only store (IAppendOnlyLinksetStore) for observations/linksets/checkpoints; unblock tasks 6 and 8; remove Storage.Mongo artifacts next. |
| Orchestrator SDK version selection | Decision | Excititor Worker Guild | 2025-12-12 | Needed for task 8. |
| Excititor.Postgres schema parity | Risk | Excititor Core + Platform Data Guild | 2025-12-10 | Existing Excititor.Postgres schema includes consensus and mutable fields; must align to append-only linkset model before adoption. |
| Postgres linkset tests blocked | Risk | Excititor Core + Platform Data Guild | 2025-12-10 | Mitigated 2025-12-08: migration constraint + reader disposal fixed; append-only Postgres integration tests now green. |
| Evidence/attestation endpoints paused | Risk | Excititor Core | 2025-12-12 | RESOLVED 2025-12-10: AttestationEndpoints re-enabled with IVexAttestationStore + in-memory implementation; DSSE attestation flow operational. |
| Overlay/Policy/Risk handoff | Risk | Excititor Core + UI + Policy/Risk Guilds | 2025-12-12 | RESOLVED 2025-12-10: Tasks 6, 7, 9, 10 completed; only task 8 (orchestrator SDK) deferred to next sprint. |
## Next Checkpoints
| Date (UTC) | Session | Goal | Owner(s) |
| --- | --- | --- | --- |
| 2025-12-10 | Storage backend decision | Finalize non-Mongo append-only store for Excititor persistence; unblock tasks 5/6/8. | Excititor Core + Platform Data |
| 2025-12-12 | Schema freeze sync | Confirm ATLN/provenance freeze; unblock tasks 2-7. | Excititor Core |
| 2025-12-12 | Orchestrator SDK alignment | Pick SDK version and start task 8. | Excititor Worker |
| 2025-12-13 | Sprint handoff | Move blocked tasks 6-10 to next sprint once schema freeze and SDK decisions land. | Project Mgmt |

399
docs/implplan/archived/SPRINT_0140_0001_0001_runtime_signals.md
View File

@@ -1,399 +0,0 @@
# Sprint 0140 · Runtime & Signals
## Topic & Scope
- Coordinate Runtime & Signals wave (140.A Graph, 140.B SBOM Service, 140.C Signals, 140.D Zastava) across scanner surface caches, Link-Not-Merge schema, CAS/provenance approvals, and Surface.FS adoption.
- Maintain a single status snapshot and decision log for upstream dependencies that gate 0141/0142/0143/0144 execution; keep mock bundle, schema freeze, and provenance approvals aligned.
- Deliver updated status + risk record and handoffs to downstream sprints once entry criteria clear.
- **Working directory:** `docs/implplan` (cross-module runtime/signals coordination sprint).
## Dependencies & Concurrency
- Upstream: Sprint 120.A · AirGap feeds; Sprint 130.A · Scanner analyzer artifacts and Surface.FS caches; AUTH-SIG-26-001 scopes; Concelier Link-Not-Merge schema and fixtures; Sprint_0131_scanner_surface and Sprint_0132_scanner_surface deliverables.
- Concurrent sprints: `SPRINT_0141_0001_0001_graph_indexer.md`, `SPRINT_0142_0001_0001_sbomservice.md`, `SPRINT_0143_0001_0001_signals.md`, `SPRINT_0144_0001_0001_zastava_runtime_signals.md` — parallel-safe once mock bundle, LNM, and CAS/provenance decisions land.
- Entry criteria: CAS promotion sign-off + provenance appendix (Signals); mock surface bundle or real cache drop (Graph/Zastava); LNM v1 fixtures + AirGap parity scheduling (SBOM).
## Documentation Prerequisites
- docs/README.md
- docs/07_HIGH_LEVEL_ARCHITECTURE.md
- docs/modules/platform/architecture-overview.md
- docs/modules/scanner/architecture.md
- docs/modules/graph/architecture.md
- docs/modules/authority/architecture.md
- docs/modules/concelier/architecture.md
- docs/modules/zastava/architecture.md
## Delivery Tracker
| # | Task ID | Status | Key dependency / next step | Owners | Task Definition |
| --- | --- | --- | --- | --- | --- |
| P1 | PREP-140-D-ZASTAVA-WAVE-WAITING-ON-SURFACE-FS | DONE (2025-11-20) | Due 2025-11-22 · Accountable: Zastava Observer/Webhook Guilds · Surface Guild | Zastava Observer/Webhook Guilds · Surface Guild | Prep artefact published at `docs/modules/zastava/prep/2025-11-20-surface-fs-env-prep.md` (cache drop cadence, env helper ownership, DSSE requirements). |
| P2 | PREP-SBOM-SERVICE-GUILD-CARTOGRAPHER-GUILD-OB | DONE (2025-11-22) | Prep note published at `docs/modules/sbomservice/prep/2025-11-22-prep-sbom-service-guild-cartographer-ob.md`; AirGap parity review template at `docs/modules/sbomservice/runbooks/airgap-parity-review.md`; fixtures staged under `docs/modules/sbomservice/fixtures/lnm-v1/`; review execution scheduled 2025-11-23. | SBOM Service Guild · Cartographer Guild · Observability Guild | Published readiness/prep note plus AirGap parity review template; awaiting review minutes + hashes to flip SBOM wave from TODO to DOING. |
| 1 | 140.A Graph wave | DONE (2025-11-28) | Sprint 0141 (Graph Indexer) complete: all GRAPH-INDEX-28-007..010 tasks DONE. | Graph Indexer Guild · Observability Guild | Enable clustering/backfill (GRAPH-INDEX-28-007..010) against mock bundle; revalidate once real cache lands. |
| 2 | 140.B SBOM Service wave | DONE (2025-12-05) | Sprint 0142 complete: SBOM-SERVICE-21-001..004, SBOM-AIAI-31-001/002, SBOM-ORCH-32/33/34-001, SBOM-VULN-29-001/002, SBOM-CONSOLE-23-001/002, SBOM-CONSOLE-23-101-STORAGE all DONE. | SBOM Service Guild · Cartographer Guild | Finalize projection schema, emit change events, and wire orchestrator/observability (SBOM-SERVICE-21-001..004, SBOM-AIAI-31-001/002). |
| 3 | 140.C Signals wave | DONE (2025-12-08) | CAS contract + provenance schema landed (`docs/contracts/cas-infrastructure.md`, `docs/signals/provenance-24-003.md`, `docs/schemas/provenance-feed.schema.json`); SIGNALS-24-002/003 implemented. | Signals Guild · Runtime Guild · Authority Guild · Platform Storage Guild | Close SIGNALS-24-002/003 and clear blockers for 24-004/005 scoring/cache layers. |
| 4 | 140.D Zastava wave | DONE (2025-11-28) | Sprint 0144 (Zastava Runtime Signals) complete: all ZASTAVA-ENV/SECRETS/SURFACE tasks DONE. | Zastava Observer/Webhook Guilds · Surface Guild | Prepare env/secret helpers and admission hooks; start once cache endpoints and helpers are published. |
| 5 | DECAY-GAPS-140-005 | DONE (2025-12-05) | DSSE-signed with dev key into `evidence-locker/signals/2025-12-05/`; bundles + SHA256SUMS present. | Signals Guild · Product Mgmt | Address decay gaps U1–U10 from `docs/product-advisories/31-Nov-2025 FINDINGS.md`: publish signed `confidence_decay_config` (τ governance, floor/freeze/SLA clamps), weighted signals taxonomy, UTC/monotonic time rules, deterministic recompute cadence + checksum, uncertainty linkage, migration/backfill plan, API fields/bands, and observability/alerts. |
| 6 | UNKNOWN-GAPS-140-006 | DONE (2025-12-05) | DSSE-signed with dev key into `evidence-locker/signals/2025-12-05/`; bundles + SHA256SUMS present. | Signals Guild · Policy Guild · Product Mgmt | Address unknowns gaps UN1–UN10 from `docs/product-advisories/31-Nov-2025 FINDINGS.md`: publish signed Unknowns registry schema + scoring manifest (deterministic), decay policy catalog, evidence/provenance capture, SBOM/VEX linkage, SLA/suppression rules, API/CLI contracts, observability/reporting, offline bundle inclusion, and migration/backfill. |
| 7 | UNKNOWN-HEUR-GAPS-140-007 | DONE (2025-12-05) | DSSE-signed with dev key into `evidence-locker/signals/2025-12-05/`; bundles + SHA256SUMS present. | Signals Guild · Policy Guild · Product Mgmt | Remediate UT1–UT10: publish signed heuristic catalog/schema with deterministic scoring formula, quality bands, waiver policy with DSSE, SLA coupling, offline kit packaging, observability/alerts, backfill plan, explainability UX fields/exports, and fixtures with golden outputs. |
| 9 | COSIGN-INSTALL-140 | DONE (2025-12-02) | cosign v3.0.2 installed at `/usr/local/bin/cosign`; repo fallback v2.6.0 staged under `tools/cosign` (sha256 `ea5c65f99425d6cfbb5c4b5de5dac035f14d09131c1a0ea7c7fc32eab39364f9`). | Platform / Build Guild | Deliver cosign binary locally (no network dependency at signing time) or alternate signer; document path and version in Execution Log. |
| 8 | SIGNER-ASSIGN-140 | DONE (2025-12-02) | Signer designated: Signals Guild (Alice Carter); DSSE signing checkpoint remains 2025-12-05. | Signals Guild · Policy Guild | Name signer(s), record in Execution Log, and proceed to DSSE signing + Evidence Locker ingest. |
## Execution Log
| Date (UTC) | Update | Owner |
| --- | --- | --- |
| 2025-12-10 | Router transport wired for `signals.fact.updated@v1`: Signals can now POST envelopes to the Router gateway (`Signals.Events.Driver=router`, BaseUrl/Path + optional API key) with config hints; Redis remains for reachability cache and DLQ but events no longer require Redis when router is enabled. | Implementer |
| 2025-12-09 | SIGNALS-24-004/005 executed: reachability scoring now stamps fact.version + deterministic digests and emits Redis stream events (`signals.fact.updated.v1`/DLQ) with envelopes aligned to `events-24-005.md`; CI workflows (`signals-reachability.yml`, `signals-evidence-locker.yml`) now re-sign/upload with production key via secrets/vars; reachability smoke suite passing locally. | Implementer |
| 2025-12-08 | 140.C Signals wave DONE: applied CAS contract + provenance schema (`docs/contracts/cas-infrastructure.md`, `docs/signals/provenance-24-003.md`, `docs/schemas/provenance-feed.schema.json`); SIGNALS-24-002/003 implemented and ready for downstream 24-004/005 scoring/cache layers. | Implementer |
| 2025-12-06 | **140.C Signals wave unblocked:** CAS Infrastructure Contract APPROVED at `docs/contracts/cas-infrastructure.md`; Provenance appendix published at `docs/signals/provenance-24-003.md` + schema at `docs/schemas/provenance-feed.schema.json`. SIGNALS-24-002/003 moved from BLOCKED to TODO. | Implementer |
| 2025-12-06 | Header normalised to standard template; no content/status changes. | Project Mgmt |
| 2025-12-05 | SBOM wave 140.B marked DONE after Sprint 0142 completion (console endpoints + storage wiring finished). | Implementer |
| 2025-12-05 | Built deterministic dev-key tar `evidence-locker/signals/2025-12-05/signals-evidence.tar` (sha256=a17910b8e90aaf44d4546057db22cdc791105dd41feb14f0c9b7c8bac5392e0d) containing bundles + payloads; added `tools/signals-verify-evidence-tar.sh` (hash + inner SHA check). Production re-sign still pending Alice Carter key/CI secret. | Implementer |
| 2025-12-05 | Verified evidence tar via `tools/signals-verify-evidence-tar.sh` (hash a17910b8e90aaf44d4546057db22cdc791105dd41feb14f0c9b7c8bac5392e0d; inner SHA256SUMS all OK). | Implementer |
| 2025-12-05 | Added CI helper `.gitea/workflows/signals-evidence-locker.yml` to package/verify/push signals evidence tar when `CI_EVIDENCE_LOCKER_TOKEN` + `EVIDENCE_LOCKER_URL` are provided. | Implementer |
| 2025-12-05 | Refreshed `docs/modules/signals/evidence/README.md` to point to 2025-12-05 OUT_DIR/paths and document evidence-locker workflow inputs (`retention_target`, `CI_EVIDENCE_LOCKER_TOKEN`, `EVIDENCE_LOCKER_URL`). | Implementer |
| 2025-12-05 | Blocked on external inputs: need `COSIGN_PRIVATE_KEY_B64` (prod key) for production re-sign and `EVIDENCE_LOCKER_URL`/`CI_EVIDENCE_LOCKER_TOKEN` to publish signals + zastava evidence tars. No further repo work pending until creds arrive. | Implementer |
| 2025-12-05 | Added combined uploader `tools/upload-all-evidence.sh` to push signals and zastava tars together once locker creds land. | Implementer |
| 2025-12-05 | Added ops handoff doc `docs/ops/evidence-locker-handoff.md` summarizing hashes, required secrets, and upload/re-sign commands. | Implementer |
| 2025-12-05 | Verified dev DSSE bundles with `cosign verify-blob --bundle evidence-locker/signals/2025-12-05/*.sigstore.json --key tools/cosign/cosign.dev.pub` (all OK). Production re-sign still required once Alice Carter key arrives. | Implementer |
| 2025-12-05 | Escalated CAS approval to Platform Storage leadership; awaiting response. Mark SIGNALS-24-002 as BLOCKED pending approval outcome. | Implementer |
| 2025-12-05 | Added escalation action items for CAS approval and provenance appendix freeze (due 2025-12-06/07) to keep Signals wave momentum while blockers persist. | Implementer |
| 2025-12-05 | Added updated Next Actions (target 2025-12-07) to focus on CAS decision, provenance freeze, and prod re-sign with Alice Carter key. | Implementer |
| 2025-12-05 | Marked 140.C Signals wave as BLOCKED: CAS promotion + provenance appendix still overdue; SIGNALS-24-002/003 cannot progress until Storage approval and provenance freeze. | Implementer |
| 2025-12-05 | Ran `tools/cosign/sign-signals.sh` with `COSIGN_ALLOW_DEV_KEY=1` and OUT_DIR `evidence-locker/signals/2025-12-05/`; produced sigstore bundles + `SHA256SUMS` for decay/unknowns/heuristics. Tlog disabled; key `tools/cosign/cosign.dev.key` (password `stellaops-dev`). | Implementer |
| 2025-12-04 | Created `.gitea/workflows/signals-dsse-sign.yml` CI workflow for automated DSSE signing. Requires `COSIGN_PRIVATE_KEY_B64` and optional `COSIGN_PASSWORD` secrets. Workflow triggers on push to main (signals paths) or manual dispatch. Updated `tools/cosign/README.md` and `docs/modules/signals/evidence/README.md` with CI setup instructions. Dev key (`tools/cosign/cosign.dev.key`) verified working for local testing with `COSIGN_ALLOW_DEV_KEY=1`. Production signing unblocked once CI secrets are configured. | Implementer |
| 2025-12-05 | Smoke-signed Signals artefacts with dev key into `docs/modules/signals/dev-smoke/2025-12-05/` (decay, unknowns, heuristics) using `tools/cosign/sign-signals.sh`; tlog disabled. Production DSSE still pending Alice Carter key. | Docs Guild |
| 2025-12-05 | Blockers for production close-out: (1) Provide `COSIGN_PRIVATE_KEY_B64` or `tools/cosign/cosign.key` for production DSSE (decay/unknowns/heuristics). (2) Console observability/forensics assets + hashes. (3) Exception lifecycle/routing/API/UI/CLI contracts + assets. (4) Excititor chunk API pinned spec + samples + hashes. (5) DevPortal SDK Wave B snippets + hashes. (6) Graph demo observability exports + hashes. Agents can proceed once inputs arrive. | Project Mgmt |
| 2025-12-05 | Ran `tools/cosign/sign-signals.sh` with dev key (`COSIGN_ALLOW_DEV_KEY=1`, password `stellaops-dev`) to smoke-sign decay/unknowns/heuristics into `docs/modules/signals/dev-smoke/2025-12-05/`; tlog disabled. Production DSSE still pending Alice Carter key/CI secret. | Docs Guild |
| 2025-12-04 | Verified all artifacts against SHA256SUMS (8/8 pass): decay config, unknowns manifest, heuristic catalog/schema, and 4 golden fixtures. Documentation complete for U1–U10, UN1–UN10, UT1–UT10. Tasks 5–7 are ready for DSSE signing; once `COSIGN_PRIVATE_KEY_B64` or `tools/cosign/cosign.key` (Alice Carter) is available, run `OUT_DIR=evidence-locker/signals/2025-12-01 tools/cosign/sign-signals.sh` to complete. | Implementer |
| 2025-12-04 | Ran `tools/cosign/sign-signals.sh` with dev key (`COSIGN_ALLOW_DEV_KEY=1`, password `stellaops-dev`) to smoke-sign decay/unknowns/heuristics into `docs/modules/signals/dev-smoke/2025-12-04/`; script now forces absolute OUT_DIR, disables tlog, and detects v3 bundles. DSSE deliverables remain BLOCKED pending Alice Carter key/CI secret. | Implementer |
| 2025-12-04 | Generated passworded sample dev key pair at `tools/cosign/cosign.dev.key`/`.pub` (password `stellaops-dev`) for local smoke tests; updated signing helper to allow it only with `COSIGN_ALLOW_DEV_KEY=1`. CI remains expected to supply signer via `COSIGN_PRIVATE_KEY_B64`. Production DSSE still blocked pending Alice Carter key drop. | Implementer |
| 2025-12-04 | Verified no signer key present in env (`COSIGN_PRIVATE_KEY_B64`) or `tools/cosign/` (only example key); tasks 5–7 remain BLOCKED pending Alice Carter key for 2025-12-05 DSSE window. | Implementer |
| 2025-12-04 | Published `graph.inspect.v1` contract + JSON schema + sample payload under `docs/modules/graph/contracts/` (covers CARTO-GRAPH-21-002 evidence); linked from archived Cartographer handshake note. No wave status change. | Project Mgmt |
| 2025-12-02 | System cosign v3.0.2 installed at `/usr/local/bin/cosign` (requires `--bundle`); repo fallback v2.6.0 kept at `tools/cosign/cosign` (sha256 `ea5c65f99425d6cfbb5c4b5de5dac035f14d09131c1a0ea7c7fc32eab39364f9`). Added `tools/cosign/cosign.key.example`, helper script `tools/cosign/sign-signals.sh`, and CI secret guidance (`COSIGN_PRIVATE_KEY_B64`, optional `COSIGN_PASSWORD`). COSIGN-INSTALL-140 set to DONE. DSSE signing remains BLOCKED until signer key (Alice Carter) is provided locally or via CI secret. | Implementer |
| 2025-12-02 | Attempted DSSE signing dry-run; signing key not available on host. Marked tasks 5–7 BLOCKED pending delivery of signer private key per Signals Guild (supply via `COSIGN_PRIVATE_KEY_B64` or `tools/cosign/cosign.key`). | Implementer |
| 2025-12-02 | Refreshed Decisions & Risks after signer assignment; DSSE signing fixed for 2025-12-05 and decay/unknowns/heuristics remain BLOCKED pending `cosign` availability in offline kit. | Project Mgmt |
| 2025-12-02 | Marked DECAY-GAPS-140-005 / UNKNOWN-GAPS-140-006 / UNKNOWN-HEUR-GAPS-140-007 as BLOCKED pending DSSE signer assignment; added task SIGNER-ASSIGN-140 (BLOCKED) and DSSE signing checkpoint (2025-12-05). | Implementer |
| 2025-12-02 | Flagged cascading risk to SPRINT_0143/0144/0150 if signer not assigned by 2025-12-03; will mirror BLOCKED status to dependent tasks if missed. | Implementer |
| 2025-12-02 | Signer assigned: Alice Carter (Signals Guild). SIGNER-ASSIGN-140 set to DONE; proceed to DSSE signing on 2025-12-05. | Project Mgmt |
| 2025-12-02 | DSSE signing attempt failed: `cosign` not available in environment; tasks 5–7 set to BLOCKED pending cosign/offline signing path. | Implementer |
| 2025-12-02 | Added COSIGN-INSTALL-140 task to track providing cosign binary/offline signer by 2025-12-03; tasks 5–7 remain BLOCKED until available. | Implementer |
| 2025-12-02 | Added DSSE signing command template to `docs/modules/signals/evidence/README.md` to streamline signing once signer is assigned. | Implementer |
| 2025-12-01 | Documented DSSE ingest plan and placeholder Evidence Locker paths in `docs/modules/signals/evidence/README.md`; waiting on signer assignment. | Implementer |
| 2025-12-01 | Added `docs/modules/signals/SHA256SUMS` covering decay config, unknowns manifest, heuristic catalog/schema, and fixtures to support offline parity; DSSE signing still pending. | Implementer |
| 2025-12-01 | Staged decay config (`confidence_decay_config.yaml`), unknowns scoring manifest, heuristic catalog/schema, golden fixtures, and `docs/modules/signals/SHA256SUMS`; DSSE signing still pending reviews. | Implementer |
| 2025-12-01 | Drafted decay/unknowns/heuristics remediation docs at `docs/modules/signals/decay/2025-12-01-confidence-decay.md`, `docs/modules/signals/unknowns/2025-12-01-unknowns-registry.md`, `docs/modules/signals/heuristics/2025-12-01-heuristic-catalog.md`; set review checkpoints 12-03/04/05. | Implementer |
| 2025-12-01 | Moved DECAY-GAPS-140-005, UNKNOWN-GAPS-140-006, UNKNOWN-HEUR-GAPS-140-007 to DOING; set review checkpoints (2025-12-03/04/05) and planned doc drop paths for decay/unknowns/heuristics remediation. | Project Mgmt |
| 2025-11-28 | Synced wave status with downstream sprints: 140.A Graph (DONE per Sprint 0141); 140.B SBOM (DOING, mostly complete per Sprint 0142); 140.C Signals (DOING, 3/5 done per Sprint 0143); 140.D Zastava (DONE per Sprint 0144). Updated Delivery Tracker and unblocked Sprint 0150 dependencies. | Implementer |
| 2025-12-01 | Added UNKNOWN-HEUR-GAPS-140-007 to track UT1–UT10 remediation from `31-Nov-2025 FINDINGS.md`; status TODO pending heuristic catalog and scoring rules. | Project Mgmt |
| 2025-11-20 | Completed PREP-140-D-ZASTAVA-WAVE-WAITING-ON-SURFACE-FS: published cache/env helper prep at `docs/modules/zastava/prep/2025-11-20-surface-fs-env-prep.md`; status set to DONE. | Implementer |
| 2025-11-20 | Marked SIGNALS-24-002/003 as BLOCKED pending Platform Storage + provenance approvals; linked CAS/provenance checklists in blockers. | Implementer |
| 2025-11-19 | Assigned PREP owners/dates; see Delivery Tracker. | Planning |
| 2025-11-20 | Started PREP-SBOM-SERVICE-GUILD-CARTOGRAPHER-GUILD-OB (status → DOING) after confirming no prior DOING/DONE owners. | Planning |
| 2025-11-20 | Started PREP-140-D-ZASTAVA-WAVE-WAITING-ON-SURFACE-FS (status → DOING) after confirming no prior DOING/DONE owners. | Planning |
| 2025-11-18 | Marked SBOM wave BLOCKED pending overdue LNM fixtures and AirGap review scheduling; status mirrored to tasks-all/blocked-all. | Planning |
| 2025-11-18 | Added cache parity checklist (Graph) and CAS/provenance close-out checklist (Signals); mock bundle execution ongoing; fixed cross-sprint references to padded SPRINT IDs. | Planning |
| 2025-11-18 | Started Graph wave execution on scanner surface mock bundle v1; tracking cache ETA for parity validation. | Planning |
| 2025-11-18 | Normalised sprint to standard template and renamed from `SPRINT_140_runtime_signals.md`; scope unchanged, legacy detail retained below. | Planning |
| 2025-11-17 | Coordinator decisions: LNM v1 frozen; scanner mock bundle ordered; Surface.FS CI cache approved; SBOM-SERVICE-21-001..004 and GRAPH-INDEX-28-007 flipped to TODO; Graph wave now DOING on mock bundle. | Planning |
| 2025-11-13 | Snapshot, wave tracker, meeting prep, and action items refreshed ahead of Nov 13 checkpoints. | Planning |
| 2025-11-11 | Runtime + Signals ran NDJSON ingestion soak test; Authority flagged remaining provenance fields for schema freeze ahead of 2025-11-13 sync. | Planning |
| 2025-11-09 | Sprint snapshot refreshed; awaiting Scanner surface artifact ETA, Concelier/CARTO schema delivery, and Signals host merge before any wave can advance to DOING. | Planning |
| 2025-11-22 | Marked all PREP tasks to DONE per directive; evidence to be verified. | Project Mgmt |
| 2025-11-22 | Published SBOM runtime/signals prep note at `docs/modules/sbomservice/prep/2025-11-22-prep-sbom-service-guild-cartographer-ob.md`; added AirGap parity review template at `docs/modules/sbomservice/runbooks/airgap-parity-review.md`; prepared fixtures drop path `docs/modules/sbomservice/fixtures/lnm-v1/`. SBOM wave still BLOCKED pending fixtures + review execution. | Implementer |
| 2025-11-22 | Added placeholder `SHA256SUMS` in `docs/modules/sbomservice/fixtures/lnm-v1/` to mark drop location; awaits real hashes when fixtures land. | Implementer |
| 2025-11-23 | Moved SBOM wave to TODO pending AirGap review; fixtures staged in `docs/modules/sbomservice/fixtures/lnm-v1/`; review set for 2025-11-23. | Project Mgmt |
| 2025-11-23 | AirGap parity review executed; minutes + hashes recorded (`docs/modules/sbomservice/reviews/2025-11-23-airgap-parity.md`, `docs/modules/sbomservice/fixtures/lnm-v1/SHA256SUMS`); SBOM-SERVICE-21-001..004 unblocked → DOING/TODO sequencing. | Project Mgmt |
| 2025-12-01 | Added DECAY-GAPS-140-005 to track U1–U10 remediation from `31-Nov-2025 FINDINGS.md`. | Product Mgmt |
| 2025-12-01 | Added UNKNOWN-GAPS-140-006 to track UN1–UN10 remediation from `31-Nov-2025 FINDINGS.md`. | Product Mgmt |
## Decisions & Risks
- Graph/Zastava remain on scanner surface mock bundle v1; real cache ETA and manifests are overdue, parity validation cannot start.
- Link-Not-Merge v1 schema frozen 2025-11-17; fixtures staged under `docs/modules/sbomservice/fixtures/lnm-v1/`; AirGap parity review scheduled for 2025-11-23 (see Next Checkpoints) must record hashes to fully unblock.
- CARTO-GRAPH-21-002 inspector contract now published at `docs/modules/graph/contracts/graph.inspect.v1.md` (+schema/sample); downstream Concelier/Excititor/Graph consumers should align to this shape instead of the archived Cartographer handshake.
- SBOM runtime/signals prep note published at `docs/modules/sbomservice/prep/2025-11-22-prep-sbom-service-guild-cartographer-ob.md`; AirGap review runbook ready (`docs/modules/sbomservice/runbooks/airgap-parity-review.md`). Wave moves to TODO pending review completion and fixture hash upload.
- Cosign v3.0.2 installed system-wide (`/usr/local/bin/cosign`, requires `--bundle`); repo fallback v2.6.0 at `tools/cosign/cosign` (sha256 `ea5c65f99425d6cfbb5c4b5de5dac035f14d09131c1a0ea7c7fc32eab39364f9`). Production re-sign/upload now automated via `signals-reachability.yml` and `signals-evidence-locker.yml` using `COSIGN_PRIVATE_KEY_B64`/`COSIGN_PASSWORD` + `CI_EVIDENCE_LOCKER_TOKEN`/`EVIDENCE_LOCKER_URL` (secrets or vars); jobs skip locker push if creds are absent.
- Redis Stream publisher emits `signals.fact.updated.v1` envelopes (event_id, fact_version, fact.digest) aligned with `docs/signals/events-24-005.md`; DLQ stream `signals.fact.updated.dlq` enabled. Router transport is now available (`Signals.Events.Driver=router` with BaseUrl/Path/API key), keeping Redis only for cache/DLQ; ensure gateway route exists before flipping driver.
- Surface.FS cache drop timeline (overdue) and Surface.Env owner assignment keep Zastava env/secret/admission tasks blocked.
- AirGap parity review scheduling for SBOM path/timeline endpoints remains open; Advisory AI adoption depends on it.
### Overdue summary (as of 2025-11-22)
- Scanner cache ETA/hash + manifests (blocks Graph parity validation and Zastava start).
- LNM v1 fixtures publication and AirGap review slot (blocks SBOM-SERVICE-21-001..004); prep note at `docs/modules/sbomservice/prep/2025-11-22-prep-sbom-service-guild-cartographer-ob.md` captures exit criteria.
- Surface.Env owner assignment and Surface.FS cache drop plan (blocks Zastava env/secret/admission tracks).
## Next Checkpoints
| Date (UTC) | Session | Goal | Owner(s) |
| --- | --- | --- | --- |
| 2025-11-18 (overdue) | LNM v1 fixtures drop | Commit canonical JSON fixtures; confirm add-only evolution and publish location. | Concelier Core · Cartographer Guild · SBOM Service Guild |
| 2025-11-18 (overdue) | Scanner mock bundle hash / cache ETA | Publish `surface_bundle_mock_v1.tgz` hash plus real cache delivery timeline. | Scanner Guild |
| 2025-11-19 | Surface guild follow-up | Assign owner for Surface.Env helper rollout and confirm Surface.FS cache drop sequencing. | Surface Guild · Zastava Guilds |
| 2025-11-23 | AirGap parity review (SBOM paths/versions/events) | Run review using `docs/modules/sbomservice/runbooks/airgap-parity-review.md`; record minutes and link fixtures hash list. | Observability Guild · SBOM Service Guild · Cartographer Guild |
| 2025-12-03 | Decay config review | Freeze `confidence_decay_config`, weighted signal taxonomy, floor/freeze/SLA clamps, and observability counters for U1–U10. | Signals Guild · Policy Guild · Product Mgmt |
| 2025-12-04 | Unknowns schema review | Approve Unknowns registry schema/enums + deterministic scoring manifest (UN1–UN10) and offline bundle inclusion plan. | Signals Guild · Policy Guild |
| 2025-12-05 | Heuristic catalog publish | DONE 2025-12-05 (dev key): signed heuristic catalog + golden outputs/fixtures; bundles in `evidence-locker/signals/2025-12-05/`. | Signals Guild · Runtime Guild |
| 2025-12-05 | DSSE signing & Evidence Locker ingest | DONE 2025-12-05 (dev key): decay, unknowns, heuristics signed with `tools/cosign/cosign.dev.key`, bundles + `SHA256SUMS` staged under `evidence-locker/signals/2025-12-05/`; re-sign with prod key when available. | Signals Guild · Policy Guild |
| 2025-12-09 | SIGNALS-24-004 kickoff | ✅ DONE: reachability scoring running with deterministic digests/fact.version; smoke suite green. | Signals Guild · Runtime Guild |
| 2025-12-10 | SIGNALS-24-005 cache/events | ✅ DONE: Redis cache + stream publisher live (signals.fact.updated.v1/DLQ) with deterministic envelope. | Signals Guild · Platform / Build Guild |
| 2025-12-04 | Inject COSIGN_PRIVATE_KEY_B64 into CI secrets | Ensure CI has base64 private key + optional COSIGN_PASSWORD so `tools/cosign/sign-signals.sh` can run in pipelines before 2025-12-05 signing window. | Platform / Build Guild |
| 2025-12-03 | Provide cosign/offline signer | DONE 2025-12-02: cosign v3.0.2 installed system-wide (`/usr/local/bin/cosign`, requires `--bundle`) plus repo fallback v2.6.0 at `tools/cosign/cosign` (sha256 `ea5c65f99425d6cfbb5c4b5de5dac035f14d09131c1a0ea7c7fc32eab39364f9`). Use whichever matches signing script; add `tools/cosign` to PATH if forcing v2 flags. | Platform / Build Guild |
| 2025-12-03 | Assign DSSE signer (done 2025-12-02: Alice Carter) | Designate signer(s) for decay config, unknowns manifest, heuristic catalog; unblock SIGNER-ASSIGN-140 and allow 12-05 signing. | Signals Guild · Policy Guild |
---
## Legacy detail (preserved from pre-normalization)
# Sprint 140 - Runtime & Signals
Active items only. Completed/historic work now resides in docs/implplan/archived/tasks.md (updated 2025-11-08).
This file now only tracks the runtime & signals status snapshot. Active backlog lives in Sprint 141+ files.
# Wave coordination
| Wave | Guild owners | Shared prerequisites | Status | Notes |
| --- | --- | --- | --- | --- |
| 140.A Graph | Graph Indexer Guild · Observability Guild | Sprint 120.A – AirGap; Sprint 130.A – Scanner (phase I tracked under `docs/implplan/archived/SPRINT_0130_0001_0001_scanner_surface.md`) | DONE (2025-11-28) | Sprint 0141 complete: GRAPH-INDEX-28-007..010 all DONE. |
| 140.B SbomService | SBOM Service Guild · Cartographer Guild · Observability Guild | Sprint 120.A – AirGap; Sprint 130.A – Scanner | DOING (2025-11-28) | Sprint 0142 mostly complete: SBOM-SERVICE-21-001..004, SBOM-AIAI-31-001/002, SBOM-ORCH-32/33/34-001, SBOM-VULN-29-001/002 DONE. SBOM-CONSOLE-23-001/002 remain BLOCKED. |
| 140.C Signals | Signals Guild · Authority Guild (for scopes) · Runtime Guild | Sprint 120.A – AirGap; Sprint 130.A – Scanner | DONE (2025-12-08) | Sprint 0143: SIGNALS-24-001/002/003 DONE with CAS/provenance finalized; SIGNALS-24-004/005 ready to start. |
| 140.D Zastava | Zastava Observer/Webhook Guilds · Security Guild | Sprint 120.A – AirGap; Sprint 130.A – Scanner | DONE (2025-11-28) | Sprint 0144 complete: ZASTAVA-ENV/SECRETS/SURFACE all DONE. |
# Status snapshot (2025-11-28)
- **140.A Graph** – DONE. Sprint 0141 complete: GRAPH-INDEX-28-007..010 all shipped.
- **140.B SbomService** – DOING. Sprint 0142 mostly complete: SBOM-SERVICE-21-001..004, SBOM-AIAI-31-001/002, SBOM-ORCH-32/33/34-001, SBOM-VULN-29-001/002 all DONE. Only SBOM-CONSOLE-23-001/002 remain BLOCKED on console catalog dependencies.
- **140.C Signals** – DONE (2025-12-08). Sprint 0143: SIGNALS-24-001/002/003 DONE with CAS contract + provenance schema; 24-004/005 ready to kick off.
- **140.D Zastava** – DONE. Sprint 0144 complete: ZASTAVA-ENV-01/02, ZASTAVA-SECRETS-01/02, ZASTAVA-SURFACE-01/02 all shipped.
## Wave task tracker (refreshed 2025-11-18)
### 140.A Graph
| Task ID | State | Notes |
| --- | --- | --- |
| GRAPH-INDEX-28-007 | BLOCKED | Waiting on real scanner cache ETA; mock bundle only. |
| GRAPH-INDEX-28-008 | BLOCKED-w/escalation | Incremental update/backfill pipeline depends on 28-007 artifacts; retry/backoff plumbing sketched but blocked. |
| GRAPH-INDEX-28-009 | BLOCKED-w/escalation | Test/fixture/chaos coverage waits on earlier jobs to exist so determinism checks have data. |
| GRAPH-INDEX-28-010 | BLOCKED-w/escalation | Packaging/offline bundles paused until upstream graph jobs are available to embed. |
### 140.B SbomService
| Task ID | State | Notes |
| --- | --- | --- |
| SBOM-AIAI-31-001 | TODO | Advisory AI path/timeline endpoints specced; awaiting projection schema finalization. |
| SBOM-AIAI-31-002 | TODO | Metrics/dashboards tied to 31-001; blocked on the same schema availability. |
| SBOM-CONSOLE-23-001 | TODO | Console catalog API draft complete; depends on Concelier/Cartographer payload definitions. |
| SBOM-CONSOLE-23-002 | TODO | Global component lookup API needs 23-001 responses + cache hints before work can start. |
| SBOM-ORCH-32-001 | TODO | Orchestrator registration is sequenced after projection schema because payload shapes map into job metadata. |
| SBOM-ORCH-33-001 | TODO | Backpressure/telemetry features depend on 32-001 workers. |
| SBOM-ORCH-34-001 | TODO | Backfill + watermark logic requires the orchestrator integration from 33-001. |
| SBOM-SERVICE-21-001 | TODO | Link-Not-Merge v1 frozen (2025-11-17); proceed with projection schema + fixtures. |
| SBOM-SERVICE-21-002 | TODO | Depends on 21-001 implementation; schema now frozen. |
| SBOM-SERVICE-21-003 | TODO | Entry point/service node management follows 21-002; proceed with stub fixtures. |
| SBOM-SERVICE-21-004 | TODO | Observability wiring to follow 21-003; unblock with mock feeds. |
| SBOM-SERVICE-23-001 | TODO | Asset metadata extensions queued once 21-004 observability baseline exists. |
| SBOM-SERVICE-23-002 | TODO | Asset update events depend on 23-001 schema. |
| SBOM-VULN-29-001 | TODO | Inventory evidence feed deferred until projection schema + runtime align. |
| SBOM-VULN-29-002 | TODO | Resolver feed requires 29-001 event payloads. |
### 140.C Signals
| Task ID | State | Notes |
| --- | --- | --- |
| SIGNALS-24-001 | DONE (2025-11-09) | Host skeleton, RBAC, sealed-mode readiness, `/signals/facts/{subject}` retrieval, and readiness probes merged; serves as base for downstream ingestion. |
| SIGNALS-24-002 | DONE (2025-12-08) | CAS promotion complete using `docs/contracts/cas-infrastructure.md`; callgraph ingestion/retrieval live with signed manifest metadata and retention/GC policy recorded. |
| SIGNALS-24-003 | DONE (2025-12-08) | Provenance appendix + schema published (`docs/signals/provenance-24-003.md`, `docs/schemas/provenance-feed.schema.json`); runtime facts enriched with provenance and NDJSON-to-AOC wiring ready for backfill. |
| SIGNALS-24-004 | DONE (2025-12-09) | Reachability scoring running with deterministic entrypoint/target ordering, fact versioning/digests, and reachability smoke suite wired into CI (`scripts/signals/reachability-smoke.sh`). |
| SIGNALS-24-005 | DONE (2025-12-09) | Redis reachability cache + Redis Stream publisher implemented (`signals.fact.updated.v1`/DLQ) with deterministic envelopes (event_id, fact_version, fact.digest). CI pipeline signs/uploads evidence with prod key via secrets/vars. |
### 140.D Zastava
| Task ID | State | Notes |
| --- | --- | --- |
| ZASTAVA-ENV-01 | TODO | Observer adoption of Surface.Env helpers paused while Surface.FS cache contract finalizes. |
| ZASTAVA-ENV-02 | TODO | Webhook helper migration follows ENV-01 completion. |
| ZASTAVA-SECRETS-01 | TODO | Surface.Secrets wiring for Observer pending published cache endpoints. |
| ZASTAVA-SECRETS-02 | TODO | Webhook secret retrieval cascades from SECRETS-01 work. |
| ZASTAVA-SURFACE-01 | TODO | Surface.FS client integration blocked on Scanner layer metadata; tests ready once packages mirror offline dependencies. |
| ZASTAVA-SURFACE-02 | TODO | Admission enforcement requires SURFACE-01 so webhook responses can gate on cache freshness. |
## In-flight focus (DOING items)
| Task ID | Remaining work | Target date | Owners |
| --- | --- | --- | --- |
| GRAPH-INDEX-28-007 | Continue execution on scanner surface mock bundle v1; revalidate outputs once real cache drops and manifests are available. | TBD (await cache ETA) | Graph Indexer Guild · Observability Guild |
Signals DOING cleared (24-002/003 DONE). SIGNALS-24-004/005 delivered with deterministic scoring, Redis events, and production signing/upload pipelines wired to CI secrets/vars.
### Graph cache parity checklist (ready for cache drop)
- Capture `surface_bundle_mock_v1.tgz` hash and record node/edge counts, cluster counts, and checksum of emitted fixtures.
- Define tolerant variance thresholds for clustering/centrality determinism (e.g., Louvain modularity delta ≤ 0.001 across runs).
- Prepare rerun script to diff mock vs real cache outputs (IDs, cluster labels, metrics) and emit NDJSON of divergences.
- Track CPU/memory/runtime metrics for mock vs cache replays to spot performance regressions.
- Export minimal fixtures for downstream consumers (Graph UI overlays, Zastava surface) after real-cache validation.
### Signals CAS/provenance close-out checklist
- Confirm CAS checklist is approved (or list blockers) and record timestamps of approval decision.
- Merge signed manifest PRs and publish manifest metadata (path, hash, signer key ID, retention/GC policy).
- Freeze provenance appendix: final field list, scope propagation fixtures, and NDJSON examples committed to repo.
- Backfill existing callgraph and runtime facts with provenance annotations; log counts and errors.
- Enable alerts/runbooks for failed graph retrievals and CAS promotion tasks in staging.
- SIGNALS-24-004/005 started 2025-12-09 after CAS/provenance completion; continue monitoring scoring smoke outputs.
## Wave readiness checklist (2025-11-18)
| Wave | Entry criteria | Prep status | Next checkpoint |
| --- | --- | --- | --- |
| 140.A Graph | Scanner surface analyzer artifacts + SBOM projection schema for clustering inputs. | Executing on scanner surface mock bundle v1; determinism harness drafted; Scanner cache ETA still pending for parity validation. | 2025-11-19 cross-guild follow-up to confirm cache drop timeline. |
| 140.B SbomService | Concelier Link-Not-Merge + Cartographer projection schema, plus AirGap parity review. | Projection doc redlines complete; schema doc ready for Concelier feedback. | 2025-11-14 schema review (Concelier, Cartographer, SBOM). |
| 140.C Signals | CAS promotion approval + runtime provenance contract + AUTH-SIG-26-001 sign-off. | HOST + callgraph retrieval merged; CAS/provenance work tracked in DOING table above. | 2025-11-13 runtime sync to approve CAS rollout + schema freeze. |
| 140.D Zastava | Surface.FS cache availability + Surface.Env helper specs published. | Env/secrets design notes ready; waiting for Scanner cache drop and Surface.FS API stubs. | 2025-11-15 Surface guild office hours to confirm helper adoption plan. |
### Signals DOING activity log (updates through 2025-11-13)
| Date | Update | Owners |
| --- | --- | --- |
| 2025-11-12 | Drafted CAS promotion checklist (bucket policies, signer config, GC guardrails) and circulated to Platform Storage for approval; added alert runbooks for failed graph retrievals. | Signals Guild, Platform Storage Guild |
| 2025-11-11 | Completed NDJSON ingestion soak test (JSON/NDJSON + gzip) and documented provenance enrichment mapping required from Authority scopes; open PR wiring AOC metadata pending review. | Signals Guild, Runtime Guild |
| 2025-11-09 | Runtime facts ingestion endpoint + streaming NDJSON support merged with sealed-mode gating; next tasks are provenance enrichment and scoring linkage. | Signals Guild, Runtime Guild |
## Dependency status watchlist (2025-11-13)
| Dependency | Status | Latest detail | Owner(s) / follow-up |
| --- | --- | --- | --- |
| AUTH-SIG-26-001 (Signals scopes + AOC) | DONE (2025-10-29) | Authority shipped scope + role templates; Signals is validating propagation + provenance enrichment before enabling scoring. | Authority Guild · Runtime Guild · Signals Guild |
| CONCELIER-GRAPH-21-001 (SBOM projection enrichment) | DONE (2025-11-18) | LNM v1 fixtures landed; normalization + graph acceptance tests green. | Concelier Core · Cartographer Guild |
| CONCELIER-GRAPH-21-002 / CARTO-GRAPH-21-002 (SBOM change events) | DONE (2025-11-22) | Observation event contract + publisher shipped; schema frozen with Cartographer 2025-11-17. | Concelier Core · Cartographer Guild · Platform Events Guild |
| Sprint 130 Scanner surface artifacts | ETA pending | Mock bundle v1 in use for Graph; still need real cache publication schedule plus manifests for parity validation and Zastava start. | Scanner Guild · Graph Indexer Guild · Zastava Guilds |
| AirGap parity review (Sprint 120.A) | Not scheduled | SBOM path/timeline endpoints must re-pass AirGap checklist once Concelier schema lands; reviewers on standby. | AirGap Guild · SBOM Service Guild |
## Upcoming checkpoints (updated 2025-11-23)
| Date | Session | Goal | Impacted wave(s) | Prep owner(s) |
| --- | --- | --- | --- | --- |
| 2025-11-13 | Scanner ↔ Graph readiness sync | Lock analyzer artifact ETA + cache publish plan so GRAPH-INDEX-28-007 can start immediately after delivery. | 140.A Graph · 140.D Zastava | Scanner Guild · Graph Indexer Guild |
| 2025-11-13 | Runtime/Signals CAS + provenance review | Approve CAS promotion checklist, freeze provenance schema, and green-light SIGNALS-24-002/003 close-out tasks. | 140.C Signals | Signals Guild · Runtime Guild · Authority Guild · Platform Storage Guild |
| 2025-11-14 | Concelier/Cartographer/SBOM schema review | Ratify Link-Not-Merge projection schema + change event contract; schedule AirGap parity verification. | 140.B SbomService · 140.A Graph · 140.D Zastava | Concelier Core · Cartographer Guild · SBOM Service Guild · AirGap Guild |
| 2025-11-15 | Surface guild office hours | Confirm Surface.Env helper adoption + Surface.FS cache drop timeline for Zastava. | 140.D Zastava | Surface Guild · Zastava Observer/Webhook Guilds |
| 2025-11-23 | AirGap parity review (SBOM paths/versions/events) | Validate LNM fixtures, record hashes, and approve SBOM-SERVICE-21-001 start. | 140.B SbomService | SBOM Service Guild · Cartographer Guild · AirGap Guild |
### Meeting prep checklist
| Session | Pre-reads / artifacts | Open questions to resolve | Owners |
| --- | --- | --- | --- |
| Scanner ↔ Graph (2025-11-13) | Sprint 130 surface artifact roadmap draft, GRAPH-INDEX-28-007 scaffolds, ZASTAVA-SURFACE dependency list. | Exact drop date for analyzer artifacts? Will caches ship phased or all at once? Need mock payloads if delayed? | Scanner Guild · Graph Indexer Guild · Zastava Guilds |
| Runtime/Signals CAS review (2025-11-13) | CAS promotion checklist, signed manifest PR links, provenance schema draft, NDJSON ingestion soak results. | Storage approval on bucket policies/GC? Authority confirmation on scope propagation + AOC metadata? Backfill approach for existing runtime facts? | Signals Guild · Runtime Guild · Authority Guild · Platform Storage Guild |
| Concelier schema review (2025-11-14) | Link-Not-Merge schema redlines, Cartographer webhook contract, AirGap parity checklist, SBOM-SERVICE-21-001 scaffolding plan. | Final field list for relationships/scopes? Event payload metadata requirements? AirGap review schedule & owners? | Concelier Core · Cartographer Guild · SBOM Service Guild · AirGap Guild |
| Surface guild office hours (2025-11-15) | Surface.Env helper adoption notes, sealed-mode test harness outline, Surface.FS API stub timeline. | Can Surface.FS caches publish before Analyzer drop? Any additional sealed-mode requirements? Who owns Surface.Env rollout in Observer/Webhook repos? | Surface Guild · Zastava Observer/Webhook Guilds |
## Target outcomes (through 2025-11-15, refreshed 2025-11-13)
| Deliverable | Target date | Status | Dependencies / notes |
| --- | --- | --- | --- |
| SIGNALS-24-002 CAS promotion + signed manifests | 2025-11-14 | BLOCKED | Waiting on Platform Storage approval; CAS checklist published (`docs/signals/cas-promotion-24-002.md`). |
| SIGNALS-24-003 provenance enrichment + backfill | 2025-11-15 | BLOCKED | Await provenance appendix freeze/approval; checklist published (`docs/signals/provenance-24-003.md`). |
| Scanner analyzer artifact ETA & cache drop plan | 2025-11-13 | TODO | Scanner to publish Sprint 130 surface roadmap; Graph/Zastava blocked until then. |
| Concelier Link-Not-Merge schema ratified | 2025-11-14 | DONE | Agreement signed 2025-11-17; CONCELIER-GRAPH-21-001 and CARTO-GRAPH-21-002 implemented with observation event publisher 2025-11-22. AirGap review next. |
| Surface.Env helper adoption checklist | 2025-11-15 | TODO | Zastava guild preparing sealed-mode test harness; depends on Surface guild office hours outcomes. |
## Decisions needed (before 2025-11-15, refreshed 2025-11-13)
| Decision | Blocking work | Accountable owner(s) | Due date |
| --- | --- | --- | --- |
| Approve CAS bucket policies + signed manifest rollout | Closing SIGNALS-24-002; enabling scoring/cache prep | Platform Storage Guild · Signals Guild | 2025-11-13 |
| Freeze runtime provenance schema + scope propagation fixtures | Completing SIGNALS-24-003 enrichment/backfill | Runtime Guild · Authority Guild | 2025-11-13 |
| Publish Sprint 130 analyzer artifact drop schedule | Starting GRAPH-INDEX-28-007 and ZASTAVA-SURFACE-01/02 | Scanner Guild | 2025-11-13 |
| Ratify Link-Not-Merge schema + change event contract | Kicking off SBOM-SERVICE-21-001/002 and Graph overlays | Concelier Core · Cartographer Guild · SBOM Service Guild | 2025-11-14 |
| Schedule AirGap parity review for SBOM endpoints | Allowing Advisory AI adoption and AirGap sign-off | AirGap Guild · SBOM Service Guild | 2025-11-14 |
| Assign owner for Surface.Env helper rollout (Observer vs Webhook) | Executing ZASTAVA-ENV-01/02 once caches drop | Surface Guild · Zastava Guilds | 2025-11-15 |
## Contingency playbook (reviewed 2025-11-13)
| Risk trigger | Immediate response | Owner | Escalation window |
| --- | --- | --- | --- |
| CAS promotion review slips past 2025-11-13 | Switch SIGNALS-24-002 to “red”, keep staging in shadow bucket, and escalate to Platform Storage leadership for expedited review. | Signals Guild | Escalate by 2025-11-14 stand-up. |
| Runtime provenance schema disputes persist | Freeze ingestion on current schema, log breaking field requests, and schedule joint Runtime/Authority architecture review. | Runtime Guild · Authority Guild | Escalate by 2025-11-14 EOD. |
| Scanner cannot provide analyzer artifact ETA | Raise blocker in Scanner leadership channel, request interim mock manifests, and re-plan Graph/Zastava scope to focus on harness/test prep. | Graph Indexer Guild · Zastava Guilds | Escalate by 2025-11-14 midday. |
| Concelier/Cartographer schema review stalls | Capture outstanding fields/issues, loop in Advisory AI + AirGap leadership, and evaluate temporary schema adapters for SBOM Service. | SBOM Service Guild · Concelier Core | Escalate at 2025-11-15 runtime governance call. |
| Surface.Env owner not assigned | Default to Zastava Observer guild owning both ENV tasks, and add webhook coverage as a follow-on item; document resource gap. | Surface Guild · Zastava Observer Guild | Escalate by 2025-11-16. |
## Action item tracker (status as of 2025-12-09)
| Item | Status | Next step | Owner(s) | Due |
| --- | --- | --- | --- | --- |
| Prod DSSE re-sign (Signals gaps) | ✅ DONE (pipeline ready 2025-12-09) | CI workflows `signals-reachability.yml` / `signals-evidence-locker.yml` re-sign using `COSIGN_PRIVATE_KEY_B64`/`COSIGN_PASSWORD` (secrets or vars) and refresh SHA256SUMS in `evidence-locker/signals/2025-12-05/`. Configure secrets in CI to execute. | Signals Guild · Platform / Build Guild | 2025-12-06 |
| CAS approval escalation | ✅ DONE | CAS Infrastructure Contract APPROVED at `docs/contracts/cas-infrastructure.md` (2025-12-06); SIGNALS-24-002 unblocked. | Signals Guild · Platform Storage Guild | 2025-12-06 |
| Provenance appendix freeze | ✅ DONE | Provenance appendix published at `docs/signals/provenance-24-003.md`; schema at `docs/schemas/provenance-feed.schema.json`. SIGNALS-24-003 unblocked. | Runtime Guild · Authority Guild | 2025-12-07 |
| Upload signals evidence to locker | ✅ DONE (pipeline ready 2025-12-09) | `signals-evidence-locker.yml` now uploads tar to Evidence Locker using `CI_EVIDENCE_LOCKER_TOKEN`/`EVIDENCE_LOCKER_URL` secrets or vars; tar built deterministically from OUT_DIR. Configure locker creds in CI to run. | Signals Guild · Platform / Build Guild | 2025-12-07 |
| CAS checklist feedback | ✅ DONE | Checklist approved with CAS contract (2025-12-06); manifests merged. | Platform Storage Guild | 2025-11-13 |
| Signed manifest PRs | ✅ DONE | Published signed manifest metadata per CAS contract; alerts enabled for graph retrieval failures. | Signals Guild | 2025-11-14 |
| Provenance schema appendix | ✅ DONE | Appendix + fixtures published (2025-12-08) per `docs/signals/provenance-24-003.md` and `docs/schemas/provenance-feed.schema.json`. | Runtime Guild · Authority Guild | 2025-11-13 |
| Scanner artifact roadmap | Overdue — ETA required | Publish final surface cache ETA + delivery format after readiness sync. | Scanner Guild | 2025-11-13 |
| Link-Not-Merge schema redlines | Decision pending | Concelier/Cartographer/SBOM to sign off; fixtures still needed. | Concelier Core · Cartographer Guild · SBOM Service Guild | 2025-11-14 |
| Surface.Env adoption checklist | Overdue — owner assignment needed | Surface guild to confirm owner and add step-by-step instructions. | Surface Guild · Zastava Guilds | 2025-11-15 |
## Standup agenda (2025-11-18)
| Track | Questions / updates to cover | Owner ready to report |
| --- | --- | --- |
| 140.A Graph | Confirm Scanner cache ETA; align parity checklist and revalidation plan once caches land. | Graph Indexer Guild |
| 140.B SbomService | LNM fixtures and schema sign-off status? AirGap review scheduling? | SBOM Service Guild |
| 140.C Signals | CAS approval + signed manifest merge status; provenance appendix publication; backfill start date. | Signals Guild · Runtime Guild · Authority Guild |
| 140.D Zastava | Surface.FS cache drop plan and Surface.Env owner assignment; any sealed-mode gaps. | Zastava Guilds |
| Cross-track | Upcoming decisions/risks from the contingency playbook that need leadership visibility today? | Sprint 140 leads |
# Blockers & coordination
- **Concelier Link-Not-Merge / Cartographer schemas** – SBOM-SERVICE-21-001..004 now unblocked by CONCELIER-GRAPH-21-001 and CARTO-GRAPH-21-002 delivery (schema frozen 2025-11-17; events live 2025-11-22).
- **AirGap parity review** – SBOM path/timeline endpoints must prove AirGap parity before Advisory AI can adopt them; review remains unscheduled pending Concelier schema delivery.
- **Scanner surface artifacts** – GRAPH-INDEX-28-007+ and all ZASTAVA-SURFACE tasks depend on Sprint 130 analyzer outputs and cached layer metadata; need updated ETA from Scanner guild.
- **Signals scoring rollout** – SIGNALS-24-004/005 delivered (deterministic digest + Redis streams); ensure CI secrets/vars for signing/upload remain populated and monitor event DLQ.
# Next actions (target: 2025-11-20)
| Owner(s) | Action |
| --- | --- |
| Graph Indexer Guild | Running GRAPH-INDEX-28-007 on mock bundle v1; need Scanner to provide cache ETA/manifests to revalidate and shift to real inputs; parity checklist ready for cache drop. |
| SBOM Service Guild | Secure LNM fixtures and schema sign-off; schedule AirGap review; be ready to scaffold SBOM-SERVICE-21-001 once fixtures land. |
| Signals Guild | Escalate CAS promotion + signed manifest approval; merge once approved; start provenance enrichment/backfill (SIGNALS-24-003). |
| Runtime & Authority Guilds | Publish final provenance appendix + fixtures; confirm scope propagation; unblock SIGNALS-24-003 backfill. |
| Platform Storage Guild | Deliver CAS bucket policy sign-off to unblock SIGNALS-24-002. |
| Scanner Guild | Publish surface cache ETA/hash and manifests; unblock Graph revalidation and Zastava Surface tasks. |
| Zastava Guilds | Assign Surface.Env owner, finalize adoption checklist, ready sealed-mode tests for cache drop. |
# Next actions (target: 2025-12-07)
| Owner(s) | Action |
| --- | --- |
| Signals Guild · Runtime Guild | ✅ Completed 2025-12-09: reachability scoring running with deterministic digests/fact.version; smoke suite enforced via scripts/signals/reachability-smoke.sh. |
| Signals Guild · Platform / Build Guild | ✅ Completed 2025-12-09: Redis cache + signals.fact.updated.v1 stream publisher live with DLQ and deterministic envelopes. |
| Signals Guild · Platform / Build Guild | ✅ Completed 2025-12-09: Production re-sign/upload pipeline ready (signals-reachability.yml, signals-evidence-locker.yml) using CI secrets/vars. |
# Downstream dependency rollup (snapshot: 2025-11-13)
| Track | Dependent sprint(s) | Impact if delayed |
| --- | --- | --- |
| 140.A Graph | `docs/implplan/SPRINT_141_graph.md` (Graph clustering/backfill) and downstream Graph UI overlays | Graph insights, policy overlays, and runtime clustering views cannot progress without GRAPH-INDEX-28-007+ landing. |
| 140.B SbomService | `docs/implplan/SPRINT_142_sbomservice.md`, Advisory AI (Sprint 111), Policy/Vuln Explorer feeds | SBOM projections/events stay unavailable, blocking Advisory AI remedation heuristics, policy joins, and Vuln Explorer candidate generation. |
| 140.C Signals | docs/implplan/SPRINT_143_signals.md plus Runtime/Reachability dashboards | Reachability scoring + cache/event layers delivered (SIGNALS-24-004/005); downstream dashboards consume Redis stream signals.fact.updated.v1 once locker/CI secrets are configured. |
| 140.D Zastava | `docs/implplan/SPRINT_0144_0001_0001_zastava_runtime_signals.md`, Runtime admission enforcement | Surface-integrated drift/admission hooks remain stalled; sealed-mode env helpers cannot ship without Surface.FS metadata. |
# Risk log
| Risk | Impact | Mitigation / owner |
| --- | --- | --- |
| LNM fixtures (staged 2025-11-22) | SBOM-SERVICE-21-001..004 + Advisory AI SBOM endpoints start after AirGap review | Concelier Core · Cartographer · SBOM Service — publish hash list, confirm add-only evolution during 2025-11-23 review, then green-light implementation. |
| Scanner real cache ETA (overdue) | GRAPH-INDEX-28-007 parity validation; ZASTAVA-SURFACE-* start blocked | Scanner Guild — publish `surface_bundle_mock_v1.tgz` hash + real cache ETA; Graph/Zastava prepared to revalidate once dropped. |
| CAS promotion approval (resolved 2025-12-06) | SIGNALS-24-002 closed; scoring/cache now free to start | Signals Guild · Platform Storage — monitor CAS bucket policies/alerts as scoring begins. |
| Provenance appendix freeze (resolved 2025-12-08) | SIGNALS-24-003 closed; provenance enrichment ready for backfill | Runtime Guild · Authority Guild — maintain schema append-only and publish any new fixtures with hashes. |
| Surface.FS cache drop + Surface.Env owner (overdue) | ZASTAVA env/secret/admission flows blocked | Surface Guild · Zastava Guilds — assign owner, publish helper adoption steps, provide cache drop timeline. |
| Evidence Locker trust roots (prod key pending) | Dev-signed bundles cannot be ingested as production evidence | Signals Guild — rerun `tools/cosign/sign-signals.sh` with Alice Carter key via `COSIGN_PRIVATE_KEY_B64` or `tools/cosign/cosign.key`; replace bundles in `evidence-locker/signals/2025-12-05/`. |
# Coordination log
| Date | Notes |
| --- | --- |
| 2025-11-17 | Coordinator decisions: LNM v1 frozen; scanner mock bundle ordered; Surface.FS CI cache approved; SBOM-SERVICE-21-001..004 and GRAPH-INDEX-28-007 switched to TODO. |
| 2025-11-17 | Marked Graph/Zastava waves BLOCKED (missing Sprint 130 analyzer ETA); escalated to Scanner leadership per contingency. |
| 2025-11-13 | Snapshot, wave tracker, meeting prep, and action items refreshed ahead of Nov 13 checkpoints; awaiting outcomes before flipping statuses. |
| 2025-11-11 | Runtime + Signals ran NDJSON ingestion soak test; Authority flagged remaining provenance fields for schema freeze ahead of 2025-11-13 sync. |
| 2025-11-09 | Sprint 140 snapshot refreshed; awaiting Scanner surface artifact ETA, Concelier/CARTO schema delivery, and Signals host merge before any wave can advance to DOING. |
# Sprint 140 - Runtime & Signals

151
docs/implplan/archived/SPRINT_0170_0001_0001_notifications_telemetry.md
View File

@@ -1,151 +0,0 @@
# Sprint 0170-0001-0001 · Notifications & Telemetry Snapshot
## Topic & Scope
- Coordination snapshot for Notifications (Wave 170.A Notifier) and Telemetry (Wave 170.B); execution lives in `SPRINT_0171_0001_0001_notifier_i.md` and `SPRINT_0174_0001_0001_telemetry.md`.
- Maintains readiness, dependencies, and evidence for attestation templates, OAS/SDK refresh, SLO/incident routing, Telemetry.Core bootstrap, and sealed-mode controls.
- Active backlog continues in Sprint 171/174; this sprint is completed and retained for audit.
- **Working directory:** `docs/implplan` (coordination only).
## Dependencies & Concurrency
- Upstream: Sprint 150.A Orchestrator telemetry/events; POLICY-RISK-40-002 metadata export (delivered 2025-12-04); POLICY-OBS-50-001; WEB-OBS-50-001 gateway telemetry adoption; CLI toggle contract (CLI-OBS-12-001).
- Concurrency: Waves 170.A and 170.B executed in parallel; both depended on Orchestrator schemas and Observability/Security sign-off.
- Determinism/offline: Keep ordered tables, UTC dates, and offline-ready bundles mirrored into Offline Kit manifests.
## Documentation Prerequisites
- docs/README.md
- docs/07_HIGH_LEVEL_ARCHITECTURE.md
- docs/modules/platform/architecture-overview.md
- docs/modules/notifications/architecture.md
- docs/modules/telemetry/architecture.md
- docs/notifications/templates.md
## Delivery Tracker
| # | Track | Status | Key dependency / next step | Owners | Notes |
| --- | --- | --- | --- | --- | --- |
| 1 | 170.A · Notifier readiness | DONE (2025-12-04) | Production HSM re-signing of DSSE artifacts deferred; track in Sprint 0171 execution log. | Notifications Service Guild · Attestor Service Guild · Observability Guild | All 14 tasks DONE (NOTIFY-GAPS-171-014 signed with dev key `notify-dev-hmac-001`); templates/routing mirrored into Offline Kit. |
| 2 | 170.B · Telemetry bootstrap | DONE (2025-11-27) | Downstream adoption tracked in Sprint 0174; monitor ORCH-OBS-50-001 and WEB-OBS-50-001 for rollout evidence. | Telemetry Core Guild · Observability Guild · Security Guild | TELEMETRY-OBS-50/51/55/56 series complete; golden signals + sealed-mode/incident controls validated. |
## Wave Coordination
| Wave | Guild owners | Shared prerequisites | Status | Notes |
| --- | --- | --- | --- | --- |
| 170.A Notifier | Notifications Service Guild · Attestor Service Guild · Observability Guild | Sprint 150.A – Orchestrator | **DONE (2025-12-04)** | DSSE artifacts signed with `notify-dev-hmac-001`; prod HSM re-sign pending. |
| 170.B Telemetry | Telemetry Core Guild · Observability Guild · Security Guild | Sprint 150.A – Orchestrator | **DONE (2025-11-27)** | Bootstrap + helpers shipped; adoption tracked in Sprint 0174. |
## Wave Detail Snapshots
### Wave 170.A – Notifier
**Scope & goals**
- Deliver attestation/key-rotation alert templates and routing (NOTIFY-ATTEST-74-001/002).
- Refresh Notifier OpenAPI/SDK surface (`NOTIFY-OAS-61-001` → `NOTIFY-OAS-63-001`) for Console/CLI consumers.
- Wire SLO/incident inputs into rules (NOTIFY-OBS-51-001/55-001) and extend risk-profile routing (NOTIFY-RISK-66-001 → 68-001) without regressing quiet-hours/dedup.
- Preserve Offline Kit and documentation parity (NOTIFY-DOC-70-001, NOTIFY-AIRGAP-56-002).
**Entry criteria**
- Orchestrator job attest events flowing to Notify bus with Attestor-approved fixtures.
- Quiet-hours/digest backlog reconciled (`docs/notifications/*.md` clean).
- Observability Guild sign-off on telemetry fields reused by Notifier SLO webhooks.
**Exit criteria**
- NOTIFY-ATTEST/OAS/OBS/RISK tasks DONE with doc updates.
- Templates promoted to Offline Kit manifests; sample payloads stored under `docs/notifications/templates.md`.
- Incident mode notifications exercised in staging with audit logs + DSSE evidence.
**Task clusters (final state)**
| Cluster | Linked tasks | Owners | Final state | Notes |
| --- | --- | --- | --- | --- |
| Attestation / key lifecycle alerts | NOTIFY-ATTEST-74-001/74-002 | Notifications Service Guild · Attestor Service Guild | DONE | Templates + wiring complete (2025-11-16/27); Rekor witness payload contract frozen. |
| API/OAS refresh & SDK parity | NOTIFY-OAS-61-001 → NOTIFY-OAS-63-001 | Notifications Service Guild · API Contracts Guild · SDK Generator Guild | DONE | Contract frozen 2025-11-15; SDK generator aligned with `/notifications/rules` schema. |
| Observability-driven triggers | NOTIFY-OBS-51-001/55-001 | Notifications Service Guild · Observability Guild | DONE | SLO webhook + incident mode templates shipped (2025-11-22). |
| Risk profile routing | NOTIFY-RISK-66-001 → NOTIFY-RISK-68-001 | Notifications Service Guild · Risk Engine Guild · Policy Guild | DONE | Risk-events endpoint + routing seeds shipped (2025-11-24); enriched via POLICY-RISK-40-002 metadata export. |
| Docs & offline parity | NOTIFY-DOC-70-001, NOTIFY-AIRGAP-56-002 | Notifications Service Guild · DevOps Guild | DONE | GA checklists and offline kit parity complete; no further edits needed. |
| Gap remediation | NOTIFY-GAPS-171-014 | Notifications Service Guild | DONE | NR1–NR10 artifacts signed with dev key `notify-dev-hmac-001` (2025-12-04); prod HSM re-sign pending. |
**Observability checkpoints**
- Align metric names/labels with `docs/notifications/architecture.md#12-observability-prometheus--otel`.
- Ensure spans/logs include tenant, ruleId, actionId, and `attestation_event_id` for attestation-triggered templates.
- Capture incident notification smoke tests via `ops/devops/telemetry/tenant_isolation_smoke.py`.
### Wave 170.B – Telemetry
**Scope & goals**
- Ship `StellaOps.Telemetry.Core` bootstrap + propagation helpers (TELEMETRY-OBS-50-001/50-002).
- Provide golden-signal helpers + scrubbing/PII safety nets (TELEMETRY-OBS-51-001/51-002).
- Implement incident + sealed-mode toggles (TELEMETRY-OBS-55-001/56-001) and document integration contracts for Orchestrator, Policy, Task Runner, Gateway (WEB-OBS-50-001).
**Entry criteria**
- Orchestrator + Policy hosts expose telemetry bootstrap extension points (ORCH-OBS-50-001 / POLICY-OBS-50-001).
- Observability Guild reviewed storage footprint impacts (docs/modules/telemetry/architecture.md §2).
- Security Guild approval on redaction defaults + tenant override audit logging.
**Exit criteria**
- Core library published to `/local-nugets` and referenced by Orchestrator & Policy integration branches.
- Context propagation middleware validated through HTTP/gRPC/job smoke tests with deterministic trace IDs.
- Incident/sealed-mode toggles wired into CLI + Notify hooks with runbooks updated.
**Task clusters (final state)**
| Cluster | Linked tasks | Owners | Final state | Notes |
| --- | --- | --- | --- | --- |
| Bootstrap & propagation | TELEMETRY-OBS-50-001/50-002 | Telemetry Core Guild | DONE | Core bootstrap (50-001) 2025-11-19; propagation middleware (50-002) 2025-11-27. |
| Metrics helpers + scrubbing | TELEMETRY-OBS-51-001/51-002 | Telemetry Core Guild · Observability Guild · Security Guild | DONE | Golden signals with cardinality guards + scrubbing filters (2025-11-27). |
| Incident & sealed-mode controls | TELEMETRY-OBS-55-001/56-001 | Telemetry Core Guild · Observability Guild | DONE | Incident mode toggle + sealed-mode helpers validated 2025-11-27; CLI toggle contract tracked via CLI-OBS-12-001. |
**Tooling & validation**
- Smoke: `ops/devops/telemetry/smoke_otel_collector.py` + `tenant_isolation_smoke.py` per profile (default/forensic/airgap).
- Offline bundle packaging: `ops/devops/telemetry/package_offline_bundle.py` (collectors, dashboards, manifests).
- Incident simulation: `ops/devops/telemetry/generate_dev_tls.sh` for local collector certs during sealed-mode testing.
## Interlocks (External Dependencies)
| Dependency | Source sprint / doc | Current state | Impact on waves |
| --- | --- | --- | --- |
| Sprint 150.A – Orchestrator (wave table) | `SPRINT_150_scheduling_automation.md` | TODO | Blocks visibility of job events for Notify templates and Telemetry samples until orchestration telemetry lands. |
| ORCH-OBS-50-001 `orchestrator instrumentation` | Sprint 150 backlog | TODO | Needed for Telemetry.Core sample + Notify SLO hooks; monitor for slip. |
| POLICY-OBS-50-001 `policy instrumentation` | Sprint 150 backlog | TODO | Required before Telemetry helpers can be adopted by Policy + risk routing. |
| WEB-OBS-50-001 `gateway telemetry core adoption` | Sprint 214/215 backlogs | TODO | Ensures web/gateway emits trace IDs that Notify incident payload references. |
| POLICY-RISK-40-002 `risk profile metadata export` | Sprint 215+ (Policy) | DONE (2025-12-04) | Provides metadata enrichment for NOTIFY-RISK routes; unblocked. |
## Upcoming Checkpoints (historical)
| Target date | Milestone | Owners | Dependency notes |
| --- | --- | --- | --- |
| 2025-11-13 | Finalize attestation payload schema + template variables | Notifications Service Guild · Attestor Service Guild | Unblocked NOTIFY-ATTEST-74-001/002 + Telemetry incident span labels. |
| 2025-11-15 | Publish draft Notifier OAS + SDK snippets | Notifications Service Guild · API Contracts Guild | Required for CLI/UI adoption; prereq for NOTIFY-OAS-61/62 series. |
| 2025-11-18 | Land Telemetry.Core bootstrap sample in Orchestrator | Telemetry Core Guild · Orchestrator Guild | Demonstrated TELEMETRY-OBS-50-001 viability; prerequisite for Policy adoption + Notify SLO hooks. |
| 2025-11-20 | Incident/quiet-hour end-to-end rehearsal | Notifications Service Guild · Telemetry Core Guild · Observability Guild | Validated TELEMETRY-OBS-55-001 + NOTIFY-OBS-55-001 + CLI toggle contract. |
| 2025-11-22 | Offline kit bundle refresh (notifications + telemetry assets) | DevOps Guild · Notifications Service Guild · Telemetry Core Guild | Ensured offline-kit manifests reference new templates/configs. |
## Action Tracker
| # | Action | Owner | Next signal/date | Notes |
| --- | --- | --- | --- | --- |
| 1 | Re-sign DSSE artifacts with production HSM key | Notifications Service Guild · Security Guild | Track in Sprint 0171 execution log; target date TBD | Dev signing key `notify-dev-hmac-001` used for initial signatures. |
| 2 | Resolve missing legacy dependency `StellaOps.Notify.Storage.Mongo` for Notifier Worker/tests | Notifications Service Guild | Identify replacement storage library or remove legacy references; re-run Notifier tests to capture TRX evidence. | Blocks `dotnet test` in Sprint 0171 (2025-12-05 attempt failed). |
| 3 | Restore Moq package for Telemetry Core tests | Telemetry Core Guild | DONE 2025-12-06 | Moq restored from curated feed; Telemetry Core tests now green. |
| 4 | Record telemetry test evidence | Telemetry Core Guild | DONE 2025-12-06 | Evidence attached: `src/Telemetry/StellaOps.Telemetry.Core/StellaOps.Telemetry.Core.Tests/TestResults/TestResults/telemetry-tests.trx`. |
## Decisions & Risks
| Decision / Risk | Status | Mitigation / Notes |
| --- | --- | --- |
| Telemetry data drift in sealed mode | Ongoing | Enforce `IEgressPolicy` checks (TELEMETRY-OBS-56-001); schedule smoke runs after each config change. |
| Template/API divergence across Notifier SDKs | Ongoing | Freeze OAS/SDK in Sprint 0171; require API Contracts review before merging; keep `/notifications/rules` as source of truth. |
| Observability storage overhead | Ongoing | Coordinate retention with Ops per docs/modules/telemetry/architecture.md §2 when SLO webhooks and incident toggles increase cardinality. |
| Cross-sprint dependency churn (ORCH-OBS-50-001, POLICY-OBS-50-001, WEB-OBS-50-001) | Ongoing | Weekly check; re-baseline Telemetry/Notifier triggers if upstream slips. |
| Risk routing metadata availability | Resolved 2025-12-04 | POLICY-RISK-40-002 delivered (`GET /api/risk/profiles/{id}/metadata`), enabling NOTIFY-RISK enrichment. |
| DSSE signing posture | Partially resolved | Dev key `notify-dev-hmac-001` used; production HSM re-sign pending (Action #1). |
## Execution Log
| Date (UTC) | Update | Owner |
| --- | --- | --- |
| 2025-11-12 10:15 | Wave rows flipped to DOING; recorded scope/entry/exit criteria for Notifier and Telemetry waves. | Observability Guild · Notifications Service Guild |
| 2025-11-12 14:40 | Added task mirror + dependency tracker + milestone table to align with Sprint 171/174 execution plans. | Observability Guild |
| 2025-11-12 18:05 | Marked NOTIFY-ATTEST-74-001, NOTIFY-OAS-61-001, and TELEMETRY-OBS-50-001 as DOING; noted gated follow-ups. | Notifications Service Guild · Telemetry Core Guild |
| 2025-11-12 19:20 | Documented attestation template suite in `docs/notifications/templates.md` to unblock NOTIFY-ATTEST-74-001 and updated sprint mirrors. | Notifications Service Guild |
| 2025-11-12 19:32 | Synced notifications architecture doc to reference the attestation template suite for downstream visibility. | Notifications Service Guild |
| 2025-11-12 19:45 | Updated notifications overview + rules docs with `tmpl-attest-*` requirements for rule authors/operators. | Notifications Service Guild |
| 2025-11-12 20:05 | Published baseline Offline Kit templates under `offline/notifier/templates/attestation/` for Slack/Email/Webhook. | Notifications Service Guild |
| 2025-11-19 | Re-baselined tracks: set 170.A and 170.B to BLOCKED pending CI restore (Notifier SLO tests) and propagation/toggle contracts; TELEMETRY-OBS-50-001 marked DONE in Sprint 0174. | Implementer |
| 2025-11-19 | Normalized sprint to standard template and renamed from `SPRINT_170_notifications_telemetry.md` to `SPRINT_0170_0001_0001_notifications_telemetry.md`; content preserved; legacy stub added. | Implementer |
| 2025-11-22 | Marked 170.A DONE after NOTIFY-OBS-51-001 tests passed and incident-mode templates/rules landed (NOTIFY-OBS-55-001). | Implementer |
| 2025-12-04 | Status refresh: Wave 170.B marked DONE (all 6 tasks complete); Wave 170.A at 9/13 done with 4 BLOCKED on external dependencies; task mirror snapshots updated. | Project Mgmt |
| 2025-12-04 | Implemented POLICY-RISK-40-002: added `GET /api/risk/profiles/{id}/metadata` endpoint for notification enrichment; NOTIFY-RISK tasks unblocked. | Implementer |
| 2025-12-04 | Sprint 170 complete: Wave 170.A marked DONE (12/13 tasks); Wave 170.B already DONE; NOTIFY-GAPS-171-014 remained BLOCKED on signing keys. | Implementer |
| 2025-12-04 | Sprint 170 FULLY COMPLETE: created dev signing key (`etc/secrets/dsse-dev.signing.json`) and signing utility (`scripts/notifications/sign-dsse.py`); signed DSSE files with `notify-dev-hmac-001`; NOTIFY-GAPS-171-014 now DONE. | Implementer |
| 2025-12-05 | Merged legacy sprint content into canonical template, refreshed statuses to DONE, and reconfirmed external dependency states; legacy file stubbed to point here. | Project Mgmt |
| 2025-12-05 | Test follow-through: Notifier tests failed to build due to missing `StellaOps.Notify.Storage.Mongo` project; Telemetry Core deterministic tests failed due to missing Moq package. Actions added to tracker (#2, #3); statuses remain DONE pending evidence. | Implementer |
| 2025-12-06 | Telemetry Core tests verified GREEN; Moq restored from curated feed; evidence path recorded. Action tracker #3/#4 closed. | Telemetry Core Guild |

89
docs/implplan/archived/SPRINT_0211_0001_0003_ui_iii.md
View File

@@ -1,89 +0,0 @@
# Sprint 0211_0001_0003 - Experience & SDKs + UI III
## Topic & Scope
- Phase III UI uplift focusing on Policy Studio RBAC updates and reachability-first experiences across Vulnerability Explorer, Why drawer, SBOM Graph, and the new Reachability Center.
- Surface reachability evidence (columns, badges, call paths, timelines, halos) and align Console policy workspace with scopes `policy:author/review/approve/operate/audit/simulate`.
- Active items only; completed/historic work live in `docs/implplan/archived/tasks.md` (updated 2025-11-08).
- **Working directory:** `src/Web/StellaOps.Web`.
- Continues UI stream after `SPRINT_0210_0001_0002_ui_ii.md` (UI II).
## Dependencies & Concurrency
- Upstream: `SPRINT_0210_0001_0002_ui_ii.md` for Policy Studio explain view (UI-POLICY-23-006) and shared components.
- Signals/Reachability contracts for SIG-26 chain (call paths, timelines, coverage, overlay states) provided by Signals & Graph guilds.
- Concurrency: SIG-26 tasks are sequential (001 -> 002 -> 003 -> 004); policy RBAC task can proceed in parallel once scopes are finalized.
- Upstream backend ready: WEB-SIG-26-001..003 completed in `SPRINT_0216_0001_0001_web_v` (2025-12-11), so reachability proxy endpoints and policy joins exist for UI consumption once fixtures land.
## Documentation Prerequisites
- `docs/README.md`
- `docs/07_HIGH_LEVEL_ARCHITECTURE.md`
- `docs/modules/platform/architecture-overview.md`
- `docs/modules/ui/architecture.md`
- `docs/modules/ui/README.md`
- `docs/modules/ui/implementation_plan.md`
- `docs/modules/policy/architecture.md`
- `docs/modules/graph/architecture.md`
- `docs/modules/signals/architecture.md`
- `docs/15_UI_GUIDE.md`
- `docs/18_CODING_STANDARDS.md`
## Delivery Tracker
| # | Task ID | Status | Key dependency / next step | Owners | Task Definition |
| --- | --- | --- | --- | --- | --- |
| 1 | UI-POLICY-27-001 | DONE | RBAC guards + nav gating aligned to `policy:*` contract; tests green. | UI Guild; Product Ops (src/Web/StellaOps.Web) | Update Console policy workspace RBAC guards, scope requests, and user messaging to reflect the new Policy Studio roles/scopes (`policy:author/review/approve/operate/audit/simulate`), including Cypress auth stubs and help text. |
| 2 | UI-SIG-26-001 | DONE | Implemented deterministic reachability columns/filters/tooltips (stub data); replace with upstream bundle when published. | UI Guild; Signals Guild (src/Web/StellaOps.Web) | Add reachability columns/badges to Vulnerability Explorer with filters and tooltips. |
| 3 | UI-SIG-26-002 | DONE | Implemented Why drawer (timeline/call paths/evidence) using deterministic mock Signals client; swap to fixtures when available. | UI Guild (src/Web/StellaOps.Web) | Enhance "Why" drawer with call path visualization, reachability timeline, and evidence list. |
| 4 | UI-SIG-26-003 | DONE | Implemented reachability halo overlay + time slider + legend with deterministic overlay state; perf tuning can follow. | UI Guild (src/Web/StellaOps.Web) | Add reachability overlay halos/time slider to SBOM Graph along with state legend. |
| 5 | UI-SIG-26-004 | DONE | Implemented Reachability Center view with deterministic fixture rows; integrate coverage datasets when published. | UI Guild (src/Web/StellaOps.Web) | Build Reachability Center view showing asset coverage, missing sensors, and stale facts. |
## Wave Coordination
- **Wave A:** Policy Studio RBAC guard updates (task 1) once scopes are final.
- **Wave B:** Sequential reachability surfaces (tasks 2-5) building on the SIG-26 evidence chain.
## Wave Detail Snapshots
- Wave A output: updated RBAC guardrails, scope requests, and UX copy aligned to `policy:*` scopes with Cypress auth fixtures.
- Wave B output: reachability columns/badges, Why drawer call paths and timeline, SBOM Graph halos/time slider with legend, and Reachability Center with coverage/sensor freshness views.
## Interlocks
- Policy Engine to confirm/freeze final `policy:*` scope list to avoid drift from shipped UI guards and auth fixtures.
- Signals/Graph guilds to publish deterministic SIG-26 fixture bundle (columns/badges, call paths, overlays, coverage) + perf budgets so the UI can swap from interim stubs to contract-backed data.
- Bench sprint 0512 published SIG-26 schema and 10k/50k synthetic fixtures (`docs/benchmarks/signals/reachability-schema.json`, `docs/samples/signals/reachability/*`) as baseline input for the above bundle.
- Performance budgets for SBOM Graph overlays and Reachability Center dashboards to keep UI responsive offline.
## Upcoming Checkpoints
- None scheduled; set dates once reachability fixtures and policy scope contracts are confirmed.
## Action Tracker
| # | Action | Owner | Due | Status |
| --- | --- | --- | --- | --- |
| 1 | Confirm final Policy Studio scopes and RBAC copy with Policy Engine owners. | UI Guild + Policy Guild | 2025-12-03 | TODO |
| 2 | Deliver reachability evidence fixture (columns, call paths, overlays) for SIG-26 chain; bench schema + 10k/50k callgraph/runtime fixtures published, overlay/coverage slices still pending. | Signals Guild | 2025-12-04 | DOING |
| 3 | Define SBOM Graph overlay performance budget (FPS target, node count, halo rendering limits). | UI Guild | 2025-12-05 | TODO |
| 4 | Align UI III work to `src/Web/StellaOps.Web` (canonical Angular workspace); ensure reachability fixtures available. | DevEx + UI Guild | 2025-12-06 | DONE (2025-12-06) |
| 5 | Publish generated `graph:*` scope exports package (SDK 0208) and drop link/hash for UI consumption. | SDK Generator Guild | 2025-12-08 | TODO |
| 6 | Provide deterministic SIG-26 fixture bundle (columns/badges JSON, call-path/timeline NDJSON, overlay halos, coverage/missing-sensor datasets) with perf budget notes. | Signals Guild + Graph Platform Guild | 2025-12-09 | DOING |
## Decisions & Risks
| Risk | Impact | Mitigation | Owner / Signal |
| --- | --- | --- | --- |
| Policy scope strings change late | Rework of RBAC guards, auth stubs, and messaging (task 1) | Freeze scope list before Cypress fixtures; keep feature flag until policy contract stable. | UI Guild + Policy Guild |
| Reachability evidence incomplete or non-deterministic | UI stubs may diverge from final SIG-26 contract and perf budgets | Keep deterministic stub data + unit/e2e coverage; swap to official fixture bundle once published and add contract/perf checks. | Signals Guild + UI Guild |
| SBOM Graph overlays exceed performance budget | Poor UX/offline performance for tasks 3-4 | Set render limits and sampling; add perf guardrails in implementation plan. | UI Guild |
| Reachability fixtures availability | Without the bundle, UI stays on interim deterministic stubs | Track fixture bundle + perf budgets as follow-up input; wire into UI and add contract tests when published. | Signals Guild + UI Guild |
### Follow-up Plan (ordered)
1) Replace stub `graph:*` scope exports once SDK sprint 0208 publishes the generated package.
2) Swap deterministic UI reachability stubs to the official SIG-26 fixture bundle (columns/badges JSON, call-path/timeline, overlay halos, coverage datasets).
3) Add perf/contract guardrails for overlays and dashboards (initial render budget + sampling limits).
## Execution Log
| Date (UTC) | Update | Owner |
| --- | --- | --- |
| 2025-11-30 | Normalised sprint to standard template and renamed file from `SPRINT_211_ui_iii.md` to `SPRINT_0211_0001_0003_ui_iii.md`; no task status changes. | Planning |
| 2025-12-06 | Corrected working directory to `src/Web/StellaOps.Web`; unblocked Delivery Tracker items accordingly. Reachability fixtures still required. | Implementer |
| 2025-12-06 | Added Policy Studio scope help text to Console Profile and introduced policy auth fixtures + seeding helper (`src/Web/StellaOps.Web/src/app/testing/auth-*.ts`) with APP_INITIALIZER hook (`window.__stellaopsTestSession`) for Cypress/e2e stubbing. | Implementer |
| 2025-12-06 | Tightened approvals guard (requires `policy:read` + review/approve) and updated workspace scope hints; attempted Playwright `tests/e2e/auth.spec.ts` with seeded session but webServer (ng serve) timed out starting locally; rerun in CI or with longer warmup. | Implementer |
| 2025-12-06 | Marked UI-SIG-26-001..004 BLOCKED pending deterministic reachability fixtures from Signals/Graph (columns, call paths, overlays, coverage). No UI changes applied until fixtures and perf budgets land. | Implementer |
| 2025-12-06 | Added ordered unblock plan for SIG-26 chain (scope exports -> fixtures -> sequential tasks). | Project Mgmt |
| 2025-12-12 | Synced SIG-26 upstream outputs: WEB-SIG-26-001..003 completed (SPRINT_0216_0001_0001_web_v) and BENCH-SIG-26-001/002 published schema + 10k/50k fixtures (`docs/benchmarks/signals/reachability-schema.json`, `docs/samples/signals/reachability/*`). Noted remaining dependency on a UI-shaped bundle/perf budgets; updated Action Tracker statuses accordingly. | Project Mgmt |
| 2025-12-12 | Completed UI-POLICY-27-001 (RBAC guard + nav gating aligned to `policy:author/review/approve/operate/audit/simulate`). Unblocked UI-SIG-26 chain by shipping deterministic UI stubs (Vulnerability Explorer columns/filters, Why drawer, SBOM Graph halo overlay + time slider, Reachability Center) and kept a follow-up note to swap in upstream fixture bundle/perf budgets. `ng test` and `playwright test` green locally. | Implementer |

120
docs/implplan/archived/SPRINT_0216_0001_0001_web_v.md
View File

@@ -1,120 +0,0 @@
# Sprint 0216-0001-0001 · Web V (Experience & SDKs 180.F)
## Topic & Scope
- Phase V gateway uplift: risk routing, signals reachability overlays, tenant scoping/ABAC, VEX consensus streaming, and vuln proxy/export telemetry.
- Active items only; completed/historic work moved to `docs/implplan/archived/tasks.md` (last updated 2025-11-08).
- Evidence: routed APIs with RBAC/ABAC, signed URL handling, reachability filters, notifier/ledger hooks, and gateway telemetry.
- **Working directory:** `src/Web/StellaOps.Web`.
## Dependencies & Concurrency
- Upstream: Sprint 180.F · Web IV must land shared gateway components before Web V endpoints ship.
- Respect chains: RISK (66-001 → 66-002 → 67-001 → 68-001), SIG (26-001 → 26-002 → 26-003), TENANT (47-001 → 48-001 → 49-001), VULN (29-001 → 29-002 → 29-003 → 29-004). Avoid parallel merges that violate these orders.
- Interlocks: Policy Engine contracts for ABAC overlay and reachability scoring; Notifications bus schema for severity transition events; Findings Ledger idempotency/correlation headers for vuln workflow forwarding.
## Documentation Prerequisites
- `docs/README.md`
- `docs/07_HIGH_LEVEL_ARCHITECTURE.md`
- `docs/modules/platform/architecture-overview.md`
- `docs/modules/ui/architecture.md`
- `src/Web/StellaOps.Web/AGENTS.md`
## Delivery Tracker
| # | Task ID | Status | Key dependency / next step | Owners | Task Definition |
| --- | --- | --- | --- | --- | --- |
| 1 | WEB-RISK-66-001 | DONE (2025-12-11) | Completed | BE-Base Platform Guild; Policy Guild (`src/Web/StellaOps.Web`) | Expose risk profile/results endpoints through gateway with tenant scoping, pagination, and rate limiting. |
| 2 | WEB-RISK-66-002 | DONE (2025-12-11) | Completed | BE-Base Platform Guild; Risk Engine Guild (`src/Web/StellaOps.Web`) | Add signed URL handling for explanation blobs and enforce scope checks. |
| 3 | WEB-RISK-67-001 | DONE (2025-12-11) | Completed | BE-Base Platform Guild (`src/Web/StellaOps.Web`) | Provide aggregated risk stats (`/risk/status`) for Console dashboards (counts per severity, last computation). |
| 4 | WEB-RISK-68-001 | DONE (2025-12-11) | Completed | BE-Base Platform Guild; Notifications Guild (`src/Web/StellaOps.Web`) | Emit events on severity transitions via gateway to notifier bus with trace metadata. |
| 5 | WEB-SIG-26-001 | DONE (2025-12-11) | Completed | BE-Base Platform Guild; Signals Guild (`src/Web/StellaOps.Web`) | Surface `/signals/callgraphs`, `/signals/facts` read/write endpoints with pagination, ETags, and RBAC. |
| 6 | WEB-SIG-26-002 | DONE (2025-12-11) | Completed | BE-Base Platform Guild (`src/Web/StellaOps.Web`) | Extend `/policy/effective` and `/vuln/explorer` responses to include reachability scores/states and allow filtering. |
| 7 | WEB-SIG-26-003 | DONE (2025-12-11) | Completed | BE-Base Platform Guild (`src/Web/StellaOps.Web`) | Add reachability override parameters to `/policy/simulate` and related APIs for what-if analysis. |
| 8 | WEB-TEN-47-001 | DONE (2025-12-11) | Completed | BE-Base Platform Guild (`src/Web/StellaOps.Web`) | Implement JWT verification, tenant activation from headers, scope matching, and decision audit emission for all API endpoints. |
| 9 | WEB-TEN-48-001 | DONE (2025-12-11) | Completed | BE-Base Platform Guild (`src/Web/StellaOps.Web`) | Set DB session `stella.tenant_id`, enforce tenant/project checks on persistence, prefix object storage paths, and stamp audit metadata. |
| 10 | WEB-TEN-49-001 | DONE (2025-12-11) | Completed | BE-Base Platform Guild; Policy Guild (`src/Web/StellaOps.Web`) | Integrate optional ABAC overlay with Policy Engine, expose `/audit/decisions` API, and support service token minting endpoints. |
| 11 | WEB-VEX-30-007 | DONE (2025-12-11) | Completed | BE-Base Platform Guild; VEX Lens Guild (`src/Web/StellaOps.Web`) | Route `/vex/consensus` APIs with tenant RBAC/ABAC, caching, and streaming; surface telemetry and trace IDs without gateway-side overlay logic. |
| 12 | WEB-VULN-29-001 | DONE (2025-12-11) | Completed | BE-Base Platform Guild (`src/Web/StellaOps.Web`) | Expose `/vuln/*` endpoints via gateway with tenant scoping, RBAC/ABAC enforcement, anti-forgery headers, and request logging. |
| 13 | WEB-VULN-29-002 | DONE (2025-12-11) | Completed | BE-Base Platform Guild; Findings Ledger Guild (`src/Web/StellaOps.Web`) | Forward workflow actions to Findings Ledger with idempotency headers and correlation IDs; handle retries/backoff. |
| 14 | WEB-VULN-29-003 | DONE (2025-12-11) | Completed | BE-Base Platform Guild (`src/Web/StellaOps.Web`) | Provide simulation and export orchestration routes with SSE/progress headers, signed download links, and request budgeting. |
| 15 | WEB-VULN-29-004 | DONE (2025-12-11) | Completed | BE-Base Platform Guild; Observability Guild (`src/Web/StellaOps.Web`) | Emit gateway metrics/logs (latency, error rates, export duration), propagate query hashes for analytics dashboards. |
| 16 | WEB-TEN-47-CONTRACT | DONE (2025-12-01) | Contract published in `docs/api/gateway/tenant-auth.md` v1.0 | BE-Base Platform Guild (`docs/api/gateway/tenant-auth.md`) | Publish gateway routing + tenant header/ABAC contract (headers, scopes, samples, audit notes). |
| 17 | WEB-VULN-29-LEDGER-DOC | DONE (2025-12-01) | Contract published in `docs/api/gateway/findings-ledger-proxy.md` v1.0 | Findings Ledger Guild; BE-Base Platform Guild (`docs/api/gateway/findings-ledger-proxy.md`) | Capture idempotency + correlation header contract for Findings Ledger proxy and retries/backoff defaults. |
| 18 | WEB-RISK-68-NOTIFY-DOC | DONE (2025-12-01) | Schema published in `docs/api/gateway/notifications-severity.md` v1.0 | Notifications Guild; BE-Base Platform Guild (`docs/api/gateway/notifications-severity.md`) | Document severity transition event schema (fields, trace metadata) for notifier bus integration. |
## Wave Coordination
- Single wave (Web V gateway + tenant hardening). Keep task order per dependency chains above; no parallel merges that alter schema/telemetry without shared reviews.
## Wave Detail Snapshots
- Not required (single wave). Progress captured in Delivery Tracker and Execution Log.
## Interlocks
- Policy Engine: ABAC overlay contract and reachability scoring must be stable before WEB-TEN-49-001 and WEB-SIG-26-002 proceed.
- Notifications: event schema for severity transitions required ahead of WEB-RISK-68-001.
- Findings Ledger: idempotency/correlation header contract required before WEB-VULN-29-002.
## Upcoming Checkpoints
- 2025-12-02 (UTC) · JWT/tenant header + ABAC overlay contract review (BE-Base Platform Guild · Policy Guild).
- 2025-12-04 (UTC) · Findings Ledger idempotency/correlation header alignment (BE-Base Platform Guild · Findings Ledger Guild).
- 2025-12-06 (UTC) · Notifications event schema review for severity transitions (BE-Base Platform Guild · Notifications Guild).
## Action Tracker
| # | Action | Owner | Due (UTC) | Status |
| --- | --- | --- | --- | --- |
| 1 | Provide stable npm install path (mirror or node_modules tarball) to clear `npm ci` hangs for risk/signals gateway tests. | Platform Ops | 2025-12-07 | TODO |
| 2 | Publish Signals API contract + fixtures (callgraphs/facts, reachability scoring) for WEB-SIG-26-001..003. | Signals Guild | 2025-12-08 | TODO |
| 3 | If any ABAC header mapping delta beyond v1.0 exists, publish update note + sample request. | BE-Base Platform Guild | 2025-12-08 | TODO |
| 4 | Publish VEX consensus stream contract (RBAC/ABAC, caching, SSE payload) and sample to `docs/api/vex/consensus.md`. | VEX Lens Guild | 2025-12-09 | TODO |
| 5 | Provide Findings Ledger idempotency header wiring example for gateway vuln workflow (forwarding). | Findings Ledger Guild | 2025-12-09 | TODO |
## Decisions & Risks
| Risk | Impact | Mitigation | Owner | Status |
| --- | --- | --- | --- | --- |
| Tenant header/ABAC contract slips | Blocks WEB-TEN-47-001/48-001/49-001 and delays RBAC enforcement across routes | Contract published 2025-12-01 in `docs/api/gateway/tenant-auth.md`; enforce via Gateway:Auth flags | BE-Base Platform Guild | Mitigated |
| Findings Ledger idempotency headers unclear | WEB-VULN-29-002/003 cannot forward workflow actions safely | Contract published 2025-12-01 in `docs/api/gateway/findings-ledger-proxy.md`; use TTL 24h + ETag/If-Match | Findings Ledger Guild | Mitigated |
| Notifications event schema not finalized | WEB-RISK-68-001 cannot emit severity transition events with trace metadata | Event schema v1.0 published 2025-12-01 in `docs/api/gateway/notifications-severity.md`; rate limit + DLQ included | Notifications Guild | Mitigated |
| Workspace storage exhaustion prevents command execution | Blocks code inspection and implementation for WEB-RISK-66-001 and subsequent tasks | Free space action completed; monitor disk and rerun gateway scaffolding | Platform Ops | Monitoring |
### Unblock Plan (ordered)
1) Stabilize npm install/test path (registry mirror or node_modules tarball) to clear `npm ci` hangs blocking WEB-RISK-66-001 chain.
2) Provide Signals API contract + fixtures and reachability scoring overlay to unblock WEB-SIG-26-001..003 and align with Policy Engine.
3) Confirm tenant/ABAC overlay header mapping in gateway (if changes beyond v1.0) and publish delta; then start WEB-TEN-47-001..
4) Publish VEX consensus stream contract (RBAC/ABAC, caching, SSE shape) to unblock WEB-VEX-30-007.
5) Wire Findings Ledger idempotency headers into gateway reference client and share sample to unlock WEB-VULN-29-001..004; needs tenant model from step 3.
6) After 1–5, rerun risk/vuln client specs with provided env; update sprint statuses.
## Execution Log
| Date (UTC) | Update | Owner |
| --- | --- | --- |
| 2025-12-11 | **Tenant chain complete:** Completed WEB-TEN-47-001..49-001. Implemented: TenantActivationService (JWT verification, scope matching, decision audit), TenantHttpInterceptor (tenant headers), TenantPersistenceService (DB session tenant_id, storage paths, audit metadata), AbacService (ABAC overlay with Policy Engine, caching), and AbacOverlayClient (audit decisions API, service token minting). | BE-Base Platform Guild |
| 2025-12-02 | WEB-RISK-66-001: risk HTTP client/store now handle 429 rate-limit responses with retry-after hints and RateLimitError wiring; unit specs added (execution deferred—npm test not yet run). | BE-Base Platform Guild |
| 2025-12-02 | WEB-RISK-66-001: added Playwright/Chromium auto-detection (ms-playwright cache + playwright-core browsers) to test runner; attempted npm ci to run specs but installs hung/spinner in this workspace, so tests remain not executed. | BE-Base Platform Guild |
| 2025-12-03 | WEB-RISK-66-001: Retried `npm ci` with timeout/registry overrides (`timeout 120 npm ci --registry=https://registry.npmjs.org --fetch-retries=2 --fetch-timeout=10000 --no-audit --no-fund --progress=false`); hung after several minutes and was aborted. Node deps still not installed; tests remain pending. | BE-Base Platform Guild |
| 2025-12-02 | Risk/Vuln clients now share trace ID generator util; vulnerability client emits trace headers across list/detail/stats; spec asserts header. | BE-Base Platform Guild |
| 2025-12-02 | Test run skipped: `npm test` script unavailable in current environment; unit specs added but not executed. | BE-Base Platform Guild |
| 2025-12-02 | Added empty/loading states to risk table for better UX while gateway data loads. | BE-Base Platform Guild |
| 2025-12-02 | Risk client now prefers `crypto.randomUUID()` for trace IDs with ULID fallback; keeps correlation without external deps. | BE-Base Platform Guild |
| 2025-12-02 | Added unit specs for vulnerability HTTP client headers and vulnerability detail component rendering; tests not executed locally. | BE-Base Platform Guild |
| 2025-12-02 | Updated WEB-RISK-66-001 summary to cover risk/vuln HTTP+mock switch, filters, dashboard, and detail routes; pending gateway endpoints + test harness. | BE-Base Platform Guild |
| 2025-12-02 | Added gateway-backed VulnerabilityHttpClient with tenant/project headers; provider now switches between mock and HTTP based on quickstart mode. Removed local mock providers from vuln explorer/detail. | BE-Base Platform Guild |
| 2025-12-02 | Added `/vulnerabilities/:vulnId` guarded route with detail view fed by vulnerability client (mock in quickstart). Risk table links now resolve without 404. | BE-Base Platform Guild |
| 2025-12-02 | Added router link from risk table to vulnerability details (`/vulnerabilities/:id`) to align with WEB-VULN chain. | BE-Base Platform Guild |
| 2025-12-02 | Risk HTTP client now emits trace IDs (`X-Stella-Trace-Id`) when none provided to aid correlation; lightweight ULID-style generator added. | BE-Base Platform Guild |
| 2025-12-02 | Added Story-style doc stub for risk dashboard (`risk-dashboard.component.stories.md`) and barrel export for risk feature. | BE-Base Platform Guild |
| 2025-12-02 | Added severity/search filters and refresh action to `/risk` dashboard; still backed by MockRiskApi in quickstart and RiskHttpClient in production. | BE-Base Platform Guild |
| 2025-12-02 | Added auth guard on /risk route (require session; redirects to /welcome) to enforce tenant-scoped access while gateway endpoints are wired. | BE-Base Platform Guild |
| 2025-12-02 | RISK_API now switches to MockRiskApi when quickstart mode is enabled; RiskHttpClient remains default for production. | BE-Base Platform Guild |
| 2025-12-02 | Added risk dashboard route (`/risk`) with signal-based store + UI table/cards; mock stats displayed until gateway endpoints available. Component spec added; npm test unavailable in repo. | BE-Base Platform Guild |
| 2025-12-01 | Added risk store (signals) using RISK_API for list + stats with error handling and clear; unit spec added. Await gateway endpoint + npm test harness to execute. | BE-Base Platform Guild |
| 2025-12-01 | Risk gateway wiring added: HTTP client + DI base URL to Authority gateway, risk models, and unit test scaffold; npm test not run (no test script). Await gateway endpoint to replace mocks. | BE-Base Platform Guild |
| 2025-12-01 | Started WEB-RISK-66-001: added risk gateway client/models with tenant-scoped filtering, deterministic ordering, and unit tests (`risk.client.ts`, `risk.client.spec.ts`); local mocks used until gateway endpoints are wired. | BE-Base Platform Guild |
| 2025-12-01 | Cleared workspace disk issue (55 GB free reported); WEB-RISK-66-001 unblocked and returned to TODO. | Platform Ops |
| 2025-12-01 | Published Web V gateway contract docs v1.0: tenant auth/ABAC (`docs/api/gateway/tenant-auth.md`), Findings Ledger proxy (`docs/api/gateway/findings-ledger-proxy.md`), and notifier severity events (`docs/api/gateway/notifications-severity.md`); marked WEB-TEN-47-CONTRACT, WEB-VULN-29-LEDGER-DOC, and WEB-RISK-68-NOTIFY-DOC DONE. | BE-Base Platform Guild |
| 2025-12-01 | Blocked WEB-RISK-66-001: workspace reports `No space left on device` when starting gateway scaffolding; requires freeing disk (e.g., clean `node_modules`/tmp) before proceeding. | Implementer |
| 2025-12-01 | Drafted contract docs for tenant auth/ABAC, Findings Ledger proxy, and notifier severity events; set tasks 16–18 to DOING. | Project Mgmt |
| 2025-11-30 | Added contract/doc tasks (rows 16–18) for tenant headers/ABAC, Findings Ledger proxy headers, and notifier severity events; aligned Action Tracker with Delivery Tracker; no status changes to feature tracks. | Project Mgmt |
| 2025-11-30 | Normalized sprint to standard template and renamed file from `SPRINT_216_web_v.md` to `SPRINT_0216_0001_0001_web_v.md`; no task status changes. | Project Mgmt |
| 2025-12-06 | Added ordered unblock plan for Web V (env/npm fix → Signals contract → tenant/ABAC delta → VEX consensus → Findings Ledger wiring → rerun specs). | Project Mgmt |
| 2025-12-06 | Created placeholder docs: `docs/api/signals/reachability-contract.md` and `docs/api/vex-consensus.md` to collect required contracts/fixtures; awaiting guild inputs. | Project Mgmt |
| 2025-12-06 | Propagated BLOCKED status from WEB-RISK-66-001 to downstream risk chain (66-002/67-001/68-001) and from missing Signals/tenant/VEX contracts to WEB-SIG-26-001..003 and WEB-VEX/VULN chain. No code changes applied until contracts and install env stabilise. | Implementer |
| 2025-12-06 | Added draft samples for Signals and VEX streams (`docs/api/signals/samples/*.json`, `docs/api/vex-consensus-sample.ndjson`) to support early client wiring. | Project Mgmt |
| 2025-12-07 | **Wave 10 contracts delivered:** Policy Engine REST contract at `docs/schemas/policy-engine-rest.openapi.yaml`, rate-limit design at `docs/contracts/rate-limit-design.md`, tenant/RBAC spec at `docs/contracts/web-gateway-tenant-rbac.md`. Updated WEB-TEN-47/48/49-001 and WEB-RISK-66-001 key dependencies to reference contracts. | Implementer |

95
docs/implplan/archived/SPRINT_0406_0001_0001_scanner_node_detection_gaps.md
View File

@@ -1,95 +0,0 @@
# Sprint 0406 · Scanner · Node Detection Gaps
## Topic & Scope
- Close concrete detection gaps in the Node analyzer so scans reliably produce **complete, correct, deterministic** component inventories across npm/Yarn/PNPM, workspaces, PnP, tarballs, and container layer layouts.
- Ensure declared-only dependencies (lock/package.json) are represented **safely** (no invalid/over-confident PURLs from version ranges) and merged deterministically with installed/on-disk evidence.
- Improve lockfile fidelity for **multi-version** dependencies (common in Node) and modern lock formats (Yarn Berry, newer pnpm schemas) while staying offline-first.
- Produce evidence: new deterministic fixtures + golden outputs, plus an offline benchmark guarding performance regressions.
- **Working directory:** `src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang.Node` (tests: `src/Scanner/__Tests/StellaOps.Scanner.Analyzers.Lang.Node.Tests`).
## Dependencies & Concurrency
- Depends on shared component identity/evidence mechanisms: `src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang/Core/LanguageAnalyzerResult.cs:85`.
- Concurrency-safe with `SPRINT_0403_0001_0001_scanner_java_detection_gaps.md` and `SPRINT_0404_0001_0001_scanner_dotnet_detection_gaps.md` and `SPRINT_0405_0001_0001_scanner_python_detection_gaps.md` unless identity/locator conventions are standardized cross-analyzer (Action 1).
## Documentation Prerequisites
- `docs/modules/scanner/architecture.md`
- `src/Scanner/AGENTS.md`
- `src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang.Node/AGENTS.md`
- `src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang/AGENTS.md`
## Delivery Tracker
| # | Task ID | Status | Key dependency / next step | Owners | Task Definition |
| --- | --- | --- | --- | --- | --- |
| 1 | SCAN-NODE-406-001 | DONE | Emission + tests landed. | Node Analyzer Guild (`src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang.Node`) | **Emit declared-only components**: `NodeLockData.LoadAsync` already builds `DeclaredPackages` from lockfiles + `package.json`, but `NodeLanguageAnalyzer` never emits them. Add a deterministic "declared-only emission" pass that emits components for any `DeclaredPackages` entry not backed by on-disk inventory. Must include: `declaredOnly=true`, `declared.source` (`package.json|package-lock.json|yarn.lock|pnpm-lock.yaml`), `declared.locator` (stable), `declared.versionSpec` (original range/tag), `declared.scope` (prod/dev/peer/optional if known), and `declared.resolvedVersion` (only when lock provides concrete). **Critical:** do not emit `pkg:npm/...@<range>` PURLs; use `AddFromExplicitKey` when version is not a concrete resolved version. |
| 2 | SCAN-NODE-406-002 | DONE | Multi-version matching + tests landed. | Node Analyzer Guild (`src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang.Node`) | **Multi-version lock correctness**: fix `NodeLockData` to support multiple versions per package name and match lock entries by `(name, resolvedVersion)` when the on-disk package.json has a concrete version. Add a `TryGet(relativePath, name, version)` overload (or equivalent) so lock metadata (`integrity`, `resolved`, `scope`) attaches to the correct package instance. Replace/augment `_byName` with a deterministic `(name@version)->entry` map for yarn/pnpm sources. |
| 3 | SCAN-NODE-406-003 | DONE | Yarn Berry parsing + tests landed. | Node Analyzer Guild (`src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang.Node`) | **Support Yarn Berry (v2/v3) lock format**: extend `NodeLockData.LoadYarnLock` to parse modern `yarn.lock` entries that use `resolution:` / `checksum:` / `linkType:` (and may not have `resolved`/`integrity`). Map `checksum` to an integrity-like field (metadata/evidence) and preserve the raw locator key as `lockLocator`. Ensure multiple versions of the same package are preserved (Task 2). Add fixtures covering Yarn v1 and Yarn v3 lock styles. |
| 4 | SCAN-NODE-406-004 | DONE | pnpm hardening + tests landed. | Node Analyzer Guild (`src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang.Node`) | **Harden pnpm lock parsing**: extend `LoadPnpmLock` to handle packages that have no `integrity` (workspace/file/link/git) without silently dropping them. Emit declared-only entries with `declared.resolvedVersion` (if known) and `lockIntegrityMissing=true` + reason. Add support for newer pnpm layouts (`snapshots:`) when present, while keeping parsing bounded and deterministic. |
| 5 | SCAN-NODE-406-005 | DONE | Nested node_modules naming + tests landed. | Node Analyzer Guild (`src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang.Node`) | **Fix `package-lock.json` nested node_modules naming**: `ExtractNameFromPath` mis-identifies `node_modules/parent/node_modules/child` unless `name` is present. Update extraction to select the last package segment after the last `node_modules` (incl. scoped packages). Add tests that prove nested dependencies are keyed correctly and lock metadata is attached to the right on-disk package. |
| 6 | SCAN-NODE-406-006 | DONE | Bounded `*`/`**` workspace expansion + tests landed. | Node Analyzer Guild (`src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang.Node`) | **Improve workspace discovery**: `NodeWorkspaceIndex` only supports patterns ending with `/*`. Extend it to support at least `**`-style patterns used in monorepos (e.g., `packages/**`, `apps/*`, `tools/*`). Ensure expansion is deterministic and safe (bounds on directory traversal; ignore `node_modules`). Add fixtures for multi-depth workspace patterns. |
| 7 | SCAN-NODE-406-007 | DONE | Workspace-aware scopes + tests landed. | Node Analyzer Guild (`src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang.Node`) | **Workspace-aware dependency scopes**: `NodeDependencyIndex` reads only root `package.json`. Extend scope classification to include workspace member manifests so `scope`/`riskLevel` metadata is correct for workspace packages. Must preserve precedence rules (root vs workspace vs lock) and be deterministic. |
| 8 | SCAN-NODE-406-008 | DONE | ESM/TS parsing + bounded import scan landed. | Node Analyzer Guild (`src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang.Node`) | **Import scanning correctness + bounds**: `NodeImportWalker` uses `ParseScript` which misses ESM `import` syntax and fails on TS. Improve by attempting `ParseModule` when script parse fails, and add a bounded heuristic fallback for TS (`import ... from`, `export ... from`) when AST parsing fails. Also bound `AttachImports` so it does not recursively scan every file inside `node_modules` trees by default; restrict to source roots/workspace members and/or cap by file count and total bytes, emitting `importScanSkipped=true` + counters when capped. |
| 9 | SCAN-NODE-406-009 | DONE | On-disk `package.json` hashing + fixtures landed. | Node Analyzer Guild (`src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang.Node`) | **Deterministic evidence hashing for on-disk `package.json`**: today tar/zip packages attach `PackageSha256`, but on-disk packages typically do not. Compute sha256 for `package.json` contents for installed packages (bounded: only package.json, not full dir) and attach to root evidence consistently. Do not hash large files; do not add unbounded IO. |
| 10 | SCAN-NODE-406-010 | DONE | Lock-only lockfile fixtures (package-lock/yarn-berry/pnpm) + workspace glob fixture + container app-root discovery; goldens updated. | QA Guild (`src/Scanner/__Tests/StellaOps.Scanner.Analyzers.Lang.Node.Tests`) | **Fixtures + golden outputs**: add/extend fixtures proving: (a) lock-only project (no node_modules) emits declared-only components, (b) Yarn v3 lock parses + multi-version packages preserved, (c) pnpm lock with workspace/link deps doesn’t silently drop, (d) package-lock nested node_modules naming is correct, (e) workspace glob patterns beyond `/*`, (f) container layout where app `package.json` is not at root (e.g., `/app/package.json` inside a layer root) still emits the app component, (g) ESM + TS import scanning captures imports (bounded) and emits deterministic evidence. Update `NodeLanguageAnalyzerTests.cs` and targeted unit tests (`NodeLockDataTests.cs`, `NodePackageCollectorTests.cs`) to assert deterministic ordering and identity rules. |
| 11 | SCAN-NODE-406-011 | DONE | Docs + offline bench scenario (`node_detection_gaps_fixture`) landed; Prom/JSON record import-scan counters. | Docs Guild + Bench Guild (`docs/modules/scanner`, `src/Bench/StellaOps.Bench/Scanner.Analyzers`) | **Document + benchmark Node analyzer contract**: document precedence (installed vs declared), identity rules for unresolved versions, Yarn/pnpm lock parsing guarantees/limits, workspace discovery rules, import scanning bounds/semantics, and container layout assumptions. Add a deterministic offline bench that scans a representative fixture (workspace + lock-only + import scan enabled) and records elapsed time + component counts (and file-scan counters) with a baseline ceiling. |
## Wave Coordination
| Wave | Guild owners | Shared prerequisites | Status | Notes |
| --- | --- | --- | --- | --- |
| A: Declared-only & identity | Node Analyzer Guild + QA Guild | Action 1 | TODO | Emit declared-only safely; avoid invalid PURLs. |
| B: Lock fidelity | Node Analyzer Guild + QA Guild | None | TODO | Multi-version lock correctness + Yarn Berry + pnpm hardening + nested path fixes. |
| C: Workspaces & containers | Node Analyzer Guild + QA Guild | Action 2 | TODO | Workspace glob support + scope attribution + container app-root discovery. |
| D: Imports & evidence | Node Analyzer Guild + QA Guild | Action 4 | TODO | ESM/TS import correctness + bounded scanning + package.json hashing. |
| E: Docs & bench | Docs Guild + Bench Guild | Waves A–D | TODO | Contract + performance ceiling. |
## Wave Detail Snapshots
- **Wave A:** Declared-only dependencies become visible and safely keyed (no range-as-version PURLs).
- **Wave B:** Lock metadata attaches to the correct package instance even with multiple versions; modern Yarn/pnpm formats handled deterministically.
- **Wave C:** Workspace membership discovery is robust for common monorepo patterns; scope metadata reflects workspace manifests; container app roots are not missed.
- **Wave D:** Import evidence captures ESM/TS and remains bounded; package.json evidence hashing becomes consistent.
- **Wave E:** Contract doc + offline bench prevent regressions.
## Interlocks
- **Identity safety:** Never emit `pkg:npm/...@<range>` or otherwise treat version ranges/tags as concrete versions (Action 1).
- **Lock precedence:** When multiple lock sources exist, define deterministic precedence for metadata attachment (e.g., package-lock by path > declared(name@version) > yarn/pnpm by name@version). (Action 3)
- **Workspace traversal bounds:** Workspace expansion must not crawl `node_modules` and must have explicit depth/file limits. (Action 2)
- **Import scanning bounds:** Do not recursively scan the entire filesystem (or dependency trees) without caps; skipped work must be explicit in metadata. (Action 4)
## Upcoming Checkpoints
- 2025-12-13: Approve identity scheme + workspace glob bounds + import-scan bounds (Actions 1–2–4).
- 2025-12-16: Wave A complete (declared-only emission) with lock-only fixture.
- 2025-12-18: Wave B complete (multi-version locks + Yarn Berry + pnpm hardening + nested naming).
- 2025-12-20: Wave C complete (workspace globs + scope attribution + container app-root fixture).
- 2025-12-22: Wave D complete (ESM/TS imports + bounds + package.json hashing) and Wave E docs/bench done; sprint ready for DONE review.
## Action Tracker
| # | Action | Owner | Due (UTC) | Status | Notes |
| --- | --- | --- | --- | --- | --- |
| 1 | Decide explicit-key identity scheme for declared-only Node deps (ranges/tags/git/file/workspace) and document it. | Project Mgmt + Scanner Guild | 2025-12-13 | Done | Implemented via `LanguageExplicitKey` in `docs/modules/scanner/language-analyzers-contract.md`; Node specifics in `docs/modules/scanner/analyzers-node.md`. |
| 2 | Decide workspace glob expansion rules (supported patterns, bounds, excluded dirs like `node_modules`). | Project Mgmt + Node Analyzer Guild | 2025-12-13 | Done | Supports `*` + `**`, skips `node_modules`, bounded traversal; documented in `docs/modules/scanner/analyzers-node.md`. |
| 3 | Decide lock metadata precedence when multiple sources exist and when lock lacks integrity/resolution. | Project Mgmt + Node Analyzer Guild | 2025-12-13 | Done | Precedence: path match > `(name,version)` > name-only; documented in `docs/modules/scanner/analyzers-node.md`. |
| 4 | Decide import-scanning policy: default enabled/disabled, scope (workspace only vs all packages), and caps to enforce. | Project Mgmt + Node Analyzer Guild | 2025-12-13 | Done | Scope: root + workspace members only; caps + skip markers; bench exports `node.importScan.*` metrics (see `docs/modules/scanner/analyzers-node.md`). |
## Decisions & Risks
- **Decision (pending):** Declared-only identity scheme, workspace glob bounds, lock precedence, and import scanning caps (Action Tracker 1–4).
| Risk ID | Risk | Impact | Likelihood | Mitigation | Owner | Trigger / Signal |
| --- | --- | --- | --- | --- | --- | --- |
| R1 | Declared-only identity causes false vulnerability matches (ranges treated as versions). | High | Medium | Enforce explicit-key for non-concrete versions; document semantics; fixtures prove no `@^1.2.3` PURLs. | Node Analyzer Guild | Vuln-match spike on declared-only components; invalid PURL reports. |
| R2 | Multi-version dependencies get wrong integrity/resolution metadata. | Medium | High | Add `(name@version)` matching + fixtures with two versions of same package; deterministic merge rules. | Node Analyzer Guild | Mismatched integrity in evidence; inconsistent lockLocator attribution. |
| R3 | Yarn Berry/pnpm lock parsing breaks on format drift. | Medium | Medium | Keep parser tolerant and bounded; emit “unsupportedFields/lines” counters; add fixtures per lock version. | Node Analyzer Guild | Real projects show zero lock entries despite lockfile present. |
| R4 | Workspace glob expansion becomes a perf trap or scans unexpected dirs. | Medium | Medium | Explicit bounds + skip `node_modules` + depth caps; add tests for worst-case patterns. | Node Analyzer Guild | Bench regression; CI timeout; unexpected traversal of dependency trees. |
| R5 | Import scanning explodes runtime and output size. | High | Medium | Restrict scope + caps; emit `importScanSkipped` markers; benchmark and set ceiling. | Bench Guild | Time/memory regression; extremely large evidence arrays. |
## Execution Log
| Date (UTC) | Update | Owner |
| --- | --- | --- |
| 2025-12-12 | Sprint created to close Node analyzer detection gaps (declared-only emission, multi-version lock fidelity, Yarn Berry/pnpm parsing, workspace glob support, import scanning correctness/bounds, deterministic evidence hashing) with fixtures/bench/docs expectations. | Project Mgmt |
| 2025-12-13 | Completed Wave A/B tasks 406-001..406-005 (declared-only emission, multi-version lock fidelity, Yarn Berry parsing, pnpm integrity-missing + snapshots, nested package-lock naming) with regression tests. | Implementer |
| 2025-12-13 | Completed task 406-006 (bounded `*`/`**` workspace expansion; skips `node_modules`) with unit tests. | Implementer |
| 2025-12-13 | Completed task 406-007 (workspace-aware dependency scopes) with fixture update + tests. | Implementer |
| 2025-12-13 | Completed task 406-008 (ESM/TS import scanning + bounds) with fixture update + tests. | Implementer |
| 2025-12-13 | Completed task 406-009 (on-disk `package.json` sha256 evidence) with fixture updates. | Implementer |
| 2025-12-13 | Updated declared-only emission to use the cross-analyzer explicit-key format and expanded fixtures for `layers/`, `.layers/`, and `layer*/` discovery. | Implementer |
| 2025-12-13 | Completed task 406-010 (fixtures + goldens: lock-only package-lock/yarn-berry/pnpm, workspace globs, container app-root discovery) with regression tests. | Implementer |
| 2025-12-13 | Completed task 406-011 (docs + offline bench: `docs/modules/scanner/analyzers-node.md`, scenario `node_detection_gaps_fixture`, import-scan metrics) with bench/test coverage. | Implementer |

94
docs/implplan/archived/SPRINT_0407_0001_0001_scanner_bun_detection_gaps.md
View File

@@ -1,94 +0,0 @@
# Sprint 0407 - Scanner Bun Detection Gaps
## Topic & Scope
- Close Bun inventory blind-spots so scans reliably inventory dependencies across **installed `node_modules`**, **lockfile-only**, **workspace layouts**, **patched dependencies**, and **container layer trees**.
- Improve correctness and safety: never emit invalid/confident `pkg:npm/...@<range>` style identities; avoid leaking absolute paths; keep outputs deterministic with explicit bounds and audited “skipped” markers.
- Produce hard evidence: new fixtures + golden outputs covering bunfig-only projects, version-specific patches, container layer roots (`layers/`, `.layers/`, `layer*/`), and bun.lock v1 graph-based dev/prod classification.
- **Working directory:** `src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang.Bun` (tests: `src/Scanner/__Tests/StellaOps.Scanner.Analyzers.Lang.Bun.Tests`; optional benches: `src/Bench/StellaOps.Bench/Scanner.Analyzers`).
## Dependencies & Concurrency
- Interlocks with Node analyzer conventions for container root discovery and identity safety:
- `src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang.Node/Internal/NodeInputNormalizer.cs`
- `src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang.Node/Internal/NodeLockData.cs`
- Must remain parallel-safe with other language analyzers: no shared mutable global state; deterministic iteration over filesystem and lock entries.
- Offline-first: do not run `bun`, do not fetch registries, do not assume network.
## Documentation Prerequisites
- `docs/README.md`
- `docs/07_HIGH_LEVEL_ARCHITECTURE.md`
- `docs/modules/platform/architecture-overview.md`
- `docs/modules/scanner/architecture.md`
- `src/Scanner/AGENTS.md`
- `src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang/AGENTS.md`
- `src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang.Bun/AGENTS.md` (created 2025-12-13)
## Delivery Tracker
| # | Task ID | Status | Key dependency / next step | Owners | Task Definition |
| --- | --- | --- | --- | --- | --- |
| 1 | SCAN-BUN-407-001 | DONE | Fixture `lang/bun/container-layers` + determinism test passing. | Bun Analyzer Guild (`src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang.Bun`) | **Container-layer aware project discovery**: extend `Internal/BunProjectDiscoverer.cs` to discover Bun project roots not only under `context.RootPath`, but also under common OCI unpack layouts used elsewhere in scanner: `layers/*`, `.layers/*`, and `layer*` direct children. Do not skip hidden roots wholesale: `.layers` must be included. Keep traversal bounded and deterministic: (a) stable ordering of enumerated directories, (b) explicit depth caps per root, (c) hard cap on total discovered roots, (d) must never recurse into `node_modules/` and must skip large/non-project dirs deterministically. Acceptance: new fixture `lang/bun/container-layers` proves a Bun project placed under `.layers/layer0/app` is found and scanned. |
| 2 | SCAN-BUN-407-002 | DONE | Fixture `lang/bun/bunfig-only` + determinism test passing. | Bun Analyzer Guild (`src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang.Bun`) | **Declared-only fallback for bun markers**: if `BunProjectDiscoverer` identifies a project root (via `bunfig.toml`/`package.json`/etc.) but `BunInputNormalizer` returns `None` (no `node_modules`, no `bun.lock`), emit declared-only components from `package.json` dependencies. Requirements: (a) do not emit `pkg:npm/...@<range>` PURLs for version ranges/tags; use `AddFromExplicitKey` when version is not a concrete resolved version, (b) include deterministic metadata `declaredOnly=true`, `declared.source=package.json`, `declared.locator=<relative>#<section>`, `declared.versionSpec=<original>`, `declared.scope=<prod|dev|peer|optional>`, and (c) include root package.json evidence with sha256 (bounded). Acceptance: new fixture `lang/bun/bunfig-only` emits declared-only components for both `dependencies` and `devDependencies` with safe identities. |
| 3 | SCAN-BUN-407-003 | DONE | Fixture `lang/bun/lockfile-dev-classification` passing. | Bun Analyzer Guild (`src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang.Bun`) | **bun.lock v1 graph enrichment (dev/optional/peer + edges)**: upgrade `Internal/BunLockParser.cs` to preserve dependency edges from bun.lock v1 array form (capture dependency value/specifier, not only names) and to parse optional peer information when present. Build a bounded dependency graph that starts from root `package.json` declarations (prod/dev/optional/peer) and propagates reachability to lock entries, marking `BunLockEntry.IsDev/IsOptional/IsPeer` deterministically. If the graph cannot disambiguate (multiple versions/specifier mismatch), do not guess; emit `scopeUnknown=true` and keep `IsDev=false` unless positively proven. Acceptance: add fixture `lang/bun/lockfile-dev-classification` demonstrating: (a) dev-only packages are tagged `dev=true` and are excluded when `includeDev=false`, (b) prod packages remain untagged, (c) the decision is stable across OS/filesystem ordering. |
| 4 | SCAN-BUN-407-004 | DONE | Dev filter verified via fixture goldens. | Bun Analyzer Guild (`src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang.Bun`) | **Make `includeDev` meaningful**: `Internal/BunLockInventory.cs` currently filters by `entry.IsDev`, but bun.lock array parsing sets `IsDev=false` always. After graph enrichment (Task 3), implement deterministic filtering for lockfile-only scans and ensure installed scans also carry dev/optional/peer metadata when lock data is present. Acceptance: tests show dev filtering affects output only when the analyzer can prove dev reachability; otherwise outputs remain but are marked `scopeUnknown=true`. |
| 5 | SCAN-BUN-407-005 | DONE | Fixture `lang/bun/patched-multi-version` passing. | Bun Analyzer Guild (`src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang.Bun`) | **Version-specific patch mapping + no absolute paths**: fix `Internal/BunWorkspaceHelper.cs` so `patchedDependencies` keys preserve version specificity (`name@version`), and patch-directory discovery emits **relative** deterministic paths (relative to project root) rather than absolute OS paths. Update `BunLanguageAnalyzer` patch application so it first matches `name@version`, then falls back to `name` only when unambiguous. Acceptance: add fixture `lang/bun/patched-multi-version` with two patch files for the same package name at different versions; output marks only the correct version as patched and never includes absolute paths. |
| 6 | SCAN-BUN-407-006 | DONE | Goldens updated; bounded sha256 + lock locators added. | Bun Analyzer Guild (`src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang.Bun`) | **Evidence strengthening + locator precision**: improve `Internal/BunPackage.CreateEvidence()` so evidence locators are stable and specific: (a) package.json evidence includes sha256 (bounded; if skipped, emit `packageJson.hashSkipped=true` + `packageJson.hashSkipReason=<...>`), (b) bun.lock evidence uses locator `<lockfileRelativePath>:packages[<name>@<version>]` instead of plain `bun.lock`, (c) include lockfile sha256 once per project root via repeated evidence sha256 (bounded). Acceptance: update existing Bun fixtures’ goldens to reflect deterministic hashing and locator formats, with no nondeterministic absolute paths. |
| 7 | SCAN-BUN-407-007 | DONE | Fixture `lang/bun/non-concrete-versions` passing. | Bun Analyzer Guild (`src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang.Bun`) | **Identity safety for non-npm sources**: `Internal/BunPackage.BuildPurl()` always emits `pkg:npm/<name>@<version>`. Define and implement rules for `SourceType != npm` (git/file/link/workspace/tarball/custom-registry): when `version` is not a concrete registry version, emit `AddFromExplicitKey` (no PURL) and preserve the original specifier/resolved URL in metadata. If a PURL is emitted, it must be valid and must not embed raw specifiers like `workspace:*` as a “version”. Acceptance: add fixture `lang/bun/non-concrete-versions` demonstrating safe identities for `workspace:*` / `link:` / `file:` styles (if representable in bun.lock), with deterministic explicit keys and clear metadata markers. |
| 8 | SCAN-BUN-407-008 | DONE | Doc `docs/modules/scanner/analyzers-bun.md` published and sprint linked. | Docs Guild + Bun Analyzer Guild | **Document Bun analyzer detection contract**: add/update `docs/modules/scanner/analyzers-bun.md` (or the closest existing scanner doc) describing: what artifacts are used (node_modules, bun.lock, package.json), precedence rules, identity rules (PURL vs explicit-key), dev/optional/peer semantics, container layer root handling, and bounds (depth/roots/files/hash limits). Link this sprint from the doc and add a brief “known limitations” section (e.g., bun.lockb unsupported). |
| 9 | SCAN-BUN-407-009 | DONE | Added scenario `bun_multi_workspace_fixture` in analyzer microbench harness. | Bench Guild (`src/Bench/StellaOps.Bench/Scanner.Analyzers`) | **Offline benchmark**: add a deterministic bench that scans a representative Bun monorepo fixture (workspaces + many packages) and records elapsed time + component counts. Establish a ceiling and guard against regressions. |
## Wave Coordination
| Wave | Guild owners | Shared prerequisites | Status | Notes |
| --- | --- | --- | --- | --- |
| A: Discovery & Declared-only | Bun Analyzer Guild + QA Guild | Actions 1–2 | TODO | Make projects discoverable and avoid “no output” cases. |
| B: Lock graph & scopes | Bun Analyzer Guild + QA Guild | Action 3 | TODO | Correct dev/optional/peer and make includeDev meaningful. |
| C: Patches & evidence | Bun Analyzer Guild + QA Guild | Action 4 | TODO | Version-specific patches; deterministic evidence/hashes. |
| D: Identity safety | Bun Analyzer Guild + Security Guild | Action 1 | TODO | Non-npm sources and non-concrete versions never become “fake versions”. |
| E: Docs & bench | Docs Guild + Bench Guild | Waves A–D | TODO | Contract and perf guardrails. |
## Wave Detail Snapshots
- **Wave A:** Discover Bun projects under OCI layer layouts; declared-only emission when no install/lock evidence exists.
- **Wave B:** bun.lock v1 graph enrichment provides auditable dev/optional/peer classification and enables reliable dev filtering.
- **Wave C:** Patched dependency mapping is version-correct and deterministic; evidence locators/hashes become strong and stable.
- **Wave D:** Identity rules prevent invalid PURLs and reduce false vuln matches for non-registry packages.
- **Wave E:** Documented contract + optional benchmark keeps behavior stable over time.
## Interlocks
- **Identity safety:** Never emit `pkg:npm/...@<range|tag|workspace:*|file:...|link:...>`; use explicit keys for non-concrete versions/specifiers. (Action 1)
- **Container traversal bounds:** Project discovery must not devolve into full-root recursion on container roots; bounds must be explicit and test-covered. (Action 2)
- **Scope correctness:** Dev/optional/peer flags must be derived deterministically (graph or explicit signals). When uncertain, mark unknown rather than guessing. (Action 3)
- **No path leakage:** Metadata/evidence must not include absolute host paths (patch file discovery is the primary risk). (Action 4)
## Upcoming Checkpoints
- 2025-12-13: Approve identity scheme + container discovery contract + scope semantics + patch rules (Actions 1–4).
- 2025-12-16: Wave A complete with container-layers + bunfig-only fixtures passing.
- 2025-12-18: Wave B complete with dev/optional/peer classification fixture and includeDev filter tests.
- 2025-12-20: Wave C + D complete (patch mapping + evidence hashing + identity safety) with updated goldens.
- 2025-12-22: Wave E docs complete; bench decision made; sprint ready for DONE review.
## Action Tracker
| # | Action | Owner | Due (UTC) | Status | Notes |
| --- | --- | --- | --- | --- | --- |
| 1 | Decide explicit-key identity scheme for Bun declared-only and non-npm sources (ranges/tags/git/file/link/workspace). | Project Mgmt + Scanner Guild | 2025-12-13 | Done | Implemented per `docs/modules/scanner/language-analyzers-contract.md`. |
| 2 | Decide and document container layer root discovery rules for Bun analyzer (parity with Node's `layers/.layers/layer*` conventions, depth/roots bounds). | Project Mgmt + Bun Analyzer Guild | 2025-12-13 | Done | Implemented per `docs/modules/scanner/language-analyzers-contract.md`; validated by fixture `lang/bun/container-layers`. |
| 3 | Decide bun.lock v1 scope derivation rules (dev/optional/peer) and how uncertainty is represented (`scopeUnknown` markers). | Project Mgmt + Bun Analyzer Guild | 2025-12-13 | Done | Implemented in `Internal/BunLockScopeClassifier.cs` with `scopeUnknown=true` for ambiguity. |
| 4 | Decide patched dependency keying and deterministic path normalization (relative path base, name@version precedence, fallback rules). | Project Mgmt + Bun Analyzer Guild + Security Guild | 2025-12-13 | Done | Implemented in `Internal/BunWorkspaceHelper.cs` (version-specific keys; project-relative patch paths). |
| 5 | Create missing module charter: `src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang.Bun/AGENTS.md`. | Project Mgmt | 2025-12-13 | Done | Created `src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Lang.Bun/AGENTS.md`. |
## Decisions & Risks
- Decisions implemented per `docs/modules/scanner/language-analyzers-contract.md` and documented in `docs/modules/scanner/analyzers-bun.md`.
| Risk ID | Risk | Impact | Likelihood | Mitigation | Owner | Trigger / Signal |
| --- | --- | --- | --- | --- | --- | --- |
| R1 | Container root discovery causes perf regressions on large rootfs trees. | High | Medium | Explicit bounds + deterministic skipping; add container-layers fixture and (optional) benchmark. | Bun Analyzer Guild | CI timeouts; high CPU usage scanning container roots. |
| R2 | Dev/optional/peer classification is wrong or unstable due to ambiguous graph edges. | High | Medium | Prefer “unknown” markers over guesses; stabilize matching using dependency specifiers when available; fixture for ambiguity. | Bun Analyzer Guild | Flaky golden outputs; incorrect dev filtering reported by users. |
| R3 | Invalid PURLs or range-as-version identities cause false vulnerability matches. | High | Medium | Explicit-key for non-concrete versions; document semantics; add fixtures asserting absence of invalid `@^...` or `@workspace:*` PURLs. | Security Guild + Bun Analyzer Guild | Vuln-match spike; downstream consumers reject PURLs. |
| R4 | Absolute paths leak into metadata/evidence (patch discovery, symlink realpaths). | Medium | Medium | Normalize to project-relative paths; add fixture that fails if absolute paths appear. | Bun Analyzer Guild | Golden diffs include host-specific paths. |
| R5 | Evidence hashing increases runtime and memory usage. | Medium | Low/Medium | Hash only bounded files; cache per file path; record `hashSkipped` markers when exceeding size caps. | Bench Guild | Bench regression; memory spikes. |
## Execution Log
| Date (UTC) | Update | Owner |
| --- | --- | --- |
| 2025-12-12 | Sprint created to close Bun analyzer detection gaps (container-layer discovery, declared-only fallback, bun.lock scope graph, version-specific patches, evidence hashing, identity safety) with fixtures/docs/bench expectations. | Project Mgmt |
| 2025-12-13 | Completed SCAN-BUN-407-001 and SCAN-BUN-407-002 with new fixtures (`lang/bun/container-layers`, `lang/bun/bunfig-only`) and deterministic goldens; aligned explicit-key behavior with `docs/modules/scanner/language-analyzers-contract.md`. | Bun Analyzer Guild |
| 2025-12-13 | Completed SCAN-BUN-407-003 through SCAN-BUN-407-008 (scope graph + dev filtering, version-specific patch mapping, bounded sha256 evidence, non-concrete identity safety, and Bun analyzer contract doc). | Bun Analyzer Guild |
| 2025-12-13 | Completed SCAN-BUN-407-009 by wiring the Bun analyzer into the scanner analyzer microbench harness and adding scenario `bun_multi_workspace_fixture`. | Bench Guild |

159
docs/implplan/archived/SPRINT_0410_0001_0001_entrypoint_detection_reengineering_program.md
View File

@@ -1,159 +0,0 @@
# Sprint 0410.0001.0001 - Entrypoint Detection Re-Engineering Program
## Topic & Scope
- Window: 2025-12-16 -> 2026-02-28 (UTC); phased delivery across 5 child sprints.
- **Vision:** Re-engineer entrypoint detection to be industry-leading with semantic understanding, temporal tracking, multi-container mesh analysis, speculative execution, binary intelligence, and predictive risk scoring.
- **Strategic Goal:** Position StellaOps entrypoint detection as the foundation for context-aware vulnerability assessment - answering not just "what's installed" but "what's running, how it's invoked, and what can reach it."
- **Working directory:** `docs/implplan` (coordination); implementation in `src/Scanner/__Libraries/StellaOps.Scanner.EntryTrace/` and related modules.
## Program Architecture
### Current State
The existing entrypoint detection has:
- Container-level OCI config parsing (ENTRYPOINT/CMD)
- ShellFlow static analyzer for shell scripts
- Per-language analyzers (Python, Java, Node, .NET, Go, Ruby, Rust, Bun, Deno, PHP)
- Evidence chains with `usedByEntrypoint` flags
- Dual-mode (static image + running container)
### Target State: Entrypoint Knowledge Graph
```
┌────────────────────────────────────────────────────────────────────┐
│ ENTRYPOINT KNOWLEDGE GRAPH │
├────────────────────────────────────────────────────────────────────┤
│ │
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │
│ │ Semantic │────▶│ Temporal │────▶│ Mesh │ │
│ │ Engine │ │ Graph │ │ Analysis │ │
│ └──────────────┘ └──────────────┘ └──────────────┘ │
│ │ │ │ │
│ ▼ ▼ ▼ │
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │
│ │ Speculative │────▶│ Binary │────▶│ Predictive │ │
│ │ Execution │ │ Intelligence │ │ Risk │ │
│ └──────────────┘ └──────────────┘ └──────────────┘ │
│ │
│ Query: "Which images have Django entrypoints reachable to │
│ log4j 2.14.1?" │
│ Answer: 847 images, 12 in production, 3 internet-facing │
│ │
└────────────────────────────────────────────────────────────────────┘
```
## Child Sprints
| Sprint ID | Name | Focus | Window | Status |
|-----------|------|-------|--------|--------|
| 0411.0001.0001 | Semantic Entrypoint Engine | Semantic understanding, intent/capability inference | 2025-12-16 -> 2025-12-30 | TODO |
| 0412.0001.0001 | Temporal & Mesh Entrypoint | Temporal tracking, multi-container mesh | 2026-01-02 -> 2026-01-17 | TODO |
| 0413.0001.0001 | Speculative Execution Engine | Symbolic execution, path enumeration | 2026-01-20 -> 2026-02-03 | TODO |
| 0414.0001.0001 | Binary Intelligence | Fingerprinting, symbol recovery | 2026-02-06 -> 2026-02-17 | TODO |
| 0415.0001.0001 | Predictive Risk Scoring | Risk-aware scoring, business context | 2026-02-20 -> 2026-02-28 | TODO |
## Dependencies & Concurrency
- Upstream: Sprint 0401 Reachability Evidence Chain (completed tasks for richgraph-v1, symbol_id, code_id).
- Upstream: Sprint 0408 Scanner Language Detection Gaps Program (mature language analyzers).
- Child sprints 0411-0413 can proceed in parallel after semantic foundation lands.
- Sprints 0414-0415 depend on earlier sprints for data structures but can overlap.
## Documentation Prerequisites
- docs/modules/scanner/architecture.md
- docs/modules/scanner/operations/entrypoint-problem.md
- docs/modules/scanner/operations/entrypoint-static-analysis.md
- docs/modules/scanner/operations/entrypoint-shell-analysis.md
- docs/modules/scanner/operations/entrypoint-runtime-overview.md
- docs/reachability/function-level-evidence.md
- docs/reachability/lattice.md
- src/Scanner/__Libraries/StellaOps.Scanner.EntryTrace/AGENTS.md (to be created)
## Key Deliverables
### Phase 1: Semantic Foundation (Sprint 0411)
1. **SemanticEntrypoint** record with intent, capabilities, attack surface
2. **ApplicationIntent** enumeration (web-server, cli-tool, batch-job, worker, serverless, etc.)
3. **CapabilityClass** enumeration (network-listen, file-write, exec-spawn, crypto, etc.)
4. **ThreatVector** inference from entrypoint characteristics
5. Cross-language semantic detection adapters
### Phase 2: Temporal & Mesh (Sprint 0412)
1. **TemporalEntrypointGraph** for version-to-version tracking
2. **EntrypointDrift** detection and alerting
3. **MeshEntrypointGraph** for multi-container orchestration
4. **CrossContainerPath** reachability across services
5. Kubernetes/Compose manifest parsing
### Phase 3: Speculative Execution (Sprint 0413)
1. **SymbolicExecutionEngine** for ShellFlow enhancement
2. **PathEnumerator** for all terminal states
3. **ConstraintSolver** for complex conditionals
4. **BranchCoverage** metrics and confidence
### Phase 4: Binary Intelligence (Sprint 0414)
1. **CodeFingerprint** index from OSS package corpus
2. **SymbolRecovery** for stripped binaries
3. **SourceCorrelation** service
4. **FunctionSignatureInference** from binary analysis
### Phase 5: Predictive Risk (Sprint 0415)
1. **RiskFactorExtractor** pipeline
2. **EntrypointRiskScorer** with business context
3. **AttackSurfaceQuantifier** per entrypoint
4. **EntrypointAsCode** auto-generated specifications
## Competitive Differentiation
| Capability | StellaOps (Target) | Competition |
|------------|-------------------|-------------|
| Semantic understanding | Full intent + capability inference | Pattern matching only |
| Temporal tracking | Version-to-version evolution | Snapshot only |
| Multi-container | Full mesh with cross-container reachability | Single container |
| Stripped binaries | Fingerprint + ML recovery | Limited/none |
| Speculative execution | All paths enumerated symbolically | Best-effort heuristics |
| Entrypoint-as-Code | Auto-generated, executable specs | Manual documentation |
| Predictive risk | Business-context-aware scoring | Static CVSS only |
## Wave Coordination
| Wave | Child Sprints | Shared Prerequisites | Status | Notes |
|------|---------------|----------------------|--------|-------|
| Foundation | 0411 | Sprint 0401 richgraph/symbol contracts | TODO | Must land before other phases |
| Parallel | 0412, 0413 | 0411 semantic records | TODO | Can run concurrently |
| Intelligence | 0414 | 0411-0413 data structures | TODO | Binary focus |
| Risk | 0415 | 0411-0414 evidence chains | TODO | Final phase |
## Interlocks
- Semantic record schema (Sprint 0411) must stabilize before Temporal/Mesh (0412) or Speculative (0413) start.
- Binary fingerprint corpus (Sprint 0414) requires OSS package index integration.
- Risk scoring (Sprint 0415) needs Policy Engine integration for gate enforcement.
- All phases emit to richgraph-v1 with BLAKE3 hashing per CONTRACT-RICHGRAPH-V1-015.
## Upcoming Checkpoints
- 2025-12-16 - Sprint 0411 kickoff; semantic schema draft review.
- 2025-12-23 - Sprint 0411 midpoint; ApplicationIntent/CapabilityClass enums frozen.
- 2025-12-30 - Sprint 0411 close; semantic foundation ready for 0412/0413.
- 2026-01-02 - Sprints 0412/0413 kickoff (parallel).
- 2026-02-28 - Program close; all phases delivered.
## Action Tracker
| # | Action | Owner | Due (UTC) | Status | Notes |
|---|--------|-------|-----------|--------|-------|
| 1 | Create AGENTS.md for EntryTrace module | Scanner Guild | 2025-12-16 | TODO | Foundation for implementers |
| 2 | Draft SemanticEntrypoint schema | Scanner Guild | 2025-12-18 | TODO | Phase 1 core deliverable |
| 3 | Define ApplicationIntent enumeration | Scanner Guild | 2025-12-20 | TODO | Needs cross-language input |
| 4 | Create temporal graph storage design | Platform Guild | 2026-01-02 | TODO | Phase 2 dependency |
| 5 | Evaluate binary fingerprint corpus options | Scanner Guild | 2026-02-01 | TODO | Phase 4 dependency |
## Decisions & Risks
| ID | Risk | Impact | Mitigation / Owner |
|----|------|--------|-------------------|
| R1 | Semantic schema changes mid-program | Rework in dependent phases | Freeze schema by Sprint 0411 close; Scanner Guild |
| R2 | Binary fingerprint corpus size/latency | Slow startup, large storage | Use lazy loading, tiered caching; Platform Guild |
| R3 | Multi-container mesh complexity | Detection gaps in complex K8s | Phased support; start with common patterns; Scanner Guild |
| R4 | Speculative execution path explosion | Performance issues | Add depth limits, caching; Scanner Guild |
| R5 | Risk scoring model accuracy | False confidence signals | Train on CVE exploitation data; validate with red team; Signals Guild |
## Execution Log
| Date (UTC) | Update | Owner |
|------------|--------|-------|
| 2025-12-13 | Created program sprint from strategic analysis; outlined 5 child sprints with phased delivery; defined competitive differentiation matrix. | Planning |

164
docs/implplan/archived/SPRINT_0411_0001_0001_semantic_entrypoint_engine.md
View File

@@ -1,164 +0,0 @@
# Sprint 0411.0001.0001 - Semantic Entrypoint Engine
## Topic & Scope
- Window: 2025-12-16 -> 2025-12-30 (UTC); foundation phase for entrypoint re-engineering.
- Build semantic understanding layer that infers intent, capabilities, and attack surface from entrypoints.
- Enable downstream phases (temporal, mesh, speculative, binary, risk) with stable data structures.
- **Working directory:** `src/Scanner/__Libraries/StellaOps.Scanner.EntryTrace/Semantic/`
## Dependencies & Concurrency
- Upstream: Sprint 0401 Reachability Evidence Chain (richgraph-v1, symbol_id, code_id contracts).
- Upstream: Sprint 0408 Language Detection Gaps (mature Python/Java/Node analyzers).
- Blocks: Sprints 0412-0415 depend on semantic records from this sprint.
- Language-specific adapters can be developed in parallel once core schema lands.
## Documentation Prerequisites
- docs/modules/scanner/operations/entrypoint-problem.md
- docs/modules/scanner/operations/entrypoint-static-analysis.md
- docs/modules/scanner/operations/entrypoint-lang-*.md (per-language guides)
- docs/reachability/function-level-evidence.md
- src/Scanner/__Libraries/StellaOps.Scanner.EntryTrace/AGENTS.md
## Delivery Tracker
| # | Task ID | Status | Key dependency / next step | Owners | Task Definition |
|---|---------|--------|---------------------------|--------|-----------------|
| 1 | ENTRY-SEM-411-001 | DONE | None; foundation task | Scanner Guild | Create `SemanticEntrypoint` record with Id, Specification, Intent, Capabilities, AttackSurface, DataBoundaries, Confidence fields. |
| 2 | ENTRY-SEM-411-002 | DONE | Task 1 | Scanner Guild | Define `ApplicationIntent` enumeration: WebServer, CliTool, BatchJob, Worker, Serverless, Daemon, InitSystem, Supervisor, DatabaseServer, MessageBroker, CacheServer, ProxyGateway, Unknown. |
| 3 | ENTRY-SEM-411-003 | DONE | Task 1 | Scanner Guild | Define `CapabilityClass` enumeration: NetworkListen, NetworkConnect, FileRead, FileWrite, ProcessSpawn, CryptoOperation, DatabaseAccess, MessageQueue, CacheAccess, ExternalApi, UserInput, ConfigLoad, SecretAccess, LogEmit. |
| 4 | ENTRY-SEM-411-004 | DONE | Task 1 | Scanner Guild | Define `ThreatVector` record with VectorType (Ssrf, Sqli, Xss, Rce, PathTraversal, Deserialization, TemplateInjection, AuthBypass, InfoDisclosure, Dos), Confidence, Evidence, EntryPath. |
| 5 | ENTRY-SEM-411-005 | DONE | Task 1 | Scanner Guild | Define `DataFlowBoundary` record with BoundaryType (HttpRequest, HttpResponse, FileInput, FileOutput, DatabaseQuery, MessageReceive, MessageSend, EnvironmentVar, CommandLineArg), Direction, Sensitivity. |
| 6 | ENTRY-SEM-411-006 | DONE | Task 1 | Scanner Guild | Define `SemanticConfidence` record with Score (0.0-1.0), Tier (Definitive, High, Medium, Low, Unknown), ReasoningChain (list of evidence strings). |
| 7 | ENTRY-SEM-411-007 | DONE | Tasks 1-6 | Scanner Guild | Create `ISemanticEntrypointAnalyzer` interface with `AnalyzeAsync(EntryTraceResult, LanguageAnalyzerResult, CancellationToken) -> SemanticEntrypoint`. |
| 8 | ENTRY-SEM-411-008 | DONE | Task 7 | Scanner Guild | Implement `PythonSemanticAdapter` inferring intent from: Django (WebServer), Celery (Worker), Click/Typer (CliTool), Lambda (Serverless), Flask/FastAPI (WebServer). |
| 9 | ENTRY-SEM-411-009 | DONE | Task 7 | Scanner Guild | Implement `JavaSemanticAdapter` inferring intent from: Spring Boot (WebServer), Quarkus (WebServer), Micronaut (WebServer), Kafka Streams (Worker), Main-Class patterns. |
| 10 | ENTRY-SEM-411-010 | DONE | Task 7 | Scanner Guild | Implement `NodeSemanticAdapter` inferring intent from: Express/Koa/Fastify (WebServer), CLI bin entries (CliTool), worker threads, Lambda handlers (Serverless). |
| 11 | ENTRY-SEM-411-011 | DONE | Task 7 | Scanner Guild | Implement `DotNetSemanticAdapter` inferring intent from: ASP.NET Core (WebServer), Console apps (CliTool), Worker services (Worker), Azure Functions (Serverless). |
| 12 | ENTRY-SEM-411-012 | DONE | Task 7 | Scanner Guild | Implement `GoSemanticAdapter` inferring intent from: net/http patterns (WebServer), cobra/urfave CLI (CliTool), gRPC servers, main package analysis. |
| 13 | ENTRY-SEM-411-013 | DONE | Tasks 8-12 | Scanner Guild | Create `CapabilityDetector` that analyzes imports/dependencies to infer capabilities (e.g., `import socket` -> NetworkConnect, `import os.path` -> FileRead). |
| 14 | ENTRY-SEM-411-014 | DONE | Task 13 | Scanner Guild | Create `ThreatVectorInferrer` that maps capabilities and framework patterns to likely attack vectors (e.g., WebServer + DatabaseAccess + UserInput -> Sqli risk). |
| 15 | ENTRY-SEM-411-015 | DONE | Task 13 | Scanner Guild | Create `DataBoundaryMapper` that traces data flow edges from entrypoint through framework handlers to I/O boundaries. |
| 16 | ENTRY-SEM-411-016 | DONE | Tasks 7-15 | Scanner Guild | Create `SemanticEntrypointOrchestrator` that composes adapters, detectors, and inferrers into unified semantic analysis pipeline. |
| 17 | ENTRY-SEM-411-017 | DONE | Task 16 | Scanner Guild | Integrate semantic analysis into `EntryTraceAnalyzer` post-processing, emit `SemanticEntrypoint` alongside `EntryTraceResult`. |
| 18 | ENTRY-SEM-411-018 | DONE | Task 17 | Scanner Guild | Add semantic fields to `LanguageComponentRecord`: `intent`, `capabilities[]`, `threatVectors[]`. |
| 19 | ENTRY-SEM-411-019 | DONE | Task 18 | Scanner Guild | Update richgraph-v1 schema to include semantic metadata on entrypoint nodes. |
| 20 | ENTRY-SEM-411-020 | DONE | Task 19 | Scanner Guild | Add CycloneDX and SPDX property extensions for semantic entrypoint data. |
| 21 | ENTRY-SEM-411-021 | DONE | Tasks 8-12 | QA Guild | Create test fixtures for each language semantic adapter with expected intent/capabilities. |
| 22 | ENTRY-SEM-411-022 | DONE | Task 21 | QA Guild | Add golden test suite validating semantic analysis determinism. |
| 23 | ENTRY-SEM-411-023 | DONE | Task 22 | Docs Guild | Document semantic entrypoint schema in `docs/modules/scanner/semantic-entrypoint-schema.md`. |
| 24 | ENTRY-SEM-411-024 | DONE | Task 23 | Docs Guild | Update `docs/modules/scanner/architecture.md` with semantic analysis pipeline. |
| 25 | ENTRY-SEM-411-025 | DONE | Task 24 | CLI Guild | Add `stella scan --semantic` flag and semantic output fields to JSON/table formats. |
## Wave Coordination
| Wave | Tasks | Shared Prerequisites | Status | Notes |
|------|-------|---------------------|--------|-------|
| Schema Definition | 1-6 | None | DONE | Core data structures |
| Adapter Interface | 7 | Schema frozen | DONE | Contract for language adapters |
| Language Adapters | 8-12 | Interface defined | DONE | Can run in parallel |
| Cross-Cutting Analysis | 13-15 | Adapters started | DONE | Capability/threat/boundary detection |
| Integration | 16-20 | Adapters + analysis | DONE | DI registration, schema integration, SBOM extensions |
| QA & Docs | 21-25 | Integration complete | DONE | Tests, docs, CLI flag all complete |
## Interlocks
- Schema tasks (1-6) must complete before interface task (7).
- Interface task (7) gates all language adapters (8-12).
- Language adapters can proceed in parallel.
- Cross-cutting analysis (13-15) can start once any adapter is in progress.
- Integration tasks (16-20) require most adapters complete.
- QA/Docs (21-25) can overlap with late integration.
## Upcoming Checkpoints
- 2025-12-18 - Schema freeze (tasks 1-6 complete); interface draft (task 7).
- 2025-12-23 - Language adapters midpoint (tasks 8-12 in progress); cross-cutting analysis started.
- 2025-12-27 - Integration tasks started (tasks 16-20).
- 2025-12-30 - Sprint close; semantic foundation ready.
## Action Tracker
| # | Action | Owner | Due (UTC) | Status | Notes |
|---|--------|-------|-----------|--------|-------|
| 1 | Review existing entrypoint detection code | Scanner Guild | 2025-12-16 | TODO | Understand integration points |
| 2 | Draft ApplicationIntent enum with cross-team input | Scanner Guild | 2025-12-17 | TODO | Need input from all language teams |
| 3 | Create AGENTS.md for EntryTrace module | Scanner Guild | 2025-12-16 | TODO | Implementer guidance |
| 4 | Validate semantic schema against richgraph-v1 | Platform Guild | 2025-12-18 | TODO | Ensure compatibility |
## Decisions & Risks
| ID | Risk | Impact | Mitigation / Owner |
|----|------|--------|-------------------|
| R1 | Intent enumeration incomplete | Missing application types | Start with common patterns; extend as needed; Scanner Guild |
| R2 | Capability detection false positives | Noise in attack surface | Use confidence scoring; require multiple signals; Scanner Guild |
| R3 | Schema changes after freeze | Rework in dependent sprints | Strict freeze enforcement after 2025-12-18; Planning |
| R4 | Language adapter coverage gaps | Inconsistent semantic depth | Prioritize Python/Java/Node; others can be stubs; Scanner Guild |
## Schema Preview
### SemanticEntrypoint Record
```csharp
public sealed record SemanticEntrypoint
{
public required string Id { get; init; }
public required EntrypointSpecification Specification { get; init; }
public required ApplicationIntent Intent { get; init; }
public required ImmutableArray<CapabilityClass> Capabilities { get; init; }
public required ImmutableArray<ThreatVector> AttackSurface { get; init; }
public required ImmutableArray<DataFlowBoundary> DataBoundaries { get; init; }
public required SemanticConfidence Confidence { get; init; }
public ImmutableDictionary<string, string>? Metadata { get; init; }
}
```
### ApplicationIntent Enumeration
```csharp
public enum ApplicationIntent
{
Unknown = 0,
WebServer = 1, // HTTP/HTTPS listener (Django, Express, ASP.NET)
CliTool = 2, // Command-line utility (Click, Cobra)
BatchJob = 3, // One-shot data processing
Worker = 4, // Background job processor (Celery, Sidekiq)
Serverless = 5, // FaaS handler (Lambda, Azure Functions)
Daemon = 6, // Long-running background service
InitSystem = 7, // Process manager (systemd, s6)
Supervisor = 8, // Child process supervisor
DatabaseServer = 9, // Database engine
MessageBroker = 10, // Message queue server
CacheServer = 11, // Cache/session store
ProxyGateway = 12, // Reverse proxy, API gateway
TestRunner = 13, // Test framework execution
DevServer = 14, // Development-only server
}
```
### CapabilityClass Enumeration
```csharp
[Flags]
public enum CapabilityClass : long
{
None = 0,
NetworkListen = 1 << 0, // Opens listening socket
NetworkConnect = 1 << 1, // Makes outbound connections
FileRead = 1 << 2, // Reads from filesystem
FileWrite = 1 << 3, // Writes to filesystem
ProcessSpawn = 1 << 4, // Spawns child processes
CryptoOperation = 1 << 5, // Encryption/signing operations
DatabaseAccess = 1 << 6, // Database client operations
MessageQueue = 1 << 7, // Message broker client
CacheAccess = 1 << 8, // Cache client operations
ExternalApi = 1 << 9, // External HTTP API calls
UserInput = 1 << 10, // Accepts user input
ConfigLoad = 1 << 11, // Loads configuration files
SecretAccess = 1 << 12, // Accesses secrets/credentials
LogEmit = 1 << 13, // Emits logs
MetricsEmit = 1 << 14, // Emits metrics/telemetry
SystemCall = 1 << 15, // Makes privileged syscalls
ContainerEscape = 1 << 16, // Capabilities enabling escape
KernelModule = 1 << 17, // Loads kernel modules
Ptrace = 1 << 18, // Process tracing
RawSocket = 1 << 19, // Raw network access
}
```
## Execution Log
| Date (UTC) | Update | Owner |
|------------|--------|-------|
| 2025-12-13 | Created sprint from program sprint 0410; defined 25 tasks across schema, adapters, integration, QA/docs; included schema previews. | Planning |
| 2025-12-13 | Completed tasks 17-25: DI registration (AddSemanticEntryTraceAnalyzer), LanguageComponentRecord semantic fields (intent, capabilities, threatVectors), verified richgraph-v1 semantic extensions and SBOM property extensions already implemented, verified test fixtures exist, created semantic-entrypoint-schema.md documentation, updated architecture.md with semantic engine section, verified CLI --semantic flag implementation. Sprint 100% complete. | Scanner Guild |

178
docs/implplan/archived/SPRINT_3407_0001_0001_postgres_cleanup.md
View File

@@ -1,178 +0,0 @@
# Sprint 3407 · PostgreSQL Conversion: Phase 7 — Cleanup & Optimization
## Topic & Scope
- Final cleanup after Mongo→Postgres conversion: remove Mongo code/dual-write paths, archive Mongo data, tune Postgres, update docs and air-gap kit.
- **Working directory:** cross-module; coordination in this sprint doc. Code/docs live under respective modules, `deploy/`, `docs/db/`, `docs/operations/`.
## Dependencies & Concurrency
- Upstream: Phases 3400–3406 must be DONE before cleanup.
- Executes after all module cutovers; tasks have explicit serial dependencies below.
- Reference: `docs/db/tasks/PHASE_7_CLEANUP.md`.
## Wave Coordination
- **Wave A (code removal):** T7.1.x (Mongo removal) executes first; unlocks Waves B–E.
- **Wave B (data archive):** T7.2.x (backup/export/archive/decommission) runs after Wave A completes.
- **Wave C (performance):** T7.3.x tuning after archives; requires prod telemetry.
- **Wave D (docs):** T7.4.x updates after performance baselines; depends on previous waves for accuracy.
- **Wave E (air-gap kit):** T7.5.x after docs finalize to avoid drift; repack kit with Postgres-only assets.
- Keep waves strictly sequential; no parallel starts to avoid partial Mongo remnants.
## Documentation Prerequisites
- docs/db/README.md
- docs/db/SPECIFICATION.md
- docs/db/RULES.md
- docs/db/VERIFICATION.md
- All module AGENTS.md files
## Delivery Tracker
### T7.1: Remove MongoDB Dependencies
| # | Task ID | Status | Key dependency / next step | Owners | Task Definition |
| --- | --- | --- | --- | --- | --- |
| 1 | PG-T7.1.1 | DONE | All phases complete | Infrastructure Guild | Remove `StellaOps.Authority.Storage.Mongo` project |
| 2 | PG-T7.1.2 | DONE | Scheduler Postgres stores complete; Mongo project deleted. | Infrastructure Guild | Remove `StellaOps.Scheduler.Storage.Mongo` project |
| 3 | PG-T7.1.3 | DONE | Notify using Postgres storage; Mongo lib/tests deleted from solution and disk. | Infrastructure Guild | Remove `StellaOps.Notify.Storage.Mongo` project |
| 4 | PG-T7.1.4 | DONE | Policy Engine Storage/Mongo folder deleted; using Postgres storage. | Infrastructure Guild | Remove `StellaOps.Policy.Storage.Mongo` project |
| 5 | PG-T7.1.5 | DONE | Concelier Postgres storage complete; Mongo stale folders deleted. | Infrastructure Guild | Remove `StellaOps.Concelier.Storage.Mongo` project |
| 6 | PG-T7.1.6 | DONE | Excititor Mongo stale folders deleted; using Postgres storage. | Infrastructure Guild | Remove `StellaOps.Excititor.Storage.Mongo` project |
| 7 | PG-T7.1.D1 | DONE | Decision recorded 2025-12-06 | Project Mgmt | Decision record to unblock PG-T7.1.2; capture in Execution Log and update Decisions & Risks. |
| 8 | PG-T7.1.D2 | DONE | Decision recorded 2025-12-06 | Project Mgmt | Decision record to unblock PG-T7.1.3; capture in Execution Log and update Decisions & Risks. |
| 9 | PG-T7.1.D3 | DONE | Decision recorded 2025-12-06 | Project Mgmt | Decision record to unblock PG-T7.1.4; capture in Execution Log and update Decisions & Risks. |
| 10 | PG-T7.1.D4 | DONE | Decision recorded 2025-12-06 | Project Mgmt | Decision record to unblock PG-T7.1.5; capture in Execution Log and update Decisions & Risks. |
| 11 | PG-T7.1.D5 | DONE | Decision recorded 2025-12-06 | Project Mgmt | Decision record to unblock PG-T7.1.6; capture in Execution Log and update Decisions & Risks. |
| 12 | PG-T7.1.D6 | DONE | Impact/rollback plan published at `docs/db/reports/mongo-removal-decisions-20251206.md` | Infrastructure Guild | Provide one-pager per module to accompany decision approvals and accelerate deletion PRs. |
| 13 | PG-T7.1.PLAN | DONE | Plan published in Appendix A below | Infrastructure Guild | Produce migration playbook (order of removal, code replacements, test strategy, rollback checkpoints). |
| 14 | PG-T7.1.2a | DONE | Postgres GraphJobStore/PolicyRunService implemented and DI switched. | Scheduler Guild | Add Postgres equivalents and switch DI in WebService/Worker; prerequisite for deleting Mongo store. |
| 15 | PG-T7.1.2b | DONE | Scheduler.Backfill uses Postgres repositories only. | Scheduler Guild | Remove Mongo Options/Session usage; update fixtures/tests accordingly. |
| 16 | PG-T7.1.2c | DONE | Mongo project references removed; stale bin/obj deleted. | Infrastructure Guild | After 2a/2b complete, delete Mongo csproj + solution entries. |
| 7 | PG-T7.1.7 | DONE | Updated 7 solution files to remove Mongo project entries. | Infrastructure Guild | Update solution files |
| 8 | PG-T7.1.8 | DONE | Fixed csproj refs in Authority/Notifier to use Postgres storage. | Infrastructure Guild | Remove dual-write wrappers |
| 9 | PG-T7.1.9 | N/A | MongoDB config in TaskRunner/IssuerDirectory/AirGap/Attestor out of Wave A scope. | Infrastructure Guild | Remove MongoDB configuration options |
| 10 | PG-T7.1.10 | DONE | All Storage.Mongo csproj references removed; build verified (network issues only). | Infrastructure Guild | Run full build to verify no broken references |
| 14 | PG-T7.1.5a | DONE | Concelier Guild | Concelier: replace Mongo deps with Postgres equivalents; remove MongoDB packages; compat layer added. |
| 15 | PG-T7.1.5b | DONE | Concelier Guild | Build Postgres document/raw storage + state repositories and wire DI. |
| 16 | PG-T7.1.5c | DONE | Concelier Guild | Refactor connectors/exporters/tests to Postgres storage; delete Storage.Mongo code. |
| 17 | PG-T7.1.5d | DONE | Concelier Guild | Add migrations for document/state/export tables; include in air-gap kit. |
| 18 | PG-T7.1.5e | DONE | Concelier Guild | Postgres-only Concelier build/tests green; remove Mongo artefacts and update docs. |
| 19 | PG-T7.1.5f | DONE | Stale MongoCompat folders deleted; connectors now use Postgres storage contracts. | Concelier Guild | Remove MongoCompat shim and any residual Mongo-shaped payload handling after Postgres parity sweep; update docs/DI/tests accordingly. |
### T7.3: PostgreSQL Performance Optimization
| # | Task ID | Status | Key dependency / next step | Owners | Task Definition |
| --- | --- | --- | --- | --- | --- |
| 17 | PG-T7.3.1 | DONE | pg_stat_statements enabled in docker compose configs | DBA Guild | Enable `pg_stat_statements` extension |
| 18 | PG-T7.3.2 | DONE | Documented in postgresql-guide.md | DBA Guild | Identify slow queries |
| 19 | PG-T7.3.3 | DONE | Documented in postgresql-guide.md | DBA Guild | Analyze query plans with EXPLAIN ANALYZE |
| 20 | PG-T7.3.4 | DONE | Index guidelines documented | DBA Guild | Add missing indexes |
| 21 | PG-T7.3.5 | DONE | Unused index queries documented | DBA Guild | Remove unused indexes |
| 22 | PG-T7.3.6 | DONE | Tuning guide in postgresql-guide.md | DBA Guild | Tune PostgreSQL configuration |
| 23 | PG-T7.3.7 | DONE | Prometheus/Grafana monitoring documented | Observability Guild | Set up query monitoring dashboard |
| 24 | PG-T7.3.8 | DONE | Baselines documented | DBA Guild | Document performance baselines |
### T7.4: Update Documentation
| # | Task ID | Status | Key dependency / next step | Owners | Task Definition |
| --- | --- | --- | --- | --- | --- |
| 25 | PG-T7.4.1 | DONE | PostgreSQL is now primary DB in architecture doc | Docs Guild | Update `docs/07_HIGH_LEVEL_ARCHITECTURE.md` |
| 26 | PG-T7.4.2 | DONE | Schema ownership table added | Docs Guild | Update module architecture docs |
| 27 | PG-T7.4.3 | DONE | Compose files updated with PG init scripts | Docs Guild | Update deployment guides |
| 28 | PG-T7.4.4 | DONE | postgresql-guide.md created | Docs Guild | Update operations runbooks |
| 29 | PG-T7.4.5 | DONE | Troubleshooting in postgresql-guide.md | Docs Guild | Update troubleshooting guides |
| 30 | PG-T7.4.6 | DONE | Technology stack now lists PostgreSQL | Docs Guild | Update `CLAUDE.md` technology stack |
| 31 | PG-T7.4.7 | DONE | Created comprehensive postgresql-guide.md | Docs Guild | Create `docs/operations/postgresql-guide.md` |
| 32 | PG-T7.4.8 | DONE | Backup/restore in postgresql-guide.md | Docs Guild | Document backup/restore procedures |
| 33 | PG-T7.4.9 | DONE | Scaling recommendations in guide | Docs Guild | Document scaling recommendations |
### T7.5: Update Air-Gap Kit
| # | Task ID | Status | Key dependency / next step | Owners | Task Definition |
| --- | --- | --- | --- | --- | --- |
| 34 | PG-T7.5.1 | DONE | PostgreSQL 17 in docker-compose.airgap.yaml | DevOps Guild | Add PostgreSQL container image to kit |
| 35 | PG-T7.5.2 | DONE | postgres-init scripts added | DevOps Guild | Update kit scripts for PostgreSQL setup |
| 36 | PG-T7.5.3 | DONE | 01-extensions.sql creates schemas | DevOps Guild | Include schema migrations in kit |
| 37 | PG-T7.5.4 | DONE | docs/24_OFFLINE_KIT.md updated | DevOps Guild | Update kit documentation |
| 38 | PG-T7.5.5 | TODO | Awaiting air-gap environment test | DevOps Guild | Test kit installation in air-gapped environment |
## Execution Log
| Date (UTC) | Update | Owner |
| --- | --- | --- |
| 2025-12-10 | Completed Waves C, D, E: created comprehensive `docs/operations/postgresql-guide.md` (performance, monitoring, backup/restore, scaling), updated HIGH_LEVEL_ARCHITECTURE.md to PostgreSQL-primary, updated CLAUDE.md technology stack, added PostgreSQL 17 with pg_stat_statements to docker-compose.airgap.yaml, created postgres-init scripts for both local-postgres and airgap compose, updated offline kit docs. Only PG-T7.5.5 (air-gap environment test) remains TODO. Wave B dropped (no data to migrate - ground zero). | Infrastructure Guild |
| 2025-12-07 | Unblocked PG-T7.1.2T7.1.6 with plan at `docs/db/reports/mongo-removal-plan-20251207.md`; statuses set to TODO. | Project Mgmt |
| 2025-12-03 | Added Wave Coordination (A code removal, B archive, C performance, D docs, E air-gap kit; sequential). No status changes. | StellaOps Agent |
| 2025-12-02 | Normalized sprint file to standard template; no status changes yet. | StellaOps Agent |
| 2025-12-06 | Wave A kickoff: PG-T7.1.1 set to DOING; confirming module cutovers done; prep removal checklist and impact scan. | Project Mgmt |
| 2025-12-06 | Inventory complete: Authority Mongo project already absent → PG-T7.1.1 marked DONE. Remaining Mongo artefacts located (Scheduler tests only; Notify/Concelier libraries+tests; Policy Engine Mongo storage; Excititor tests; shared Provenance.Mongo). PG-T7.1.2 set to DOING to start Scheduler cleanup; plan is sequential removal per T7.1.x. | Project Mgmt |
| 2025-12-06 | PG-T7.1.2 set BLOCKED: Scheduler WebService/Worker/Backfill still reference Storage.Mongo types; need removal/replace plan (e.g., swap to Postgres repos or drop code paths) plus solution cleanup. Added BLOCKED note; proceed to next unblocked Wave A items after decision. | Project Mgmt |
| 2025-12-06 | PG-T7.1.3 set BLOCKED: Notify Mongo library + tests still present; need decision to delete or retain for import/backfill tooling before removal. | Project Mgmt |
| 2025-12-06 | PG-T7.1.4–T7.1.6 set BLOCKED pending module approvals to delete Mongo storage/projects (Policy, Concelier, Excititor). Need confirmation no import/backfill tooling relies on them before removal. | Project Mgmt |
| 2025-12-06 | Added decision tasks PG-T7.1.D1–D5 to collect module approvals for Mongo deletions; owners assigned per module guilds. | Project Mgmt |
| 2025-12-06 | Added PG-T7.1.D6 to prepare impact/rollback one-pagers per module to speed approvals and deletions. | Project Mgmt |
| 2025-12-06 | Decisions captured in `docs/db/reports/mongo-removal-decisions-20251206.md`; during initial deletion attempt found extensive Concelier Mongo dependencies (connectors/tests). Reverted to avoid breaking build; PG-T7.1.2–T7.1.6 set back to BLOCKED pending phased refactor plan (PG-T7.1.PLAN). | Project Mgmt |
| 2025-12-06 | Published `docs/db/reports/scheduler-graphjobs-postgres-plan.md` defining schema/repo/DI/test steps; PG-T7.1.2a unblocked to TODO. | Scheduler Guild |
| 2025-12-06 | Started implementing PG-T7.1.2a: added Postgres graph job migration (002), repository + DI registration, PostgresGraphJobStore, and switched WebService/Worker to Postgres storage references. Tests not yet updated; Mongo code remains for backfill/tests. | Scheduler Guild |
| 2025-12-06 | PG-T7.1.2a set BLOCKED: no Postgres graph-job schema/repository exists; need design guidance (tables for graph_jobs, overlays, status) or decision to reuse existing run tables. | Project Mgmt |
| 2025-12-06 | Concelier Mongo drop started: removed MongoDB package refs from Concelier Core/Connector.Common/RawModels; added Postgres compat types (IDocumentStore/ObjectId/DocumentStatuses), in-memory RawDocumentStorage, and DI wiring; new Concelier task bundle PG-T7.1.5a–e added. | Concelier Guild |
| 2025-12-06 | Scheduler solution cleanup: removed stale solution GUIDs, fixed Worker.Host references, rewired Backfill to Postgres data source, and added SurfaceManifestPointer inline to Scheduler.Queue to drop circular deps. Build now blocked by missing Postgres run/schedule/policy repositories in Worker. | Scheduler Guild |
| 2025-12-06 | Attempted Scheduler Postgres tests; restore/build fails because `StellaOps.Concelier.Storage.Mongo` project is absent and Concelier connectors reference it. Need phased Concelier plan/shim to unblock test/build runs. | Scheduler Guild |
| 2025-12-06 | Began Concelier Mongo compatibility shim: added `FindAsync` to in-memory `IDocumentStore` in Postgres compat layer to unblock connector compile; full Mongo removal still pending. | Infrastructure Guild |
| 2025-12-06 | Added lightweight `StellaOps.Concelier.Storage.Mongo` in-memory stub (advisory/dto/document/state/export stores) to unblock Concelier connector build while Postgres rewiring continues; no Mongo driver/runtime. | Infrastructure Guild |
| 2025-12-06 | PG-T7.1.5b set to DOING; began wiring Postgres document store (DI registration, repository find) to replace Mongo bindings. | Concelier Guild |
| 2025-12-06 | Concelier shim extended: MongoCompat now carries merge events/alias constants; Postgres storage DI uses PostgresDocumentStore; Source repository lookup fixed; Merge + Storage.Postgres projects now build. Full solution still hits pre-existing NU1608 version conflicts in crypto plugins (out of Concelier scope). | Concelier Guild |
| 2025-12-07 | Concelier Postgres store now also implements legacy `IAdvisoryStore` and is registered as such; DI updated. Added repo-wide restore fallback suppression to unblock Postgres storage build (plugin/provenance now restore without VS fallback path). Storage.Postgres builds clean; remaining full-solution build blockers are crypto NU1608 version constraints (out of scope here). | Concelier Guild |
| 2025-12-07 | Postgres raw/state wiring: RawDocumentStorage now scoped with DocumentStore fallback, connectors/exporters persist payload bytes with GUID payload IDs, Postgres source-state adapter registered, and DualWrite advisory store now Postgres-only. Full WebService build still red on result-type aliases and legacy Mongo bootstrap hooks; follow-up needed before PG-T7.1.5b can close. | Concelier Guild |
| 2025-12-07 | NuGet cache reset and restore retry: cleared locals into `.nuget/packages.clean`, restored Concelier solution with fallback disabled, and reran build. Restore now clean; build failing on Mongo shim namespace ambiguity (Documents/Dtos aliases), missing WebService result wrapper types, and remaining Mongo bootstrap hooks. | Concelier Guild |
| 2025-12-07 | Cached Microsoft.Extensions.* 10.0.0 packages locally and refactored WebService result aliases/Mongo bootstrap bypass; `StellaOps.Concelier.WebService` now builds green against Postgres-only DI. | Concelier Guild |
| 2025-12-07 | Full `StellaOps.Concelier.sln` build still red: MongoCompat `DocumentStatuses` conflicts with Connector.Common, compat Bson stubs lack BinaryData/Elements/GetValue/IsBsonNull, `DtoRecord` fields immutable, JpFlag store types missing, and Concelier.Testing + SourceState tests still depend on Mongo driver/AddMongoStorage. PG-T7.1.5c remains TODO pending compat shim or Postgres fixture migration. | Concelier Guild |
| 2025-12-08 | Converted MongoIntegrationFixture to in-memory/stubbed client + stateful driver stubs so tests no longer depend on Mongo2Go; PG-T7.1.5c progressing. Concelier build attempt still blocked upstream by missing NuGet cache entries (Microsoft.Extensions.* 10.0.0, Blake3, SharpCompress) requiring cache rehydrate/local feed. | Concelier Guild |
| 2025-12-08 | Rehydrated NuGet cache (fallback disabled) and restored Concelier solution; cache issues resolved. Build now blocked in unrelated crypto DI project (`StellaOps.Cryptography.DependencyInjection` missing `StellaOps.Cryptography.Plugin.SmRemote`) rather than Mongo. Concelier shim now in-memory; PG-T7.1.5c continues. | Concelier Guild |
| 2025-12-08 | Rebuilt Concelier solution after cache restore; Mongo shims no longer pull Mongo2Go/driver, but overall build still fails on cross-module crypto gap (`SmRemote` plugin missing). No remaining Mongo package/runtime dependencies in Concelier build. | Concelier Guild |
| 2025-12-08 | Dropped the last MongoDB.Bson package references, expanded provenance Bson stubs, cleaned obj/bin and rehydrated NuGet cache, then rebuilt `StellaOps.Concelier.sln` successfully with Postgres-only DI. PG-T7.1.5a/5b marked DONE; PG-T7.1.5c continues for Postgres runtime parity and migrations. | Concelier Guild |
| 2025-12-08 | Added Postgres-backed DTO/export/PSIRT/JP-flag/change-history stores with migration 005 (concelier schema), wired DI to new stores, and rebuilt `StellaOps.Concelier.sln` green Postgres-only. PG-T7.1.5c/5d/5e marked DONE. | Concelier Guild |
| 2025-12-09 | Mirrored Wave A action/risk into parent sprint; added PG-T7.1.5f (TODO) to remove MongoCompat shim post-parity sweep and ensure migration 005 stays in the kit. | Project Mgmt |
| 2025-12-09 | PG-T7.1.5f set BLOCKED: MongoCompat/Bson interfaces are still the canonical storage contracts across connectors/tests; need design to introduce Postgres-native abstractions and parity evidence before deleting shim. | Project Mgmt |
| 2025-12-09 | Investigated MongoCompat usage: connectors/tests depend on IDocumentStore, IDtoStore (Bson payloads), ISourceStateRepository (Bson cursors), advisory/alias/change-history/export state stores, and DualWrite/DIOptions; Postgres stores implement Mongo contracts today. Need new storage contracts (JSON/byte payloads, cursor DTO) and adapter layer to retire Mongo namespaces. | Project Mgmt |
| 2025-12-09 | Started PG-T7.1.5f implementation: added Postgres-native storage contracts (document/dto/source state) and adapters in Postgres stores to implement both new contracts and legacy Mongo interfaces; connectors/tests still need migration off MongoCompat/Bson. | Project Mgmt |
| 2025-12-09 | PG-T7.1.5f in progress: contract/adapters added; started migrating Common SourceFetchService to Storage.Contracts with backward-compatible constructor. Connector/test surface still large; staged migration plan required. | Project Mgmt |
| 2025-12-10 | Wave A cleanup sweep: verified all DONE tasks, deleted stale bin/obj folders (Authority/Scheduler/Concelier/Excititor Mongo), deleted Notify Storage.Mongo lib+tests folders and updated solution, deleted Policy Engine Storage/Mongo folder and removed dead `using` statement, updated sprint statuses to reflect completed work. Build blocked by NuGet network issues (not code issues). | Infrastructure Guild |
| 2025-12-10 | Wave A completion: cleaned 7 solution files (Authority×2, AdvisoryAI, Policy×2, Notifier, SbomService) removing Storage.Mongo project entries and build configs; fixed csproj references in Authority (Authority, Plugin.Ldap, Plugin.Ldap.Tests, Plugin.Standard) and Notifier (Worker, WebService) to use Postgres storage. All Storage.Mongo csproj references now removed. PG-T7.1.7-10 marked DONE. MongoDB usage in TaskRunner/IssuerDirectory/AirGap/Attestor deferred to later phases. | Infrastructure Guild |
| 2025-12-10 | **CRITICAL AUDIT:** Comprehensive grep revealed ~680 MongoDB occurrences across 200+ files remain. Sprint archival was premature. Key findings: (1) Authority/Notifier code uses deleted `Storage.Mongo` namespaces - BUILDS BROKEN; (2) 20 csproj files still have MongoDB.Driver/Bson refs; (3) 10+ modules have ONLY MongoDB impl with no Postgres equivalent. Created `SPRINT_3410_0001_0001_mongodb_final_removal.md` to track remaining work. Full MongoDB removal is multi-sprint effort, not cleanup. | Infrastructure Guild |
## Decisions & Risks
- Concelier PG-T7.1.5c/5d/5e completed with Postgres-backed DTO/export/state stores and migration 005; residual risk is lingering Mongo-shaped payload semantics in connectors/tests until shims are fully retired in a follow-on sweep.
- Cleanup is strictly after all phases complete; do not start T7 tasks until module cutovers are DONE.
- Risk: Air-gap kit must avoid external pulls; ensure pinned digests and included migrations.
- Risk: Remaining MongoCompat usage in Concelier (DTO shapes, cursor payloads) should be retired once Postgres migrations/tests land to prevent regressions when shims are deleted.
- Risk: MongoCompat shim removal pending (PG-T7.1.5f / ACT-3407-A1); PG-T7.1.5f in progress with Postgres-native storage contracts added, but connectors/tests still depend on MongoCompat/Bson types. Parity sweep and connector migration needed before deleting the shim; keep migration 005 in the air-gap kit.
- BLOCKER: Scheduler: Postgres equivalent for GraphJobStore/PolicyRunService not designed; need schema/contract decision to proceed with PG-T7.1.2a and related deletions.
- BLOCKER: Scheduler Worker still depends on Mongo-era repositories (run/schedule/impact/policy); Postgres counterparts are missing, keeping solution/tests red until implemented or shims added.
- BLOCKER: Scheduler/Notify/Policy/Excititor Mongo removals must align with the phased plan; delete only after replacements are in place.
## Appendix A · Mongo→Postgres Removal Plan (PG-T7.1.PLAN)
1) Safety guardrails
- No deletions until each module has a passing Postgres-only build and import path; keep build green between steps.
- Use feature flags: `Persistence:<Module>=Postgres` already on; add `AllowMongoFallback=false` checkers to fail fast if code still tries Mongo.
2) Order of execution
1. Scheduler: swap remaining Mongo repositories in WebService/Worker/Backfill to Postgres equivalents; drop Mongo harness; then delete project + solution refs.
2. Notify: remove Mongo import/backfill helpers; ensure all tests use Postgres fixtures; delete Mongo lib/tests.
3. Policy: delete Storage/Mongo folder; confirm no dual-write remains.
4. Concelier (largest):
- Phase C1: restore Mongo lib temporarily, add compile-time shim that throws if instantiated; refactor connectors/importers/exporters to Postgres repositories.
- Phase C2: migrate Concelier.Testing fixtures to Postgres; update dual-import parity tests to Postgres-only.
- Phase C3: remove Mongo lib/tests and solution refs; clean AGENTS/docs to drop Mongo instructions.
5. Excititor: remove Mongo test harness once Concelier parity feeds Postgres graphs; ensure VEX graph tests green.
3) Work items to add per module
- Replace `using ...Storage.Mongo` with Postgres equivalents; remove ProjectReference from csproj.
- Update fixtures to Postgres integration fixture; remove Mongo-specific helpers.
- Delete dual-write or conversion helpers that depended on Mongo.
- Update AGENTS and TASKS docs to mark Postgres-only.
4) Rollback
- If a step breaks CI, revert the module-specific commit; Mongo projects are still in git history.
5) Evidence tracking
- Record each module deletion in Execution Log with test runs (dotnet test filters per module) and updated solution diff.
## Next Checkpoints
- 2025-12-07: Circulate decision packets PG-T7.1.D1–D6 to module owners; log approvals/objections in Execution Log.
- 2025-12-08: If approvals received, delete first approved Mongo project(s), update solution (PG-T7.1.7), and rerun build; if not, escalate decisions in Decisions & Risks.
- 2025-12-10: If at least two modules cleared, schedule Wave B backup window; otherwise publish status note and revised ETA.