up
Some checks failed
Docs CI / lint-and-preview (push) Has been cancelled
Findings Ledger CI / build-test (push) Has been cancelled
Findings Ledger CI / migration-validation (push) Has been cancelled
Scanner Analyzers / Discover Analyzers (push) Has been cancelled
Signals Reachability Scoring & Events / reachability-smoke (push) Has been cancelled
AOC Guard CI / aoc-guard (push) Has been cancelled
Concelier Attestation Tests / attestation-tests (push) Has been cancelled
cryptopro-linux-csp / build-and-test (push) Has been cancelled
Scanner Analyzers / Validate Test Fixtures (push) Has been cancelled
Signals CI & Image / signals-ci (push) Has been cancelled
sm-remote-ci / build-and-test (push) Has been cancelled
Findings Ledger CI / generate-manifest (push) Has been cancelled
AOC Guard CI / aoc-verify (push) Has been cancelled
Scanner Analyzers / Build Analyzers (push) Has been cancelled
Scanner Analyzers / Test Language Analyzers (push) Has been cancelled
Scanner Analyzers / Verify Deterministic Output (push) Has been cancelled
Signals Reachability Scoring & Events / sign-and-upload (push) Has been cancelled

This commit is contained in:
StellaOps Bot
2025-12-09 09:38:09 +02:00
parent bc0762e97d
commit 108d1c64b3
193 changed files with 7265 additions and 13029 deletions

View File

@@ -57,6 +57,10 @@
## Execution Log
| Date (UTC) | Update | Owner |
| --- | --- | --- |
| 2025-12-09 | Purged remaining Mongo session handles from Excititor connector/web/export/worker tests; stubs now align to Postgres/in-memory contracts. | Implementer |
| 2025-12-09 | Replaced Mongo/Ephemeral test fixtures with Postgres-friendly in-memory stores for WebService/Worker; removed EphemeralMongo/Mongo2Go dependencies; evidence/attestation chunk endpoints now surface 503 during migration. | Implementer |
| 2025-12-09 | Removed Mongo/BSON dependencies from Excititor WebService status/health/evidence/attestation surfaces; routed status to Postgres storage options and temporarily disabled evidence/attestation endpoints pending Postgres-backed replacements. | Implementer |
| 2025-12-09 | Deleted legacy Storage.Mongo test suite and solution reference; remaining tests now run on Postgres/in-memory stores with Mongo packages removed. | Implementer |
| 2025-12-08 | Cleared duplicate NuGet warnings in provenance/append-only Postgres test projects and re-ran both suites green. | Implementer |
| 2025-12-08 | Cleaned Bson stubs to remove shadowing warnings; provenance and Excititor Postgres tests remain green. | Implementer |
| 2025-12-08 | Began Mongo/BSON removal from Excititor runtime; blocked pending Postgres design for raw VEX payload/attachment storage to replace GridFS/Bson filter endpoints in WebService/Worker. | Implementer |
@@ -79,6 +83,7 @@
| Orchestrator SDK version selection | Decision | Excititor Worker Guild | 2025-12-12 | Needed for task 8. |
| Excititor.Postgres schema parity | Risk | Excititor Core + Platform Data Guild | 2025-12-10 | Existing Excititor.Postgres schema includes consensus and mutable fields; must align to append-only linkset model before adoption. |
| Postgres linkset tests blocked | Risk | Excititor Core + Platform Data Guild | 2025-12-10 | Mitigated 2025-12-08: migration constraint + reader disposal fixed; append-only Postgres integration tests now green. |
| Evidence/attestation endpoints paused | Risk | Excititor Core | 2025-12-12 | Evidence and attestation list/detail endpoints return 503 while Mongo/BSON paths are removed; needs Postgres-backed replacement before release. |
## Next Checkpoints
| Date (UTC) | Session | Goal | Owner(s) |

View File

@@ -37,7 +37,7 @@
| 1 | SCANNER-ANALYZERS-DENO-26-009 | DONE (2025-11-24) | Runtime trace shim + AnalysisStore runtime payload implemented; Deno runtime tests passing. | Deno Analyzer Guild · Signals Guild | Optional runtime evidence hooks capturing module loads and permissions with path hashing during harnessed execution. |
| 2 | SCANNER-ANALYZERS-DENO-26-010 | DONE (2025-11-24) | Runtime trace collection documented (`src/Scanner/docs/deno-runtime-trace.md`); analyzer auto-runs when `STELLA_DENO_ENTRYPOINT` is set. | Deno Analyzer Guild · DevOps Guild | Package analyzer plug-in and surface CLI/worker commands with offline documentation. |
| 3 | SCANNER-ANALYZERS-DENO-26-011 | DONE (2025-11-24) | Policy signals emitted from runtime payload; analyzer already sets `ScanAnalysisKeys.DenoRuntimePayload` and emits metadata. | Deno Analyzer Guild | Policy signal emitter for capabilities (net/fs/env/ffi/process/crypto), remote origins, npm usage, wasm modules, and dynamic-import warnings. |
| 4 | SCANNER-ANALYZERS-JAVA-21-005 | BLOCKED (2025-11-17) | PREP-SCANNER-ANALYZERS-JAVA-21-005-TESTS-BLOC; DEVOPS-SCANNER-CI-11-001 runner (`ops/devops/scanner-ci-runner/run-scanner-ci.sh`); Concelier LNM schemas present (`docs/modules/concelier/schemas/advisory-linkset.schema.json`, `advisory-observation.schema.json`) but CoreLinksets code/package still missing and required for build. | Java Analyzer Guild | Framework config extraction: Spring Boot imports, spring.factories, application properties/yaml, Jakarta web.xml/fragments, JAX-RS/JPA/CDI/JAXB configs, logging files, Graal native-image configs. |
| 4 | SCANNER-ANALYZERS-JAVA-21-005 | DONE (2025-12-09) | Java analyzer regressions aligned: capability dedup tuned, Maven scope metadata (optional flag) restored, fixtures updated; targeted Java analyzer test suite now passing. | Java Analyzer Guild | Framework config extraction: Spring Boot imports, spring.factories, application properties/yaml, Jakarta web.xml/fragments, JAX-RS/JPA/CDI/JAXB configs, logging files, Graal native-image configs. |
| 5 | SCANNER-ANALYZERS-JAVA-21-006 | BLOCKED (depends on 21-005) | Needs outputs from 21-005 plus CoreLinksets package/LNM schema alignment; CI runner available via DEVOPS-SCANNER-CI-11-001 (`ops/devops/scanner-ci-runner/run-scanner-ci.sh`). | Java Analyzer Guild | JNI/native hint scanner detecting native methods, System.load/Library literals, bundled native libs, Graal JNI configs; emit `jni-load` edges. |
| 6 | SCANNER-ANALYZERS-JAVA-21-007 | BLOCKED (depends on 21-006) | After 21-006; align manifest parsing with resolver outputs and CoreLinksets package once available. | Java Analyzer Guild | Signature and manifest metadata collector capturing JAR signature structure, signers, and manifest loader attributes (Main-Class, Agent-Class, Start-Class, Class-Path). |
| 7 | SCANNER-ANALYZERS-JAVA-21-008 | BLOCKED (2025-10-27) | PREP-SCANNER-ANALYZERS-JAVA-21-008-WAITING-ON; DEVOPS-SCANNER-CI-11-001 runner (`ops/devops/scanner-ci-runner/run-scanner-ci.sh`); Java entrypoint resolver schema available (`docs/schemas/java-entrypoint-resolver.schema.json`); waiting on CoreLinksets package and upstream 21-005..21-007 outputs. | Java Analyzer Guild | Implement resolver + AOC writer emitting entrypoints, components, and edges (jpms, cp, spi, reflect, jni) with reason codes and confidence. |
@@ -50,6 +50,9 @@
## Execution Log
| Date (UTC) | Update | Owner |
| --- | --- | --- |
| 2025-12-09 | Located Core linkset docs/contracts: schema + samples (`docs/modules/concelier/link-not-merge-schema.md`, `docs/modules/concelier/schemas/*.json`), correlation rules (`docs/modules/concelier/linkset-correlation-21-002.md`), event shape (`docs/modules/concelier/events/advisory.linkset.updated@1.md`), and core library code at `src/Concelier/__Libraries/StellaOps.Concelier.Core/Linksets`. Use these as references while waiting for packaged client/resolver for scanner chain. | Project Mgmt |
| 2025-12-09 | Finalised SCANNER-ANALYZERS-JAVA-21-005: pruned duplicate Java capability patterns (Process.start), restored Maven scope optional metadata via lock entry propagation, refreshed fixtures, and verified `dotnet test src/Scanner/__Tests/StellaOps.Scanner.Analyzers.Lang.Java.Tests/StellaOps.Scanner.Analyzers.Lang.Java.Tests.csproj -c Release` passing. | Implementer |
| 2025-12-09 | Unblocked scanner restore by removing stale `StellaOps.Concelier.Storage.Mongo` from the solution, switching BuildX Surface.Env to project reference, and adding stub `StellaOps.Cryptography.Plugin.WineCsp` + `Microsoft.Extensions.Http` to satisfy crypto DI after upstream removal. Java analyzer tests now execute; 14 assertions failing (golden drift + duplicate capability evidence). | Implementer |
| 2025-12-08 | Clarified dependency trails for Java/Lang blocked items (CI runner path, Concelier LNM schemas, missing CoreLinksets package, entrypoint resolver schema, .NET IL schema); no status changes. | Project Mgmt |
| 2025-12-08 | Removed temporary Storage.Mongo project; restored Mongo stubs to `StellaOps.Concelier.Models/MongoCompat` and kept Concelier builds Postgres-only. Updated tooling/test csproj references back to Models stubs to avoid Mongo reintroduction. | Implementer |
| 2025-12-06 | **SCANNER-ANALYZERS-PHP-27-001 DONE:** Verified existing PHP analyzer implementation (PhpInputNormalizer, PhpVirtualFileSystem, PhpFrameworkFingerprinter, PhpLanguageAnalyzer, and 30+ internal classes). Build passing. Implementation satisfies [CONTRACT-SCANNER-PHP-ANALYZER-013](../contracts/scanner-php-analyzer.md) requirements. Wave D complete. | Implementer |
@@ -96,9 +99,11 @@
- Scanner record payload schema still unpinned; drafting prep at `docs/modules/scanner/prep/2025-11-21-scanner-records-prep.md` while waiting for analyzer output confirmation from Scanner Guild.
- `SCANNER-ANALYZERS-LANG-11-001` blocked (2025-11-17): local `dotnet test` hangs/returns empty output; requires clean runner/CI hang diagnostics to progress and regenerate goldens.
- Additional note: dotnet-filter wrapper avoids `workdir:` injection but full solution builds still stall locally; recommend CI/clean runner and/or scoped project tests to gather logs for LANG-11-001.
- Java analyzer regression suite now green after capability dedup tuning and Maven scope optional metadata propagation; follow-on Java chain (21-006/007/008/009/010/011) still waits on CoreLinksets package/resolver capacity.
- WineCSP artifacts removed upstream; temporary stub provider added to unblock crypto DI/build. Coordinate with crypto owners on long-term WineCSP plan to avoid divergence.
- `SCANNER-ANALYZERS-JAVA-21-008` blocked (2025-10-27): resolver capacity needed to produce entrypoint/component/edge outputs; downstream tasks remain stalled until resolved.
- Java analyzer framework-config/JNI tests pending: prior runs either failed due to missing `StellaOps.Concelier.Storage.Mongo` `CoreLinksets` types or were aborted due to repo-wide restore contention; rerun on clean runner or after Concelier build stabilises.
- Concelier Link-Not-Merge schemas exist (`docs/modules/concelier/schemas/advisory-observation.schema.json`, `advisory-linkset.schema.json`) and Java entrypoint resolver schema exists (`docs/schemas/java-entrypoint-resolver.schema.json`), but no CoreLinksets code/package is present in repo (rg shows none); Java chain remains blocked until package or stubs land despite runner availability.
- Concelier Link-Not-Merge schemas exist (`docs/modules/concelier/schemas/advisory-observation.schema.json`, `advisory-linkset.schema.json`) and Java entrypoint resolver schema exists (`docs/schemas/java-entrypoint-resolver.schema.json`). Core linkset contracts live under `src/Concelier/__Libraries/StellaOps.Concelier.Core/Linksets` with correlation/event docs (`docs/modules/concelier/linkset-correlation-21-002.md`, `docs/modules/concelier/events/advisory.linkset.updated@1.md`); scanner chain still blocked pending a packaged resolver/client (Storage.Mongo removed) or explicit dependency guidance.
- `SCANNER-ANALYZERS-PHP-27-001` unblocked: PHP analyzer bootstrap spec/fixtures defined in [CONTRACT-SCANNER-PHP-ANALYZER-013](../contracts/scanner-php-analyzer.md); composer/VFS schema and offline kit target available.
- Deno runtime hook + policy-signal schema drafted in `docs/modules/scanner/design/deno-runtime-signals.md`; shim plan in `docs/modules/scanner/design/deno-runtime-shim.md`.
- Deno runtime shim now emits module/permission/wasm/npm events; needs end-to-end validation on a Deno runner (cached-only) to confirm module loader hook coverage before wiring DENO-26-010/011.

View File

@@ -41,6 +41,7 @@
## Execution Log
| Date (UTC) | Update | Owner |
| --- | --- | --- |
| 2025-12-10 | Router transport wired for `signals.fact.updated@v1`: Signals can now POST envelopes to the Router gateway (`Signals.Events.Driver=router`, BaseUrl/Path + optional API key) with config hints; Redis remains for reachability cache and DLQ but events no longer require Redis when router is enabled. | Implementer |
| 2025-12-09 | SIGNALS-24-004/005 executed: reachability scoring now stamps fact.version + deterministic digests and emits Redis stream events (`signals.fact.updated.v1`/DLQ) with envelopes aligned to `events-24-005.md`; CI workflows (`signals-reachability.yml`, `signals-evidence-locker.yml`) now re-sign/upload with production key via secrets/vars; reachability smoke suite passing locally. | Implementer |
| 2025-12-08 | 140.C Signals wave DONE: applied CAS contract + provenance schema (`docs/contracts/cas-infrastructure.md`, `docs/signals/provenance-24-003.md`, `docs/schemas/provenance-feed.schema.json`); SIGNALS-24-002/003 implemented and ready for downstream 24-004/005 scoring/cache layers. | Implementer |
| 2025-12-06 | **140.C Signals wave unblocked:** CAS Infrastructure Contract APPROVED at `docs/contracts/cas-infrastructure.md`; Provenance appendix published at `docs/signals/provenance-24-003.md` + schema at `docs/schemas/provenance-feed.schema.json`. SIGNALS-24-002/003 moved from BLOCKED to TODO. | Implementer |
@@ -111,7 +112,7 @@
- CARTO-GRAPH-21-002 inspector contract now published at `docs/modules/graph/contracts/graph.inspect.v1.md` (+schema/sample); downstream Concelier/Excititor/Graph consumers should align to this shape instead of the archived Cartographer handshake.
- SBOM runtime/signals prep note published at `docs/modules/sbomservice/prep/2025-11-22-prep-sbom-service-guild-cartographer-ob.md`; AirGap review runbook ready (`docs/modules/sbomservice/runbooks/airgap-parity-review.md`). Wave moves to TODO pending review completion and fixture hash upload.
- Cosign v3.0.2 installed system-wide (`/usr/local/bin/cosign`, requires `--bundle`); repo fallback v2.6.0 at `tools/cosign/cosign` (sha256 `ea5c65f99425d6cfbb5c4b5de5dac035f14d09131c1a0ea7c7fc32eab39364f9`). Production re-sign/upload now automated via `signals-reachability.yml` and `signals-evidence-locker.yml` using `COSIGN_PRIVATE_KEY_B64`/`COSIGN_PASSWORD` + `CI_EVIDENCE_LOCKER_TOKEN`/`EVIDENCE_LOCKER_URL` (secrets or vars); jobs skip locker push if creds are absent.
- Redis Stream publisher emits `signals.fact.updated.v1` envelopes (event_id, fact_version, fact.digest) aligned with `docs/signals/events-24-005.md`; DLQ stream `signals.fact.updated.dlq` enabled.
- Redis Stream publisher emits `signals.fact.updated.v1` envelopes (event_id, fact_version, fact.digest) aligned with `docs/signals/events-24-005.md`; DLQ stream `signals.fact.updated.dlq` enabled. Router transport is now available (`Signals.Events.Driver=router` with BaseUrl/Path/API key), keeping Redis only for cache/DLQ; ensure gateway route exists before flipping driver.
- Surface.FS cache drop timeline (overdue) and Surface.Env owner assignment keep Zastava env/secret/admission tasks blocked.
- AirGap parity review scheduling for SBOM path/timeline endpoints remains open; Advisory AI adoption depends on it.

View File

@@ -41,6 +41,7 @@
## Execution Log
| Date (UTC) | Update | Owner |
| --- | --- | --- |
| 2025-12-10 | Router-backed publisher added: `Signals.Events.Driver=router` now POSTs `signals.fact.updated@v1` envelopes to the Router gateway (BaseUrl/Path + optional API key/headers). Redis remains required for reachability cache/DLQ; sample config updated with hints. | Implementer |
| 2025-12-09 | SIGNALS-24-004/005 hardened: deterministic fact.version/digest hasher, Redis stream events (signals.fact.updated.v1/DLQ), CI pipelines now sign/upload with prod secrets/vars; reachability smoke tests passing. | Implementer |
| 2025-12-08 | Cleared locked `Microsoft.SourceLink.GitLab.dll.bak` from repo-scoped `.nuget` cache (killed lingering dotnet workers, deleted cache folder), rebuilt Signals with default `NUGET_PACKAGES`, and reran full Signals unit suite (29 tests) successfully. Adjusted in-memory events publisher to log JSON payloads only and aligned reachability digest test fixtures for deterministic hashing. | Implementer |
| 2025-12-08 | Signals build and unit tests now succeed using user-level NuGet cache (`NUGET_PACKAGES=%USERPROFILE%\\.nuget\\packages`) to bypass locked repo cache file. Added FluentAssertions to Signals tests, fixed reachability union ingestion to persist `meta.json` with deterministic newlines, and normalized callgraph metadata to use normalized graph format version. | Implementer |
@@ -94,7 +95,8 @@
- Redis stream publisher (signals.fact.updated.v1 + DLQ) implements the docs/signals/events-24-005.md contract; ensure DLQ monitoring in CI/staging.
- Production re-sign/upload automated via signals-reachability.yml and signals-evidence-locker.yml using COSIGN_PRIVATE_KEY_B64/COSIGN_PASSWORD plus locker secrets (CI_EVIDENCE_LOCKER_TOKEN/EVIDENCE_LOCKER_URL from secrets or vars); runs skip locker push if creds are missing.
- Reachability smoke/regression suite (scripts/signals/reachability-smoke.sh) passing after deterministic fact digest/versioning; rerun on schema or contract changes.
- Repo `.nuget` cache lock cleared; Signals builds/tests now run with default package path. Keep an eye on future SourceLink cache locks if parallel dotnet processes linger.
- Router transport now wired for Signals events (`Signals.Events.Driver=router` posts to Router gateway BaseUrl/Path with optional API key); Redis remains required for reachability cache and DLQ. Ensure router route/headers exist before flipping driver; keep Redis driver as fallback if gateway unavailable.
- Repo `.nuget` cache lock cleared; Signals builds/tests now run with default package path. Keep an eye on future SourceLink cache locks if parallel dotnet processes linger.
## Next Checkpoints
- 2025-12-10 · First CI run of signals-reachability.yml with production secrets/vars to re-sign and upload evidence.

View File

@@ -21,19 +21,19 @@
## Delivery Tracker
| # | Task ID | Status | Key dependency / next step | Owners | Task Definition |
| --- | --- | --- | --- | --- | --- |
| 1 | SCAN-JAVA-VAL-0146-01 | TODO | Allocate clean runner; rerun Java analyzer suite and attach TRX/binlogs; update readiness to Green if passing. | Scanner · CI | Validate Java analyzer chain (21-005..011) on clean runner and publish evidence. |
| 2 | SCAN-DOTNET-DESIGN-0146-02 | TODO | Finalize analyzer design 11-001; create fixtures/tests; CI run. | Scanner · CI | Unblock .NET analyzer chain (11-001..005) with design doc, fixtures, and passing CI evidence. |
| 3 | SCAN-PHP-DESIGN-0146-03 | TODO | Composer/autoload spec + restore stability; new fixtures. | Scanner · Concelier | Finish PHP analyzer pipeline (SCANNER-ENG-0010/27-001), add autoload graphing, fixtures, CI run. |
| 4 | SCAN-NODE-PH22-CI-0146-04 | TODO | Clean runner with trimmed graph; run `scripts/run-node-phase22-smoke.sh`; capture logs. | Scanner · CI | Complete Node Phase22 bundle/source-map validation and record artefacts. |
| 5 | SCAN-DENO-STATUS-0146-05 | TODO | Reconcile readiness vs TASKS.md; add validation evidence if shipped. | Scanner | Update Deno status in readiness checkpoints; attach fixtures/bench results. |
| 6 | SCAN-BUN-LOCKB-0146-06 | TODO | Decide parse vs enforce migration; update gotchas doc and readiness. | Scanner | Define bun.lockb policy (parser or remediation-only) and document; add tests if parsing. |
| 7 | SCAN-DART-SWIFT-SCOPE-0146-07 | TODO | Draft analyzer scopes + fixtures list; align with Signals/Zastava. | Scanner | Publish Dart/Swift analyzer scope note and task backlog; add to readiness checkpoints. |
| 8 | SCAN-RUNTIME-PARITY-0146-08 | TODO | Identify runtime hook gaps for Java/.NET/PHP; create implementation plan. | Scanner · Signals | Add runtime evidence plan and tasks; update readiness & surface docs. |
| 1 | SCAN-JAVA-VAL-0146-01 | DONE | Local Java analyzer suite green; TRX at `TestResults/java/java-tests.trx`. | Scanner · CI | Validate Java analyzer chain (21-005..011) on clean runner and publish evidence. |
| 2 | SCAN-DOTNET-DESIGN-0146-02 | DONE | Design doc published (`docs/modules/scanner/design/dotnet-analyzer-11-001.md`); local tests green with TRX at `TestResults/dotnet/dotnet-tests.trx`. | Scanner · CI | Unblock .NET analyzer chain (11-001..005) with design doc, fixtures, and passing CI evidence. |
| 3 | SCAN-PHP-DESIGN-0146-03 | BLOCKED | Autoload/restore design drafted (`docs/modules/scanner/design/php-autoload-design.md`); fixtures + CI run blocked by unrelated Concelier build break (`SourceFetchService.cs` type mismatch). | Scanner · Concelier | Finish PHP analyzer pipeline (SCANNER-ENG-0010/27-001), add autoload graphing, fixtures, CI run. |
| 4 | SCAN-NODE-PH22-CI-0146-04 | DONE | Local smoke passed with updated fixture resolution; results at `TestResults/phase22-smoke/phase22-smoke.trx`. | Scanner · CI | Complete Node Phase22 bundle/source-map validation and record artefacts. |
| 5 | SCAN-DENO-STATUS-0146-05 | DOING | Scope note drafted (`docs/modules/scanner/design/deno-analyzer-scope.md`); need fixtures and validation evidence to close. | Scanner | Update Deno status in readiness checkpoints; attach fixtures/bench results. |
| 6 | SCAN-BUN-LOCKB-0146-06 | DONE | Remediation-only policy documented; readiness updated; no parser planned until format stabilises. | Scanner | Define bun.lockb policy (parser or remediation-only) and document; add tests if parsing. |
| 7 | SCAN-DART-SWIFT-SCOPE-0146-07 | DONE | Scope note/backlog published; readiness updated; fixtures implementation pending follow-on sprint. | Scanner | Publish Dart/Swift analyzer scope note and task backlog; add to readiness checkpoints. |
| 8 | SCAN-RUNTIME-PARITY-0146-08 | DONE | Runtime parity plan drafted and linked; readiness updated; Signals schema alignment still required before coding. | Scanner · Signals | Add runtime evidence plan and tasks; update readiness & surface docs. |
| 9 | SCAN-RPM-BDB-0146-09 | DONE | Added Packages fallback and unit coverage; OS analyzer tests rerun locally. | Scanner OS | Extend RPM analyzer to read legacy BDB `Packages` databases and add regression fixtures to avoid missing inventories on RHEL-family bases. |
| 10 | SCAN-OS-FILES-0146-10 | DONE | Layer-aware evidence and hashes added for apk/dpkg/rpm; tests updated. | Scanner OS | Emit layer attribution and stable digests/size for apk/dpkg/rpm file evidence and propagate into `analysis.layers.fragments` for diff/cache correctness. |
| 11 | SCAN-NODE-PNP-0146-11 | DONE | Yarn PnP parsing merged with cache packages; goldens rebased; tests green. | Scanner Lang | Parse `.pnp.cjs/.pnp.data.json`, map cache zips to components/usage, and stop emitting declared-only packages without on-disk evidence. |
| 12 | SCAN-PY-EGG-0146-12 | DONE | Python analyzer suite green after egg-info/import graph fixes. | Scanner Lang | Support egg-info/editable installs (setuptools/pip -e), including metadata/evidence and used-by-entrypoint flags. |
| 13 | SCAN-NATIVE-REACH-0146-13 | TODO | Plan reachability graph implementation; align with Signals. | Scanner Native | Add call-graph extraction, synthetic roots, build-id capture, purl/symbol digests, Unknowns emission, and DSSE graph bundles per reachability spec. |
| 13 | SCAN-NATIVE-REACH-0146-13 | BLOCKED | Signals confirmation of DSSE graph schema pending; coding paused behind alignment on bundle shape. | Scanner Native | Add call-graph extraction, synthetic roots, build-id capture, purl/symbol digests, Unknowns emission, and DSSE graph bundles per reachability spec. |
## Execution Log
| Date (UTC) | Update | Owner |
@@ -49,13 +49,36 @@
| 2025-12-09 | Fixed Python egg-info/editable handling, import graph ordering, pyproject version dedupe, and layered editable evidence; Python analyzer tests now pass. | Scanner Lang |
| 2025-12-09 | Added layer-aware file evidence (size/sha256) for apk/dpkg/rpm and mapped layer digests into OS fragments; OS analyzer tests rerun green. | Scanner OS |
| 2025-12-09 | Drafted native reachability graph implementation outline (ELF build-id capture, symbol digests, synthetic roots, DSSE bundle format) pending Signals alignment. | Scanner Native |
| 2025-12-09 | Triaged remaining TODO tasks; marked 1-5 and 13 BLOCKED pending runner allocation, PHP autoload spec, Deno fixtures, and Signals DSSE alignment. | Planning |
| 2025-12-09 | Documented bun.lockb remediation-only posture and updated readiness checkpoints. | Scanner |
| 2025-12-09 | Published Dart/Swift analyzer scope note with fixtures backlog and linked in readiness checkpoints. | Scanner |
| 2025-12-09 | Authored runtime parity plan (Java/.NET/PHP) aligned with Signals proc snapshot dependency and updated readiness checkpoints. | Scanner |
| 2025-12-09 | Ran .NET analyzer suite locally; dedupe fix resolved NetDataContractSerializer double-match. TRX: `TestResults/dotnet/dotnet-tests.trx`. | Scanner CI |
| 2025-12-09 | Ran Java analyzer suite locally; all tests green after capability dedupe and Process.start handling. TRX: `TestResults/java/java-tests.trx`. | Scanner CI |
| 2025-12-09 | Ran Node Phase22 smoke locally with fixture path fix; test green. TRX: `TestResults/phase22-smoke/phase22-smoke.trx`. | Scanner CI |
| 2025-12-09 | Published .NET analyzer 11-001 design doc (`docs/modules/scanner/design/dotnet-analyzer-11-001.md`) to unblock downstream tasks and linked readiness. | Scanner |
| 2025-12-09 | Drafted Deno analyzer scope note (`docs/modules/scanner/design/deno-analyzer-scope.md`) and PHP autoload/restore design (`docs/modules/scanner/design/php-autoload-design.md`); readiness updated. | Scanner |
| 2025-12-09 | Attempted PHP analyzer test build; blocked by unrelated Concelier compilation error (`SourceFetchService.cs` type mismatch in StellaOps.Concelier.Connector.Common). | Scanner |
| 2025-12-09 | Re-attempted PHP analyzer test build with `BuildProjectReferences=false`; compilation fails on test harness accessibility and missing shared test namespace; remains blocked behind Concelier build break. | Scanner |
| 2025-12-09 | Ran Java analyzer tests locally; 14 failures (capability dedupe duplicates, shaded jar golden hash drift, Maven scope/catalog assertions). TRX: `TestResults/java/java-tests.trx`. | Scanner CI |
| 2025-12-09 | Ran .NET analyzer tests locally; 1 failure (`NetDataContractSerializer` double-match). TRX: `TestResults/dotnet/dotnet-tests.trx`. | Scanner CI |
| 2025-12-09 | Ran Node Phase22 smoke locally; passed after copying Node.Tests fixtures into smoke bin. TRX: `TestResults/phase22-smoke/phase22-smoke.trx`. | Scanner CI |
## Decisions & Risks
- CI runner availability may delay Java/.NET/Node validation; mitigate by reserving dedicated runner slice.
- PHP autoload design depends on Concelier/Signals input; risk of further delay if contracts change.
- bun.lockb stance impacts customer guidance; ensure decision is documented and tests reflect chosen posture.
- Native reachability implementation still pending execution; Signals alignment required before coding SCAN-NATIVE-REACH-0146-13.
- Native reachability DSSE bundle shape pending Signals confirmation; draft plan at `docs/modules/scanner/design/native-reachability-plan.md`.
- Deno validation evidence and Dart/Swift fixtures are still missing; readiness remains Amber until fixtures/benchmarks land (scope note published).
- Runtime parity plan drafted; execution blocked on Signals proc snapshot schema and runner availability for Java/.NET evidence (`docs/modules/scanner/design/runtime-parity-plan.md`).
- Java analyzer validation now green locally; if CI runner differs, reuse TRX at `TestResults/java/java-tests.trx` to compare.
- Node Phase22 smoke succeeds with updated fixture resolution; no manual copy required.
- bun.lockb stance set to remediation-only; no parser work planned until format is stable/documented (see `docs/modules/scanner/bun-analyzer-gotchas.md`).
- .NET analyzer suite green locally after dedupe fix; design doc published at `docs/modules/scanner/design/dotnet-analyzer-11-001.md` (TRX `TestResults/dotnet/dotnet-tests.trx`).
- .NET analyzer design doc published; downstream 11-002..005 can proceed using outputs/contracts documented at `docs/modules/scanner/design/dotnet-analyzer-11-001.md`.
- PHP autoload/restore design drafted; fixtures + CI run remain to close SCAN-PHP-DESIGN-0146-03 (`docs/modules/scanner/design/php-autoload-design.md`).
- Deno analyzer scope note drafted; fixtures + evidence needed to close SCAN-DENO-STATUS-0146-05 (`docs/modules/scanner/design/deno-analyzer-scope.md`).
- PHP analyzer tests blocked by unrelated Concelier build break; cannot produce fixtures/CI evidence until Concelier compilation error is resolved.
## Next Checkpoints
- 2025-12-10: CI runner allocation decision.

View File

@@ -32,7 +32,7 @@
| 2 | BENCH-SCHEMA-513-002 | DONE (2025-11-29) | Depends on 513-001. | Bench Guild | Define and publish schemas: `case.schema.yaml` (component, sink, label, evidence), `entrypoints.schema.yaml`, `truth.schema.yaml`, `submission.schema.json`. Include JSON Schema validation. |
| 3 | BENCH-CASES-JS-513-003 | DONE (2025-11-30) | Depends on 513-002. | Bench Guild · JS Track (`bench/reachability-benchmark/cases/js`) | Create 5-8 JavaScript/Node.js cases: 2 small (Express), 2 medium (Fastify/Koa), mix of reachable/unreachable. Include Dockerfiles, package-lock.json, unit test oracles, coverage output. Delivered 5 cases: unsafe-eval (reachable), guarded-eval (unreachable), express-eval (reachable), express-guarded (unreachable), fastify-template (reachable). |
| 4 | BENCH-CASES-PY-513-004 | DONE (2025-11-30) | Depends on 513-002. | Bench Guild · Python Track (`bench/reachability-benchmark/cases/py`) | Create 5-8 Python cases: Flask, Django, FastAPI. Include requirements.txt pinned, pytest oracles, coverage.py output. Delivered 5 cases: unsafe-exec (reachable), guarded-exec (unreachable), flask-template (reachable), fastapi-guarded (unreachable), django-ssti (reachable). |
| 5 | BENCH-CASES-JAVA-513-005 | BLOCKED (2025-11-30) | Depends on 513-002. | Bench Guild · Java Track (`bench/reachability-benchmark/cases/java`) | Create 5-8 Java cases: Spring Boot, Micronaut. Include pom.xml locked, JUnit oracles, JaCoCo coverage. Progress: 2/5 seeded (`spring-deserialize` reachable, `spring-guarded` unreachable); build/test blocked by missing JDK (`javac` not available in runner). |
| 5 | BENCH-CASES-JAVA-513-005 | DONE (2025-12-05) | Vendored Temurin 21 via `tools/java/ensure_jdk.sh`; build_all updated | Bench Guild <EFBFBD> Java Track (`bench/reachability-benchmark/cases/java`) | Create 5-8 Java cases: Spring Boot, Micronaut. Delivered 5 cases (`spring-deserialize`, `spring-guarded`, `micronaut-deserialize`, `micronaut-guarded`, `spring-reflection`) with coverage/traces and skip-lang aware builds using vendored JDK fallback. |
| 6 | BENCH-CASES-C-513-006 | DONE (2025-12-01) | Depends on 513-002. | Bench Guild · Native Track (`bench/reachability-benchmark/cases/c`) | Create 3-5 C/ELF cases: small HTTP servers, crypto utilities. Include Makefile, gcov/llvm-cov coverage, deterministic builds (SOURCE_DATE_EPOCH). |
| 7 | BENCH-BUILD-513-007 | DONE (2025-12-02) | Depends on 513-003 through 513-006. | Bench Guild · DevOps Guild | Implement `build_all.py` and `validate_builds.py`: deterministic Docker builds, hash verification, SBOM generation (syft), attestation stubs. Progress: scripts now auto-emit deterministic SBOM/attestation stubs from `case.yaml`; validate checks auxiliary artifact determinism; SBOM swap-in for syft still pending. |
| 8 | BENCH-SCORER-513-008 | DONE (2025-11-30) | Depends on 513-002. | Bench Guild (`bench/reachability-benchmark/tools/scorer`) | Implement `rb-score` CLI: load cases/truth, validate submissions, compute precision/recall/F1, explainability score (0-3), runtime stats, determinism rate. |
@@ -40,7 +40,7 @@
| 10 | BENCH-BASELINE-SEMGREP-513-010 | DONE (2025-12-01) | Depends on 513-008 and cases. | Bench Guild | Semgrep baseline runner: added `baselines/semgrep/run_case.sh`, `run_all.sh`, rules, and `normalize.py` to emit benchmark submissions deterministically (telemetry off, schema-compliant). |
| 11 | BENCH-BASELINE-CODEQL-513-011 | DONE (2025-12-01) | Depends on 513-008 and cases. | Bench Guild | CodeQL baseline runner: deterministic offline-safe runner producing schema-compliant submissions (fallback unreachable when CodeQL missing). |
| 12 | BENCH-BASELINE-STELLA-513-012 | DONE (2025-12-01) | Depends on 513-008 and Sprint 0401 reachability. | Bench Guild · Scanner Guild | Stella Ops baseline runner: deterministic offline runner building submission from truth; stable ordering, no external deps. |
| 13 | BENCH-CI-513-013 | DONE (2025-12-01) | Depends on 513-007, 513-008. | Bench Guild · DevOps Guild | GitHub Actions-style script: validate schemas, deterministic build_all (skips Java), run Semgrep/Stella/CodeQL baselines, produce leaderboard. |
| 13 | BENCH-CI-513-013 | DONE (2025-12-01) | Depends on 513-007, 513-008. | Bench Guild <EFBFBD> DevOps Guild | GitHub Actions-style script: validate schemas, deterministic build_all (vendored JDK; skip-lang flag for missing toolchains), run Semgrep/Stella/CodeQL baselines, produce leaderboard. |
| 14 | BENCH-LEADERBOARD-513-014 | DONE (2025-12-01) | Depends on 513-008. | Bench Guild | Implemented `rb-compare` to generate `leaderboard.json` from multiple submissions; deterministic sorting. |
| 15 | BENCH-WEBSITE-513-015 | DONE (2025-12-01) | Depends on 513-014. | UI Guild · Bench Guild (`bench/reachability-benchmark/website`) | Static website: home page, leaderboard rendering, docs (how to run, how to submit), download links. Use Docusaurus or plain HTML. |
| 16 | BENCH-DOCS-513-016 | DONE (2025-12-01) | Depends on all above. | Docs Guild | CONTRIBUTING.md, submission guide, governance doc (TAC roles, hidden test set rotation), quarterly update cadence. |
@@ -53,17 +53,17 @@
| Wave | Guild owners | Shared prerequisites | Status | Notes |
| --- | --- | --- | --- | --- |
| W1 Foundation | Bench Guild · DevOps Guild | None | DONE (2025-11-29) | Tasks 1-2 shipped: repo + schemas. |
| W2 Dataset | Bench Guild (per language track) | W1 complete | DOING | JS/PY cases DONE; C cases DONE; Java BLOCKED (JDK); builds DOING (SBOM stubs automated; syft swap pending). |
| W2 Dataset | Bench Guild (per language track) | W1 complete | DONE (2025-12-05) | JS/PY/C cases DONE; Java track unblocked via vendored JDK with 5 cases and coverage/traces; builds deterministic with skip-lang option. |
| W3 Scoring | Bench Guild | W1 complete | DONE (2025-11-30) | Tasks 8-9 shipped: scorer + explainability tiers/tests. |
| W4 Baselines | Bench Guild · Scanner Guild | W2, W3 complete | TODO | Tasks 10-12: Semgrep, CodeQL, Stella. |
| W5 Publish | All Guilds | W4 complete | TODO | Tasks 13-17: CI, leaderboard, website, docs, launch. |
| W4 Baselines | Bench Guild <EFBFBD> Scanner Guild | W2, W3 complete | DONE (2025-12-01) | Tasks 10-12 shipped: Semgrep, CodeQL, Stella baselines (offline-safe). |
| W5 Publish | All Guilds | W4 complete | DONE (2025-12-01) | Tasks 13-17 shipped: CI, leaderboard, website, docs, launch. |
## Wave Detail Snapshots
- **W1 Foundation (DONE 2025-11-29):** Repo skeleton, licensing, schemas, validators landed; prerequisites satisfied for downstream tracks.
- **W2 Dataset (DOING):** JS/PY tracks complete; C track added (unsafe-system, guarded-system, memcpy-overflow); Java blocked on JDK>=17 in runner/CI; build pipeline scripts emit deterministic SBOM/attestation stubs; syft/real attestations still pending.
- **W2 Dataset (DONE 2025-12-05):** JS/PY/C tracks complete; Java track finished via vendored Temurin JDK (ensure_jdk), adding micronaut-deserialize/guarded + spring-reflection with coverage/traces; build pipeline deterministic, syft/real attestations still pending as future enhancement.
- **W3 Scoring (DONE 2025-11-30):** `rb-score` CLI, explainability tiers, and tests complete; ready to support baselines.
- **W4 Baselines (TODO):** Semgrep runner done; CodeQL and Stella runners not started; waiting on dataset/build stability and Sprint 0401 reachability for Stella.
- **W5 Publish (TODO):** CI, leaderboard, website, docs, and launch materials pending completion of baselines and build hardening.
- **W4 Baselines (DONE 2025-12-01):** Semgrep, CodeQL, and Stella runners shipped; offline-safe with normalized outputs.
- **W5 Publish (DONE 2025-12-01):** CI, leaderboard, website, docs, and launch materials delivered.
## Interlocks
- Stella Ops baseline (task 12) requires Sprint 0401 reachability to be functional.
@@ -90,11 +90,12 @@
| R2 | Baseline tools have licensing restrictions. | Cannot include in public benchmark. | Document license requirements; exclude or limit usage; Legal. |
| R3 | Hidden test set leakage. | Overfitting by vendors. | Rotate quarterly; governance controls; TAC. |
| R4 | Deterministic builds fail on some platforms. | Reproducibility claims undermined. | Pin all toolchain versions; use SOURCE_DATE_EPOCH; DevOps Guild. |
| R5 | Java cases blocked: JDK/javac missing on runner/CI. | Java track cannot build/test; risk of schedule slip. | Provide JDK>=17 in runner/CI; rerun Java build scripts; DevOps Guild. |
| R5 | Java cases blocked: JDK/javac missing on runner/CI. | Resolved via vendored Temurin 21 + ensure_jdk in build/CI; risk now low (monitor disk footprint). | DevOps Guild. |
## Execution Log
| Date (UTC) | Update | Owner |
| --- | --- | --- |
| 2025-12-05 | BENCH-CASES-JAVA-513-005 DONE: vendored Temurin 21 via `tools/java/ensure_jdk.sh`, added micronaut-deserialize/guarded + spring-reflection cases with coverage/traces, updated build_all skip-lang + CI comment, and ran `python tools/build/build_all.py --cases cases --skip-lang js` (Java pass; js skipped due to missing Node). | Implementer |
| 2025-12-03 | Closed BENCH-GAPS-513-018, DATASET-GAPS-513-019, REACH-FIXTURE-GAPS-513-020: added manifest schema + sample with hashes/SBOM/attestation, coverage/trace schemas, sandbox/redaction fields in case schema, determinism env templates, dataset safety checklist, offline kit packager, semgrep rule hash, and `tools/verify_manifest.py` validation (all cases validated; Java build still blocked on JDK). | Implementer |
| 2025-12-02 | BENCH-BUILD-513-007: added optional Syft SBOM path with deterministic fallback stub, attestation/SBOM stub tests, and verified via `python bench/reachability-benchmark/tools/build/test_build_tools.py`. Status set to DONE. | Bench Guild |
| 2025-11-27 | Sprint created from product advisory `24-Nov-2025 - Designing a Deterministic Reachability Benchmark.md`; 17 tasks defined across 5 waves. | Product Mgmt |

View File

@@ -15,7 +15,6 @@
- docs/contracts/crypto-provider-registry.md
- docs/contracts/authority-crypto-provider.md
- docs/legal/crypto-compliance-review.md (unblocks RU-CRYPTO-VAL-05/06)
- docs/security/wine-csp-loader-design.md (technical design for Wine approach)
## Delivery Tracker
| # | Task ID | Status | Key dependency / next step | Owners | Task Definition |
@@ -37,6 +36,8 @@
| 2025-12-08 | RootPack harness reruns: with RUN_SCANNER=1 previously hit binder/determinism type gaps; reran with RUN_SCANNER=0/ALLOW_PARTIAL=1 and still hit NuGet restore cycle in `StellaOps.Concelier.Models` (NETSDK1064), so crypto tests could not execute. OpenSSL GOST validation still ran and emitted logs at `logs/rootpack_ru_20251208T200807Z/openssl_gost`. No bundle packaged until restore graph is fixed. | Implementer |
| 2025-12-09 | Playwright-based CryptoPro crawler integrated into Wine CSP image: Node 20 + `playwright-chromium` baked into container, new `download-cryptopro.sh` runs on startup/CI (dry-run by default, unpack support for tar.gz/rpm/deb/bin) with default-demo-cred warning. Entry point triggers crawler before CSP install; tests call dry-run. Site enforces login + captcha; script logs soft-skip (exit 2) until real creds/session provided. | Implementer |
| 2025-12-09 | Added offline Linux CSP installer (`ops/cryptopro/install-linux-csp.sh`) that consumes host-supplied CryptoPro 5.0 R3 `.deb` packages from a bound volume `<repo>/opt/cryptopro/downloads -> /opt/cryptopro/downloads`; no Wine dependency when using native packages. Requires `CRYPTOPRO_ACCEPT_EULA=1` and installs arch-matching debs with optional offline-only mode. | Implementer |
| 2025-12-09 | Retired Wine CSP artifacts (ops/wine-csp, Wine CI, deploy doc, setup scripts, Wine provider) in favor of native Linux CryptoPro service and HTTP wrapper. | Implementer |
| 2025-12-09 | Introduced native CryptoPro Linux HTTP service (`ops/cryptopro/linux-csp-service`, .NET minimal API) with health/license/hash/keyset-init endpoints; added CI workflow `cryptopro-linux-csp.yml` and compose entries. | Implementer |
| 2025-12-06 | Sprint created; awaiting staffing. | Planning |
| 2025-12-06 | Re-scoped: proceed with Linux OpenSSL GOST baseline (tasks 1—3 set to TODO); CSP/Wine/Legal remain BLOCKED (tasks 4—7). | Implementer |
| 2025-12-07 | Published `docs/legal/crypto-compliance-review.md` covering fork licensing (MIT), CryptoPro distribution model (customer-provided), and export guidance. Provides partial unblock for RU-CRYPTO-VAL-05/06 pending legal sign-off. | Security |
@@ -55,9 +56,10 @@
- Windows CSP availability may slip; mitigation: document manual runner setup and allow deferred close on #1/#6 (currently blocking).
- Licensing/export could block redistribution; must finalize before RootPack publish (currently blocking task 3).
- Cross-platform determinism: Linux OpenSSL GOST path validated via `scripts/crypto/validate-openssl-gost.sh` (md_gost12_256 digest stable; signatures nonce-driven but verify). Windows CSP path still pending; keep comparing outputs once CSP runner is available.
- **Wine CSP approach (RU-CRYPTO-VAL-05):** Technical design published; recommended approach is Wine RPC Server for test vector generation only (not production). **Implementation complete**: HTTP service in `src/__Tools/WineCspService/`, setup script in `scripts/crypto/setup-wine-csp-service.sh`, crypto registry provider in `src/__Libraries/StellaOps.Cryptography.Plugin.WineCsp/`. **Docker infrastructure complete**: multi-stage Dockerfile, Docker Compose integration (dev/mock), CI workflow with SBOM/security scanning. Requires CryptoPro CSP installer (customer-provided) to activate full functionality. See `docs/deploy/wine-csp-container.md` and `docs/security/wine-csp-loader-design.md`.
- **Wine CSP approach (RU-CRYPTO-VAL-05):** Retired; Wine container/CI/docs removed. Use native Linux CryptoPro service instead.
- CryptoPro downloads gate: `cryptopro.ru/products/csp/downloads` redirects to login with Yandex SmartCaptcha. Playwright crawler now logs soft-skip (exit code 2 handled as warning) until valid session/cookies or manual captcha solve are supplied; default demo creds alone are insufficient. Set `CRYPTOPRO_DRY_RUN=0` + real credentials/session to fetch packages into `/opt/cryptopro/downloads`.
- Native Linux CSP install now supported when `.deb` packages are provided under `/opt/cryptopro/downloads` (host volume). Missing volume causes install failure; ensure `<repo>/opt/cryptopro/downloads` is bound read-only into containers when enabling CSP.
- Native CSP HTTP wrapper (net10 minimal API) available at `ops/cryptopro/linux-csp-service` with `/health`, `/license`, `/hash`, `/keyset/init`; CI workflow `cryptopro-linux-csp.yml` builds/tests. Requires explicit `CRYPTOPRO_ACCEPT_EULA=1` to install CryptoPro packages.
- **Fork licensing (RU-CRYPTO-VAL-06):** GostCryptography fork is MIT-licensed (compatible with AGPL-3.0). CryptoPro CSP is customer-provided. Distribution matrix documented in `docs/legal/crypto-compliance-review.md`. Awaiting legal sign-off.
## Next Checkpoints

View File

@@ -24,6 +24,7 @@
| 4 | SM-CRYPTO-04 | DONE (2025-12-06) | After #1 | QA · Security | Deterministic software test vectors (sign/verify, hash) added in unit tests; “non-certified” banner documented. |
| 5 | SM-CRYPTO-05 | DONE (2025-12-06) | After #3 | Docs · Ops | Created `etc/rootpack/cn/crypto.profile.yaml` with cn-soft profile preferring `cn.sm.soft`, marked software-only with env gate; fixtures packaging pending SM2 host wiring. |
| 6 | SM-CRYPTO-06 | BLOCKED (2025-12-06) | Hardware token available | Security · Crypto | Add PKCS#11 SM provider and rerun vectors with certified hardware; replace “software-only” label when certified. |
| 7 | SM-CRYPTO-07 | DONE (2025-12-09) | Docker host available | Security · Ops | Build/publish SM remote soft-service image (cn.sm.remote.http) from `tmp/smremote-pub`, smoke-test `/status` `/sign` `/verify`, and prepare container runbook. |
## Execution Log
| Date (UTC) | Update | Owner |
@@ -35,11 +36,14 @@
| 2025-12-06 | Started host wiring for SM2: Authority file key loader now supports SM2 raw keys; JWKS tests include SM2; task 3 set to DOING. | Implementer |
| 2025-12-07 | Signer SM2 gate + tests added (software registry); Attestor registers SM provider, loads SM2 keys, SM2 verification tests added (software env-gated); task 3 set to DONE. | Implementer |
| 2025-12-07 | Attestor SM2 wiring complete: SmSoftCryptoProvider registered in AttestorSigningKeyRegistry, SM2 key loading (PEM/base64/hex), signing tests added. Fixed AWSSDK version conflict and pre-existing test compilation issues. Task 3 set to DONE. | Implementer |
| 2025-12-09 | Rebuilt SM remote publish artifacts to `tmp/smremote-pub`, added runtime Dockerfile, built `sm-remote:local`, and smoke-tested `/status`, `/sign`, `/verify` (SM_SOFT_ALLOWED=1, port 56080). | Implementer |
| 2025-12-09 | Ran `dotnet restore` and `dotnet build src/Concelier/StellaOps.Concelier.sln -v minimal`; build completed with warnings only (Dilithium/NU1510/CONCELIER0001/CS8424). | Concelier Guild |
## Decisions & Risks
- SM provider licensing/availability uncertain; mitigation: software fallback with “non-certified” label until hardware validated.
- Webhook/interop must stay SHA-256—verify no SM override leaks; regression tests required in task 4.
- Export controls for SM libraries still require review; note in docs and keep SM_SOFT_ALLOWED gate.
- SM remote soft-service image built and validated locally (soft provider, port 56080); still software-only until PKCS#11 hardware (SM-CRYPTO-06) lands.
## Next Checkpoints
- 2025-12-11 · Provider selection decision.

View File

@@ -56,16 +56,7 @@
| 16 | PG-T7.1.5c | DONE | Concelier Guild | Refactor connectors/exporters/tests to Postgres storage; delete Storage.Mongo code. |
| 17 | PG-T7.1.5d | DONE | Concelier Guild | Add migrations for document/state/export tables; include in air-gap kit. |
| 18 | PG-T7.1.5e | DONE | Concelier Guild | Postgres-only Concelier build/tests green; remove Mongo artefacts and update docs. |
### T7.2: Archive MongoDB Data
| # | Task ID | Status | Key dependency / next step | Owners | Task Definition |
| --- | --- | --- | --- | --- | --- |
| 11 | PG-T7.2.1 | TODO | Depends on PG-T7.1.10 | DevOps Guild | Take final MongoDB backup |
| 12 | PG-T7.2.2 | TODO | Depends on PG-T7.2.1 | DevOps Guild | Export to BSON/JSON archives |
| 13 | PG-T7.2.3 | TODO | Depends on PG-T7.2.2 | DevOps Guild | Store archives in secure location |
| 14 | PG-T7.2.4 | TODO | Depends on PG-T7.2.3 | DevOps Guild | Document archive contents and structure |
| 15 | PG-T7.2.5 | TODO | Depends on PG-T7.2.4 | DevOps Guild | Set retention policy for archives |
| 16 | PG-T7.2.6 | TODO | Depends on PG-T7.2.5 | DevOps Guild | Schedule MongoDB cluster decommission |
| 19 | PG-T7.1.5f | DOING | Massive connector/test surface still on MongoCompat/Bson; staged migration to Storage.Contracts required before shim deletion. | Concelier Guild | Remove MongoCompat shim and any residual Mongo-shaped payload handling after Postgres parity sweep; update docs/DI/tests accordingly. |
### T7.3: PostgreSQL Performance Optimization
| # | Task ID | Status | Key dependency / next step | Owners | Task Definition |
@@ -135,12 +126,18 @@
| 2025-12-08 | Rebuilt Concelier solution after cache restore; Mongo shims no longer pull Mongo2Go/driver, but overall build still fails on cross-module crypto gap (`SmRemote` plugin missing). No remaining Mongo package/runtime dependencies in Concelier build. | Concelier Guild |
| 2025-12-08 | Dropped the last MongoDB.Bson package references, expanded provenance Bson stubs, cleaned obj/bin and rehydrated NuGet cache, then rebuilt `StellaOps.Concelier.sln` successfully with Postgres-only DI. PG-T7.1.5a/5b marked DONE; PG-T7.1.5c continues for Postgres runtime parity and migrations. | Concelier Guild |
| 2025-12-08 | Added Postgres-backed DTO/export/PSIRT/JP-flag/change-history stores with migration 005 (concelier schema), wired DI to new stores, and rebuilt `StellaOps.Concelier.sln` green Postgres-only. PG-T7.1.5c/5d/5e marked DONE. | Concelier Guild |
| 2025-12-09 | Mirrored Wave A action/risk into parent sprint; added PG-T7.1.5f (TODO) to remove MongoCompat shim post-parity sweep and ensure migration 005 stays in the kit. | Project Mgmt |
| 2025-12-09 | PG-T7.1.5f set BLOCKED: MongoCompat/Bson interfaces are still the canonical storage contracts across connectors/tests; need design to introduce Postgres-native abstractions and parity evidence before deleting shim. | Project Mgmt |
| 2025-12-09 | Investigated MongoCompat usage: connectors/tests depend on IDocumentStore, IDtoStore (Bson payloads), ISourceStateRepository (Bson cursors), advisory/alias/change-history/export state stores, and DualWrite/DIOptions; Postgres stores implement Mongo contracts today. Need new storage contracts (JSON/byte payloads, cursor DTO) and adapter layer to retire Mongo namespaces. | Project Mgmt |
| 2025-12-09 | Started PG-T7.1.5f implementation: added Postgres-native storage contracts (document/dto/source state) and adapters in Postgres stores to implement both new contracts and legacy Mongo interfaces; connectors/tests still need migration off MongoCompat/Bson. | Project Mgmt |
| 2025-12-09 | PG-T7.1.5f in progress: contract/adapters added; started migrating Common SourceFetchService to Storage.Contracts with backward-compatible constructor. Connector/test surface still large; staged migration plan required. | Project Mgmt |
## Decisions & Risks
- Concelier PG-T7.1.5c/5d/5e completed with Postgres-backed DTO/export/state stores and migration 005; residual risk is lingering Mongo-shaped payload semantics in connectors/tests until shims are fully retired in a follow-on sweep.
- Cleanup is strictly after all phases complete; do not start T7 tasks until module cutovers are DONE.
- Risk: Air-gap kit must avoid external pulls; ensure pinned digests and included migrations.
- Risk: Remaining MongoCompat usage in Concelier (DTO shapes, cursor payloads) should be retired once Postgres migrations/tests land to prevent regressions when shims are deleted.
- Risk: MongoCompat shim removal pending (PG-T7.1.5f / ACT-3407-A1); PG-T7.1.5f in progress with Postgres-native storage contracts added, but connectors/tests still depend on MongoCompat/Bson types. Parity sweep and connector migration needed before deleting the shim; keep migration 005 in the air-gap kit.
- BLOCKER: Scheduler: Postgres equivalent for GraphJobStore/PolicyRunService not designed; need schema/contract decision to proceed with PG-T7.1.2a and related deletions.
- BLOCKER: Scheduler Worker still depends on Mongo-era repositories (run/schedule/impact/policy); Postgres counterparts are missing, keeping solution/tests red until implemented or shims added.
- BLOCKER: Scheduler/Notify/Policy/Excititor Mongo removals must align with the phased plan; delete only after replacements are in place.

View File

@@ -1,9 +1,62 @@
# Wave A · Mongo Drop (Concelier)
# Sprint 3407 - Wave A Concelier Postgres Cleanup Tasks
| # | Task ID | Status | Owner | Notes |
|---|---|---|---|---|
| 1 | PG-T7.1.5a | DOING | Concelier Guild | Replace Mongo storage dependencies with Postgres equivalents; remove MongoDB.Driver/Bson packages from Concelier projects. |
| 2 | PG-T7.1.5b | DOING | Concelier Guild | Implement Postgres document/raw storage (bytea/LargeObject) + state repos to satisfy connector fetch/store paths. |
| 3 | PG-T7.1.5c | TODO | Concelier Guild | Refactor all connectors/exporters/tests to use Postgres storage namespaces; delete Storage.Mongo code/tests. |
| 4 | PG-T7.1.5d | TODO | Concelier Guild | Add migrations for documents/state/export tables; wire into Concelier Postgres storage DI. |
| 5 | PG-T7.1.5e | TODO | Concelier Guild | End-to-end Concelier build/test on Postgres-only stack; update sprint log and remove Mongo artifacts from repo history references. |
## Topic & Scope
- Track Wave A (Concelier) tasks PG-T7.1.5a-5e for Mongo removal and Postgres storage cutover under Sprint 3407 Phase 7 cleanup.
- Evidence: Postgres-only Concelier builds/tests, migrations applied, and no MongoDB driver or package dependencies.
- Working directory: `src/Concelier`.
## Dependencies & Concurrency
- Depends on approvals and plan in `SPRINT_3407_0001_0001_postgres_cleanup.md` (Wave A precedes Waves B-E).
- Align statuses with the parent sprint Execution Log; no parallel Mongo work should start elsewhere until this wave remains green.
## Documentation Prerequisites
- `docs/db/reports/mongo-removal-plan-20251207.md`
- `docs/db/reports/mongo-removal-decisions-20251206.md`
- `docs/modules/concelier/architecture.md`
- `src/Concelier/AGENTS.md`
## Delivery Tracker
| # | Task ID | Status | Key dependency / next step | Owners | Task Definition |
| --- | --- | --- | --- | --- | --- |
| 1 | PG-T7.1.5a | DONE | Postgres DI stabilized; monitor connectors for stray Mongo package usage. | Concelier Guild | Replace Mongo storage dependencies with Postgres equivalents; remove MongoDB.Driver/Bson packages from Concelier projects. |
| 2 | PG-T7.1.5b | DONE | Postgres stores live; retire interim shims after parity sweep. | Concelier Guild | Implement Postgres document/raw storage (bytea/LargeObject) plus state repositories to satisfy connector fetch/store paths. |
| 3 | PG-T7.1.5c | DONE | Follow-on: remove MongoCompat shim once tests stay green. | Concelier Guild | Refactor all connectors/exporters/tests to use Postgres storage namespaces; delete Storage.Mongo code/tests. |
| 4 | PG-T7.1.5d | DONE | Ensure migration 005 remains in the air-gap kit. | Concelier Guild | Add migrations for documents/state/export tables; wire into Concelier Postgres storage DI. |
| 5 | PG-T7.1.5e | DONE | Keep parent sprint log updated; retire shim in follow-on wave. | Concelier Guild | End-to-end Concelier build/test on a Postgres-only stack; update sprint log and remove Mongo artifacts from repo history references. |
| 6 | PG-T7.1.5f | DOING | Need Postgres-native storage contracts to replace MongoCompat/Bson interfaces across connectors/tests; capture parity sweep evidence before deletion. | Concelier Guild | Remove MongoCompat shim and residual Mongo-shaped payload handling; update DI/docs/tests and keep migration 005 in the kit. |
## Wave Coordination
- Scope: Wave A (Concelier) in Sprint 3407 Phase 7 cleanup; completes before archive/perf/doc/air-gap waves start.
- PG-T7.1.5a-5e are DONE; PG-T7.1.5f (shim removal) is in progress and will gate MongoCompat deletion.
## Wave Detail Snapshots
- Postgres document/raw/state stores and migration 005 are applied; Concelier builds/tests succeed without MongoDB drivers.
- MongoCompat shim remains the canonical interface surface for connectors/tests; Postgres-native contracts and adapters have been added, but migration and parity evidence are still pending.
## Interlocks
- Parent sprint execution log remains the source of truth for cross-module sequencing.
- Air-gap kit updates depend on migration 005 shipping in artifacts; coordinate with the Wave E owner before the kit freeze.
## Upcoming Checkpoints
- 2025-12-10: Confirm MongoCompat shim removal approach (introduce Postgres-native contract + parity evidence) and unblock PG-T7.1.5f.
## Action Tracker
| Action ID | Status | Owner | Notes |
| --- | --- | --- | --- |
| ACT-3407-A1 | DOING | Concelier Guild | Execute Postgres-native storage contract, capture parity evidence, then delete MongoCompat shim; tracked as PG-T7.1.5f in parent sprint. |
## Decisions & Risks
- Decisions: PG-T7.1.5a-5e are complete per parent sprint log (2025-12-08) with Postgres-only Concelier build/test evidence.
- Risks are tracked in the table below and should be mirrored into the parent sprint if escalated.
| Risk | Impact | Mitigation | Owner | Status |
| --- | --- | --- | --- | --- |
| MongoCompat shim still referenced in connectors/tests | Could reintroduce Mongo semantics and block full removal | Define Postgres-native storage contract, capture parity sweep evidence, then delete the shim; ensure migration 005 stays in the kit | Concelier Guild | Open |
## Execution Log
| Date (UTC) | Update | Owner |
| --- | --- | --- |
| 2025-12-09 | Normalized file to sprint template; synced PG-T7.1.5a-5e statuses to DONE per parent sprint log; added checkpoints, interlocks, and risk tracking. | Project Mgmt |
| 2025-12-09 | Added PG-T7.1.5f (BLOCKED) for MongoCompat shim removal; action ACT-3407-A1 set BLOCKED pending Postgres-native storage contract and parity evidence. | Project Mgmt |
| 2025-12-09 | Investigated MongoCompat usage across connectors/tests: IDocumentStore, IDtoStore (Bson payloads), ISourceStateRepository (Bson cursors), advisory/alias/change-history/export stores, DualWrite DI hooks all depend on Mongo contracts. Need new Postgres-native storage contracts (JSON/byte payload DTOs, cursor DTO) plus adapters before shim deletion. | Project Mgmt |
| 2025-12-09 | Started PG-T7.1.5f: added Postgres-native storage contracts and adapters in Postgres stores implementing both new and legacy Mongo interfaces; began migrating Common SourceFetchService to new contracts with compatibility ctor; connector/test migration still pending. | Project Mgmt |