stella-ops.org/git.stella-ops.org
1
0
Fork 0
Code Issues Pull Requests Actions 2 Releases Wiki Activity

feat: Document completed tasks for KMS, Cryptography, and Plugin Libraries

Browse Source 
- Added detailed task completion records for KMS interface implementation and CLI support for file-based keys.
- Documented security enhancements including Argon2id password hashing, audit event contracts, and rate limiting configurations.
- Included scoped service support and integration updates for the Plugin platform, ensuring proper DI handling and testing coverage.
This commit is contained in:
master
2025-10-31 14:37:45 +02:00
parent 240e8ff25d
commit 15b4a1de6a
312 changed files with 6399 additions and 3319 deletions
Download Patch File Download Diff File Expand all files Collapse all files

18
src/Concelier/__Libraries/StellaOps.Concelier.Connector.Acsc/TASKS.completed.md Normal file
View File

@@ -0,0 +1,18 @@
# Completed Tasks
|FEEDCONN-ACSC-02-001 Source discovery & feed contract|BE-Conn-ACSC|Research|**DONE (2025-10-11)** – Catalogued feed slugs `/acsc/view-all-content/{alerts,advisories,news,publications,threats}/rss`; every endpoint currently negotiates HTTP/2 then aborts with `INTERNAL_ERROR` (curl exit 92) and hanging >600 s when forcing `--http1.1`. Documented traces + mitigations in `docs/concelier-connector-research-20251011.md` and opened `FEEDCONN-SHARED-HTTP2-001` for shared handler tweaks (force `RequestVersionOrLower`, jittered retries, relay option).|
|FEEDCONN-ACSC-02-002 Fetch pipeline & cursor persistence|BE-Conn-ACSC|Source.Common, Storage.Mongo|**DONE (2025-10-12)** – HTTP client now pins `HttpRequestMessage.VersionPolicy = RequestVersionOrLower`, forces `AutomaticDecompression = GZip | Deflate`, and sends `User-Agent: StellaOps/Concelier (+https://stella-ops.org)` via `AddAcscConnector`. Fetch pipeline implemented in `AcscConnector` with relay-aware fallback (`AcscProbeJob` seeds preference), deterministic cursor updates (`preferredEndpoint`, published timestamp per feed), and metadata-deduped documents. Unit tests `AcscConnectorFetchTests` + `AcscHttpClientConfigurationTests` cover direct/relay flows and client wiring.|
|FEEDCONN-ACSC-02-003 Parser & DTO sanitiser|BE-Conn-ACSC|Source.Common|**DONE (2025-10-12)** – Added `AcscFeedParser` to sanitise RSS payloads, collapse multi-paragraph summaries, dedupe references, and surface `serialNumber`/`advisoryType` fields as structured metadata + alias candidates. `ParseAsync` now materialises `acsc.feed.v1` DTOs, promotes documents to `pending-map`, and advances cursor state. Covered by `AcscConnectorParseTests`.|
|FEEDCONN-ACSC-02-004 Canonical mapper + range primitives|BE-Conn-ACSC|Models|**DONE (2025-10-12)** – Introduced `AcscMapper` and wired `MapAsync` to emit canonical advisories with normalized aliases, source-tagged references, and optional vendor `affectedPackages` derived from “Systems/Products affected” fields. Documents transition to `mapped`, advisories persist via `IAdvisoryStore`, and metrics/logging capture mapped counts. `AcscConnectorParseTests` exercise fetch→parse→map flow.|
|FEEDCONN-ACSC-02-005 Deterministic fixtures & regression tests|QA|Testing|**DONE (2025-10-12)** – `AcscConnectorParseTests` now snapshots fetch→parse→map output via `Acsc/Fixtures/acsc-advisories.snapshot.json`; set `UPDATE_ACSC_FIXTURES=1` to regenerate. Tests assert DTO status transitions, advisory persistence, and state cleanup.|
|FEEDCONN-ACSC-02-006 Diagnostics & documentation|DevEx|Docs|**DONE (2025-10-12)** – Added module README describing configuration, job schedules, metrics (including new `acsc.map.success` counter), relay behaviour, and fixture workflow. Diagnostics updated to count map successes alongside existing fetch/parse metrics.|
|FEEDCONN-ACSC-02-007 Feed retention & pagination validation|BE-Conn-ACSC|Research|**DONE (2025-10-11)** – Relay sampling shows retention ≥ July 2025; need to re-run once direct HTTP/2 path is stable to see if feed caps at ~50 items and whether `?page=` exists. Pending action tracked in shared HTTP downgrade task.|
|FEEDCONN-ACSC-02-008 HTTP client compatibility plan|BE-Conn-ACSC|Source.Common|**DONE (2025-10-11)** – Reproduced Akamai resets, drafted downgrade plan (two-stage HTTP/2 retry + relay fallback), and filed `FEEDCONN-SHARED-HTTP2-001`; module README TODO will host the per-environment knob matrix.|

14
src/Concelier/__Libraries/StellaOps.Concelier.Connector.Acsc/TASKS.md
View File

@@ -1,11 +1,3 @@
# TASKS
| Task | Owner(s) | Depends on | Notes |
|---|---|---|---|
|FEEDCONN-ACSC-02-001 Source discovery & feed contract|BE-Conn-ACSC|Research|**DONE (2025-10-11)** – Catalogued feed slugs `/acsc/view-all-content/{alerts,advisories,news,publications,threats}/rss`; every endpoint currently negotiates HTTP/2 then aborts with `INTERNAL_ERROR` (curl exit 92) and hanging >600 s when forcing `--http1.1`. Documented traces + mitigations in `docs/concelier-connector-research-20251011.md` and opened `FEEDCONN-SHARED-HTTP2-001` for shared handler tweaks (force `RequestVersionOrLower`, jittered retries, relay option).|
|FEEDCONN-ACSC-02-002 Fetch pipeline & cursor persistence|BE-Conn-ACSC|Source.Common, Storage.Mongo|**DONE (2025-10-12)** – HTTP client now pins `HttpRequestMessage.VersionPolicy = RequestVersionOrLower`, forces `AutomaticDecompression = GZip | Deflate`, and sends `User-Agent: StellaOps/Concelier (+https://stella-ops.org)` via `AddAcscConnector`. Fetch pipeline implemented in `AcscConnector` with relay-aware fallback (`AcscProbeJob` seeds preference), deterministic cursor updates (`preferredEndpoint`, published timestamp per feed), and metadata-deduped documents. Unit tests `AcscConnectorFetchTests` + `AcscHttpClientConfigurationTests` cover direct/relay flows and client wiring.|
|FEEDCONN-ACSC-02-003 Parser & DTO sanitiser|BE-Conn-ACSC|Source.Common|**DONE (2025-10-12)** – Added `AcscFeedParser` to sanitise RSS payloads, collapse multi-paragraph summaries, dedupe references, and surface `serialNumber`/`advisoryType` fields as structured metadata + alias candidates. `ParseAsync` now materialises `acsc.feed.v1` DTOs, promotes documents to `pending-map`, and advances cursor state. Covered by `AcscConnectorParseTests`.|
|FEEDCONN-ACSC-02-004 Canonical mapper + range primitives|BE-Conn-ACSC|Models|**DONE (2025-10-12)** – Introduced `AcscMapper` and wired `MapAsync` to emit canonical advisories with normalized aliases, source-tagged references, and optional vendor `affectedPackages` derived from “Systems/Products affected” fields. Documents transition to `mapped`, advisories persist via `IAdvisoryStore`, and metrics/logging capture mapped counts. `AcscConnectorParseTests` exercise fetch→parse→map flow.|
|FEEDCONN-ACSC-02-005 Deterministic fixtures & regression tests|QA|Testing|**DONE (2025-10-12)** – `AcscConnectorParseTests` now snapshots fetch→parse→map output via `Acsc/Fixtures/acsc-advisories.snapshot.json`; set `UPDATE_ACSC_FIXTURES=1` to regenerate. Tests assert DTO status transitions, advisory persistence, and state cleanup.|
|FEEDCONN-ACSC-02-006 Diagnostics & documentation|DevEx|Docs|**DONE (2025-10-12)** – Added module README describing configuration, job schedules, metrics (including new `acsc.map.success` counter), relay behaviour, and fixture workflow. Diagnostics updated to count map successes alongside existing fetch/parse metrics.|
|FEEDCONN-ACSC-02-007 Feed retention & pagination validation|BE-Conn-ACSC|Research|**DONE (2025-10-11)** – Relay sampling shows retention ≥ July 2025; need to re-run once direct HTTP/2 path is stable to see if feed caps at ~50 items and whether `?page=` exists. Pending action tracked in shared HTTP downgrade task.|
|FEEDCONN-ACSC-02-008 HTTP client compatibility plan|BE-Conn-ACSC|Source.Common|**DONE (2025-10-11)** – Reproduced Akamai resets, drafted downgrade plan (two-stage HTTP/2 retry + relay fallback), and filed `FEEDCONN-SHARED-HTTP2-001`; module README TODO will host the per-environment knob matrix.|
# TASKS
| Task | Owner(s) | Depends on | Notes |
|---|---|---|---|

18
src/Concelier/__Libraries/StellaOps.Concelier.Connector.Cccs/TASKS.completed.md Normal file
View File

@@ -0,0 +1,18 @@
# Completed Tasks
|FEEDCONN-CCCS-02-001 Catalogue official CCCS advisory feeds|BE-Conn-CCCS|Research|**DONE (2025-10-11)** – Resolved RSS→Atom redirects (`/api/cccs/rss/v1/get?...` → `/api/cccs/atom/v1/get?...`), confirmed feed caps at 50 entries with inline HTML bodies, no `Last-Modified`/`ETag`, and `updated` timestamps in UTC. Findings and packet captures parked in `docs/concelier-connector-research-20251011.md`; retention sweep follow-up tracked in 02-007.|
|FEEDCONN-CCCS-02-002 Implement fetch & source state handling|BE-Conn-CCCS|Source.Common, Storage.Mongo|**DONE (2025-10-14)** – `CccsConnector.FetchAsync` now hydrates feeds via `CccsFeedClient`, persists per-entry JSON payloads with SHA256 dedupe and cursor state, throttles requests, and records taxonomy + language metadata in document state.|
|FEEDCONN-CCCS-02-003 DTO/parser implementation|BE-Conn-CCCS|Source.Common|**DONE (2025-10-14)** – Added `CccsHtmlParser` to sanitize Atom body HTML, extract serial/date/product bullets, collapse whitespace, and emit normalized reference URLs; `ParseAsync` now persists DTO records under schema `cccs.dto.v1`.|
|FEEDCONN-CCCS-02-004 Canonical mapping & range primitives|BE-Conn-CCCS|Models|**DONE (2025-10-14)** – `CccsMapper` now materializes canonical advisories (aliases from serial/source/CVEs, references incl. canonical URL, vendor package records) with provenance masks; `MapAsync` stores results in `AdvisoryStore`.|
|FEEDCONN-CCCS-02-005 Deterministic fixtures & tests|QA|Testing|**DONE (2025-10-14)** – Added English/French fixtures plus parser + connector end-to-end tests (`StellaOps.Concelier.Connector.Cccs.Tests`). Canned HTTP handler + Mongo fixture enables fetch→parse→map regression; fixtures refresh via `UPDATE_CCCS_FIXTURES=1`.|
|FEEDCONN-CCCS-02-006 Observability & documentation|DevEx|Docs|**DONE (2025-10-15)** – Added `CccsDiagnostics` meter (fetch/parse/map counters), enriched connector logs with document counts, and published `docs/modules/concelier/operations/connectors/cccs.md` covering config, telemetry, and sanitiser guidance.|
|FEEDCONN-CCCS-02-007 Historical advisory harvesting plan|BE-Conn-CCCS|Research|**DONE (2025-10-15)** – Measured `/api/cccs/threats/v1/get` inventory (~5.1k rows/lang; earliest 2018-06-08), documented backfill workflow + language split strategy, and linked the runbook for Offline Kit execution.|
|FEEDCONN-CCCS-02-008 Raw DOM parsing refinement|BE-Conn-CCCS|Source.Common|**DONE (2025-10-15)** – Parser now walks unsanitised DOM (heading + nested list coverage), sanitizer keeps `<h#>`/`section` nodes, and regression fixtures/tests assert EN/FR list handling + preserved HTML structure.|

8
src/Concelier/__Libraries/StellaOps.Concelier.Connector.Cccs/TASKS.md
View File

@@ -1,12 +1,4 @@
# TASKS
| Task | Owner(s) | Depends on | Notes |
|---|---|---|---|
|FEEDCONN-CCCS-02-001 Catalogue official CCCS advisory feeds|BE-Conn-CCCS|Research|**DONE (2025-10-11)** – Resolved RSS→Atom redirects (`/api/cccs/rss/v1/get?...` → `/api/cccs/atom/v1/get?...`), confirmed feed caps at 50 entries with inline HTML bodies, no `Last-Modified`/`ETag`, and `updated` timestamps in UTC. Findings and packet captures parked in `docs/concelier-connector-research-20251011.md`; retention sweep follow-up tracked in 02-007.|
|FEEDCONN-CCCS-02-002 Implement fetch & source state handling|BE-Conn-CCCS|Source.Common, Storage.Mongo|**DONE (2025-10-14)** – `CccsConnector.FetchAsync` now hydrates feeds via `CccsFeedClient`, persists per-entry JSON payloads with SHA256 dedupe and cursor state, throttles requests, and records taxonomy + language metadata in document state.|
|FEEDCONN-CCCS-02-003 DTO/parser implementation|BE-Conn-CCCS|Source.Common|**DONE (2025-10-14)** – Added `CccsHtmlParser` to sanitize Atom body HTML, extract serial/date/product bullets, collapse whitespace, and emit normalized reference URLs; `ParseAsync` now persists DTO records under schema `cccs.dto.v1`.|
|FEEDCONN-CCCS-02-004 Canonical mapping & range primitives|BE-Conn-CCCS|Models|**DONE (2025-10-14)** – `CccsMapper` now materializes canonical advisories (aliases from serial/source/CVEs, references incl. canonical URL, vendor package records) with provenance masks; `MapAsync` stores results in `AdvisoryStore`.|
|FEEDCONN-CCCS-02-005 Deterministic fixtures & tests|QA|Testing|**DONE (2025-10-14)** – Added English/French fixtures plus parser + connector end-to-end tests (`StellaOps.Concelier.Connector.Cccs.Tests`). Canned HTTP handler + Mongo fixture enables fetch→parse→map regression; fixtures refresh via `UPDATE_CCCS_FIXTURES=1`.|
|FEEDCONN-CCCS-02-006 Observability & documentation|DevEx|Docs|**DONE (2025-10-15)** – Added `CccsDiagnostics` meter (fetch/parse/map counters), enriched connector logs with document counts, and published `docs/modules/concelier/operations/connectors/cccs.md` covering config, telemetry, and sanitiser guidance.|
|FEEDCONN-CCCS-02-007 Historical advisory harvesting plan|BE-Conn-CCCS|Research|**DONE (2025-10-15)** – Measured `/api/cccs/threats/v1/get` inventory (~5.1k rows/lang; earliest 2018-06-08), documented backfill workflow + language split strategy, and linked the runbook for Offline Kit execution.|
|FEEDCONN-CCCS-02-008 Raw DOM parsing refinement|BE-Conn-CCCS|Source.Common|**DONE (2025-10-15)** – Parser now walks unsanitised DOM (heading + nested list coverage), sanitizer keeps `<h#>`/`section` nodes, and regression fixtures/tests assert EN/FR list handling + preserved HTML structure.|
|FEEDCONN-CCCS-02-009 Normalized versions rollout (Oct 2025)|BE-Conn-CCCS|Merge coordination (`FEEDMERGE-COORD-02-900`)|**TODO (due 2025-10-21)** – Implement trailing-version split helper per Merge guidance (see `../Merge/RANGE_PRIMITIVES_COORDINATION.md` “Helper snippets”) to emit `NormalizedVersions` via `SemVerRangeRuleBuilder`; refresh mapper tests/fixtures to assert provenance notes (`cccs:{serial}:{index}`) and confirm merge counters drop.<br>2025-10-29: See `docs/dev/normalized-rule-recipes.md` for ready-made helper + regex snippet; wire into `BuildPackages` and update fixtures with `UPDATE_CCCS_FIXTURES=1`.|

20
src/Concelier/__Libraries/StellaOps.Concelier.Connector.CertBund/TASKS.completed.md Normal file
View File

@@ -0,0 +1,20 @@
# Completed Tasks
|FEEDCONN-CERTBUND-02-001 Research CERT-Bund advisory endpoints|BE-Conn-CERTBUND|Research|**DONE (2025-10-11)** – Confirmed public RSS at `https://wid.cert-bund.de/content/public/securityAdvisory/rss` (HTTP 200 w/out cookies), 250-item window, German titles/categories, and detail links pointing to Angular SPA. Captured header profile (no cache hints) and logged open item to discover the JSON API used by `portal` frontend.|
|FEEDCONN-CERTBUND-02-002 Fetch job & state persistence|BE-Conn-CERTBUND|Source.Common, Storage.Mongo|**DONE (2025-10-14)** – `CertBundConnector.FetchAsync` consumes RSS via session-bootstrapped client, stores per-advisory JSON documents with metadata + SHA, throttles detail requests, and maintains cursor state (pending docs/mappings, known advisory IDs, last published).|
|FEEDCONN-CERTBUND-02-003 Parser/DTO implementation|BE-Conn-CERTBUND|Source.Common|**DONE (2025-10-14)** – Detail JSON piped through `CertBundDetailParser` (raw DOM sanitised to HTML), capturing severity, CVEs, product list, and references into DTO records (`cert-bund.detail.v1`).|
|FEEDCONN-CERTBUND-02-004 Canonical mapping & range primitives|BE-Conn-CERTBUND|Models|**DONE (2025-10-14)** – `CertBundMapper` emits canonical advisories (aliases, references, vendor package ranges, provenance) with severity normalisation and deterministic ordering.|
|FEEDCONN-CERTBUND-02-005 Regression fixtures & tests|QA|Testing|**DONE (2025-10-14)** – Added `StellaOps.Concelier.Connector.CertBund.Tests` covering fetch→parse→map against canned RSS/JSON fixtures; integration harness uses Mongo2Go + canned HTTP handler; fixtures regenerate via `UPDATE_CERTBUND_FIXTURES=1`.|
|FEEDCONN-CERTBUND-02-006 Telemetry & documentation|DevEx|Docs|**DONE (2025-10-15)** – Added `CertBundDiagnostics` (meter `StellaOps.Concelier.Connector.CertBund`) with fetch/parse/map counters + histograms, recorded coverage days, wired stage summary logs, and published the ops runbook (`docs/modules/concelier/operations/connectors/certbund.md`).|
|FEEDCONN-CERTBUND-02-007 Feed history & locale assessment|BE-Conn-CERTBUND|Research|**DONE (2025-10-15)** – Measured RSS retention (~6 days/≈250 items), captured connector-driven backfill guidance in the runbook, and aligned locale guidance (preserve `language=de`, Docs glossary follow-up). **Next:** coordinate with Tools to land the state-seeding helper so scripted backfills replace manual Mongo tweaks.|
|FEEDCONN-CERTBUND-02-008 Session bootstrap & cookie strategy|BE-Conn-CERTBUND|Source.Common|**DONE (2025-10-14)** – Feed client primes the portal session (cookie container via `SocketsHttpHandler`), shares cookies across detail requests, and documents bootstrap behaviour in options (`PortalBootstrapUri`).|
|FEEDCONN-CERTBUND-02-009 Offline Kit export packaging|BE-Conn-CERTBUND, Docs|Offline Kit|**DONE (2025-10-17)** – Added `src/Tools/certbund_offline_snapshot.py` to capture search/export JSON, emit deterministic manifests + SHA files, and refreshed docs (`docs/modules/concelier/operations/connectors/certbund.md`, `docs/24_OFFLINE_KIT.md`) with offline-kit instructions and manifest layout guidance. Seed data README/ignore rules cover local snapshot hygiene.|

9
src/Concelier/__Libraries/StellaOps.Concelier.Connector.CertBund/TASKS.md
View File

@@ -1,13 +1,4 @@
# TASKS
| Task | Owner(s) | Depends on | Notes |
|---|---|---|---|
|FEEDCONN-CERTBUND-02-001 Research CERT-Bund advisory endpoints|BE-Conn-CERTBUND|Research|**DONE (2025-10-11)** – Confirmed public RSS at `https://wid.cert-bund.de/content/public/securityAdvisory/rss` (HTTP 200 w/out cookies), 250-item window, German titles/categories, and detail links pointing to Angular SPA. Captured header profile (no cache hints) and logged open item to discover the JSON API used by `portal` frontend.|
|FEEDCONN-CERTBUND-02-002 Fetch job & state persistence|BE-Conn-CERTBUND|Source.Common, Storage.Mongo|**DONE (2025-10-14)** – `CertBundConnector.FetchAsync` consumes RSS via session-bootstrapped client, stores per-advisory JSON documents with metadata + SHA, throttles detail requests, and maintains cursor state (pending docs/mappings, known advisory IDs, last published).|
|FEEDCONN-CERTBUND-02-003 Parser/DTO implementation|BE-Conn-CERTBUND|Source.Common|**DONE (2025-10-14)** – Detail JSON piped through `CertBundDetailParser` (raw DOM sanitised to HTML), capturing severity, CVEs, product list, and references into DTO records (`cert-bund.detail.v1`).|
|FEEDCONN-CERTBUND-02-004 Canonical mapping & range primitives|BE-Conn-CERTBUND|Models|**DONE (2025-10-14)** – `CertBundMapper` emits canonical advisories (aliases, references, vendor package ranges, provenance) with severity normalisation and deterministic ordering.|
|FEEDCONN-CERTBUND-02-005 Regression fixtures & tests|QA|Testing|**DONE (2025-10-14)** – Added `StellaOps.Concelier.Connector.CertBund.Tests` covering fetch→parse→map against canned RSS/JSON fixtures; integration harness uses Mongo2Go + canned HTTP handler; fixtures regenerate via `UPDATE_CERTBUND_FIXTURES=1`.|
|FEEDCONN-CERTBUND-02-006 Telemetry & documentation|DevEx|Docs|**DONE (2025-10-15)** – Added `CertBundDiagnostics` (meter `StellaOps.Concelier.Connector.CertBund`) with fetch/parse/map counters + histograms, recorded coverage days, wired stage summary logs, and published the ops runbook (`docs/modules/concelier/operations/connectors/certbund.md`).|
|FEEDCONN-CERTBUND-02-007 Feed history & locale assessment|BE-Conn-CERTBUND|Research|**DONE (2025-10-15)** – Measured RSS retention (~6 days/≈250 items), captured connector-driven backfill guidance in the runbook, and aligned locale guidance (preserve `language=de`, Docs glossary follow-up). **Next:** coordinate with Tools to land the state-seeding helper so scripted backfills replace manual Mongo tweaks.|
|FEEDCONN-CERTBUND-02-008 Session bootstrap & cookie strategy|BE-Conn-CERTBUND|Source.Common|**DONE (2025-10-14)** – Feed client primes the portal session (cookie container via `SocketsHttpHandler`), shares cookies across detail requests, and documents bootstrap behaviour in options (`PortalBootstrapUri`).|
|FEEDCONN-CERTBUND-02-009 Offline Kit export packaging|BE-Conn-CERTBUND, Docs|Offline Kit|**DONE (2025-10-17)** – Added `src/Tools/certbund_offline_snapshot.py` to capture search/export JSON, emit deterministic manifests + SHA files, and refreshed docs (`docs/modules/concelier/operations/connectors/certbund.md`, `docs/24_OFFLINE_KIT.md`) with offline-kit instructions and manifest layout guidance. Seed data README/ignore rules cover local snapshot hygiene.|
|FEEDCONN-CERTBUND-02-010 Normalized range translator|BE-Conn-CERTBUND|Merge coordination (`FEEDMERGE-COORD-02-900`)|**TODO (due 2025-10-22)** – Translate `product.Versions` phrases (e.g., `2023.1 bis 2024.2`, `alle`) into comparator strings for `SemVerRangeRuleBuilder`, emit `NormalizedVersions` with `certbund:{advisoryId}:{vendor}` provenance, and extend tests/README with localisation notes.|

24
src/Concelier/__Libraries/StellaOps.Concelier.Connector.CertCc/TASKS.completed.md Normal file
View File

@@ -0,0 +1,24 @@
# Completed Tasks
|Document CERT/CC advisory sources|BE-Conn-CERTCC|Research|**DONE (2025-10-10)** – Catalogued Atom feed + VINCE API endpoints and archive references in `README.md`; include polling/backoff guidance.|
|Fetch pipeline & state tracking|BE-Conn-CERTCC|Source.Common, Storage.Mongo|**DONE (2025-10-12)** – Summary planner + fetch job persist monthly/yearly VINCE JSON to `DocumentStore`, hydrate the `TimeWindowCursorState`, and snapshot regression (`dotnet test` 2025-10-12) confirmed deterministic resume behaviour.|
|VINCE note detail fetcher|BE-Conn-CERTCC|Source.Common, Storage.Mongo|**DONE (2025-10-12)** – Detail bundle fetch now enqueues VU identifiers and persists note/vendors/vuls/vendors-vuls documents with ETag/Last-Modified metadata, tolerating missing optional endpoints without wedging the cursor.|
|DTO & parser implementation|BE-Conn-CERTCC|Source.Common|**DONE (2025-10-12)** – VINCE DTO aggregate materialises note/vendor/vulnerability payloads, normalises markdown to HTML-safe fragments, and surfaces vendor impact statements covered by parser unit tests.|
|Canonical mapping & range primitives|BE-Conn-CERTCC|Models|**DONE (2025-10-12)** – Mapper emits aliases (VU#, CVE), vendor range primitives, and normalizedVersions (`scheme=certcc.vendor`) with provenance masks; `certcc-advisories.snapshot.json` validates canonical output after schema sync.|
|Deterministic fixtures/tests|QA|Testing|**DONE (2025-10-11)** – Snapshot harness regenerated (`certcc-*.snapshot.json`), request ordering assertions added, and `UPDATE_CERTCC_FIXTURES` workflow verified for CI determinism.|
|Connector test harness remediation|BE-Conn-CERTCC, QA|Testing|**DONE (2025-10-11)** – Connector test harness now rebuilds `FakeTimeProvider`, wires `AddSourceCommon`, and drives canned VINCE responses across fetch→parse→map with recorded-request assertions.|
|Snapshot coverage handoff|QA|Models, Merge|**DONE (2025-10-11)** – Fixtures + request/advisory snapshots refreshed, README documents `UPDATE_CERTCC_FIXTURES` workflow, and recorded-request ordering is enforced for QA handoff.|
|FEEDCONN-CERTCC-02-010 Partial-detail graceful degradation|BE-Conn-CERTCC|Connector plan|**DONE (2025-10-12)** – Detail fetch now catches 404/410/403 responses for optional endpoints, logs missing bundles, feeds empty payloads into parsing, and ships regression coverage for mixed responses.|
|FEEDCONN-CERTCC-02-012 Schema sync & snapshot regen follow-up|QA, BE-Conn-CERTCC|Models `FEEDMODELS-SCHEMA-01-001`/`-002`/`-003`, Storage `FEEDSTORAGE-DATA-02-001`|**DONE (2025-10-12)** – Snapshot suite rerun, fixtures updated, and handoff notes (`FEEDCONN-CERTCC-02-012_HANDOFF.md`) document normalizedVersions/provenance expectations for Merge backfill.|
|Telemetry & documentation|DevEx|Docs|**DONE (2025-10-12)** – `CertCcDiagnostics` now publishes summary/detail/parse/map metrics, README documents meter names, and structured logging guidance is captured for Ops handoff.|

17
src/Concelier/__Libraries/StellaOps.Concelier.Connector.CertCc/TASKS.md
View File

@@ -1,14 +1,3 @@
# TASKS
| Task | Owner(s) | Depends on | Notes |
|---|---|---|---|
|Document CERT/CC advisory sources|BE-Conn-CERTCC|Research|**DONE (2025-10-10)** – Catalogued Atom feed + VINCE API endpoints and archive references in `README.md`; include polling/backoff guidance.|
|Fetch pipeline & state tracking|BE-Conn-CERTCC|Source.Common, Storage.Mongo|**DONE (2025-10-12)** – Summary planner + fetch job persist monthly/yearly VINCE JSON to `DocumentStore`, hydrate the `TimeWindowCursorState`, and snapshot regression (`dotnet test` 2025-10-12) confirmed deterministic resume behaviour.|
|VINCE note detail fetcher|BE-Conn-CERTCC|Source.Common, Storage.Mongo|**DONE (2025-10-12)** – Detail bundle fetch now enqueues VU identifiers and persists note/vendors/vuls/vendors-vuls documents with ETag/Last-Modified metadata, tolerating missing optional endpoints without wedging the cursor.|
|DTO & parser implementation|BE-Conn-CERTCC|Source.Common|**DONE (2025-10-12)** – VINCE DTO aggregate materialises note/vendor/vulnerability payloads, normalises markdown to HTML-safe fragments, and surfaces vendor impact statements covered by parser unit tests.|
|Canonical mapping & range primitives|BE-Conn-CERTCC|Models|**DONE (2025-10-12)** – Mapper emits aliases (VU#, CVE), vendor range primitives, and normalizedVersions (`scheme=certcc.vendor`) with provenance masks; `certcc-advisories.snapshot.json` validates canonical output after schema sync.|
|Deterministic fixtures/tests|QA|Testing|**DONE (2025-10-11)** – Snapshot harness regenerated (`certcc-*.snapshot.json`), request ordering assertions added, and `UPDATE_CERTCC_FIXTURES` workflow verified for CI determinism.|
|Connector test harness remediation|BE-Conn-CERTCC, QA|Testing|**DONE (2025-10-11)** – Connector test harness now rebuilds `FakeTimeProvider`, wires `AddSourceCommon`, and drives canned VINCE responses across fetch→parse→map with recorded-request assertions.|
|Snapshot coverage handoff|QA|Models, Merge|**DONE (2025-10-11)** – Fixtures + request/advisory snapshots refreshed, README documents `UPDATE_CERTCC_FIXTURES` workflow, and recorded-request ordering is enforced for QA handoff.|
|FEEDCONN-CERTCC-02-010 Partial-detail graceful degradation|BE-Conn-CERTCC|Connector plan|**DONE (2025-10-12)** – Detail fetch now catches 404/410/403 responses for optional endpoints, logs missing bundles, feeds empty payloads into parsing, and ships regression coverage for mixed responses.|
|FEEDCONN-CERTCC-02-012 Schema sync & snapshot regen follow-up|QA, BE-Conn-CERTCC|Models `FEEDMODELS-SCHEMA-01-001`/`-002`/`-003`, Storage `FEEDSTORAGE-DATA-02-001`|**DONE (2025-10-12)** – Snapshot suite rerun, fixtures updated, and handoff notes (`FEEDCONN-CERTCC-02-012_HANDOFF.md`) document normalizedVersions/provenance expectations for Merge backfill.|
|Telemetry & documentation|DevEx|Docs|**DONE (2025-10-12)** – `CertCcDiagnostics` now publishes summary/detail/parse/map metrics, README documents meter names, and structured logging guidance is captured for Ops handoff.|
# TASKS
| Task | Owner(s) | Depends on | Notes |
|---|---|---|---|

18
src/Concelier/__Libraries/StellaOps.Concelier.Connector.CertFr/TASKS.completed.md Normal file
View File

@@ -0,0 +1,18 @@
# Completed Tasks
|RSS/list fetcher with sliding window|BE-Conn-CertFr|Source.Common|**DONE** – RSS/list ingestion implemented with sliding date cursor.|
|Detail page fetch and sanitizer|BE-Conn-CertFr|Source.Common|**DONE** – HTML sanitizer trims boilerplate prior to DTO mapping.|
|Extractor and schema validation of DTO|BE-Conn-CertFr, QA|Source.Common|**DONE** – DTO parsing validates structure before persistence.|
|Canonical mapping (aliases, refs, severity text)|BE-Conn-CertFr|Models|**DONE** – mapper emits enrichment references with severity text.|
|Watermark plus dedupe by sha256|BE-Conn-CertFr|Storage.Mongo|**DONE** – SHA comparisons skip unchanged docs; covered by duplicate/not-modified connector tests.|
|Golden fixtures and determinism tests|QA|Source.CertFr|**DONE** – snapshot fixtures added in `CertFrConnectorTests` to enforce deterministic output.|
|Mark failure/backoff on fetch errors|BE-Conn-CertFr|Storage.Mongo|**DONE** – fetch path now marks failures/backoff and tests assert state repository updates.|
|Conditional fetch caching|BE-Conn-CertFr|Source.Common|**DONE** – ETag/Last-Modified support wired via `SourceFetchService` and verified in not-modified test.|

14
src/Concelier/__Libraries/StellaOps.Concelier.Connector.CertFr/TASKS.md
View File

@@ -1,11 +1,3 @@
# TASKS
| Task | Owner(s) | Depends on | Notes |
|---|---|---|---|
|RSS/list fetcher with sliding window|BE-Conn-CertFr|Source.Common|**DONE** – RSS/list ingestion implemented with sliding date cursor.|
|Detail page fetch and sanitizer|BE-Conn-CertFr|Source.Common|**DONE** – HTML sanitizer trims boilerplate prior to DTO mapping.|
|Extractor and schema validation of DTO|BE-Conn-CertFr, QA|Source.Common|**DONE** – DTO parsing validates structure before persistence.|
|Canonical mapping (aliases, refs, severity text)|BE-Conn-CertFr|Models|**DONE** – mapper emits enrichment references with severity text.|
|Watermark plus dedupe by sha256|BE-Conn-CertFr|Storage.Mongo|**DONE** – SHA comparisons skip unchanged docs; covered by duplicate/not-modified connector tests.|
|Golden fixtures and determinism tests|QA|Source.CertFr|**DONE** – snapshot fixtures added in `CertFrConnectorTests` to enforce deterministic output.|
|Mark failure/backoff on fetch errors|BE-Conn-CertFr|Storage.Mongo|**DONE** – fetch path now marks failures/backoff and tests assert state repository updates.|
|Conditional fetch caching|BE-Conn-CertFr|Source.Common|**DONE** – ETag/Last-Modified support wired via `SourceFetchService` and verified in not-modified test.|
# TASKS
| Task | Owner(s) | Depends on | Notes |
|---|---|---|---|

16
src/Concelier/__Libraries/StellaOps.Concelier.Connector.CertIn/TASKS.completed.md Normal file
View File

@@ -0,0 +1,16 @@
# Completed Tasks
|Index/detail crawler with windowing|BE-Conn-CertIn|Source.Common|**DONE** – index/detail fetch implemented with window overlap and pagination.|
|Extractor (title/CVEs/mitigation)|BE-Conn-CertIn|Source.Common|**DONE** – parser normalizes encodings, CVE lists, and mitigation text.|
|DTO validation and sanitizer|BE-Conn-CertIn, QA|Source.Common|**DONE** – HTML sanitizer produces DTO before persistence.|
|Canonical mapping (aliases, refs)|BE-Conn-CertIn|Models|**DONE** – mapper creates CERT-IN aliases plus typed references.|
|State/dedupe and fixtures|BE-Conn-CertIn, QA|Storage.Mongo|**DONE** – snapshot/resume tests cover dedupe and cursor handling.|
|Mark failure/backoff on fetch errors|BE-Conn-CertIn|Storage.Mongo|**DONE** – fetch pipeline marks failures/backoff with unit coverage.|
|Conditional fetch caching|BE-Conn-CertIn|Source.Common|**DONE** – connector reuses ETag/Last-Modified; tests verify not-modified flow.|

13
src/Concelier/__Libraries/StellaOps.Concelier.Connector.CertIn/TASKS.md
View File

@@ -1,10 +1,3 @@
# TASKS
| Task | Owner(s) | Depends on | Notes |
|---|---|---|---|
|Index/detail crawler with windowing|BE-Conn-CertIn|Source.Common|**DONE** – index/detail fetch implemented with window overlap and pagination.|
|Extractor (title/CVEs/mitigation)|BE-Conn-CertIn|Source.Common|**DONE** – parser normalizes encodings, CVE lists, and mitigation text.|
|DTO validation and sanitizer|BE-Conn-CertIn, QA|Source.Common|**DONE** – HTML sanitizer produces DTO before persistence.|
|Canonical mapping (aliases, refs)|BE-Conn-CertIn|Models|**DONE** – mapper creates CERT-IN aliases plus typed references.|
|State/dedupe and fixtures|BE-Conn-CertIn, QA|Storage.Mongo|**DONE** – snapshot/resume tests cover dedupe and cursor handling.|
|Mark failure/backoff on fetch errors|BE-Conn-CertIn|Storage.Mongo|**DONE** – fetch pipeline marks failures/backoff with unit coverage.|
|Conditional fetch caching|BE-Conn-CertIn|Source.Common|**DONE** – connector reuses ETag/Last-Modified; tests verify not-modified flow.|
# TASKS
| Task | Owner(s) | Depends on | Notes |
|---|---|---|---|

32
src/Concelier/__Libraries/StellaOps.Concelier.Connector.Common/TASKS.completed.md Normal file
View File

@@ -0,0 +1,32 @@
# Completed Tasks
|Register source HTTP clients with allowlists and timeouts|BE-Conn-Shared|Source.Common|**DONE** – `AddSourceHttpClient` wires named clients with host allowlists/timeouts.|
|Implement retry/backoff with jitter and 429 handling|BE-Conn-Shared|Source.Common|**DONE** – `SourceRetryPolicy` retries with 429/5xx handling and exponential backoff.|
|Conditional GET helpers (ETag/Last-Modified)|BE-Conn-Shared|Source.Common|**DONE** – `SourceFetchRequest` + fetch result propagate etag/last-modified for NotModified handling.|
|Windowed cursor and pagination utilities|BE-Conn-Shared|Source.Common|**DONE** – `TimeWindowCursorPlanner` + `PaginationPlanner` centralize sliding windows and additional page indices.|
|JSON/XML schema validators with rich errors|BE-Conn-Shared, QA|Source.Common|DONE – JsonSchemaValidator surfaces keyword/path/message details + tests.|
|Raw document capture helper|BE-Conn-Shared|Storage.Mongo|**DONE** – `SourceFetchService` stores raw payload + headers with sha256 metadata.|
|Canned HTTP test harness|QA|Source.Common|DONE – enriched `CannedHttpMessageHandler` with method-aware queues, request capture, fallbacks, and helpers + unit coverage.|
|HTML sanitization and URL normalization utilities|BE-Conn-Shared|Source.Common|DONE – `HtmlContentSanitizer` + `UrlNormalizer` provide safe fragments and canonical links for connectors.|
|PDF-to-text sandbox helper|BE-Conn-Shared|Source.Common|DONE – `PdfTextExtractor` uses PdfPig to yield deterministic text with options + tests.|
|PURL and SemVer helper library|BE-Conn-Shared|Models|DONE – `PackageCoordinateHelper` exposes normalized purl + SemVer parsing utilities backed by normalization.|
|Telemetry wiring (logs/metrics/traces)|BE-Conn-Shared|Observability|DONE – `SourceDiagnostics` emits Activity/Meter signals integrated into fetch pipeline and WebService OTEL setup.|
|Shared jitter source in retry policy|BE-Conn-Shared|Source.Common|**DONE** – `SourceRetryPolicy` now consumes injected `CryptoJitterSource` for thread-safe jitter.|
|Allow per-request Accept header overrides|BE-Conn-Shared|Source.Common|**DONE** – `SourceFetchRequest.AcceptHeaders` honored by `SourceFetchService` plus unit tests for overrides.|
|FEEDCONN-SHARED-HTTP2-001 HTTP version fallback policy|BE-Conn-Shared, Source.Common|Source.Common|**DONE (2025-10-11)** – `AddSourceHttpClient` now honours per-connector HTTP version/ policy, exposes handler customisation, and defaults to downgrade-friendly settings; unit tests cover handler configuration hook.|
|FEEDCONN-SHARED-TLS-001 Sovereign trust store support|BE-Conn-Shared, Ops|Source.Common|**DONE (2025-10-11)** – `SourceHttpClientOptions` now exposes `TrustedRootCertificates`, `ServerCertificateCustomValidation`, and `AllowInvalidServerCertificates`, and `AddSourceHttpClient` runs the shared configuration binder so connectors can pull `concelier:httpClients|sources:<name>:http` settings (incl. Offline Kit relative PEM paths via `concelier:offline:root`). Tests cover handler wiring. Ops follow-up: package RU trust roots for Offline Kit distribution.|

21
src/Concelier/__Libraries/StellaOps.Concelier.Connector.Common/TASKS.md
View File

@@ -1,19 +1,4 @@
# TASKS
| Task | Owner(s) | Depends on | Notes |
|---|---|---|---|
|Register source HTTP clients with allowlists and timeouts|BE-Conn-Shared|Source.Common|**DONE** – `AddSourceHttpClient` wires named clients with host allowlists/timeouts.|
|Implement retry/backoff with jitter and 429 handling|BE-Conn-Shared|Source.Common|**DONE** – `SourceRetryPolicy` retries with 429/5xx handling and exponential backoff.|
|Conditional GET helpers (ETag/Last-Modified)|BE-Conn-Shared|Source.Common|**DONE** – `SourceFetchRequest` + fetch result propagate etag/last-modified for NotModified handling.|
|Windowed cursor and pagination utilities|BE-Conn-Shared|Source.Common|**DONE** – `TimeWindowCursorPlanner` + `PaginationPlanner` centralize sliding windows and additional page indices.|
|JSON/XML schema validators with rich errors|BE-Conn-Shared, QA|Source.Common|DONE – JsonSchemaValidator surfaces keyword/path/message details + tests.|
|Raw document capture helper|BE-Conn-Shared|Storage.Mongo|**DONE** – `SourceFetchService` stores raw payload + headers with sha256 metadata.|
|Canned HTTP test harness|QA|Source.Common|DONE – enriched `CannedHttpMessageHandler` with method-aware queues, request capture, fallbacks, and helpers + unit coverage.|
|HTML sanitization and URL normalization utilities|BE-Conn-Shared|Source.Common|DONE – `HtmlContentSanitizer` + `UrlNormalizer` provide safe fragments and canonical links for connectors.|
|PDF-to-text sandbox helper|BE-Conn-Shared|Source.Common|DONE – `PdfTextExtractor` uses PdfPig to yield deterministic text with options + tests.|
|PURL and SemVer helper library|BE-Conn-Shared|Models|DONE – `PackageCoordinateHelper` exposes normalized purl + SemVer parsing utilities backed by normalization.|
|Telemetry wiring (logs/metrics/traces)|BE-Conn-Shared|Observability|DONE – `SourceDiagnostics` emits Activity/Meter signals integrated into fetch pipeline and WebService OTEL setup.|
|Shared jitter source in retry policy|BE-Conn-Shared|Source.Common|**DONE** – `SourceRetryPolicy` now consumes injected `CryptoJitterSource` for thread-safe jitter.|
|Allow per-request Accept header overrides|BE-Conn-Shared|Source.Common|**DONE** – `SourceFetchRequest.AcceptHeaders` honored by `SourceFetchService` plus unit tests for overrides.|
|FEEDCONN-SHARED-HTTP2-001 HTTP version fallback policy|BE-Conn-Shared, Source.Common|Source.Common|**DONE (2025-10-11)** – `AddSourceHttpClient` now honours per-connector HTTP version/ policy, exposes handler customisation, and defaults to downgrade-friendly settings; unit tests cover handler configuration hook.|
|FEEDCONN-SHARED-TLS-001 Sovereign trust store support|BE-Conn-Shared, Ops|Source.Common|**DONE (2025-10-11)** – `SourceHttpClientOptions` now exposes `TrustedRootCertificates`, `ServerCertificateCustomValidation`, and `AllowInvalidServerCertificates`, and `AddSourceHttpClient` runs the shared configuration binder so connectors can pull `concelier:httpClients|sources:<name>:http` settings (incl. Offline Kit relative PEM paths via `concelier:offline:root`). Tests cover handler wiring. Ops follow-up: package RU trust roots for Offline Kit distribution.|
# TASKS
| Task | Owner(s) | Depends on | Notes |
|---|---|---|---|
|FEEDCONN-SHARED-STATE-003 Source state seeding helper|Tools Guild, BE-Conn-MSRC|Tools|**DOING (2025-10-19)** – Provide a reusable CLI/utility to seed `pendingDocuments`/`pendingMappings` for connectors (MSRC backfills require scripted CVRF + detail injection). Coordinate with MSRC team for expected JSON schema and handoff once prototype lands. Prereqs confirmed none (2025-10-19).|

20
src/Concelier/__Libraries/StellaOps.Concelier.Connector.Cve/TASKS.completed.md Normal file
View File

@@ -0,0 +1,20 @@
# Completed Tasks
|Define CVE data source + contract|BE-Conn-CVE|Research|**DONE (2025-10-10)** – Connector targets the CVE Services JSON 5 API with authenticated windowed queries documented in `CveOptions` (`CVE-API-*` headers, pagination semantics, failure backoff).|
|Fetch/cursor implementation|BE-Conn-CVE|Source.Common, Storage.Mongo|**DONE (2025-10-10)** – Time-window + page-aware cursor with SourceFetchService fetching list/detail pairs, resumable state persisted via `CveCursor`.|
|DTOs & parser|BE-Conn-CVE|Source.Common|**DONE (2025-10-10)** – `CveRecordParser` and DTOs capture aliases, references, metrics, vendor ranges; sanitises text and timestamps.|
|Canonical mapping & range primitives|BE-Conn-CVE|Models|**DONE (2025-10-10)** – `CveMapper` emits canonical advisories, vendor range primitives, SemVer/range statuses, references, CVSS normalization.<br>2025-10-11 research trail: confirm subsequent MR adds `NormalizedVersions` shaped like `[{"scheme":"semver","type":"range","min":"<min>","minInclusive":true,"max":"<max>","maxInclusive":false,"notes":"nvd:CVE-2025-XXXX"}]` so storage provenance joins continue to work.|
|Deterministic tests & fixtures|QA|Testing|**DONE (2025-10-10)** – Added `StellaOps.Concelier.Connector.Cve.Tests` harness with canned fixtures + snapshot regression covering fetch/parse/map.|
|Observability & docs|DevEx|Docs|**DONE (2025-10-10)** – Diagnostics meter (`cve.fetch.*`, etc.) wired; options/usage documented via `CveServiceCollectionExtensions`.|
|Operator rollout playbook|BE-Conn-CVE, Ops|Docs|**DONE (2025-10-12)** – Refreshed `docs/modules/concelier/operations/connectors/cve-kev.md` with credential checklist, smoke book, PromQL guardrails, and linked Grafana pack (`docs/modules/concelier/operations/connectors/cve-kev-grafana-dashboard.json`).|
|Live smoke & monitoring|QA, BE-Conn-CVE|WebService, Observability|**DONE (2025-10-15)** – Executed connector harness smoke using CVE Services sample window (CVE-2024-0001), confirmed fetch/parse/map telemetry (`cve.fetch.*`, `cve.map.success`) all incremented once, and archived the summary log + Grafana import guidance in `docs/modules/concelier/operations/connectors/cve-kev.md` (“Staging smoke 2025-10-15”).|
|FEEDCONN-CVE-02-003 Normalized versions rollout|BE-Conn-CVE|Models `FEEDMODELS-SCHEMA-01-003`, Normalization playbook|**DONE (2025-10-12)** – Confirmed SemVer primitives map to normalized rules with `cve:{cveId}:{identifier}` notes and refreshed snapshots; `dotnet test src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Cve.Tests` passes on net10 preview.|

9
src/Concelier/__Libraries/StellaOps.Concelier.Connector.Cve/TASKS.md
View File

@@ -1,12 +1,3 @@
# TASKS
| Task | Owner(s) | Depends on | Notes |
|---|---|---|---|
|Define CVE data source + contract|BE-Conn-CVE|Research|**DONE (2025-10-10)** – Connector targets the CVE Services JSON 5 API with authenticated windowed queries documented in `CveOptions` (`CVE-API-*` headers, pagination semantics, failure backoff).|
|Fetch/cursor implementation|BE-Conn-CVE|Source.Common, Storage.Mongo|**DONE (2025-10-10)** – Time-window + page-aware cursor with SourceFetchService fetching list/detail pairs, resumable state persisted via `CveCursor`.|
|DTOs & parser|BE-Conn-CVE|Source.Common|**DONE (2025-10-10)** – `CveRecordParser` and DTOs capture aliases, references, metrics, vendor ranges; sanitises text and timestamps.|
|Canonical mapping & range primitives|BE-Conn-CVE|Models|**DONE (2025-10-10)** – `CveMapper` emits canonical advisories, vendor range primitives, SemVer/range statuses, references, CVSS normalization.<br>2025-10-11 research trail: confirm subsequent MR adds `NormalizedVersions` shaped like `[{"scheme":"semver","type":"range","min":"<min>","minInclusive":true,"max":"<max>","maxInclusive":false,"notes":"nvd:CVE-2025-XXXX"}]` so storage provenance joins continue to work.|
|Deterministic tests & fixtures|QA|Testing|**DONE (2025-10-10)** – Added `StellaOps.Concelier.Connector.Cve.Tests` harness with canned fixtures + snapshot regression covering fetch/parse/map.|
|Observability & docs|DevEx|Docs|**DONE (2025-10-10)** – Diagnostics meter (`cve.fetch.*`, etc.) wired; options/usage documented via `CveServiceCollectionExtensions`.|
|Operator rollout playbook|BE-Conn-CVE, Ops|Docs|**DONE (2025-10-12)** – Refreshed `docs/modules/concelier/operations/connectors/cve-kev.md` with credential checklist, smoke book, PromQL guardrails, and linked Grafana pack (`docs/modules/concelier/operations/connectors/cve-kev-grafana-dashboard.json`).|
|Live smoke & monitoring|QA, BE-Conn-CVE|WebService, Observability|**DONE (2025-10-15)** – Executed connector harness smoke using CVE Services sample window (CVE-2024-0001), confirmed fetch/parse/map telemetry (`cve.fetch.*`, `cve.map.success`) all incremented once, and archived the summary log + Grafana import guidance in `docs/modules/concelier/operations/connectors/cve-kev.md` (“Staging smoke 2025-10-15”).|
|FEEDCONN-CVE-02-003 Normalized versions rollout|BE-Conn-CVE|Models `FEEDMODELS-SCHEMA-01-003`, Normalization playbook|**DONE (2025-10-12)** – Confirmed SemVer primitives map to normalized rules with `cve:{cveId}:{identifier}` notes and refreshed snapshots; `dotnet test src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Cve.Tests` passes on net10 preview.|

26
src/Concelier/__Libraries/StellaOps.Concelier.Connector.Distro.RedHat/TASKS.completed.md Normal file
View File

@@ -0,0 +1,26 @@
# Completed Tasks
|Hydra fetch with after= cursor|BE-Conn-RH|Source.Common|**DONE** – windowed paging with overlap, ETag/Last-Modified persisted.|
|DTOs for Security Data + OVAL|BE-Conn-RH|Tests|**DONE** – CSAF payloads serialized into `redhat.csaf.v2` DTOs.|
|NEVRA parser/comparer (complete)|BE-Conn-RH|Models|**DONE** – parser/comparer shipped with coverage; add edge cases as needed.|
|Mapper to canonical rpm/cpe affected|BE-Conn-RH|Models|**DONE** – maps fixed/known ranges, CPE provenance, status ranges.|
|Job scheduler registration aligns with Options pipeline|BE-Conn-RH|Core|**DONE** – registered fetch/parse/map via JobSchedulerBuilder, preserving option overrides and tightening cron/timeouts.|
|Watermark persistence + resume|BE-Conn-RH|Storage.Mongo|**DONE** – cursor updates via SourceStateRepository.|
|Precedence tests vs NVD|QA|Merge|**DONE** – Added AffectedPackagePrecedenceResolver + tests ensuring Red Hat CPEs override NVD ranges.|
|Golden mapping fixtures|QA|Fixtures|**DONE** – fixture validation test now snapshots RHSA-2025:0001/0002/0003 with env-driven regeneration.|
|Job scheduling defaults for source:redhat tasks|BE-Core|JobScheduler|**DONE** – Cron windows + per-job timeouts defined for fetch/parse/map.|
|Express unaffected/investigation statuses without overloading range fields|BE-Conn-RH|Models|**DONE** – Introduced AffectedPackageStatus collection and updated mapper/tests.|
|Reference dedupe & ordering in mapper|BE-Conn-RH|Models|DONE – mapper consolidates by URL, merges metadata, deterministic ordering validated in tests.|
|Hydra summary fetch through SourceFetchService|BE-Conn-RH|Source.Common|DONE – summary pages now fetched via SourceFetchService with cache + conditional headers.|

18
src/Concelier/__Libraries/StellaOps.Concelier.Connector.Distro.RedHat/TASKS.md
View File

@@ -1,16 +1,4 @@
# TASKS
| Task | Owner(s) | Depends on | Notes |
|---|---|---|---|
|Hydra fetch with after= cursor|BE-Conn-RH|Source.Common|**DONE** – windowed paging with overlap, ETag/Last-Modified persisted.|
|DTOs for Security Data + OVAL|BE-Conn-RH|Tests|**DONE** – CSAF payloads serialized into `redhat.csaf.v2` DTOs.|
|NEVRA parser/comparer (complete)|BE-Conn-RH|Models|**DONE** – parser/comparer shipped with coverage; add edge cases as needed.|
|Mapper to canonical rpm/cpe affected|BE-Conn-RH|Models|**DONE** – maps fixed/known ranges, CPE provenance, status ranges.|
|Job scheduler registration aligns with Options pipeline|BE-Conn-RH|Core|**DONE** – registered fetch/parse/map via JobSchedulerBuilder, preserving option overrides and tightening cron/timeouts.|
|Watermark persistence + resume|BE-Conn-RH|Storage.Mongo|**DONE** – cursor updates via SourceStateRepository.|
|Precedence tests vs NVD|QA|Merge|**DONE** – Added AffectedPackagePrecedenceResolver + tests ensuring Red Hat CPEs override NVD ranges.|
|Golden mapping fixtures|QA|Fixtures|**DONE** – fixture validation test now snapshots RHSA-2025:0001/0002/0003 with env-driven regeneration.|
|Job scheduling defaults for source:redhat tasks|BE-Core|JobScheduler|**DONE** – Cron windows + per-job timeouts defined for fetch/parse/map.|
|Express unaffected/investigation statuses without overloading range fields|BE-Conn-RH|Models|**DONE** – Introduced AffectedPackageStatus collection and updated mapper/tests.|
|Reference dedupe & ordering in mapper|BE-Conn-RH|Models|DONE – mapper consolidates by URL, merges metadata, deterministic ordering validated in tests.|
|Hydra summary fetch through SourceFetchService|BE-Conn-RH|Source.Common|DONE – summary pages now fetched via SourceFetchService with cache + conditional headers.|
# TASKS
| Task | Owner(s) | Depends on | Notes |
|---|---|---|---|
|Fixture validation sweep|QA|None|**DOING (2025-10-19)** – Prereqs confirmed none; continuing RHSA fixture regeneration and diff review alongside mapper provenance updates.<br>2025-10-29: Added `scripts/update-redhat-fixtures.sh` to regenerate golden snapshots with `UPDATE_GOLDENS=1`; run it before reviews to capture CSAF contract deltas.|

10
src/Concelier/__Libraries/StellaOps.Concelier.Connector.Distro.Ubuntu/TASKS.completed.md Normal file
View File

@@ -0,0 +1,10 @@
# Completed Tasks
| Task | Status | Notes |
|---|---|---|
|Discover data model & pagination for `notices.json`|DONE|Connector now walks `offset`/`limit` pages (configurable page size) until MaxNoticesPerFetch satisfied, reusing cached pages when unchanged.|
|Design cursor & state model|DONE|Cursor tracks last published timestamp plus processed USN identifiers with overlap logic.|
|Implement fetch/parse pipeline|DONE|Index fetch hydrates per-notice DTOs, stores metadata, and maps without dedicated detail fetches.|
|Emit RangePrimitives + telemetry|DONE|Each package emits EVR primitives with `ubuntu.release` and `ubuntu.pocket` extensions for dashboards.|
|Add integration tests|DONE|Fixture-driven fetch→map suite covers resolved and ESM pockets, including conditional GET behaviour.|
|NormalizedVersions rollout|DONE (2025-10-11)|EVR ranges now project `normalizedVersions` with `ubuntu:<release>` notes; tests assert canonical rule emission.|

14
src/Concelier/__Libraries/StellaOps.Concelier.Connector.Distro.Ubuntu/TASKS.md
View File

@@ -1,10 +1,4 @@
# Ubuntu Connector TODOs
| Task | Status | Notes |
|---|---|---|
|Discover data model & pagination for `notices.json`|DONE|Connector now walks `offset`/`limit` pages (configurable page size) until MaxNoticesPerFetch satisfied, reusing cached pages when unchanged.|
|Design cursor & state model|DONE|Cursor tracks last published timestamp plus processed USN identifiers with overlap logic.|
|Implement fetch/parse pipeline|DONE|Index fetch hydrates per-notice DTOs, stores metadata, and maps without dedicated detail fetches.|
|Emit RangePrimitives + telemetry|DONE|Each package emits EVR primitives with `ubuntu.release` and `ubuntu.pocket` extensions for dashboards.|
|Add integration tests|DONE|Fixture-driven fetch→map suite covers resolved and ESM pockets, including conditional GET behaviour.|
|NormalizedVersions rollout|DONE (2025-10-11)|EVR ranges now project `normalizedVersions` with `ubuntu:<release>` notes; tests assert canonical rule emission.|
# Ubuntu Connector TODOs
| Task | Status | Notes |
|---|---|---|

34
src/Concelier/__Libraries/StellaOps.Concelier.Connector.Ghsa/TASKS.completed.md Normal file
View File

@@ -0,0 +1,34 @@
# Completed Tasks
|Select GHSA data source & auth model|BE-Conn-GHSA|Research|**DONE (2025-10-10)** – Adopted GitHub Security Advisories REST (global) endpoint with bearer token + API version headers documented in `GhsaOptions`.|
|Fetch pipeline & state management|BE-Conn-GHSA|Source.Common, Storage.Mongo|**DONE (2025-10-10)** – Implemented list/detail fetch using `GhsaCursor` (time window + page), resumable SourceState and backoff controls.|
|DTO & parser implementation|BE-Conn-GHSA|Source.Common|**DONE (2025-10-10)** – Added `GhsaRecordParser`/DTOs extracting aliases, references, severity, vulnerable ranges, patched versions.|
|Canonical mapping & range primitives|BE-Conn-GHSA|Models|**DONE (2025-10-10)** – `GhsaMapper` emits GHSA advisories with SemVer packages, vendor extensions (ecosystem/package) and deterministic references.<br>2025-10-11 research trail: upcoming normalized array should follow `[{"scheme":"semver","type":"range","min":"<min>","minInclusive":true,"max":"<max>","maxInclusive":false,"notes":"ghsa:GHSA-xxxx"}]`; include patched-only advisories as `lt`/`lte` when no explicit floor.|
|Deterministic fixtures & tests|QA|Testing|**DONE (2025-10-10)** – New `StellaOps.Concelier.Connector.Ghsa.Tests` regression covers fetch/parse/map via canned GHSA fixtures and snapshot assertions.|
|Telemetry & documentation|DevEx|Docs|**DONE (2025-10-10)** – Diagnostics meter (`ghsa.fetch.*`) wired; DI extension documents token/headers and job registrations.|
|GitHub quota monitoring & retries|BE-Conn-GHSA, Observability|Source.Common|**DONE (2025-10-12)** – Rate-limit metrics/logs added, retry/backoff handles 403 secondary limits, and ops runbook documents dashboards + mitigation steps.|
|Production credential & scheduler rollout|Ops, BE-Conn-GHSA|Docs, WebService|**DONE (2025-10-12)** – Scheduler defaults registered via `JobSchedulerBuilder`, credential provisioning documented (Compose/Helm samples), and staged backfill guidance captured in `docs/modules/concelier/operations/connectors/ghsa.md`.|
|FEEDCONN-GHSA-04-002 Conflict regression fixtures|BE-Conn-GHSA, QA|Merge `FEEDMERGE-ENGINE-04-001`|**DONE (2025-10-12)** – Added `conflict-ghsa.canonical.json` + `GhsaConflictFixtureTests`; SemVer ranges and credits align with merge precedence triple and shareable with QA. Validation: `dotnet test src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Ghsa.Tests/StellaOps.Concelier.Connector.Ghsa.Tests.csproj --filter GhsaConflictFixtureTests`.|
|FEEDCONN-GHSA-02-004 GHSA credits & ecosystem severity mapping|BE-Conn-GHSA|Models `FEEDMODELS-SCHEMA-01-002`|**DONE (2025-10-11)** – Mapper emits advisory credits with provenance masks, fixtures assert role/contact ordering, and severity normalization remains unchanged.|
|FEEDCONN-GHSA-02-007 Credit parity regression fixtures|BE-Conn-GHSA, QA|Source.Nvd, Source.Osv|**DONE (2025-10-12)** – Parity fixtures regenerated via `src/Tools/FixtureUpdater`, normalized SemVer notes verified against GHSA/NVD/OSV snapshots, and the fixtures guide now documents the headroom checks.|
|FEEDCONN-GHSA-02-001 Normalized versions rollout|BE-Conn-GHSA|Models `FEEDMODELS-SCHEMA-01-003`, Normalization playbook|**DONE (2025-10-11)** – GHSA mapper now emits SemVer primitives + normalized ranges, fixtures refreshed, connector tests passing; report logged via FEEDMERGE-COORD-02-900.|
|FEEDCONN-GHSA-02-005 Quota monitoring hardening|BE-Conn-GHSA, Observability|Source.Common metrics|**DONE (2025-10-12)** – Diagnostics expose headroom histograms/gauges, warning logs dedupe below the configured threshold, and the ops runbook gained alerting and mitigation guidance.|
|FEEDCONN-GHSA-02-006 Scheduler rollout integration|BE-Conn-GHSA, Ops|Job scheduler|**DONE (2025-10-12)** – Dependency routine tests assert cron/timeouts, and the runbook highlights cron overrides plus backoff toggles for staged rollouts.|
|FEEDCONN-GHSA-04-003 Description/CWE/metric parity rollout|BE-Conn-GHSA|Models, Core|**DONE (2025-10-15)** – Mapper emits advisory description, CWE weaknesses, and canonical CVSS metric id with updated fixtures (`osv-ghsa.osv.json` parity suite) and connector regression covers the new fields. Reported completion to Merge coordination.|
|FEEDCONN-GHSA-04-004 Canonical metric fallback coverage|BE-Conn-GHSA|Models, Merge|**DONE (2025-10-16)** – Ensure canonical metric ids remain populated when GitHub omits CVSS vectors/scores; add fixtures capturing severity-only advisories, document precedence with Merge, and emit analytics to track fallback usage.<br>2025-10-16: Mapper now emits `ghsa:severity/<level>` canonical ids when vectors are missing, diagnostics expose `ghsa.map.canonical_metric_fallbacks`, conflict/mapper fixtures updated, and runbook documents Merge precedence. Tests: `dotnet test src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Ghsa.Tests/StellaOps.Concelier.Connector.Ghsa.Tests.csproj`.|

16
src/Concelier/__Libraries/StellaOps.Concelier.Connector.Ghsa/TASKS.md
View File

@@ -1,19 +1,3 @@
# TASKS
| Task | Owner(s) | Depends on | Notes |
|---|---|---|---|
|Select GHSA data source & auth model|BE-Conn-GHSA|Research|**DONE (2025-10-10)** – Adopted GitHub Security Advisories REST (global) endpoint with bearer token + API version headers documented in `GhsaOptions`.|
|Fetch pipeline & state management|BE-Conn-GHSA|Source.Common, Storage.Mongo|**DONE (2025-10-10)** – Implemented list/detail fetch using `GhsaCursor` (time window + page), resumable SourceState and backoff controls.|
|DTO & parser implementation|BE-Conn-GHSA|Source.Common|**DONE (2025-10-10)** – Added `GhsaRecordParser`/DTOs extracting aliases, references, severity, vulnerable ranges, patched versions.|
|Canonical mapping & range primitives|BE-Conn-GHSA|Models|**DONE (2025-10-10)** – `GhsaMapper` emits GHSA advisories with SemVer packages, vendor extensions (ecosystem/package) and deterministic references.<br>2025-10-11 research trail: upcoming normalized array should follow `[{"scheme":"semver","type":"range","min":"<min>","minInclusive":true,"max":"<max>","maxInclusive":false,"notes":"ghsa:GHSA-xxxx"}]`; include patched-only advisories as `lt`/`lte` when no explicit floor.|
|Deterministic fixtures & tests|QA|Testing|**DONE (2025-10-10)** – New `StellaOps.Concelier.Connector.Ghsa.Tests` regression covers fetch/parse/map via canned GHSA fixtures and snapshot assertions.|
|Telemetry & documentation|DevEx|Docs|**DONE (2025-10-10)** – Diagnostics meter (`ghsa.fetch.*`) wired; DI extension documents token/headers and job registrations.|
|GitHub quota monitoring & retries|BE-Conn-GHSA, Observability|Source.Common|**DONE (2025-10-12)** – Rate-limit metrics/logs added, retry/backoff handles 403 secondary limits, and ops runbook documents dashboards + mitigation steps.|
|Production credential & scheduler rollout|Ops, BE-Conn-GHSA|Docs, WebService|**DONE (2025-10-12)** – Scheduler defaults registered via `JobSchedulerBuilder`, credential provisioning documented (Compose/Helm samples), and staged backfill guidance captured in `docs/modules/concelier/operations/connectors/ghsa.md`.|
|FEEDCONN-GHSA-04-002 Conflict regression fixtures|BE-Conn-GHSA, QA|Merge `FEEDMERGE-ENGINE-04-001`|**DONE (2025-10-12)** – Added `conflict-ghsa.canonical.json` + `GhsaConflictFixtureTests`; SemVer ranges and credits align with merge precedence triple and shareable with QA. Validation: `dotnet test src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Ghsa.Tests/StellaOps.Concelier.Connector.Ghsa.Tests.csproj --filter GhsaConflictFixtureTests`.|
|FEEDCONN-GHSA-02-004 GHSA credits & ecosystem severity mapping|BE-Conn-GHSA|Models `FEEDMODELS-SCHEMA-01-002`|**DONE (2025-10-11)** – Mapper emits advisory credits with provenance masks, fixtures assert role/contact ordering, and severity normalization remains unchanged.|
|FEEDCONN-GHSA-02-007 Credit parity regression fixtures|BE-Conn-GHSA, QA|Source.Nvd, Source.Osv|**DONE (2025-10-12)** – Parity fixtures regenerated via `src/Tools/FixtureUpdater`, normalized SemVer notes verified against GHSA/NVD/OSV snapshots, and the fixtures guide now documents the headroom checks.|
|FEEDCONN-GHSA-02-001 Normalized versions rollout|BE-Conn-GHSA|Models `FEEDMODELS-SCHEMA-01-003`, Normalization playbook|**DONE (2025-10-11)** – GHSA mapper now emits SemVer primitives + normalized ranges, fixtures refreshed, connector tests passing; report logged via FEEDMERGE-COORD-02-900.|
|FEEDCONN-GHSA-02-005 Quota monitoring hardening|BE-Conn-GHSA, Observability|Source.Common metrics|**DONE (2025-10-12)** – Diagnostics expose headroom histograms/gauges, warning logs dedupe below the configured threshold, and the ops runbook gained alerting and mitigation guidance.|
|FEEDCONN-GHSA-02-006 Scheduler rollout integration|BE-Conn-GHSA, Ops|Job scheduler|**DONE (2025-10-12)** – Dependency routine tests assert cron/timeouts, and the runbook highlights cron overrides plus backoff toggles for staged rollouts.|
|FEEDCONN-GHSA-04-003 Description/CWE/metric parity rollout|BE-Conn-GHSA|Models, Core|**DONE (2025-10-15)** – Mapper emits advisory description, CWE weaknesses, and canonical CVSS metric id with updated fixtures (`osv-ghsa.osv.json` parity suite) and connector regression covers the new fields. Reported completion to Merge coordination.|
|FEEDCONN-GHSA-04-004 Canonical metric fallback coverage|BE-Conn-GHSA|Models, Merge|**DONE (2025-10-16)** – Ensure canonical metric ids remain populated when GitHub omits CVSS vectors/scores; add fixtures capturing severity-only advisories, document precedence with Merge, and emit analytics to track fallback usage.<br>2025-10-16: Mapper now emits `ghsa:severity/<level>` canonical ids when vectors are missing, diagnostics expose `ghsa.map.canonical_metric_fallbacks`, conflict/mapper fixtures updated, and runbook documents Merge precedence. Tests: `dotnet test src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Ghsa.Tests/StellaOps.Concelier.Connector.Ghsa.Tests.csproj`.|

24
src/Concelier/__Libraries/StellaOps.Concelier.Connector.Ics.Cisa/TASKS.completed.md Normal file
View File

@@ -0,0 +1,24 @@
# Completed Tasks
|FEEDCONN-ICSCISA-02-001 Document CISA ICS feed contract|BE-Conn-ICS-CISA|Research|**DONE (2025-10-11)** – `https://www.cisa.gov/cybersecurity-advisories/ics-advisories.xml` and legacy `/sites/default/files/feeds/...` return Akamai 403 even with browser UA; HTML landing page blocked as well. Logged full headers (x-reference-error, AkamaiGHost) in `docs/concelier-connector-research-20251011.md` and initiated GovDelivery access request.|
|FEEDCONN-ICSCISA-02-002 Fetch pipeline & cursor storage|BE-Conn-ICS-CISA|Source.Common, Storage.Mongo|**DONE (2025-10-16)** – Confirmed proxy knobs + cursor state behave with the refreshed fixtures; ops runbook now captures proxy usage/validation so the fetch stage is production-ready.|
|FEEDCONN-ICSCISA-02-003 DTO/parser implementation|BE-Conn-ICS-CISA|Source.Common|**DONE (2025-10-16)** – Feed parser fixtures updated to retain vendor PDFs as attachments while maintaining reference coverage; console diagnostics removed.|
|FEEDCONN-ICSCISA-02-004 Canonical mapping & range primitives|BE-Conn-ICS-CISA|Models|**DONE (2025-10-16)** – `TryCreateSemVerPrimitive` flow + Mongo deserialiser now persist `exactValue` (`4.2` → `4.2.0`), unblocking canonical snapshots.|
|FEEDCONN-ICSCISA-02-005 Deterministic fixtures/tests|QA|Testing|**DONE (2025-10-16)** – `dotnet test src/Concelier/__Tests/StellaOps.Concelier.Connector.Ics.Cisa.Tests/...` passes; fixtures assert attachment handling + SemVer semantics.|
|FEEDCONN-ICSCISA-02-006 Telemetry & documentation|DevEx|Docs|**DONE (2025-10-16)** – Ops guide documents attachment checks, SemVer exact values, and proxy guidance; diagnostics remain unchanged.|
|FEEDCONN-ICSCISA-02-007 Detail document inventory|BE-Conn-ICS-CISA|Research|**DONE (2025-10-16)** – Validated canned detail pages vs feed output so attachment inventories stay aligned; archived expectations noted in `HANDOVER.md`.|
|FEEDCONN-ICSCISA-02-008 Distribution fallback strategy|BE-Conn-ICS-CISA|Research|**DONE (2025-10-11)** – Outlined GovDelivery token request, HTML scrape + email digest fallback, and dependency on Ops for credential workflow; awaiting decision before fetch implementation.|
|FEEDCONN-ICSCISA-02-009 GovDelivery credential onboarding|Ops, BE-Conn-ICS-CISA|Ops|**DONE (2025-10-14)** – GovDelivery onboarding runbook captured in `docs/modules/concelier/operations/connectors/ics-cisa.md`; secret vault path and Offline Kit handling documented.|
|FEEDCONN-ICSCISA-02-010 Mitigation & SemVer polish|BE-Conn-ICS-CISA|02-003, 02-004|**DONE (2025-10-16)** – Attachment + mitigation references now land as expected and SemVer primitives carry exact values; end-to-end suite green (see `HANDOVER.md`).|
|FEEDCONN-ICSCISA-02-011 Docs & telemetry refresh|DevEx|02-006|**DONE (2025-10-16)** – Ops documentation refreshed (attachments, SemVer validation, proxy knobs) and telemetry notes verified.|

11
src/Concelier/__Libraries/StellaOps.Concelier.Connector.Ics.Cisa/TASKS.md
View File

@@ -1,15 +1,4 @@
# TASKS
| Task | Owner(s) | Depends on | Notes |
|---|---|---|---|
|FEEDCONN-ICSCISA-02-001 Document CISA ICS feed contract|BE-Conn-ICS-CISA|Research|**DONE (2025-10-11)** – `https://www.cisa.gov/cybersecurity-advisories/ics-advisories.xml` and legacy `/sites/default/files/feeds/...` return Akamai 403 even with browser UA; HTML landing page blocked as well. Logged full headers (x-reference-error, AkamaiGHost) in `docs/concelier-connector-research-20251011.md` and initiated GovDelivery access request.|
|FEEDCONN-ICSCISA-02-002 Fetch pipeline & cursor storage|BE-Conn-ICS-CISA|Source.Common, Storage.Mongo|**DONE (2025-10-16)** – Confirmed proxy knobs + cursor state behave with the refreshed fixtures; ops runbook now captures proxy usage/validation so the fetch stage is production-ready.|
|FEEDCONN-ICSCISA-02-003 DTO/parser implementation|BE-Conn-ICS-CISA|Source.Common|**DONE (2025-10-16)** – Feed parser fixtures updated to retain vendor PDFs as attachments while maintaining reference coverage; console diagnostics removed.|
|FEEDCONN-ICSCISA-02-004 Canonical mapping & range primitives|BE-Conn-ICS-CISA|Models|**DONE (2025-10-16)** – `TryCreateSemVerPrimitive` flow + Mongo deserialiser now persist `exactValue` (`4.2` → `4.2.0`), unblocking canonical snapshots.|
|FEEDCONN-ICSCISA-02-005 Deterministic fixtures/tests|QA|Testing|**DONE (2025-10-16)** – `dotnet test src/Concelier/__Tests/StellaOps.Concelier.Connector.Ics.Cisa.Tests/...` passes; fixtures assert attachment handling + SemVer semantics.|
|FEEDCONN-ICSCISA-02-006 Telemetry & documentation|DevEx|Docs|**DONE (2025-10-16)** – Ops guide documents attachment checks, SemVer exact values, and proxy guidance; diagnostics remain unchanged.|
|FEEDCONN-ICSCISA-02-007 Detail document inventory|BE-Conn-ICS-CISA|Research|**DONE (2025-10-16)** – Validated canned detail pages vs feed output so attachment inventories stay aligned; archived expectations noted in `HANDOVER.md`.|
|FEEDCONN-ICSCISA-02-008 Distribution fallback strategy|BE-Conn-ICS-CISA|Research|**DONE (2025-10-11)** – Outlined GovDelivery token request, HTML scrape + email digest fallback, and dependency on Ops for credential workflow; awaiting decision before fetch implementation.|
|FEEDCONN-ICSCISA-02-009 GovDelivery credential onboarding|Ops, BE-Conn-ICS-CISA|Ops|**DONE (2025-10-14)** – GovDelivery onboarding runbook captured in `docs/modules/concelier/operations/connectors/ics-cisa.md`; secret vault path and Offline Kit handling documented.|
|FEEDCONN-ICSCISA-02-010 Mitigation & SemVer polish|BE-Conn-ICS-CISA|02-003, 02-004|**DONE (2025-10-16)** – Attachment + mitigation references now land as expected and SemVer primitives carry exact values; end-to-end suite green (see `HANDOVER.md`).|
|FEEDCONN-ICSCISA-02-011 Docs & telemetry refresh|DevEx|02-006|**DONE (2025-10-16)** – Ops documentation refreshed (attachments, SemVer validation, proxy knobs) and telemetry notes verified.|
|FEEDCONN-ICSCISA-02-012 Normalized version decision|BE-Conn-ICS-CISA|Merge coordination (`FEEDMERGE-COORD-02-900`)|**TODO (due 2025-10-23)** – Promote existing `SemVerPrimitive` exact values into `NormalizedVersions` via `.ToNormalizedVersionRule("ics-cisa:{advisoryId}:{product}")`, add regression coverage, and open Models ticket if non-SemVer firmware requires a new scheme.<br>2025-10-29: Follow `docs/dev/normalized-rule-recipes.md` §2 to call `ToNormalizedVersionRule` and ensure mixed firmware strings log a Models ticket when regex extraction fails.|

16
src/Concelier/__Libraries/StellaOps.Concelier.Connector.Ics.Kaspersky/TASKS.completed.md Normal file
View File

@@ -0,0 +1,16 @@
# Completed Tasks
|List/detail fetcher with windowing|BE-Conn-ICS-Kaspersky|Source.Common|**DONE** – feed client paginates and fetches detail pages with window overlap.|
|Extractor (vendors/models/CVEs)|BE-Conn-ICS-Kaspersky|Source.Common|**DONE** – parser normalizes vendor/model taxonomy into DTO.|
|DTO validation and sanitizer|BE-Conn-ICS-Kaspersky, QA|Source.Common|**DONE** – HTML parsed into DTO with sanitizer guardrails.|
|Canonical mapping (affected, refs)|BE-Conn-ICS-Kaspersky|Models|**DONE** – mapper outputs `ics-vendor` affected entries with provenance.|
|State/dedupe and fixtures|BE-Conn-ICS-Kaspersky, QA|Storage.Mongo|**DONE** – duplicate-content and resume tests exercise SHA gating + cursor hygiene.|
|Backoff on fetch failures|BE-Conn-ICS-Kaspersky|Storage.Mongo|**DONE** – feed/page failures mark source_state with timed backoff.|
|Conditional fetch caching|BE-Conn-ICS-Kaspersky|Source.Common|**DONE** – fetch cache persists ETag/Last-Modified; not-modified scenarios validated in tests.|

13
src/Concelier/__Libraries/StellaOps.Concelier.Connector.Ics.Kaspersky/TASKS.md
View File

@@ -1,10 +1,3 @@
# TASKS
| Task | Owner(s) | Depends on | Notes |
|---|---|---|---|
|List/detail fetcher with windowing|BE-Conn-ICS-Kaspersky|Source.Common|**DONE** – feed client paginates and fetches detail pages with window overlap.|
|Extractor (vendors/models/CVEs)|BE-Conn-ICS-Kaspersky|Source.Common|**DONE** – parser normalizes vendor/model taxonomy into DTO.|
|DTO validation and sanitizer|BE-Conn-ICS-Kaspersky, QA|Source.Common|**DONE** – HTML parsed into DTO with sanitizer guardrails.|
|Canonical mapping (affected, refs)|BE-Conn-ICS-Kaspersky|Models|**DONE** – mapper outputs `ics-vendor` affected entries with provenance.|
|State/dedupe and fixtures|BE-Conn-ICS-Kaspersky, QA|Storage.Mongo|**DONE** – duplicate-content and resume tests exercise SHA gating + cursor hygiene.|
|Backoff on fetch failures|BE-Conn-ICS-Kaspersky|Storage.Mongo|**DONE** – feed/page failures mark source_state with timed backoff.|
|Conditional fetch caching|BE-Conn-ICS-Kaspersky|Source.Common|**DONE** – fetch cache persists ETag/Last-Modified; not-modified scenarios validated in tests.|
# TASKS
| Task | Owner(s) | Depends on | Notes |
|---|---|---|---|

22
src/Concelier/__Libraries/StellaOps.Concelier.Connector.Jvn/TASKS.completed.md Normal file
View File

@@ -0,0 +1,22 @@
# Completed Tasks
|MyJVN client (JVNRSS+VULDEF) with windowing|BE-Conn-JVN|Source.Common|**DONE** – windowed overview/detail fetch with rate limit handling implemented.|
|Schema/XSD validation and DTO sanitizer|BE-Conn-JVN, QA|Source.Common|**DONE** – parser validates XML against schema before DTO persistence.|
|Canonical mapping (aliases, jp_flags, refs)|BE-Conn-JVN|Models|**DONE** – mapper populates aliases, jp_flags, references while skipping non-actionable affected entries.|
|SourceState and idempotent dedupe|BE-Conn-JVN|Storage.Mongo|**DONE** – cursor tracks pending docs/mappings with resume support.|
|Golden fixtures and determinism tests|QA|Source.Jvn|**DONE** – deterministic snapshot test in `JvnConnectorTests` now passes with offline fixtures.|
|Async-safe overview query building|BE-Conn-JVN|Source.Common|DONE – `MyJvnClient` now builds query strings synchronously without blocking calls.|
|Reference dedupe + deterministic ordering|BE-Conn-JVN|Models|DONE – mapper merges by URL, retains richer metadata, sorts deterministically.|
|Console logging remediation|BE-Conn-JVN|Observability|**DONE** – connector now uses structured `ILogger` debug entries instead of console writes.|
|Offline fixtures for connector tests|QA|Source.Jvn|**DONE** – tests rely solely on canned HTTP responses and local fixtures.|
|Update VULDEF schema for vendor attribute|BE-Conn-JVN, QA|Source.Jvn|**DONE** – embedded XSD updated (vendor/product attrs, impact item), parser tightened, fixtures & snapshots refreshed.|

16
src/Concelier/__Libraries/StellaOps.Concelier.Connector.Jvn/TASKS.md
View File

@@ -1,13 +1,3 @@
# TASKS
| Task | Owner(s) | Depends on | Notes |
|---|---|---|---|
|MyJVN client (JVNRSS+VULDEF) with windowing|BE-Conn-JVN|Source.Common|**DONE** – windowed overview/detail fetch with rate limit handling implemented.|
|Schema/XSD validation and DTO sanitizer|BE-Conn-JVN, QA|Source.Common|**DONE** – parser validates XML against schema before DTO persistence.|
|Canonical mapping (aliases, jp_flags, refs)|BE-Conn-JVN|Models|**DONE** – mapper populates aliases, jp_flags, references while skipping non-actionable affected entries.|
|SourceState and idempotent dedupe|BE-Conn-JVN|Storage.Mongo|**DONE** – cursor tracks pending docs/mappings with resume support.|
|Golden fixtures and determinism tests|QA|Source.Jvn|**DONE** – deterministic snapshot test in `JvnConnectorTests` now passes with offline fixtures.|
|Async-safe overview query building|BE-Conn-JVN|Source.Common|DONE – `MyJvnClient` now builds query strings synchronously without blocking calls.|
|Reference dedupe + deterministic ordering|BE-Conn-JVN|Models|DONE – mapper merges by URL, retains richer metadata, sorts deterministically.|
|Console logging remediation|BE-Conn-JVN|Observability|**DONE** – connector now uses structured `ILogger` debug entries instead of console writes.|
|Offline fixtures for connector tests|QA|Source.Jvn|**DONE** – tests rely solely on canned HTTP responses and local fixtures.|
|Update VULDEF schema for vendor attribute|BE-Conn-JVN, QA|Source.Jvn|**DONE** – embedded XSD updated (vendor/product attrs, impact item), parser tightened, fixtures & snapshots refreshed.|
# TASKS
| Task | Owner(s) | Depends on | Notes |
|---|---|---|---|

20
src/Concelier/__Libraries/StellaOps.Concelier.Connector.Kev/TASKS.completed.md Normal file
View File

@@ -0,0 +1,20 @@
# Completed Tasks
|Review KEV JSON schema & cadence|BE-Conn-KEV|Research|**DONE** – Feed defaults lock to the public JSON catalog; AGENTS notes call out daily cadence and allowlist requirements.|
|Fetch & cursor implementation|BE-Conn-KEV|Source.Common, Storage.Mongo|**DONE** – SourceFetchService drives ETag/Last-Modified aware fetches with SourceState cursor tracking documents + catalog metadata.|
|DTO/parser implementation|BE-Conn-KEV|Source.Common|**DONE** – `KevCatalogDto`/`KevVulnerabilityDto` deserialize payloads with logging for catalog version/releases before DTO persistence.|
|Canonical mapping & range primitives|BE-Conn-KEV|Models|**DONE** – Mapper produces vendor RangePrimitives (due dates, CWE list, ransomware flag, catalog metadata) and deduplicated references.|
|Deterministic fixtures/tests|QA|Testing|**DONE** – End-to-end fetch→parse→map test with canned catalog + snapshot (`UPDATE_KEV_FIXTURES=1`) guards determinism.|
|Telemetry & docs|DevEx|Docs|**DONE** – Connector emits structured logs + meters for catalog entries/advisories and AGENTS docs cover cadence/allowlist guidance.|
|Schema validation & anomaly surfacing|BE-Conn-KEV, QA|Source.Common|**DONE (2025-10-12)** – Wired `IJsonSchemaValidator` + embedded schema, added failure reasons (`schema`, `download`, `invalidJson`, etc.), anomaly counters (`missingCveId`, `countMismatch`, `nullEntry`), and kept `dotnet test src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Kev.Tests` passing.|
|Metrics export wiring|DevOps, DevEx|Observability|**DONE (2025-10-12)** – Added `kev.fetch.*` counters, parse failure/anomaly tags, refreshed ops runbook + Grafana dashboard (`docs/modules/concelier/operations/connectors/cve-kev-grafana-dashboard.json`) with PromQL guidance.|
|FEEDCONN-KEV-02-003 Normalized versions propagation|BE-Conn-KEV|Models `FEEDMODELS-SCHEMA-01-003`, Normalization playbook|**DONE (2025-10-12)** – Validated catalog/date/due normalized rules emission + ordering; fixtures assert rule set and `dotnet test src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Kev.Tests` remains green.|

9
src/Concelier/__Libraries/StellaOps.Concelier.Connector.Kev/TASKS.md
View File

@@ -1,12 +1,3 @@
# TASKS
| Task | Owner(s) | Depends on | Notes |
|---|---|---|---|
|Review KEV JSON schema & cadence|BE-Conn-KEV|Research|**DONE** – Feed defaults lock to the public JSON catalog; AGENTS notes call out daily cadence and allowlist requirements.|
|Fetch & cursor implementation|BE-Conn-KEV|Source.Common, Storage.Mongo|**DONE** – SourceFetchService drives ETag/Last-Modified aware fetches with SourceState cursor tracking documents + catalog metadata.|
|DTO/parser implementation|BE-Conn-KEV|Source.Common|**DONE** – `KevCatalogDto`/`KevVulnerabilityDto` deserialize payloads with logging for catalog version/releases before DTO persistence.|
|Canonical mapping & range primitives|BE-Conn-KEV|Models|**DONE** – Mapper produces vendor RangePrimitives (due dates, CWE list, ransomware flag, catalog metadata) and deduplicated references.|
|Deterministic fixtures/tests|QA|Testing|**DONE** – End-to-end fetch→parse→map test with canned catalog + snapshot (`UPDATE_KEV_FIXTURES=1`) guards determinism.|
|Telemetry & docs|DevEx|Docs|**DONE** – Connector emits structured logs + meters for catalog entries/advisories and AGENTS docs cover cadence/allowlist guidance.|
|Schema validation & anomaly surfacing|BE-Conn-KEV, QA|Source.Common|**DONE (2025-10-12)** – Wired `IJsonSchemaValidator` + embedded schema, added failure reasons (`schema`, `download`, `invalidJson`, etc.), anomaly counters (`missingCveId`, `countMismatch`, `nullEntry`), and kept `dotnet test src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Kev.Tests` passing.|
|Metrics export wiring|DevOps, DevEx|Observability|**DONE (2025-10-12)** – Added `kev.fetch.*` counters, parse failure/anomaly tags, refreshed ops runbook + Grafana dashboard (`docs/modules/concelier/operations/connectors/cve-kev-grafana-dashboard.json`) with PromQL guidance.|
|FEEDCONN-KEV-02-003 Normalized versions propagation|BE-Conn-KEV|Models `FEEDMODELS-SCHEMA-01-003`, Normalization playbook|**DONE (2025-10-12)** – Validated catalog/date/due normalized rules emission + ordering; fixtures assert rule set and `dotnet test src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Kev.Tests` remains green.|

16
src/Concelier/__Libraries/StellaOps.Concelier.Connector.Kisa/TASKS.completed.md Normal file
View File

@@ -0,0 +1,16 @@
# Completed Tasks
|FEEDCONN-KISA-02-001 Research KISA advisory feeds|BE-Conn-KISA|Research|**DONE (2025-10-11)** – Located public RSS endpoints (`https://knvd.krcert.or.kr/rss/securityInfo.do`, `.../securityNotice.do`) returning UTF-8 XML with 10-item windows and canonical `detailDos.do?IDX=` links. Logged output structure + header profile in `docs/concelier-connector-research-20251011.md`; outstanding work is parsing the SPA detail payload.|
|FEEDCONN-KISA-02-002 Fetch pipeline & source state|BE-Conn-KISA|Source.Common, Storage.Mongo|**DONE (2025-10-14)** – `KisaConnector.FetchAsync` pulls RSS, sets `Accept-Language: ko-KR`, persists detail JSON with IDX metadata, throttles requests, and tracks cursor state (pending docs/mappings, known IDs, published timestamp).|
|FEEDCONN-KISA-02-003 Parser & DTO implementation|BE-Conn-KISA|Source.Common|**DONE (2025-10-14)** – Detail API parsed via `KisaDetailParser` (Hangul NFC normalisation, sanitised HTML, CVE extraction, references/products captured into DTO `kisa.detail.v1`).|
|FEEDCONN-KISA-02-004 Canonical mapping & range primitives|BE-Conn-KISA|Models|**DONE (2025-10-14)** – `KisaMapper` emits vendor packages with range strings, aliases (IDX/CVEs), references, and provenance; advisories default to `ko` language and normalised severity.|
|FEEDCONN-KISA-02-005 Deterministic fixtures & tests|QA|Testing|**DONE (2025-10-14)** – Added `StellaOps.Concelier.Connector.Kisa.Tests` with Korean fixtures and fetch→parse→map regression; fixtures regenerate via `UPDATE_KISA_FIXTURES=1`.|
|FEEDCONN-KISA-02-006 Telemetry & documentation|DevEx|Docs|**DONE (2025-10-14)** – Added diagnostics-backed telemetry, structured logs, regression coverage, and published localisation notes in `docs/dev/kisa_connector_notes.md` + fixture guidance for Docs/QA.|
|FEEDCONN-KISA-02-007 RSS contract & localisation brief|BE-Conn-KISA|Research|**DONE (2025-10-11)** – Documented RSS URLs, confirmed UTF-8 payload (no additional cookies required), and drafted localisation plan (Hangul glossary + optional MT plugin). Remaining open item: capture SPA detail API contract for full-text translations.|

15
src/Concelier/__Libraries/StellaOps.Concelier.Connector.Kisa/TASKS.md
View File

@@ -1,11 +1,4 @@
# TASKS
| Task | Owner(s) | Depends on | Notes |
|---|---|---|---|
|FEEDCONN-KISA-02-001 Research KISA advisory feeds|BE-Conn-KISA|Research|**DONE (2025-10-11)** – Located public RSS endpoints (`https://knvd.krcert.or.kr/rss/securityInfo.do`, `.../securityNotice.do`) returning UTF-8 XML with 10-item windows and canonical `detailDos.do?IDX=` links. Logged output structure + header profile in `docs/concelier-connector-research-20251011.md`; outstanding work is parsing the SPA detail payload.|
|FEEDCONN-KISA-02-002 Fetch pipeline & source state|BE-Conn-KISA|Source.Common, Storage.Mongo|**DONE (2025-10-14)** – `KisaConnector.FetchAsync` pulls RSS, sets `Accept-Language: ko-KR`, persists detail JSON with IDX metadata, throttles requests, and tracks cursor state (pending docs/mappings, known IDs, published timestamp).|
|FEEDCONN-KISA-02-003 Parser & DTO implementation|BE-Conn-KISA|Source.Common|**DONE (2025-10-14)** – Detail API parsed via `KisaDetailParser` (Hangul NFC normalisation, sanitised HTML, CVE extraction, references/products captured into DTO `kisa.detail.v1`).|
|FEEDCONN-KISA-02-004 Canonical mapping & range primitives|BE-Conn-KISA|Models|**DONE (2025-10-14)** – `KisaMapper` emits vendor packages with range strings, aliases (IDX/CVEs), references, and provenance; advisories default to `ko` language and normalised severity.|
|FEEDCONN-KISA-02-005 Deterministic fixtures & tests|QA|Testing|**DONE (2025-10-14)** – Added `StellaOps.Concelier.Connector.Kisa.Tests` with Korean fixtures and fetch→parse→map regression; fixtures regenerate via `UPDATE_KISA_FIXTURES=1`.|
|FEEDCONN-KISA-02-006 Telemetry & documentation|DevEx|Docs|**DONE (2025-10-14)** – Added diagnostics-backed telemetry, structured logs, regression coverage, and published localisation notes in `docs/dev/kisa_connector_notes.md` + fixture guidance for Docs/QA.|
|FEEDCONN-KISA-02-007 RSS contract & localisation brief|BE-Conn-KISA|Research|**DONE (2025-10-11)** – Documented RSS URLs, confirmed UTF-8 payload (no additional cookies required), and drafted localisation plan (Hangul glossary + optional MT plugin). Remaining open item: capture SPA detail API contract for full-text translations.|
|FEEDCONN-KISA-02-008 Firmware scheme proposal|BE-Conn-KISA, Models|Merge coordination (`FEEDMERGE-COORD-02-900`)|**TODO (due 2025-10-24)** – Define transformation for Hangul-labelled firmware ranges (`XFU 1.0.1.0084 ~ 2.0.1.0034`), propose `kisa.build` (or equivalent) scheme to Models, implement normalized rule emission/tests once scheme approved, and update localisation notes.|
# TASKS
| Task | Owner(s) | Depends on | Notes |
|---|---|---|---|
|FEEDCONN-KISA-02-008 Firmware scheme proposal|BE-Conn-KISA, Models|Merge coordination (`FEEDMERGE-COORD-02-900`)|**TODO (due 2025-10-24)** – Define transformation for Hangul-labelled firmware ranges (`XFU 1.0.1.0084 ~ 2.0.1.0034`), propose `kisa.build` (or equivalent) scheme to Models, implement normalized rule emission/tests once scheme approved, and update localisation notes.|

32
src/Concelier/__Libraries/StellaOps.Concelier.Connector.Nvd/TASKS.completed.md Normal file
View File

@@ -0,0 +1,32 @@
# Completed Tasks
|Fetch job with sliding modified windows|BE-Conn-Nvd|Source.Common|**DONE** – windowed fetch implemented with overlap and raw doc persistence.|
|DTO schema + validation|BE-Conn-Nvd|Source.Common|**DONE** – schema validator enforced before DTO persistence.|
|Mapper to canonical model|BE-Conn-Nvd|Models|**DONE** – `NvdMapper` populates CVSS/CWE/CPE data.<br>2025-10-11 research trail: upcoming normalized rules must serialize as `[{"scheme":"semver","type":"range","min":"<floor>","minInclusive":true,"max":"<ceiling>","maxInclusive":false,"notes":"nvd:CVE-2025-XXXX"}]`; keep notes consistent with CVE IDs for provenance joins.|
|Watermark repo usage|BE-Conn-Nvd|Storage.Mongo|**DONE** – cursor tracks windowStart/windowEnd and updates SourceState.|
|Integration test fixture isolation|QA|Storage.Mongo|**DONE** – connector tests reset Mongo/time fixtures between runs to avoid cross-test bleed.|
|Tests: golden pages + resume|QA|Tests|**DONE** – snapshot and resume coverage added across `NvdConnectorTests`.|
|Observability|BE-Conn-Nvd|Core|**DONE** – `NvdDiagnostics` meter tracks attempts/documents/failures with collector tests.|
|Change history snapshotting|BE-Conn-Nvd|Storage.Mongo|DONE – connector now records per-CVE snapshots with top-level diff metadata whenever canonical advisories change.|
|Pagination for windows over page limit|BE-Conn-Nvd|Source.Common|**DONE** – additional page fetcher honors `startIndex`; covered by multipage tests.|
|Schema validation quarantine path|BE-Conn-Nvd|Storage.Mongo|**DONE** – schema failures mark documents failed and metrics assert quarantine.|
|FEEDCONN-NVD-04-002 Conflict regression fixtures|BE-Conn-Nvd, QA|Merge `FEEDMERGE-ENGINE-04-001`|**DONE (2025-10-12)** – Published `conflict-nvd.canonical.json` + mapper test; includes CVSS 3.1 + CWE reference and normalized CPE range feeding the conflict triple. Validation: `dotnet test src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Nvd.Tests/StellaOps.Concelier.Connector.Nvd.Tests.csproj --filter NvdConflictFixtureTests`.|
|FEEDCONN-NVD-02-004 NVD CVSS & CWE precedence payloads|BE-Conn-Nvd|Models `FEEDMODELS-SCHEMA-01-002`|**DONE (2025-10-11)** – CVSS metrics now carry provenance masks, CWE weaknesses emit normalized references, and fixtures cover the additional precedence data.|
|FEEDCONN-NVD-02-005 NVD merge/export parity regression|BE-Conn-Nvd, BE-Merge|Merge `FEEDMERGE-ENGINE-04-003`|**DONE (2025-10-12)** – Canonical merge parity fixtures captured, regression test validates credit/reference union, and exporter snapshot check guarantees parity through JSON exports.|
|FEEDCONN-NVD-02-002 Normalized versions rollout|BE-Conn-Nvd|Models `FEEDMODELS-SCHEMA-01-003`, Normalization playbook|**DONE (2025-10-11)** – SemVer primitives + normalized rules emitting for parseable ranges, fixtures/tests refreshed, coordination pinged via FEEDMERGE-COORD-02-900.|
|FEEDCONN-NVD-04-003 Description/CWE/metric parity rollout|BE-Conn-Nvd|Models, Core|**DONE (2025-10-15)** – Mapper now surfaces normalized description text, CWE weaknesses, and canonical CVSS metric id. Snapshots (`conflict-nvd.canonical.json`) refreshed and completion relayed to Merge coordination.|

15
src/Concelier/__Libraries/StellaOps.Concelier.Connector.Nvd/TASKS.md
View File

@@ -1,18 +1,3 @@
# TASKS
| Task | Owner(s) | Depends on | Notes |
|---|---|---|---|
|Fetch job with sliding modified windows|BE-Conn-Nvd|Source.Common|**DONE** – windowed fetch implemented with overlap and raw doc persistence.|
|DTO schema + validation|BE-Conn-Nvd|Source.Common|**DONE** – schema validator enforced before DTO persistence.|
|Mapper to canonical model|BE-Conn-Nvd|Models|**DONE** – `NvdMapper` populates CVSS/CWE/CPE data.<br>2025-10-11 research trail: upcoming normalized rules must serialize as `[{"scheme":"semver","type":"range","min":"<floor>","minInclusive":true,"max":"<ceiling>","maxInclusive":false,"notes":"nvd:CVE-2025-XXXX"}]`; keep notes consistent with CVE IDs for provenance joins.|
|Watermark repo usage|BE-Conn-Nvd|Storage.Mongo|**DONE** – cursor tracks windowStart/windowEnd and updates SourceState.|
|Integration test fixture isolation|QA|Storage.Mongo|**DONE** – connector tests reset Mongo/time fixtures between runs to avoid cross-test bleed.|
|Tests: golden pages + resume|QA|Tests|**DONE** – snapshot and resume coverage added across `NvdConnectorTests`.|
|Observability|BE-Conn-Nvd|Core|**DONE** – `NvdDiagnostics` meter tracks attempts/documents/failures with collector tests.|
|Change history snapshotting|BE-Conn-Nvd|Storage.Mongo|DONE – connector now records per-CVE snapshots with top-level diff metadata whenever canonical advisories change.|
|Pagination for windows over page limit|BE-Conn-Nvd|Source.Common|**DONE** – additional page fetcher honors `startIndex`; covered by multipage tests.|
|Schema validation quarantine path|BE-Conn-Nvd|Storage.Mongo|**DONE** – schema failures mark documents failed and metrics assert quarantine.|
|FEEDCONN-NVD-04-002 Conflict regression fixtures|BE-Conn-Nvd, QA|Merge `FEEDMERGE-ENGINE-04-001`|**DONE (2025-10-12)** – Published `conflict-nvd.canonical.json` + mapper test; includes CVSS 3.1 + CWE reference and normalized CPE range feeding the conflict triple. Validation: `dotnet test src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Nvd.Tests/StellaOps.Concelier.Connector.Nvd.Tests.csproj --filter NvdConflictFixtureTests`.|
|FEEDCONN-NVD-02-004 NVD CVSS & CWE precedence payloads|BE-Conn-Nvd|Models `FEEDMODELS-SCHEMA-01-002`|**DONE (2025-10-11)** – CVSS metrics now carry provenance masks, CWE weaknesses emit normalized references, and fixtures cover the additional precedence data.|
|FEEDCONN-NVD-02-005 NVD merge/export parity regression|BE-Conn-Nvd, BE-Merge|Merge `FEEDMERGE-ENGINE-04-003`|**DONE (2025-10-12)** – Canonical merge parity fixtures captured, regression test validates credit/reference union, and exporter snapshot check guarantees parity through JSON exports.|
|FEEDCONN-NVD-02-002 Normalized versions rollout|BE-Conn-Nvd|Models `FEEDMODELS-SCHEMA-01-003`, Normalization playbook|**DONE (2025-10-11)** – SemVer primitives + normalized rules emitting for parseable ranges, fixtures/tests refreshed, coordination pinged via FEEDMERGE-COORD-02-900.|
|FEEDCONN-NVD-04-003 Description/CWE/metric parity rollout|BE-Conn-Nvd|Models, Core|**DONE (2025-10-15)** – Mapper now surfaces normalized description text, CWE weaknesses, and canonical CVSS metric id. Snapshots (`conflict-nvd.canonical.json`) refreshed and completion relayed to Merge coordination.|

36
src/Concelier/__Libraries/StellaOps.Concelier.Connector.Osv/TASKS.completed.md Normal file
View File

@@ -0,0 +1,36 @@
# Completed Tasks
|Ecosystem fetchers (npm, pypi, maven, go, crates)|BE-Conn-OSV|Source.Common|**DONE** – archive fetch loop iterates ecosystems with pagination + change gating.|
|OSV options & HttpClient configuration|BE-Conn-OSV|Source.Common|**DONE** – `OsvOptions` + `AddOsvConnector` configure allowlisted HttpClient.|
|DTO validation + sanitizer|BE-Conn-OSV|Source.Common|**DONE** – JSON deserialization sanitizes payloads before persistence; schema enforcement deferred.|
|Mapper to canonical SemVer ranges|BE-Conn-OSV|Models|**DONE** – `OsvMapper` emits SemVer ranges with provenance metadata.<br>2025-10-11 research trail: ensure `NormalizedVersions` array uses payloads such as `[{"scheme":"semver","type":"range","min":"<min>","minInclusive":true,"max":"<max>","maxInclusive":false,"notes":"osv:GHI-2025-0001"}]` so storage merges align with GHSA parity tests.|
|Alias consolidation (GHSA/CVE)|BE-Merge|Merge|DONE – OSV advisory records now emit GHSA/CVE aliases captured by alias graph tests.|
|Tests: snapshot per ecosystem|QA|Tests|DONE – deterministic snapshots added for npm and PyPI advisories.|
|Cursor persistence and hash gating|BE-Conn-OSV|Storage.Mongo|**DONE** – `OsvCursor` tracks per-ecosystem metadata and SHA gating.|
|Parity checks vs GHSA data|QA|Merge|DONE – `OsvGhsaParityRegressionTests` keep OSV ↔ GHSA fixtures green; regeneration workflow documented in docs/19_TEST_SUITE_OVERVIEW.md.|
|Connector DI routine & job registration|BE-Conn-OSV|Core|**DONE** – DI routine registers fetch/parse/map jobs with scheduler.|
|Implement OSV fetch/parse/map skeleton|BE-Conn-OSV|Source.Common|**DONE** – connector now persists documents, DTOs, and canonical advisories.|
|FEEDCONN-OSV-02-004 OSV references & credits alignment|BE-Conn-OSV|Models `FEEDMODELS-SCHEMA-01-002`|**DONE (2025-10-11)** – Mapper normalizes references with provenance masks, emits advisory credits, and regression fixtures/assertions cover the new fields.|
|FEEDCONN-OSV-02-005 Fixture updater workflow|BE-Conn-OSV, QA|Docs|**DONE (2025-10-12)** – Canonical PURL derivation now covers Go + scoped npm advisories without upstream `purl`; legacy invalid npm names still fall back to `ecosystem:name`. OSV/GHSA/NVD suites and normalization/storage tests rerun clean.|
|FEEDCONN-OSV-02-003 Normalized versions rollout|BE-Conn-OSV|Models `FEEDMODELS-SCHEMA-01-003`, Normalization playbook|**DONE (2025-10-11)** – `OsvMapper` now emits SemVer primitives + normalized rules with `osv:{ecosystem}:{advisoryId}:{identifier}` notes; npm/PyPI/Parity fixtures refreshed; merge coordination pinged (OSV handoff).|
|FEEDCONN-OSV-04-003 Parity fixture refresh|QA, BE-Conn-OSV|Normalized versions rollout, GHSA parity tests|**DONE (2025-10-12)** – Parity fixtures include normalizedVersions notes (`osv:<ecosystem>:<id>:<purl>`); regression math rerun via `dotnet test src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Osv.Tests` and docs flagged for workflow sync.|
|FEEDCONN-OSV-04-002 Conflict regression fixtures|BE-Conn-OSV, QA|Merge `FEEDMERGE-ENGINE-04-001`|**DONE (2025-10-12)** – Added `conflict-osv.canonical.json` + regression asserting SemVer range + CVSS medium severity; dataset matches GHSA/NVD fixtures for merge tests. Validation: `dotnet test src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Osv.Tests/StellaOps.Concelier.Connector.Osv.Tests.csproj --filter OsvConflictFixtureTests`.|
|FEEDCONN-OSV-04-004 Description/CWE/metric parity rollout|BE-Conn-OSV|Models, Core|**DONE (2025-10-15)** – OSV mapper writes advisory descriptions, `database_specific.cwe_ids` weaknesses, and canonical CVSS metric id. Parity fixtures (`osv-ghsa.*`, `osv-npm.snapshot.json`, `osv-pypi.snapshot.json`) refreshed and status communicated to Merge coordination.|
|FEEDCONN-OSV-04-005 Canonical metric fallbacks & CWE notes|BE-Conn-OSV|Models, Merge|**DONE (2025-10-16)** – Add fallback logic and metrics for advisories lacking CVSS vectors, enrich CWE provenance notes, and document merge/export expectations; refresh parity fixtures accordingly.<br>2025-10-16: Mapper now emits `osv:severity/<level>` canonical ids for severity-only advisories, weakness provenance carries `database_specific.cwe_ids`, diagnostics expose `osv.map.canonical_metric_fallbacks`, parity fixtures regenerated, and ops notes added in `docs/modules/concelier/operations/connectors/osv.md`. Tests: `dotnet test src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Osv.Tests/StellaOps.Concelier.Connector.Osv.Tests.csproj`.|

17
src/Concelier/__Libraries/StellaOps.Concelier.Connector.Osv/TASKS.md
View File

@@ -1,20 +1,3 @@
# TASKS
| Task | Owner(s) | Depends on | Notes |
|---|---|---|---|
|Ecosystem fetchers (npm, pypi, maven, go, crates)|BE-Conn-OSV|Source.Common|**DONE** – archive fetch loop iterates ecosystems with pagination + change gating.|
|OSV options & HttpClient configuration|BE-Conn-OSV|Source.Common|**DONE** – `OsvOptions` + `AddOsvConnector` configure allowlisted HttpClient.|
|DTO validation + sanitizer|BE-Conn-OSV|Source.Common|**DONE** – JSON deserialization sanitizes payloads before persistence; schema enforcement deferred.|
|Mapper to canonical SemVer ranges|BE-Conn-OSV|Models|**DONE** – `OsvMapper` emits SemVer ranges with provenance metadata.<br>2025-10-11 research trail: ensure `NormalizedVersions` array uses payloads such as `[{"scheme":"semver","type":"range","min":"<min>","minInclusive":true,"max":"<max>","maxInclusive":false,"notes":"osv:GHI-2025-0001"}]` so storage merges align with GHSA parity tests.|
|Alias consolidation (GHSA/CVE)|BE-Merge|Merge|DONE – OSV advisory records now emit GHSA/CVE aliases captured by alias graph tests.|
|Tests: snapshot per ecosystem|QA|Tests|DONE – deterministic snapshots added for npm and PyPI advisories.|
|Cursor persistence and hash gating|BE-Conn-OSV|Storage.Mongo|**DONE** – `OsvCursor` tracks per-ecosystem metadata and SHA gating.|
|Parity checks vs GHSA data|QA|Merge|DONE – `OsvGhsaParityRegressionTests` keep OSV ↔ GHSA fixtures green; regeneration workflow documented in docs/19_TEST_SUITE_OVERVIEW.md.|
|Connector DI routine & job registration|BE-Conn-OSV|Core|**DONE** – DI routine registers fetch/parse/map jobs with scheduler.|
|Implement OSV fetch/parse/map skeleton|BE-Conn-OSV|Source.Common|**DONE** – connector now persists documents, DTOs, and canonical advisories.|
|FEEDCONN-OSV-02-004 OSV references & credits alignment|BE-Conn-OSV|Models `FEEDMODELS-SCHEMA-01-002`|**DONE (2025-10-11)** – Mapper normalizes references with provenance masks, emits advisory credits, and regression fixtures/assertions cover the new fields.|
|FEEDCONN-OSV-02-005 Fixture updater workflow|BE-Conn-OSV, QA|Docs|**DONE (2025-10-12)** – Canonical PURL derivation now covers Go + scoped npm advisories without upstream `purl`; legacy invalid npm names still fall back to `ecosystem:name`. OSV/GHSA/NVD suites and normalization/storage tests rerun clean.|
|FEEDCONN-OSV-02-003 Normalized versions rollout|BE-Conn-OSV|Models `FEEDMODELS-SCHEMA-01-003`, Normalization playbook|**DONE (2025-10-11)** – `OsvMapper` now emits SemVer primitives + normalized rules with `osv:{ecosystem}:{advisoryId}:{identifier}` notes; npm/PyPI/Parity fixtures refreshed; merge coordination pinged (OSV handoff).|
|FEEDCONN-OSV-04-003 Parity fixture refresh|QA, BE-Conn-OSV|Normalized versions rollout, GHSA parity tests|**DONE (2025-10-12)** – Parity fixtures include normalizedVersions notes (`osv:<ecosystem>:<id>:<purl>`); regression math rerun via `dotnet test src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Osv.Tests` and docs flagged for workflow sync.|
|FEEDCONN-OSV-04-002 Conflict regression fixtures|BE-Conn-OSV, QA|Merge `FEEDMERGE-ENGINE-04-001`|**DONE (2025-10-12)** – Added `conflict-osv.canonical.json` + regression asserting SemVer range + CVSS medium severity; dataset matches GHSA/NVD fixtures for merge tests. Validation: `dotnet test src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Osv.Tests/StellaOps.Concelier.Connector.Osv.Tests.csproj --filter OsvConflictFixtureTests`.|
|FEEDCONN-OSV-04-004 Description/CWE/metric parity rollout|BE-Conn-OSV|Models, Core|**DONE (2025-10-15)** – OSV mapper writes advisory descriptions, `database_specific.cwe_ids` weaknesses, and canonical CVSS metric id. Parity fixtures (`osv-ghsa.*`, `osv-npm.snapshot.json`, `osv-pypi.snapshot.json`) refreshed and status communicated to Merge coordination.|
|FEEDCONN-OSV-04-005 Canonical metric fallbacks & CWE notes|BE-Conn-OSV|Models, Merge|**DONE (2025-10-16)** – Add fallback logic and metrics for advisories lacking CVSS vectors, enrich CWE provenance notes, and document merge/export expectations; refresh parity fixtures accordingly.<br>2025-10-16: Mapper now emits `osv:severity/<level>` canonical ids for severity-only advisories, weakness provenance carries `database_specific.cwe_ids`, diagnostics expose `osv.map.canonical_metric_fallbacks`, parity fixtures regenerated, and ops notes added in `docs/modules/concelier/operations/connectors/osv.md`. Tests: `dotnet test src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Osv.Tests/StellaOps.Concelier.Connector.Osv.Tests.csproj`.|

18
src/Concelier/__Libraries/StellaOps.Concelier.Connector.Ru.Bdu/TASKS.completed.md Normal file
View File

@@ -0,0 +1,18 @@
# Completed Tasks
|FEEDCONN-RUBDU-02-001 Identify BDU data source & schema|BE-Conn-BDU|Research|**DONE (2025-10-11)** – Candidate endpoints (`https://bdu.fstec.ru/component/rsform/form/7-bdu?format=xml`, `...?format=json`) return 403/404 even with `--insecure` because TLS chain requires Russian Trusted Sub CA and WAF expects referer/session headers. Documented request/response samples in `docs/concelier-connector-research-20251011.md`; blocked until trusted root + access strategy from Ops.|
|FEEDCONN-RUBDU-02-002 Fetch pipeline & cursor handling|BE-Conn-BDU|Source.Common, Storage.Mongo|**DONE (2025-10-14)** – Connector streams `vulxml.zip` through cached fetches, persists JSON payloads via `RawDocumentStorage`, and tracks cursor pending sets. Added cache fallback + deterministic SHA logging and state updates tied to `TimeProvider`.|
|FEEDCONN-RUBDU-02-003 DTO/parser implementation|BE-Conn-BDU|Source.Common|**DONE (2025-10-14)** – `RuBduXmlParser` now captures identifiers, source links, CVSS 2/3 metrics, CWE arrays, and environment/software metadata with coverage for multi-entry fixtures.|
|FEEDCONN-RUBDU-02-004 Canonical mapping & range primitives|BE-Conn-BDU|Models|**DONE (2025-10-14)** – `RuBduMapper` emits vendor/ICS packages with normalized `ru-bdu.raw` rules, dual status provenance, alias/reference hydration (CVE, external, source), and CVSS severity normalisation.|
|FEEDCONN-RUBDU-02-005 Deterministic fixtures & regression tests|QA|Testing|**DONE (2025-10-14)** – Added connector harness snapshot suite with canned archive, state/documents/dtos/advisories snapshots under `Fixtures/`, gated by `UPDATE_BDU_FIXTURES`.|
|FEEDCONN-RUBDU-02-006 Telemetry & documentation|DevEx|Docs|**DONE (2025-10-14)** – Introduced `RuBduDiagnostics` meter (fetch/parse/map counters & histograms) and authored connector README covering configuration, trusted roots, telemetry, and offline behaviour.|
|FEEDCONN-RUBDU-02-007 Access & export options assessment|BE-Conn-BDU|Research|**DONE (2025-10-14)** – Documented archive access constraints, offline mirroring expectations, and export packaging in `src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Ru.Bdu/README.md` + flagged Offline Kit bundling requirements.|
|FEEDCONN-RUBDU-02-008 Trusted root onboarding plan|BE-Conn-BDU|Source.Common|**DONE (2025-10-14)** – Validated Russian Trusted Root/Sub CA bundle wiring (`certificates/russian_trusted_bundle.pem`), updated Offline Kit guidance, and surfaced `concelier:httpClients:source.bdu:trustedRootPaths` sample configuration.|

8
src/Concelier/__Libraries/StellaOps.Concelier.Connector.Ru.Bdu/TASKS.md
View File

@@ -1,11 +1,3 @@
# TASKS
| Task | Owner(s) | Depends on | Notes |
|---|---|---|---|
|FEEDCONN-RUBDU-02-001 Identify BDU data source & schema|BE-Conn-BDU|Research|**DONE (2025-10-11)** – Candidate endpoints (`https://bdu.fstec.ru/component/rsform/form/7-bdu?format=xml`, `...?format=json`) return 403/404 even with `--insecure` because TLS chain requires Russian Trusted Sub CA and WAF expects referer/session headers. Documented request/response samples in `docs/concelier-connector-research-20251011.md`; blocked until trusted root + access strategy from Ops.|
|FEEDCONN-RUBDU-02-002 Fetch pipeline & cursor handling|BE-Conn-BDU|Source.Common, Storage.Mongo|**DONE (2025-10-14)** – Connector streams `vulxml.zip` through cached fetches, persists JSON payloads via `RawDocumentStorage`, and tracks cursor pending sets. Added cache fallback + deterministic SHA logging and state updates tied to `TimeProvider`.|
|FEEDCONN-RUBDU-02-003 DTO/parser implementation|BE-Conn-BDU|Source.Common|**DONE (2025-10-14)** – `RuBduXmlParser` now captures identifiers, source links, CVSS 2/3 metrics, CWE arrays, and environment/software metadata with coverage for multi-entry fixtures.|
|FEEDCONN-RUBDU-02-004 Canonical mapping & range primitives|BE-Conn-BDU|Models|**DONE (2025-10-14)** – `RuBduMapper` emits vendor/ICS packages with normalized `ru-bdu.raw` rules, dual status provenance, alias/reference hydration (CVE, external, source), and CVSS severity normalisation.|
|FEEDCONN-RUBDU-02-005 Deterministic fixtures & regression tests|QA|Testing|**DONE (2025-10-14)** – Added connector harness snapshot suite with canned archive, state/documents/dtos/advisories snapshots under `Fixtures/`, gated by `UPDATE_BDU_FIXTURES`.|
|FEEDCONN-RUBDU-02-006 Telemetry & documentation|DevEx|Docs|**DONE (2025-10-14)** – Introduced `RuBduDiagnostics` meter (fetch/parse/map counters & histograms) and authored connector README covering configuration, trusted roots, telemetry, and offline behaviour.|
|FEEDCONN-RUBDU-02-007 Access & export options assessment|BE-Conn-BDU|Research|**DONE (2025-10-14)** – Documented archive access constraints, offline mirroring expectations, and export packaging in `src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Connector.Ru.Bdu/README.md` + flagged Offline Kit bundling requirements.|
|FEEDCONN-RUBDU-02-008 Trusted root onboarding plan|BE-Conn-BDU|Source.Common|**DONE (2025-10-14)** – Validated Russian Trusted Root/Sub CA bundle wiring (`certificates/russian_trusted_bundle.pem`), updated Offline Kit guidance, and surfaced `concelier:httpClients:source.bdu:trustedRootPaths` sample configuration.|

18
src/Concelier/__Libraries/StellaOps.Concelier.Connector.Ru.Nkcki/TASKS.completed.md Normal file
View File

@@ -0,0 +1,18 @@
# Completed Tasks
|FEEDCONN-NKCKI-02-001 Research NKTsKI advisory feeds|BE-Conn-Nkcki|Research|**DONE (2025-10-11)** – Candidate RSS locations (`https://cert.gov.ru/rss/advisories.xml`, `https://www.cert.gov.ru/...`) return 403/404 even with `Accept-Language: ru-RU` and `--insecure`; site is Bitrix-backed and expects Russian Trusted Sub CA plus session cookies. Logged packet captures + needed cert list in `docs/concelier-connector-research-20251011.md`; waiting on Ops for sanctioned trust bundle.|
|FEEDCONN-NKCKI-02-002 Fetch pipeline & state persistence|BE-Conn-Nkcki|Source.Common, Storage.Mongo|**DONE (2025-10-13)** – Listing fetch now honours `maxListingPagesPerFetch`, persists cache hits when listing access fails, and records telemetry via `RuNkckiDiagnostics`. Cursor tracking covers pending documents/mappings and the known bulletin ring buffer.|
|FEEDCONN-NKCKI-02-003 DTO & parser implementation|BE-Conn-Nkcki|Source.Common|**DONE (2025-10-13)** – Parser normalises nested arrays (ICS categories, vulnerable software lists, optional tags), flattens multiline `software_text`, and guarantees deterministic ordering for URLs and tags.|
|FEEDCONN-NKCKI-02-004 Canonical mapping & range primitives|BE-Conn-Nkcki|Models|**DONE (2025-10-13)** – Mapper splits structured software entries, emits SemVer range primitives + normalized rules, deduplicates references, and surfaces CVSS v4 metadata alongside existing metrics.|
|FEEDCONN-NKCKI-02-005 Deterministic fixtures & tests|QA|Testing|**DONE (2025-10-13)** – Fixtures refreshed with multi-page pagination + multi-entry bulletins. Tests exercise cache replay and rely on bundled OpenSSL 1.1 libs in `src/Tools/openssl/linux-x64` to keep Mongo2Go green on modern distros.|
|FEEDCONN-NKCKI-02-006 Telemetry & documentation|DevEx|Docs|**DONE (2025-10-13)** – Added connector-specific metrics (`nkcki.*`) and documented configuration/operational guidance in `docs/modules/concelier/operations/connectors/nkcki.md`.|
|FEEDCONN-NKCKI-02-007 Archive ingestion strategy|BE-Conn-Nkcki|Research|**DONE (2025-10-13)** – Documented Bitrix pagination/backfill plan (cache-first, offline replay, HTML/PDF capture) in `docs/modules/concelier/operations/connectors/nkcki.md`.|
|FEEDCONN-NKCKI-02-008 Access enablement plan|BE-Conn-Nkcki|Source.Common|**DONE (2025-10-11)** – Documented trust-store requirement, optional SOCKS proxy fallback, and monitoring plan; shared TLS support now available via `SourceHttpClientOptions.TrustedRootCertificates` (`concelier:httpClients:source.nkcki:*`), awaiting Ops-sourced cert bundle before fetch implementation.|

8
src/Concelier/__Libraries/StellaOps.Concelier.Connector.Ru.Nkcki/TASKS.md
View File

@@ -1,11 +1,3 @@
# TASKS
| Task | Owner(s) | Depends on | Notes |
|---|---|---|---|
|FEEDCONN-NKCKI-02-001 Research NKTsKI advisory feeds|BE-Conn-Nkcki|Research|**DONE (2025-10-11)** – Candidate RSS locations (`https://cert.gov.ru/rss/advisories.xml`, `https://www.cert.gov.ru/...`) return 403/404 even with `Accept-Language: ru-RU` and `--insecure`; site is Bitrix-backed and expects Russian Trusted Sub CA plus session cookies. Logged packet captures + needed cert list in `docs/concelier-connector-research-20251011.md`; waiting on Ops for sanctioned trust bundle.|
|FEEDCONN-NKCKI-02-002 Fetch pipeline & state persistence|BE-Conn-Nkcki|Source.Common, Storage.Mongo|**DONE (2025-10-13)** – Listing fetch now honours `maxListingPagesPerFetch`, persists cache hits when listing access fails, and records telemetry via `RuNkckiDiagnostics`. Cursor tracking covers pending documents/mappings and the known bulletin ring buffer.|
|FEEDCONN-NKCKI-02-003 DTO & parser implementation|BE-Conn-Nkcki|Source.Common|**DONE (2025-10-13)** – Parser normalises nested arrays (ICS categories, vulnerable software lists, optional tags), flattens multiline `software_text`, and guarantees deterministic ordering for URLs and tags.|
|FEEDCONN-NKCKI-02-004 Canonical mapping & range primitives|BE-Conn-Nkcki|Models|**DONE (2025-10-13)** – Mapper splits structured software entries, emits SemVer range primitives + normalized rules, deduplicates references, and surfaces CVSS v4 metadata alongside existing metrics.|
|FEEDCONN-NKCKI-02-005 Deterministic fixtures & tests|QA|Testing|**DONE (2025-10-13)** – Fixtures refreshed with multi-page pagination + multi-entry bulletins. Tests exercise cache replay and rely on bundled OpenSSL 1.1 libs in `src/Tools/openssl/linux-x64` to keep Mongo2Go green on modern distros.|
|FEEDCONN-NKCKI-02-006 Telemetry & documentation|DevEx|Docs|**DONE (2025-10-13)** – Added connector-specific metrics (`nkcki.*`) and documented configuration/operational guidance in `docs/modules/concelier/operations/connectors/nkcki.md`.|
|FEEDCONN-NKCKI-02-007 Archive ingestion strategy|BE-Conn-Nkcki|Research|**DONE (2025-10-13)** – Documented Bitrix pagination/backfill plan (cache-first, offline replay, HTML/PDF capture) in `docs/modules/concelier/operations/connectors/nkcki.md`.|
|FEEDCONN-NKCKI-02-008 Access enablement plan|BE-Conn-Nkcki|Source.Common|**DONE (2025-10-11)** – Documented trust-store requirement, optional SOCKS proxy fallback, and monitoring plan; shared TLS support now available via `SourceHttpClientOptions.TrustedRootCertificates` (`concelier:httpClients:source.nkcki:*`), awaiting Ops-sourced cert bundle before fetch implementation.|

7
src/Concelier/__Libraries/StellaOps.Concelier.Connector.StellaOpsMirror/TASKS.completed.md Normal file
View File

@@ -0,0 +1,7 @@
# Completed Tasks
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| FEEDCONN-STELLA-08-001 | DONE (2025-10-20) | BE-Conn-Stella | CONCELIER-EXPORT-08-201 | Implement Concelier mirror fetcher hitting `https://<domain>.stella-ops.org/concelier/exports/index.json`, verify signatures/digests, and persist raw documents with provenance. | Fetch job downloads mirror manifest, verifies digest/signature, stores raw docs with tests covering happy-path + tampered manifest. *(Completed 2025-10-20: detached JWS + digest enforcement, metadata persisted, and regression coverage via `dotnet test src/Concelier/__Tests/StellaOps.Concelier.Connector.StellaOpsMirror.Tests/StellaOps.Concelier.Connector.StellaOpsMirror.Tests.csproj`.)* |
| FEEDCONN-STELLA-08-002 | DONE (2025-10-20) | BE-Conn-Stella | FEEDCONN-STELLA-08-001 | Map mirror payloads into canonical advisory DTOs with provenance referencing mirror domain + original source metadata. | Mapper produces advisories/aliases/affected with mirror provenance; fixtures assert canonical parity with upstream JSON exporters. |
| FEEDCONN-STELLA-08-003 | DONE (2025-10-20) | BE-Conn-Stella | FEEDCONN-STELLA-08-002 | Add incremental cursor + resume support (per-export fingerprint) and document configuration for downstream Concelier instances. | Connector resumes from last export, handles deletion/delta cases, docs updated with config sample; integration test covers resume + new export scenario. |

3
src/Concelier/__Libraries/StellaOps.Concelier.Connector.StellaOpsMirror/TASKS.md
View File

@@ -2,6 +2,3 @@
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| FEEDCONN-STELLA-08-001 | DONE (2025-10-20) | BE-Conn-Stella | CONCELIER-EXPORT-08-201 | Implement Concelier mirror fetcher hitting `https://<domain>.stella-ops.org/concelier/exports/index.json`, verify signatures/digests, and persist raw documents with provenance. | Fetch job downloads mirror manifest, verifies digest/signature, stores raw docs with tests covering happy-path + tampered manifest. *(Completed 2025-10-20: detached JWS + digest enforcement, metadata persisted, and regression coverage via `dotnet test src/Concelier/__Tests/StellaOps.Concelier.Connector.StellaOpsMirror.Tests/StellaOps.Concelier.Connector.StellaOpsMirror.Tests.csproj`.)* |
| FEEDCONN-STELLA-08-002 | DONE (2025-10-20) | BE-Conn-Stella | FEEDCONN-STELLA-08-001 | Map mirror payloads into canonical advisory DTOs with provenance referencing mirror domain + original source metadata. | Mapper produces advisories/aliases/affected with mirror provenance; fixtures assert canonical parity with upstream JSON exporters. |
| FEEDCONN-STELLA-08-003 | DONE (2025-10-20) | BE-Conn-Stella | FEEDCONN-STELLA-08-002 | Add incremental cursor + resume support (per-export fingerprint) and document configuration for downstream Concelier instances. | Connector resumes from last export, handles deletion/delta cases, docs updated with config sample; integration test covers resume + new export scenario. |

20
src/Concelier/__Libraries/StellaOps.Concelier.Connector.Vndr.Adobe/TASKS.completed.md Normal file
View File

@@ -0,0 +1,20 @@
# Completed Tasks
|Index discovery and sliding window fetch|BE-Conn-Adobe|Source.Common|DONE — Support backfill; honor robots/ToS.|
|Detail extractor (products/components/fixes)|BE-Conn-Adobe|Source.Common|DONE — Normalizes metadata and CVE/product capture.|
|DTO schema and validation pipeline|BE-Conn-Adobe, QA|Source.Common|DONE — JSON schema enforced during parse.|
|Canonical mapping plus psirt_flags|BE-Conn-Adobe|Models|DONE — Emits canonical advisory and Adobe psirt flag.|
|SourceState plus sha256 short-circuit|BE-Conn-Adobe|Storage.Mongo|DONE — Idempotence guarantee.|
|Golden fixtures and determinism tests|QA|Source.Vndr.Adobe|**DONE** — connector tests assert snapshot determinism for dual advisories.|
|Mark failed parse DTOs|BE-Conn-Adobe|Storage.Mongo|**DONE** — parse failures now mark documents `Failed` and tests cover the path.|
|Reference dedupe & ordering|BE-Conn-Adobe|Models|**DONE** — mapper groups references by URL with deterministic ordering.|
|NormalizedVersions emission|BE-Conn-Adobe|Models|**DONE** (2025-10-11) — EVR-like version metadata now projects `normalizedVersions` with `adobe:<product>:<platform>` notes; regression fixtures refreshed to assert rule output.|

15
src/Concelier/__Libraries/StellaOps.Concelier.Connector.Vndr.Adobe/TASKS.md
View File

@@ -1,12 +1,3 @@
# TASKS
| Task | Owner(s) | Depends on | Notes |
|---|---|---|---|
|Index discovery and sliding window fetch|BE-Conn-Adobe|Source.Common|DONE — Support backfill; honor robots/ToS.|
|Detail extractor (products/components/fixes)|BE-Conn-Adobe|Source.Common|DONE — Normalizes metadata and CVE/product capture.|
|DTO schema and validation pipeline|BE-Conn-Adobe, QA|Source.Common|DONE — JSON schema enforced during parse.|
|Canonical mapping plus psirt_flags|BE-Conn-Adobe|Models|DONE — Emits canonical advisory and Adobe psirt flag.|
|SourceState plus sha256 short-circuit|BE-Conn-Adobe|Storage.Mongo|DONE — Idempotence guarantee.|
|Golden fixtures and determinism tests|QA|Source.Vndr.Adobe|**DONE** — connector tests assert snapshot determinism for dual advisories.|
|Mark failed parse DTOs|BE-Conn-Adobe|Storage.Mongo|**DONE** — parse failures now mark documents `Failed` and tests cover the path.|
|Reference dedupe & ordering|BE-Conn-Adobe|Models|**DONE** — mapper groups references by URL with deterministic ordering.|
|NormalizedVersions emission|BE-Conn-Adobe|Models|**DONE** (2025-10-11) — EVR-like version metadata now projects `normalizedVersions` with `adobe:<product>:<platform>` notes; regression fixtures refreshed to assert rule output.|
# TASKS
| Task | Owner(s) | Depends on | Notes |
|---|---|---|---|

18
src/Concelier/__Libraries/StellaOps.Concelier.Connector.Vndr.Apple/TASKS.completed.md Normal file
View File

@@ -0,0 +1,18 @@
# Completed Tasks
|Catalogue Apple security bulletin sources|BE-Conn-Apple|Research|**DONE** – Feed contract documented in README (Software Lookup Service JSON + HT article hub) with rate-limit notes.|
|Fetch pipeline & state persistence|BE-Conn-Apple|Source.Common, Storage.Mongo|**DONE** – Index fetch + detail ingestion with SourceState cursoring/allowlists committed; awaiting live smoke run before enabling in scheduler defaults.|
|Parser & DTO implementation|BE-Conn-Apple|Source.Common|**DONE** – AngleSharp detail parser produces canonical DTO payloads (CVE list, timestamps, affected tables) persisted via DTO store.|
|Canonical mapping & range primitives|BE-Conn-Apple|Models|**DONE** – Mapper now emits SemVer-derived normalizedVersions with `apple:<platform>:<product>` notes; fixtures updated to assert canonical rules while we continue tracking multi-device coverage in follow-up tasks.<br>2025-10-11 research trail: confirmed payload aligns with `[{"scheme":"semver","type":"range","min":"<build-start>","minInclusive":true,"max":"<build-end>","maxInclusive":false,"notes":"apple:ios:17.1"}]`; continue using `notes` to surface build identifiers for storage provenance.|
|Deterministic fixtures/tests|QA|Testing|**DONE (2025-10-12)** – Parser now scopes references to article content, sorts affected rows deterministically, and regenerated fixtures (125326/125328/106355/HT214108/HT215500) produce stable JSON + sanitizer HTML in English.|
|Telemetry & documentation|DevEx|Docs|**DONE (2025-10-12)** – OpenTelemetry pipeline exports `StellaOps.Concelier.Connector.Vndr.Apple`; runbook `docs/modules/concelier/operations/connectors/apple.md` added with metrics + monitoring guidance.|
|Live HTML regression sweep|QA|Source.Common|**DONE (2025-10-12)** – Captured latest support.apple.com articles for 125326/125328/106355/HT214108/HT215500, trimmed nav noise, and committed sanitized HTML + expected DTOs with invariant timestamps.|
|Fixture regeneration tooling|DevEx|Testing|**DONE (2025-10-12)** – `scripts/update-apple-fixtures.(sh|ps1)` set the env flag + sentinel, forward through WSLENV, and clean up after regeneration; README references updated usage.|

8
src/Concelier/__Libraries/StellaOps.Concelier.Connector.Vndr.Apple/TASKS.md
View File

@@ -1,11 +1,3 @@
# TASKS
| Task | Owner(s) | Depends on | Notes |
|---|---|---|---|
|Catalogue Apple security bulletin sources|BE-Conn-Apple|Research|**DONE** – Feed contract documented in README (Software Lookup Service JSON + HT article hub) with rate-limit notes.|
|Fetch pipeline & state persistence|BE-Conn-Apple|Source.Common, Storage.Mongo|**DONE** – Index fetch + detail ingestion with SourceState cursoring/allowlists committed; awaiting live smoke run before enabling in scheduler defaults.|
|Parser & DTO implementation|BE-Conn-Apple|Source.Common|**DONE** – AngleSharp detail parser produces canonical DTO payloads (CVE list, timestamps, affected tables) persisted via DTO store.|
|Canonical mapping & range primitives|BE-Conn-Apple|Models|**DONE** – Mapper now emits SemVer-derived normalizedVersions with `apple:<platform>:<product>` notes; fixtures updated to assert canonical rules while we continue tracking multi-device coverage in follow-up tasks.<br>2025-10-11 research trail: confirmed payload aligns with `[{"scheme":"semver","type":"range","min":"<build-start>","minInclusive":true,"max":"<build-end>","maxInclusive":false,"notes":"apple:ios:17.1"}]`; continue using `notes` to surface build identifiers for storage provenance.|
|Deterministic fixtures/tests|QA|Testing|**DONE (2025-10-12)** – Parser now scopes references to article content, sorts affected rows deterministically, and regenerated fixtures (125326/125328/106355/HT214108/HT215500) produce stable JSON + sanitizer HTML in English.|
|Telemetry & documentation|DevEx|Docs|**DONE (2025-10-12)** – OpenTelemetry pipeline exports `StellaOps.Concelier.Connector.Vndr.Apple`; runbook `docs/modules/concelier/operations/connectors/apple.md` added with metrics + monitoring guidance.|
|Live HTML regression sweep|QA|Source.Common|**DONE (2025-10-12)** – Captured latest support.apple.com articles for 125326/125328/106355/HT214108/HT215500, trimmed nav noise, and committed sanitized HTML + expected DTOs with invariant timestamps.|
|Fixture regeneration tooling|DevEx|Testing|**DONE (2025-10-12)** – `scripts/update-apple-fixtures.(sh|ps1)` set the env flag + sentinel, forward through WSLENV, and clean up after regeneration; README references updated usage.|

22
src/Concelier/__Libraries/StellaOps.Concelier.Connector.Vndr.Chromium/TASKS.completed.md Normal file
View File

@@ -0,0 +1,22 @@
# Completed Tasks
| CH1 | Blog crawl + cursor | Conn | DONE | Common | Sliding window feed reader with cursor persisted. |
| CH2 | Post parser → DTO (CVEs, versions, refs) | QA | DONE | | AngleSharp parser normalizes CVEs, versions, references. |
| CH3 | Canonical mapping (aliases/refs/affected-hint)| Conn | DONE | Models | Deterministic advisory mapping with psirt flags. |
| CH4 | Snapshot tests + resume | QA | DONE | Storage | Deterministic snapshot plus resume scenario via Mongo state. |
| CH5 | Observability | QA | DONE | | Metered fetch/parse/map counters. |
| CH6 | SourceState + SHA dedupe | Conn | DONE | Storage | Cursor tracks SHA cache to skip unchanged posts. |
| CH7 | Stabilize resume integration (preserve pending docs across provider instances) | QA | DONE | Storage.Mongo | Resume integration test exercises pending docs across providers via shared Mongo. |
| CH8 | Mark failed parse documents | Conn | DONE | Storage.Mongo | Parse pipeline marks failures; unit tests assert status transitions. |
| CH9 | Reference dedupe & ordering | Conn | DONE | Models | Mapper groups references by URL and sorts deterministically. |
| CH10 | Range primitives + provenance instrumentation | Conn | DONE | Models, Storage.Mongo | Vendor primitives + logging in place, resume metrics updated, snapshots refreshed. |

24
src/Concelier/__Libraries/StellaOps.Concelier.Connector.Vndr.Chromium/TASKS.md
View File

@@ -1,17 +1,7 @@
# Source.Vndr.Chromium — Task Board
| ID | Task | Owner | Status | Depends On | Notes |
|------|-----------------------------------------------|-------|--------|------------|-------|
| CH1 | Blog crawl + cursor | Conn | DONE | Common | Sliding window feed reader with cursor persisted. |
| CH2 | Post parser → DTO (CVEs, versions, refs) | QA | DONE | | AngleSharp parser normalizes CVEs, versions, references. |
| CH3 | Canonical mapping (aliases/refs/affected-hint)| Conn | DONE | Models | Deterministic advisory mapping with psirt flags. |
| CH4 | Snapshot tests + resume | QA | DONE | Storage | Deterministic snapshot plus resume scenario via Mongo state. |
| CH5 | Observability | QA | DONE | | Metered fetch/parse/map counters. |
| CH6 | SourceState + SHA dedupe | Conn | DONE | Storage | Cursor tracks SHA cache to skip unchanged posts. |
| CH7 | Stabilize resume integration (preserve pending docs across provider instances) | QA | DONE | Storage.Mongo | Resume integration test exercises pending docs across providers via shared Mongo. |
| CH8 | Mark failed parse documents | Conn | DONE | Storage.Mongo | Parse pipeline marks failures; unit tests assert status transitions. |
| CH9 | Reference dedupe & ordering | Conn | DONE | Models | Mapper groups references by URL and sorts deterministically. |
| CH10 | Range primitives + provenance instrumentation | Conn | DONE | Models, Storage.Mongo | Vendor primitives + logging in place, resume metrics updated, snapshots refreshed. |
## Changelog
- YYYY-MM-DD: Created.
# Source.Vndr.Chromium — Task Board
| ID | Task | Owner | Status | Depends On | Notes |
|------|-----------------------------------------------|-------|--------|------------|-------|
## Changelog
- YYYY-MM-DD: Created.

18
src/Concelier/__Libraries/StellaOps.Concelier.Connector.Vndr.Cisco/TASKS.completed.md Normal file
View File

@@ -0,0 +1,18 @@
# Completed Tasks
|FEEDCONN-CISCO-02-001 Confirm Cisco PSIRT data source|BE-Conn-Cisco|Research|**DONE (2025-10-11)** – Selected openVuln REST API (`https://apix.cisco.com/security/advisories/v2/…`) as primary (structured JSON, CSAF/CVRF links) with RSS as fallback. Documented OAuth2 client-credentials flow (`cloudsso.cisco.com/as/token.oauth2`), baseline quotas (5 req/s, 30 req/min, 5 000 req/day), and pagination contract (`pageIndex`, `pageSize≤100`) in `docs/concelier-connector-research-20251011.md`.|
|FEEDCONN-CISCO-02-002 Fetch pipeline & state persistence|BE-Conn-Cisco|Source.Common, Storage.Mongo|**DONE (2025-10-14)** – Fetch job now streams openVuln pages with OAuth bearer handler, honours 429 `Retry-After`, persists per-advisory JSON + metadata into GridFS, and updates cursor (`lastModified`, advisory ID, pending docs).|
|FEEDCONN-CISCO-02-003 Parser & DTO implementation|BE-Conn-Cisco|Source.Common|**DONE (2025-10-14)** – DTO factory normalizes SIR, folds CSAF product statuses, and persists `cisco.dto.v1` payloads (see `CiscoDtoFactory`).|
|FEEDCONN-CISCO-02-004 Canonical mapping & range primitives|BE-Conn-Cisco|Models|**DONE (2025-10-14)** – `CiscoMapper` emits canonical advisories with vendor + SemVer primitives, provenance, and status tags.|
|FEEDCONN-CISCO-02-005 Deterministic fixtures & tests|QA|Testing|**DONE (2025-10-14)** – Added unit tests (`StellaOps.Concelier.Connector.Vndr.Cisco.Tests`) exercising DTO/mapper pipelines; `dotnet test` validated.|
|FEEDCONN-CISCO-02-006 Telemetry & documentation|DevEx|Docs|**DONE (2025-10-14)** – Cisco diagnostics counters exposed and ops runbook updated with telemetry guidance (`docs/modules/concelier/operations/connectors/cisco.md`).|
|FEEDCONN-CISCO-02-007 API selection decision memo|BE-Conn-Cisco|Research|**DONE (2025-10-11)** – Drafted decision matrix: openVuln (structured/delta filters, OAuth throttle) vs RSS (delayed/minimal metadata). Pending OAuth onboarding (`FEEDCONN-CISCO-02-008`) before final recommendation circulated.|
|FEEDCONN-CISCO-02-008 OAuth client provisioning|Ops, BE-Conn-Cisco|Ops|**DONE (2025-10-14)** – `docs/modules/concelier/operations/connectors/cisco.md` documents OAuth provisioning/rotation, quotas, and Offline Kit distribution guidance.|

8
src/Concelier/__Libraries/StellaOps.Concelier.Connector.Vndr.Cisco/TASKS.md
View File

@@ -1,12 +1,4 @@
# TASKS
| Task | Owner(s) | Depends on | Notes |
|---|---|---|---|
|FEEDCONN-CISCO-02-001 Confirm Cisco PSIRT data source|BE-Conn-Cisco|Research|**DONE (2025-10-11)** – Selected openVuln REST API (`https://apix.cisco.com/security/advisories/v2/…`) as primary (structured JSON, CSAF/CVRF links) with RSS as fallback. Documented OAuth2 client-credentials flow (`cloudsso.cisco.com/as/token.oauth2`), baseline quotas (5 req/s, 30 req/min, 5 000 req/day), and pagination contract (`pageIndex`, `pageSize≤100`) in `docs/concelier-connector-research-20251011.md`.|
|FEEDCONN-CISCO-02-002 Fetch pipeline & state persistence|BE-Conn-Cisco|Source.Common, Storage.Mongo|**DONE (2025-10-14)** – Fetch job now streams openVuln pages with OAuth bearer handler, honours 429 `Retry-After`, persists per-advisory JSON + metadata into GridFS, and updates cursor (`lastModified`, advisory ID, pending docs).|
|FEEDCONN-CISCO-02-003 Parser & DTO implementation|BE-Conn-Cisco|Source.Common|**DONE (2025-10-14)** – DTO factory normalizes SIR, folds CSAF product statuses, and persists `cisco.dto.v1` payloads (see `CiscoDtoFactory`).|
|FEEDCONN-CISCO-02-004 Canonical mapping & range primitives|BE-Conn-Cisco|Models|**DONE (2025-10-14)** – `CiscoMapper` emits canonical advisories with vendor + SemVer primitives, provenance, and status tags.|
|FEEDCONN-CISCO-02-005 Deterministic fixtures & tests|QA|Testing|**DONE (2025-10-14)** – Added unit tests (`StellaOps.Concelier.Connector.Vndr.Cisco.Tests`) exercising DTO/mapper pipelines; `dotnet test` validated.|
|FEEDCONN-CISCO-02-006 Telemetry & documentation|DevEx|Docs|**DONE (2025-10-14)** – Cisco diagnostics counters exposed and ops runbook updated with telemetry guidance (`docs/modules/concelier/operations/connectors/cisco.md`).|
|FEEDCONN-CISCO-02-007 API selection decision memo|BE-Conn-Cisco|Research|**DONE (2025-10-11)** – Drafted decision matrix: openVuln (structured/delta filters, OAuth throttle) vs RSS (delayed/minimal metadata). Pending OAuth onboarding (`FEEDCONN-CISCO-02-008`) before final recommendation circulated.|
|FEEDCONN-CISCO-02-008 OAuth client provisioning|Ops, BE-Conn-Cisco|Ops|**DONE (2025-10-14)** – `docs/modules/concelier/operations/connectors/cisco.md` documents OAuth provisioning/rotation, quotas, and Offline Kit distribution guidance.|
|FEEDCONN-CISCO-02-009 Normalized SemVer promotion|BE-Conn-Cisco|Merge coordination (`FEEDMERGE-COORD-02-900`)|**TODO (due 2025-10-21)** – Use helper from `../Merge/RANGE_PRIMITIVES_COORDINATION.md` to convert `SemVerPrimitive` outputs into `NormalizedVersionRule` with provenance (`cisco:{productId}`), update mapper/tests, and confirm merge normalized-rule counters drop.|

18
src/Concelier/__Libraries/StellaOps.Concelier.Connector.Vndr.Msrc/TASKS.completed.md Normal file
View File

@@ -0,0 +1,18 @@
# Completed Tasks
|FEEDCONN-MSRC-02-001 Document MSRC Security Update Guide API|BE-Conn-MSRC|Research|**DONE (2025-10-11)** – Confirmed REST endpoint (`https://api.msrc.microsoft.com/sug/v2.0/en-US/vulnerabilities`) + CVRF ZIP download flow, required Azure AD client-credentials scope (`api://api.msrc.microsoft.com/.default`), mandatory `api-version=2024-08-01` header, and delta params (`lastModifiedStartDateTime`, `lastModifiedEndDateTime`). Findings recorded in `docs/concelier-connector-research-20251011.md`.|
|FEEDCONN-MSRC-02-002 Fetch pipeline & source state|BE-Conn-MSRC|Source.Common, Storage.Mongo|**DONE (2025-10-15)** – Added `MsrcApiClient` + token provider, cursor overlap handling, and detail persistence via GridFS (metadata carries CVRF URL + timestamps). State tracks `lastModifiedCursor` with configurable overlap/backoff. **Next:** coordinate with Tools on shared state-seeding helper once CVRF download flag stabilises.|
|FEEDCONN-MSRC-02-003 Parser & DTO implementation|BE-Conn-MSRC|Source.Common|**DONE (2025-10-15)** – Implemented `MsrcDetailParser`/DTOs capturing threats, remediations, KB IDs, CVEs, CVSS, and affected products (build/platform metadata preserved).|
|FEEDCONN-MSRC-02-004 Canonical mapping & range primitives|BE-Conn-MSRC|Models|**DONE (2025-10-15)** – `MsrcMapper` emits aliases (MSRC ID/CVE/KB), references (release notes + CVRF), vendor packages with `msrc.build` normalized rules, and CVSS provenance.|
|FEEDCONN-MSRC-02-005 Deterministic fixtures/tests|QA|Testing|**DONE (2025-10-15)** – Added `StellaOps.Concelier.Connector.Vndr.Msrc.Tests` with canned token/summary/detail responses and snapshot assertions via Mongo2Go. Fixtures regenerate via `UPDATE_MSRC_FIXTURES`.|
|FEEDCONN-MSRC-02-006 Telemetry & documentation|DevEx|Docs|**DONE (2025-10-15)** – Introduced `MsrcDiagnostics` meter (summary/detail/parse/map metrics), structured fetch logs, README updates, and Ops brief `docs/modules/concelier/operations/connectors/msrc.md` covering AAD onboarding + CVRF handling.|
|FEEDCONN-MSRC-02-007 API contract comparison memo|BE-Conn-MSRC|Research|**DONE (2025-10-11)** – Completed memo outline recommending dual-path (REST for incremental, CVRF for offline); implementation hinges on `FEEDCONN-MSRC-02-008` AAD onboarding for token acquisition.|
|FEEDCONN-MSRC-02-008 Azure AD application onboarding|Ops, BE-Conn-MSRC|Ops|**DONE (2025-10-15)** – Coordinated Ops handoff; drafted AAD onboarding brief (`docs/modules/concelier/operations/connectors/msrc.md`) with app registration requirements, secret rotation policy, sample configuration, and CVRF mirroring guidance for Offline Kit.|

8
src/Concelier/__Libraries/StellaOps.Concelier.Connector.Vndr.Msrc/TASKS.md
View File

@@ -1,11 +1,3 @@
# TASKS
| Task | Owner(s) | Depends on | Notes |
|---|---|---|---|
|FEEDCONN-MSRC-02-001 Document MSRC Security Update Guide API|BE-Conn-MSRC|Research|**DONE (2025-10-11)** – Confirmed REST endpoint (`https://api.msrc.microsoft.com/sug/v2.0/en-US/vulnerabilities`) + CVRF ZIP download flow, required Azure AD client-credentials scope (`api://api.msrc.microsoft.com/.default`), mandatory `api-version=2024-08-01` header, and delta params (`lastModifiedStartDateTime`, `lastModifiedEndDateTime`). Findings recorded in `docs/concelier-connector-research-20251011.md`.|
|FEEDCONN-MSRC-02-002 Fetch pipeline & source state|BE-Conn-MSRC|Source.Common, Storage.Mongo|**DONE (2025-10-15)** – Added `MsrcApiClient` + token provider, cursor overlap handling, and detail persistence via GridFS (metadata carries CVRF URL + timestamps). State tracks `lastModifiedCursor` with configurable overlap/backoff. **Next:** coordinate with Tools on shared state-seeding helper once CVRF download flag stabilises.|
|FEEDCONN-MSRC-02-003 Parser & DTO implementation|BE-Conn-MSRC|Source.Common|**DONE (2025-10-15)** – Implemented `MsrcDetailParser`/DTOs capturing threats, remediations, KB IDs, CVEs, CVSS, and affected products (build/platform metadata preserved).|
|FEEDCONN-MSRC-02-004 Canonical mapping & range primitives|BE-Conn-MSRC|Models|**DONE (2025-10-15)** – `MsrcMapper` emits aliases (MSRC ID/CVE/KB), references (release notes + CVRF), vendor packages with `msrc.build` normalized rules, and CVSS provenance.|
|FEEDCONN-MSRC-02-005 Deterministic fixtures/tests|QA|Testing|**DONE (2025-10-15)** – Added `StellaOps.Concelier.Connector.Vndr.Msrc.Tests` with canned token/summary/detail responses and snapshot assertions via Mongo2Go. Fixtures regenerate via `UPDATE_MSRC_FIXTURES`.|
|FEEDCONN-MSRC-02-006 Telemetry & documentation|DevEx|Docs|**DONE (2025-10-15)** – Introduced `MsrcDiagnostics` meter (summary/detail/parse/map metrics), structured fetch logs, README updates, and Ops brief `docs/modules/concelier/operations/connectors/msrc.md` covering AAD onboarding + CVRF handling.|
|FEEDCONN-MSRC-02-007 API contract comparison memo|BE-Conn-MSRC|Research|**DONE (2025-10-11)** – Completed memo outline recommending dual-path (REST for incremental, CVRF for offline); implementation hinges on `FEEDCONN-MSRC-02-008` AAD onboarding for token acquisition.|
|FEEDCONN-MSRC-02-008 Azure AD application onboarding|Ops, BE-Conn-MSRC|Ops|**DONE (2025-10-15)** – Coordinated Ops handoff; drafted AAD onboarding brief (`docs/modules/concelier/operations/connectors/msrc.md`) with app registration requirements, secret rotation policy, sample configuration, and CVRF mirroring guidance for Offline Kit.|

22
src/Concelier/__Libraries/StellaOps.Concelier.Connector.Vndr.Oracle/TASKS.completed.md Normal file
View File

@@ -0,0 +1,22 @@
# Completed Tasks
|Oracle options & HttpClient configuration|BE-Conn-Oracle|Source.Common|**DONE** – `AddOracleConnector` wires options and allowlisted HttpClient.|
|CPU calendar plus advisory fetchers|BE-Conn-Oracle|Source.Common|**DONE** – resume/backfill scenario covered with new integration test and fetch cache pruning verified.|
|Extractor for products/components/fix levels|BE-Conn-Oracle|Source.Common|**DONE** – HTML risk matrices parsed into vendor packages with fix heuristics and normalized versions.|
|DTO schema and validation|BE-Conn-Oracle, QA|Source.Common|**DONE** – `OracleDtoValidator` enforces required fields and quarantines malformed payloads.|
|Canonical mapping with psirt_flags|BE-Conn-Oracle|Models|**DONE** – mapper now emits CVE aliases, patch references, and vendor affected packages under psirt flag provenance.|
|SourceState and dedupe|BE-Conn-Oracle|Storage.Mongo|**DONE** – cursor fetch cache tracks SHA/ETag to skip unchanged advisories and clear pending work.|
|Golden fixtures and precedence tests (later with merge)|QA|Source.Vndr.Oracle|**DONE** – snapshot fixtures and psirt flag assertions added in `OracleConnectorTests`.|
|Dependency injection routine & job registration|BE-Conn-Oracle|Core|**DONE** – `OracleDependencyInjectionRoutine` registers connector and fetch/parse/map jobs with scheduler defaults.|
|Implement Oracle connector skeleton|BE-Conn-Oracle|Source.Common|**DONE** – fetch/parse/map pipeline persists documents, DTOs, advisories, psirt flags.|
|Range primitives & provenance backfill|BE-Conn-Oracle|Models, Storage.Mongo|**DONE** – vendor primitives emitted (extensions + fix parsing), provenance tagging/logging extended, snapshots refreshed.|

16
src/Concelier/__Libraries/StellaOps.Concelier.Connector.Vndr.Oracle/TASKS.md
View File

@@ -1,13 +1,3 @@
# TASKS
| Task | Owner(s) | Depends on | Notes |
|---|---|---|---|
|Oracle options & HttpClient configuration|BE-Conn-Oracle|Source.Common|**DONE** – `AddOracleConnector` wires options and allowlisted HttpClient.|
|CPU calendar plus advisory fetchers|BE-Conn-Oracle|Source.Common|**DONE** – resume/backfill scenario covered with new integration test and fetch cache pruning verified.|
|Extractor for products/components/fix levels|BE-Conn-Oracle|Source.Common|**DONE** – HTML risk matrices parsed into vendor packages with fix heuristics and normalized versions.|
|DTO schema and validation|BE-Conn-Oracle, QA|Source.Common|**DONE** – `OracleDtoValidator` enforces required fields and quarantines malformed payloads.|
|Canonical mapping with psirt_flags|BE-Conn-Oracle|Models|**DONE** – mapper now emits CVE aliases, patch references, and vendor affected packages under psirt flag provenance.|
|SourceState and dedupe|BE-Conn-Oracle|Storage.Mongo|**DONE** – cursor fetch cache tracks SHA/ETag to skip unchanged advisories and clear pending work.|
|Golden fixtures and precedence tests (later with merge)|QA|Source.Vndr.Oracle|**DONE** – snapshot fixtures and psirt flag assertions added in `OracleConnectorTests`.|
|Dependency injection routine & job registration|BE-Conn-Oracle|Core|**DONE** – `OracleDependencyInjectionRoutine` registers connector and fetch/parse/map jobs with scheduler defaults.|
|Implement Oracle connector skeleton|BE-Conn-Oracle|Source.Common|**DONE** – fetch/parse/map pipeline persists documents, DTOs, advisories, psirt flags.|
|Range primitives & provenance backfill|BE-Conn-Oracle|Models, Storage.Mongo|**DONE** – vendor primitives emitted (extensions + fix parsing), provenance tagging/logging extended, snapshots refreshed.|
# TASKS
| Task | Owner(s) | Depends on | Notes |
|---|---|---|---|

22
src/Concelier/__Libraries/StellaOps.Concelier.Connector.Vndr.Vmware/TASKS.completed.md Normal file
View File

@@ -0,0 +1,22 @@
# Completed Tasks
| VM1 | Advisory listing discovery + cursor | Conn | DONE | Common | **DONE** – fetch pipeline uses index JSON with sliding cursor + processed id tracking. |
| VM2 | VMSA parser → DTO | QA | DONE | | **DONE** – JSON DTO deserialization wired with sanitization. |
| VM3 | Canonical mapping (aliases/affected/refs) | Conn | DONE | Models | **DONE** – `VmwareMapper` emits aliases/affected/reference ordering and persists PSIRT flags via `PsirtFlagStore`. |
| VM4 | Snapshot tests + resume | QA | DONE | Storage | **DONE** – integration test validates snapshot output and resume flow with cached state. |
| VM5 | Observability | QA | DONE | | **DONE** – diagnostics meter exposes fetch/parse/map metrics and structured logs. |
| VM6 | SourceState + hash dedupe | Conn | DONE | Storage | **DONE** – fetch cache stores sha/etag to skip unchanged advisories during resume. |
| VM6a | Options & HttpClient configuration | Conn | DONE | Source.Common | **DONE** – `AddVmwareConnector` configures allowlisted HttpClient + options. |
| VM7 | Dependency injection routine & scheduler registration | Conn | DONE | Core | **DONE** – `VmwareDependencyInjectionRoutine` registers fetch/parse/map jobs. |
| VM8 | Replace stub plugin with connector pipeline skeleton | Conn | DONE | Source.Common | **DONE** – connector implements fetch/parse/map persisting docs, DTOs, advisories. |
| VM9 | Range primitives + provenance diagnostics refresh | Conn | DONE | Models, Storage.Mongo | Vendor primitives emitted (SemVer + vendor extensions), provenance tags/logging updated, snapshots refreshed. |

24
src/Concelier/__Libraries/StellaOps.Concelier.Connector.Vndr.Vmware/TASKS.md
View File

@@ -1,17 +1,7 @@
# Source.Vndr.Vmware — Task Board
| ID | Task | Owner | Status | Depends On | Notes |
|------|-----------------------------------------------|-------|--------|------------|-------|
| VM1 | Advisory listing discovery + cursor | Conn | DONE | Common | **DONE** – fetch pipeline uses index JSON with sliding cursor + processed id tracking. |
| VM2 | VMSA parser → DTO | QA | DONE | | **DONE** – JSON DTO deserialization wired with sanitization. |
| VM3 | Canonical mapping (aliases/affected/refs) | Conn | DONE | Models | **DONE** – `VmwareMapper` emits aliases/affected/reference ordering and persists PSIRT flags via `PsirtFlagStore`. |
| VM4 | Snapshot tests + resume | QA | DONE | Storage | **DONE** – integration test validates snapshot output and resume flow with cached state. |
| VM5 | Observability | QA | DONE | | **DONE** – diagnostics meter exposes fetch/parse/map metrics and structured logs. |
| VM6 | SourceState + hash dedupe | Conn | DONE | Storage | **DONE** – fetch cache stores sha/etag to skip unchanged advisories during resume. |
| VM6a | Options & HttpClient configuration | Conn | DONE | Source.Common | **DONE** – `AddVmwareConnector` configures allowlisted HttpClient + options. |
| VM7 | Dependency injection routine & scheduler registration | Conn | DONE | Core | **DONE** – `VmwareDependencyInjectionRoutine` registers fetch/parse/map jobs. |
| VM8 | Replace stub plugin with connector pipeline skeleton | Conn | DONE | Source.Common | **DONE** – connector implements fetch/parse/map persisting docs, DTOs, advisories. |
| VM9 | Range primitives + provenance diagnostics refresh | Conn | DONE | Models, Storage.Mongo | Vendor primitives emitted (SemVer + vendor extensions), provenance tags/logging updated, snapshots refreshed. |
## Changelog
- YYYY-MM-DD: Created.
# Source.Vndr.Vmware — Task Board
| ID | Task | Owner | Status | Depends On | Notes |
|------|-----------------------------------------------|-------|--------|------------|-------|
## Changelog
- YYYY-MM-DD: Created.

11
src/Concelier/__Libraries/StellaOps.Concelier.Core/TASKS.completed.md Normal file
View File

@@ -0,0 +1,11 @@
# Completed Tasks
| ID | Status | Owner(s) | Depends on | Notes |
|---|---|---|---|---|
| CONCELIER-CORE-AOC-19-001 `AOC write guard` | DONE (2025-10-29) | Concelier Core Guild | WEB-AOC-19-001 | Implement repository interceptor that inspects write payloads for forbidden AOC keys, validates provenance/signature presence, and maps violations to `ERR_AOC_00x`. |
| CONCELIER-CORE-AOC-19-002 `Deterministic linkset extraction` | DONE (2025-10-31) | Concelier Core Guild | CONCELIER-CORE-AOC-19-001 | Build canonical linkset mappers for CVE/GHSA/PURL/CPE/reference extraction from upstream raw payloads, ensuring reconciled-from metadata is tracked and deterministic. |
| CONCELIER-CORE-AOC-19-003 `Idempotent append-only upsert` | DONE (2025-10-28) | Concelier Core Guild | CONCELIER-STORE-AOC-19-002 | Implement idempotent upsert path using `(vendor, upstreamId, contentHash, tenant)` key, emitting supersedes pointers for new revisions and preventing duplicate inserts. |
| ID | Status | Owner(s) | Depends on | Notes |
|----|--------|----------|------------|-------|
| CONCELIER-GRAPH-24-001 `Advisory overlay inputs` | DONE (2025-10-29) | Concelier Core Guild | CONCELIER-POLICY-23-001 | Expose raw advisory observations/linksets with tenant filters for overlay services; no derived counts/severity in ingestion. |

222
src/Concelier/__Libraries/StellaOps.Concelier.Core/TASKS.md
View File

@@ -1,122 +1,118 @@
# TASKS — Epic 1: Aggregation-Only Contract
> **AOC Reminder:** ingestion aggregates and links only—no precedence, normalization, or severity computation. Derived data lives in Policy/overlay services.
| ID | Status | Owner(s) | Depends on | Notes |
|---|---|---|---|---|
| CONCELIER-CORE-AOC-19-001 `AOC write guard` | DONE (2025-10-29) | Concelier Core Guild | WEB-AOC-19-001 | Implement repository interceptor that inspects write payloads for forbidden AOC keys, validates provenance/signature presence, and maps violations to `ERR_AOC_00x`. |
> Docs alignment (2025-10-26): Behaviour/spec captured in `docs/ingestion/aggregation-only-contract.md` and architecture overview §2.
> Implementation (2025-10-29): Added `AdvisoryRawWriteGuard` + DI extensions wrapping `AocWriteGuard`, throwing domain-specific `ConcelierAocGuardException` with `ERR_AOC_00x` mappings. Unit tests cover valid/missing-tenant/signature cases.
> Coordination (2025-10-27): Authority `dotnet test` run is currently blocked because `AdvisoryObservationQueryService.BuildAliasLookup` returns `ImmutableHashSet<string?>`; please normalise these lookups to `ImmutableHashSet<string>` (trim nulls) so downstream builds succeed.
| CONCELIER-CORE-AOC-19-002 `Deterministic linkset extraction` | DONE (2025-10-31) | Concelier Core Guild | CONCELIER-CORE-AOC-19-001 | Build canonical linkset mappers for CVE/GHSA/PURL/CPE/reference extraction from upstream raw payloads, ensuring reconciled-from metadata is tracked and deterministic. |
> 2025-10-31: Added advisory linkset mapper + DI registration, normalized PURL/CPE canonicalization, persisted `reconciled_from` pointers, and refreshed observation factory/tests for new raw linkset shape.
> Docs alignment (2025-10-26): Linkset expectations detailed in AOC reference §4 and policy-engine architecture §2.1.
| CONCELIER-CORE-AOC-19-003 `Idempotent append-only upsert` | DONE (2025-10-28) | Concelier Core Guild | CONCELIER-STORE-AOC-19-002 | Implement idempotent upsert path using `(vendor, upstreamId, contentHash, tenant)` key, emitting supersedes pointers for new revisions and preventing duplicate inserts. |
> 2025-10-28: Advisory raw ingestion now strips client-supplied supersedes hints, logs ignored pointers, and surfaces repository-supplied supersedes identifiers; service tests cover duplicate handling and append-only semantics.
> Docs alignment (2025-10-26): Deployment guide + observability guide describe supersedes metrics; ensure implementation emits `aoc_violation_total` on failure.
# TASKS — Epic 1: Aggregation-Only Contract
> **AOC Reminder:** ingestion aggregates and links only—no precedence, normalization, or severity computation. Derived data lives in Policy/overlay services.
| ID | Status | Owner(s) | Depends on | Notes |
|---|---|---|---|---|
> Docs alignment (2025-10-26): Behaviour/spec captured in `docs/ingestion/aggregation-only-contract.md` and architecture overview §2.
> Implementation (2025-10-29): Added `AdvisoryRawWriteGuard` + DI extensions wrapping `AocWriteGuard`, throwing domain-specific `ConcelierAocGuardException` with `ERR_AOC_00x` mappings. Unit tests cover valid/missing-tenant/signature cases.
> Coordination (2025-10-27): Authority `dotnet test` run is currently blocked because `AdvisoryObservationQueryService.BuildAliasLookup` returns `ImmutableHashSet<string?>`; please normalise these lookups to `ImmutableHashSet<string>` (trim nulls) so downstream builds succeed.
> 2025-10-31: Added advisory linkset mapper + DI registration, normalized PURL/CPE canonicalization, persisted `reconciled_from` pointers, and refreshed observation factory/tests for new raw linkset shape.
> Docs alignment (2025-10-26): Linkset expectations detailed in AOC reference §4 and policy-engine architecture §2.1.
> 2025-10-28: Advisory raw ingestion now strips client-supplied supersedes hints, logs ignored pointers, and surfaces repository-supplied supersedes identifiers; service tests cover duplicate handling and append-only semantics.
> Docs alignment (2025-10-26): Deployment guide + observability guide describe supersedes metrics; ensure implementation emits `aoc_violation_total` on failure.
| CONCELIER-CORE-AOC-19-004 `Remove ingestion normalization` | DOING (2025-10-28) | Concelier Core Guild | CONCELIER-CORE-AOC-19-002, POLICY-AOC-19-003 | Strip normalization/dedup/severity logic from ingestion pipelines, delegate derived computations to Policy Engine, and update exporters/tests to consume raw documents only.<br>2025-10-29 19:05Z: Audit completed for `AdvisoryRawService`/Mongo repo to confirm alias order/dedup removal persists; identified remaining normalization in observation/linkset factory that will be revised to surface raw duplicates for Policy ingestion. Change sketch + regression matrix drafted under `docs/dev/aoc-normalization-removal-notes.md` (pending commit). |
> Docs alignment (2025-10-26): Architecture overview emphasises policy-only derivation; coordinate with Policy Engine guild for rollout.
> 2025-10-29: `AdvisoryRawService` now preserves upstream alias/linkset ordering (trim-only) and updated AOC documentation reflects the behaviour; follow-up to ensure policy consumers handle duplicates remains open.
| CONCELIER-CORE-AOC-19-013 `Authority tenant scope smoke coverage` | TODO | Concelier Core Guild | AUTH-AOC-19-002 | Extend Concelier smoke/e2e fixtures to configure `requiredTenants` and assert cross-tenant rejection with updated Authority tokens. | Coordinate deliverable so Authority docs (`AUTH-AOC-19-003`) can close once tests are in place. |
## Policy Engine v2
| ID | Status | Owner(s) | Depends on | Notes |
|----|--------|----------|------------|-------|
| CONCELIER-POLICY-20-002 `Linkset enrichment for policy` | TODO | Concelier Core Guild, Policy Guild | CONCELIER-CORE-AOC-19-002, POLICY-ENGINE-20-001 | Strengthen linkset builders with vendor-specific equivalence tables, NEVRA/PURL normalization, and version range parsing to maximize policy join recall; update fixtures + docs. |
> 2025-10-31: Base advisory linkset mapper landed under `CONCELIER-CORE-AOC-19-002`; policy enrichment work can now proceed with mapper outputs and observation schema fixtures.
## Graph Explorer v1
| ID | Status | Owner(s) | Depends on | Notes |
|----|--------|----------|------------|-------|
| CONCELIER-CORE-AOC-19-013 `Authority tenant scope smoke coverage` | TODO | Concelier Core Guild | AUTH-AOC-19-002 | Extend Concelier smoke/e2e fixtures to configure `requiredTenants` and assert cross-tenant rejection with updated Authority tokens. | Coordinate deliverable so Authority docs (`AUTH-AOC-19-003`) can close once tests are in place. |
## Policy Engine v2
| ID | Status | Owner(s) | Depends on | Notes |
|----|--------|----------|------------|-------|
| CONCELIER-POLICY-20-002 `Linkset enrichment for policy` | TODO | Concelier Core Guild, Policy Guild | CONCELIER-CORE-AOC-19-002, POLICY-ENGINE-20-001 | Strengthen linkset builders with vendor-specific equivalence tables, NEVRA/PURL normalization, and version range parsing to maximize policy join recall; update fixtures + docs. |
> 2025-10-31: Base advisory linkset mapper landed under `CONCELIER-CORE-AOC-19-002`; policy enrichment work can now proceed with mapper outputs and observation schema fixtures.
## Graph Explorer v1
| ID | Status | Owner(s) | Depends on | Notes |
|----|--------|----------|------------|-------|
| CONCELIER-GRAPH-21-001 `SBOM projection enrichment` | BLOCKED (2025-10-27) | Concelier Core Guild, Cartographer Guild | CONCELIER-POLICY-20-002, CARTO-GRAPH-21-002 | Extend SBOM normalization to emit full relationship graph (depends_on/contains/provides), scope tags, entrypoint annotations, and component metadata required by Cartographer. |
> 2025-10-27: Waiting on policy-driven linkset enrichment (`CONCELIER-POLICY-20-002`) and Cartographer API contract (`CARTO-GRAPH-21-002`) to define required relationship payloads. Without those schemas the projection changes cannot be implemented deterministically.
> 2025-10-29: Cross-guild handshake captured in `docs/dev/cartographer-graph-handshake.md`; begin drafting enrichment plan once Cartographer ships the inspector schema/query patterns.
| CONCELIER-GRAPH-21-002 `Change events` | BLOCKED (2025-10-27) | Concelier Core Guild, Scheduler Guild | CONCELIER-GRAPH-21-001 | Publish change events (new SBOM version, relationship delta) for Cartographer build queue; ensure events include tenant/context metadata. |
> 2025-10-27: Depends on `CONCELIER-GRAPH-21-001`; event schema hinges on finalized projection output and Cartographer webhook contract, both pending.
> 2025-10-29: Action item from handshake doc — prepare sample `sbom.relationship.changed` payload + replay notes once schema lands; coordinate with Scheduler for queue semantics.
## Link-Not-Merge v1
| ID | Status | Owner(s) | Depends on | Notes |
|----|--------|----------|------------|-------|
| CONCELIER-LNM-21-001 `Advisory observation schema` | TODO | Concelier Core Guild | CONCELIER-CORE-AOC-19-001 | Introduce immutable `advisory_observations` model with AOC metadata, raw payload pointers, normalized fields, and tenancy guardrails; publish schema definition. `DOCS-LNM-22-001` blocked pending this deliverable. |
| CONCELIER-LNM-21-002 `Linkset builder` | TODO | Concelier Core Guild, Data Science Guild | CONCELIER-LNM-21-001 | Implement correlation pipeline (alias graph, PURL overlap, CVSS vector equality, fuzzy title match) that produces `advisory_linksets` with confidence + conflict annotations. Docs note: unblock `DOCS-LNM-22-001` once builder lands. |
| CONCELIER-LNM-21-003 `Conflict annotator` | TODO | Concelier Core Guild | CONCELIER-LNM-21-002 | Detect field disagreements (severity, CVSS, ranges, references) and record structured conflicts on linksets; surface to API/UI. Docs awaiting structured conflict payloads. |
| CONCELIER-LNM-21-004 `Merge code removal` | TODO | Concelier Core Guild | CONCELIER-LNM-21-002 | Excise existing merge/dedup logic, enforce immutability on observations, and add guards/tests to prevent future merges. |
| CONCELIER-LNM-21-005 `Event emission` | TODO | Concelier Core Guild, Platform Events Guild | CONCELIER-LNM-21-002 | Emit `advisory.linkset.updated` events with delta payloads for downstream Policy Engine/Cartographer consumers; ensure idempotent delivery. |
## Policy Engine + Editor v1
| ID | Status | Owner(s) | Depends on | Notes |
|----|--------|----------|------------|-------|
| CONCELIER-POLICY-23-001 `Evidence indexes` | TODO | Concelier Core Guild | CONCELIER-LNM-21-002 | Add secondary indexes/materialized views to accelerate policy lookups (alias, severity per observation, correlation confidence). Document query contracts for runtime. |
| CONCELIER-POLICY-23-002 `Event guarantees` | TODO | Concelier Core Guild, Platform Events Guild | CONCELIER-LNM-21-005 | Ensure `advisory.linkset.updated` emits at-least-once with idempotent keys and include policy-relevant metadata (confidence, conflict summary). |
## Graph & Vuln Explorer v1
| ID | Status | Owner(s) | Depends on | Notes |
|----|--------|----------|------------|-------|
| CONCELIER-GRAPH-24-001 `Advisory overlay inputs` | DONE (2025-10-29) | Concelier Core Guild | CONCELIER-POLICY-23-001 | Expose raw advisory observations/linksets with tenant filters for overlay services; no derived counts/severity in ingestion. |
> 2025-10-29: Filter-aware lookup path and /concelier/observations coverage landed; overlay services can consume raw advisory feeds deterministically.
## Reachability v1
| ID | Status | Owner(s) | Depends on | Notes |
|----|--------|----------|------------|-------|
| CONCELIER-SIG-26-001 `Vulnerable symbol exposure` | TODO | Concelier Core Guild, Signals Guild | SIGNALS-24-002 | Expose advisory metadata (affected symbols/functions) via API to enrich reachability scoring; update fixtures. |
## Orchestrator Dashboard
| ID | Status | Owner(s) | Depends on | Notes |
|----|--------|----------|------------|-------|
| CONCELIER-ORCH-32-001 `Source registry integration` | TODO | Concelier Core Guild | ORCH-SVC-32-001, AUTH-ORCH-32-001 | Register Concelier data sources with orchestrator (metadata, schedules, rate policies) and wire provenance IDs/security scopes. |
| CONCELIER-ORCH-32-002 `Worker SDK adoption` | TODO | Concelier Core Guild | CONCELIER-ORCH-32-001, WORKER-GO-32-001, WORKER-PY-32-001 | Embed orchestrator worker SDK in ingestion loops, emit heartbeats/progress/artifact hashes, and enforce idempotency keys. |
| CONCELIER-ORCH-33-001 `Control hook compliance` | TODO | Concelier Core Guild | CONCELIER-ORCH-32-002, ORCH-SVC-33-001, ORCH-SVC-33-002 | Honor orchestrator throttle/pause/retry actions, surface structured error classes, and persist safe checkpoints for resume. |
| CONCELIER-ORCH-34-001 `Backfill + ledger linkage` | TODO | Concelier Core Guild | CONCELIER-ORCH-33-001, ORCH-SVC-33-003, ORCH-SVC-34-001 | Execute orchestrator-driven backfills, reuse artifact hashes to avoid duplicates, and link provenance to run ledger exports. |
## Authority-Backed Scopes & Tenancy (Epic 14)
| ID | Status | Owner(s) | Depends on | Notes |
|----|--------|----------|------------|-------|
| CONCELIER-TEN-48-001 `Tenant-aware linking` | TODO | Concelier Core Guild | AUTH-TEN-47-001 | Ensure advisory normalization/linking runs per tenant with RLS enforcing isolation; emit capability endpoint reporting `merge=false`; update events with tenant context. |
## Observability & Forensics (Epic 15)
| ID | Status | Owner(s) | Depends on | Notes |
|----|--------|----------|------------|-------|
| CONCELIER-OBS-50-001 `Telemetry adoption` | TODO | Concelier Core Guild, Observability Guild | TELEMETRY-OBS-50-001, TELEMETRY-OBS-50-002 | Replace ad-hoc logging with telemetry core across ingestion/linking pipelines; ensure spans/logs include tenant, source vendor, upstream id, content hash, and trace IDs. |
| CONCELIER-OBS-51-001 `Metrics & SLOs` | TODO | Concelier Core Guild, DevOps Guild | CONCELIER-OBS-50-001, TELEMETRY-OBS-51-001 | Emit metrics for ingest latency (cold/warm), queue depth, aoc violation rate, and publish SLO burn-rate alerts (ingest P95 <30s cold / <5s warm). Ship dashboards + alert configs. |
| CONCELIER-OBS-52-001 `Timeline events` | TODO | Concelier Core Guild | CONCELIER-OBS-50-001, TIMELINE-OBS-52-002 | Emit `timeline_event` records for advisory ingest/normalization/linkset creation with provenance, trace IDs, conflict summaries, and evidence placeholders. |
| CONCELIER-OBS-53-001 `Evidence snapshots` | TODO | Concelier Core Guild, Evidence Locker Guild | CONCELIER-OBS-52-001, EVID-OBS-53-002 | Produce advisory evaluation bundle payloads (raw doc, linkset, normalization diff) for evidence locker; ensure Merkle manifests seeded with content hashes. |
| CONCELIER-OBS-54-001 `Attestation & verification` | TODO | Concelier Core Guild, Provenance Guild | CONCELIER-OBS-53-001, PROV-OBS-54-001 | Attach DSSE attestations for advisory processing batches, expose verification API to confirm bundle integrity, and link attestation IDs back to timeline + ledger. |
| CONCELIER-OBS-55-001 `Incident mode hooks` | TODO | Concelier Core Guild, DevOps Guild | CONCELIER-OBS-51-001, DEVOPS-OBS-55-001 | Increase sampling, capture raw payload snapshots, and extend retention under incident mode; emit activation events + guardrails against PII leak. |
## Air-Gapped Mode (Epic 16)
| ID | Status | Owner(s) | Depends on | Notes |
|----|--------|----------|------------|-------|
| CONCELIER-AIRGAP-56-001 `Mirror ingestion adapters` | TODO | Concelier Core Guild | AIRGAP-IMP-57-002, MIRROR-CRT-56-001 | Add mirror source adapters reading advisories from imported bundles, preserving source metadata and bundle IDs. Ensure ingestion remains append-only. |
| CONCELIER-AIRGAP-56-002 `Bundle catalog linking` | TODO | Concelier Core Guild, AirGap Importer Guild | CONCELIER-AIRGAP-56-001, AIRGAP-IMP-57-001 | Persist `bundle_id`, `merkle_root`, and time anchor references on observations/linksets for provenance. |
| CONCELIER-AIRGAP-57-001 `Sealed-mode source restrictions` | TODO | Concelier Core Guild, AirGap Policy Guild | CONCELIER-AIRGAP-56-001, AIRGAP-POL-56-001 | Enforce sealed-mode egress rules by disallowing non-mirror connectors and surfacing remediation errors. |
| CONCELIER-AIRGAP-57-002 `Staleness annotations` | TODO | Concelier Core Guild, AirGap Time Guild | CONCELIER-AIRGAP-56-002, AIRGAP-TIME-58-001 | Compute staleness metadata for advisories per bundle and expose via API for Console/CLI badges. |
| CONCELIER-AIRGAP-58-001 `Portable advisory evidence` | TODO | Concelier Core Guild, Evidence Locker Guild | CONCELIER-OBS-53-001, EVID-OBS-54-001 | Package advisory evidence fragments into portable evidence bundles for cross-domain transfer. |
## SDKs & OpenAPI (Epic 17)
| ID | Status | Owner(s) | Depends on | Notes |
|----|--------|----------|------------|-------|
| CONCELIER-OAS-61-001 `Spec coverage` | TODO | Concelier Core Guild, API Contracts Guild | OAS-61-001 | Update Concelier OAS with advisory observation/linkset endpoints, standard pagination, and source provenance fields. |
| CONCELIER-OAS-61-002 `Examples library` | TODO | Concelier Core Guild | CONCELIER-OAS-61-001 | Provide rich examples for advisories, linksets, conflict annotations used by SDK + docs. |
| CONCELIER-OAS-62-001 `SDK smoke tests` | TODO | Concelier Core Guild, SDK Generator Guild | CONCELIER-OAS-61-001, SDKGEN-63-001 | Add SDK tests covering advisory search, pagination, and conflict handling; ensure source metadata surfaced. |
| CONCELIER-OAS-63-001 `Deprecation headers` | TODO | Concelier Core Guild, API Governance Guild | APIGOV-63-001 | Implement deprecation header support and timeline events for retiring endpoints. |
## Risk Profiles (Epic 18)
| ID | Status | Owner(s) | Depends on | Notes |
|----|--------|----------|------------|-------|
| CONCELIER-RISK-66-001 `CVSS/KEV providers` | TODO | Concelier Core Guild, Risk Engine Guild | RISK-ENGINE-67-001 | Expose CVSS, KEV, fix availability data via provider APIs with source metadata preserved. |
| CONCELIER-RISK-66-002 `Fix availability signals` | TODO | Concelier Core Guild | CONCELIER-RISK-66-001 | Provide structured fix availability and release metadata consumable by risk engine; document provenance. |
| CONCELIER-RISK-67-001 `Source consensus metrics` | TODO | Concelier Core Guild | CONCELIER-RISK-66-001 | Add consensus counts and confidence scores for linked advisories; ensure explainability includes source digests. |
| CONCELIER-RISK-68-001 `Policy Studio integration` | TODO | Concelier Core Guild, Policy Studio Guild | POLICY-RISK-68-001 | Surface advisory fields in Policy Studio profile editor (signal pickers, reducers). |
| CONCELIER-RISK-69-001 `Notification hooks` | TODO | Concelier Core Guild, Notifications Guild | CONCELIER-RISK-66-002 | Emit events when advisory signals change impacting risk scores (e.g., fix available). |
## Attestor Console (Epic 19)
| ID | Status | Owner(s) | Depends on | Notes |
|----|--------|----------|------------|-------|
| CONCELIER-ATTEST-73-001 `ScanResults attestation inputs` | TODO | Concelier Core Guild, Attestor Service Guild | ATTEST-TYPES-72-001 | Provide normalized advisory data and linkset digests needed for ScanResults attestations. |
| CONCELIER-ATTEST-73-002 `Transparency metadata` | TODO | Concelier Core Guild | CONCELIER-ATTEST-73-001 | Ensure Conseiller exposes source digests for transparency proofs and explainability. |
## Link-Not-Merge v1
| ID | Status | Owner(s) | Depends on | Notes |
|----|--------|----------|------------|-------|
| CONCELIER-LNM-21-001 `Advisory observation schema` | TODO | Concelier Core Guild | CONCELIER-CORE-AOC-19-001 | Introduce immutable `advisory_observations` model with AOC metadata, raw payload pointers, normalized fields, and tenancy guardrails; publish schema definition. `DOCS-LNM-22-001` blocked pending this deliverable. |
| CONCELIER-LNM-21-002 `Linkset builder` | TODO | Concelier Core Guild, Data Science Guild | CONCELIER-LNM-21-001 | Implement correlation pipeline (alias graph, PURL overlap, CVSS vector equality, fuzzy title match) that produces `advisory_linksets` with confidence + conflict annotations. Docs note: unblock `DOCS-LNM-22-001` once builder lands. |
| CONCELIER-LNM-21-003 `Conflict annotator` | TODO | Concelier Core Guild | CONCELIER-LNM-21-002 | Detect field disagreements (severity, CVSS, ranges, references) and record structured conflicts on linksets; surface to API/UI. Docs awaiting structured conflict payloads. |
| CONCELIER-LNM-21-004 `Merge code removal` | TODO | Concelier Core Guild | CONCELIER-LNM-21-002 | Excise existing merge/dedup logic, enforce immutability on observations, and add guards/tests to prevent future merges. |
| CONCELIER-LNM-21-005 `Event emission` | TODO | Concelier Core Guild, Platform Events Guild | CONCELIER-LNM-21-002 | Emit `advisory.linkset.updated` events with delta payloads for downstream Policy Engine/Cartographer consumers; ensure idempotent delivery. |
## Policy Engine + Editor v1
| ID | Status | Owner(s) | Depends on | Notes |
|----|--------|----------|------------|-------|
| CONCELIER-POLICY-23-001 `Evidence indexes` | TODO | Concelier Core Guild | CONCELIER-LNM-21-002 | Add secondary indexes/materialized views to accelerate policy lookups (alias, severity per observation, correlation confidence). Document query contracts for runtime. |
| CONCELIER-POLICY-23-002 `Event guarantees` | TODO | Concelier Core Guild, Platform Events Guild | CONCELIER-LNM-21-005 | Ensure `advisory.linkset.updated` emits at-least-once with idempotent keys and include policy-relevant metadata (confidence, conflict summary). |
## Graph & Vuln Explorer v1
| ID | Status | Owner(s) | Depends on | Notes |
|----|--------|----------|------------|-------|
> 2025-10-29: Filter-aware lookup path and /concelier/observations coverage landed; overlay services can consume raw advisory feeds deterministically.
## Reachability v1
| ID | Status | Owner(s) | Depends on | Notes |
|----|--------|----------|------------|-------|
| CONCELIER-SIG-26-001 `Vulnerable symbol exposure` | TODO | Concelier Core Guild, Signals Guild | SIGNALS-24-002 | Expose advisory metadata (affected symbols/functions) via API to enrich reachability scoring; update fixtures. |
## Orchestrator Dashboard
| ID | Status | Owner(s) | Depends on | Notes |
|----|--------|----------|------------|-------|
| CONCELIER-ORCH-32-001 `Source registry integration` | TODO | Concelier Core Guild | ORCH-SVC-32-001, AUTH-ORCH-32-001 | Register Concelier data sources with orchestrator (metadata, schedules, rate policies) and wire provenance IDs/security scopes. |
| CONCELIER-ORCH-32-002 `Worker SDK adoption` | TODO | Concelier Core Guild | CONCELIER-ORCH-32-001, WORKER-GO-32-001, WORKER-PY-32-001 | Embed orchestrator worker SDK in ingestion loops, emit heartbeats/progress/artifact hashes, and enforce idempotency keys. |
| CONCELIER-ORCH-33-001 `Control hook compliance` | TODO | Concelier Core Guild | CONCELIER-ORCH-32-002, ORCH-SVC-33-001, ORCH-SVC-33-002 | Honor orchestrator throttle/pause/retry actions, surface structured error classes, and persist safe checkpoints for resume. |
| CONCELIER-ORCH-34-001 `Backfill + ledger linkage` | TODO | Concelier Core Guild | CONCELIER-ORCH-33-001, ORCH-SVC-33-003, ORCH-SVC-34-001 | Execute orchestrator-driven backfills, reuse artifact hashes to avoid duplicates, and link provenance to run ledger exports. |
## Authority-Backed Scopes & Tenancy (Epic 14)
| ID | Status | Owner(s) | Depends on | Notes |
|----|--------|----------|------------|-------|
| CONCELIER-TEN-48-001 `Tenant-aware linking` | TODO | Concelier Core Guild | AUTH-TEN-47-001 | Ensure advisory normalization/linking runs per tenant with RLS enforcing isolation; emit capability endpoint reporting `merge=false`; update events with tenant context. |
## Observability & Forensics (Epic 15)
| ID | Status | Owner(s) | Depends on | Notes |
|----|--------|----------|------------|-------|
| CONCELIER-OBS-50-001 `Telemetry adoption` | TODO | Concelier Core Guild, Observability Guild | TELEMETRY-OBS-50-001, TELEMETRY-OBS-50-002 | Replace ad-hoc logging with telemetry core across ingestion/linking pipelines; ensure spans/logs include tenant, source vendor, upstream id, content hash, and trace IDs. |
| CONCELIER-OBS-51-001 `Metrics & SLOs` | TODO | Concelier Core Guild, DevOps Guild | CONCELIER-OBS-50-001, TELEMETRY-OBS-51-001 | Emit metrics for ingest latency (cold/warm), queue depth, aoc violation rate, and publish SLO burn-rate alerts (ingest P95 <30s cold / <5s warm). Ship dashboards + alert configs. |
| CONCELIER-OBS-52-001 `Timeline events` | TODO | Concelier Core Guild | CONCELIER-OBS-50-001, TIMELINE-OBS-52-002 | Emit `timeline_event` records for advisory ingest/normalization/linkset creation with provenance, trace IDs, conflict summaries, and evidence placeholders. |
| CONCELIER-OBS-53-001 `Evidence snapshots` | TODO | Concelier Core Guild, Evidence Locker Guild | CONCELIER-OBS-52-001, EVID-OBS-53-002 | Produce advisory evaluation bundle payloads (raw doc, linkset, normalization diff) for evidence locker; ensure Merkle manifests seeded with content hashes. |
| CONCELIER-OBS-54-001 `Attestation & verification` | TODO | Concelier Core Guild, Provenance Guild | CONCELIER-OBS-53-001, PROV-OBS-54-001 | Attach DSSE attestations for advisory processing batches, expose verification API to confirm bundle integrity, and link attestation IDs back to timeline + ledger. |
| CONCELIER-OBS-55-001 `Incident mode hooks` | TODO | Concelier Core Guild, DevOps Guild | CONCELIER-OBS-51-001, DEVOPS-OBS-55-001 | Increase sampling, capture raw payload snapshots, and extend retention under incident mode; emit activation events + guardrails against PII leak. |
## Air-Gapped Mode (Epic 16)
| ID | Status | Owner(s) | Depends on | Notes |
|----|--------|----------|------------|-------|
| CONCELIER-AIRGAP-56-001 `Mirror ingestion adapters` | TODO | Concelier Core Guild | AIRGAP-IMP-57-002, MIRROR-CRT-56-001 | Add mirror source adapters reading advisories from imported bundles, preserving source metadata and bundle IDs. Ensure ingestion remains append-only. |
| CONCELIER-AIRGAP-56-002 `Bundle catalog linking` | TODO | Concelier Core Guild, AirGap Importer Guild | CONCELIER-AIRGAP-56-001, AIRGAP-IMP-57-001 | Persist `bundle_id`, `merkle_root`, and time anchor references on observations/linksets for provenance. |
| CONCELIER-AIRGAP-57-001 `Sealed-mode source restrictions` | TODO | Concelier Core Guild, AirGap Policy Guild | CONCELIER-AIRGAP-56-001, AIRGAP-POL-56-001 | Enforce sealed-mode egress rules by disallowing non-mirror connectors and surfacing remediation errors. |
| CONCELIER-AIRGAP-57-002 `Staleness annotations` | TODO | Concelier Core Guild, AirGap Time Guild | CONCELIER-AIRGAP-56-002, AIRGAP-TIME-58-001 | Compute staleness metadata for advisories per bundle and expose via API for Console/CLI badges. |
| CONCELIER-AIRGAP-58-001 `Portable advisory evidence` | TODO | Concelier Core Guild, Evidence Locker Guild | CONCELIER-OBS-53-001, EVID-OBS-54-001 | Package advisory evidence fragments into portable evidence bundles for cross-domain transfer. |
## SDKs & OpenAPI (Epic 17)
| ID | Status | Owner(s) | Depends on | Notes |
|----|--------|----------|------------|-------|
| CONCELIER-OAS-61-001 `Spec coverage` | TODO | Concelier Core Guild, API Contracts Guild | OAS-61-001 | Update Concelier OAS with advisory observation/linkset endpoints, standard pagination, and source provenance fields. |
| CONCELIER-OAS-61-002 `Examples library` | TODO | Concelier Core Guild | CONCELIER-OAS-61-001 | Provide rich examples for advisories, linksets, conflict annotations used by SDK + docs. |
| CONCELIER-OAS-62-001 `SDK smoke tests` | TODO | Concelier Core Guild, SDK Generator Guild | CONCELIER-OAS-61-001, SDKGEN-63-001 | Add SDK tests covering advisory search, pagination, and conflict handling; ensure source metadata surfaced. |
| CONCELIER-OAS-63-001 `Deprecation headers` | TODO | Concelier Core Guild, API Governance Guild | APIGOV-63-001 | Implement deprecation header support and timeline events for retiring endpoints. |
## Risk Profiles (Epic 18)
| ID | Status | Owner(s) | Depends on | Notes |
|----|--------|----------|------------|-------|
| CONCELIER-RISK-66-001 `CVSS/KEV providers` | TODO | Concelier Core Guild, Risk Engine Guild | RISK-ENGINE-67-001 | Expose CVSS, KEV, fix availability data via provider APIs with source metadata preserved. |
| CONCELIER-RISK-66-002 `Fix availability signals` | TODO | Concelier Core Guild | CONCELIER-RISK-66-001 | Provide structured fix availability and release metadata consumable by risk engine; document provenance. |
| CONCELIER-RISK-67-001 `Source consensus metrics` | TODO | Concelier Core Guild | CONCELIER-RISK-66-001 | Add consensus counts and confidence scores for linked advisories; ensure explainability includes source digests. |
| CONCELIER-RISK-68-001 `Policy Studio integration` | TODO | Concelier Core Guild, Policy Studio Guild | POLICY-RISK-68-001 | Surface advisory fields in Policy Studio profile editor (signal pickers, reducers). |
| CONCELIER-RISK-69-001 `Notification hooks` | TODO | Concelier Core Guild, Notifications Guild | CONCELIER-RISK-66-002 | Emit events when advisory signals change impacting risk scores (e.g., fix available). |
## Attestor Console (Epic 19)
| ID | Status | Owner(s) | Depends on | Notes |
|----|--------|----------|------------|-------|
| CONCELIER-ATTEST-73-001 `ScanResults attestation inputs` | TODO | Concelier Core Guild, Attestor Service Guild | ATTEST-TYPES-72-001 | Provide normalized advisory data and linkset digests needed for ScanResults attestations. |
| CONCELIER-ATTEST-73-002 `Transparency metadata` | TODO | Concelier Core Guild | CONCELIER-ATTEST-73-001 | Ensure Conseiller exposes source digests for transparency proofs and explainability. |

22
src/Concelier/__Libraries/StellaOps.Concelier.Exporter.Json/TASKS.completed.md Normal file
View File

@@ -0,0 +1,22 @@
# Completed Tasks
|Directory layout strategy (vuln-list mirror)|BE-Export|Models|DONE – `VulnListJsonExportPathResolver` maps CVE, GHSA, distro, and vendor identifiers into vuln-list style paths.|
|Deterministic serializer|BE-Export|Models|DONE – Canonical serializer + snapshot builder emit stable JSON across runs.|
|ExportState read/write|BE-Export|Storage.Mongo|DONE – `JsonFeedExporter` reads prior state, stores digests/cursors, and skips unchanged exports.|
|JsonExportJob wiring|BE-Export|Core|DONE – Job scheduler options now configurable via DI; JSON job registered with scheduler.|
|Snapshot tests for file tree|QA|Exporters|DONE – Added resolver/exporter tests asserting tree layout and deterministic behavior.|
|Parity smoke vs upstream vuln-list|QA|Exporters|DONE – `JsonExporterParitySmokeTests` covers common ecosystems against vuln-list layout.|
|Stream advisories during export|BE-Export|Storage.Mongo|DONE – exporter + streaming-only test ensures single enumeration and per-file digest capture.|
|Emit export manifest with digest metadata|BE-Export|Exporters|DONE – manifest now includes per-file digests/sizes alongside tree digest.|
|Surface new advisory fields (description/CWEs/canonical metric)|BE-Export|Models, Core|DONE (2025-10-15) – JSON exporter validated with new fixtures ensuring description/CWEs/canonical metric are preserved in outputs; `dotnet test src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Exporter.Json.Tests` run 2025-10-15 for regression coverage.|
|CONCELIER-EXPORT-08-201 – Mirror bundle + domain manifest|Team Concelier Export|FEEDCORE-ENGINE-07-001|DONE (2025-10-19) – Mirror bundle writer emits domain aggregates + manifests with cosign-compatible JWS signatures; index/tests updated via `dotnet test src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Exporter.Json.Tests/StellaOps.Concelier.Exporter.Json.Tests.csproj` (2025-10-19).|

10
src/Concelier/__Libraries/StellaOps.Concelier.Exporter.Json/TASKS.md
View File

@@ -1,13 +1,3 @@
# TASKS
| Task | Owner(s) | Depends on | Notes |
|---|---|---|---|
|Directory layout strategy (vuln-list mirror)|BE-Export|Models|DONE – `VulnListJsonExportPathResolver` maps CVE, GHSA, distro, and vendor identifiers into vuln-list style paths.|
|Deterministic serializer|BE-Export|Models|DONE – Canonical serializer + snapshot builder emit stable JSON across runs.|
|ExportState read/write|BE-Export|Storage.Mongo|DONE – `JsonFeedExporter` reads prior state, stores digests/cursors, and skips unchanged exports.|
|JsonExportJob wiring|BE-Export|Core|DONE – Job scheduler options now configurable via DI; JSON job registered with scheduler.|
|Snapshot tests for file tree|QA|Exporters|DONE – Added resolver/exporter tests asserting tree layout and deterministic behavior.|
|Parity smoke vs upstream vuln-list|QA|Exporters|DONE – `JsonExporterParitySmokeTests` covers common ecosystems against vuln-list layout.|
|Stream advisories during export|BE-Export|Storage.Mongo|DONE – exporter + streaming-only test ensures single enumeration and per-file digest capture.|
|Emit export manifest with digest metadata|BE-Export|Exporters|DONE – manifest now includes per-file digests/sizes alongside tree digest.|
|Surface new advisory fields (description/CWEs/canonical metric)|BE-Export|Models, Core|DONE (2025-10-15) – JSON exporter validated with new fixtures ensuring description/CWEs/canonical metric are preserved in outputs; `dotnet test src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Exporter.Json.Tests` run 2025-10-15 for regression coverage.|
|CONCELIER-EXPORT-08-201 – Mirror bundle + domain manifest|Team Concelier Export|FEEDCORE-ENGINE-07-001|DONE (2025-10-19) – Mirror bundle writer emits domain aggregates + manifests with cosign-compatible JWS signatures; index/tests updated via `dotnet test src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Exporter.Json.Tests/StellaOps.Concelier.Exporter.Json.Tests.csproj` (2025-10-19).|

26
src/Concelier/__Libraries/StellaOps.Concelier.Exporter.TrivyDb/TASKS.completed.md Normal file
View File

@@ -0,0 +1,26 @@
# Completed Tasks
|Fix method name typo GetExportRoot' -> GetExportRoot|BE-Export|Exporters|DONE – `TrivyDbExportOptions.GetExportRoot` helper added with unit coverage.|
|Implement BoltDB builder integration (v0 via trivy-db CLI)|BE-Export|Env|DONE – `TrivyDbBoltBuilder` shells `trivy-db build` against our JSON tree with deterministic packaging.|
|Pack db.tar.gz + metadata.json|BE-Export|Exporters|DONE – Builder output re-packed with fixed timestamps and zeroed gzip mtime.|
|ORAS push support|BE-Export|Exporters|DONE – Optional `TrivyDbOrasPusher` shells `oras cp --from-oci-layout` with configurable args/env.|
|Offline bundle toggle|BE-Export|Exporters|DONE – Deterministic OCI layout bundle emitted when enabled.|
|Deterministic ordering of advisories|BE-Export|Models|DONE – exporter now loads advisories, sorts by advisoryKey, and emits sorted JSON trees with deterministic OCI payloads.|
|End-to-end tests with small dataset|QA|Exporters|DONE – added deterministic round-trip test covering OCI layout, media types, and digest stability w/ repeated inputs.|
|ExportState persistence & idempotence|BE-Export|Storage.Mongo|DONE – baseline resets wired into `ExportStateManager`, planner signals resets after delta runs, and exporters update state w/ repository-aware baseline rotation + tests.|
|Streamed package building to avoid large copies|BE-Export|Exporters|DONE – metadata/config now reuse backing arrays and OCI writer streams directly without double buffering.|
|Plan incremental/delta exports|BE-Export|Exporters|DONE – state captures per-file manifests, planner schedules delta vs full resets, layer reuse smoke test verifies OCI reuse, and operator guide documents the validation flow.|
|Advisory schema parity export (description/CWEs/canonical metric)|BE-Export|Models, Core|DONE (2025-10-15) – exporter/test fixtures updated to handle description/CWEs/canonical metric fields during Trivy DB packaging; `dotnet test src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Exporter.TrivyDb.Tests` re-run 2025-10-15 to confirm coverage.|
|CONCELIER-EXPORT-08-202 – Mirror-ready Trivy DB bundles|Team Concelier Export|CONCELIER-EXPORT-08-201|**DONE (2025-10-19)** – Added mirror export options and writer emitting `mirror/index.json` plus per-domain `manifest.json`/`metadata.json`/`db.tar.gz` with deterministic SHA-256 digests; regression covered via `dotnet test src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Exporter.TrivyDb.Tests/StellaOps.Concelier.Exporter.TrivyDb.Tests.csproj`.|

12
src/Concelier/__Libraries/StellaOps.Concelier.Exporter.TrivyDb/TASKS.md
View File

@@ -1,15 +1,3 @@
# TASKS
| Task | Owner(s) | Depends on | Notes |
|---|---|---|---|
|Fix method name typo GetExportRoot' -> GetExportRoot|BE-Export|Exporters|DONE – `TrivyDbExportOptions.GetExportRoot` helper added with unit coverage.|
|Implement BoltDB builder integration (v0 via trivy-db CLI)|BE-Export|Env|DONE – `TrivyDbBoltBuilder` shells `trivy-db build` against our JSON tree with deterministic packaging.|
|Pack db.tar.gz + metadata.json|BE-Export|Exporters|DONE – Builder output re-packed with fixed timestamps and zeroed gzip mtime.|
|ORAS push support|BE-Export|Exporters|DONE – Optional `TrivyDbOrasPusher` shells `oras cp --from-oci-layout` with configurable args/env.|
|Offline bundle toggle|BE-Export|Exporters|DONE – Deterministic OCI layout bundle emitted when enabled.|
|Deterministic ordering of advisories|BE-Export|Models|DONE – exporter now loads advisories, sorts by advisoryKey, and emits sorted JSON trees with deterministic OCI payloads.|
|End-to-end tests with small dataset|QA|Exporters|DONE – added deterministic round-trip test covering OCI layout, media types, and digest stability w/ repeated inputs.|
|ExportState persistence & idempotence|BE-Export|Storage.Mongo|DONE – baseline resets wired into `ExportStateManager`, planner signals resets after delta runs, and exporters update state w/ repository-aware baseline rotation + tests.|
|Streamed package building to avoid large copies|BE-Export|Exporters|DONE – metadata/config now reuse backing arrays and OCI writer streams directly without double buffering.|
|Plan incremental/delta exports|BE-Export|Exporters|DONE – state captures per-file manifests, planner schedules delta vs full resets, layer reuse smoke test verifies OCI reuse, and operator guide documents the validation flow.|
|Advisory schema parity export (description/CWEs/canonical metric)|BE-Export|Models, Core|DONE (2025-10-15) – exporter/test fixtures updated to handle description/CWEs/canonical metric fields during Trivy DB packaging; `dotnet test src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Exporter.TrivyDb.Tests` re-run 2025-10-15 to confirm coverage.|
|CONCELIER-EXPORT-08-202 – Mirror-ready Trivy DB bundles|Team Concelier Export|CONCELIER-EXPORT-08-201|**DONE (2025-10-19)** – Added mirror export options and writer emitting `mirror/index.json` plus per-domain `manifest.json`/`metadata.json`/`db.tar.gz` with deterministic SHA-256 digests; regression covered via `dotnet test src/Concelier/StellaOps.Concelier.PluginBinaries/StellaOps.Concelier.Exporter.TrivyDb.Tests/StellaOps.Concelier.Exporter.TrivyDb.Tests.csproj`.|

37
src/Concelier/__Libraries/StellaOps.Concelier.Merge/TASKS.completed.md Normal file
View File

@@ -0,0 +1,37 @@
# Completed Tasks
|Identity graph and alias resolver|BE-Merge|Models, Storage.Mongo|DONE – `AdvisoryIdentityResolver` builds alias-driven clusters with canonical key selection + unit coverage.|
|Precedence policy engine|BE-Merge|Architecture|**DONE** – precedence defaults enforced by `AdvisoryPrecedenceMerger`/`AdvisoryPrecedenceDefaults` with distro/PSIRT overriding registry feeds and CERT/KEV enrichers.|
|NEVRA comparer plus tests|BE-Merge (Distro WG)|Source.Distro fixtures|DONE – Added Nevra parser/comparer with tilde-aware rpm ordering and unit coverage.|
|Debian EVR comparer plus tests|BE-Merge (Distro WG)|Debian fixtures|DONE – DebianEvr comparer mirrors dpkg ordering with tilde/epoch handling and unit coverage.|
|SemVer range resolver plus tests|BE-Merge (OSS WG)|OSV/GHSA fixtures|DONE – SemanticVersionRangeResolver covers introduced/fixed/lastAffected semantics with SemVer ordering tests.|
|Canonical hash and merge_event writer|BE-Merge|Models, Storage.Mongo|DONE – Hash calculator + MergeEventWriter compute canonical SHA-256 digests and persist merge events.|
|Conflict detection and metrics|BE-Merge|Core|**DONE** – merge meters emit override/conflict counters and structured audits (`AdvisoryPrecedenceMerger`).|
|FEEDMERGE-ENGINE-04-001 GHSA/NVD/OSV conflict rules|BE-Merge|Core, Storage.Mongo|DONE – `AdvisoryMergeService` applies `CanonicalMerger` output before precedence merge, replacing source advisories with the canonical transcript. **Coordination:** connector fixture owners should surface canonical deltas to Merge QA before regression sign-off.|
|FEEDMERGE-ENGINE-04-002 Override metrics instrumentation|BE-Merge|Observability|DONE – merge events persist `MergeFieldDecision` records enabling analytics on precedence/freshness decisions. **Next:** hand off metrics schema to Ops for dashboard wiring.|
|FEEDMERGE-ENGINE-04-003 Reference & credit union pipeline|BE-Merge|Models|DONE – canonical merge preserves union semantics while respecting precedence, validated via updated credit union tests.|
|End-to-end determinism test|QA|Merge, key connectors|**DONE** – `MergePrecedenceIntegrationTests.MergePipeline_IsDeterministicAcrossRuns` guards determinism.|
|FEEDMERGE-QA-04-001 End-to-end conflict regression suite|QA|Merge|DONE – `AdvisoryMergeServiceTests.MergeAsync_AppliesCanonicalRulesAndPersistsDecisions` exercises GHSA/NVD/OSV conflict path and merge-event analytics. **Reminder:** QA to sync with connector teams once new fixture triples land.|
|Override audit logging|BE-Merge|Observability|DONE – override audits now emit structured logs plus bounded-tag metrics suitable for prod telemetry.|
|Configurable precedence table|BE-Merge|Architecture|DONE – precedence options bind via concelier:merge:precedence:ranks with docs/tests covering operator workflow.|
|Merge pipeline parity for new advisory fields|BE-Merge|Models, Core|DONE (2025-10-15) – merge service now surfaces description/CWE/canonical metric decisions with updated metrics/tests.|
|Connector coordination for new advisory fields|Connector Leads, BE-Merge|Models, Core|**DONE (2025-10-15)** – GHSA, NVD, and OSV connectors now emit advisory descriptions, CWE weaknesses, and canonical metric ids. Fixtures refreshed (GHSA connector regression suite, `conflict-nvd.canonical.json`, OSV parity snapshots) and completion recorded in coordination log.|
|FEEDMERGE-ENGINE-07-001 Conflict sets & explainers|BE-Merge|FEEDSTORAGE-DATA-07-001|**DONE (2025-10-20)** – Merge surfaces conflict explainers with replay hashes via `MergeConflictSummary`; API exposes structured payloads and integration tests cover deterministic `asOf` hashes.|
> Remark (2025-10-20): `AdvisoryMergeService` now returns conflict summaries with deterministic hashes; WebService replay endpoint emits typed explainers verified by new tests.

42
src/Concelier/__Libraries/StellaOps.Concelier.Merge/TASKS.md
View File

@@ -1,33 +1,15 @@
# TASKS
| Task | Owner(s) | Depends on | Notes |
|---|---|---|---|
|Identity graph and alias resolver|BE-Merge|Models, Storage.Mongo|DONE – `AdvisoryIdentityResolver` builds alias-driven clusters with canonical key selection + unit coverage.|
|Precedence policy engine|BE-Merge|Architecture|**DONE** – precedence defaults enforced by `AdvisoryPrecedenceMerger`/`AdvisoryPrecedenceDefaults` with distro/PSIRT overriding registry feeds and CERT/KEV enrichers.|
|NEVRA comparer plus tests|BE-Merge (Distro WG)|Source.Distro fixtures|DONE – Added Nevra parser/comparer with tilde-aware rpm ordering and unit coverage.|
|Debian EVR comparer plus tests|BE-Merge (Distro WG)|Debian fixtures|DONE – DebianEvr comparer mirrors dpkg ordering with tilde/epoch handling and unit coverage.|
|SemVer range resolver plus tests|BE-Merge (OSS WG)|OSV/GHSA fixtures|DONE – SemanticVersionRangeResolver covers introduced/fixed/lastAffected semantics with SemVer ordering tests.|
|Canonical hash and merge_event writer|BE-Merge|Models, Storage.Mongo|DONE – Hash calculator + MergeEventWriter compute canonical SHA-256 digests and persist merge events.|
|Conflict detection and metrics|BE-Merge|Core|**DONE** – merge meters emit override/conflict counters and structured audits (`AdvisoryPrecedenceMerger`).|
|FEEDMERGE-ENGINE-04-001 GHSA/NVD/OSV conflict rules|BE-Merge|Core, Storage.Mongo|DONE – `AdvisoryMergeService` applies `CanonicalMerger` output before precedence merge, replacing source advisories with the canonical transcript. **Coordination:** connector fixture owners should surface canonical deltas to Merge QA before regression sign-off.|
|FEEDMERGE-ENGINE-04-002 Override metrics instrumentation|BE-Merge|Observability|DONE – merge events persist `MergeFieldDecision` records enabling analytics on precedence/freshness decisions. **Next:** hand off metrics schema to Ops for dashboard wiring.|
|FEEDMERGE-ENGINE-04-003 Reference & credit union pipeline|BE-Merge|Models|DONE – canonical merge preserves union semantics while respecting precedence, validated via updated credit union tests.|
|End-to-end determinism test|QA|Merge, key connectors|**DONE** – `MergePrecedenceIntegrationTests.MergePipeline_IsDeterministicAcrossRuns` guards determinism.|
|FEEDMERGE-QA-04-001 End-to-end conflict regression suite|QA|Merge|DONE – `AdvisoryMergeServiceTests.MergeAsync_AppliesCanonicalRulesAndPersistsDecisions` exercises GHSA/NVD/OSV conflict path and merge-event analytics. **Reminder:** QA to sync with connector teams once new fixture triples land.|
|Override audit logging|BE-Merge|Observability|DONE – override audits now emit structured logs plus bounded-tag metrics suitable for prod telemetry.|
|Configurable precedence table|BE-Merge|Architecture|DONE – precedence options bind via concelier:merge:precedence:ranks with docs/tests covering operator workflow.|
# TASKS
| Task | Owner(s) | Depends on | Notes |
|---|---|---|---|
|Range primitives backlog|BE-Merge|Connector WGs|**DOING** – Coordinate remaining connectors (`Acsc`, `Cccs`, `CertBund`, `CertCc`, `Cve`, `Ghsa`, `Ics.Cisa`, `Kisa`, `Ru.Bdu`, `Ru.Nkcki`, `Vndr.Apple`, `Vndr.Cisco`, `Vndr.Msrc`) to emit canonical RangePrimitives with provenance tags; track progress/fixtures here.<br>2025-10-11: Storage alignment notes + sample normalized rule JSON now captured in `RANGE_PRIMITIVES_COORDINATION.md` (see “Storage alignment quick reference”).<br>2025-10-11 18:45Z: GHSA normalized rules landed; OSV connector picked up next for rollout.<br>2025-10-11 21:10Z: `docs/dev/merge_semver_playbook.md` Section 8 now documents the persisted Mongo projection (SemVer + NEVRA) for connector reviewers.<br>2025-10-11 21:30Z: Added `docs/dev/normalized_versions_rollout.md` dashboard to centralize connector status and upcoming milestones.<br>2025-10-11 21:55Z: Merge now emits `concelier.merge.normalized_rules*` counters and unions connector-provided normalized arrays; see new test coverage in `AdvisoryPrecedenceMergerTests.Merge_RecordsNormalizedRuleMetrics`.<br>2025-10-12 17:05Z: CVE + KEV normalized rule verification complete; OSV parity fixtures revalidated—downstream parity/monitoring tasks may proceed.<br>2025-10-19 14:35Z: Prerequisites reviewed (none outstanding); FEEDMERGE-COORD-02-900 remains in DOING with connector follow-ups unchanged.<br>2025-10-19 15:25Z: Refreshed `RANGE_PRIMITIVES_COORDINATION.md` matrix + added targeted follow-ups (Cccs, CertBund, ICS-CISA, Kisa, Vndr.Cisco) with delivery dates 2025-10-21 → 2025-10-25; monitoring merge counters for regression.<br>2025-10-29: Added merge-time warnings highlighting sources/package types when ranges emit without normalized rules to accelerate backlog triage.|
|Range primitives backlog|BE-Merge|Connector WGs|**DOING** – Coordinate remaining connectors (`Acsc`, `Cccs`, `CertBund`, `CertCc`, `Cve`, `Ghsa`, `Ics.Cisa`, `Kisa`, `Ru.Bdu`, `Ru.Nkcki`, `Vndr.Apple`, `Vndr.Cisco`, `Vndr.Msrc`) to emit canonical RangePrimitives with provenance tags; track progress/fixtures here.<br>2025-10-11: Storage alignment notes + sample normalized rule JSON now captured in `RANGE_PRIMITIVES_COORDINATION.md` (see “Storage alignment quick reference”).<br>2025-10-11 18:45Z: GHSA normalized rules landed; OSV connector picked up next for rollout.<br>2025-10-11 21:10Z: `docs/dev/merge_semver_playbook.md` Section 8 now documents the persisted Mongo projection (SemVer + NEVRA) for connector reviewers.<br>2025-10-11 21:30Z: Added `docs/dev/normalized_versions_rollout.md` dashboard to centralize connector status and upcoming milestones.<br>2025-10-11 21:55Z: Merge now emits `concelier.merge.normalized_rules*` counters and unions connector-provided normalized arrays; see new test coverage in `AdvisoryPrecedenceMergerTests.Merge_RecordsNormalizedRuleMetrics`.<br>2025-10-12 17:05Z: CVE + KEV normalized rule verification complete; OSV parity fixtures revalidated—downstream parity/monitoring tasks may proceed.<br>2025-10-19 14:35Z: Prerequisites reviewed (none outstanding); FEEDMERGE-COORD-02-900 remains in DOING with connector follow-ups unchanged.<br>2025-10-19 15:25Z: Refreshed `RANGE_PRIMITIVES_COORDINATION.md` matrix + added targeted follow-ups (Cccs, CertBund, ICS-CISA, Kisa, Vndr.Cisco) with delivery dates 2025-10-21 → 2025-10-25; monitoring merge counters for regression.<br>2025-10-20 19:30Z: Coordination matrix + rollout dashboard updated with current connector statuses and due dates; flagged Slack escalation plan if Cccs/Cisco miss 2025-10-21 and documented Acsc kickoff window for 2025-10-24.|
|Merge pipeline parity for new advisory fields|BE-Merge|Models, Core|DONE (2025-10-15) – merge service now surfaces description/CWE/canonical metric decisions with updated metrics/tests.|
|Connector coordination for new advisory fields|Connector Leads, BE-Merge|Models, Core|**DONE (2025-10-15)** – GHSA, NVD, and OSV connectors now emit advisory descriptions, CWE weaknesses, and canonical metric ids. Fixtures refreshed (GHSA connector regression suite, `conflict-nvd.canonical.json`, OSV parity snapshots) and completion recorded in coordination log.|
|FEEDMERGE-ENGINE-07-001 Conflict sets & explainers|BE-Merge|FEEDSTORAGE-DATA-07-001|**DONE (2025-10-20)** – Merge surfaces conflict explainers with replay hashes via `MergeConflictSummary`; API exposes structured payloads and integration tests cover deterministic `asOf` hashes.|
> Remark (2025-10-20): `AdvisoryMergeService` now returns conflict summaries with deterministic hashes; WebService replay endpoint emits typed explainers verified by new tests.
|Range primitives backlog|BE-Merge|Connector WGs|**DOING** – Coordinate remaining connectors (`Acsc`, `Cccs`, `CertBund`, `CertCc`, `Cve`, `Ghsa`, `Ics.Cisa`, `Kisa`, `Ru.Bdu`, `Ru.Nkcki`, `Vndr.Apple`, `Vndr.Cisco`, `Vndr.Msrc`) to emit canonical RangePrimitives with provenance tags; track progress/fixtures here.<br>2025-10-11: Storage alignment notes + sample normalized rule JSON now captured in `RANGE_PRIMITIVES_COORDINATION.md` (see “Storage alignment quick reference”).<br>2025-10-11 18:45Z: GHSA normalized rules landed; OSV connector picked up next for rollout.<br>2025-10-11 21:10Z: `docs/dev/merge_semver_playbook.md` Section 8 now documents the persisted Mongo projection (SemVer + NEVRA) for connector reviewers.<br>2025-10-11 21:30Z: Added `docs/dev/normalized_versions_rollout.md` dashboard to centralize connector status and upcoming milestones.<br>2025-10-11 21:55Z: Merge now emits `concelier.merge.normalized_rules*` counters and unions connector-provided normalized arrays; see new test coverage in `AdvisoryPrecedenceMergerTests.Merge_RecordsNormalizedRuleMetrics`.<br>2025-10-12 17:05Z: CVE + KEV normalized rule verification complete; OSV parity fixtures revalidated—downstream parity/monitoring tasks may proceed.<br>2025-10-19 14:35Z: Prerequisites reviewed (none outstanding); FEEDMERGE-COORD-02-900 remains in DOING with connector follow-ups unchanged.<br>2025-10-19 15:25Z: Refreshed `RANGE_PRIMITIVES_COORDINATION.md` matrix + added targeted follow-ups (Cccs, CertBund, ICS-CISA, Kisa, Vndr.Cisco) with delivery dates 2025-10-21 → 2025-10-25; monitoring merge counters for regression.<br>2025-10-20 19:30Z: Coordination matrix + rollout dashboard updated with current connector statuses and due dates; flagged Slack escalation plan if Cccs/Cisco miss 2025-10-21 and documented Acsc kickoff window for 2025-10-24.|
|FEEDMERGE-COORD-02-901 Connector deadline check-ins|BE-Merge|FEEDMERGE-COORD-02-900|**TODO (due 2025-10-21)** – Confirm Cccs/Cisco normalized-rule branches land, capture `concelier.merge.normalized_rules*` counter screenshots, and update coordination docs with the results.<br>2025-10-29: Merge now emits `Normalized version rules missing...` warnings (see `docs/dev/normalized-rule-recipes.md` §4); include zero-warning excerpt plus Grafana counter snapshot when closing this task.|
|FEEDMERGE-COORD-02-902 ICS-CISA normalized-rule decision support|BE-Merge, Models|FEEDMERGE-COORD-02-900|**TODO (due 2025-10-23)** – Review ICS-CISA sample advisories, confirm SemVer reuse vs new firmware scheme, pre-stage Models ticket template, and document outcome in coordination docs + tracker files.<br>2025-10-29: Recipes doc (§2–§3) outlines SemVer promotion + fallback logging—attach decision summary + log sample when handing off to Models.|
|FEEDMERGE-COORD-02-903 KISA firmware scheme review|BE-Merge, Models|FEEDMERGE-COORD-02-900|**TODO (due 2025-10-24)** – Pair with KISA team on proposed firmware scheme (`kisa.build` or variant), ensure builder alignment, open Models ticket if required, and log decision in coordination docs + tracker files.|
## Link-Not-Merge v1 Transition
| Task | Owner(s) | Depends on | Notes |
|---|---|---|---|
|MERGE-LNM-21-001 Migration plan authoring|BE-Merge, Architecture Guild|CONCELIER-LNM-21-101|Draft `no-merge` migration playbook, documenting backfill strategy, feature flag rollout, and rollback steps for legacy merge pipeline deprecation.|
|MERGE-LNM-21-002 Merge service deprecation|BE-Merge|MERGE-LNM-21-001|Refactor or retire `AdvisoryMergeService` and related pipelines, ensuring callers transition to observation/linkset APIs; add compile-time analyzer preventing merge service usage.|
|MERGE-LNM-21-003 Determinism/test updates|QA Guild, BE-Merge|MERGE-LNM-21-002|Replace merge determinism suites with observation/linkset regression tests verifying no data mutation and conflicts remain visible.|
|FEEDMERGE-COORD-02-903 KISA firmware scheme review|BE-Merge, Models|FEEDMERGE-COORD-02-900|**TODO (due 2025-10-24)** – Pair with KISA team on proposed firmware scheme (`kisa.build` or variant), ensure builder alignment, open Models ticket if required, and log decision in coordination docs + tracker files.|
## Link-Not-Merge v1 Transition
| Task | Owner(s) | Depends on | Notes |
|---|---|---|---|
|MERGE-LNM-21-001 Migration plan authoring|BE-Merge, Architecture Guild|CONCELIER-LNM-21-101|Draft `no-merge` migration playbook, documenting backfill strategy, feature flag rollout, and rollback steps for legacy merge pipeline deprecation.|
|MERGE-LNM-21-002 Merge service deprecation|BE-Merge|MERGE-LNM-21-001|Refactor or retire `AdvisoryMergeService` and related pipelines, ensuring callers transition to observation/linkset APIs; add compile-time analyzer preventing merge service usage.|
|MERGE-LNM-21-003 Determinism/test updates|QA Guild, BE-Merge|MERGE-LNM-21-002|Replace merge determinism suites with observation/linkset regression tests verifying no data mutation and conflicts remain visible.|

34
src/Concelier/__Libraries/StellaOps.Concelier.Models/TASKS.completed.md Normal file
View File

@@ -0,0 +1,34 @@
# Completed Tasks
|Canonical JSON serializer with stable ordering|BE-Merge|Models|DONE – `CanonicalJsonSerializer` ensures deterministic property ordering.|
|Equality/comparison helpers for ranges|BE-Merge|Models|DONE – added `AffectedVersionRangeComparer` & equality comparer.|
|Type enums/constants for AffectedPackage.Type|BE-Merge|Models|DONE – introduced `AffectedPackageTypes`.|
|Validation helpers (lightweight)|BE-Merge|Models|DONE – added `Validation` static helpers and URL guard.|
|Snapshot serializer for tests|QA|Models|DONE – `SnapshotSerializer` emits canonical JSON.|
|Docs: field provenance guidelines|BE-Merge|Models|DONE – see `PROVENANCE_GUIDELINES.md`.|
|Canonical record definitions kept in sync|BE-Merge|Models|DONE – documented in `CANONICAL_RECORDS.md`; update alongside model changes.|
|Alias scheme registry and validation helpers|BE-Merge|Models|DONE – see `AliasSchemes` & `AliasSchemeRegistry` plus validation integration/tests.|
|Range primitives for SemVer/EVR/NEVRA metadata|BE-Merge|Models|DONE – SemVer/Evr/Nevra primitives now project canonical normalized rules; range helpers emit fallback rules for legacy inputs and tests cover canonical string generation so connectors can populate `normalizedVersions` deterministically.|
|Provenance envelope field masks|BE-Merge|Models|DONE – `AdvisoryProvenance.fieldMask` added with diagnostics/tests/docs refreshed; connectors can now emit canonical masks for QA dashboards.|
|Backward-compatibility playbook|BE-Merge, QA|Models|DONE – see `BACKWARD_COMPATIBILITY.md` for evolution policy/test checklist.|
|Golden canonical examples|QA|Models|DONE – added `/p:UpdateGoldens=true` test hook wiring `UPDATE_GOLDENS=1` so canonical fixtures regenerate via `dotnet test`; docs/tests unchanged.|
|Serialization determinism regression tests|QA|Models|DONE – locale-stability tests hash canonical serializer output across multiple cultures and runs.|
|Severity normalization helpers|BE-Merge|Models|DONE – helper now normalizes compound vendor labels/priority tiers with expanded synonym coverage and regression tests.|
|AffectedPackage status glossary & guardrails|BE-Merge|Models|DONE – catalog now exposes deterministic listing, TryNormalize helpers, and synonym coverage for vendor phrases (not vulnerable, workaround available, etc.).|
|Advisory schema parity (description, CWE collection, canonical metric id)|BE-Merge, BE-Core|Core, Exporters|DONE (2025-10-15) – extended `Advisory`/related records with description/CWEs/canonical metric id plus serializer/tests updated; exporters validated via new coverage.|

22
src/Concelier/__Libraries/StellaOps.Concelier.Models/TASKS.md
View File

@@ -1,19 +1,3 @@
# TASKS
| Task | Owner(s) | Depends on | Notes |
|---|---|---|---|
|Canonical JSON serializer with stable ordering|BE-Merge|Models|DONE – `CanonicalJsonSerializer` ensures deterministic property ordering.|
|Equality/comparison helpers for ranges|BE-Merge|Models|DONE – added `AffectedVersionRangeComparer` & equality comparer.|
|Type enums/constants for AffectedPackage.Type|BE-Merge|Models|DONE – introduced `AffectedPackageTypes`.|
|Validation helpers (lightweight)|BE-Merge|Models|DONE – added `Validation` static helpers and URL guard.|
|Snapshot serializer for tests|QA|Models|DONE – `SnapshotSerializer` emits canonical JSON.|
|Docs: field provenance guidelines|BE-Merge|Models|DONE – see `PROVENANCE_GUIDELINES.md`.|
|Canonical record definitions kept in sync|BE-Merge|Models|DONE – documented in `CANONICAL_RECORDS.md`; update alongside model changes.|
|Alias scheme registry and validation helpers|BE-Merge|Models|DONE – see `AliasSchemes` & `AliasSchemeRegistry` plus validation integration/tests.|
|Range primitives for SemVer/EVR/NEVRA metadata|BE-Merge|Models|DONE – SemVer/Evr/Nevra primitives now project canonical normalized rules; range helpers emit fallback rules for legacy inputs and tests cover canonical string generation so connectors can populate `normalizedVersions` deterministically.|
|Provenance envelope field masks|BE-Merge|Models|DONE – `AdvisoryProvenance.fieldMask` added with diagnostics/tests/docs refreshed; connectors can now emit canonical masks for QA dashboards.|
|Backward-compatibility playbook|BE-Merge, QA|Models|DONE – see `BACKWARD_COMPATIBILITY.md` for evolution policy/test checklist.|
|Golden canonical examples|QA|Models|DONE – added `/p:UpdateGoldens=true` test hook wiring `UPDATE_GOLDENS=1` so canonical fixtures regenerate via `dotnet test`; docs/tests unchanged.|
|Serialization determinism regression tests|QA|Models|DONE – locale-stability tests hash canonical serializer output across multiple cultures and runs.|
|Severity normalization helpers|BE-Merge|Models|DONE – helper now normalizes compound vendor labels/priority tiers with expanded synonym coverage and regression tests.|
|AffectedPackage status glossary & guardrails|BE-Merge|Models|DONE – catalog now exposes deterministic listing, TryNormalize helpers, and synonym coverage for vendor phrases (not vulnerable, workaround available, etc.).|
|Advisory schema parity (description, CWE collection, canonical metric id)|BE-Merge, BE-Core|Core, Exporters|DONE (2025-10-15) – extended `Advisory`/related records with description/CWEs/canonical metric id plus serializer/tests updated; exporters validated via new coverage.|
# TASKS
| Task | Owner(s) | Depends on | Notes |
|---|---|---|---|

16
src/Concelier/__Libraries/StellaOps.Concelier.Normalization/TASKS.completed.md Normal file
View File

@@ -0,0 +1,16 @@
# Completed Tasks
|Canonical NEVRA/EVR parsing helpers|BE-Norm (Distro WG)|Models|DONE – `Normalization.Distro` exposes parsers + canonical formatters consumed by Merge comparers/tests.|
|PURL/CPE identifier normalization|BE-Norm (OSS WG)|Models|DONE – canonical PURL/CPE helpers feed connectors and exporter tooling.|
|CPE normalization escape handling|BE-Norm (OSS WG)|Normalization identifiers|DONE – percent-decoding, edition sub-field expansion, and deterministic escaping landed in `Cpe23` with new tests covering boundary cases.|
|CVSS metric normalization & severity bands|BE-Norm (Risk WG)|Models|DONE – `CvssMetricNormalizer` unifies vectors, recomputes scores/severities, and is wired through NVD/RedHat/JVN mappers with unit coverage.|
|Description and locale normalization pipeline|BE-Norm (I18N)|Source connectors|DONE – `DescriptionNormalizer` strips markup, collapses whitespace, and provides locale fallback used by core mappers.|
|SemVer normalized rule emitter (FEEDNORM-NORM-02-001)|BE-Norm (SemVer WG)|Models, `FASTER_MODELING_AND_NORMALIZATION.md`|**DONE (2025-10-12)** – `SemVerRangeRuleBuilder` now parses comparator chains without comma delimiters, supports multi-segment `||` ranges, pushes exact-value metadata, and new tests document the contract for connector teams.|
|SemVer normalized rule convenience API|BE-Norm (SemVer WG)|SemVer normalized rule emitter|**DONE (2025-10-15)** – added `SemVerRangeRuleBuilder.BuildNormalizedRules` projection helper and unit coverage for empty/standard ranges so callers can access normalized rules without materializing primitives.|

13
src/Concelier/__Libraries/StellaOps.Concelier.Normalization/TASKS.md
View File

@@ -1,10 +1,3 @@
# TASKS
| Task | Owner(s) | Depends on | Notes |
|---|---|---|---|
|Canonical NEVRA/EVR parsing helpers|BE-Norm (Distro WG)|Models|DONE – `Normalization.Distro` exposes parsers + canonical formatters consumed by Merge comparers/tests.|
|PURL/CPE identifier normalization|BE-Norm (OSS WG)|Models|DONE – canonical PURL/CPE helpers feed connectors and exporter tooling.|
|CPE normalization escape handling|BE-Norm (OSS WG)|Normalization identifiers|DONE – percent-decoding, edition sub-field expansion, and deterministic escaping landed in `Cpe23` with new tests covering boundary cases.|
|CVSS metric normalization & severity bands|BE-Norm (Risk WG)|Models|DONE – `CvssMetricNormalizer` unifies vectors, recomputes scores/severities, and is wired through NVD/RedHat/JVN mappers with unit coverage.|
|Description and locale normalization pipeline|BE-Norm (I18N)|Source connectors|DONE – `DescriptionNormalizer` strips markup, collapses whitespace, and provides locale fallback used by core mappers.|
|SemVer normalized rule emitter (FEEDNORM-NORM-02-001)|BE-Norm (SemVer WG)|Models, `FASTER_MODELING_AND_NORMALIZATION.md`|**DONE (2025-10-12)** – `SemVerRangeRuleBuilder` now parses comparator chains without comma delimiters, supports multi-segment `||` ranges, pushes exact-value metadata, and new tests document the contract for connector teams.|
|SemVer normalized rule convenience API|BE-Norm (SemVer WG)|SemVer normalized rule emitter|**DONE (2025-10-15)** – added `SemVerRangeRuleBuilder.BuildNormalizedRules` projection helper and unit coverage for empty/standard ranges so callers can access normalized rules without materializing primitives.|
# TASKS
| Task | Owner(s) | Depends on | Notes |
|---|---|---|---|

8
src/Concelier/__Libraries/StellaOps.Concelier.Storage.Mongo/TASKS.completed.md Normal file
View File

@@ -0,0 +1,8 @@
# Completed Tasks
| ID | Status | Owner(s) | Depends on | Notes |
|---|---|---|---|---|
| CONCELIER-STORE-AOC-19-001 `advisory_raw schema validator` | DONE (2025-10-28) | Concelier Storage Guild | Mongo cluster ops sign-off | Author MongoDB JSON schema enforcing required fields (`source`, `upstream`, `content`, `linkset`, `tenant`) and forbidding normalized/severity fields. Include migration toggles for staged rollout. |
| CONCELIER-STORE-AOC-19-002 `idempotency unique index` | DONE (2025-10-28) | Concelier Storage Guild | CONCELIER-STORE-AOC-19-001 | Create compound unique index on `(source.vendor, upstream.upstream_id, upstream.content_hash, tenant)` with backfill script verifying existing data, and document offline validator bootstrap. |
| CONCELIER-STORE-AOC-19-003 `append-only supersedes migration` | DONE (2025-10-28) | Concelier Storage Guild | CONCELIER-STORE-AOC-19-002 | Introduce migration that freezes legacy `advisories` writes, copies data into `_backup_*`, and backfills supersedes pointers for raw revisions. Provide rollback plan. |
| CONCELIER-STORE-AOC-19-004 `validator deployment playbook` | DONE (2025-10-28) | Concelier Storage Guild, DevOps Guild | CONCELIER-STORE-AOC-19-001 | Update `MIGRATIONS.md` and Offline Kit docs to cover enabling validators, rolling restarts, and validator smoke tests for air-gapped installs. |

56
src/Concelier/__Libraries/StellaOps.Concelier.Storage.Mongo/TASKS.md
View File

@@ -1,30 +1,26 @@
# TASKS — Epic 1: Aggregation-Only Contract
> **AOC Reminder:** storage enforces append-only raw documents; no precedence/severity/normalization in ingestion collections.
| ID | Status | Owner(s) | Depends on | Notes |
|---|---|---|---|---|
| CONCELIER-STORE-AOC-19-001 `advisory_raw schema validator` | DONE (2025-10-28) | Concelier Storage Guild | Mongo cluster ops sign-off | Author MongoDB JSON schema enforcing required fields (`source`, `upstream`, `content`, `linkset`, `tenant`) and forbidding normalized/severity fields. Include migration toggles for staged rollout. |
> 2025-10-28: Added configurable validator migration (`20251028_advisory_raw_validator`), bootstrapper collection registration, storage options toggle, and Mongo migration tests covering schema + enforcement levels.
> Docs alignment (2025-10-26): Validator expectations + deployment steps documented in `docs/deploy/containers.md` §1.
| CONCELIER-STORE-AOC-19-002 `idempotency unique index` | DONE (2025-10-28) | Concelier Storage Guild | CONCELIER-STORE-AOC-19-001 | Create compound unique index on `(source.vendor, upstream.upstream_id, upstream.content_hash, tenant)` with backfill script verifying existing data, and document offline validator bootstrap. |
> 2025-10-28: Added `20251028_advisory_raw_idempotency_index` migration that detects duplicate raw advisories before creating the unique compound index, wired into DI, and extended migration tests to cover index shape + duplicate handling with supporting package updates.
> Docs alignment (2025-10-26): Idempotency contract + supersedes metrics in `docs/ingestion/aggregation-only-contract.md` §7 and observability guide.
| CONCELIER-STORE-AOC-19-003 `append-only supersedes migration` | DONE (2025-10-28) | Concelier Storage Guild | CONCELIER-STORE-AOC-19-002 | Introduce migration that freezes legacy `advisories` writes, copies data into `_backup_*`, and backfills supersedes pointers for raw revisions. Provide rollback plan. |
> 2025-10-28: Added supersedes backfill migration (`20251028_advisory_supersedes_backfill`) that renames `advisory` to a read-only view, snapshots data into `_backup_20251028`, and walks raw revisions to populate deterministic supersedes chains with integration coverage and operator scripts.
> Docs alignment (2025-10-26): Rollback guidance added to `docs/deploy/containers.md` §6.
| CONCELIER-STORE-AOC-19-004 `validator deployment playbook` | DONE (2025-10-28) | Concelier Storage Guild, DevOps Guild | CONCELIER-STORE-AOC-19-001 | Update `MIGRATIONS.md` and Offline Kit docs to cover enabling validators, rolling restarts, and validator smoke tests for air-gapped installs. |
> 2025-10-28: Documented duplicate audit + migration workflow in `docs/deploy/containers.md`, Offline Kit guide, and `MIGRATIONS.md`; published `ops/devops/scripts/check-advisory-raw-duplicates.js` for staging/offline clusters.
> Docs alignment (2025-10-26): Offline kit requirements documented in `docs/deploy/containers.md` §5.
## Policy Engine v2
| ID | Status | Owner(s) | Depends on | Notes |
|----|--------|----------|------------|-------|
| CONCELIER-POLICY-20-003 `Selection cursors` | TODO | Concelier Storage Guild | CONCELIER-STORE-AOC-19-002, POLICY-ENGINE-20-003 | Add advisory/vex selection cursors (per policy run) with change stream checkpoints, indexes, and offline migration scripts to support incremental evaluations. |
## Link-Not-Merge v1
| ID | Status | Owner(s) | Depends on | Notes |
|----|--------|----------|------------|-------|
| CONCELIER-LNM-21-101 `Observations collections` | TODO | Concelier Storage Guild | CONCELIER-LNM-21-001 | Provision `advisory_observations` and `advisory_linksets` collections with hashed shard keys, TTL for ingest metadata, and required indexes (`aliases`, `purls`, `observation_ids`). |
| CONCELIER-LNM-21-102 `Migration tooling` | TODO | Concelier Storage Guild, DevOps Guild | CONCELIER-LNM-21-101 | Backfill legacy merged advisories into observation/linkset collections, create tombstones for merged docs, and supply rollback scripts. |
| CONCELIER-LNM-21-103 `Blob/store wiring` | TODO | Concelier Storage Guild | CONCELIER-LNM-21-101 | Store large raw payloads in object storage with pointers from observations; update bootstrapper/offline kit to seed sample blobs. |
# TASKS — Epic 1: Aggregation-Only Contract
> **AOC Reminder:** storage enforces append-only raw documents; no precedence/severity/normalization in ingestion collections.
| ID | Status | Owner(s) | Depends on | Notes |
|---|---|---|---|---|
> 2025-10-28: Added configurable validator migration (`20251028_advisory_raw_validator`), bootstrapper collection registration, storage options toggle, and Mongo migration tests covering schema + enforcement levels.
> Docs alignment (2025-10-26): Validator expectations + deployment steps documented in `docs/deploy/containers.md` §1.
> 2025-10-28: Added `20251028_advisory_raw_idempotency_index` migration that detects duplicate raw advisories before creating the unique compound index, wired into DI, and extended migration tests to cover index shape + duplicate handling with supporting package updates.
> Docs alignment (2025-10-26): Idempotency contract + supersedes metrics in `docs/ingestion/aggregation-only-contract.md` §7 and observability guide.
> 2025-10-28: Added supersedes backfill migration (`20251028_advisory_supersedes_backfill`) that renames `advisory` to a read-only view, snapshots data into `_backup_20251028`, and walks raw revisions to populate deterministic supersedes chains with integration coverage and operator scripts.
> Docs alignment (2025-10-26): Rollback guidance added to `docs/deploy/containers.md` §6.
> 2025-10-28: Documented duplicate audit + migration workflow in `docs/deploy/containers.md`, Offline Kit guide, and `MIGRATIONS.md`; published `ops/devops/scripts/check-advisory-raw-duplicates.js` for staging/offline clusters.
> Docs alignment (2025-10-26): Offline kit requirements documented in `docs/deploy/containers.md` §5.
## Policy Engine v2
| ID | Status | Owner(s) | Depends on | Notes |
|----|--------|----------|------------|-------|
| CONCELIER-POLICY-20-003 `Selection cursors` | TODO | Concelier Storage Guild | CONCELIER-STORE-AOC-19-002, POLICY-ENGINE-20-003 | Add advisory/vex selection cursors (per policy run) with change stream checkpoints, indexes, and offline migration scripts to support incremental evaluations. |
## Link-Not-Merge v1
| ID | Status | Owner(s) | Depends on | Notes |
|----|--------|----------|------------|-------|
| CONCELIER-LNM-21-101 `Observations collections` | TODO | Concelier Storage Guild | CONCELIER-LNM-21-001 | Provision `advisory_observations` and `advisory_linksets` collections with hashed shard keys, TTL for ingest metadata, and required indexes (`aliases`, `purls`, `observation_ids`). |
| CONCELIER-LNM-21-102 `Migration tooling` | TODO | Concelier Storage Guild, DevOps Guild | CONCELIER-LNM-21-101 | Backfill legacy merged advisories into observation/linkset collections, create tombstones for merged docs, and supply rollback scripts. |
| CONCELIER-LNM-21-103 `Blob/store wiring` | TODO | Concelier Storage Guild | CONCELIER-LNM-21-101 | Store large raw payloads in object storage with pointers from observations; update bootstrapper/offline kit to seed sample blobs. |