Files
git.stella-ops.org/src/StellaOps.Concelier.Merge/RANGE_PRIMITIVES_COORDINATION.md

9.4 KiB
Raw Blame History

Range Primitive Coordination (Sprint 2)

Status date: 2025-10-11

Why this exists

  • SemVer range outputs must follow the embedded rule guidance in ../FASTER_MODELING_AND_NORMALIZATION.md (array of {scheme,type,min/max/value,notes}).
  • Merge will rely on normalized rules plus existing RangePrimitives (SemVer/NEVRA/EVR) to dedupe ranges and compute deterministic hashes.
  • Connector teams are mid-flight; this playbook restarts coordination so every feed delivers the normalized payload needed by the conflict resolver work in Sprint 3.

Upstream dependencies

  • Models (FEEDMODELS-SCHEMA-01-003, FEEDMODELS-SCHEMA-02-900) extends RangePrimitives.SemVer metadata and introduces NormalizedVersionRule arrays on affected packages.
  • Normalization (FEEDNORM-NORM-02-001) provides SemVerRangeRuleBuilder used by OSS connectors (GHSA/OSV/NVD) to emit canonical rule docs plus provenance notes.
  • Storage.Mongo (FEEDSTORAGE-DATA-02-001) dual-write/dual-read modifications for the new arrays; required before staging rollout.
  • Merge (FEEDMERGE-ENGINE-02-002) unions/dedupes normalized rules across sources once connectors publish them.

Until these blocks land, connectors should stage changes behind a feature flag or fixture branch so we can flip on normalized writes in sync.

Connector adoption matrix

Connector Owner team Current state (2025-10-11) Required actions for SemVer guidance Coordination notes
Acsc BE-Conn-ACSC All tasks still TODO Blocked on initial ingest work; align DTO design with normalized rule array before mapper lands. Schedule pairing once SemVerRangeRuleBuilder API is published; ensure fixtures capture vendor/device taxonomy for provenance notes.
Cccs BE-Conn-CCCS All tasks still TODO Same as Acsc; design DTOs with normalized rule hooks from day one. Provide sample rule snippets in kickoff; share Mongo dual-write plan once storage flag is ready.
CertBund BE-Conn-CERTBUND All tasks still TODO Ensure canonical mapper emits vendor range primitives plus normalized rules for product firmware. Needs language/localisation guidance; coordinate with Localization WG for deterministic casing.
CertCc BE-Conn-CERTCC Fetch in progress, mapping TODO Map VINCE vendor/product data into RangePrimitives with certcc.vendor extensions; build normalized SemVer ranges when version strings surface. Follow up on 2025-10-14 to review VINCE payload examples and confirm builder requirements.
Cve BE-Conn-CVE Mapping/tests DONE (legacy SemVer) Refactor CveMapper to call the shared builder and populate NormalizedVersions + provenance notes once models land. Prepare MR behind ENABLE_NORMALIZED_VERSIONS flag; regression fixtures already cover version ranges—extend snapshots to cover rule arrays.
Ghsa BE-Conn-GHSA Normalized rules emitted (2025-10-11) Maintain SemVer builder integration; share regression diffs if schema shifts occur. Fixtures refreshed with ghsa:{identifier} notes; OSV rollout next in queue—await connector handoff update.
Osv BE-Conn-OSV Normalized rules emitted (2025-10-11) Keep SemVer builder wiring current; extend notes if new ecosystems appear. npm/PyPI parity snapshots updated with osv:{ecosystem}:{advisoryId}:{identifier} notes; merge analytics notified.
Nvd BE-Conn-NVD Normalized rules emitted (2025-10-11) Maintain SemVer coverage for ecosystem ranges; keep notes aligned with CVE IDs. CPE ranges now emit semver primitives when versions parse; fixtures refreshed, report sent to FEEDMERGE-COORD-02-900.
Cve BE-Conn-CVE Normalized rules emitted (2025-10-11) Maintain SemVer notes for vendor ecosystems; backfill additional fixture coverage as CVE payloads expand. Connector outputs cve:{cveId}:{identifier} notes; npm parity test fixtures updated and merge ping acknowledged.
Ics.Cisa BE-Conn-ICS-CISA All tasks TODO When defining product schema, plan for SemVer or vendor version rules (many advisories use firmware revisions). Gather sample advisories and confirm whether ranges are SemVer or vendor-specific so we can introduce scheme identifiers early.
Kisa BE-Conn-KISA All tasks TODO Ensure DTO parsing captures version strings despite localisation; feed into normalized rule builder once ready. Requires translation samples; request help from Localization WG before mapper implementation.
Ru.Bdu BE-Conn-BDU All tasks TODO Map product releases into normalized rules; add provenance notes referencing BDU advisory identifiers. Verify we have UTF-8 safe handling in builder; share sample sanitized inputs.
Ru.Nkcki BE-Conn-Nkcki All tasks TODO Similar to BDU; capture vendor firmware/build numbers and map into normalized rules. Coordinate with Localization WG for Cyrillic transliteration strategy.
Vndr.Apple BE-Conn-Apple Mapper/tests/telemetry marked DOING Continue extending vendor range primitives (apple.version, apple.build) and adopt normalized rule arrays for OS build spans. Request builder integration review on 2025-10-16; ensure fixtures cover multi-range tables and include provenance notes.
Vndr.Cisco BE-Conn-Cisco Emits SemVer primitives with vendor notes Parser maps versions into SemVer primitives with cisco.productId vendor extensions; sample fixtures landing in StellaOps.Concelier.Connector.Vndr.Cisco.Tests. No custom comparer required; SemVer + vendor metadata suffices.
Vndr.Msrc BE-Conn-MSRC All tasks TODO Canonical mapper must output product/build coverage as normalized rules (likely msrc.patch scheme) with provenance referencing KB IDs. Sync with Models on adding scheme identifiers for MSRC packages; plan fixture coverage for monthly rollups.

Storage alignment quick reference (2025-10-11)

  • NormalizedVersionDocumentFactory copies each NormalizedVersionRule into Mongo with the shape { packageId, packageType, scheme, type, style, min, minInclusive, max, maxInclusive, value, notes, decisionReason, constraint, source, recordedAt }. style is currently a direct echo of type but reserved for future vendor comparers—no connector action required.
  • constraint is hydrated only when NormalizedVersionRule matches a legacy VersionRange primitive. Preserve notes (e.g., nvd:cve-2025-1234) so storage can join rules back to their provenance and carry decision reasoning.
  • Valid scheme values today are semver, nevra, and evr. Raise a Models ticket before introducing additional scheme identifiers (e.g., apple.build, ios.semver).
  • Prefer normalized type tokens from NormalizedVersionRuleTypes (range, exact, lt, lte, gt, gte). Builders already coerce casing/format—avoid custom strings.
  • Ensure AffectedPackage.Identifier/Type and Provenance collections are populated; storage falls back to package-level provenance if range-level data is absent, but loses traceability if both are empty.
  • Snapshot of an emitted document (SemVer range) for reference:
    {
      "packageId": "pkg:npm/example",
      "packageType": "npm",
      "scheme": "semver",
      "type": "range",
      "style": "range",
      "min": "1.2.3",
      "minInclusive": true,
      "max": "2.0.0",
      "maxInclusive": false,
      "value": null,
      "notes": "ghsa:GHSA-xxxx-yyyy",
      "decisionReason": "ghsa-precedence-over-nvd",
      "constraint": ">= 1.2.3 < 2.0.0",
      "source": "ghsa",
      "recordedAt": "2025-10-11T00:00:00Z"
    }
    
  • For distro sources emitting NEVRA/EVR primitives, expect the same envelope with scheme swapped accordingly. Example (nevra):
    {
      "packageId": "bash",
      "packageType": "rpm",
      "scheme": "nevra",
      "type": "range",
      "style": "range",
      "min": "0:4.4.18-2.el7",
      "minInclusive": true,
      "max": "0:4.4.20-1.el7",
      "maxInclusive": false,
      "value": null,
      "notes": "redhat:RHSA-2025:1234",
      "decisionReason": "rhel-priority-over-nvd",
      "constraint": "<= 0:4.4.20-1.el7",
      "source": "redhat",
      "recordedAt": "2025-10-11T00:00:00Z"
    }
    

Immediate next steps

  • Normalization team to share draft SemVerRangeRuleBuilder API by 2025-10-13 for review; Merge will circulate feedback within 24 hours.
  • Connector owners to prepare fixture pull requests demonstrating sample normalized rule arrays (even if feature-flagged) by 2025-10-17.
  • Merge team will run a cross-connector review on 2025-10-18 to confirm consistent field usage and provenance tagging before enabling merge union logic.
  • Schedule held for 2025-10-14 14:00 UTC to review the CERT/CC staging VINCE advisory sample once enableDetailMapping is flipped; capture findings in #concelier-merge with snapshot diffs.

Tracking & follow-up

  • Capture connector progress updates in stand-ups twice per week; link PRs/issues back to this document and the rollout dashboard (docs/dev/normalized_versions_rollout.md).
  • Monitor merge counters concelier.merge.normalized_rules and concelier.merge.normalized_rules_missing to spot advisories that still lack normalized arrays after precedence merge.
  • When a connector is ready to emit normalized rules, update its module TASKS.md status and ping Merge in #concelier-merge with fixture diff screenshots.
  • If new schemes or comparer logic is required (e.g., Cisco IOS), open a Models issue referencing FEEDMODELS-SCHEMA-02-900 before implementing.