Files
git.stella-ops.org/src/StellaOps.Concelier.Merge/RANGE_PRIMITIVES_COORDINATION.md
master 48f3071e2a Add tests and implement StubBearer authentication for Signer endpoints
- Created SignerEndpointsTests to validate the SignDsse and VerifyReferrers endpoints.
- Implemented StubBearerAuthenticationDefaults and StubBearerAuthenticationHandler for token-based authentication.
- Developed ConcelierExporterClient for managing Trivy DB settings and export operations.
- Added TrivyDbSettingsPageComponent for UI interactions with Trivy DB settings, including form handling and export triggering.
- Implemented styles and HTML structure for Trivy DB settings page.
- Created NotifySmokeCheck tool for validating Redis event streams and Notify deliveries.
2025-10-21 09:37:07 +03:00

98 lines
9.2 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# Range Primitive Coordination (Sprint 2)
_Status date: 2025-10-20_
## Why this exists
- SemVer range outputs must follow the embedded rule guidance in `../FASTER_MODELING_AND_NORMALIZATION.md` (array of `{scheme,type,min/max/value,notes}`).
- Merge will rely on normalized rules plus existing `RangePrimitives` (SemVer/NEVRA/EVR) to dedupe ranges and compute deterministic hashes.
- Connector teams are mid-flight; this playbook restarts coordination so every feed delivers the normalized payload needed by the conflict resolver work in Sprint 3.
## Upstream dependencies
- **Models** (`FEEDMODELS-SCHEMA-01-003`, `FEEDMODELS-SCHEMA-02-900`) extends `RangePrimitives.SemVer` metadata and introduces `NormalizedVersionRule` arrays on affected packages.
- **Normalization** (`FEEDNORM-NORM-02-001`) provides `SemVerRangeRuleBuilder` used by OSS connectors (GHSA/OSV/NVD) to emit canonical rule docs plus provenance notes.
- **Storage.Mongo** (`FEEDSTORAGE-DATA-02-001`) dual-write/dual-read modifications for the new arrays; required before staging rollout.
- **Merge** (`FEEDMERGE-ENGINE-02-002`) unions/dedupes normalized rules across sources once connectors publish them.
Until these blocks land, connectors should stage changes behind a feature flag or fixture branch so we can flip on normalized writes in sync.
## Connector adoption matrix
| Connector | Owner team | Current state (2025-10-20) | Required actions for normalized rules | Coordination notes |
|-----------|------------|----------------------------|--------------------------------------|--------------------|
| Acsc | BE-Conn-ACSC | ❌ Not started mapper emits legacy range strings only | Stage `SemVerRangeRuleBuilder` integration once relay HTTP/2 fixes stabilise; target kickoff 2025-10-24. | Pair with Merge on sample payloads; ensure fixtures capture vendor/device taxonomy for provenance notes. |
| Cccs | BE-Conn-CCCS | ⚠️ DOING helper branch under review (due 2025-10-21) | Wire trailing-version split helper, emit `NormalizedVersions` with `cccs:{serial}:{index}` notes, refresh fixtures/tests. | Share MR link before 2025-10-21 stand-up; Merge to validate counters once fixtures land. |
| CertBund | BE-Conn-CERTBUND | ⚠️ In progress localisation work pending (due 2025-10-22) | Translate `product.Versions` phrases (`bis`, `alle`) into builder inputs; emit provenance `certbund:{advisoryId}:{vendor}`; update README/tests. | Localization WG drafting deterministic casing guidance; expect sample payloads 2025-10-21. |
| CertCc | BE-Conn-CERTCC | ✅ Complete emitting `certcc.vendor` rules since 2025-10-12 | Keep builder contract stable; bubble any VINCE payload changes. | Merge verified counters drop on 2025-10-19 run; no follow-up. |
| Cve | BE-Conn-CVE | ✅ Complete SemVer rules emitted 2025-10-12 | Maintain provenance notes (`cve:{cveId}:{identifier}`) and extend fixtures as schema grows. | Latest nightly confirms normalized counters at expected baseline. |
| Ghsa | BE-Conn-GHSA | ✅ Complete normalized rollout live 2025-10-11 | Monitor schema diffs; keep fixtures synced with GHSA provenance notes. | Coordinate with OSV on shared ecosystems; no open issues. |
| Osv | BE-Conn-OSV | ✅ Complete normalized rules shipping 2025-10-11 | Track new ecosystems; ensure notes stay aligned with `osv:{ecosystem}:{advisoryId}:{identifier}`. | Merge analytics watching npm/PyPI parity; no action needed. |
| Nvd | BE-Conn-NVD | ✅ Complete normalized SemVer output live 2025-10-11 | Maintain CVE-aligned provenance; monitor MR toggles if schema shifts. | Next check: confirm export parity once storage migration flips on 2025-10-23. |
| Kev | BE-Conn-KEV | ✅ Complete catalog/due-date rules emitted 2025-10-12 | Keep schedule metadata synced with CISA feed. | Acts as flag-only enrich; no additional merge work required. |
| Ics.Cisa | BE-Conn-ICS-CISA | ⚠️ Pending decision (due 2025-10-23) | Promote existing SemVer primitives into normalized rules; open Models ticket if firmware requires new scheme. | Provide sample advisories to Merge by 2025-10-22 for schema review. |
| Kisa | BE-Conn-KISA | ⚠️ Proposal drafting (due 2025-10-24) | Finalise `kisa.build` (or alternate) scheme with Models, then emit normalized rules and update localisation notes/tests. | Localization WG prepping translation samples; Merge to review scheme request immediately. |
| Ru.Bdu | BE-Conn-BDU | ✅ Complete emitting `ru-bdu.raw` rules since 2025-10-14 | Monitor UTF-8 sanitisation; keep provenance notes aligned with advisory ids. | Storage snapshot verified 2025-10-19; counters green. |
| Ru.Nkcki | BE-Conn-Nkcki | ✅ Complete SemVer + normalized rules live 2025-10-13 | Maintain Cyrillic provenance fields and SemVer coverage. | Localization WG confirmed transliteration guidance; no open items. |
| Vndr.Apple | BE-Conn-Apple | ✅ Complete `apple.build` SemVer rules live 2025-10-11 | Keep fixtures covering multi-range tables; notify Merge of schema evolutions. | Prepare follow-up for macOS/iOS beta channels by 2025-10-26. |
| Vndr.Cisco | BE-Conn-Cisco | ⚠️ DOING normalized promotion branch open (due 2025-10-21) | Use helper to convert SemVer primitives into rule arrays with `cisco:{productId}` notes; refresh tests. | OAuth throttling validated; Merge to rerun counters post-merge. |
| Vndr.Msrc | BE-Conn-MSRC | ✅ Complete `msrc.build` rules live 2025-10-15 | Monitor monthly rollup coverage and provenance notes. | Merge verified rule ingestion 2025-10-19; no outstanding actions. |
## Storage alignment quick reference (2025-10-11)
- `NormalizedVersionDocumentFactory` copies each `NormalizedVersionRule` into Mongo with the shape `{ packageId, packageType, scheme, type, style, min, minInclusive, max, maxInclusive, value, notes, decisionReason, constraint, source, recordedAt }`. `style` is currently a direct echo of `type` but reserved for future vendor comparers—no connector action required.
- `constraint` is hydrated only when `NormalizedVersionRule` matches a legacy `VersionRange` primitive. Preserve `notes` (e.g., `nvd:cve-2025-1234`) so storage can join rules back to their provenance and carry decision reasoning.
- Valid `scheme` values today are `semver`, `nevra`, and `evr`. Raise a Models ticket before introducing additional scheme identifiers (e.g., `apple.build`, `ios.semver`).
- Prefer normalized `type` tokens from `NormalizedVersionRuleTypes` (`range`, `exact`, `lt`, `lte`, `gt`, `gte`). Builders already coerce casing/format—avoid custom strings.
- Ensure `AffectedPackage.Identifier`/`Type` and `Provenance` collections are populated; storage falls back to package-level provenance if range-level data is absent, but loses traceability if both are empty.
- Snapshot of an emitted document (SemVer range) for reference:
```json
{
"packageId": "pkg:npm/example",
"packageType": "npm",
"scheme": "semver",
"type": "range",
"style": "range",
"min": "1.2.3",
"minInclusive": true,
"max": "2.0.0",
"maxInclusive": false,
"value": null,
"notes": "ghsa:GHSA-xxxx-yyyy",
"decisionReason": "ghsa-precedence-over-nvd",
"constraint": ">= 1.2.3 < 2.0.0",
"source": "ghsa",
"recordedAt": "2025-10-11T00:00:00Z"
}
```
- For distro sources emitting NEVRA/EVR primitives, expect the same envelope with `scheme` swapped accordingly. Example (`nevra`):
```json
{
"packageId": "bash",
"packageType": "rpm",
"scheme": "nevra",
"type": "range",
"style": "range",
"min": "0:4.4.18-2.el7",
"minInclusive": true,
"max": "0:4.4.20-1.el7",
"maxInclusive": false,
"value": null,
"notes": "redhat:RHSA-2025:1234",
"decisionReason": "rhel-priority-over-nvd",
"constraint": "<= 0:4.4.20-1.el7",
"source": "redhat",
"recordedAt": "2025-10-11T00:00:00Z"
}
```
## Immediate next steps
- **2025-10-21** Cccs and Cisco teams to merge normalized-rule branches, regenerate fixtures, and post counter screenshots.
- **2025-10-22** CertBund translator review with Localization WG; confirm localisation glossary + deterministic casing before merge.
- **2025-10-23** ICS-CISA to confirm SemVer vs firmware scheme; escalate Models ticket if new scheme required.
- **2025-10-24** KISA firmware scheme proposal due; Merge to review immediately and unblock builder integration.
- **2025-10-25** Merge cross-connector review to validate counters, provenance notes, and storage projections before flipping default union logic.
## Tracking & follow-up
- Track due dates above; if a connector slips past its deadline, flag in `#concelier-merge` stand-up and open a blocker ticket referencing FEEDMERGE-COORD-02-900.
- Capture connector progress updates in stand-ups twice per week; link PRs/issues back to this document and the rollout dashboard (`docs/dev/normalized_versions_rollout.md`).
- Monitor merge counters `concelier.merge.normalized_rules` and `concelier.merge.normalized_rules_missing` to spot advisories that still lack normalized arrays after precedence merge.
- When a connector is ready to emit normalized rules, update its module `TASKS.md` status and ping Merge in `#concelier-merge` with fixture diff screenshots.
- If new schemes or comparer logic is required (e.g., Cisco IOS), open a Models issue referencing `FEEDMODELS-SCHEMA-02-900` before implementing.