- Created SignerEndpointsTests to validate the SignDsse and VerifyReferrers endpoints. - Implemented StubBearerAuthenticationDefaults and StubBearerAuthenticationHandler for token-based authentication. - Developed ConcelierExporterClient for managing Trivy DB settings and export operations. - Added TrivyDbSettingsPageComponent for UI interactions with Trivy DB settings, including form handling and export triggering. - Implemented styles and HTML structure for Trivy DB settings page. - Created NotifySmokeCheck tool for validating Redis event streams and Notify deliveries.
98 lines
9.2 KiB
Markdown
98 lines
9.2 KiB
Markdown
# Range Primitive Coordination (Sprint 2)
|
||
|
||
_Status date: 2025-10-20_
|
||
|
||
## Why this exists
|
||
- SemVer range outputs must follow the embedded rule guidance in `../FASTER_MODELING_AND_NORMALIZATION.md` (array of `{scheme,type,min/max/value,notes}`).
|
||
- Merge will rely on normalized rules plus existing `RangePrimitives` (SemVer/NEVRA/EVR) to dedupe ranges and compute deterministic hashes.
|
||
- Connector teams are mid-flight; this playbook restarts coordination so every feed delivers the normalized payload needed by the conflict resolver work in Sprint 3.
|
||
|
||
## Upstream dependencies
|
||
- **Models** (`FEEDMODELS-SCHEMA-01-003`, `FEEDMODELS-SCHEMA-02-900`) – extends `RangePrimitives.SemVer` metadata and introduces `NormalizedVersionRule` arrays on affected packages.
|
||
- **Normalization** (`FEEDNORM-NORM-02-001`) – provides `SemVerRangeRuleBuilder` used by OSS connectors (GHSA/OSV/NVD) to emit canonical rule docs plus provenance notes.
|
||
- **Storage.Mongo** (`FEEDSTORAGE-DATA-02-001`) – dual-write/dual-read modifications for the new arrays; required before staging rollout.
|
||
- **Merge** (`FEEDMERGE-ENGINE-02-002`) – unions/dedupes normalized rules across sources once connectors publish them.
|
||
|
||
Until these blocks land, connectors should stage changes behind a feature flag or fixture branch so we can flip on normalized writes in sync.
|
||
|
||
## Connector adoption matrix
|
||
| Connector | Owner team | Current state (2025-10-20) | Required actions for normalized rules | Coordination notes |
|
||
|-----------|------------|----------------------------|--------------------------------------|--------------------|
|
||
| Acsc | BE-Conn-ACSC | ❌ Not started – mapper emits legacy range strings only | Stage `SemVerRangeRuleBuilder` integration once relay HTTP/2 fixes stabilise; target kickoff 2025-10-24. | Pair with Merge on sample payloads; ensure fixtures capture vendor/device taxonomy for provenance notes. |
|
||
| Cccs | BE-Conn-CCCS | ⚠️ DOING – helper branch under review (due 2025-10-21) | Wire trailing-version split helper, emit `NormalizedVersions` with `cccs:{serial}:{index}` notes, refresh fixtures/tests. | Share MR link before 2025-10-21 stand-up; Merge to validate counters once fixtures land. |
|
||
| CertBund | BE-Conn-CERTBUND | ⚠️ In progress – localisation work pending (due 2025-10-22) | Translate `product.Versions` phrases (`bis`, `alle`) into builder inputs; emit provenance `certbund:{advisoryId}:{vendor}`; update README/tests. | Localization WG drafting deterministic casing guidance; expect sample payloads 2025-10-21. |
|
||
| CertCc | BE-Conn-CERTCC | ✅ Complete – emitting `certcc.vendor` rules since 2025-10-12 | Keep builder contract stable; bubble any VINCE payload changes. | Merge verified counters drop on 2025-10-19 run; no follow-up. |
|
||
| Cve | BE-Conn-CVE | ✅ Complete – SemVer rules emitted 2025-10-12 | Maintain provenance notes (`cve:{cveId}:{identifier}`) and extend fixtures as schema grows. | Latest nightly confirms normalized counters at expected baseline. |
|
||
| Ghsa | BE-Conn-GHSA | ✅ Complete – normalized rollout live 2025-10-11 | Monitor schema diffs; keep fixtures synced with GHSA provenance notes. | Coordinate with OSV on shared ecosystems; no open issues. |
|
||
| Osv | BE-Conn-OSV | ✅ Complete – normalized rules shipping 2025-10-11 | Track new ecosystems; ensure notes stay aligned with `osv:{ecosystem}:{advisoryId}:{identifier}`. | Merge analytics watching npm/PyPI parity; no action needed. |
|
||
| Nvd | BE-Conn-NVD | ✅ Complete – normalized SemVer output live 2025-10-11 | Maintain CVE-aligned provenance; monitor MR toggles if schema shifts. | Next check: confirm export parity once storage migration flips on 2025-10-23. |
|
||
| Kev | BE-Conn-KEV | ✅ Complete – catalog/due-date rules emitted 2025-10-12 | Keep schedule metadata synced with CISA feed. | Acts as flag-only enrich; no additional merge work required. |
|
||
| Ics.Cisa | BE-Conn-ICS-CISA | ⚠️ Pending decision (due 2025-10-23) | Promote existing SemVer primitives into normalized rules; open Models ticket if firmware requires new scheme. | Provide sample advisories to Merge by 2025-10-22 for schema review. |
|
||
| Kisa | BE-Conn-KISA | ⚠️ Proposal drafting (due 2025-10-24) | Finalise `kisa.build` (or alternate) scheme with Models, then emit normalized rules and update localisation notes/tests. | Localization WG prepping translation samples; Merge to review scheme request immediately. |
|
||
| Ru.Bdu | BE-Conn-BDU | ✅ Complete – emitting `ru-bdu.raw` rules since 2025-10-14 | Monitor UTF-8 sanitisation; keep provenance notes aligned with advisory ids. | Storage snapshot verified 2025-10-19; counters green. |
|
||
| Ru.Nkcki | BE-Conn-Nkcki | ✅ Complete – SemVer + normalized rules live 2025-10-13 | Maintain Cyrillic provenance fields and SemVer coverage. | Localization WG confirmed transliteration guidance; no open items. |
|
||
| Vndr.Apple | BE-Conn-Apple | ✅ Complete – `apple.build` SemVer rules live 2025-10-11 | Keep fixtures covering multi-range tables; notify Merge of schema evolutions. | Prepare follow-up for macOS/iOS beta channels by 2025-10-26. |
|
||
| Vndr.Cisco | BE-Conn-Cisco | ⚠️ DOING – normalized promotion branch open (due 2025-10-21) | Use helper to convert SemVer primitives into rule arrays with `cisco:{productId}` notes; refresh tests. | OAuth throttling validated; Merge to rerun counters post-merge. |
|
||
| Vndr.Msrc | BE-Conn-MSRC | ✅ Complete – `msrc.build` rules live 2025-10-15 | Monitor monthly rollup coverage and provenance notes. | Merge verified rule ingestion 2025-10-19; no outstanding actions. |
|
||
|
||
## Storage alignment quick reference (2025-10-11)
|
||
- `NormalizedVersionDocumentFactory` copies each `NormalizedVersionRule` into Mongo with the shape `{ packageId, packageType, scheme, type, style, min, minInclusive, max, maxInclusive, value, notes, decisionReason, constraint, source, recordedAt }`. `style` is currently a direct echo of `type` but reserved for future vendor comparers—no connector action required.
|
||
- `constraint` is hydrated only when `NormalizedVersionRule` matches a legacy `VersionRange` primitive. Preserve `notes` (e.g., `nvd:cve-2025-1234`) so storage can join rules back to their provenance and carry decision reasoning.
|
||
- Valid `scheme` values today are `semver`, `nevra`, and `evr`. Raise a Models ticket before introducing additional scheme identifiers (e.g., `apple.build`, `ios.semver`).
|
||
- Prefer normalized `type` tokens from `NormalizedVersionRuleTypes` (`range`, `exact`, `lt`, `lte`, `gt`, `gte`). Builders already coerce casing/format—avoid custom strings.
|
||
- Ensure `AffectedPackage.Identifier`/`Type` and `Provenance` collections are populated; storage falls back to package-level provenance if range-level data is absent, but loses traceability if both are empty.
|
||
- Snapshot of an emitted document (SemVer range) for reference:
|
||
```json
|
||
{
|
||
"packageId": "pkg:npm/example",
|
||
"packageType": "npm",
|
||
"scheme": "semver",
|
||
"type": "range",
|
||
"style": "range",
|
||
"min": "1.2.3",
|
||
"minInclusive": true,
|
||
"max": "2.0.0",
|
||
"maxInclusive": false,
|
||
"value": null,
|
||
"notes": "ghsa:GHSA-xxxx-yyyy",
|
||
"decisionReason": "ghsa-precedence-over-nvd",
|
||
"constraint": ">= 1.2.3 < 2.0.0",
|
||
"source": "ghsa",
|
||
"recordedAt": "2025-10-11T00:00:00Z"
|
||
}
|
||
```
|
||
- For distro sources emitting NEVRA/EVR primitives, expect the same envelope with `scheme` swapped accordingly. Example (`nevra`):
|
||
```json
|
||
{
|
||
"packageId": "bash",
|
||
"packageType": "rpm",
|
||
"scheme": "nevra",
|
||
"type": "range",
|
||
"style": "range",
|
||
"min": "0:4.4.18-2.el7",
|
||
"minInclusive": true,
|
||
"max": "0:4.4.20-1.el7",
|
||
"maxInclusive": false,
|
||
"value": null,
|
||
"notes": "redhat:RHSA-2025:1234",
|
||
"decisionReason": "rhel-priority-over-nvd",
|
||
"constraint": "<= 0:4.4.20-1.el7",
|
||
"source": "redhat",
|
||
"recordedAt": "2025-10-11T00:00:00Z"
|
||
}
|
||
```
|
||
|
||
## Immediate next steps
|
||
- **2025-10-21** – Cccs and Cisco teams to merge normalized-rule branches, regenerate fixtures, and post counter screenshots.
|
||
- **2025-10-22** – CertBund translator review with Localization WG; confirm localisation glossary + deterministic casing before merge.
|
||
- **2025-10-23** – ICS-CISA to confirm SemVer vs firmware scheme; escalate Models ticket if new scheme required.
|
||
- **2025-10-24** – KISA firmware scheme proposal due; Merge to review immediately and unblock builder integration.
|
||
- **2025-10-25** – Merge cross-connector review to validate counters, provenance notes, and storage projections before flipping default union logic.
|
||
|
||
## Tracking & follow-up
|
||
- Track due dates above; if a connector slips past its deadline, flag in `#concelier-merge` stand-up and open a blocker ticket referencing FEEDMERGE-COORD-02-900.
|
||
- Capture connector progress updates in stand-ups twice per week; link PRs/issues back to this document and the rollout dashboard (`docs/dev/normalized_versions_rollout.md`).
|
||
- Monitor merge counters `concelier.merge.normalized_rules` and `concelier.merge.normalized_rules_missing` to spot advisories that still lack normalized arrays after precedence merge.
|
||
- When a connector is ready to emit normalized rules, update its module `TASKS.md` status and ping Merge in `#concelier-merge` with fixture diff screenshots.
|
||
- If new schemes or comparer logic is required (e.g., Cisco IOS), open a Models issue referencing `FEEDMODELS-SCHEMA-02-900` before implementing.
|