9253620833
Some checks failed
Docs CI / lint-and-preview (push) Has been cancelled
Temp commit to debug
113 lines
7.2 KiB
Markdown
113 lines
7.2 KiB
Markdown
# Normalized Version Rule Recipes
|
||
|
||
_Status: 2025-10-29_
|
||
|
||
This guide captures the minimum wiring required for connectors and Merge coordination tasks to finish the normalized-version rollout that unblocks FEEDMERGE-COORD-02-9xx.
|
||
|
||
## 1. Quick-start checklist
|
||
|
||
1. Ensure your mapper already emits `AffectedPackage.VersionRanges` (SemVer, NEVRA, EVR). If you only have vendor/product strings, capture the raw range text before trimming so it can feed the helper.
|
||
2. Call `SemVerRangeRuleBuilder.BuildNormalizedRules(rawRange, patchedVersion, provenance)` for each range and place the result in `AffectedPackage.NormalizedVersions`.
|
||
3. Set a provenance note in the format `connector:{advisoryId}:{index}` so Merge can differentiate connector-provided rules from canonical fallbacks.
|
||
4. Verify with `dotnet test` that the connector snapshot fixtures now include the `normalizedVersions` array and update fixtures by setting the connector-specific `UPDATE_*_FIXTURES=1` environment variable.
|
||
5. Tail Merge logs (or the test output) for the new warning `Normalized version rules missing for {AdvisoryKey}`; an empty warning stream means the connector/merge artefacts are ready to close FEEDMERGE-COORD-02-901/902.
|
||
|
||
### 1.1 Rollout status (2025-11-04)
|
||
|
||
| Connector | Status | Notes / next steps |
|
||
|-----------|--------|--------------------|
|
||
| `vendor.acsc` | ⚠️ Pending | Upstream feed only supplies vendor/product strings. Waiting on ASD/ACSC feed update to expose explicit ranges. Track via FEEDCONN-ACSC-02-010. |
|
||
| `vendor.cccs` | ⚠️ Pending | Advisory payloads lack structured version ranges; all entries remain vendor identifiers. Coordinate with CCCS to obtain machine-readable version metadata or fall back to Model heuristics. |
|
||
| `certbund` | ⚠️ Pending | `product.Versions` contains natural-language German ranges. Parser spec drafted (`CERTBUND-NORM-01`); awaiting implementation before emitting normalized rules. |
|
||
| `vendor.cisco` | ⚠️ Pending | Current API exposes product IDs only. Engage Cisco PSIRT (FEEDCONN-CISCO-02-014) to surface affected version expressions. |
|
||
| `vendor.apple` | ✅ Done | SemVer-style helpers in `AppleMapper` emit normalized rules with provenance annotations. |
|
||
| `vendor.msrc` | ✅ Done | `MsrcMapper` maps KB build numbers to exact rules; normalized output guarded by fixtures. |
|
||
| `vendor.ghsa` | ✅ Done | SemVer + vendor fallback rules emitted (`CreateSemVerVersionArtifacts`). |
|
||
| `vendor.kisa` | ✅ Partial | Normalized SemVer rules for structured firmware ranges. Fallback vendor strings remain for prose-only advisories; continue capturing new patterns in fixtures. |
|
||
| `ics.cisa` | ✅ Done | Firmware helper emits normalized range matrix using `SemVerRangeRuleBuilder`. |
|
||
| `certcc` | ✅ Done | Vendor comparator transforms range fragments to normalized rules. |
|
||
| `cve.nvd` | ✅ Done | Full SemVer builder and provenance mapping rolled out (FEEDCONN-CVE-02-015). |
|
||
| `ru.bdu` | ⚠️ Pending | Feed only includes product codenames. Normalized rules blocked until Roskomnadzor publishes range schema. |
|
||
| `ru.nkcki` | ✅ Done | SemVer-style range parser covers vendor firmware records; remaining prose ranges logged with `Normalized version rules missing`. |
|
||
|
||
## 2. Code snippet: SemVer connector (CCCS/Cisco/ICS-CISA)
|
||
|
||
```csharp
|
||
using StellaOps.Concelier.Normalization.SemVer;
|
||
|
||
private static IReadOnlyList<AffectedPackage> BuildPackages(MyDto dto, DateTimeOffset recordedAt)
|
||
{
|
||
var packages = new List<AffectedPackage>();
|
||
|
||
foreach (var entry in dto.AffectedEntries.Select((value, index) => (value, index)))
|
||
{
|
||
var rangeText = entry.value.Range?.Trim();
|
||
var patched = entry.value.FixedVersion;
|
||
var provenance = $"{MyConnectorPlugin.SourceName}:{dto.AdvisoryId}:{entry.index}";
|
||
|
||
var normalizedRules = SemVerRangeRuleBuilder.BuildNormalizedRules(rangeText, patched, provenance);
|
||
var primitives = SemVerRangeRuleBuilder.Build(rangeText, patched, provenance)
|
||
.Select(result => result.Primitive.ToAffectedVersionRange(provenance))
|
||
.ToArray();
|
||
|
||
packages.Add(new AffectedPackage(
|
||
AffectedPackageTypes.SemVer,
|
||
entry.value.PackageId,
|
||
versionRanges: primitives,
|
||
normalizedVersions: normalizedRules,
|
||
provenance: new[]
|
||
{
|
||
new AdvisoryProvenance(
|
||
MyConnectorPlugin.SourceName,
|
||
"package",
|
||
entry.value.PackageId,
|
||
recordedAt,
|
||
new[] { ProvenanceFieldMasks.AffectedPackages })
|
||
}));
|
||
}
|
||
|
||
return packages;
|
||
}
|
||
```
|
||
|
||
A few notes:
|
||
|
||
- If you already have `SemVerPrimitive` instances, call `.ToNormalizedVersionRule(provenance)` on each primitive instead of rebuilding from raw strings.
|
||
- Use `SemVerRangeRuleBuilder.BuildNormalizedRules` when the connector only tracks raw range text plus an optional fixed/patched version.
|
||
- For products that encode ranges like `"ExampleOS 4.12 - 4.14"`, run a small regex to peel off the version substring (see §3) and use the same provenance note when emitting the rule and the original range primitive.
|
||
|
||
## 3. Parsing helper for trailing version phrases
|
||
|
||
Many of the overdue connectors store affected products as natural-language phrases. The following helper normalises common patterns (`1.2 - 1.4`, `<= 3.5`, `Version 7.2 and later`).
|
||
|
||
```csharp
|
||
private static string? TryExtractRangeSuffix(string productString)
|
||
{
|
||
if (string.IsNullOrWhiteSpace(productString))
|
||
{
|
||
return null;
|
||
}
|
||
|
||
var match = Regex.Match(productString, "(?<range>(?:<=?|>=?)?\s*\d+(?:\.\d+){0,2}(?:\s*-\s*\d+(?:\.\d+){0,2})?)", RegexOptions.CultureInvariant);
|
||
return match.Success ? match.Groups["range"].Value.Trim() : null;
|
||
}
|
||
```
|
||
|
||
Once you extract the `range` fragment, feed it to `SemVerRangeRuleBuilder.BuildNormalizedRules(range, null, provenance)`. Keep the original product string as-is so operators can still see the descriptive text.
|
||
|
||
## 4. Merge dashboard hygiene
|
||
|
||
- Run `dotnet build src/Concelier/__Libraries/StellaOps.Concelier.Merge/StellaOps.Concelier.Merge.csproj` after wiring a connector to confirm no warnings appear.
|
||
- Merge counter tag pairs to watch in Grafana/CI logs:
|
||
- `concelier.merge.normalized_rules{package_type="npm"}` – increases once the connector emits normalized arrays.
|
||
- `concelier.merge.normalized_rules_missing{package_type="vendor"}` – should trend to zero once rollout completes.
|
||
- The Merge service now logs `Normalized version rules missing for {AdvisoryKey}; sources=...; packageTypes=...` when a connector still needs to supply normalized rules. Use this as the acceptance gate for FEEDMERGE-COORD-02-901/902.
|
||
|
||
## 5. Documentation touchpoints
|
||
|
||
- Update the connector `TASKS.md` entry with the date you flipped on normalized rules and note the provenance format you chose.
|
||
- Record any locale-specific parsing (e.g., German `bis`) in the connector README so future contributors can regenerate fixtures confidently.
|
||
- When opening the PR, include `dotnet test` output covering the connector tests so reviewers see the normalized array diff.
|
||
|
||
Once each connector follows the steps above, we can mark FEEDCONN-CCCS-02-009, FEEDCONN-CISCO-02-009, FEEDCONN-CERTBUND-02-010, FEEDCONN-ICSCISA-02-012, and the FEEDMERGE-COORD-02-90x tasks as resolved.
|