feat: Enhance MongoDB storage with event publishing and outbox support
- Added `MongoAdvisoryObservationEventPublisher` and `NatsAdvisoryObservationEventPublisher` for event publishing. - Registered `IAdvisoryObservationEventPublisher` to choose between NATS and MongoDB based on configuration. - Introduced `MongoAdvisoryObservationEventOutbox` for outbox pattern implementation. - Updated service collection to include new event publishers and outbox. - Added a new hosted service `AdvisoryObservationTransportWorker` for processing events. feat: Update project dependencies - Added `NATS.Client.Core` package to the project for NATS integration. test: Add unit tests for AdvisoryLinkset normalization - Created `AdvisoryLinksetNormalizationConfidenceTests` to validate confidence score calculations. fix: Adjust confidence assertion in `AdvisoryObservationAggregationTests` - Updated confidence assertion to allow a range instead of a fixed value. test: Implement tests for AdvisoryObservationEventFactory - Added `AdvisoryObservationEventFactoryTests` to ensure correct mapping and hashing of observation events. chore: Configure test project for Findings Ledger - Created `Directory.Build.props` for test project configuration. - Added `StellaOps.Findings.Ledger.Exports.Unit.csproj` for unit tests related to findings ledger exports. feat: Implement export contracts for findings ledger - Defined export request and response contracts in `ExportContracts.cs`. - Created various export item records for findings, VEX, advisories, and SBOMs. feat: Add export functionality to Findings Ledger Web Service - Implemented endpoints for exporting findings, VEX, advisories, and SBOMs. - Integrated `ExportQueryService` for handling export logic and pagination. test: Add tests for Node language analyzer phase 22 - Implemented `NodePhase22SampleLoaderTests` to validate loading of NDJSON fixtures. - Created sample NDJSON file for testing. chore: Set up isolated test environment for Node tests - Added `node-isolated.runsettings` for isolated test execution. - Created `node-tests-isolated.sh` script for running tests in isolation.
This commit is contained in:
56
docs/modules/concelier/linkset-correlation-21-002.md
Normal file
56
docs/modules/concelier/linkset-correlation-21-002.md
Normal file
@@ -0,0 +1,56 @@
|
||||
# CONCELIER-LNM-21-002 · Linkset correlation rules (v1)
|
||||
|
||||
Purpose: unblock CONCELIER-LNM-21-002 by freezing correlation/precedence rules and providing fixtures so builders and downstream consumers can proceed.
|
||||
|
||||
## Scope
|
||||
- Applies to linksets produced from `advisory_observations` (LNM v1).
|
||||
- Correlation is aggregation-only: no value synthesis or merge; emit conflicts instead of collapsing fields.
|
||||
- Output persists in `advisory_linksets` and drives `advisory.linkset.updated@1` events.
|
||||
|
||||
## Deterministic confidence calculation (0–1)
|
||||
```
|
||||
confidence = clamp(
|
||||
0.40 * alias_score +
|
||||
0.25 * purl_overlap_score +
|
||||
0.15 * cpe_overlap_score +
|
||||
0.10 * severity_agreement +
|
||||
0.05 * reference_overlap +
|
||||
0.05 * freshness_score
|
||||
)
|
||||
```
|
||||
- `alias_score`: 1 if any alias exact-match across observations; 0.5 if vendor ID prefixes match; else 0.
|
||||
- `purl_overlap_score`: 1 if same pkg+version range intersects; 0.6 if same pkg family but disjoint ranges; 0 otherwise. Use semver/rpm/deb comparers as in LNM v1.
|
||||
- `cpe_overlap_score`: 1 if any CPE exact-match; 0.5 if same vendor/product, any version; else 0.
|
||||
- `severity_agreement`: 1 if CVSS base score delta ≤ 0.1; 0.5 if ≤ 1.0; else 0. Use max of available CVSS per observation.
|
||||
- `reference_overlap`: fraction of shared reference URLs (case-normalized) between the pair with the highest overlap across the set.
|
||||
- `freshness_score`: 1 when `fetchedAt` spread ≤ 48h; linearly decays to 0 at 14 days.
|
||||
- Sort observations before scoring by `(source.vendor, advisoryId, fetchedAt)`; reuse that order for hashing and for output arrays.
|
||||
|
||||
## Conflict emission (add-only)
|
||||
Emit a conflict entry per divergent field group:
|
||||
- `severity-mismatch`: CVSS base score delta > 1.0 or vector differs.
|
||||
- `affected-range-divergence`: version ranges do not intersect.
|
||||
- `reference-clash`: no shared references and source vendors differ.
|
||||
- `alias-inconsistency`: aliases disjoint across observations.
|
||||
- `metadata-gap`: required fields missing on any observation.
|
||||
Each conflict includes `field`, `reason`, and `values` (array of `source: value` strings) and is stable-sorted by `field` then `reason`.
|
||||
|
||||
## Linkset output shape additions
|
||||
- `key.confidence`: populated from formula above.
|
||||
- `conflicts[]`: as defined; may be empty but never null.
|
||||
- `normalized` retains add-only fields from `link-not-merge-schema.md`; do not drop raw ranges even when disjoint.
|
||||
- `provenance.hashes`: sorted list of `observationHash` values; used by replay bundles.
|
||||
|
||||
## Fixtures
|
||||
- `docs/samples/lnm/linkset-lnm-21-002-sample.json`: two-source agreement (high confidence, no conflicts).
|
||||
- `docs/samples/lnm/linkset-lnm-21-002-conflict.json`: three-source disagreement showing conflict records and confidence < 0.7.
|
||||
All fixtures use ASCII ordering and ISO-8601 UTC timestamps and may be used as golden outputs in tests.
|
||||
|
||||
## Implementation checklist
|
||||
- Builder must refuse to overwrite existing linkset when incoming hash list unchanged.
|
||||
- Correlation job idempotency key: `hash(tenantId|aliasSet|purlSet|fetchedAtBucket)`.
|
||||
- Telemetry: counter `concelier.linkset.builder.conflict_total{field,reason}` and histogram `concelier.linkset.builder.confidence` (0–1 buckets).
|
||||
- Event emission: include `confidence` and `conflicts` summary in `advisory.linkset.updated@1`; keep arrays sorted as above.
|
||||
|
||||
## Change control
|
||||
- Add-only. Adjusting weights or conflict codes requires new version `advisory.linkset.updated@2` and a sprint note.
|
||||
Reference in New Issue
Block a user