stella-ops.org/git.stella-ops.org
1
0
Fork 0
Code Issues Pull Requests Actions 2 Releases Wiki Activity

feat: Enhance MongoDB storage with event publishing and outbox support

Browse Source 
- Added `MongoAdvisoryObservationEventPublisher` and `NatsAdvisoryObservationEventPublisher` for event publishing.
- Registered `IAdvisoryObservationEventPublisher` to choose between NATS and MongoDB based on configuration.
- Introduced `MongoAdvisoryObservationEventOutbox` for outbox pattern implementation.
- Updated service collection to include new event publishers and outbox.
- Added a new hosted service `AdvisoryObservationTransportWorker` for processing events.

feat: Update project dependencies

- Added `NATS.Client.Core` package to the project for NATS integration.

test: Add unit tests for AdvisoryLinkset normalization

- Created `AdvisoryLinksetNormalizationConfidenceTests` to validate confidence score calculations.

fix: Adjust confidence assertion in `AdvisoryObservationAggregationTests`

- Updated confidence assertion to allow a range instead of a fixed value.

test: Implement tests for AdvisoryObservationEventFactory

- Added `AdvisoryObservationEventFactoryTests` to ensure correct mapping and hashing of observation events.

chore: Configure test project for Findings Ledger

- Created `Directory.Build.props` for test project configuration.
- Added `StellaOps.Findings.Ledger.Exports.Unit.csproj` for unit tests related to findings ledger exports.

feat: Implement export contracts for findings ledger

- Defined export request and response contracts in `ExportContracts.cs`.
- Created various export item records for findings, VEX, advisories, and SBOMs.

feat: Add export functionality to Findings Ledger Web Service

- Implemented endpoints for exporting findings, VEX, advisories, and SBOMs.
- Integrated `ExportQueryService` for handling export logic and pagination.

test: Add tests for Node language analyzer phase 22

- Implemented `NodePhase22SampleLoaderTests` to validate loading of NDJSON fixtures.
- Created sample NDJSON file for testing.

chore: Set up isolated test environment for Node tests

- Added `node-isolated.runsettings` for isolated test execution.
- Created `node-tests-isolated.sh` script for running tests in isolation.
This commit is contained in:
master
2025-11-20 23:08:45 +02:00
parent f0e74d2ee8
commit 2e276d6676
49 changed files with 1996 additions and 113 deletions
Download Patch File Download Diff File Expand all files Collapse all files

44
docs/modules/concelier/events/advisory.observation.updated@1.md Normal file
View File

@@ -0,0 +1,44 @@
# advisory.observation.updated@1 · Event contract
Purpose: unblock CONCELIER-GRAPH-21-002 by freezing the platform event shape for observation changes emitted by Concelier. This is the only supported event for observation churn; downstreams subscribe for evidence fan-out and replay bundles.
## Envelope & transport
- Subject: `concelier.advisory.observation.updated.v1`
- Type/version: `advisory.observation.updated@1`
- Transport: NATS (primary), Redis Stream `concelier:advisory.observation.updated:v1` (fallback). Both carry the same DSSE envelope.
- DSSE payloadType: `application/vnd.stellaops.advisory.observation.updated.v1+json`.
- Signature: Ed25519 via Platform Events signer; attach Rekor UUID when available. Offline kits treat the envelope as the source of truth.
## Payload (JSON)
| Field | Type | Rules |
| --- | --- | --- |
| `eventId` | string (uuid) | Generated by publisher; idempotency key.
| `tenantId` | string | `urn:tenant:{uuid}`; required for multi-tenant routing.
| `observationId` | string (ObjectId) | Mongo `_id` of the observation document.
| `advisoryId` | string | Upstream advisory identifier (e.g., CVE, GHSA, vendor id).
| `source` | object | `{ vendor, stream, api, collectorVersion }`; lowercase vendor, non-empty.
| `linksetSummary` | object | `{ aliases: string[], purls: string[], cpes?: string[], scopes?: string[], relationships?: object[] }` all arrays pre-sorted ASCII.
| `supersedesId` | string (ObjectId, optional) | Previous observation `_id` if this is a new revision; omitted otherwise.
| `documentSha` | string | SHA-256 of raw upstream document.
| `observationHash` | string | Stable hash over canonicalized observation JSON (tenant, source, advisoryId, documentSha, fetchedAt).
| `ingestedAt` | string (ISO-8601 UTC) | Timestamp when appended.
| `traceId` | string (optional) | Propagated from ingest job/request; aids join with logs/metrics.
| `replayCursor` | string | Monotone cursor for offline bundle ordering (tick from change stream resume token).
### Determinism & ordering
- Arrays sorted ASCII; objects field-sorted when hashing.
- `eventId` + `replayCursor` provide exactly-once consumer handling; duplicates must be ignored when `observationHash` unchanged.
- No judgments: only raw facts and hash pointers; any derived severity/merge content is forbidden.
### Error contracts for Scheduler
- Retryable NATS/Redis failures use backoff capped at 30s; after 5 attempts, emit `concelier.events.dlq` with the same envelope and `error` field describing transport failure.
- Consumers must NACK on schema validation failure; publisher logs `ERR_EVENT_SCHEMA` and quarantines the offending observation id.
## Sample payload
See `advisory.observation.updated@1.sample.json` (canonical field order, ASCII sorted arrays). Hashes intentionally short for readability; replace with real values in tests.
## Schema
`advisory.observation.updated@1.schema.json` provides a JSON Schema (draft 2020-12) for runtime validation; any additional fields are rejected.
## Compatibility note
Sprint tracker referenced `sbom.observation.updated`; this contract standardises on `advisory.observation.updated@1`. If a legacy alias is required for interim consumers, mirror the envelope on subject `surface.sbom.observation.updated.v1` with identical payload.