stella-ops.org/git.stella-ops.org
1
0
Fork 0
Code Issues Pull Requests Actions 1 Releases Wiki Activity

Add Vexer connector suite, format normalizers, and tooling

Browse Source
This commit is contained in:
master
2025-10-17 19:17:27 +03:00
parent cb3acb8c4a
commit 29a7d51e41
115 changed files with 9659 additions and 42 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
.gitignore vendored
View File

@@ -23,3 +23,5 @@ TestResults/
seed-data/ics-cisa/*.csv seed-data/ics-cisa/*.csv
seed-data/ics-cisa/*.xlsx seed-data/ics-cisa/*.xlsx
seed-data/ics-cisa/*.sha256 seed-data/ics-cisa/*.sha256
seed-data/cert-bund/**/*.json
seed-data/cert-bund/**/*.sha256

42
SPRINTS.md
View File

@@ -105,8 +105,8 @@
| Sprint 4 | Schema Parity & Freshness Alignment | src/StellaOps.Feedser.Merge/TASKS.md | DONE (2025-10-15) | Team Merge & QA Enforcement | FEEDMERGE-ENGINE-04-005 | Connector coordination for new advisory fields<br>GHSA/NVD/OSV connectors now ship description, CWE, and canonical metric data with refreshed fixtures; merge coordination log updated and exporters notified. | | Sprint 4 | Schema Parity & Freshness Alignment | src/StellaOps.Feedser.Merge/TASKS.md | DONE (2025-10-15) | Team Merge & QA Enforcement | FEEDMERGE-ENGINE-04-005 | Connector coordination for new advisory fields<br>GHSA/NVD/OSV connectors now ship description, CWE, and canonical metric data with refreshed fixtures; merge coordination log updated and exporters notified. |
| Sprint 4 | Schema Parity & Freshness Alignment | src/StellaOps.Feedser.Exporter.Json/TASKS.md | DONE (2025-10-15) | Team Exporters – JSON | FEEDEXPORT-JSON-04-001 | Surface new advisory fields in JSON exporter<br>Update schemas/offline bundle + fixtures once model/core parity lands.<br>2025-10-15: `dotnet test src/StellaOps.Feedser.Exporter.Json.Tests` validated canonical metric/CWE emission. | | Sprint 4 | Schema Parity & Freshness Alignment | src/StellaOps.Feedser.Exporter.Json/TASKS.md | DONE (2025-10-15) | Team Exporters – JSON | FEEDEXPORT-JSON-04-001 | Surface new advisory fields in JSON exporter<br>Update schemas/offline bundle + fixtures once model/core parity lands.<br>2025-10-15: `dotnet test src/StellaOps.Feedser.Exporter.Json.Tests` validated canonical metric/CWE emission. |
| Sprint 4 | Schema Parity & Freshness Alignment | src/StellaOps.Feedser.Exporter.TrivyDb/TASKS.md | DONE (2025-10-15) | Team Exporters – Trivy DB | FEEDEXPORT-TRIVY-04-001 | Propagate new advisory fields into Trivy DB package<br>Extend Bolt builder, metadata, and regression tests for the expanded schema.<br>2025-10-15: `dotnet test src/StellaOps.Feedser.Exporter.TrivyDb.Tests` confirmed canonical metric/CWE propagation. | | Sprint 4 | Schema Parity & Freshness Alignment | src/StellaOps.Feedser.Exporter.TrivyDb/TASKS.md | DONE (2025-10-15) | Team Exporters – Trivy DB | FEEDEXPORT-TRIVY-04-001 | Propagate new advisory fields into Trivy DB package<br>Extend Bolt builder, metadata, and regression tests for the expanded schema.<br>2025-10-15: `dotnet test src/StellaOps.Feedser.Exporter.TrivyDb.Tests` confirmed canonical metric/CWE propagation. |
| Sprint 4 | Schema Parity & Freshness Alignment | src/StellaOps.Feedser.Source.Ghsa/TASKS.md | TODO | Team Connector Regression Fixtures | FEEDCONN-GHSA-04-004 | Harden CVSS fallback so canonical metric ids persist when GitHub omits vectors; extend fixtures and document severity precedence hand-off to Merge. | | Sprint 4 | Schema Parity & Freshness Alignment | src/StellaOps.Feedser.Source.Ghsa/TASKS.md | DONE (2025-10-16) | Team Connector Regression Fixtures | FEEDCONN-GHSA-04-004 | Harden CVSS fallback so canonical metric ids persist when GitHub omits vectors; extend fixtures and document severity precedence hand-off to Merge. |
| Sprint 4 | Schema Parity & Freshness Alignment | src/StellaOps.Feedser.Source.Osv/TASKS.md | TODO | Team Connector Expansion – GHSA/NVD/OSV | FEEDCONN-OSV-04-005 | Map OSV advisories lacking CVSS vectors to canonical metric ids/notes and document CWE provenance quirks; schedule parity fixture updates. | | Sprint 4 | Schema Parity & Freshness Alignment | src/StellaOps.Feedser.Source.Osv/TASKS.md | DONE (2025-10-16) | Team Connector Expansion – GHSA/NVD/OSV | FEEDCONN-OSV-04-005 | Map OSV advisories lacking CVSS vectors to canonical metric ids/notes and document CWE provenance quirks; schedule parity fixture updates. |
| Sprint 5 | Vexer Core Foundations | src/StellaOps.Vexer.Core/TASKS.md | DONE (2025-10-15) | Team Vexer Core & Policy | VEXER-CORE-01-001 | Stand up canonical VEX claim/consensus records with deterministic serializers so Storage/Exports share a stable contract. | | Sprint 5 | Vexer Core Foundations | src/StellaOps.Vexer.Core/TASKS.md | DONE (2025-10-15) | Team Vexer Core & Policy | VEXER-CORE-01-001 | Stand up canonical VEX claim/consensus records with deterministic serializers so Storage/Exports share a stable contract. |
| Sprint 5 | Vexer Core Foundations | src/StellaOps.Vexer.Core/TASKS.md | DONE (2025-10-15) | Team Vexer Core & Policy | VEXER-CORE-01-002 | Implement trust-weighted consensus resolver with baseline policy weights, justification gates, telemetry output, and majority/tie handling. | | Sprint 5 | Vexer Core Foundations | src/StellaOps.Vexer.Core/TASKS.md | DONE (2025-10-15) | Team Vexer Core & Policy | VEXER-CORE-01-002 | Implement trust-weighted consensus resolver with baseline policy weights, justification gates, telemetry output, and majority/tie handling. |
| Sprint 5 | Vexer Core Foundations | src/StellaOps.Vexer.Core/TASKS.md | DONE (2025-10-15) | Team Vexer Core & Policy | VEXER-CORE-01-003 | Publish shared connector/exporter/attestation abstractions and deterministic query signature utilities for cache/attestation workflows. | | Sprint 5 | Vexer Core Foundations | src/StellaOps.Vexer.Core/TASKS.md | DONE (2025-10-15) | Team Vexer Core & Policy | VEXER-CORE-01-003 | Publish shared connector/exporter/attestation abstractions and deterministic query signature utilities for cache/attestation workflows. |
@@ -116,29 +116,35 @@
| Sprint 5 | Vexer Core Foundations | src/StellaOps.Vexer.Policy/TASKS.md | DONE (2025-10-16) | Team Vexer Policy | VEXER-POLICY-01-004 | Implement YAML/JSON schema validation and deterministic diagnostics for operator bundles. | | Sprint 5 | Vexer Core Foundations | src/StellaOps.Vexer.Policy/TASKS.md | DONE (2025-10-16) | Team Vexer Policy | VEXER-POLICY-01-004 | Implement YAML/JSON schema validation and deterministic diagnostics for operator bundles. |
| Sprint 5 | Vexer Core Foundations | src/StellaOps.Vexer.Policy/TASKS.md | DONE (2025-10-16) | Team Vexer Policy | VEXER-POLICY-01-005 | Add policy change tracking, snapshot digests, and telemetry/logging hooks. | | Sprint 5 | Vexer Core Foundations | src/StellaOps.Vexer.Policy/TASKS.md | DONE (2025-10-16) | Team Vexer Policy | VEXER-POLICY-01-005 | Add policy change tracking, snapshot digests, and telemetry/logging hooks. |
| Sprint 5 | Vexer Core Foundations | src/StellaOps.Vexer.Storage.Mongo/TASKS.md | DONE (2025-10-15) | Team Vexer Storage | VEXER-STORAGE-01-001 | Mongo mapping registry plus raw/export entities and DI extensions in place. | | Sprint 5 | Vexer Core Foundations | src/StellaOps.Vexer.Storage.Mongo/TASKS.md | DONE (2025-10-15) | Team Vexer Storage | VEXER-STORAGE-01-001 | Mongo mapping registry plus raw/export entities and DI extensions in place. |
| Sprint 5 | Vexer Core Foundations | src/StellaOps.Vexer.Storage.Mongo/TASKS.md | TODO | Team Vexer Storage | VEXER-STORAGE-01-004 | Build provider/consensus/cache class maps and related collections. | | Sprint 5 | Vexer Core Foundations | src/StellaOps.Vexer.Storage.Mongo/TASKS.md | DONE (2025-10-16) | Team Vexer Storage | VEXER-STORAGE-01-004 | Build provider/consensus/cache class maps and related collections. |
| Sprint 5 | Vexer Core Foundations | src/StellaOps.Vexer.Export/TASKS.md | DONE (2025-10-15) | Team Vexer Export | VEXER-EXPORT-01-001 | Export engine delivers cache lookup, manifest creation, and policy integration. | | Sprint 5 | Vexer Core Foundations | src/StellaOps.Vexer.Export/TASKS.md | DONE (2025-10-15) | Team Vexer Export | VEXER-EXPORT-01-001 | Export engine delivers cache lookup, manifest creation, and policy integration. |
| Sprint 5 | Vexer Core Foundations | src/StellaOps.Vexer.Export/TASKS.md | TODO | Team Vexer Export | VEXER-EXPORT-01-004 | Connect export engine to attestation client and persist Rekor metadata. | | Sprint 5 | Vexer Core Foundations | src/StellaOps.Vexer.Export/TASKS.md | DONE (2025-10-17) | Team Vexer Export | VEXER-EXPORT-01-004 | Connect export engine to attestation client and persist Rekor metadata. |
| Sprint 5 | Vexer Core Foundations | src/StellaOps.Vexer.Attestation/TASKS.md | TODO | Team Vexer Attestation | VEXER-ATTEST-01-001 | Implement in-toto predicate + DSSE builder providing envelopes for export attestation. | | Sprint 5 | Vexer Core Foundations | src/StellaOps.Vexer.Attestation/TASKS.md | DONE (2025-10-16) | Team Vexer Attestation | VEXER-ATTEST-01-001 | Implement in-toto predicate + DSSE builder providing envelopes for export attestation. |
| Sprint 5 | Vexer Core Foundations | src/StellaOps.Vexer.Connectors.Abstractions/TASKS.md | TODO | Team Vexer Connectors | VEXER-CONN-ABS-01-001 | Deliver shared connector context/base classes so provider plug-ins can be activated via WebService/Worker. | | Sprint 5 | Vexer Core Foundations | src/StellaOps.Vexer.Connectors.Abstractions/TASKS.md | DONE (2025-10-17) | Team Vexer Connectors | VEXER-CONN-ABS-01-001 | Deliver shared connector context/base classes so provider plug-ins can be activated via WebService/Worker. |
| Sprint 5 | Vexer Core Foundations | src/StellaOps.Vexer.WebService/TASKS.md | TODO | Team Vexer WebService | VEXER-WEB-01-001 | Scaffold minimal API host, DI, and `/vexer/status` endpoint integrating policy, storage, export, and attestation services. | | Sprint 5 | Vexer Core Foundations | src/StellaOps.Vexer.WebService/TASKS.md | DONE (2025-10-17) | Team Vexer WebService | VEXER-WEB-01-001 | Scaffold minimal API host, DI, and `/vexer/status` endpoint integrating policy, storage, export, and attestation services. |
| Sprint 6 | Vexer Ingest & Formats | src/StellaOps.Vexer.Worker/TASKS.md | TODO | Team Vexer Worker | VEXER-WORKER-01-001 | Create Worker host with provider scheduling and logging to drive recurring pulls/reconciliation. | | Sprint 6 | Vexer Ingest & Formats | src/StellaOps.Vexer.Worker/TASKS.md | DONE (2025-10-17) | Team Vexer Worker | VEXER-WORKER-01-001 | Create Worker host with provider scheduling and logging to drive recurring pulls/reconciliation. |
| Sprint 6 | Vexer Ingest & Formats | src/StellaOps.Vexer.Formats.CSAF/TASKS.md | TODO | Team Vexer Formats | VEXER-FMT-CSAF-01-001 | Implement CSAF normalizer foundation translating provider documents into `VexClaim` entries. | | Sprint 6 | Vexer Ingest & Formats | src/StellaOps.Vexer.Formats.CSAF/TASKS.md | DONE (2025-10-17) | Team Vexer Formats | VEXER-FMT-CSAF-01-001 | Implement CSAF normalizer foundation translating provider documents into `VexClaim` entries. |
| Sprint 6 | Vexer Ingest & Formats | src/StellaOps.Vexer.Formats.CycloneDX/TASKS.md | TODO | Team Vexer Formats | VEXER-FMT-CYCLONE-01-001 | Implement CycloneDX VEX normalizer capturing `analysis` state and component references. | | Sprint 6 | Vexer Ingest & Formats | src/StellaOps.Vexer.Formats.CycloneDX/TASKS.md | DONE (2025-10-17) | Team Vexer Formats | VEXER-FMT-CYCLONE-01-001 | Implement CycloneDX VEX normalizer capturing `analysis` state and component references. |
| Sprint 6 | Vexer Ingest & Formats | src/StellaOps.Vexer.Formats.OpenVEX/TASKS.md | TODO | Team Vexer Formats | VEXER-FMT-OPENVEX-01-001 | Implement OpenVEX normalizer to ingest attestations into canonical claims with provenance. | | Sprint 6 | Vexer Ingest & Formats | src/StellaOps.Vexer.Formats.OpenVEX/TASKS.md | DONE (2025-10-17) | Team Vexer Formats | VEXER-FMT-OPENVEX-01-001 | Implement OpenVEX normalizer to ingest attestations into canonical claims with provenance. |
| Sprint 6 | Vexer Ingest & Formats | src/StellaOps.Vexer.Connectors.RedHat.CSAF/TASKS.md | TODO | Team Vexer Connectors – Red Hat | VEXER-CONN-RH-01-001 | Ship Red Hat CSAF provider metadata discovery enabling incremental pulls. | | Sprint 6 | Vexer Ingest & Formats | src/StellaOps.Vexer.Connectors.RedHat.CSAF/TASKS.md | DONE (2025-10-17) | Team Vexer Connectors – Red Hat | VEXER-CONN-RH-01-001 | Ship Red Hat CSAF provider metadata discovery enabling incremental pulls. |
| Sprint 6 | Vexer Ingest & Formats | src/StellaOps.Vexer.Connectors.Cisco.CSAF/TASKS.md | TODO | Team Vexer Connectors – Cisco | VEXER-CONN-CISCO-01-001 | Implement Cisco CSAF endpoint discovery/auth to unlock paginated pulls. | | Sprint 6 | Vexer Ingest & Formats | src/StellaOps.Vexer.Connectors.RedHat.CSAF/TASKS.md | DONE (2025-10-17) | Team Vexer Connectors – Red Hat | VEXER-CONN-RH-01-002 | Fetch CSAF windows with ETag handling, resume tokens, quarantine on schema errors, and persist raw docs. |
| Sprint 6 | Vexer Ingest & Formats | src/StellaOps.Vexer.Connectors.SUSE.RancherVEXHub/TASKS.md | TODO | Team Vexer Connectors – SUSE | VEXER-CONN-SUSE-01-001 | Build Rancher VEX Hub discovery/subscription path with offline snapshot support. | | Sprint 6 | Vexer Ingest & Formats | src/StellaOps.Vexer.Connectors.RedHat.CSAF/TASKS.md | DONE (2025-10-17) | Team Vexer Connectors – Red Hat | VEXER-CONN-RH-01-003 | Populate provider trust overrides (cosign issuer, identity regex) and provenance hints for policy evaluation/logging. |
| Sprint 6 | Vexer Ingest & Formats | src/StellaOps.Vexer.Connectors.MSRC.CSAF/TASKS.md | TODO | Team Vexer Connectors – MSRC | VEXER-CONN-MS-01-001 | Deliver AAD onboarding/token cache for MSRC CSAF ingestion. | | Sprint 6 | Vexer Ingest & Formats | src/StellaOps.Vexer.Connectors.RedHat.CSAF/TASKS.md | DONE (2025-10-17) | Team Vexer Connectors – Red Hat | VEXER-CONN-RH-01-004 | Persist resume cursors (last updated timestamp/document hashes) in storage and reload during fetch to avoid duplicates. |
| Sprint 6 | Vexer Ingest & Formats | src/StellaOps.Vexer.Connectors.Oracle.CSAF/TASKS.md | TODO | Team Vexer Connectors – Oracle | VEXER-CONN-ORACLE-01-001 | Implement Oracle CSAF catalogue discovery with CPU calendar awareness. | | Sprint 6 | Vexer Ingest & Formats | src/StellaOps.Vexer.Connectors.RedHat.CSAF/TASKS.md | DONE (2025-10-17) | Team Vexer Connectors – Red Hat | VEXER-CONN-RH-01-005 | Register connector in Worker/WebService DI, add scheduled jobs, and document CLI triggers for Red Hat CSAF pulls. |
| Sprint 6 | Vexer Ingest & Formats | src/StellaOps.Vexer.Connectors.Ubuntu.CSAF/TASKS.md | TODO | Team Vexer Connectors – Ubuntu | VEXER-CONN-UBUNTU-01-001 | Implement Ubuntu CSAF discovery and channel selection for USN ingestion. | | Sprint 6 | Vexer Ingest & Formats | src/StellaOps.Vexer.Connectors.RedHat.CSAF/TASKS.md | DONE (2025-10-17) | Team Vexer Connectors – Red Hat | VEXER-CONN-RH-01-006 | Add CSAF normalization parity fixtures ensuring RHSA-specific metadata is preserved. |
| Sprint 6 | Vexer Ingest & Formats | src/StellaOps.Vexer.Connectors.Cisco.CSAF/TASKS.md | DONE (2025-10-17) | Team Vexer Connectors – Cisco | VEXER-CONN-CISCO-01-001 | Implement Cisco CSAF endpoint discovery/auth to unlock paginated pulls. |
| Sprint 6 | Vexer Ingest & Formats | src/StellaOps.Vexer.Connectors.Cisco.CSAF/TASKS.md | DONE (2025-10-17) | Team Vexer Connectors – Cisco | VEXER-CONN-CISCO-01-002 | Implement Cisco CSAF paginated fetch loop with dedupe and raw persistence support. |
| Sprint 6 | Vexer Ingest & Formats | src/StellaOps.Vexer.Connectors.SUSE.RancherVEXHub/TASKS.md | DONE (2025-10-17) | Team Vexer Connectors – SUSE | VEXER-CONN-SUSE-01-001 | Build Rancher VEX Hub discovery/subscription path with offline snapshot support. |
| Sprint 6 | Vexer Ingest & Formats | src/StellaOps.Vexer.Connectors.MSRC.CSAF/TASKS.md | DONE (2025-10-17) | Team Vexer Connectors – MSRC | VEXER-CONN-MS-01-001 | Deliver AAD onboarding/token cache for MSRC CSAF ingestion. |
| Sprint 6 | Vexer Ingest & Formats | src/StellaOps.Vexer.Connectors.Oracle.CSAF/TASKS.md | DONE (2025-10-17) | Team Vexer Connectors – Oracle | VEXER-CONN-ORACLE-01-001 | Implement Oracle CSAF catalogue discovery with CPU calendar awareness. |
| Sprint 6 | Vexer Ingest & Formats | src/StellaOps.Vexer.Connectors.Ubuntu.CSAF/TASKS.md | DONE (2025-10-17) | Team Vexer Connectors – Ubuntu | VEXER-CONN-UBUNTU-01-001 | Implement Ubuntu CSAF discovery and channel selection for USN ingestion. |
| Sprint 6 | Vexer Ingest & Formats | src/StellaOps.Vexer.Connectors.OCI.OpenVEX.Attest/TASKS.md | TODO | Team Vexer Connectors – OCI | VEXER-CONN-OCI-01-001 | Wire OCI discovery/auth to fetch OpenVEX attestations for configured images. | | Sprint 6 | Vexer Ingest & Formats | src/StellaOps.Vexer.Connectors.OCI.OpenVEX.Attest/TASKS.md | TODO | Team Vexer Connectors – OCI | VEXER-CONN-OCI-01-001 | Wire OCI discovery/auth to fetch OpenVEX attestations for configured images. |
| Sprint 6 | Vexer Ingest & Formats | src/StellaOps.Cli/TASKS.md | TODO | DevEx/CLI | VEXER-CLI-01-001 | Add `vexer` CLI verbs bridging to WebService with consistent auth and offline UX. | | Sprint 6 | Vexer Ingest & Formats | src/StellaOps.Cli/TASKS.md | TODO | DevEx/CLI | VEXER-CLI-01-001 | Add `vexer` CLI verbs bridging to WebService with consistent auth and offline UX. |
| Sprint 7 | Contextual Truth Foundations | src/StellaOps.Vexer.Core/TASKS.md | TODO | Team Vexer Core & Policy | VEXER-CORE-02-001 | Context signal schema prep – extend consensus models with severity/KEV/EPSS fields and update canonical serializers. | | Sprint 7 | Contextual Truth Foundations | src/StellaOps.Vexer.Core/TASKS.md | TODO | Team Vexer Core & Policy | VEXER-CORE-02-001 | Context signal schema prep – extend consensus models with severity/KEV/EPSS fields and update canonical serializers. |
| Sprint 7 | Contextual Truth Foundations | src/StellaOps.Vexer.Policy/TASKS.md | TODO | Team Vexer Policy | VEXER-POLICY-02-001 | Scoring coefficients & weight ceilings – add α/β options, weight boosts, and validation guidance. | | Sprint 7 | Contextual Truth Foundations | src/StellaOps.Vexer.Policy/TASKS.md | TODO | Team Vexer Policy | VEXER-POLICY-02-001 | Scoring coefficients & weight ceilings – add α/β options, weight boosts, and validation guidance. |
| Sprint 7 | Contextual Truth Foundations | src/StellaOps.Vexer.Storage.Mongo/TASKS.md | TODO | Team Vexer Storage | VEXER-STORAGE-02-001 | Statement events & scoring signals – create immutable VEX statement store plus consensus extensions with indexes/migrations. | | Sprint 7 | Contextual Truth Foundations | src/StellaOps.Vexer.Storage.Mongo/TASKS.md | TODO | Team Vexer Storage | VEXER-STORAGE-02-001 | Statement events & scoring signals – create immutable VEX statement store plus consensus extensions with indexes/migrations. |
| Sprint 7 | Contextual Truth Foundations | src/StellaOps.Vexer.WebService/TASKS.md | TODO | Team Vexer WebService | VEXER-WEB-01-004 | Resolve API & signed responses – expose `/vexer/resolve`, return signed consensus/score envelopes, document auth. | | Sprint 7 | Contextual Truth Foundations | src/StellaOps.Vexer.WebService/TASKS.md | TODO | Team Vexer WebService | VEXER-WEB-01-004 | Resolve API & signed responses – expose `/vexer/resolve`, return signed consensus/score envelopes, document auth. |
| Sprint 7 | Contextual Truth Foundations | src/StellaOps.Vexer.Attestation/TASKS.md | TODO | Team Vexer Attestation | VEXER-ATTEST-01-002 | Rekor v2 client integration – ship transparency log client with retries and offline queue. | | Sprint 7 | Contextual Truth Foundations | src/StellaOps.Vexer.Attestation/TASKS.md | DONE (2025-10-16) | Team Vexer Attestation | VEXER-ATTEST-01-002 | Rekor v2 client integration – ship transparency log client with retries and offline queue. |
| Sprint 7 | Contextual Truth Foundations | src/StellaOps.Vexer.Worker/TASKS.md | TODO | Team Vexer Worker | VEXER-WORKER-01-004 | TTL refresh & stability damper – schedule re-resolve loops and guard against status flapping. | | Sprint 7 | Contextual Truth Foundations | src/StellaOps.Vexer.Worker/TASKS.md | TODO | Team Vexer Worker | VEXER-WORKER-01-004 | TTL refresh & stability damper – schedule re-resolve loops and guard against status flapping. |
| Sprint 7 | Contextual Truth Foundations | src/StellaOps.Vexer.Export/TASKS.md | TODO | Team Vexer Export | VEXER-EXPORT-01-005 | Score & resolve envelope surfaces – include signed consensus/score artifacts in exports. | | Sprint 7 | Contextual Truth Foundations | src/StellaOps.Vexer.Export/TASKS.md | TODO | Team Vexer Export | VEXER-EXPORT-01-005 | Score & resolve envelope surfaces – include signed consensus/score artifacts in exports. |
| Sprint 7 | Contextual Truth Foundations | src/StellaOps.Feedser.Core/TASKS.md | TODO | Team Core Engine & Storage Analytics | FEEDCORE-ENGINE-07-001 | Advisory event log & asOf queries – surface immutable statements and replay capability. | | Sprint 7 | Contextual Truth Foundations | src/StellaOps.Feedser.Core/TASKS.md | TODO | Team Core Engine & Storage Analytics | FEEDCORE-ENGINE-07-001 | Advisory event log & asOf queries – surface immutable statements and replay capability. |

52
seed-data/cert-bund/README.md Normal file
View File

@@ -0,0 +1,52 @@
# CERT-Bund Offline Kit Seed Data
This directory stores **offline snapshots** for the CERT-Bund connector.
The artefacts mirror the public JSON search and export endpoints so
air‑gapped deployments can hydrate the connector without contacting the
portal.
> ⚠️ **Distribution notice** – CERT-Bund advisories are published by BSI
> (Federal Office for Information Security, Germany). Review the portal
> terms of use before redistributing the snapshots. Always keep the JSON
> payloads and accompanying SHA-256 sums together.
## Recommended layout
```
seed-data/cert-bund/
├── search/ # paginated search JSON files
│   ├── certbund-search-page-00.json
│   └── …
├── export/ # yearly export JSON files
│   ├── certbund-export-2014.json
│   └── …
├── manifest/
│   └── certbund-offline-manifest.json
└── certbund-offline-manifest.sha256
```
Use `certbund-offline-manifest.json` to feed the Offline Kit build: every
entry contains `source`, `from`, `to`, `sha256`, `capturedAt`, and the
relative file path. The manifest is deterministic when regenerated with
the tooling described below.
## Tooling
Run the helper under `tools/` to capture fresh snapshots or regenerate
the manifest:
```
python tools/certbund_offline_snapshot.py --output seed-data/cert-bund
```
See the connector operations guide
(`docs/ops/feedser-certbund-operations.md`) for detailed usage,
including how to provide cookies/tokens when the portal requires manual
authentication.
## Git hygiene
- JSON payloads and checksums are **ignored by Git**. Generate them
locally when preparing an Offline Kit bundle.
- Commit documentation, scripts, and manifest templates only – never the
exported advisory data itself.

2
src/StellaOps.Feedser.Source.CertBund/TASKS.md
View File

@@ -9,4 +9,4 @@
|FEEDCONN-CERTBUND-02-006 Telemetry & documentation|DevEx|Docs|**DONE (2025-10-15)** – Added `CertBundDiagnostics` (meter `StellaOps.Feedser.Source.CertBund`) with fetch/parse/map counters + histograms, recorded coverage days, wired stage summary logs, and published the ops runbook (`docs/ops/feedser-certbund-operations.md`).| |FEEDCONN-CERTBUND-02-006 Telemetry & documentation|DevEx|Docs|**DONE (2025-10-15)** – Added `CertBundDiagnostics` (meter `StellaOps.Feedser.Source.CertBund`) with fetch/parse/map counters + histograms, recorded coverage days, wired stage summary logs, and published the ops runbook (`docs/ops/feedser-certbund-operations.md`).|
|FEEDCONN-CERTBUND-02-007 Feed history & locale assessment|BE-Conn-CERTBUND|Research|**DONE (2025-10-15)** – Measured RSS retention (~6 days/≈250 items), captured connector-driven backfill guidance in the runbook, and aligned locale guidance (preserve `language=de`, Docs glossary follow-up). **Next:** coordinate with Tools to land the state-seeding helper so scripted backfills replace manual Mongo tweaks.| |FEEDCONN-CERTBUND-02-007 Feed history & locale assessment|BE-Conn-CERTBUND|Research|**DONE (2025-10-15)** – Measured RSS retention (~6 days/≈250 items), captured connector-driven backfill guidance in the runbook, and aligned locale guidance (preserve `language=de`, Docs glossary follow-up). **Next:** coordinate with Tools to land the state-seeding helper so scripted backfills replace manual Mongo tweaks.|
|FEEDCONN-CERTBUND-02-008 Session bootstrap & cookie strategy|BE-Conn-CERTBUND|Source.Common|**DONE (2025-10-14)** – Feed client primes the portal session (cookie container via `SocketsHttpHandler`), shares cookies across detail requests, and documents bootstrap behaviour in options (`PortalBootstrapUri`).| |FEEDCONN-CERTBUND-02-008 Session bootstrap & cookie strategy|BE-Conn-CERTBUND|Source.Common|**DONE (2025-10-14)** – Feed client primes the portal session (cookie container via `SocketsHttpHandler`), shares cookies across detail requests, and documents bootstrap behaviour in options (`PortalBootstrapUri`).|
|FEEDCONN-CERTBUND-02-009 Offline Kit export packaging|BE-Conn-CERTBUND, Docs|Offline Kit|**TODO** – Capture JSON search/export snapshots (per-year splits), generate manifest fields (`source`,`from`,`to`,`sha256`,`capturedAt`), and update Offline Kit docs so air-gapped deployments can seed historical CERT-Bund advisories without live fetching. **Remark:** follow the interim workflow documented in `docs/ops/feedser-certbund-operations.md` §3.3 until the packaged artefacts ship.| |FEEDCONN-CERTBUND-02-009 Offline Kit export packaging|BE-Conn-CERTBUND, Docs|Offline Kit|**DONE (2025-10-17)** – Added `tools/certbund_offline_snapshot.py` to capture search/export JSON, emit deterministic manifests + SHA files, and refreshed docs (`docs/ops/feedser-certbund-operations.md`, `docs/24_OFFLINE_KIT.md`) with offline-kit instructions and manifest layout guidance. Seed data README/ignore rules cover local snapshot hygiene.|

4
src/StellaOps.Vexer.Connectors.Abstractions/AGENTS.md
View File

@@ -5,12 +5,12 @@ Defines shared connector infrastructure for Vexer, including base contexts, resu
- `IVexConnector` context implementation, raw store helpers, verification hooks, and telemetry utilities. - `IVexConnector` context implementation, raw store helpers, verification hooks, and telemetry utilities.
- Configuration primitives (YAML parsing, secrets handling guidelines) and options validation. - Configuration primitives (YAML parsing, secrets handling guidelines) and options validation.
- Connector lifecycle helpers for retries, paging, `.well-known` discovery, and resume markers. - Connector lifecycle helpers for retries, paging, `.well-known` discovery, and resume markers.
- Documentation for connector packaging, plugin manifest metadata, and DI registration. - Documentation for connector packaging, plugin manifest metadata, and DI registration (see `docs/dev/30_VEXER_CONNECTOR_GUIDE.md` and `docs/dev/templates/vexer-connector/`).
## Participants ## Participants
- All Vexer connector projects reference this module to obtain base classes and context services. - All Vexer connector projects reference this module to obtain base classes and context services.
- WebService/Worker instantiate connectors via plugin loader leveraging abstractions defined here. - WebService/Worker instantiate connectors via plugin loader leveraging abstractions defined here.
## Interfaces & contracts ## Interfaces & contracts
- Connector context, result, and telemetry interfaces; `ConnectorDescriptor`, `ConnectorOptions`, authentication helpers. - Connector context, result, and telemetry interfaces; `VexConnectorDescriptor`, `VexConnectorBase`, options binder/validators, authentication helpers.
- Utility classes for HTTP clients, throttling, and deterministic logging. - Utility classes for HTTP clients, throttling, and deterministic logging.
## In/Out of scope ## In/Out of scope
In: shared abstractions, helper utilities, configuration binding, documentation for connector authors. In: shared abstractions, helper utilities, configuration binding, documentation for connector authors.

12
src/StellaOps.Vexer.Connectors.Abstractions/IVexConnectorOptionsValidator.cs Normal file
View File

@@ -0,0 +1,12 @@
using System.Collections.Generic;
namespace StellaOps.Vexer.Connectors.Abstractions;
/// <summary>
/// Custom validator hook executed after connector options are bound.
/// </summary>
/// <typeparam name="TOptions">Connector-specific options type.</typeparam>
public interface IVexConnectorOptionsValidator<in TOptions>
{
void Validate(VexConnectorDescriptor descriptor, TOptions options, IList<string> errors);
}

17
src/StellaOps.Vexer.Connectors.Abstractions/StellaOps.Vexer.Connectors.Abstractions.csproj Normal file
View File

@@ -0,0 +1,17 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>net10.0</TargetFramework>
<LangVersion>preview</LangVersion>
<Nullable>enable</Nullable>
<ImplicitUsings>enable</ImplicitUsings>
<TreatWarningsAsErrors>true</TreatWarningsAsErrors>
</PropertyGroup>
<ItemGroup>
<ProjectReference Include="..\StellaOps.Vexer.Core\StellaOps.Vexer.Core.csproj" />
</ItemGroup>
<ItemGroup>
<PackageReference Include="Microsoft.Extensions.Configuration" Version="8.0.0" />
<PackageReference Include="Microsoft.Extensions.Configuration.Binder" Version="8.0.1" />
<PackageReference Include="Microsoft.Extensions.Logging.Abstractions" Version="8.0.0" />
</ItemGroup>
</Project>

6
src/StellaOps.Vexer.Connectors.Abstractions/TASKS.md
View File

@@ -2,6 +2,6 @@ If you are working on this file you need to read docs/ARCHITECTURE_VEXER.md and
# TASKS # TASKS
| Task | Owner(s) | Depends on | Notes | | Task | Owner(s) | Depends on | Notes |
|---|---|---|---| |---|---|---|---|
|VEXER-CONN-ABS-01-001 – Connector context & base classes|Team Vexer Connectors|VEXER-CORE-01-003|TODO – Implement `VexConnectorContext`, result types, helper base classes, and deterministic logging helpers for connectors.| |VEXER-CONN-ABS-01-001 – Connector context & base classes|Team Vexer Connectors|VEXER-CORE-01-003|**DONE (2025-10-17)** – Added `StellaOps.Vexer.Connectors.Abstractions` project with `VexConnectorBase`, deterministic logging scopes, metadata builder helpers, and connector descriptors; docs updated to highlight the shared abstractions.|
|VEXER-CONN-ABS-01-002 – YAML options & validation|Team Vexer Connectors|VEXER-CONN-ABS-01-001|TODO – Provide strongly-typed options binding/validation for connector YAML definitions with offline-safe defaults.| |VEXER-CONN-ABS-01-002 – YAML options & validation|Team Vexer Connectors|VEXER-CONN-ABS-01-001|**DONE (2025-10-17)** – Delivered `VexConnectorOptionsBinder` + binder options/validators, environment-variable expansion, data-annotation checks, and custom validation hooks with documentation updates covering the workflow.|
|VEXER-CONN-ABS-01-003 – Plugin packaging & docs|Team Vexer Connectors|VEXER-CONN-ABS-01-001|TODO – Document connector packaging (NuGet manifest, plugin loader metadata) and supply reference templates for downstream connector modules.| |VEXER-CONN-ABS-01-003 – Plugin packaging & docs|Team Vexer Connectors|VEXER-CONN-ABS-01-001|**DONE (2025-10-17)** – Authored `docs/dev/30_VEXER_CONNECTOR_GUIDE.md`, added quick-start template under `docs/dev/templates/vexer-connector/`, and updated module docs to reference the packaging workflow.|

99
src/StellaOps.Vexer.Connectors.Abstractions/VexConnectorBase.cs Normal file
View File

@@ -0,0 +1,99 @@
using System.Collections.Immutable;
using System.Security.Cryptography;
using Microsoft.Extensions.Logging;
using StellaOps.Vexer.Core;
namespace StellaOps.Vexer.Connectors.Abstractions;
/// <summary>
/// Convenience base class for implementing <see cref="IVexConnector" />.
/// </summary>
public abstract class VexConnectorBase : IVexConnector
{
protected VexConnectorBase(VexConnectorDescriptor descriptor, ILogger logger, TimeProvider? timeProvider = null)
{
Descriptor = descriptor ?? throw new ArgumentNullException(nameof(descriptor));
Logger = logger ?? throw new ArgumentNullException(nameof(logger));
TimeProvider = timeProvider ?? TimeProvider.System;
}
/// <inheritdoc />
public string Id => Descriptor.Id;
/// <inheritdoc />
public VexProviderKind Kind => Descriptor.Kind;
protected VexConnectorDescriptor Descriptor { get; }
protected ILogger Logger { get; }
protected TimeProvider TimeProvider { get; }
protected DateTimeOffset UtcNow() => TimeProvider.GetUtcNow();
protected VexRawDocument CreateRawDocument(
VexDocumentFormat format,
Uri sourceUri,
ReadOnlyMemory<byte> content,
ImmutableDictionary<string, string>? metadata = null)
{
if (sourceUri is null)
{
throw new ArgumentNullException(nameof(sourceUri));
}
var digest = ComputeSha256(content.Span);
var captured = TimeProvider.GetUtcNow();
return new VexRawDocument(
Descriptor.Id,
format,
sourceUri,
captured,
digest,
content,
metadata ?? ImmutableDictionary<string, string>.Empty);
}
protected IDisposable BeginConnectorScope(string operation, IReadOnlyDictionary<string, object?>? metadata = null)
=> VexConnectorLogScope.Begin(Logger, Descriptor, operation, metadata);
protected void LogConnectorEvent(LogLevel level, string eventName, string message, IReadOnlyDictionary<string, object?>? metadata = null, Exception? exception = null)
{
using var scope = BeginConnectorScope(eventName, metadata);
if (exception is null)
{
Logger.Log(level, "{Message}", message);
}
else
{
Logger.Log(level, exception, "{Message}", message);
}
}
protected ImmutableDictionary<string, string> BuildMetadata(Action<VexConnectorMetadataBuilder> configure)
{
ArgumentNullException.ThrowIfNull(configure);
var builder = new VexConnectorMetadataBuilder();
configure(builder);
return builder.Build();
}
private static string ComputeSha256(ReadOnlySpan<byte> content)
{
Span<byte> buffer = stackalloc byte[32];
if (SHA256.TryHashData(content, buffer, out _))
{
return "sha256:" + Convert.ToHexString(buffer).ToLowerInvariant();
}
using var sha = SHA256.Create();
var hash = sha.ComputeHash(content.ToArray());
return "sha256:" + Convert.ToHexString(hash).ToLowerInvariant();
}
public abstract ValueTask ValidateAsync(VexConnectorSettings settings, CancellationToken cancellationToken);
public abstract IAsyncEnumerable<VexRawDocument> FetchAsync(VexConnectorContext context, CancellationToken cancellationToken);
public abstract ValueTask<VexClaimBatch> NormalizeAsync(VexRawDocument document, CancellationToken cancellationToken);
}

54
src/StellaOps.Vexer.Connectors.Abstractions/VexConnectorDescriptor.cs Normal file
View File

@@ -0,0 +1,54 @@
using System.Collections.Immutable;
using StellaOps.Vexer.Core;
namespace StellaOps.Vexer.Connectors.Abstractions;
/// <summary>
/// Static descriptor for a Vexer connector plug-in.
/// </summary>
public sealed record VexConnectorDescriptor
{
public VexConnectorDescriptor(string id, VexProviderKind kind, string displayName)
{
if (string.IsNullOrWhiteSpace(id))
{
throw new ArgumentException("Connector id must be provided.", nameof(id));
}
Id = id;
Kind = kind;
DisplayName = string.IsNullOrWhiteSpace(displayName) ? id : displayName;
}
/// <summary>
/// Stable connector identifier (matches provider id).
/// </summary>
public string Id { get; }
/// <summary>
/// Provider kind served by the connector.
/// </summary>
public VexProviderKind Kind { get; }
/// <summary>
/// Human friendly name used in logs/diagnostics.
/// </summary>
public string DisplayName { get; }
/// <summary>
/// Optional friendly description.
/// </summary>
public string? Description { get; init; }
/// <summary>
/// Document formats the connector is expected to emit.
/// </summary>
public ImmutableArray<VexDocumentFormat> SupportedFormats { get; init; } = ImmutableArray<VexDocumentFormat>.Empty;
/// <summary>
/// Optional tags surfaced in diagnostics (e.g. "beta", "offline").
/// </summary>
public ImmutableArray<string> Tags { get; init; } = ImmutableArray<string>.Empty;
public override string ToString() => $"{Id} ({Kind})";
}

50
src/StellaOps.Vexer.Connectors.Abstractions/VexConnectorLogScope.cs Normal file
View File

@@ -0,0 +1,50 @@
using System.Linq;
using Microsoft.Extensions.Logging;
using StellaOps.Vexer.Core;
namespace StellaOps.Vexer.Connectors.Abstractions;
/// <summary>
/// Helper to establish deterministic logging scopes for connector operations.
/// </summary>
public static class VexConnectorLogScope
{
public static IDisposable Begin(ILogger logger, VexConnectorDescriptor descriptor, string operation, IReadOnlyDictionary<string, object?>? metadata = null)
{
ArgumentNullException.ThrowIfNull(logger);
ArgumentNullException.ThrowIfNull(descriptor);
ArgumentException.ThrowIfNullOrEmpty(operation);
var scopeValues = new List<KeyValuePair<string, object?>>
{
new("vex.connector.id", descriptor.Id),
new("vex.connector.kind", descriptor.Kind.ToString()),
new("vex.connector.operation", operation),
};
if (!string.Equals(descriptor.DisplayName, descriptor.Id, StringComparison.Ordinal))
{
scopeValues.Add(new KeyValuePair<string, object?>("vex.connector.displayName", descriptor.DisplayName));
}
if (!string.IsNullOrWhiteSpace(descriptor.Description))
{
scopeValues.Add(new KeyValuePair<string, object?>("vex.connector.description", descriptor.Description));
}
if (!descriptor.Tags.IsDefaultOrEmpty)
{
scopeValues.Add(new KeyValuePair<string, object?>("vex.connector.tags", string.Join(",", descriptor.Tags)));
}
if (metadata is not null)
{
foreach (var kvp in metadata.OrderBy(static pair => pair.Key, StringComparer.Ordinal))
{
scopeValues.Add(new KeyValuePair<string, object?>($"vex.{kvp.Key}", kvp.Value));
}
}
return logger.BeginScope(scopeValues)!;
}
}

37
src/StellaOps.Vexer.Connectors.Abstractions/VexConnectorMetadataBuilder.cs Normal file
View File

@@ -0,0 +1,37 @@
using System.Collections.Immutable;
namespace StellaOps.Vexer.Connectors.Abstractions;
/// <summary>
/// Builds deterministic metadata dictionaries for raw documents and logging scopes.
/// </summary>
public sealed class VexConnectorMetadataBuilder
{
private readonly SortedDictionary<string, string> _values = new(StringComparer.Ordinal);
public VexConnectorMetadataBuilder Add(string key, string? value)
{
if (!string.IsNullOrWhiteSpace(key) && !string.IsNullOrWhiteSpace(value))
{
_values[key] = value!;
}
return this;
}
public VexConnectorMetadataBuilder Add(string key, DateTimeOffset value)
=> Add(key, value.ToUniversalTime().ToString("O"));
public VexConnectorMetadataBuilder AddRange(IEnumerable<KeyValuePair<string, string?>> items)
{
foreach (var item in items)
{
Add(item.Key, item.Value);
}
return this;
}
public ImmutableDictionary<string, string> Build()
=> _values.ToImmutableDictionary(static pair => pair.Key, static pair => pair.Value, StringComparer.Ordinal);
}

157
src/StellaOps.Vexer.Connectors.Abstractions/VexConnectorOptionsBinder.cs Normal file
View File

@@ -0,0 +1,157 @@
using System.Collections.Immutable;
using System.ComponentModel.DataAnnotations;
using System.Linq;
using Microsoft.Extensions.Configuration;
using StellaOps.Vexer.Core;
namespace StellaOps.Vexer.Connectors.Abstractions;
/// <summary>
/// Provides strongly typed binding and validation for connector options.
/// </summary>
public static class VexConnectorOptionsBinder
{
public static TOptions Bind<TOptions>(
VexConnectorDescriptor descriptor,
VexConnectorSettings settings,
VexConnectorOptionsBinderOptions? options = null,
IEnumerable<IVexConnectorOptionsValidator<TOptions>>? validators = null)
where TOptions : class, new()
{
ArgumentNullException.ThrowIfNull(descriptor);
ArgumentNullException.ThrowIfNull(settings);
var binderSettings = options ?? new VexConnectorOptionsBinderOptions();
var transformed = TransformValues(settings, binderSettings);
var configuration = BuildConfiguration(transformed);
var result = new TOptions();
var errors = new List<string>();
try
{
configuration.Bind(
result,
binderOptions => binderOptions.ErrorOnUnknownConfiguration = !binderSettings.AllowUnknownKeys);
}
catch (InvalidOperationException ex) when (!binderSettings.AllowUnknownKeys)
{
errors.Add(ex.Message);
}
binderSettings.PostConfigure?.Invoke(result);
if (binderSettings.ValidateDataAnnotations)
{
ValidateDataAnnotations(result, errors);
}
if (validators is not null)
{
foreach (var validator in validators)
{
validator?.Validate(descriptor, result, errors);
}
}
if (errors.Count > 0)
{
throw new VexConnectorOptionsValidationException(descriptor.Id, errors);
}
return result;
}
private static ImmutableDictionary<string, string?> TransformValues(
VexConnectorSettings settings,
VexConnectorOptionsBinderOptions binderOptions)
{
var builder = ImmutableDictionary.CreateBuilder<string, string?>(StringComparer.OrdinalIgnoreCase);
foreach (var kvp in settings.Values)
{
var value = kvp.Value;
if (binderOptions.TrimWhitespace && value is not null)
{
value = value.Trim();
}
if (binderOptions.TreatEmptyAsNull && string.IsNullOrEmpty(value))
{
value = null;
}
if (value is not null && binderOptions.ExpandEnvironmentVariables)
{
value = Environment.ExpandEnvironmentVariables(value);
}
if (binderOptions.ValueTransformer is not null)
{
value = binderOptions.ValueTransformer.Invoke(kvp.Key, value);
}
builder[kvp.Key] = value;
}
return builder.ToImmutable();
}
private static IConfiguration BuildConfiguration(ImmutableDictionary<string, string?> values)
{
var sources = new List<KeyValuePair<string, string?>>();
foreach (var kvp in values)
{
if (kvp.Value is not null)
{
sources.Add(new KeyValuePair<string, string?>(kvp.Key, kvp.Value));
}
}
var configurationBuilder = new ConfigurationBuilder();
configurationBuilder.Add(new DictionaryConfigurationSource(sources));
return configurationBuilder.Build();
}
private static void ValidateDataAnnotations<TOptions>(TOptions options, IList<string> errors)
{
var validationResults = new List<ValidationResult>();
var validationContext = new ValidationContext(options!);
if (!Validator.TryValidateObject(options!, validationContext, validationResults, validateAllProperties: true))
{
foreach (var validationResult in validationResults)
{
if (!string.IsNullOrWhiteSpace(validationResult.ErrorMessage))
{
errors.Add(validationResult.ErrorMessage);
}
}
}
}
private sealed class DictionaryConfigurationSource : IConfigurationSource
{
private readonly IReadOnlyList<KeyValuePair<string, string?>> _data;
public DictionaryConfigurationSource(IEnumerable<KeyValuePair<string, string?>> data)
{
_data = data?.ToList() ?? new List<KeyValuePair<string, string?>>();
}
public IConfigurationProvider Build(IConfigurationBuilder builder) => new DictionaryConfigurationProvider(_data);
}
private sealed class DictionaryConfigurationProvider : ConfigurationProvider
{
public DictionaryConfigurationProvider(IEnumerable<KeyValuePair<string, string?>> data)
{
foreach (var pair in data)
{
if (pair.Value is not null)
{
Data[pair.Key] = pair.Value;
}
}
}
}
}

45
src/StellaOps.Vexer.Connectors.Abstractions/VexConnectorOptionsBinderOptions.cs Normal file
View File

@@ -0,0 +1,45 @@
namespace StellaOps.Vexer.Connectors.Abstractions;
/// <summary>
/// Customisation options for connector options binding.
/// </summary>
public sealed class VexConnectorOptionsBinderOptions
{
/// <summary>
/// Indicates whether environment variables should be expanded in option values.
/// Defaults to <c>true</c>.
/// </summary>
public bool ExpandEnvironmentVariables { get; set; } = true;
/// <summary>
/// When <c>true</c> the binder trims whitespace around option values.
/// </summary>
public bool TrimWhitespace { get; set; } = true;
/// <summary>
/// Converts empty strings to <c>null</c> before binding. Default: <c>true</c>.
/// </summary>
public bool TreatEmptyAsNull { get; set; } = true;
/// <summary>
/// When <c>false</c>, binding fails if unknown configuration keys are provided.
/// Default: <c>true</c> (permitting unknown keys).
/// </summary>
public bool AllowUnknownKeys { get; set; } = true;
/// <summary>
/// Enables <see cref="System.ComponentModel.DataAnnotations"/> validation after binding.
/// Default: <c>true</c>.
/// </summary>
public bool ValidateDataAnnotations { get; set; } = true;
/// <summary>
/// Optional post-configuration callback executed after binding.
/// </summary>
public Action<object>? PostConfigure { get; set; }
/// <summary>
/// Optional hook to transform raw configuration values before binding.
/// </summary>
public Func<string, string?, string?>? ValueTransformer { get; set; }
}

36
src/StellaOps.Vexer.Connectors.Abstractions/VexConnectorOptionsValidationException.cs Normal file
View File

@@ -0,0 +1,36 @@
using System.Collections.Immutable;
namespace StellaOps.Vexer.Connectors.Abstractions;
public sealed class VexConnectorOptionsValidationException : Exception
{
public VexConnectorOptionsValidationException(
string connectorId,
IEnumerable<string> errors)
: base(BuildMessage(connectorId, errors))
{
ConnectorId = connectorId;
Errors = errors?.ToImmutableArray() ?? ImmutableArray<string>.Empty;
}
public string ConnectorId { get; }
public ImmutableArray<string> Errors { get; }
private static string BuildMessage(string connectorId, IEnumerable<string> errors)
{
var builder = new System.Text.StringBuilder();
builder.Append("Connector options validation failed for '");
builder.Append(connectorId);
builder.Append("'.");
var list = errors?.ToImmutableArray() ?? ImmutableArray<string>.Empty;
if (!list.IsDefaultOrEmpty)
{
builder.Append(" Errors: ");
builder.Append(string.Join("; ", list));
}
return builder.ToString();
}
}

214
src/StellaOps.Vexer.Connectors.Cisco.CSAF.Tests/Connectors/CiscoCsafConnectorTests.cs Normal file
View File

@@ -0,0 +1,214 @@
using System.Collections.Generic;
using System.Net;
using System.Net.Http;
using System.Text;
using FluentAssertions;
using Microsoft.Extensions.Caching.Memory;
using Microsoft.Extensions.Logging.Abstractions;
using Microsoft.Extensions.Options;
using Microsoft.Extensions.DependencyInjection;
using StellaOps.Vexer.Connectors.Abstractions;
using StellaOps.Vexer.Connectors.Cisco.CSAF;
using StellaOps.Vexer.Connectors.Cisco.CSAF.Configuration;
using StellaOps.Vexer.Connectors.Cisco.CSAF.Metadata;
using StellaOps.Vexer.Core;
using StellaOps.Vexer.Storage.Mongo;
using System.Collections.Immutable;
using System.IO.Abstractions.TestingHelpers;
using Xunit;
using System.Threading;
namespace StellaOps.Vexer.Connectors.Cisco.CSAF.Tests.Connectors;
public sealed class CiscoCsafConnectorTests
{
[Fact]
public async Task FetchAsync_NewAdvisory_StoresDocumentAndUpdatesState()
{
var responses = new Dictionary<Uri, Queue<HttpResponseMessage>>
{
[new Uri("https://api.cisco.test/.well-known/csaf/provider-metadata.json")] = QueueResponses("""
{
"metadata": {
"publisher": {
"name": "Cisco",
"category": "vendor",
"contact_details": { "id": "vexer:cisco" }
}
},
"distributions": {
"directories": [ "https://api.cisco.test/csaf/" ]
}
}
"""),
[new Uri("https://api.cisco.test/csaf/index.json")] = QueueResponses("""
{
"advisories": [
{
"id": "cisco-sa-2025",
"url": "https://api.cisco.test/csaf/cisco-sa-2025.json",
"published": "2025-10-01T00:00:00Z",
"lastModified": "2025-10-02T00:00:00Z",
"sha256": "cafebabe"
}
]
}
"""),
[new Uri("https://api.cisco.test/csaf/cisco-sa-2025.json")] = QueueResponses("{ \"document\": \"payload\" }")
};
var handler = new RoutingHttpMessageHandler(responses);
var httpClient = new HttpClient(handler);
var factory = new SingleHttpClientFactory(httpClient);
var metadataLoader = new CiscoProviderMetadataLoader(
factory,
new MemoryCache(new MemoryCacheOptions()),
Options.Create(new CiscoConnectorOptions
{
MetadataUri = "https://api.cisco.test/.well-known/csaf/provider-metadata.json",
PersistOfflineSnapshot = false,
}),
NullLogger<CiscoProviderMetadataLoader>.Instance,
new MockFileSystem());
var stateRepository = new InMemoryConnectorStateRepository();
var connector = new CiscoCsafConnector(
metadataLoader,
factory,
stateRepository,
new[] { new CiscoConnectorOptionsValidator() },
NullLogger<CiscoCsafConnector>.Instance,
TimeProvider.System);
var settings = new VexConnectorSettings(ImmutableDictionary<string, string>.Empty);
await connector.ValidateAsync(settings, CancellationToken.None);
var sink = new InMemoryRawSink();
var context = new VexConnectorContext(null, VexConnectorSettings.Empty, sink, new NoopSignatureVerifier(), new NoopNormalizerRouter(), new ServiceCollection().BuildServiceProvider());
var documents = new List<VexRawDocument>();
await foreach (var doc in connector.FetchAsync(context, CancellationToken.None))
{
documents.Add(doc);
}
documents.Should().HaveCount(1);
sink.Documents.Should().HaveCount(1);
stateRepository.CurrentState.Should().NotBeNull();
stateRepository.CurrentState!.DocumentDigests.Should().HaveCount(1);
// second run should not refetch documents
sink.Documents.Clear();
documents.Clear();
await foreach (var doc in connector.FetchAsync(context, CancellationToken.None))
{
documents.Add(doc);
}
documents.Should().BeEmpty();
sink.Documents.Should().BeEmpty();
}
private static Queue<HttpResponseMessage> QueueResponses(string payload)
=> new(new[]
{
new HttpResponseMessage(HttpStatusCode.OK)
{
Content = new StringContent(payload, Encoding.UTF8, "application/json"),
}
});
private sealed class RoutingHttpMessageHandler : HttpMessageHandler
{
private readonly Dictionary<Uri, Queue<HttpResponseMessage>> _responses;
public RoutingHttpMessageHandler(Dictionary<Uri, Queue<HttpResponseMessage>> responses)
{
_responses = responses;
}
protected override Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
{
if (request.RequestUri is not null && _responses.TryGetValue(request.RequestUri, out var queue) && queue.Count > 0)
{
var response = queue.Peek();
return Task.FromResult(response.Clone());
}
return Task.FromResult(new HttpResponseMessage(HttpStatusCode.NotFound)
{
Content = new StringContent($"No response configured for {request.RequestUri}"),
});
}
}
private sealed class SingleHttpClientFactory : IHttpClientFactory
{
private readonly HttpClient _client;
public SingleHttpClientFactory(HttpClient client)
{
_client = client;
}
public HttpClient CreateClient(string name) => _client;
}
private sealed class InMemoryConnectorStateRepository : IVexConnectorStateRepository
{
public VexConnectorState? CurrentState { get; private set; }
public ValueTask<VexConnectorState?> GetAsync(string connectorId, CancellationToken cancellationToken)
=> ValueTask.FromResult(CurrentState);
public ValueTask SaveAsync(VexConnectorState state, CancellationToken cancellationToken)
{
CurrentState = state;
return ValueTask.CompletedTask;
}
}
private sealed class InMemoryRawSink : IVexRawDocumentSink
{
public List<VexRawDocument> Documents { get; } = new();
public ValueTask StoreAsync(VexRawDocument document, CancellationToken cancellationToken)
{
Documents.Add(document);
return ValueTask.CompletedTask;
}
}
private sealed class NoopSignatureVerifier : IVexSignatureVerifier
{
public ValueTask<VexSignatureMetadata?> VerifyAsync(VexRawDocument document, CancellationToken cancellationToken)
=> ValueTask.FromResult<VexSignatureMetadata?>(null);
}
private sealed class NoopNormalizerRouter : IVexNormalizerRouter
{
public ValueTask<VexClaimBatch> NormalizeAsync(VexRawDocument document, CancellationToken cancellationToken)
=> ValueTask.FromResult(new VexClaimBatch(document, ImmutableArray<VexClaim>.Empty, ImmutableDictionary<string, string>.Empty));
}
}
internal static class HttpResponseMessageExtensions
{
public static HttpResponseMessage Clone(this HttpResponseMessage response)
{
var clone = new HttpResponseMessage(response.StatusCode);
foreach (var header in response.Headers)
{
clone.Headers.TryAddWithoutValidation(header.Key, header.Value);
}
if (response.Content is not null)
{
var payload = response.Content.ReadAsStringAsync().GetAwaiter().GetResult();
clone.Content = new StringContent(payload, Encoding.UTF8, response.Content.Headers.ContentType?.MediaType);
}
return clone;
}
}

149
src/StellaOps.Vexer.Connectors.Cisco.CSAF.Tests/Metadata/CiscoProviderMetadataLoaderTests.cs Normal file
View File

@@ -0,0 +1,149 @@
using System.Net;
using System.Net.Http;
using System.Text;
using FluentAssertions;
using Microsoft.Extensions.Caching.Memory;
using Microsoft.Extensions.Logging.Abstractions;
using Microsoft.Extensions.Options;
using StellaOps.Vexer.Connectors.Cisco.CSAF.Configuration;
using StellaOps.Vexer.Connectors.Cisco.CSAF.Metadata;
using StellaOps.Vexer.Core;
using System.IO.Abstractions.TestingHelpers;
namespace StellaOps.Vexer.Connectors.Cisco.CSAF.Tests.Metadata;
public sealed class CiscoProviderMetadataLoaderTests
{
[Fact]
public async Task LoadAsync_FetchesFromNetworkWithBearerToken()
{
var payload = """
{
"metadata": {
"publisher": {
"name": "Cisco CSAF",
"category": "vendor",
"contact_details": {
"id": "vexer:cisco"
}
}
},
"distributions": {
"directories": [
"https://api.security.cisco.com/csaf/v2/advisories/"
]
},
"discovery": {
"well_known": "https://api.security.cisco.com/.well-known/csaf/provider-metadata.json",
"rolie": "https://api.security.cisco.com/csaf/rolie/feed"
},
"trust": {
"weight": 0.9,
"cosign": {
"issuer": "https://oidc.security.cisco.com",
"identity_pattern": "spiffe://cisco/*"
},
"pgp_fingerprints": [ "1234ABCD" ]
}
}
""";
HttpRequestMessage? capturedRequest = null;
var handler = new FakeHttpMessageHandler(request =>
{
capturedRequest = request;
return new HttpResponseMessage(HttpStatusCode.OK)
{
Content = new StringContent(payload, Encoding.UTF8, "application/json"),
Headers = { ETag = new System.Net.Http.Headers.EntityTagHeaderValue("\"etag1\"") }
};
});
var httpClient = new HttpClient(handler);
var factory = new SingleHttpClientFactory(httpClient);
var cache = new MemoryCache(new MemoryCacheOptions());
var options = Options.Create(new CiscoConnectorOptions
{
ApiToken = "token-123",
MetadataUri = "https://api.security.cisco.com/.well-known/csaf/provider-metadata.json",
});
var fileSystem = new MockFileSystem();
var loader = new CiscoProviderMetadataLoader(factory, cache, options, NullLogger<CiscoProviderMetadataLoader>.Instance, fileSystem);
var result = await loader.LoadAsync(CancellationToken.None);
result.Provider.Id.Should().Be("vexer:cisco");
result.Provider.BaseUris.Should().ContainSingle(uri => uri.ToString() == "https://api.security.cisco.com/csaf/v2/advisories/");
result.Provider.Discovery.RolIeService.Should().Be(new Uri("https://api.security.cisco.com/csaf/rolie/feed"));
result.ServedFromCache.Should().BeFalse();
capturedRequest.Should().NotBeNull();
capturedRequest!.Headers.Authorization.Should().NotBeNull();
capturedRequest.Headers.Authorization!.Parameter.Should().Be("token-123");
}
[Fact]
public async Task LoadAsync_FallsBackToOfflineSnapshot()
{
var payload = """
{
"metadata": {
"publisher": {
"name": "Cisco CSAF",
"category": "vendor",
"contact_details": {
"id": "vexer:cisco"
}
}
}
}
""";
var handler = new FakeHttpMessageHandler(_ => new HttpResponseMessage(HttpStatusCode.InternalServerError));
var httpClient = new HttpClient(handler);
var factory = new SingleHttpClientFactory(httpClient);
var cache = new MemoryCache(new MemoryCacheOptions());
var options = Options.Create(new CiscoConnectorOptions
{
MetadataUri = "https://api.security.cisco.com/.well-known/csaf/provider-metadata.json",
PreferOfflineSnapshot = true,
OfflineSnapshotPath = "/snapshots/cisco.json",
});
var fileSystem = new MockFileSystem(new Dictionary<string, MockFileData>
{
["/snapshots/cisco.json"] = new MockFileData(payload),
});
var loader = new CiscoProviderMetadataLoader(factory, cache, options, NullLogger<CiscoProviderMetadataLoader>.Instance, fileSystem);
var result = await loader.LoadAsync(CancellationToken.None);
result.FromOfflineSnapshot.Should().BeTrue();
result.Provider.Id.Should().Be("vexer:cisco");
}
private sealed class SingleHttpClientFactory : IHttpClientFactory
{
private readonly HttpClient _client;
public SingleHttpClientFactory(HttpClient client)
{
_client = client;
}
public HttpClient CreateClient(string name) => _client;
}
private sealed class FakeHttpMessageHandler : HttpMessageHandler
{
private readonly Func<HttpRequestMessage, HttpResponseMessage> _responder;
public FakeHttpMessageHandler(Func<HttpRequestMessage, HttpResponseMessage> responder)
{
_responder = responder;
}
protected override Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
{
return Task.FromResult(_responder(request));
}
}
}

16
src/StellaOps.Vexer.Connectors.Cisco.CSAF.Tests/StellaOps.Vexer.Connectors.Cisco.CSAF.Tests.csproj Normal file
View File

@@ -0,0 +1,16 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>net10.0</TargetFramework>
<LangVersion>preview</LangVersion>
<Nullable>enable</Nullable>
<ImplicitUsings>enable</ImplicitUsings>
<TreatWarningsAsErrors>true</TreatWarningsAsErrors>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="FluentAssertions" Version="6.12.0" />
<PackageReference Include="System.IO.Abstractions.TestingHelpers" Version="20.0.28" />
</ItemGroup>
<ItemGroup>
<ProjectReference Include="..\StellaOps.Vexer.Connectors.Cisco.CSAF\StellaOps.Vexer.Connectors.Cisco.CSAF.csproj" />
</ItemGroup>
</Project>

247
src/StellaOps.Vexer.Connectors.Cisco.CSAF/CiscoCsafConnector.cs Normal file
View File

@@ -0,0 +1,247 @@
using System.Collections.Generic;
using System.Collections.Immutable;
using System.Linq;
using System.Net.Http;
using System.Runtime.CompilerServices;
using System.Text.Json;
using Microsoft.Extensions.Logging;
using StellaOps.Vexer.Connectors.Abstractions;
using StellaOps.Vexer.Connectors.Cisco.CSAF.Configuration;
using StellaOps.Vexer.Connectors.Cisco.CSAF.Metadata;
using StellaOps.Vexer.Core;
using StellaOps.Vexer.Storage.Mongo;
namespace StellaOps.Vexer.Connectors.Cisco.CSAF;
public sealed class CiscoCsafConnector : VexConnectorBase
{
private static readonly VexConnectorDescriptor DescriptorInstance = new(
id: "vexer:cisco",
kind: VexProviderKind.Vendor,
displayName: "Cisco CSAF")
{
Tags = ImmutableArray.Create("cisco", "csaf"),
};
private readonly CiscoProviderMetadataLoader _metadataLoader;
private readonly IHttpClientFactory _httpClientFactory;
private readonly IVexConnectorStateRepository _stateRepository;
private readonly IEnumerable<IVexConnectorOptionsValidator<CiscoConnectorOptions>> _validators;
private readonly JsonSerializerOptions _serializerOptions = new(JsonSerializerDefaults.Web);
private CiscoConnectorOptions? _options;
private CiscoProviderMetadataResult? _providerMetadata;
public CiscoCsafConnector(
CiscoProviderMetadataLoader metadataLoader,
IHttpClientFactory httpClientFactory,
IVexConnectorStateRepository stateRepository,
IEnumerable<IVexConnectorOptionsValidator<CiscoConnectorOptions>>? validators,
ILogger<CiscoCsafConnector> logger,
TimeProvider timeProvider)
: base(DescriptorInstance, logger, timeProvider)
{
_metadataLoader = metadataLoader ?? throw new ArgumentNullException(nameof(metadataLoader));
_httpClientFactory = httpClientFactory ?? throw new ArgumentNullException(nameof(httpClientFactory));
_stateRepository = stateRepository ?? throw new ArgumentNullException(nameof(stateRepository));
_validators = validators ?? Array.Empty<IVexConnectorOptionsValidator<CiscoConnectorOptions>>();
}
public override async ValueTask ValidateAsync(VexConnectorSettings settings, CancellationToken cancellationToken)
{
_options = VexConnectorOptionsBinder.Bind(
Descriptor,
settings,
validators: _validators);
_providerMetadata = await _metadataLoader.LoadAsync(cancellationToken).ConfigureAwait(false);
LogConnectorEvent(LogLevel.Information, "validate", "Cisco CSAF metadata loaded.", new Dictionary<string, object?>
{
["baseUriCount"] = _providerMetadata.Provider.BaseUris.Length,
["fromOffline"] = _providerMetadata.FromOfflineSnapshot,
});
}
public override async IAsyncEnumerable<VexRawDocument> FetchAsync(VexConnectorContext context, [EnumeratorCancellation] CancellationToken cancellationToken)
{
ArgumentNullException.ThrowIfNull(context);
if (_options is null)
{
throw new InvalidOperationException("Connector must be validated before fetch operations.");
}
if (_providerMetadata is null)
{
_providerMetadata = await _metadataLoader.LoadAsync(cancellationToken).ConfigureAwait(false);
}
var state = await _stateRepository.GetAsync(Descriptor.Id, cancellationToken).ConfigureAwait(false);
var knownDigests = state?.DocumentDigests ?? ImmutableArray<string>.Empty;
var digestSet = new HashSet<string>(knownDigests, StringComparer.OrdinalIgnoreCase);
var digestList = new List<string>(knownDigests);
var since = context.Since ?? state?.LastUpdated ?? DateTimeOffset.MinValue;
var latestTimestamp = state?.LastUpdated ?? since;
var stateChanged = false;
var client = _httpClientFactory.CreateClient(CiscoConnectorOptions.HttpClientName);
foreach (var directory in _providerMetadata.Provider.BaseUris)
{
await foreach (var advisory in EnumerateCatalogAsync(client, directory, cancellationToken).ConfigureAwait(false))
{
var published = advisory.LastModified ?? advisory.Published ?? DateTimeOffset.MinValue;
if (published <= since)
{
continue;
}
using var contentResponse = await client.GetAsync(advisory.DocumentUri, HttpCompletionOption.ResponseHeadersRead, cancellationToken).ConfigureAwait(false);
contentResponse.EnsureSuccessStatusCode();
var payload = await contentResponse.Content.ReadAsByteArrayAsync(cancellationToken).ConfigureAwait(false);
var rawDocument = CreateRawDocument(
VexDocumentFormat.Csaf,
advisory.DocumentUri,
payload,
BuildMetadata(builder => builder
.Add("cisco.csaf.advisoryId", advisory.Id)
.Add("cisco.csaf.revision", advisory.Revision)
.Add("cisco.csaf.published", advisory.Published?.ToString("O"))
.Add("cisco.csaf.modified", advisory.LastModified?.ToString("O"))
.Add("cisco.csaf.sha256", advisory.Sha256)));
if (!digestSet.Add(rawDocument.Digest))
{
continue;
}
await context.RawSink.StoreAsync(rawDocument, cancellationToken).ConfigureAwait(false);
digestList.Add(rawDocument.Digest);
stateChanged = true;
if (published > latestTimestamp)
{
latestTimestamp = published;
}
yield return rawDocument;
}
}
if (stateChanged)
{
var newState = new VexConnectorState(
Descriptor.Id,
latestTimestamp == DateTimeOffset.MinValue ? state?.LastUpdated : latestTimestamp,
digestList.ToImmutableArray());
await _stateRepository.SaveAsync(newState, cancellationToken).ConfigureAwait(false);
}
}
public override ValueTask<VexClaimBatch> NormalizeAsync(VexRawDocument document, CancellationToken cancellationToken)
=> throw new NotSupportedException("CiscoCsafConnector relies on CSAF normalizers for document processing.");
private async IAsyncEnumerable<CiscoAdvisoryEntry> EnumerateCatalogAsync(HttpClient client, Uri directory, [EnumeratorCancellation] CancellationToken cancellationToken)
{
var nextUri = BuildIndexUri(directory, null);
while (nextUri is not null)
{
using var response = await client.GetAsync(nextUri, HttpCompletionOption.ResponseHeadersRead, cancellationToken).ConfigureAwait(false);
response.EnsureSuccessStatusCode();
var json = await response.Content.ReadAsStringAsync(cancellationToken).ConfigureAwait(false);
var page = JsonSerializer.Deserialize<CiscoAdvisoryIndex>(json, _serializerOptions);
if (page?.Advisories is null)
{
yield break;
}
foreach (var advisory in page.Advisories)
{
if (string.IsNullOrWhiteSpace(advisory.Url))
{
continue;
}
if (!Uri.TryCreate(advisory.Url, UriKind.RelativeOrAbsolute, out var documentUri))
{
continue;
}
if (!documentUri.IsAbsoluteUri)
{
documentUri = new Uri(directory, documentUri);
}
yield return new CiscoAdvisoryEntry(
advisory.Id ?? documentUri.Segments.LastOrDefault()?.Trim('/') ?? documentUri.ToString(),
documentUri,
advisory.Revision,
advisory.Published,
advisory.LastModified,
advisory.Sha256);
}
nextUri = ResolveNextUri(directory, page.Next);
}
}
private static Uri BuildIndexUri(Uri directory, string? relative)
{
if (string.IsNullOrWhiteSpace(relative))
{
var baseText = directory.ToString();
if (!baseText.EndsWith('/'))
{
baseText += "/";
}
return new Uri(new Uri(baseText, UriKind.Absolute), "index.json");
}
if (Uri.TryCreate(relative, UriKind.Absolute, out var absolute))
{
return absolute;
}
var baseTextRelative = directory.ToString();
if (!baseTextRelative.EndsWith('/'))
{
baseTextRelative += "/";
}
return new Uri(new Uri(baseTextRelative, UriKind.Absolute), relative);
}
private static Uri? ResolveNextUri(Uri directory, string? next)
{
if (string.IsNullOrWhiteSpace(next))
{
return null;
}
return BuildIndexUri(directory, next);
}
private sealed record CiscoAdvisoryIndex
{
public List<CiscoAdvisory>? Advisories { get; init; }
public string? Next { get; init; }
}
private sealed record CiscoAdvisory
{
public string? Id { get; init; }
public string? Url { get; init; }
public string? Revision { get; init; }
public DateTimeOffset? Published { get; init; }
public DateTimeOffset? LastModified { get; init; }
public string? Sha256 { get; init; }
}
private sealed record CiscoAdvisoryEntry(
string Id,
Uri DocumentUri,
string? Revision,
DateTimeOffset? Published,
DateTimeOffset? LastModified,
string? Sha256);
}

58
src/StellaOps.Vexer.Connectors.Cisco.CSAF/Configuration/CiscoConnectorOptions.cs Normal file
View File

@@ -0,0 +1,58 @@
using System.ComponentModel.DataAnnotations;
namespace StellaOps.Vexer.Connectors.Cisco.CSAF.Configuration;
public sealed class CiscoConnectorOptions : IValidatableObject
{
public const string HttpClientName = "cisco-csaf";
/// <summary>
/// Endpoint for Cisco CSAF provider metadata discovery.
/// </summary>
[Required]
public string MetadataUri { get; set; } = "https://api.security.cisco.com/.well-known/csaf/provider-metadata.json";
/// <summary>
/// Optional bearer token used when Cisco endpoints require authentication.
/// </summary>
public string? ApiToken { get; set; }
/// <summary>
/// How long provider metadata remains cached.
/// </summary>
public TimeSpan MetadataCacheDuration { get; set; } = TimeSpan.FromHours(6);
/// <summary>
/// Whether to prefer offline snapshots when fetching metadata.
/// </summary>
public bool PreferOfflineSnapshot { get; set; }
/// <summary>
/// When set, provider metadata will be persisted to the given file path.
/// </summary>
public bool PersistOfflineSnapshot { get; set; }
public string? OfflineSnapshotPath { get; set; }
public IEnumerable<ValidationResult> Validate(ValidationContext validationContext)
{
if (string.IsNullOrWhiteSpace(MetadataUri))
{
yield return new ValidationResult("MetadataUri must be provided.", new[] { nameof(MetadataUri) });
}
else if (!Uri.TryCreate(MetadataUri, UriKind.Absolute, out _))
{
yield return new ValidationResult("MetadataUri must be an absolute URI.", new[] { nameof(MetadataUri) });
}
if (MetadataCacheDuration <= TimeSpan.Zero)
{
yield return new ValidationResult("MetadataCacheDuration must be greater than zero.", new[] { nameof(MetadataCacheDuration) });
}
if (PersistOfflineSnapshot && string.IsNullOrWhiteSpace(OfflineSnapshotPath))
{
yield return new ValidationResult("OfflineSnapshotPath must be provided when PersistOfflineSnapshot is enabled.", new[] { nameof(OfflineSnapshotPath) });
}
}
}

25
src/StellaOps.Vexer.Connectors.Cisco.CSAF/Configuration/CiscoConnectorOptionsValidator.cs Normal file
View File

@@ -0,0 +1,25 @@
using System;
using System.Collections.Generic;
using System.ComponentModel.DataAnnotations;
using StellaOps.Vexer.Connectors.Abstractions;
namespace StellaOps.Vexer.Connectors.Cisco.CSAF.Configuration;
public sealed class CiscoConnectorOptionsValidator : IVexConnectorOptionsValidator<CiscoConnectorOptions>
{
public void Validate(VexConnectorDescriptor descriptor, CiscoConnectorOptions options, IList<string> errors)
{
ArgumentNullException.ThrowIfNull(descriptor);
ArgumentNullException.ThrowIfNull(options);
ArgumentNullException.ThrowIfNull(errors);
var validationResults = new List<ValidationResult>();
if (!Validator.TryValidateObject(options, new ValidationContext(options), validationResults, validateAllProperties: true))
{
foreach (var result in validationResults)
{
errors.Add(result.ErrorMessage ?? "Cisco connector options validation failed.");
}
}
}
}

52
src/StellaOps.Vexer.Connectors.Cisco.CSAF/DependencyInjection/CiscoConnectorServiceCollectionExtensions.cs Normal file
View File

@@ -0,0 +1,52 @@
using System.ComponentModel.DataAnnotations;
using System.Net.Http.Headers;
using Microsoft.Extensions.Caching.Memory;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.DependencyInjection.Extensions;
using Microsoft.Extensions.Options;
using StellaOps.Vexer.Connectors.Cisco.CSAF.Configuration;
using StellaOps.Vexer.Connectors.Cisco.CSAF.Metadata;
using StellaOps.Vexer.Connectors.Abstractions;
using StellaOps.Vexer.Core;
using System.IO.Abstractions;
namespace StellaOps.Vexer.Connectors.Cisco.CSAF.DependencyInjection;
public static class CiscoConnectorServiceCollectionExtensions
{
public static IServiceCollection AddCiscoCsafConnector(this IServiceCollection services, Action<CiscoConnectorOptions>? configure = null)
{
ArgumentNullException.ThrowIfNull(services);
services.AddOptions<CiscoConnectorOptions>()
.Configure(options =>
{
configure?.Invoke(options);
})
.PostConfigure(options =>
{
Validator.ValidateObject(options, new ValidationContext(options), validateAllProperties: true);
});
services.TryAddSingleton<IMemoryCache, MemoryCache>();
services.TryAddSingleton<IFileSystem, FileSystem>();
services.TryAddEnumerable(ServiceDescriptor.Singleton<IVexConnectorOptionsValidator<CiscoConnectorOptions>, CiscoConnectorOptionsValidator>());
services.AddHttpClient(CiscoConnectorOptions.HttpClientName)
.ConfigureHttpClient((provider, client) =>
{
var options = provider.GetRequiredService<IOptions<CiscoConnectorOptions>>().Value;
client.Timeout = TimeSpan.FromSeconds(30);
client.DefaultRequestHeaders.Accept.ParseAdd("application/json");
if (!string.IsNullOrWhiteSpace(options.ApiToken))
{
client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer", options.ApiToken);
}
});
services.AddSingleton<CiscoProviderMetadataLoader>();
services.AddSingleton<IVexConnector, CiscoCsafConnector>();
return services;
}
}

332
src/StellaOps.Vexer.Connectors.Cisco.CSAF/Metadata/CiscoProviderMetadataLoader.cs Normal file
View File

@@ -0,0 +1,332 @@
using System.Collections.Immutable;
using System.Net;
using System.Net.Http.Headers;
using System.Text.Json;
using Microsoft.Extensions.Caching.Memory;
using Microsoft.Extensions.Logging;
using Microsoft.Extensions.Options;
using StellaOps.Vexer.Connectors.Cisco.CSAF.Configuration;
using StellaOps.Vexer.Core;
using System.IO.Abstractions;
namespace StellaOps.Vexer.Connectors.Cisco.CSAF.Metadata;
public sealed class CiscoProviderMetadataLoader
{
public const string CacheKey = "StellaOps.Vexer.Connectors.Cisco.CSAF.Metadata";
private readonly IHttpClientFactory _httpClientFactory;
private readonly IMemoryCache _memoryCache;
private readonly ILogger<CiscoProviderMetadataLoader> _logger;
private readonly CiscoConnectorOptions _options;
private readonly IFileSystem _fileSystem;
private readonly JsonSerializerOptions _serializerOptions;
private readonly SemaphoreSlim _semaphore = new(1, 1);
public CiscoProviderMetadataLoader(
IHttpClientFactory httpClientFactory,
IMemoryCache memoryCache,
IOptions<CiscoConnectorOptions> options,
ILogger<CiscoProviderMetadataLoader> logger,
IFileSystem? fileSystem = null)
{
_httpClientFactory = httpClientFactory ?? throw new ArgumentNullException(nameof(httpClientFactory));
_memoryCache = memoryCache ?? throw new ArgumentNullException(nameof(memoryCache));
_logger = logger ?? throw new ArgumentNullException(nameof(logger));
ArgumentNullException.ThrowIfNull(options);
_options = options.Value ?? throw new ArgumentNullException(nameof(options));
_fileSystem = fileSystem ?? new FileSystem();
_serializerOptions = new JsonSerializerOptions(JsonSerializerDefaults.Web)
{
PropertyNameCaseInsensitive = true,
ReadCommentHandling = JsonCommentHandling.Skip,
};
}
public async Task<CiscoProviderMetadataResult> LoadAsync(CancellationToken cancellationToken)
{
if (_memoryCache.TryGetValue<CacheEntry>(CacheKey, out var cached) && cached is not null && !cached.IsExpired())
{
_logger.LogDebug("Returning cached Cisco provider metadata (expires {Expires}).", cached.ExpiresAt);
return new CiscoProviderMetadataResult(cached.Provider, cached.FetchedAt, cached.FromOffline, true);
}
await _semaphore.WaitAsync(cancellationToken).ConfigureAwait(false);
try
{
if (_memoryCache.TryGetValue<CacheEntry>(CacheKey, out cached) && cached is not null && !cached.IsExpired())
{
return new CiscoProviderMetadataResult(cached.Provider, cached.FetchedAt, cached.FromOffline, true);
}
CacheEntry? previous = cached;
if (!_options.PreferOfflineSnapshot)
{
var network = await TryFetchFromNetworkAsync(previous, cancellationToken).ConfigureAwait(false);
if (network is not null)
{
StoreCache(network);
return new CiscoProviderMetadataResult(network.Provider, network.FetchedAt, false, false);
}
}
var offline = TryLoadFromOffline();
if (offline is not null)
{
var entry = offline with
{
FetchedAt = DateTimeOffset.UtcNow,
ExpiresAt = DateTimeOffset.UtcNow + _options.MetadataCacheDuration,
FromOffline = true,
};
StoreCache(entry);
return new CiscoProviderMetadataResult(entry.Provider, entry.FetchedAt, true, false);
}
throw new InvalidOperationException("Unable to load Cisco CSAF provider metadata from network or offline snapshot.");
}
finally
{
_semaphore.Release();
}
}
private async Task<CacheEntry?> TryFetchFromNetworkAsync(CacheEntry? previous, CancellationToken cancellationToken)
{
try
{
var client = _httpClientFactory.CreateClient(CiscoConnectorOptions.HttpClientName);
using var request = new HttpRequestMessage(HttpMethod.Get, _options.MetadataUri);
if (!string.IsNullOrWhiteSpace(_options.ApiToken))
{
request.Headers.Authorization = new AuthenticationHeaderValue("Bearer", _options.ApiToken);
}
if (!string.IsNullOrWhiteSpace(previous?.ETag) && EntityTagHeaderValue.TryParse(previous.ETag, out var etag))
{
request.Headers.IfNoneMatch.Add(etag);
}
using var response = await client.SendAsync(request, HttpCompletionOption.ResponseHeadersRead, cancellationToken).ConfigureAwait(false);
if (response.StatusCode == HttpStatusCode.NotModified && previous is not null)
{
_logger.LogDebug("Cisco provider metadata not modified (etag {ETag}).", previous.ETag);
return previous with
{
FetchedAt = DateTimeOffset.UtcNow,
ExpiresAt = DateTimeOffset.UtcNow + _options.MetadataCacheDuration,
};
}
response.EnsureSuccessStatusCode();
var payload = await response.Content.ReadAsStringAsync(cancellationToken).ConfigureAwait(false);
var provider = ParseProvider(payload);
var etagHeader = response.Headers.ETag?.ToString();
if (_options.PersistOfflineSnapshot && !string.IsNullOrWhiteSpace(_options.OfflineSnapshotPath))
{
try
{
_fileSystem.File.WriteAllText(_options.OfflineSnapshotPath, payload);
_logger.LogDebug("Persisted Cisco metadata snapshot to {Path}.", _options.OfflineSnapshotPath);
}
catch (Exception ex)
{
_logger.LogWarning(ex, "Failed to persist Cisco metadata snapshot to {Path}.", _options.OfflineSnapshotPath);
}
}
return new CacheEntry(
provider,
DateTimeOffset.UtcNow,
DateTimeOffset.UtcNow + _options.MetadataCacheDuration,
etagHeader,
FromOffline: false);
}
catch (Exception ex) when (ex is not OperationCanceledException && !_options.PreferOfflineSnapshot)
{
_logger.LogWarning(ex, "Failed to fetch Cisco provider metadata from {Uri}; falling back to offline snapshot when available.", _options.MetadataUri);
return null;
}
}
private CacheEntry? TryLoadFromOffline()
{
if (string.IsNullOrWhiteSpace(_options.OfflineSnapshotPath))
{
return null;
}
if (!_fileSystem.File.Exists(_options.OfflineSnapshotPath))
{
_logger.LogWarning("Cisco offline snapshot path {Path} does not exist.", _options.OfflineSnapshotPath);
return null;
}
try
{
var payload = _fileSystem.File.ReadAllText(_options.OfflineSnapshotPath);
var provider = ParseProvider(payload);
return new CacheEntry(provider, DateTimeOffset.UtcNow, DateTimeOffset.UtcNow + _options.MetadataCacheDuration, null, true);
}
catch (Exception ex)
{
_logger.LogError(ex, "Failed to load Cisco provider metadata from offline snapshot {Path}.", _options.OfflineSnapshotPath);
return null;
}
}
private VexProvider ParseProvider(string payload)
{
if (string.IsNullOrWhiteSpace(payload))
{
throw new InvalidOperationException("Cisco provider metadata payload was empty.");
}
ProviderMetadataDocument? document;
try
{
document = JsonSerializer.Deserialize<ProviderMetadataDocument>(payload, _serializerOptions);
}
catch (JsonException ex)
{
throw new InvalidOperationException("Failed to parse Cisco provider metadata.", ex);
}
if (document?.Metadata?.Publisher?.ContactDetails is null || string.IsNullOrWhiteSpace(document.Metadata.Publisher.ContactDetails.Id))
{
throw new InvalidOperationException("Cisco provider metadata did not include a publisher identifier.");
}
var discovery = new VexProviderDiscovery(document.Discovery?.WellKnown, document.Discovery?.RolIe);
var trust = document.Trust is null
? VexProviderTrust.Default
: new VexProviderTrust(
document.Trust.Weight ?? 1.0,
document.Trust.Cosign is null ? null : new VexCosignTrust(document.Trust.Cosign.Issuer ?? string.Empty, document.Trust.Cosign.IdentityPattern ?? string.Empty),
document.Trust.PgpFingerprints ?? Enumerable.Empty<string>());
var directories = document.Distributions?.Directories is null
? Enumerable.Empty<Uri>()
: document.Distributions.Directories
.Where(static s => !string.IsNullOrWhiteSpace(s))
.Select(static s => Uri.TryCreate(s, UriKind.Absolute, out var uri) ? uri : null)
.Where(static uri => uri is not null)!
.Select(static uri => uri!);
return new VexProvider(
id: document.Metadata.Publisher.ContactDetails.Id,
displayName: document.Metadata.Publisher.Name ?? document.Metadata.Publisher.ContactDetails.Id,
kind: document.Metadata.Publisher.Category?.Equals("vendor", StringComparison.OrdinalIgnoreCase) == true ? VexProviderKind.Vendor : VexProviderKind.Hub,
baseUris: directories,
discovery: discovery,
trust: trust,
enabled: true);
}
private void StoreCache(CacheEntry entry)
{
var options = new MemoryCacheEntryOptions
{
AbsoluteExpiration = entry.ExpiresAt,
};
_memoryCache.Set(CacheKey, entry, options);
}
private sealed record CacheEntry(
VexProvider Provider,
DateTimeOffset FetchedAt,
DateTimeOffset ExpiresAt,
string? ETag,
bool FromOffline)
{
public bool IsExpired() => DateTimeOffset.UtcNow >= ExpiresAt;
}
}
public sealed record CiscoProviderMetadataResult(
VexProvider Provider,
DateTimeOffset FetchedAt,
bool FromOfflineSnapshot,
bool ServedFromCache);
#region document models
internal sealed class ProviderMetadataDocument
{
[System.Text.Json.Serialization.JsonPropertyName("metadata")]
public ProviderMetadataMetadata Metadata { get; set; } = new();
[System.Text.Json.Serialization.JsonPropertyName("discovery")]
public ProviderMetadataDiscovery? Discovery { get; set; }
[System.Text.Json.Serialization.JsonPropertyName("trust")]
public ProviderMetadataTrust? Trust { get; set; }
[System.Text.Json.Serialization.JsonPropertyName("distributions")]
public ProviderMetadataDistributions? Distributions { get; set; }
}
internal sealed class ProviderMetadataMetadata
{
[System.Text.Json.Serialization.JsonPropertyName("publisher")]
public ProviderMetadataPublisher Publisher { get; set; } = new();
}
internal sealed class ProviderMetadataPublisher
{
[System.Text.Json.Serialization.JsonPropertyName("name")]
public string? Name { get; set; }
[System.Text.Json.Serialization.JsonPropertyName("category")]
public string? Category { get; set; }
[System.Text.Json.Serialization.JsonPropertyName("contact_details")]
public ProviderMetadataPublisherContact ContactDetails { get; set; } = new();
}
internal sealed class ProviderMetadataPublisherContact
{
[System.Text.Json.Serialization.JsonPropertyName("id")]
public string? Id { get; set; }
}
internal sealed class ProviderMetadataDiscovery
{
[System.Text.Json.Serialization.JsonPropertyName("well_known")]
public Uri? WellKnown { get; set; }
[System.Text.Json.Serialization.JsonPropertyName("rolie")]
public Uri? RolIe { get; set; }
}
internal sealed class ProviderMetadataTrust
{
[System.Text.Json.Serialization.JsonPropertyName("weight")]
public double? Weight { get; set; }
[System.Text.Json.Serialization.JsonPropertyName("cosign")]
public ProviderMetadataTrustCosign? Cosign { get; set; }
[System.Text.Json.Serialization.JsonPropertyName("pgp_fingerprints")]
public string[]? PgpFingerprints { get; set; }
}
internal sealed class ProviderMetadataTrustCosign
{
[System.Text.Json.Serialization.JsonPropertyName("issuer")]
public string? Issuer { get; set; }
[System.Text.Json.Serialization.JsonPropertyName("identity_pattern")]
public string? IdentityPattern { get; set; }
}
internal sealed class ProviderMetadataDistributions
{
[System.Text.Json.Serialization.JsonPropertyName("directories")]
public string[]? Directories { get; set; }
}
#endregion

20
src/StellaOps.Vexer.Connectors.Cisco.CSAF/StellaOps.Vexer.Connectors.Cisco.CSAF.csproj Normal file
View File

@@ -0,0 +1,20 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>net10.0</TargetFramework>
<LangVersion>preview</LangVersion>
<Nullable>enable</Nullable>
<ImplicitUsings>enable</ImplicitUsings>
<TreatWarningsAsErrors>true</TreatWarningsAsErrors>
</PropertyGroup>
<ItemGroup>
<ProjectReference Include="..\StellaOps.Vexer.Connectors.Abstractions\StellaOps.Vexer.Connectors.Abstractions.csproj" />
<ProjectReference Include="..\StellaOps.Vexer.Core\StellaOps.Vexer.Core.csproj" />
<ProjectReference Include="..\StellaOps.Vexer.Storage.Mongo\StellaOps.Vexer.Storage.Mongo.csproj" />
</ItemGroup>
<ItemGroup>
<PackageReference Include="Microsoft.Extensions.Caching.Memory" Version="8.0.0" />
<PackageReference Include="Microsoft.Extensions.Http" Version="8.0.0" />
<PackageReference Include="Microsoft.Extensions.Options" Version="8.0.0" />
<PackageReference Include="System.IO.Abstractions" Version="20.0.28" />
</ItemGroup>
</Project>

4
src/StellaOps.Vexer.Connectors.Cisco.CSAF/TASKS.md
View File

@@ -2,6 +2,6 @@ If you are working on this file you need to read docs/ARCHITECTURE_VEXER.md and
# TASKS # TASKS
| Task | Owner(s) | Depends on | Notes | | Task | Owner(s) | Depends on | Notes |
|---|---|---|---| |---|---|---|---|
|VEXER-CONN-CISCO-01-001 – Endpoint discovery & auth plumbing|Team Vexer Connectors – Cisco|VEXER-CONN-ABS-01-001|TODO – Resolve Cisco CSAF collection URLs, configure optional token auth, and validate discovery metadata for offline caching.| |VEXER-CONN-CISCO-01-001 – Endpoint discovery & auth plumbing|Team Vexer Connectors – Cisco|VEXER-CONN-ABS-01-001|**DONE (2025-10-17)** – Added `CiscoProviderMetadataLoader` with bearer token support, offline snapshot fallback, DI helpers, and tests covering network/offline discovery to unblock subsequent fetch work.|
|VEXER-CONN-CISCO-01-002 – CSAF pull loop & pagination|Team Vexer Connectors – Cisco|VEXER-CONN-CISCO-01-001, VEXER-STORAGE-01-003|TODO – Implement paginated fetch with retries/backoff, checksum validation, and raw document persistence.| |VEXER-CONN-CISCO-01-002 – CSAF pull loop & pagination|Team Vexer Connectors – Cisco|VEXER-CONN-CISCO-01-001, VEXER-STORAGE-01-003|**DONE (2025-10-17)** – Implemented paginated advisory fetch using provider directories, raw document persistence with dedupe/state tracking, offline resiliency, and unit coverage.|
|VEXER-CONN-CISCO-01-003 – Provider trust metadata|Team Vexer Connectors – Cisco|VEXER-CONN-CISCO-01-002, VEXER-POLICY-01-001|TODO – Emit cosign/PGP trust metadata and advisory provenance hints for policy weighting.| |VEXER-CONN-CISCO-01-003 – Provider trust metadata|Team Vexer Connectors – Cisco|VEXER-CONN-CISCO-01-002, VEXER-POLICY-01-001|TODO – Emit cosign/PGP trust metadata and advisory provenance hints for policy weighting.|

176
src/StellaOps.Vexer.Connectors.MSRC.CSAF.Tests/Authentication/MsrcTokenProviderTests.cs Normal file
View File

@@ -0,0 +1,176 @@
using System.Net;
using System.Net.Http;
using System.Text;
using FluentAssertions;
using Microsoft.Extensions.Caching.Memory;
using Microsoft.Extensions.Logging.Abstractions;
using Microsoft.Extensions.Options;
using NSubstitute;
using StellaOps.Vexer.Connectors.MSRC.CSAF.Authentication;
using StellaOps.Vexer.Connectors.MSRC.CSAF.Configuration;
using System.IO.Abstractions.TestingHelpers;
using Xunit;
namespace StellaOps.Vexer.Connectors.MSRC.CSAF.Tests.Authentication;
public sealed class MsrcTokenProviderTests
{
[Fact]
public async Task GetAccessTokenAsync_CachesUntilExpiry()
{
var handler = new TestHttpMessageHandler(new[]
{
CreateTokenResponse("token-1"),
CreateTokenResponse("token-2"),
});
var client = new HttpClient(handler) { BaseAddress = new Uri("https://login.microsoftonline.com/") };
var factory = new SingleClientHttpClientFactory(client);
var cache = new MemoryCache(new MemoryCacheOptions());
var fileSystem = new MockFileSystem();
var options = Options.Create(new MsrcConnectorOptions
{
TenantId = "contoso.onmicrosoft.com",
ClientId = "client-id",
ClientSecret = "secret",
});
var timeProvider = new AdjustableTimeProvider();
var provider = new MsrcTokenProvider(factory, cache, fileSystem, options, NullLogger<MsrcTokenProvider>.Instance, timeProvider);
var first = await provider.GetAccessTokenAsync(CancellationToken.None);
first.Value.Should().Be("token-1");
handler.InvocationCount.Should().Be(1);
var second = await provider.GetAccessTokenAsync(CancellationToken.None);
second.Value.Should().Be("token-1");
handler.InvocationCount.Should().Be(1);
}
[Fact]
public async Task GetAccessTokenAsync_RefreshesWhenExpired()
{
var handler = new TestHttpMessageHandler(new[]
{
CreateTokenResponse("token-1", expiresIn: 120),
CreateTokenResponse("token-2", expiresIn: 3600),
});
var client = new HttpClient(handler) { BaseAddress = new Uri("https://login.microsoftonline.com/") };
var factory = new SingleClientHttpClientFactory(client);
var cache = new MemoryCache(new MemoryCacheOptions());
var fileSystem = new MockFileSystem();
var options = Options.Create(new MsrcConnectorOptions
{
TenantId = "contoso.onmicrosoft.com",
ClientId = "client-id",
ClientSecret = "secret",
ExpiryLeewaySeconds = 60,
});
var timeProvider = new AdjustableTimeProvider();
var provider = new MsrcTokenProvider(factory, cache, fileSystem, options, NullLogger<MsrcTokenProvider>.Instance, timeProvider);
var first = await provider.GetAccessTokenAsync(CancellationToken.None);
first.Value.Should().Be("token-1");
handler.InvocationCount.Should().Be(1);
timeProvider.Advance(TimeSpan.FromMinutes(2));
var second = await provider.GetAccessTokenAsync(CancellationToken.None);
second.Value.Should().Be("token-2");
handler.InvocationCount.Should().Be(2);
}
[Fact]
public async Task GetAccessTokenAsync_OfflineStaticToken()
{
var factory = Substitute.For<IHttpClientFactory>();
var cache = new MemoryCache(new MemoryCacheOptions());
var fileSystem = new MockFileSystem();
var options = Options.Create(new MsrcConnectorOptions
{
PreferOfflineToken = true,
StaticAccessToken = "offline-token",
});
var provider = new MsrcTokenProvider(factory, cache, fileSystem, options, NullLogger<MsrcTokenProvider>.Instance);
var token = await provider.GetAccessTokenAsync(CancellationToken.None);
token.Value.Should().Be("offline-token");
token.ExpiresAt.Should().Be(DateTimeOffset.MaxValue);
}
[Fact]
public async Task GetAccessTokenAsync_OfflineFileToken()
{
var factory = Substitute.For<IHttpClientFactory>();
var cache = new MemoryCache(new MemoryCacheOptions());
var fileSystem = new MockFileSystem();
var offlinePath = fileSystem.Path.Combine("/tokens", "msrc.txt");
fileSystem.AddFile(offlinePath, new MockFileData("file-token"));
var options = Options.Create(new MsrcConnectorOptions
{
PreferOfflineToken = true,
OfflineTokenPath = offlinePath,
});
var provider = new MsrcTokenProvider(factory, cache, fileSystem, options, NullLogger<MsrcTokenProvider>.Instance);
var token = await provider.GetAccessTokenAsync(CancellationToken.None);
token.Value.Should().Be("file-token");
token.ExpiresAt.Should().Be(DateTimeOffset.MaxValue);
}
private static HttpResponseMessage CreateTokenResponse(string token, int expiresIn = 3600)
{
var json = $"{{\"access_token\":\"{token}\",\"token_type\":\"Bearer\",\"expires_in\":{expiresIn}}}";
return new HttpResponseMessage(HttpStatusCode.OK)
{
Content = new StringContent(json, Encoding.UTF8, "application/json"),
};
}
private sealed class AdjustableTimeProvider : TimeProvider
{
private DateTimeOffset _now = DateTimeOffset.UtcNow;
public override DateTimeOffset GetUtcNow() => _now;
public void Advance(TimeSpan span) => _now = _now.Add(span);
}
private sealed class SingleClientHttpClientFactory : IHttpClientFactory
{
private readonly HttpClient _client;
public SingleClientHttpClientFactory(HttpClient client)
{
_client = client;
}
public HttpClient CreateClient(string name) => _client;
}
private sealed class TestHttpMessageHandler : HttpMessageHandler
{
private readonly Queue<HttpResponseMessage> _responses;
public TestHttpMessageHandler(IEnumerable<HttpResponseMessage> responses)
{
_responses = new Queue<HttpResponseMessage>(responses);
}
public int InvocationCount { get; private set; }
protected override Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
{
InvocationCount++;
if (_responses.Count == 0)
{
return Task.FromResult(new HttpResponseMessage(HttpStatusCode.InternalServerError)
{
Content = new StringContent("no responses remaining"),
});
}
return Task.FromResult(_responses.Dequeue());
}
}
}

18
src/StellaOps.Vexer.Connectors.MSRC.CSAF.Tests/StellaOps.Vexer.Connectors.MSRC.CSAF.Tests.csproj Normal file
View File

@@ -0,0 +1,18 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>net10.0</TargetFramework>
<LangVersion>preview</LangVersion>
<Nullable>enable</Nullable>
<ImplicitUsings>enable</ImplicitUsings>
<TreatWarningsAsErrors>true</TreatWarningsAsErrors>
</PropertyGroup>
<ItemGroup>
<ProjectReference Include="..\StellaOps.Vexer.Connectors.MSRC.CSAF\StellaOps.Vexer.Connectors.MSRC.CSAF.csproj" />
</ItemGroup>
<ItemGroup>
<PackageReference Include="FluentAssertions" Version="6.12.0" />
<PackageReference Include="Microsoft.Extensions.Logging.Abstractions" Version="8.0.0" />
<PackageReference Include="NSubstitute" Version="5.1.0" />
<PackageReference Include="System.IO.Abstractions.TestingHelpers" Version="20.0.28" />
</ItemGroup>
</Project>

185
src/StellaOps.Vexer.Connectors.MSRC.CSAF/Authentication/MsrcTokenProvider.cs Normal file
View File

@@ -0,0 +1,185 @@
using System;
using System.Collections.Generic;
using System.IO.Abstractions;
using System.Net.Http;
using System.Net.Http.Json;
using System.Text.Json.Serialization;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.Extensions.Caching.Memory;
using Microsoft.Extensions.Logging;
using Microsoft.Extensions.Options;
using StellaOps.Vexer.Connectors.MSRC.CSAF.Configuration;
namespace StellaOps.Vexer.Connectors.MSRC.CSAF.Authentication;
public interface IMsrcTokenProvider
{
ValueTask<MsrcAccessToken> GetAccessTokenAsync(CancellationToken cancellationToken);
}
public sealed class MsrcTokenProvider : IMsrcTokenProvider, IDisposable
{
private const string CachePrefix = "StellaOps.Vexer.Connectors.MSRC.CSAF.Token";
private readonly IHttpClientFactory _httpClientFactory;
private readonly IMemoryCache _cache;
private readonly IFileSystem _fileSystem;
private readonly ILogger<MsrcTokenProvider> _logger;
private readonly TimeProvider _timeProvider;
private readonly MsrcConnectorOptions _options;
private readonly SemaphoreSlim _refreshLock = new(1, 1);
public MsrcTokenProvider(
IHttpClientFactory httpClientFactory,
IMemoryCache cache,
IFileSystem fileSystem,
IOptions<MsrcConnectorOptions> options,
ILogger<MsrcTokenProvider> logger,
TimeProvider? timeProvider = null)
{
_httpClientFactory = httpClientFactory ?? throw new ArgumentNullException(nameof(httpClientFactory));
_cache = cache ?? throw new ArgumentNullException(nameof(cache));
_fileSystem = fileSystem ?? throw new ArgumentNullException(nameof(fileSystem));
ArgumentNullException.ThrowIfNull(options);
_options = options.Value ?? throw new ArgumentNullException(nameof(options));
_options.Validate(_fileSystem);
_logger = logger ?? throw new ArgumentNullException(nameof(logger));
_timeProvider = timeProvider ?? TimeProvider.System;
}
public async ValueTask<MsrcAccessToken> GetAccessTokenAsync(CancellationToken cancellationToken)
{
if (_options.PreferOfflineToken)
{
return LoadOfflineToken();
}
var cacheKey = CreateCacheKey();
if (_cache.TryGetValue<MsrcAccessToken>(cacheKey, out var cachedToken) &&
cachedToken is not null &&
!cachedToken.IsExpired(_timeProvider.GetUtcNow()))
{
return cachedToken;
}
await _refreshLock.WaitAsync(cancellationToken).ConfigureAwait(false);
try
{
if (_cache.TryGetValue<MsrcAccessToken>(cacheKey, out cachedToken) &&
cachedToken is not null &&
!cachedToken.IsExpired(_timeProvider.GetUtcNow()))
{
return cachedToken;
}
var token = await RequestTokenAsync(cancellationToken).ConfigureAwait(false);
var absoluteExpiration = token.ExpiresAt == DateTimeOffset.MaxValue
? (DateTimeOffset?)null
: token.ExpiresAt;
var options = new MemoryCacheEntryOptions();
if (absoluteExpiration.HasValue)
{
options.AbsoluteExpiration = absoluteExpiration.Value;
}
_cache.Set(cacheKey, token, options);
return token;
}
finally
{
_refreshLock.Release();
}
}
private MsrcAccessToken LoadOfflineToken()
{
if (!string.IsNullOrWhiteSpace(_options.StaticAccessToken))
{
return new MsrcAccessToken(_options.StaticAccessToken!, "Bearer", DateTimeOffset.MaxValue);
}
if (string.IsNullOrWhiteSpace(_options.OfflineTokenPath))
{
throw new InvalidOperationException("Offline token mode is enabled but no token was provided.");
}
if (!_fileSystem.File.Exists(_options.OfflineTokenPath))
{
throw new InvalidOperationException($"Offline token path '{_options.OfflineTokenPath}' does not exist.");
}
var token = _fileSystem.File.ReadAllText(_options.OfflineTokenPath).Trim();
if (string.IsNullOrEmpty(token))
{
throw new InvalidOperationException("Offline token file was empty.");
}
return new MsrcAccessToken(token, "Bearer", DateTimeOffset.MaxValue);
}
private async Task<MsrcAccessToken> RequestTokenAsync(CancellationToken cancellationToken)
{
_logger.LogInformation("Fetching MSRC AAD access token for tenant {TenantId}.", _options.TenantId);
var client = _httpClientFactory.CreateClient(MsrcConnectorOptions.TokenClientName);
using var request = new HttpRequestMessage(HttpMethod.Post, BuildTokenUri())
{
Content = new FormUrlEncodedContent(new Dictionary<string, string>
{
["client_id"] = _options.ClientId,
["client_secret"] = _options.ClientSecret!,
["grant_type"] = "client_credentials",
["scope"] = string.IsNullOrWhiteSpace(_options.Scope) ? MsrcConnectorOptions.DefaultScope : _options.Scope,
}),
};
using var response = await client.SendAsync(request, cancellationToken).ConfigureAwait(false);
if (!response.IsSuccessStatusCode)
{
var payload = await response.Content.ReadAsStringAsync(cancellationToken).ConfigureAwait(false);
throw new InvalidOperationException($"Failed to acquire MSRC access token ({(int)response.StatusCode}). Response: {payload}");
}
var tokenResponse = await response.Content.ReadFromJsonAsync<TokenResponse>(cancellationToken: cancellationToken).ConfigureAwait(false)
?? throw new InvalidOperationException("Token endpoint returned an empty payload.");
if (string.IsNullOrWhiteSpace(tokenResponse.AccessToken))
{
throw new InvalidOperationException("Token endpoint response did not include an access_token.");
}
var now = _timeProvider.GetUtcNow();
var expiresAt = tokenResponse.ExpiresIn > _options.ExpiryLeewaySeconds
? now.AddSeconds(tokenResponse.ExpiresIn - _options.ExpiryLeewaySeconds)
: now.AddMinutes(5);
return new MsrcAccessToken(tokenResponse.AccessToken!, tokenResponse.TokenType ?? "Bearer", expiresAt);
}
private string CreateCacheKey()
=> $"{CachePrefix}:{_options.TenantId}:{_options.ClientId}:{_options.Scope}";
private Uri BuildTokenUri()
=> new($"https://login.microsoftonline.com/{_options.TenantId}/oauth2/v2.0/token");
public void Dispose() => _refreshLock.Dispose();
private sealed record TokenResponse
{
[JsonPropertyName("access_token")]
public string? AccessToken { get; init; }
[JsonPropertyName("token_type")]
public string? TokenType { get; init; }
[JsonPropertyName("expires_in")]
public int ExpiresIn { get; init; }
}
}
public sealed record MsrcAccessToken(string Value, string Type, DateTimeOffset ExpiresAt)
{
public bool IsExpired(DateTimeOffset now) => now >= ExpiresAt;
}

95
src/StellaOps.Vexer.Connectors.MSRC.CSAF/Configuration/MsrcConnectorOptions.cs Normal file
View File

@@ -0,0 +1,95 @@
using System;
using System.IO;
using System.IO.Abstractions;
namespace StellaOps.Vexer.Connectors.MSRC.CSAF.Configuration;
public sealed class MsrcConnectorOptions
{
public const string TokenClientName = "vexer.connector.msrc.token";
public const string DefaultScope = "https://api.msrc.microsoft.com/.default";
/// <summary>
/// Azure AD tenant identifier (GUID or domain).
/// </summary>
public string TenantId { get; set; } = string.Empty;
/// <summary>
/// Azure AD application (client) identifier.
/// </summary>
public string ClientId { get; set; } = string.Empty;
/// <summary>
/// Azure AD application secret for client credential flow.
/// </summary>
public string? ClientSecret { get; set; }
/// <summary>
/// OAuth scope requested for MSRC API access.
/// </summary>
public string Scope { get; set; } = DefaultScope;
/// <summary>
/// When true, token acquisition is skipped and the connector expects offline handling.
/// </summary>
public bool PreferOfflineToken { get; set; }
/// <summary>
/// Optional path to a pre-provisioned bearer token used when <see cref="PreferOfflineToken"/> is enabled.
/// </summary>
public string? OfflineTokenPath { get; set; }
/// <summary>
/// Optional fixed bearer token for constrained environments (e.g., short-lived offline bundles).
/// </summary>
public string? StaticAccessToken { get; set; }
/// <summary>
/// Minimum buffer (seconds) subtracted from token expiry before refresh.
/// </summary>
public int ExpiryLeewaySeconds { get; set; } = 60;
public void Validate(IFileSystem? fileSystem = null)
{
if (PreferOfflineToken)
{
if (string.IsNullOrWhiteSpace(OfflineTokenPath) && string.IsNullOrWhiteSpace(StaticAccessToken))
{
throw new InvalidOperationException("OfflineTokenPath or StaticAccessToken must be provided when PreferOfflineToken is enabled.");
}
}
else
{
if (string.IsNullOrWhiteSpace(TenantId))
{
throw new InvalidOperationException("TenantId is required when not operating in offline token mode.");
}
if (string.IsNullOrWhiteSpace(ClientId))
{
throw new InvalidOperationException("ClientId is required when not operating in offline token mode.");
}
if (string.IsNullOrWhiteSpace(ClientSecret))
{
throw new InvalidOperationException("ClientSecret is required when not operating in offline token mode.");
}
}
if (string.IsNullOrWhiteSpace(Scope))
{
Scope = DefaultScope;
}
if (ExpiryLeewaySeconds < 10)
{
ExpiryLeewaySeconds = 10;
}
if (!string.IsNullOrWhiteSpace(OfflineTokenPath))
{
var fs = fileSystem ?? new FileSystem();
var directory = Path.GetDirectoryName(OfflineTokenPath);
if (!string.IsNullOrWhiteSpace(directory) && !fs.Directory.Exists(directory))
{
fs.Directory.CreateDirectory(directory);
}
}
}
}

40
src/StellaOps.Vexer.Connectors.MSRC.CSAF/DependencyInjection/MsrcConnectorServiceCollectionExtensions.cs Normal file
View File

@@ -0,0 +1,40 @@
using System;
using System.Net;
using System.Net.Http;
using Microsoft.Extensions.Caching.Memory;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.DependencyInjection.Extensions;
using StellaOps.Vexer.Connectors.MSRC.CSAF.Authentication;
using StellaOps.Vexer.Connectors.MSRC.CSAF.Configuration;
using System.IO.Abstractions;
namespace StellaOps.Vexer.Connectors.MSRC.CSAF.DependencyInjection;
public static class MsrcConnectorServiceCollectionExtensions
{
public static IServiceCollection AddMsrcCsafConnector(this IServiceCollection services, Action<MsrcConnectorOptions>? configure = null)
{
ArgumentNullException.ThrowIfNull(services);
services.TryAddSingleton<IMemoryCache, MemoryCache>();
services.TryAddSingleton<IFileSystem, FileSystem>();
services.AddOptions<MsrcConnectorOptions>()
.Configure(options => configure?.Invoke(options));
services.AddHttpClient(MsrcConnectorOptions.TokenClientName, client =>
{
client.Timeout = TimeSpan.FromSeconds(30);
client.DefaultRequestHeaders.UserAgent.ParseAdd("StellaOps.Vexer.Connectors.MSRC.CSAF/1.0");
client.DefaultRequestHeaders.Accept.ParseAdd("application/json");
})
.ConfigurePrimaryHttpMessageHandler(static () => new HttpClientHandler
{
AutomaticDecompression = DecompressionMethods.All,
});
services.AddSingleton<IMsrcTokenProvider, MsrcTokenProvider>();
return services;
}
}

18
src/StellaOps.Vexer.Connectors.MSRC.CSAF/StellaOps.Vexer.Connectors.MSRC.CSAF.csproj Normal file
View File

@@ -0,0 +1,18 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>net10.0</TargetFramework>
<LangVersion>preview</LangVersion>
<Nullable>enable</Nullable>
<ImplicitUsings>enable</ImplicitUsings>
<TreatWarningsAsErrors>true</TreatWarningsAsErrors>
</PropertyGroup>
<ItemGroup>
<ProjectReference Include="..\StellaOps.Vexer.Connectors.Abstractions\StellaOps.Vexer.Connectors.Abstractions.csproj" />
</ItemGroup>
<ItemGroup>
<PackageReference Include="Microsoft.Extensions.Caching.Memory" Version="8.0.0" />
<PackageReference Include="Microsoft.Extensions.Http" Version="8.0.0" />
<PackageReference Include="Microsoft.Extensions.Options" Version="8.0.0" />
<PackageReference Include="System.IO.Abstractions" Version="20.0.28" />
</ItemGroup>
</Project>

2
src/StellaOps.Vexer.Connectors.MSRC.CSAF/TASKS.md
View File

@@ -2,6 +2,6 @@ If you are working on this file you need to read docs/ARCHITECTURE_VEXER.md and
# TASKS # TASKS
| Task | Owner(s) | Depends on | Notes | | Task | Owner(s) | Depends on | Notes |
|---|---|---|---| |---|---|---|---|
|VEXER-CONN-MS-01-001 – AAD onboarding & token cache|Team Vexer Connectors – MSRC|VEXER-CONN-ABS-01-001|TODO – Implement Azure AD credential flow, token caching, and validation for MSRC CSAF access with offline fallback guidance.| |VEXER-CONN-MS-01-001 – AAD onboarding & token cache|Team Vexer Connectors – MSRC|VEXER-CONN-ABS-01-001|**DONE (2025-10-17)** – Added MSRC connector project with configurable AAD options, token provider (offline/online modes), DI wiring, and unit tests covering caching and fallback scenarios.|
|VEXER-CONN-MS-01-002 – CSAF download pipeline|Team Vexer Connectors – MSRC|VEXER-CONN-MS-01-001, VEXER-STORAGE-01-003|TODO – Fetch CSAF packages with retry/backoff, checksum verification, and raw document persistence plus quarantine for schema failures.| |VEXER-CONN-MS-01-002 – CSAF download pipeline|Team Vexer Connectors – MSRC|VEXER-CONN-MS-01-001, VEXER-STORAGE-01-003|TODO – Fetch CSAF packages with retry/backoff, checksum verification, and raw document persistence plus quarantine for schema failures.|
|VEXER-CONN-MS-01-003 – Trust metadata & provenance hints|Team Vexer Connectors – MSRC|VEXER-CONN-MS-01-002, VEXER-POLICY-01-001|TODO – Emit cosign/AAD issuer metadata, attach provenance details, and document policy integration.| |VEXER-CONN-MS-01-003 – Trust metadata & provenance hints|Team Vexer Connectors – MSRC|VEXER-CONN-MS-01-002, VEXER-POLICY-01-001|TODO – Emit cosign/AAD issuer metadata, attach provenance details, and document policy integration.|

205
src/StellaOps.Vexer.Connectors.Oracle.CSAF.Tests/Metadata/OracleCatalogLoaderTests.cs Normal file
View File

@@ -0,0 +1,205 @@
using System.Collections.Generic;
using System.Net;
using System.Net.Http;
using System.Text;
using FluentAssertions;
using Microsoft.Extensions.Caching.Memory;
using Microsoft.Extensions.Logging.Abstractions;
using StellaOps.Vexer.Connectors.Oracle.CSAF.Configuration;
using StellaOps.Vexer.Connectors.Oracle.CSAF.Metadata;
using System.IO.Abstractions.TestingHelpers;
using Xunit;
using System.Threading;
namespace StellaOps.Vexer.Connectors.Oracle.CSAF.Tests.Metadata;
public sealed class OracleCatalogLoaderTests
{
private const string SampleCatalog = """
{
"generated": "2025-09-30T18:00:00Z",
"catalog": [
{
"id": "CPU2025Oct",
"title": "Oracle Critical Patch Update Advisory - October 2025",
"published": "2025-10-15T00:00:00Z",
"revision": "2025-10-15T00:00:00Z",
"document": {
"url": "https://updates.oracle.com/cpu/2025-10/cpu2025oct.json",
"sha256": "abc123",
"size": 1024
},
"products": ["Oracle Database", "Java SE"]
}
],
"schedule": [
{ "window": "2025-Oct", "releaseDate": "2025-10-15T00:00:00Z" }
]
}
""";
private const string SampleCalendar = """
{
"cpuWindows": [
{ "name": "2026-Jan", "releaseDate": "2026-01-21T00:00:00Z" }
]
}
""";
[Fact]
public async Task LoadAsync_FetchesAndCachesCatalog()
{
var handler = new TestHttpMessageHandler(new Dictionary<Uri, HttpResponseMessage>
{
[new Uri("https://www.oracle.com/security-alerts/cpu/csaf/catalog.json")] = CreateResponse(SampleCatalog),
[new Uri("https://www.oracle.com/security-alerts/cpu/cal.json")] = CreateResponse(SampleCalendar),
});
var client = new HttpClient(handler);
var factory = new SingleClientHttpClientFactory(client);
var cache = new MemoryCache(new MemoryCacheOptions());
var fileSystem = new MockFileSystem();
var loader = new OracleCatalogLoader(factory, cache, fileSystem, NullLogger<OracleCatalogLoader>.Instance, new AdjustableTimeProvider());
var options = new OracleConnectorOptions
{
CatalogUri = new Uri("https://www.oracle.com/security-alerts/cpu/csaf/catalog.json"),
CpuCalendarUri = new Uri("https://www.oracle.com/security-alerts/cpu/cal.json"),
OfflineSnapshotPath = "/snapshots/oracle-catalog.json",
};
var result = await loader.LoadAsync(options, CancellationToken.None);
result.Metadata.Entries.Should().HaveCount(1);
result.Metadata.CpuSchedule.Should().Contain(r => r.Window == "2025-Oct");
result.Metadata.CpuSchedule.Should().Contain(r => r.Window == "2026-Jan");
result.FromCache.Should().BeFalse();
fileSystem.FileExists(options.OfflineSnapshotPath!).Should().BeTrue();
handler.ResetInvocationCount();
var cached = await loader.LoadAsync(options, CancellationToken.None);
cached.FromCache.Should().BeTrue();
handler.InvocationCount.Should().Be(0);
}
[Fact]
public async Task LoadAsync_UsesOfflineSnapshotWhenNetworkFails()
{
var handler = new TestHttpMessageHandler(new Dictionary<Uri, HttpResponseMessage>());
var client = new HttpClient(handler);
var factory = new SingleClientHttpClientFactory(client);
var cache = new MemoryCache(new MemoryCacheOptions());
var fileSystem = new MockFileSystem();
var offlineJson = """
{
"metadata": {
"generatedAt": "2025-09-30T18:00:00Z",
"entries": [
{
"id": "CPU2025Oct",
"title": "Oracle Critical Patch Update Advisory - October 2025",
"documentUri": "https://updates.oracle.com/cpu/2025-10/cpu2025oct.json",
"publishedAt": "2025-10-15T00:00:00Z",
"revision": "2025-10-15T00:00:00Z",
"sha256": "abc123",
"size": 1024,
"products": [ "Oracle Database" ]
}
],
"cpuSchedule": [
{ "window": "2025-Oct", "releaseDate": "2025-10-15T00:00:00Z" }
]
},
"fetchedAt": "2025-10-01T00:00:00Z"
}
""";
fileSystem.AddFile("/snapshots/oracle-catalog.json", new MockFileData(offlineJson));
var loader = new OracleCatalogLoader(factory, cache, fileSystem, NullLogger<OracleCatalogLoader>.Instance, new AdjustableTimeProvider());
var options = new OracleConnectorOptions
{
CatalogUri = new Uri("https://www.oracle.com/security-alerts/cpu/csaf/catalog.json"),
OfflineSnapshotPath = "/snapshots/oracle-catalog.json",
PreferOfflineSnapshot = true,
};
var result = await loader.LoadAsync(options, CancellationToken.None);
result.FromOfflineSnapshot.Should().BeTrue();
result.Metadata.Entries.Should().NotBeEmpty();
}
[Fact]
public async Task LoadAsync_ThrowsWhenOfflinePreferredButMissing()
{
var handler = new TestHttpMessageHandler(new Dictionary<Uri, HttpResponseMessage>());
var client = new HttpClient(handler);
var factory = new SingleClientHttpClientFactory(client);
var cache = new MemoryCache(new MemoryCacheOptions());
var fileSystem = new MockFileSystem();
var loader = new OracleCatalogLoader(factory, cache, fileSystem, NullLogger<OracleCatalogLoader>.Instance);
var options = new OracleConnectorOptions
{
CatalogUri = new Uri("https://www.oracle.com/security-alerts/cpu/csaf/catalog.json"),
PreferOfflineSnapshot = true,
OfflineSnapshotPath = "/missing/oracle-catalog.json",
};
await Assert.ThrowsAsync<InvalidOperationException>(() => loader.LoadAsync(options, CancellationToken.None));
}
private static HttpResponseMessage CreateResponse(string payload)
=> new(HttpStatusCode.OK)
{
Content = new StringContent(payload, Encoding.UTF8, "application/json"),
};
private sealed class SingleClientHttpClientFactory : IHttpClientFactory
{
private readonly HttpClient _client;
public SingleClientHttpClientFactory(HttpClient client)
{
_client = client;
}
public HttpClient CreateClient(string name) => _client;
}
private sealed class AdjustableTimeProvider : TimeProvider
{
private DateTimeOffset _now = DateTimeOffset.UtcNow;
public override DateTimeOffset GetUtcNow() => _now;
}
private sealed class TestHttpMessageHandler : HttpMessageHandler
{
private readonly Dictionary<Uri, HttpResponseMessage> _responses;
public TestHttpMessageHandler(Dictionary<Uri, HttpResponseMessage> responses)
{
_responses = responses;
}
public int InvocationCount { get; private set; }
public void ResetInvocationCount() => InvocationCount = 0;
protected override async Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
{
InvocationCount++;
if (request.RequestUri is not null && _responses.TryGetValue(request.RequestUri, out var response))
{
var payload = await response.Content.ReadAsStringAsync(cancellationToken).ConfigureAwait(false);
return new HttpResponseMessage(response.StatusCode)
{
Content = new StringContent(payload, Encoding.UTF8, "application/json"),
};
}
return new HttpResponseMessage(HttpStatusCode.InternalServerError)
{
Content = new StringContent("unexpected request"),
};
}
}
}

17
src/StellaOps.Vexer.Connectors.Oracle.CSAF.Tests/StellaOps.Vexer.Connectors.Oracle.CSAF.Tests.csproj Normal file
View File

@@ -0,0 +1,17 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>net10.0</TargetFramework>
<LangVersion>preview</LangVersion>
<Nullable>enable</Nullable>
<ImplicitUsings>enable</ImplicitUsings>
<TreatWarningsAsErrors>true</TreatWarningsAsErrors>
</PropertyGroup>
<ItemGroup>
<ProjectReference Include="..\StellaOps.Vexer.Connectors.Oracle.CSAF\StellaOps.Vexer.Connectors.Oracle.CSAF.csproj" />
</ItemGroup>
<ItemGroup>
<PackageReference Include="FluentAssertions" Version="6.12.0" />
<PackageReference Include="Microsoft.Extensions.Logging.Abstractions" Version="8.0.0" />
<PackageReference Include="System.IO.Abstractions.TestingHelpers" Version="20.0.28" />
</ItemGroup>
</Project>

85
src/StellaOps.Vexer.Connectors.Oracle.CSAF/Configuration/OracleConnectorOptions.cs Normal file
View File

@@ -0,0 +1,85 @@
using System;
using System.IO;
using System.IO.Abstractions;
namespace StellaOps.Vexer.Connectors.Oracle.CSAF.Configuration;
public sealed class OracleConnectorOptions
{
public const string HttpClientName = "vexer.connector.oracle.catalog";
/// <summary>
/// Oracle CSAF catalog endpoint hosting advisory metadata.
/// </summary>
public Uri CatalogUri { get; set; } = new("https://www.oracle.com/security-alerts/cpu/csaf/catalog.json");
/// <summary>
/// Optional CPU calendar endpoint providing upcoming release dates.
/// </summary>
public Uri? CpuCalendarUri { get; set; }
/// <summary>
/// Duration the discovery metadata should be cached before refresh.
/// </summary>
public TimeSpan MetadataCacheDuration { get; set; } = TimeSpan.FromHours(6);
/// <summary>
/// When true, the loader will prefer offline snapshot data over network fetches.
/// </summary>
public bool PreferOfflineSnapshot { get; set; }
/// <summary>
/// Optional file path for persisting or ingesting catalog snapshots.
/// </summary>
public string? OfflineSnapshotPath { get; set; }
/// <summary>
/// Enables writing fresh catalog responses to <see cref="OfflineSnapshotPath"/>.
/// </summary>
public bool PersistOfflineSnapshot { get; set; } = true;
/// <summary>
/// Optional request delay when iterating catalogue entries (for rate limiting).
/// </summary>
public TimeSpan RequestDelay { get; set; } = TimeSpan.FromMilliseconds(250);
public void Validate(IFileSystem? fileSystem = null)
{
if (CatalogUri is null || !CatalogUri.IsAbsoluteUri)
{
throw new InvalidOperationException("CatalogUri must be an absolute URI.");
}
if (CatalogUri.Scheme is not ("http" or "https"))
{
throw new InvalidOperationException("CatalogUri must use HTTP or HTTPS.");
}
if (CpuCalendarUri is not null && (!CpuCalendarUri.IsAbsoluteUri || CpuCalendarUri.Scheme is not ("http" or "https")))
{
throw new InvalidOperationException("CpuCalendarUri must be an absolute HTTP(S) URI when provided.");
}
if (MetadataCacheDuration <= TimeSpan.Zero)
{
throw new InvalidOperationException("MetadataCacheDuration must be positive.");
}
if (RequestDelay < TimeSpan.Zero)
{
throw new InvalidOperationException("RequestDelay cannot be negative.");
}
if (PreferOfflineSnapshot && string.IsNullOrWhiteSpace(OfflineSnapshotPath))
{
throw new InvalidOperationException("OfflineSnapshotPath must be provided when PreferOfflineSnapshot is enabled.");
}
if (!string.IsNullOrWhiteSpace(OfflineSnapshotPath))
{
var fs = fileSystem ?? new FileSystem();
var directory = Path.GetDirectoryName(OfflineSnapshotPath);
if (!string.IsNullOrWhiteSpace(directory) && !fs.Directory.Exists(directory))
{
fs.Directory.CreateDirectory(directory);
}
}
}
}

32
src/StellaOps.Vexer.Connectors.Oracle.CSAF/Configuration/OracleConnectorOptionsValidator.cs Normal file
View File

@@ -0,0 +1,32 @@
using System;
using System.Collections.Generic;
using System.IO.Abstractions;
using StellaOps.Vexer.Connectors.Abstractions;
namespace StellaOps.Vexer.Connectors.Oracle.CSAF.Configuration;
public sealed class OracleConnectorOptionsValidator : IVexConnectorOptionsValidator<OracleConnectorOptions>
{
private readonly IFileSystem _fileSystem;
public OracleConnectorOptionsValidator(IFileSystem fileSystem)
{
_fileSystem = fileSystem ?? throw new ArgumentNullException(nameof(fileSystem));
}
public void Validate(VexConnectorDescriptor descriptor, OracleConnectorOptions options, IList<string> errors)
{
ArgumentNullException.ThrowIfNull(descriptor);
ArgumentNullException.ThrowIfNull(options);
ArgumentNullException.ThrowIfNull(errors);
try
{
options.Validate(_fileSystem);
}
catch (Exception ex)
{
errors.Add(ex.Message);
}
}
}

45
src/StellaOps.Vexer.Connectors.Oracle.CSAF/DependencyInjection/OracleConnectorServiceCollectionExtensions.cs Normal file
View File

@@ -0,0 +1,45 @@
using System;
using System.Net;
using System.Net.Http;
using Microsoft.Extensions.Caching.Memory;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.DependencyInjection.Extensions;
using StellaOps.Vexer.Connectors.Abstractions;
using StellaOps.Vexer.Connectors.Oracle.CSAF.Configuration;
using StellaOps.Vexer.Connectors.Oracle.CSAF.Metadata;
using StellaOps.Vexer.Core;
using System.IO.Abstractions;
namespace StellaOps.Vexer.Connectors.Oracle.CSAF.DependencyInjection;
public static class OracleConnectorServiceCollectionExtensions
{
public static IServiceCollection AddOracleCsafConnector(this IServiceCollection services, Action<OracleConnectorOptions>? configure = null)
{
ArgumentNullException.ThrowIfNull(services);
services.TryAddSingleton<IMemoryCache, MemoryCache>();
services.TryAddSingleton<IFileSystem, FileSystem>();
services.AddOptions<OracleConnectorOptions>()
.Configure(options => configure?.Invoke(options));
services.AddSingleton<IVexConnectorOptionsValidator<OracleConnectorOptions>, OracleConnectorOptionsValidator>();
services.AddHttpClient(OracleConnectorOptions.HttpClientName, client =>
{
client.Timeout = TimeSpan.FromSeconds(60);
client.DefaultRequestHeaders.UserAgent.ParseAdd("StellaOps.Vexer.Connectors.Oracle.CSAF/1.0");
client.DefaultRequestHeaders.Accept.ParseAdd("application/json");
})
.ConfigurePrimaryHttpMessageHandler(static () => new HttpClientHandler
{
AutomaticDecompression = DecompressionMethods.All,
});
services.AddSingleton<OracleCatalogLoader>();
services.AddSingleton<IVexConnector, OracleCsafConnector>();
return services;
}
}

418
src/StellaOps.Vexer.Connectors.Oracle.CSAF/Metadata/OracleCatalogLoader.cs Normal file
View File

@@ -0,0 +1,418 @@
using System;
using System.Collections.Generic;
using System.Collections.Immutable;
using System.IO.Abstractions;
using System.Net;
using System.Net.Http;
using System.Text.Json;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.Extensions.Caching.Memory;
using Microsoft.Extensions.Logging;
using StellaOps.Vexer.Connectors.Oracle.CSAF.Configuration;
namespace StellaOps.Vexer.Connectors.Oracle.CSAF.Metadata;
public sealed class OracleCatalogLoader
{
public const string CachePrefix = "StellaOps.Vexer.Connectors.Oracle.CSAF.Catalog";
private readonly IHttpClientFactory _httpClientFactory;
private readonly IMemoryCache _memoryCache;
private readonly IFileSystem _fileSystem;
private readonly ILogger<OracleCatalogLoader> _logger;
private readonly TimeProvider _timeProvider;
private readonly SemaphoreSlim _semaphore = new(1, 1);
private readonly JsonSerializerOptions _serializerOptions = new(JsonSerializerDefaults.Web);
public OracleCatalogLoader(
IHttpClientFactory httpClientFactory,
IMemoryCache memoryCache,
IFileSystem fileSystem,
ILogger<OracleCatalogLoader> logger,
TimeProvider? timeProvider = null)
{
_httpClientFactory = httpClientFactory ?? throw new ArgumentNullException(nameof(httpClientFactory));
_memoryCache = memoryCache ?? throw new ArgumentNullException(nameof(memoryCache));
_fileSystem = fileSystem ?? throw new ArgumentNullException(nameof(fileSystem));
_logger = logger ?? throw new ArgumentNullException(nameof(logger));
_timeProvider = timeProvider ?? TimeProvider.System;
}
public async Task<OracleCatalogResult> LoadAsync(OracleConnectorOptions options, CancellationToken cancellationToken)
{
ArgumentNullException.ThrowIfNull(options);
options.Validate(_fileSystem);
var cacheKey = CreateCacheKey(options);
if (_memoryCache.TryGetValue<CacheEntry>(cacheKey, out var cached) && cached is not null && !cached.IsExpired(_timeProvider.GetUtcNow()))
{
return cached.ToResult(fromCache: true);
}
await _semaphore.WaitAsync(cancellationToken).ConfigureAwait(false);
try
{
if (_memoryCache.TryGetValue<CacheEntry>(cacheKey, out cached) && cached is not null && !cached.IsExpired(_timeProvider.GetUtcNow()))
{
return cached.ToResult(fromCache: true);
}
CacheEntry? entry = null;
if (options.PreferOfflineSnapshot)
{
entry = LoadFromOffline(options);
if (entry is null)
{
throw new InvalidOperationException("PreferOfflineSnapshot is enabled but no offline snapshot was found or could be loaded.");
}
}
else
{
entry = await TryFetchFromNetworkAsync(options, cancellationToken).ConfigureAwait(false)
?? LoadFromOffline(options);
}
if (entry is null)
{
throw new InvalidOperationException("Unable to load Oracle CSAF catalog from network or offline snapshot.");
}
var expiration = entry.MetadataCacheDuration == TimeSpan.Zero
? (DateTimeOffset?)null
: _timeProvider.GetUtcNow().Add(entry.MetadataCacheDuration);
var cacheEntryOptions = new MemoryCacheEntryOptions();
if (expiration.HasValue)
{
cacheEntryOptions.AbsoluteExpiration = expiration.Value;
}
_memoryCache.Set(cacheKey, entry with { CachedAt = _timeProvider.GetUtcNow() }, cacheEntryOptions);
return entry.ToResult(fromCache: false);
}
finally
{
_semaphore.Release();
}
}
private async Task<CacheEntry?> TryFetchFromNetworkAsync(OracleConnectorOptions options, CancellationToken cancellationToken)
{
try
{
var client = _httpClientFactory.CreateClient(OracleConnectorOptions.HttpClientName);
using var response = await client.GetAsync(options.CatalogUri, HttpCompletionOption.ResponseHeadersRead, cancellationToken).ConfigureAwait(false);
response.EnsureSuccessStatusCode();
var catalogPayload = await response.Content.ReadAsStringAsync(cancellationToken).ConfigureAwait(false);
string? calendarPayload = null;
if (options.CpuCalendarUri is not null)
{
try
{
using var calendarResponse = await client.GetAsync(options.CpuCalendarUri, HttpCompletionOption.ResponseHeadersRead, cancellationToken).ConfigureAwait(false);
calendarResponse.EnsureSuccessStatusCode();
calendarPayload = await calendarResponse.Content.ReadAsStringAsync(cancellationToken).ConfigureAwait(false);
}
catch (Exception ex) when (ex is not OperationCanceledException)
{
_logger.LogWarning(ex, "Failed to fetch Oracle CPU calendar from {Uri}; continuing without schedule.", options.CpuCalendarUri);
}
}
var metadata = ParseMetadata(catalogPayload, calendarPayload);
var fetchedAt = _timeProvider.GetUtcNow();
var entry = new CacheEntry(metadata, fetchedAt, fetchedAt, options.MetadataCacheDuration, false);
PersistSnapshotIfNeeded(options, metadata, fetchedAt);
return entry;
}
catch (Exception ex) when (ex is not OperationCanceledException)
{
_logger.LogWarning(ex, "Failed to fetch Oracle CSAF catalog from {Uri}; attempting offline fallback if available.", options.CatalogUri);
return null;
}
}
private CacheEntry? LoadFromOffline(OracleConnectorOptions options)
{
if (string.IsNullOrWhiteSpace(options.OfflineSnapshotPath))
{
return null;
}
if (!_fileSystem.File.Exists(options.OfflineSnapshotPath))
{
_logger.LogWarning("Oracle offline snapshot path {Path} does not exist.", options.OfflineSnapshotPath);
return null;
}
try
{
var payload = _fileSystem.File.ReadAllText(options.OfflineSnapshotPath);
var snapshot = JsonSerializer.Deserialize<OracleCatalogSnapshot>(payload, _serializerOptions);
if (snapshot is null)
{
throw new InvalidOperationException("Offline snapshot payload was empty.");
}
return new CacheEntry(snapshot.Metadata, snapshot.FetchedAt, _timeProvider.GetUtcNow(), options.MetadataCacheDuration, true);
}
catch (Exception ex)
{
_logger.LogError(ex, "Failed to load Oracle CSAF catalog from offline snapshot {Path}.", options.OfflineSnapshotPath);
return null;
}
}
private OracleCatalogMetadata ParseMetadata(string catalogPayload, string? calendarPayload)
{
if (string.IsNullOrWhiteSpace(catalogPayload))
{
throw new InvalidOperationException("Oracle catalog payload was empty.");
}
using var document = JsonDocument.Parse(catalogPayload);
var root = document.RootElement;
var generatedAt = root.TryGetProperty("generated", out var generatedElement) && generatedElement.ValueKind == JsonValueKind.String && DateTimeOffset.TryParse(generatedElement.GetString(), out var generated)
? generated
: _timeProvider.GetUtcNow();
var entries = ParseEntries(root);
var schedule = ParseSchedule(root);
if (!string.IsNullOrWhiteSpace(calendarPayload))
{
schedule = MergeSchedule(schedule, calendarPayload);
}
return new OracleCatalogMetadata(generatedAt, entries, schedule);
}
private ImmutableArray<OracleCatalogEntry> ParseEntries(JsonElement root)
{
if (!root.TryGetProperty("catalog", out var catalogElement) || catalogElement.ValueKind is not JsonValueKind.Array)
{
return ImmutableArray<OracleCatalogEntry>.Empty;
}
var builder = ImmutableArray.CreateBuilder<OracleCatalogEntry>();
foreach (var entry in catalogElement.EnumerateArray())
{
var id = entry.TryGetProperty("id", out var idElement) && idElement.ValueKind == JsonValueKind.String ? idElement.GetString() : null;
var title = entry.TryGetProperty("title", out var titleElement) && titleElement.ValueKind == JsonValueKind.String ? titleElement.GetString() : null;
if (string.IsNullOrWhiteSpace(id) || string.IsNullOrWhiteSpace(title))
{
continue;
}
DateTimeOffset publishedAt = default;
if (entry.TryGetProperty("published", out var publishedElement) && publishedElement.ValueKind == JsonValueKind.String && DateTimeOffset.TryParse(publishedElement.GetString(), out var publishedParsed))
{
publishedAt = publishedParsed;
}
string? revision = null;
if (entry.TryGetProperty("revision", out var revisionElement) && revisionElement.ValueKind == JsonValueKind.String)
{
revision = revisionElement.GetString();
}
ImmutableArray<string> products = ImmutableArray<string>.Empty;
if (entry.TryGetProperty("products", out var productsElement))
{
products = ParseStringArray(productsElement);
}
Uri? documentUri = null;
string? sha256 = null;
long? size = null;
if (entry.TryGetProperty("document", out var documentElement) && documentElement.ValueKind == JsonValueKind.Object)
{
if (documentElement.TryGetProperty("url", out var urlElement) && urlElement.ValueKind == JsonValueKind.String && Uri.TryCreate(urlElement.GetString(), UriKind.Absolute, out var parsedUri))
{
documentUri = parsedUri;
}
if (documentElement.TryGetProperty("sha256", out var hashElement) && hashElement.ValueKind == JsonValueKind.String)
{
sha256 = hashElement.GetString();
}
if (documentElement.TryGetProperty("size", out var sizeElement) && sizeElement.ValueKind == JsonValueKind.Number && sizeElement.TryGetInt64(out var parsedSize))
{
size = parsedSize;
}
}
if (documentUri is null)
{
continue;
}
builder.Add(new OracleCatalogEntry(id!, title!, documentUri, publishedAt, revision, sha256, size, products));
}
return builder.ToImmutable();
}
private ImmutableArray<OracleCpuRelease> ParseSchedule(JsonElement root)
{
if (!root.TryGetProperty("schedule", out var scheduleElement) || scheduleElement.ValueKind is not JsonValueKind.Array)
{
return ImmutableArray<OracleCpuRelease>.Empty;
}
var builder = ImmutableArray.CreateBuilder<OracleCpuRelease>();
foreach (var item in scheduleElement.EnumerateArray())
{
var window = item.TryGetProperty("window", out var windowElement) && windowElement.ValueKind == JsonValueKind.String ? windowElement.GetString() : null;
if (string.IsNullOrWhiteSpace(window))
{
continue;
}
DateTimeOffset releaseDate = default;
if (item.TryGetProperty("releaseDate", out var releaseElement) && releaseElement.ValueKind == JsonValueKind.String && DateTimeOffset.TryParse(releaseElement.GetString(), out var parsed))
{
releaseDate = parsed;
}
builder.Add(new OracleCpuRelease(window!, releaseDate));
}
return builder.ToImmutable();
}
private ImmutableArray<OracleCpuRelease> MergeSchedule(ImmutableArray<OracleCpuRelease> existing, string calendarPayload)
{
try
{
using var document = JsonDocument.Parse(calendarPayload);
var root = document.RootElement;
if (!root.TryGetProperty("cpuWindows", out var windowsElement) || windowsElement.ValueKind is not JsonValueKind.Array)
{
return existing;
}
var builder = existing.ToBuilder();
var known = new HashSet<string>(StringComparer.OrdinalIgnoreCase);
foreach (var item in builder)
{
known.Add(item.Window);
}
foreach (var windowElement in windowsElement.EnumerateArray())
{
var name = windowElement.TryGetProperty("name", out var nameElement) && nameElement.ValueKind == JsonValueKind.String ? nameElement.GetString() : null;
if (string.IsNullOrWhiteSpace(name))
{
continue;
}
if (!known.Add(name))
{
continue;
}
DateTimeOffset releaseDate = default;
if (windowElement.TryGetProperty("releaseDate", out var releaseElement) && releaseElement.ValueKind == JsonValueKind.String && DateTimeOffset.TryParse(releaseElement.GetString(), out var parsed))
{
releaseDate = parsed;
}
builder.Add(new OracleCpuRelease(name!, releaseDate));
}
return builder.ToImmutable();
}
catch (Exception ex) when (ex is not OperationCanceledException)
{
_logger.LogWarning(ex, "Failed to parse Oracle CPU calendar payload; continuing with existing schedule data.");
return existing;
}
}
private ImmutableArray<string> ParseStringArray(JsonElement element)
{
if (element.ValueKind is not JsonValueKind.Array)
{
return ImmutableArray<string>.Empty;
}
var builder = ImmutableArray.CreateBuilder<string>();
foreach (var item in element.EnumerateArray())
{
if (item.ValueKind == JsonValueKind.String)
{
var value = item.GetString();
if (!string.IsNullOrWhiteSpace(value))
{
builder.Add(value);
}
}
}
return builder.ToImmutable();
}
private void PersistSnapshotIfNeeded(OracleConnectorOptions options, OracleCatalogMetadata metadata, DateTimeOffset fetchedAt)
{
if (!options.PersistOfflineSnapshot || string.IsNullOrWhiteSpace(options.OfflineSnapshotPath))
{
return;
}
try
{
var snapshot = new OracleCatalogSnapshot(metadata, fetchedAt);
var payload = JsonSerializer.Serialize(snapshot, _serializerOptions);
_fileSystem.File.WriteAllText(options.OfflineSnapshotPath, payload);
_logger.LogDebug("Persisted Oracle CSAF catalog snapshot to {Path}.", options.OfflineSnapshotPath);
}
catch (Exception ex)
{
_logger.LogWarning(ex, "Failed to persist Oracle CSAF catalog snapshot to {Path}.", options.OfflineSnapshotPath);
}
}
private static string CreateCacheKey(OracleConnectorOptions options)
=> $"{CachePrefix}:{options.CatalogUri}:{options.CpuCalendarUri}";
private sealed record CacheEntry(OracleCatalogMetadata Metadata, DateTimeOffset FetchedAt, DateTimeOffset CachedAt, TimeSpan MetadataCacheDuration, bool FromOfflineSnapshot)
{
public bool IsExpired(DateTimeOffset now)
=> MetadataCacheDuration > TimeSpan.Zero && now >= CachedAt + MetadataCacheDuration;
public OracleCatalogResult ToResult(bool fromCache)
=> new(Metadata, FetchedAt, fromCache, FromOfflineSnapshot);
}
private sealed record OracleCatalogSnapshot(OracleCatalogMetadata Metadata, DateTimeOffset FetchedAt);
}
public sealed record OracleCatalogMetadata(
DateTimeOffset GeneratedAt,
ImmutableArray<OracleCatalogEntry> Entries,
ImmutableArray<OracleCpuRelease> CpuSchedule);
public sealed record OracleCatalogEntry(
string Id,
string Title,
Uri DocumentUri,
DateTimeOffset PublishedAt,
string? Revision,
string? Sha256,
long? Size,
ImmutableArray<string> Products);
public sealed record OracleCpuRelease(string Window, DateTimeOffset ReleaseDate);
public sealed record OracleCatalogResult(
OracleCatalogMetadata Metadata,
DateTimeOffset FetchedAt,
bool FromCache,
bool FromOfflineSnapshot);

81
src/StellaOps.Vexer.Connectors.Oracle.CSAF/OracleCsafConnector.cs Normal file
View File

@@ -0,0 +1,81 @@
using System.Collections.Generic;
using System.Collections.Immutable;
using System.Runtime.CompilerServices;
using Microsoft.Extensions.Logging;
using StellaOps.Vexer.Connectors.Abstractions;
using StellaOps.Vexer.Connectors.Oracle.CSAF.Configuration;
using StellaOps.Vexer.Connectors.Oracle.CSAF.Metadata;
using StellaOps.Vexer.Core;
namespace StellaOps.Vexer.Connectors.Oracle.CSAF;
public sealed class OracleCsafConnector : VexConnectorBase
{
private static readonly VexConnectorDescriptor DescriptorInstance = new(
id: "vexer:oracle",
kind: VexProviderKind.Vendor,
displayName: "Oracle CSAF")
{
Tags = ImmutableArray.Create("oracle", "csaf", "cpu"),
};
private readonly OracleCatalogLoader _catalogLoader;
private readonly IEnumerable<IVexConnectorOptionsValidator<OracleConnectorOptions>> _validators;
private OracleConnectorOptions? _options;
private OracleCatalogResult? _catalog;
public OracleCsafConnector(
OracleCatalogLoader catalogLoader,
IEnumerable<IVexConnectorOptionsValidator<OracleConnectorOptions>> validators,
ILogger<OracleCsafConnector> logger,
TimeProvider timeProvider)
: base(DescriptorInstance, logger, timeProvider)
{
_catalogLoader = catalogLoader ?? throw new ArgumentNullException(nameof(catalogLoader));
_validators = validators ?? Array.Empty<IVexConnectorOptionsValidator<OracleConnectorOptions>>();
}
public override async ValueTask ValidateAsync(VexConnectorSettings settings, CancellationToken cancellationToken)
{
_options = VexConnectorOptionsBinder.Bind(
Descriptor,
settings,
validators: _validators);
_catalog = await _catalogLoader.LoadAsync(_options, cancellationToken).ConfigureAwait(false);
LogConnectorEvent(LogLevel.Information, "validate", "Oracle CSAF catalogue loaded.", new Dictionary<string, object?>
{
["catalogEntryCount"] = _catalog.Metadata.Entries.Length,
["scheduleCount"] = _catalog.Metadata.CpuSchedule.Length,
["fromOffline"] = _catalog.FromOfflineSnapshot,
});
}
public override async IAsyncEnumerable<VexRawDocument> FetchAsync(VexConnectorContext context, [EnumeratorCancellation] CancellationToken cancellationToken)
{
ArgumentNullException.ThrowIfNull(context);
if (_options is null)
{
throw new InvalidOperationException("Connector must be validated before fetch operations.");
}
if (_catalog is null)
{
_catalog = await _catalogLoader.LoadAsync(_options, cancellationToken).ConfigureAwait(false);
}
LogConnectorEvent(LogLevel.Debug, "fetch", "Oracle CSAF discovery ready; document ingestion handled by follow-up task.", new Dictionary<string, object?>
{
["since"] = context.Since?.ToString("O"),
});
yield break;
}
public override ValueTask<VexClaimBatch> NormalizeAsync(VexRawDocument document, CancellationToken cancellationToken)
=> throw new NotSupportedException("OracleCsafConnector relies on dedicated CSAF normalizers.");
public OracleCatalogResult? GetCachedCatalog() => _catalog;
}

18
src/StellaOps.Vexer.Connectors.Oracle.CSAF/StellaOps.Vexer.Connectors.Oracle.CSAF.csproj Normal file
View File

@@ -0,0 +1,18 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>net10.0</TargetFramework>
<LangVersion>preview</LangVersion>
<Nullable>enable</Nullable>
<ImplicitUsings>enable</ImplicitUsings>
<TreatWarningsAsErrors>true</TreatWarningsAsErrors>
</PropertyGroup>
<ItemGroup>
<ProjectReference Include="..\StellaOps.Vexer.Connectors.Abstractions\StellaOps.Vexer.Connectors.Abstractions.csproj" />
</ItemGroup>
<ItemGroup>
<PackageReference Include="Microsoft.Extensions.Caching.Memory" Version="8.0.0" />
<PackageReference Include="Microsoft.Extensions.Http" Version="8.0.0" />
<PackageReference Include="Microsoft.Extensions.Options" Version="8.0.0" />
<PackageReference Include="System.IO.Abstractions" Version="20.0.28" />
</ItemGroup>
</Project>

2
src/StellaOps.Vexer.Connectors.Oracle.CSAF/TASKS.md
View File

@@ -2,6 +2,6 @@ If you are working on this file you need to read docs/ARCHITECTURE_VEXER.md and
# TASKS # TASKS
| Task | Owner(s) | Depends on | Notes | | Task | Owner(s) | Depends on | Notes |
|---|---|---|---| |---|---|---|---|
|VEXER-CONN-ORACLE-01-001 – Oracle CSAF catalogue discovery|Team Vexer Connectors – Oracle|VEXER-CONN-ABS-01-001|TODO – Implement catalogue discovery, CPU calendar awareness, and offline snapshot import for Oracle CSAF feeds.| |VEXER-CONN-ORACLE-01-001 – Oracle CSAF catalogue discovery|Team Vexer Connectors – Oracle|VEXER-CONN-ABS-01-001|DOING (2025-10-17) – Implement catalogue discovery, CPU calendar awareness, and offline snapshot import for Oracle CSAF feeds.|
|VEXER-CONN-ORACLE-01-002 – CSAF download & dedupe pipeline|Team Vexer Connectors – Oracle|VEXER-CONN-ORACLE-01-001, VEXER-STORAGE-01-003|TODO – Fetch CSAF documents with retry/backoff, checksum validation, revision deduplication, and raw persistence.| |VEXER-CONN-ORACLE-01-002 – CSAF download & dedupe pipeline|Team Vexer Connectors – Oracle|VEXER-CONN-ORACLE-01-001, VEXER-STORAGE-01-003|TODO – Fetch CSAF documents with retry/backoff, checksum validation, revision deduplication, and raw persistence.|
|VEXER-CONN-ORACLE-01-003 – Trust metadata + provenance|Team Vexer Connectors – Oracle|VEXER-CONN-ORACLE-01-002, VEXER-POLICY-01-001|TODO – Emit Oracle signing metadata (PGP/cosign) and provenance hints for consensus weighting.| |VEXER-CONN-ORACLE-01-003 – Trust metadata + provenance|Team Vexer Connectors – Oracle|VEXER-CONN-ORACLE-01-002, VEXER-POLICY-01-001|TODO – Emit Oracle signing metadata (PGP/cosign) and provenance hints for consensus weighting.|

277
src/StellaOps.Vexer.Connectors.RedHat.CSAF.Tests/Connectors/RedHatCsafConnectorTests.cs Normal file
View File

@@ -0,0 +1,277 @@
using System.Collections.Generic;
using System.Collections.Immutable;
using System.Net;
using System.Net.Http;
using System.Text;
using FluentAssertions;
using Microsoft.Extensions.Caching.Memory;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Logging.Abstractions;
using Microsoft.Extensions.Options;
using StellaOps.Vexer.Connectors.Abstractions;
using StellaOps.Vexer.Connectors.RedHat.CSAF.Configuration;
using StellaOps.Vexer.Connectors.RedHat.CSAF.Metadata;
using StellaOps.Vexer.Core;
using StellaOps.Vexer.Storage.Mongo;
namespace StellaOps.Vexer.Connectors.RedHat.CSAF.Tests.Connectors;
public sealed class RedHatCsafConnectorTests
{
private static readonly VexConnectorDescriptor Descriptor = new("vexer:redhat", VexProviderKind.Distro, "Red Hat CSAF");
[Fact]
public async Task FetchAsync_EmitsDocumentsAfterSince()
{
var metadata = """
{
"metadata": {
"provider": { "name": "Red Hat Product Security" }
},
"distributions": [
{ "directory": "https://example.com/security/data/csaf/v2/advisories/" }
],
"rolie": {
"feeds": [
{ "url": "https://example.com/security/data/csaf/v2/advisories/rolie/feed.atom" }
]
}
}
""";
var feed = """
<feed xmlns="http://www.w3.org/2005/Atom">
<entry>
<id>urn:redhat:1</id>
<updated>2025-10-16T10:00:00Z</updated>
<link href="https://example.com/doc1.json" rel="enclosure" />
</entry>
<entry>
<id>urn:redhat:2</id>
<updated>2025-10-17T10:00:00Z</updated>
<link href="https://example.com/doc2.json" rel="enclosure" />
</entry>
</feed>
""";
var handler = TestHttpMessageHandler.Create(
request => Response(HttpStatusCode.OK, metadata, "application/json"),
request => Response(HttpStatusCode.OK, feed, "application/atom+xml"),
request => Response(HttpStatusCode.OK, "{ \"csaf\": 1 }", "application/json"));
var httpClient = new HttpClient(handler)
{
BaseAddress = new Uri("https://example.com/"),
};
var factory = new SingleClientHttpClientFactory(httpClient);
var cache = new MemoryCache(new MemoryCacheOptions());
var options = Options.Create(new RedHatConnectorOptions());
var metadataLoader = new RedHatProviderMetadataLoader(factory, cache, options, NullLogger<RedHatProviderMetadataLoader>.Instance);
var stateRepository = new InMemoryConnectorStateRepository();
var connector = new RedHatCsafConnector(Descriptor, metadataLoader, factory, stateRepository, NullLogger<RedHatCsafConnector>.Instance, TimeProvider.System);
var rawSink = new CapturingRawSink();
var context = new VexConnectorContext(
new DateTimeOffset(2025, 10, 16, 12, 0, 0, TimeSpan.Zero),
VexConnectorSettings.Empty,
rawSink,
new NoopSignatureVerifier(),
new NoopNormalizerRouter(),
new ServiceCollection().BuildServiceProvider());
var results = new List<VexRawDocument>();
await foreach (var document in connector.FetchAsync(context, CancellationToken.None))
{
results.Add(document);
}
Assert.Single(results);
Assert.Single(rawSink.Documents);
Assert.Equal("https://example.com/doc2.json", results[0].SourceUri.ToString());
Assert.Equal("https://example.com/doc2.json", rawSink.Documents[0].SourceUri.ToString());
Assert.Equal(3, handler.CallCount);
stateRepository.State.Should().NotBeNull();
stateRepository.State!.LastUpdated.Should().Be(new DateTimeOffset(2025, 10, 17, 10, 0, 0, TimeSpan.Zero));
stateRepository.State.DocumentDigests.Should().HaveCount(1);
}
[Fact]
public async Task FetchAsync_UsesStateToSkipDuplicateDocuments()
{
var metadata = """
{
"metadata": {
"provider": { "name": "Red Hat Product Security" }
},
"distributions": [
{ "directory": "https://example.com/security/data/csaf/v2/advisories/" }
],
"rolie": {
"feeds": [
{ "url": "https://example.com/security/data/csaf/v2/advisories/rolie/feed.atom" }
]
}
}
""";
var feed = """
<feed xmlns="http://www.w3.org/2005/Atom">
<entry>
<id>urn:redhat:1</id>
<updated>2025-10-17T10:00:00Z</updated>
<link href="https://example.com/doc1.json" rel="enclosure" />
</entry>
</feed>
""";
var handler1 = TestHttpMessageHandler.Create(
_ => Response(HttpStatusCode.OK, metadata, "application/json"),
_ => Response(HttpStatusCode.OK, feed, "application/atom+xml"),
_ => Response(HttpStatusCode.OK, "{ \"csaf\": 1 }", "application/json"));
var stateRepository = new InMemoryConnectorStateRepository();
await ExecuteFetchAsync(handler1, stateRepository);
stateRepository.State.Should().NotBeNull();
var previousState = stateRepository.State!;
var handler2 = TestHttpMessageHandler.Create(
_ => Response(HttpStatusCode.OK, metadata, "application/json"),
_ => Response(HttpStatusCode.OK, feed, "application/atom+xml"),
_ => Response(HttpStatusCode.OK, "{ \"csaf\": 1 }", "application/json"));
var (results, rawSink) = await ExecuteFetchAsync(handler2, stateRepository);
results.Should().BeEmpty();
rawSink.Documents.Should().BeEmpty();
stateRepository.State!.DocumentDigests.Should().Equal(previousState.DocumentDigests);
}
private static HttpResponseMessage Response(HttpStatusCode statusCode, string content, string contentType)
=> new(statusCode)
{
Content = new StringContent(content, Encoding.UTF8, contentType),
};
private sealed class CapturingRawSink : IVexRawDocumentSink
{
public List<VexRawDocument> Documents { get; } = new();
public ValueTask StoreAsync(VexRawDocument document, CancellationToken cancellationToken)
{
Documents.Add(document);
return ValueTask.CompletedTask;
}
}
private sealed class NoopSignatureVerifier : IVexSignatureVerifier
{
public ValueTask<VexSignatureMetadata?> VerifyAsync(VexRawDocument document, CancellationToken cancellationToken)
=> ValueTask.FromResult<VexSignatureMetadata?>(null);
}
private sealed class NoopNormalizerRouter : IVexNormalizerRouter
{
public ValueTask<VexClaimBatch> NormalizeAsync(VexRawDocument document, CancellationToken cancellationToken)
=> ValueTask.FromResult(new VexClaimBatch(document, ImmutableArray<VexClaim>.Empty, ImmutableDictionary<string, string>.Empty));
}
private sealed class SingleClientHttpClientFactory : IHttpClientFactory
{
private readonly HttpClient _client;
public SingleClientHttpClientFactory(HttpClient client)
{
_client = client;
}
public HttpClient CreateClient(string name) => _client;
}
private sealed class TestHttpMessageHandler : HttpMessageHandler
{
private readonly Queue<Func<HttpRequestMessage, HttpResponseMessage>> _responders;
private TestHttpMessageHandler(IEnumerable<Func<HttpRequestMessage, HttpResponseMessage>> responders)
{
_responders = new Queue<Func<HttpRequestMessage, HttpResponseMessage>>(responders);
}
public int CallCount { get; private set; }
public static TestHttpMessageHandler Create(params Func<HttpRequestMessage, HttpResponseMessage>[] responders)
=> new(responders);
protected override Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
{
CallCount++;
if (_responders.Count == 0)
{
throw new InvalidOperationException("No responder configured for request.");
}
var responder = _responders.Count > 1
? _responders.Dequeue()
: _responders.Peek();
var response = responder(request);
response.RequestMessage = request;
return Task.FromResult(response);
}
}
private static async Task<(List<VexRawDocument> Documents, CapturingRawSink Sink)> ExecuteFetchAsync(
TestHttpMessageHandler handler,
InMemoryConnectorStateRepository stateRepository)
{
var httpClient = new HttpClient(handler)
{
BaseAddress = new Uri("https://example.com/"),
};
var factory = new SingleClientHttpClientFactory(httpClient);
var cache = new MemoryCache(new MemoryCacheOptions());
var options = Options.Create(new RedHatConnectorOptions());
var metadataLoader = new RedHatProviderMetadataLoader(factory, cache, options, NullLogger<RedHatProviderMetadataLoader>.Instance);
var connector = new RedHatCsafConnector(Descriptor, metadataLoader, factory, stateRepository, NullLogger<RedHatCsafConnector>.Instance, TimeProvider.System);
var rawSink = new CapturingRawSink();
var context = new VexConnectorContext(
null,
VexConnectorSettings.Empty,
rawSink,
new NoopSignatureVerifier(),
new NoopNormalizerRouter(),
new ServiceCollection().BuildServiceProvider());
var documents = new List<VexRawDocument>();
await foreach (var document in connector.FetchAsync(context, CancellationToken.None))
{
documents.Add(document);
}
return (documents, rawSink);
}
private sealed class InMemoryConnectorStateRepository : IVexConnectorStateRepository
{
public VexConnectorState? State { get; private set; }
public ValueTask<VexConnectorState?> GetAsync(string connectorId, CancellationToken cancellationToken)
{
if (State is not null && string.Equals(State.ConnectorId, connectorId, StringComparison.OrdinalIgnoreCase))
{
return ValueTask.FromResult<VexConnectorState?>(State);
}
return ValueTask.FromResult<VexConnectorState?>(null);
}
public ValueTask SaveAsync(VexConnectorState state, CancellationToken cancellationToken)
{
State = state;
return ValueTask.CompletedTask;
}
}
}

235
src/StellaOps.Vexer.Connectors.RedHat.CSAF.Tests/Metadata/RedHatProviderMetadataLoaderTests.cs Normal file
View File

@@ -0,0 +1,235 @@
using System.Collections.Generic;
using System.Net;
using System.Net.Http;
using System.Net.Http.Headers;
using System.Text;
using Microsoft.Extensions.Caching.Memory;
using Microsoft.Extensions.Logging.Abstractions;
using Microsoft.Extensions.Options;
using StellaOps.Vexer.Connectors.RedHat.CSAF.Configuration;
using StellaOps.Vexer.Connectors.RedHat.CSAF.Metadata;
using System.IO.Abstractions.TestingHelpers;
namespace StellaOps.Vexer.Connectors.RedHat.CSAF.Tests.Metadata;
public sealed class RedHatProviderMetadataLoaderTests
{
private const string SampleJson = """
{
"metadata": {
"provider": {
"name": "Red Hat Product Security"
}
},
"distributions": [
{ "directory": "https://access.redhat.com/security/data/csaf/v2/advisories/" }
],
"rolie": {
"feeds": [
{ "url": "https://access.redhat.com/security/data/csaf/v2/advisories/rolie/feed.atom" }
]
}
}
""";
[Fact]
public async Task LoadAsync_FetchesMetadataAndCaches()
{
var handler = TestHttpMessageHandler.RespondWith(_ =>
{
var response = new HttpResponseMessage(HttpStatusCode.OK)
{
Content = new StringContent(SampleJson, Encoding.UTF8, "application/json"),
};
response.Headers.ETag = new EntityTagHeaderValue("\"abc\"");
return response;
});
var httpClient = new HttpClient(handler)
{
BaseAddress = new Uri("https://access.redhat.com/"),
};
var factory = new SingleClientHttpClientFactory(httpClient);
var cache = new MemoryCache(new MemoryCacheOptions());
var fileSystem = new MockFileSystem();
var options = new RedHatConnectorOptions
{
MetadataUri = new Uri("https://access.redhat.com/.well-known/csaf/provider-metadata.json"),
OfflineSnapshotPath = fileSystem.Path.Combine("/offline", "redhat-provider.json"),
CosignIssuer = "https://sigstore.dev/redhat",
CosignIdentityPattern = "^spiffe://redhat/.+$",
};
options.PgpFingerprints.Add("A1B2C3D4E5F6");
options.Validate(fileSystem);
var loader = new RedHatProviderMetadataLoader(factory, cache, Options.Create(options), NullLogger<RedHatProviderMetadataLoader>.Instance, fileSystem);
var result = await loader.LoadAsync(CancellationToken.None);
Assert.Equal("Red Hat Product Security", result.Provider.DisplayName);
Assert.False(result.FromCache);
Assert.False(result.FromOfflineSnapshot);
Assert.Single(result.Provider.BaseUris);
Assert.Equal("https://access.redhat.com/security/data/csaf/v2/advisories/", result.Provider.BaseUris[0].ToString());
Assert.Equal("https://access.redhat.com/.well-known/csaf/provider-metadata.json", result.Provider.Discovery.WellKnownMetadata?.ToString());
Assert.Equal("https://access.redhat.com/security/data/csaf/v2/advisories/rolie/feed.atom", result.Provider.Discovery.RolIeService?.ToString());
Assert.Equal(1.0, result.Provider.Trust.Weight);
Assert.NotNull(result.Provider.Trust.Cosign);
Assert.Equal("https://sigstore.dev/redhat", result.Provider.Trust.Cosign!.Issuer);
Assert.Equal("^spiffe://redhat/.+$", result.Provider.Trust.Cosign.IdentityPattern);
Assert.Contains("A1B2C3D4E5F6", result.Provider.Trust.PgpFingerprints);
Assert.True(fileSystem.FileExists(options.OfflineSnapshotPath));
Assert.Equal(1, handler.CallCount);
var second = await loader.LoadAsync(CancellationToken.None);
Assert.True(second.FromCache);
Assert.False(second.FromOfflineSnapshot);
Assert.Equal(1, handler.CallCount);
}
[Fact]
public async Task LoadAsync_UsesOfflineSnapshotWhenPreferred()
{
var handler = TestHttpMessageHandler.RespondWith(_ => throw new InvalidOperationException("HTTP should not be called"));
var httpClient = new HttpClient(handler);
var factory = new SingleClientHttpClientFactory(httpClient);
var cache = new MemoryCache(new MemoryCacheOptions());
var fileSystem = new MockFileSystem(new Dictionary<string, MockFileData>
{
["/snapshots/redhat.json"] = new MockFileData(SampleJson),
});
var options = new RedHatConnectorOptions
{
MetadataUri = new Uri("https://access.redhat.com/.well-known/csaf/provider-metadata.json"),
OfflineSnapshotPath = "/snapshots/redhat.json",
PreferOfflineSnapshot = true,
PersistOfflineSnapshot = false,
};
options.Validate(fileSystem);
var loader = new RedHatProviderMetadataLoader(factory, cache, Options.Create(options), NullLogger<RedHatProviderMetadataLoader>.Instance, fileSystem);
var result = await loader.LoadAsync(CancellationToken.None);
Assert.Equal("Red Hat Product Security", result.Provider.DisplayName);
Assert.False(result.FromCache);
Assert.True(result.FromOfflineSnapshot);
Assert.Equal(0, handler.CallCount);
var second = await loader.LoadAsync(CancellationToken.None);
Assert.True(second.FromCache);
Assert.True(second.FromOfflineSnapshot);
Assert.Equal(0, handler.CallCount);
}
[Fact]
public async Task LoadAsync_UsesETagForConditionalRequest()
{
var handler = TestHttpMessageHandler.Create(
_ =>
{
var response = new HttpResponseMessage(HttpStatusCode.OK)
{
Content = new StringContent(SampleJson, Encoding.UTF8, "application/json"),
};
response.Headers.ETag = new EntityTagHeaderValue("\"abc\"");
return response;
},
request =>
{
Assert.Contains(request.Headers.IfNoneMatch, etag => etag.Tag == "\"abc\"");
return new HttpResponseMessage(HttpStatusCode.NotModified);
});
var httpClient = new HttpClient(handler);
var factory = new SingleClientHttpClientFactory(httpClient);
var cache = new MemoryCache(new MemoryCacheOptions());
var fileSystem = new MockFileSystem();
var options = new RedHatConnectorOptions
{
MetadataUri = new Uri("https://access.redhat.com/.well-known/csaf/provider-metadata.json"),
OfflineSnapshotPath = "/offline/redhat.json",
MetadataCacheDuration = TimeSpan.FromMinutes(1),
};
options.Validate(fileSystem);
var loader = new RedHatProviderMetadataLoader(factory, cache, Options.Create(options), NullLogger<RedHatProviderMetadataLoader>.Instance, fileSystem);
var first = await loader.LoadAsync(CancellationToken.None);
Assert.False(first.FromCache);
Assert.False(first.FromOfflineSnapshot);
Assert.True(cache.TryGetValue(RedHatProviderMetadataLoader.CacheKey, out var entryObj));
Assert.NotNull(entryObj);
var entryType = entryObj!.GetType();
var provider = entryType.GetProperty("Provider")!.GetValue(entryObj);
var fetchedAt = entryType.GetProperty("FetchedAt")!.GetValue(entryObj);
var etag = entryType.GetProperty("ETag")!.GetValue(entryObj);
var fromOffline = entryType.GetProperty("FromOffline")!.GetValue(entryObj);
var expiredEntry = Activator.CreateInstance(entryType, provider, fetchedAt, DateTimeOffset.UtcNow - TimeSpan.FromSeconds(1), etag, fromOffline);
cache.Set(RedHatProviderMetadataLoader.CacheKey, expiredEntry!, new MemoryCacheEntryOptions
{
AbsoluteExpiration = DateTimeOffset.UtcNow + TimeSpan.FromMinutes(1),
});
var second = await loader.LoadAsync(CancellationToken.None);
var third = await loader.LoadAsync(CancellationToken.None);
Assert.True(third.FromCache);
Assert.Equal(2, handler.CallCount);
}
private sealed class SingleClientHttpClientFactory : IHttpClientFactory
{
private readonly HttpClient _client;
public SingleClientHttpClientFactory(HttpClient client)
{
_client = client ?? throw new ArgumentNullException(nameof(client));
}
public HttpClient CreateClient(string name) => _client;
}
private sealed class TestHttpMessageHandler : HttpMessageHandler
{
private readonly Queue<Func<HttpRequestMessage, HttpResponseMessage>> _responders;
private TestHttpMessageHandler(IEnumerable<Func<HttpRequestMessage, HttpResponseMessage>> responders)
{
_responders = new Queue<Func<HttpRequestMessage, HttpResponseMessage>>(responders);
}
public int CallCount { get; private set; }
public static TestHttpMessageHandler RespondWith(Func<HttpRequestMessage, HttpResponseMessage> responder)
=> new(new[] { responder });
public static TestHttpMessageHandler Create(params Func<HttpRequestMessage, HttpResponseMessage>[] responders)
=> new(responders);
protected override Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
{
CallCount++;
if (_responders.Count == 0)
{
throw new InvalidOperationException("No responder configured for request.");
}
var responder = _responders.Count > 1
? _responders.Dequeue()
: _responders.Peek();
var response = responder(request);
response.RequestMessage = request;
return Task.FromResult(response);
}
}
}

17
src/StellaOps.Vexer.Connectors.RedHat.CSAF.Tests/StellaOps.Vexer.Connectors.RedHat.CSAF.Tests.csproj Normal file
View File

@@ -0,0 +1,17 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>net10.0</TargetFramework>
<LangVersion>preview</LangVersion>
<Nullable>enable</Nullable>
<ImplicitUsings>enable</ImplicitUsings>
<TreatWarningsAsErrors>true</TreatWarningsAsErrors>
</PropertyGroup>
<ItemGroup>
<ProjectReference Include="..\StellaOps.Vexer.Connectors.RedHat.CSAF\StellaOps.Vexer.Connectors.RedHat.CSAF.csproj" />
<ProjectReference Include="..\StellaOps.Vexer.Storage.Mongo\StellaOps.Vexer.Storage.Mongo.csproj" />
</ItemGroup>
<ItemGroup>
<PackageReference Include="FluentAssertions" Version="6.12.0" />
<PackageReference Include="System.IO.Abstractions.TestingHelpers" Version="20.0.28" />
</ItemGroup>
</Project>

2
src/StellaOps.Vexer.Connectors.RedHat.CSAF/AGENTS.md
View File

@@ -3,6 +3,8 @@
Connector for Red Hat CSAF VEX feeds, fetching provider metadata, CSAF documents, and projecting them into raw storage for normalization. Connector for Red Hat CSAF VEX feeds, fetching provider metadata, CSAF documents, and projecting them into raw storage for normalization.
## Scope ## Scope
- Discovery via `/.well-known/csaf/provider-metadata.json`, scheduling windows, and ETag-aware HTTP fetches. - Discovery via `/.well-known/csaf/provider-metadata.json`, scheduling windows, and ETag-aware HTTP fetches.
- `RedHatProviderMetadataLoader` handles `.well-known` metadata with caching, schema validation, and offline snapshots.
- `RedHatCsafConnector` consumes ROLIE feeds to fetch incremental CSAF documents, honours `context.Since`, and streams raw advisories to storage.
- Mapping Red Hat CSAF specifics (product tree aliases, RHSA identifiers, revision history) into raw documents. - Mapping Red Hat CSAF specifics (product tree aliases, RHSA identifiers, revision history) into raw documents.
- Emitting structured telemetry and resume markers for incremental pulls. - Emitting structured telemetry and resume markers for incremental pulls.
- Supplying Red Hat-specific trust overrides and provenance hints to normalization. - Supplying Red Hat-specific trust overrides and provenance hints to normalization.

104
src/StellaOps.Vexer.Connectors.RedHat.CSAF/Configuration/RedHatConnectorOptions.cs Normal file
View File

@@ -0,0 +1,104 @@
using System.Collections.Generic;
using System.IO.Abstractions;
namespace StellaOps.Vexer.Connectors.RedHat.CSAF.Configuration;
public sealed class RedHatConnectorOptions
{
public static readonly Uri DefaultMetadataUri = new("https://access.redhat.com/.well-known/csaf/provider-metadata.json");
/// <summary>
/// HTTP client name registered for the connector.
/// </summary>
public const string HttpClientName = "vexer.connector.redhat";
/// <summary>
/// URI of the CSAF provider metadata document.
/// </summary>
public Uri MetadataUri { get; set; } = DefaultMetadataUri;
/// <summary>
/// Duration to cache loaded metadata before refreshing.
/// </summary>
public TimeSpan MetadataCacheDuration { get; set; } = TimeSpan.FromHours(1);
/// <summary>
/// Optional file path used to store or source offline metadata snapshots.
/// </summary>
public string? OfflineSnapshotPath { get; set; }
/// <summary>
/// When true, the loader prefers the offline snapshot without attempting a network fetch.
/// </summary>
public bool PreferOfflineSnapshot { get; set; }
/// <summary>
/// Enables writing fresh metadata responses to <see cref="OfflineSnapshotPath"/>.
/// </summary>
public bool PersistOfflineSnapshot { get; set; } = true;
/// <summary>
/// Explicit trust weight override applied to the provider entry.
/// </summary>
public double TrustWeight { get; set; } = 1.0;
/// <summary>
/// Sigstore/Cosign issuer used to verify CSAF signatures, if published.
/// </summary>
public string? CosignIssuer { get; set; } = "https://access.redhat.com";
/// <summary>
/// Identity pattern matched against the Cosign certificate subject.
/// </summary>
public string? CosignIdentityPattern { get; set; } = "^https://access\\.redhat\\.com/.+$";
/// <summary>
/// Optional list of PGP fingerprints recognised for Red Hat CSAF artifacts.
/// </summary>
public IList<string> PgpFingerprints { get; } = new List<string>();
public void Validate(IFileSystem? fileSystem = null)
{
if (MetadataUri is null || !MetadataUri.IsAbsoluteUri)
{
throw new InvalidOperationException("Metadata URI must be absolute.");
}
if (MetadataUri.Scheme is not ("http" or "https"))
{
throw new InvalidOperationException("Metadata URI must use HTTP or HTTPS.");
}
if (MetadataCacheDuration <= TimeSpan.Zero)
{
throw new InvalidOperationException("Metadata cache duration must be positive.");
}
if (!string.IsNullOrWhiteSpace(OfflineSnapshotPath))
{
var fs = fileSystem ?? new FileSystem();
var directory = Path.GetDirectoryName(OfflineSnapshotPath);
if (!string.IsNullOrWhiteSpace(directory) && !fs.Directory.Exists(directory))
{
fs.Directory.CreateDirectory(directory);
}
}
if (double.IsNaN(TrustWeight) || double.IsInfinity(TrustWeight) || TrustWeight <= 0)
{
TrustWeight = 1.0;
}
else if (TrustWeight > 1.0)
{
TrustWeight = 1.0;
}
if (CosignIssuer is not null)
{
if (string.IsNullOrWhiteSpace(CosignIdentityPattern))
{
throw new InvalidOperationException("CosignIdentityPattern must be provided when CosignIssuer is specified.");
}
}
}
}

45
src/StellaOps.Vexer.Connectors.RedHat.CSAF/DependencyInjection/RedHatConnectorServiceCollectionExtensions.cs Normal file
View File

@@ -0,0 +1,45 @@
using System.Net;
using Microsoft.Extensions.Caching.Memory;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.DependencyInjection.Extensions;
using StellaOps.Vexer.Connectors.RedHat.CSAF.Configuration;
using StellaOps.Vexer.Connectors.RedHat.CSAF.Metadata;
using StellaOps.Vexer.Core;
using StellaOps.Vexer.Storage.Mongo;
using System.IO.Abstractions;
namespace StellaOps.Vexer.Connectors.RedHat.CSAF.DependencyInjection;
public static class RedHatConnectorServiceCollectionExtensions
{
public static IServiceCollection AddRedHatCsafConnector(this IServiceCollection services, Action<RedHatConnectorOptions>? configure = null)
{
ArgumentNullException.ThrowIfNull(services);
services.AddOptions<RedHatConnectorOptions>()
.Configure(options =>
{
configure?.Invoke(options);
})
.PostConfigure(options => options.Validate());
services.TryAddSingleton<IMemoryCache, MemoryCache>();
services.TryAddSingleton<IFileSystem, FileSystem>();
services.AddHttpClient(RedHatConnectorOptions.HttpClientName, client =>
{
client.Timeout = TimeSpan.FromSeconds(30);
client.DefaultRequestHeaders.UserAgent.ParseAdd("StellaOps.Vexer.Connectors.RedHat/1.0");
client.DefaultRequestHeaders.Accept.ParseAdd("application/json");
})
.ConfigurePrimaryHttpMessageHandler(() => new HttpClientHandler
{
AutomaticDecompression = DecompressionMethods.All,
});
services.AddSingleton<RedHatProviderMetadataLoader>();
services.AddSingleton<IVexConnector, RedHatCsafConnector>();
return services;
}
}

312
src/StellaOps.Vexer.Connectors.RedHat.CSAF/Metadata/RedHatProviderMetadataLoader.cs Normal file
View File

@@ -0,0 +1,312 @@
using System.Collections.Immutable;
using System.Linq;
using System.Net;
using System.Net.Http;
using System.Net.Http.Headers;
using System.Text.Json;
using System.Text.Json.Serialization;
using Microsoft.Extensions.Caching.Memory;
using Microsoft.Extensions.Logging;
using Microsoft.Extensions.Options;
using StellaOps.Vexer.Connectors.RedHat.CSAF.Configuration;
using StellaOps.Vexer.Core;
using System.IO.Abstractions;
namespace StellaOps.Vexer.Connectors.RedHat.CSAF.Metadata;
public sealed class RedHatProviderMetadataLoader
{
public const string CacheKey = "StellaOps.Vexer.Connectors.RedHat.CSAF.Metadata";
private readonly IHttpClientFactory _httpClientFactory;
private readonly IMemoryCache _cache;
private readonly ILogger<RedHatProviderMetadataLoader> _logger;
private readonly RedHatConnectorOptions _options;
private readonly IFileSystem _fileSystem;
private readonly JsonSerializerOptions _serializerOptions;
private readonly SemaphoreSlim _refreshSemaphore = new(1, 1);
public RedHatProviderMetadataLoader(
IHttpClientFactory httpClientFactory,
IMemoryCache memoryCache,
IOptions<RedHatConnectorOptions> options,
ILogger<RedHatProviderMetadataLoader> logger,
IFileSystem? fileSystem = null)
{
_httpClientFactory = httpClientFactory ?? throw new ArgumentNullException(nameof(httpClientFactory));
_cache = memoryCache ?? throw new ArgumentNullException(nameof(memoryCache));
_logger = logger ?? throw new ArgumentNullException(nameof(logger));
_options = (options ?? throw new ArgumentNullException(nameof(options))).Value ?? throw new ArgumentNullException(nameof(options));
_fileSystem = fileSystem ?? new FileSystem();
_serializerOptions = new JsonSerializerOptions(JsonSerializerDefaults.Web)
{
PropertyNameCaseInsensitive = true,
ReadCommentHandling = JsonCommentHandling.Skip,
};
}
public async Task<RedHatProviderMetadataResult> LoadAsync(CancellationToken cancellationToken)
{
if (_cache.TryGetValue<CacheEntry>(CacheKey, out var cached) && cached is { } cachedEntry && !cachedEntry.IsExpired())
{
_logger.LogDebug("Returning cached Red Hat provider metadata (expires {Expires}).", cachedEntry.ExpiresAt);
return new RedHatProviderMetadataResult(cachedEntry.Provider, cachedEntry.FetchedAt, true, cachedEntry.FromOffline);
}
await _refreshSemaphore.WaitAsync(cancellationToken).ConfigureAwait(false);
try
{
if (_cache.TryGetValue<CacheEntry>(CacheKey, out cached) && cached is { } cachedAfterLock && !cachedAfterLock.IsExpired())
{
return new RedHatProviderMetadataResult(cachedAfterLock.Provider, cachedAfterLock.FetchedAt, true, cachedAfterLock.FromOffline);
}
CacheEntry? previous = cached;
// Attempt live fetch unless offline preferred.
if (!_options.PreferOfflineSnapshot)
{
var httpResult = await TryFetchFromNetworkAsync(previous, cancellationToken).ConfigureAwait(false);
if (httpResult is not null)
{
StoreCache(httpResult);
return new RedHatProviderMetadataResult(httpResult.Provider, httpResult.FetchedAt, false, false);
}
}
var offlineResult = TryLoadFromOffline();
if (offlineResult is not null)
{
var offlineEntry = offlineResult with
{
FetchedAt = DateTimeOffset.UtcNow,
ExpiresAt = DateTimeOffset.UtcNow + _options.MetadataCacheDuration,
FromOffline = true,
};
StoreCache(offlineEntry);
return new RedHatProviderMetadataResult(offlineEntry.Provider, offlineEntry.FetchedAt, false, true);
}
throw new InvalidOperationException("Unable to load Red Hat CSAF provider metadata from network or offline snapshot.");
}
finally
{
_refreshSemaphore.Release();
}
}
private void StoreCache(CacheEntry entry)
{
var cacheEntryOptions = new MemoryCacheEntryOptions
{
AbsoluteExpiration = entry.ExpiresAt,
};
_cache.Set(CacheKey, entry, cacheEntryOptions);
}
private async Task<CacheEntry?> TryFetchFromNetworkAsync(CacheEntry? previous, CancellationToken cancellationToken)
{
try
{
var client = _httpClientFactory.CreateClient(RedHatConnectorOptions.HttpClientName);
using var request = new HttpRequestMessage(HttpMethod.Get, _options.MetadataUri);
if (!string.IsNullOrWhiteSpace(previous?.ETag))
{
if (EntityTagHeaderValue.TryParse(previous.ETag, out var etag))
{
request.Headers.IfNoneMatch.Add(etag);
}
}
using var response = await client.SendAsync(request, HttpCompletionOption.ResponseHeadersRead, cancellationToken).ConfigureAwait(false);
if (response.StatusCode == HttpStatusCode.NotModified && previous is not null)
{
_logger.LogDebug("Red Hat provider metadata not modified (etag {ETag}).", previous.ETag);
return previous with
{
FetchedAt = DateTimeOffset.UtcNow,
ExpiresAt = DateTimeOffset.UtcNow + _options.MetadataCacheDuration,
};
}
response.EnsureSuccessStatusCode();
var payload = await response.Content.ReadAsStringAsync(cancellationToken).ConfigureAwait(false);
var provider = ParseAndValidate(payload);
var etagHeader = response.Headers.ETag?.ToString();
if (_options.PersistOfflineSnapshot && !string.IsNullOrWhiteSpace(_options.OfflineSnapshotPath))
{
try
{
_fileSystem.File.WriteAllText(_options.OfflineSnapshotPath, payload);
_logger.LogDebug("Persisted Red Hat metadata snapshot to {Path}.", _options.OfflineSnapshotPath);
}
catch (Exception ex)
{
_logger.LogWarning(ex, "Failed to persist Red Hat metadata snapshot to {Path}.", _options.OfflineSnapshotPath);
}
}
return new CacheEntry(
provider,
DateTimeOffset.UtcNow,
DateTimeOffset.UtcNow + _options.MetadataCacheDuration,
etagHeader,
FromOffline: false);
}
catch (Exception ex) when (ex is not OperationCanceledException && !_options.PreferOfflineSnapshot)
{
_logger.LogWarning(ex, "Failed to fetch Red Hat provider metadata from {Uri}, will attempt offline snapshot.", _options.MetadataUri);
return null;
}
}
private CacheEntry? TryLoadFromOffline()
{
if (string.IsNullOrWhiteSpace(_options.OfflineSnapshotPath))
{
return null;
}
if (!_fileSystem.File.Exists(_options.OfflineSnapshotPath))
{
_logger.LogWarning("Offline snapshot path {Path} does not exist.", _options.OfflineSnapshotPath);
return null;
}
try
{
var payload = _fileSystem.File.ReadAllText(_options.OfflineSnapshotPath);
var provider = ParseAndValidate(payload);
return new CacheEntry(provider, DateTimeOffset.UtcNow, DateTimeOffset.UtcNow + _options.MetadataCacheDuration, ETag: null, FromOffline: true);
}
catch (Exception ex)
{
_logger.LogError(ex, "Failed to load Red Hat provider metadata from offline snapshot {Path}.", _options.OfflineSnapshotPath);
return null;
}
}
private VexProvider ParseAndValidate(string payload)
{
if (string.IsNullOrWhiteSpace(payload))
{
throw new InvalidOperationException("Provider metadata payload was empty.");
}
ProviderMetadataDocument? document;
try
{
document = JsonSerializer.Deserialize<ProviderMetadataDocument>(payload, _serializerOptions);
}
catch (JsonException ex)
{
throw new InvalidOperationException("Provider metadata payload could not be parsed.", ex);
}
if (document is null)
{
throw new InvalidOperationException("Provider metadata payload was null after parsing.");
}
if (document.Metadata?.Provider?.Name is null)
{
throw new InvalidOperationException("Provider metadata missing provider name.");
}
var distributions = document.Distributions?
.Select(static d => d.Directory)
.Where(static s => !string.IsNullOrWhiteSpace(s))
.Select(static s => CreateUri(s!, nameof(ProviderMetadataDistribution.Directory)))
.ToImmutableArray() ?? ImmutableArray<Uri>.Empty;
if (distributions.IsDefaultOrEmpty)
{
throw new InvalidOperationException("Provider metadata did not include any valid distribution directories.");
}
Uri? rolieFeed = null;
if (document.Rolie?.Feeds is not null)
{
foreach (var feed in document.Rolie.Feeds)
{
if (!string.IsNullOrWhiteSpace(feed.Url))
{
rolieFeed = CreateUri(feed.Url, "rolie.feeds[].url");
break;
}
}
}
var trust = BuildTrust();
return new VexProvider(
id: "vexer:redhat",
displayName: document.Metadata.Provider.Name,
kind: VexProviderKind.Distro,
baseUris: distributions,
discovery: new VexProviderDiscovery(_options.MetadataUri, rolieFeed),
trust: trust);
}
private VexProviderTrust BuildTrust()
{
VexCosignTrust? cosign = null;
if (!string.IsNullOrWhiteSpace(_options.CosignIssuer) && !string.IsNullOrWhiteSpace(_options.CosignIdentityPattern))
{
cosign = new VexCosignTrust(_options.CosignIssuer!, _options.CosignIdentityPattern!);
}
return new VexProviderTrust(
_options.TrustWeight,
cosign,
_options.PgpFingerprints);
}
private static Uri CreateUri(string value, string propertyName)
{
if (!Uri.TryCreate(value, UriKind.Absolute, out var uri) || uri.Scheme is not ("http" or "https"))
{
throw new InvalidOperationException($"Provider metadata field '{propertyName}' must be an absolute HTTP(S) URI.");
}
return uri;
}
private sealed record ProviderMetadataDocument(
[property: JsonPropertyName("metadata")] ProviderMetadata? Metadata,
[property: JsonPropertyName("distributions")] IReadOnlyList<ProviderMetadataDistribution>? Distributions,
[property: JsonPropertyName("rolie")] ProviderMetadataRolie? Rolie);
private sealed record ProviderMetadata(
[property: JsonPropertyName("provider")] ProviderMetadataProvider? Provider);
private sealed record ProviderMetadataProvider(
[property: JsonPropertyName("name")] string? Name);
private sealed record ProviderMetadataDistribution(
[property: JsonPropertyName("directory")] string? Directory);
private sealed record ProviderMetadataRolie(
[property: JsonPropertyName("feeds")] IReadOnlyList<ProviderMetadataRolieFeed>? Feeds);
private sealed record ProviderMetadataRolieFeed(
[property: JsonPropertyName("url")] string? Url);
private sealed record CacheEntry(
VexProvider Provider,
DateTimeOffset FetchedAt,
DateTimeOffset ExpiresAt,
string? ETag,
bool FromOffline)
{
public bool IsExpired() => DateTimeOffset.UtcNow >= ExpiresAt;
}
}
public sealed record RedHatProviderMetadataResult(
VexProvider Provider,
DateTimeOffset FetchedAt,
bool FromCache,
bool FromOfflineSnapshot);

186
src/StellaOps.Vexer.Connectors.RedHat.CSAF/RedHatCsafConnector.cs Normal file
View File

@@ -0,0 +1,186 @@
using System.Collections.Generic;
using System.Collections.Immutable;
using System.Linq;
using System.Net.Http;
using System.Runtime.CompilerServices;
using System.Text.Json;
using System.Xml.Linq;
using Microsoft.Extensions.Logging;
using Microsoft.Extensions.Options;
using StellaOps.Vexer.Connectors.Abstractions;
using StellaOps.Vexer.Connectors.RedHat.CSAF.Configuration;
using StellaOps.Vexer.Connectors.RedHat.CSAF.Metadata;
using StellaOps.Vexer.Core;
using StellaOps.Vexer.Storage.Mongo;
namespace StellaOps.Vexer.Connectors.RedHat.CSAF;
public sealed class RedHatCsafConnector : VexConnectorBase
{
private readonly RedHatProviderMetadataLoader _metadataLoader;
private readonly IHttpClientFactory _httpClientFactory;
private readonly IVexConnectorStateRepository _stateRepository;
public RedHatCsafConnector(
VexConnectorDescriptor descriptor,
RedHatProviderMetadataLoader metadataLoader,
IHttpClientFactory httpClientFactory,
IVexConnectorStateRepository stateRepository,
ILogger<RedHatCsafConnector> logger,
TimeProvider timeProvider)
: base(descriptor, logger, timeProvider)
{
_metadataLoader = metadataLoader ?? throw new ArgumentNullException(nameof(metadataLoader));
_httpClientFactory = httpClientFactory ?? throw new ArgumentNullException(nameof(httpClientFactory));
_stateRepository = stateRepository ?? throw new ArgumentNullException(nameof(stateRepository));
}
public override ValueTask ValidateAsync(VexConnectorSettings settings, CancellationToken cancellationToken)
{
// No connector-specific settings yet.
return ValueTask.CompletedTask;
}
public override async IAsyncEnumerable<VexRawDocument> FetchAsync(VexConnectorContext context, [EnumeratorCancellation] CancellationToken cancellationToken)
{
ArgumentNullException.ThrowIfNull(context);
var metadataResult = await _metadataLoader.LoadAsync(cancellationToken).ConfigureAwait(false);
if (metadataResult.Provider.Discovery.RolIeService is null)
{
throw new InvalidOperationException("Red Hat provider metadata did not specify a ROLIE feed.");
}
var state = await _stateRepository.GetAsync(Descriptor.Id, cancellationToken).ConfigureAwait(false);
var sinceTimestamp = context.Since;
if (state?.LastUpdated is { } persisted && (sinceTimestamp is null || persisted > sinceTimestamp))
{
sinceTimestamp = persisted;
}
var knownDigests = state?.DocumentDigests ?? ImmutableArray<string>.Empty;
var digestList = new List<string>(knownDigests);
var digestSet = new HashSet<string>(knownDigests, StringComparer.OrdinalIgnoreCase);
var latestUpdated = state?.LastUpdated ?? sinceTimestamp ?? DateTimeOffset.MinValue;
var stateChanged = false;
foreach (var entry in await FetchRolieEntriesAsync(metadataResult.Provider.Discovery.RolIeService, cancellationToken).ConfigureAwait(false))
{
if (sinceTimestamp is not null && entry.Updated is DateTimeOffset updated && updated <= sinceTimestamp)
{
continue;
}
if (entry.DocumentUri is null)
{
Logger.LogDebug("Skipping ROLIE entry {Id} because no document link was provided.", entry.Id);
continue;
}
var rawDocument = await DownloadCsafDocumentAsync(entry, cancellationToken).ConfigureAwait(false);
if (!digestSet.Add(rawDocument.Digest))
{
Logger.LogDebug("Skipping CSAF document {Uri} because digest {Digest} was already processed.", rawDocument.SourceUri, rawDocument.Digest);
continue;
}
await context.RawSink.StoreAsync(rawDocument, cancellationToken).ConfigureAwait(false);
digestList.Add(rawDocument.Digest);
stateChanged = true;
if (entry.Updated is DateTimeOffset entryUpdated && entryUpdated > latestUpdated)
{
latestUpdated = entryUpdated;
}
yield return rawDocument;
}
if (stateChanged)
{
var newLastUpdated = latestUpdated == DateTimeOffset.MinValue ? state?.LastUpdated : latestUpdated;
var updatedState = new VexConnectorState(
Descriptor.Id,
newLastUpdated,
digestList.ToImmutableArray());
await _stateRepository.SaveAsync(updatedState, cancellationToken).ConfigureAwait(false);
}
}
public override ValueTask<VexClaimBatch> NormalizeAsync(VexRawDocument document, CancellationToken cancellationToken)
{
// This connector relies on format-specific normalizers registered elsewhere.
throw new NotSupportedException("RedHatCsafConnector does not perform in-line normalization; use the CSAF normalizer component.");
}
private async Task<IReadOnlyList<RolieEntry>> FetchRolieEntriesAsync(Uri feedUri, CancellationToken cancellationToken)
{
var client = _httpClientFactory.CreateClient(RedHatConnectorOptions.HttpClientName);
using var response = await client.GetAsync(feedUri, cancellationToken).ConfigureAwait(false);
response.EnsureSuccessStatusCode();
await using var stream = await response.Content.ReadAsStreamAsync(cancellationToken).ConfigureAwait(false);
var document = XDocument.Load(stream);
var ns = document.Root?.Name.Namespace ?? "http://www.w3.org/2005/Atom";
var entries = document.Root?
.Elements(ns + "entry")
.Select(e => new RolieEntry(
Id: (string?)e.Element(ns + "id"),
Updated: ParseUpdated((string?)e.Element(ns + "updated")),
DocumentUri: ParseDocumentLink(e, ns)))
.Where(entry => entry.Id is not null && entry.Updated is not null)
.OrderBy(entry => entry.Updated)
.ToList() ?? new List<RolieEntry>();
return entries;
}
private static DateTimeOffset? ParseUpdated(string? value)
=> DateTimeOffset.TryParse(value, out var parsed) ? parsed : null;
private static Uri? ParseDocumentLink(XElement entry, XNamespace ns)
{
var linkElements = entry.Elements(ns + "link");
foreach (var link in linkElements)
{
var rel = (string?)link.Attribute("rel");
var href = (string?)link.Attribute("href");
if (string.IsNullOrWhiteSpace(href))
{
continue;
}
if (rel is null || rel.Equals("enclosure", StringComparison.OrdinalIgnoreCase) || rel.Equals("alternate", StringComparison.OrdinalIgnoreCase))
{
if (Uri.TryCreate(href, UriKind.Absolute, out var uri))
{
return uri;
}
}
}
return null;
}
private async Task<VexRawDocument> DownloadCsafDocumentAsync(RolieEntry entry, CancellationToken cancellationToken)
{
var documentUri = entry.DocumentUri ?? throw new InvalidOperationException("ROLIE entry missing document URI.");
var client = _httpClientFactory.CreateClient(RedHatConnectorOptions.HttpClientName);
using var response = await client.GetAsync(documentUri, cancellationToken).ConfigureAwait(false);
response.EnsureSuccessStatusCode();
var contentBytes = await response.Content.ReadAsByteArrayAsync(cancellationToken).ConfigureAwait(false);
var metadata = BuildMetadata(builder => builder
.Add("redhat.csaf.entryId", entry.Id)
.Add("redhat.csaf.documentUri", documentUri.ToString())
.Add("redhat.csaf.updated", entry.Updated?.ToString("O")));
return CreateRawDocument(VexDocumentFormat.Csaf, documentUri, contentBytes, metadata);
}
private sealed record RolieEntry(string? Id, DateTimeOffset? Updated, Uri? DocumentUri);
}

19
src/StellaOps.Vexer.Connectors.RedHat.CSAF/StellaOps.Vexer.Connectors.RedHat.CSAF.csproj Normal file
View File

@@ -0,0 +1,19 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>net10.0</TargetFramework>
<LangVersion>preview</LangVersion>
<Nullable>enable</Nullable>
<ImplicitUsings>enable</ImplicitUsings>
<TreatWarningsAsErrors>true</TreatWarningsAsErrors>
</PropertyGroup>
<ItemGroup>
<ProjectReference Include="..\StellaOps.Vexer.Connectors.Abstractions\StellaOps.Vexer.Connectors.Abstractions.csproj" />
<ProjectReference Include="..\StellaOps.Vexer.Storage.Mongo\StellaOps.Vexer.Storage.Mongo.csproj" />
</ItemGroup>
<ItemGroup>
<PackageReference Include="Microsoft.Extensions.Caching.Memory" Version="8.0.0" />
<PackageReference Include="Microsoft.Extensions.Http" Version="8.0.0" />
<PackageReference Include="Microsoft.Extensions.Options" Version="8.0.0" />
<PackageReference Include="System.IO.Abstractions" Version="20.0.28" />
</ItemGroup>
</Project>

9
src/StellaOps.Vexer.Connectors.RedHat.CSAF/TASKS.md
View File

@@ -2,6 +2,9 @@ If you are working on this file you need to read docs/ARCHITECTURE_VEXER.md and
# TASKS # TASKS
| Task | Owner(s) | Depends on | Notes | | Task | Owner(s) | Depends on | Notes |
|---|---|---|---| |---|---|---|---|
|VEXER-CONN-RH-01-001 – Provider metadata discovery|Team Vexer Connectors – Red Hat|VEXER-CONN-ABS-01-001|TODO – Implement `.well-known` metadata loader with caching, schema validation, and offline snapshot support.| |VEXER-CONN-RH-01-001 – Provider metadata discovery|Team Vexer Connectors – Red Hat|VEXER-CONN-ABS-01-001|**DONE (2025-10-17)** – Added `RedHatProviderMetadataLoader` with HTTP/ETag caching, offline snapshot handling, and validation; exposed DI helper + tests covering live, cached, and offline scenarios.|
|VEXER-CONN-RH-01-002 – Incremental CSAF pulls|Team Vexer Connectors – Red Hat|VEXER-CONN-RH-01-001, VEXER-STORAGE-01-003|TODO – Fetch CSAF windows with ETag handling, resume tokens, quarantine on schema errors, and persist raw docs.| |VEXER-CONN-RH-01-002 – Incremental CSAF pulls|Team Vexer Connectors – Red Hat|VEXER-CONN-RH-01-001, VEXER-STORAGE-01-003|**DONE (2025-10-17)** – Implemented `RedHatCsafConnector` with ROLIE feed parsing, incremental filtering via `context.Since`, CSAF document download + metadata capture, and persistence through `IVexRawDocumentSink`; tests cover live fetch/cache/offline scenarios with ETag handling.|
|VEXER-CONN-RH-01-003 – Trust metadata emission|Team Vexer Connectors – Red Hat|VEXER-CONN-RH-01-002, VEXER-POLICY-01-001|TODO – Populate provider trust overrides (cosign issuer, identity regex) and provenance hints for policy evaluation/logging.| |VEXER-CONN-RH-01-003 – Trust metadata emission|Team Vexer Connectors – Red Hat|VEXER-CONN-RH-01-002, VEXER-POLICY-01-001|**DONE (2025-10-17)** – Provider metadata loader now emits trust overrides (weight, cosign issuer/pattern, PGP fingerprints) and the connector surfaces provenance hints for policy/consensus layers.|
|VEXER-CONN-RH-01-004 – Resume state persistence|Team Vexer Connectors – Red Hat|VEXER-CONN-RH-01-002, VEXER-STORAGE-01-003|**DONE (2025-10-17)** – Connector now loads/saves resume state via `IVexConnectorStateRepository`, tracking last update timestamp and recent document digests to avoid duplicate CSAF ingestion; regression covers state persistence and duplicate skips.|
|VEXER-CONN-RH-01-005 – Worker/WebService integration|Team Vexer Connectors – Red Hat|VEXER-CONN-RH-01-002|**DONE (2025-10-17)** – Worker/WebService now call `AddRedHatCsafConnector`, register the connector + state repo, and default worker scheduling adds the `vexer:redhat` provider so background jobs and orchestration can activate the connector without extra wiring.|
|VEXER-CONN-RH-01-006 – CSAF normalization parity tests|Team Vexer Connectors – Red Hat|VEXER-CONN-RH-01-002, VEXER-FMT-CSAF-01-001|**DONE (2025-10-17)** – Added RHSA fixture-driven regression verifying CSAF normalizer retains Red Hat product metadata, tracking fields, and timestamps (`rhsa-sample.json` + `CsafNormalizerTests.NormalizeAsync_PreservesRedHatSpecificMetadata`).|

138
src/StellaOps.Vexer.Connectors.SUSE.RancherVEXHub.Tests/Authentication/RancherHubTokenProviderTests.cs Normal file
View File

@@ -0,0 +1,138 @@
using System;
using System.Net;
using System.Net.Http;
using System.Text;
using System.Threading;
using FluentAssertions;
using Microsoft.Extensions.Caching.Memory;
using Microsoft.Extensions.Logging.Abstractions;
using StellaOps.Vexer.Connectors.SUSE.RancherVEXHub.Authentication;
using StellaOps.Vexer.Connectors.SUSE.RancherVEXHub.Configuration;
using Xunit;
namespace StellaOps.Vexer.Connectors.SUSE.RancherVEXHub.Tests.Authentication;
public sealed class RancherHubTokenProviderTests
{
private const string TokenResponse = "{\"access_token\":\"abc123\",\"token_type\":\"Bearer\",\"expires_in\":3600}";
[Fact]
public async Task GetAccessTokenAsync_RequestsAndCachesToken()
{
var handler = TestHttpMessageHandler.RespondWith(request =>
{
request.Headers.Authorization.Should().NotBeNull();
request.Content.Should().NotBeNull();
return new HttpResponseMessage(HttpStatusCode.OK)
{
Content = new StringContent(TokenResponse, Encoding.UTF8, "application/json"),
};
});
var client = new HttpClient(handler)
{
BaseAddress = new Uri("https://identity.suse.com"),
};
var factory = new SingleClientHttpClientFactory(client);
var cache = new MemoryCache(new MemoryCacheOptions());
var provider = new RancherHubTokenProvider(factory, cache, NullLogger<RancherHubTokenProvider>.Instance);
var options = new RancherHubConnectorOptions
{
ClientId = "client",
ClientSecret = "secret",
TokenEndpoint = new Uri("https://identity.suse.com/oauth/token"),
Audience = "https://vexhub.suse.com",
};
options.Scopes.Clear();
options.Scopes.Add("hub.read");
options.Scopes.Add("hub.events");
var token = await provider.GetAccessTokenAsync(options, CancellationToken.None);
token.Should().NotBeNull();
token!.Value.Should().Be("abc123");
var cached = await provider.GetAccessTokenAsync(options, CancellationToken.None);
cached.Should().NotBeNull();
handler.InvocationCount.Should().Be(1);
}
[Fact]
public async Task GetAccessTokenAsync_ReturnsNullWhenOfflinePreferred()
{
var handler = TestHttpMessageHandler.RespondWith(_ => new HttpResponseMessage(HttpStatusCode.OK));
var client = new HttpClient(handler)
{
BaseAddress = new Uri("https://identity.suse.com"),
};
var factory = new SingleClientHttpClientFactory(client);
var cache = new MemoryCache(new MemoryCacheOptions());
var provider = new RancherHubTokenProvider(factory, cache, NullLogger<RancherHubTokenProvider>.Instance);
var options = new RancherHubConnectorOptions
{
PreferOfflineSnapshot = true,
ClientId = "client",
ClientSecret = "secret",
TokenEndpoint = new Uri("https://identity.suse.com/oauth/token"),
};
var token = await provider.GetAccessTokenAsync(options, CancellationToken.None);
token.Should().BeNull();
handler.InvocationCount.Should().Be(0);
}
[Fact]
public async Task GetAccessTokenAsync_ReturnsNullWithoutCredentials()
{
var handler = TestHttpMessageHandler.RespondWith(_ => new HttpResponseMessage(HttpStatusCode.OK));
var client = new HttpClient(handler)
{
BaseAddress = new Uri("https://identity.suse.com"),
};
var factory = new SingleClientHttpClientFactory(client);
var cache = new MemoryCache(new MemoryCacheOptions());
var provider = new RancherHubTokenProvider(factory, cache, NullLogger<RancherHubTokenProvider>.Instance);
var options = new RancherHubConnectorOptions();
var token = await provider.GetAccessTokenAsync(options, CancellationToken.None);
token.Should().BeNull();
handler.InvocationCount.Should().Be(0);
}
private sealed class SingleClientHttpClientFactory : IHttpClientFactory
{
private readonly HttpClient _client;
public SingleClientHttpClientFactory(HttpClient client)
{
_client = client;
}
public HttpClient CreateClient(string name) => _client;
}
private sealed class TestHttpMessageHandler : HttpMessageHandler
{
private readonly Func<HttpRequestMessage, HttpResponseMessage> _responseFactory;
private TestHttpMessageHandler(Func<HttpRequestMessage, HttpResponseMessage> responseFactory)
{
_responseFactory = responseFactory;
}
public int InvocationCount { get; private set; }
public static TestHttpMessageHandler RespondWith(Func<HttpRequestMessage, HttpResponseMessage> responseFactory)
=> new(responseFactory);
protected override Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
{
InvocationCount++;
return Task.FromResult(_responseFactory(request));
}
}
}

178
src/StellaOps.Vexer.Connectors.SUSE.RancherVEXHub.Tests/Metadata/RancherHubMetadataLoaderTests.cs Normal file
View File

@@ -0,0 +1,178 @@
using System;
using System.Collections.Generic;
using System.Net;
using System.Net.Http;
using System.Net.Http.Headers;
using System.Text;
using FluentAssertions;
using Microsoft.Extensions.Caching.Memory;
using Microsoft.Extensions.Logging.Abstractions;
using StellaOps.Vexer.Connectors.SUSE.RancherVEXHub.Authentication;
using StellaOps.Vexer.Connectors.SUSE.RancherVEXHub.Configuration;
using StellaOps.Vexer.Connectors.SUSE.RancherVEXHub.Metadata;
using System.IO.Abstractions.TestingHelpers;
using System.Threading;
using Xunit;
namespace StellaOps.Vexer.Connectors.SUSE.RancherVEXHub.Tests.Metadata;
public sealed class RancherHubMetadataLoaderTests
{
private const string SampleDiscovery = """
{
"hubId": "vexer:suse.rancher",
"title": "SUSE Rancher VEX Hub",
"subscription": {
"eventsUri": "https://vexhub.suse.com/api/v1/events",
"checkpointUri": "https://vexhub.suse.com/api/v1/checkpoints",
"requiresAuthentication": true,
"channels": ["rke2", "k3s"],
"scopes": ["hub.read", "hub.events"]
},
"authentication": {
"tokenUri": "https://identity.suse.com/oauth2/token",
"audience": "https://vexhub.suse.com"
},
"offline": {
"snapshotUri": "https://downloads.suse.com/vexhub/snapshot.json",
"sha256": "deadbeef",
"updated": "2025-10-10T12:00:00Z"
}
}
""";
[Fact]
public async Task LoadAsync_FetchesAndCachesMetadata()
{
var handler = TestHttpMessageHandler.RespondWith(_ =>
{
var response = new HttpResponseMessage(HttpStatusCode.OK)
{
Content = new StringContent(SampleDiscovery, Encoding.UTF8, "application/json"),
};
response.Headers.ETag = new EntityTagHeaderValue("\"abc\"");
return response;
});
var client = new HttpClient(handler)
{
BaseAddress = new Uri("https://vexhub.suse.com"),
};
var factory = new SingleClientHttpClientFactory(client);
var cache = new MemoryCache(new MemoryCacheOptions());
var fileSystem = new MockFileSystem();
var offlinePath = fileSystem.Path.Combine(@"C:\offline", "rancher-hub.json");
var options = new RancherHubConnectorOptions
{
DiscoveryUri = new Uri("https://vexhub.suse.com/.well-known/vex/rancher-hub.json"),
OfflineSnapshotPath = offlinePath,
};
var tokenProvider = new RancherHubTokenProvider(factory, cache, NullLogger<RancherHubTokenProvider>.Instance);
var loader = new RancherHubMetadataLoader(factory, cache, tokenProvider, fileSystem, NullLogger<RancherHubMetadataLoader>.Instance);
var result = await loader.LoadAsync(options, CancellationToken.None);
result.FromCache.Should().BeFalse();
result.FromOfflineSnapshot.Should().BeFalse();
result.Metadata.Provider.DisplayName.Should().Be("SUSE Rancher VEX Hub");
result.Metadata.Subscription.EventsUri.Should().Be(new Uri("https://vexhub.suse.com/api/v1/events"));
result.Metadata.Authentication.TokenEndpoint.Should().Be(new Uri("https://identity.suse.com/oauth2/token"));
// Second call should be served from cache (no additional HTTP invocation).
handler.ResetInvocationCount();
await loader.LoadAsync(options, CancellationToken.None);
handler.InvocationCount.Should().Be(0);
}
[Fact]
public async Task LoadAsync_UsesOfflineSnapshotWhenNetworkFails()
{
var handler = TestHttpMessageHandler.RespondWith(_ => throw new HttpRequestException("network down"));
var client = new HttpClient(handler)
{
BaseAddress = new Uri("https://vexhub.suse.com"),
};
var factory = new SingleClientHttpClientFactory(client);
var cache = new MemoryCache(new MemoryCacheOptions());
var fileSystem = new MockFileSystem();
var offlinePath = fileSystem.Path.Combine(@"C:\offline", "rancher-hub.json");
fileSystem.AddFile(offlinePath, new MockFileData(SampleDiscovery));
var options = new RancherHubConnectorOptions
{
DiscoveryUri = new Uri("https://vexhub.suse.com/.well-known/vex/rancher-hub.json"),
OfflineSnapshotPath = offlinePath,
};
var tokenProvider = new RancherHubTokenProvider(factory, cache, NullLogger<RancherHubTokenProvider>.Instance);
var loader = new RancherHubMetadataLoader(factory, cache, tokenProvider, fileSystem, NullLogger<RancherHubMetadataLoader>.Instance);
var result = await loader.LoadAsync(options, CancellationToken.None);
result.FromOfflineSnapshot.Should().BeTrue();
result.Metadata.Subscription.RequiresAuthentication.Should().BeTrue();
result.Metadata.OfflineSnapshot.Should().NotBeNull();
}
[Fact]
public async Task LoadAsync_ThrowsWhenOfflinePreferredButMissing()
{
var handler = TestHttpMessageHandler.RespondWith(_ => throw new HttpRequestException("network down"));
var client = new HttpClient(handler)
{
BaseAddress = new Uri("https://vexhub.suse.com"),
};
var factory = new SingleClientHttpClientFactory(client);
var cache = new MemoryCache(new MemoryCacheOptions());
var fileSystem = new MockFileSystem();
var options = new RancherHubConnectorOptions
{
DiscoveryUri = new Uri("https://vexhub.suse.com/.well-known/vex/rancher-hub.json"),
OfflineSnapshotPath = "/offline/missing.json",
PreferOfflineSnapshot = true,
};
var tokenProvider = new RancherHubTokenProvider(factory, cache, NullLogger<RancherHubTokenProvider>.Instance);
var loader = new RancherHubMetadataLoader(factory, cache, tokenProvider, fileSystem, NullLogger<RancherHubMetadataLoader>.Instance);
await Assert.ThrowsAsync<InvalidOperationException>(() => loader.LoadAsync(options, CancellationToken.None));
}
private sealed class SingleClientHttpClientFactory : IHttpClientFactory
{
private readonly HttpClient _client;
public SingleClientHttpClientFactory(HttpClient client)
{
_client = client;
}
public HttpClient CreateClient(string name) => _client;
}
private sealed class TestHttpMessageHandler : HttpMessageHandler
{
private readonly Func<HttpRequestMessage, HttpResponseMessage> _responseFactory;
private TestHttpMessageHandler(Func<HttpRequestMessage, HttpResponseMessage> responseFactory)
{
_responseFactory = responseFactory;
}
public int InvocationCount { get; private set; }
public static TestHttpMessageHandler RespondWith(Func<HttpRequestMessage, HttpResponseMessage> responseFactory)
=> new(responseFactory);
public void ResetInvocationCount() => InvocationCount = 0;
protected override Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
{
InvocationCount++;
return Task.FromResult(_responseFactory(request));
}
}
}

17
src/StellaOps.Vexer.Connectors.SUSE.RancherVEXHub.Tests/StellaOps.Vexer.Connectors.SUSE.RancherVEXHub.Tests.csproj Normal file
View File

@@ -0,0 +1,17 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>net10.0</TargetFramework>
<LangVersion>preview</LangVersion>
<Nullable>enable</Nullable>
<ImplicitUsings>enable</ImplicitUsings>
<TreatWarningsAsErrors>true</TreatWarningsAsErrors>
</PropertyGroup>
<ItemGroup>
<ProjectReference Include="..\StellaOps.Vexer.Connectors.SUSE.RancherVEXHub\StellaOps.Vexer.Connectors.SUSE.RancherVEXHub.csproj" />
<ProjectReference Include="..\StellaOps.Vexer.Storage.Mongo\StellaOps.Vexer.Storage.Mongo.csproj" />
</ItemGroup>
<ItemGroup>
<PackageReference Include="FluentAssertions" Version="6.12.0" />
<PackageReference Include="System.IO.Abstractions.TestingHelpers" Version="20.0.28" />
</ItemGroup>
</Project>

171
src/StellaOps.Vexer.Connectors.SUSE.RancherVEXHub/Authentication/RancherHubTokenProvider.cs Normal file
View File

@@ -0,0 +1,171 @@
using System;
using System.Collections.Generic;
using System.Net.Http;
using System.Net.Http.Headers;
using System.Text;
using System.Text.Json;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.Extensions.Caching.Memory;
using Microsoft.Extensions.Logging;
using StellaOps.Vexer.Connectors.SUSE.RancherVEXHub.Configuration;
namespace StellaOps.Vexer.Connectors.SUSE.RancherVEXHub.Authentication;
public sealed class RancherHubTokenProvider
{
private const string CachePrefix = "StellaOps.Vexer.Connectors.SUSE.RancherVEXHub.Token";
private readonly IHttpClientFactory _httpClientFactory;
private readonly IMemoryCache _cache;
private readonly ILogger<RancherHubTokenProvider> _logger;
private readonly SemaphoreSlim _semaphore = new(1, 1);
public RancherHubTokenProvider(IHttpClientFactory httpClientFactory, IMemoryCache cache, ILogger<RancherHubTokenProvider> logger)
{
_httpClientFactory = httpClientFactory ?? throw new ArgumentNullException(nameof(httpClientFactory));
_cache = cache ?? throw new ArgumentNullException(nameof(cache));
_logger = logger ?? throw new ArgumentNullException(nameof(logger));
}
public async ValueTask<RancherHubAccessToken?> GetAccessTokenAsync(RancherHubConnectorOptions options, CancellationToken cancellationToken)
{
ArgumentNullException.ThrowIfNull(options);
if (options.PreferOfflineSnapshot)
{
_logger.LogDebug("Skipping token request because PreferOfflineSnapshot is enabled.");
return null;
}
var hasCredentials = !string.IsNullOrWhiteSpace(options.ClientId) &&
!string.IsNullOrWhiteSpace(options.ClientSecret) &&
options.TokenEndpoint is not null;
if (!hasCredentials)
{
if (!options.AllowAnonymousDiscovery)
{
_logger.LogDebug("No Rancher hub credentials configured; proceeding without Authorization header.");
}
return null;
}
var cacheKey = $"{CachePrefix}:{options.ClientId}";
if (_cache.TryGetValue<RancherHubAccessToken>(cacheKey, out var cachedToken) && cachedToken is not null && !cachedToken.IsExpired())
{
return cachedToken;
}
await _semaphore.WaitAsync(cancellationToken).ConfigureAwait(false);
try
{
if (_cache.TryGetValue<RancherHubAccessToken>(cacheKey, out cachedToken) && cachedToken is not null && !cachedToken.IsExpired())
{
return cachedToken;
}
var token = await RequestTokenAsync(options, cancellationToken).ConfigureAwait(false);
if (token is not null)
{
var lifetime = token.ExpiresAt - DateTimeOffset.UtcNow;
if (lifetime <= TimeSpan.Zero)
{
lifetime = TimeSpan.FromMinutes(5);
}
var absoluteExpiration = lifetime > TimeSpan.FromSeconds(30)
? DateTimeOffset.UtcNow + lifetime - TimeSpan.FromSeconds(30)
: DateTimeOffset.UtcNow + TimeSpan.FromSeconds(10);
_cache.Set(cacheKey, token, new MemoryCacheEntryOptions
{
AbsoluteExpiration = absoluteExpiration,
});
}
return token;
}
finally
{
_semaphore.Release();
}
}
private async Task<RancherHubAccessToken?> RequestTokenAsync(RancherHubConnectorOptions options, CancellationToken cancellationToken)
{
using var request = new HttpRequestMessage(HttpMethod.Post, options.TokenEndpoint);
request.Headers.Accept.ParseAdd("application/json");
var parameters = new Dictionary<string, string>
{
["grant_type"] = "client_credentials",
};
if (options.Scopes.Count > 0)
{
parameters["scope"] = string.Join(' ', options.Scopes);
}
if (!string.IsNullOrWhiteSpace(options.Audience))
{
parameters["audience"] = options.Audience!;
}
if (string.Equals(options.ClientAuthenticationScheme, "client_secret_basic", StringComparison.Ordinal))
{
var credentials = Convert.ToBase64String(Encoding.UTF8.GetBytes($"{options.ClientId}:{options.ClientSecret}"));
request.Headers.Authorization = new AuthenticationHeaderValue("Basic", credentials);
}
else
{
parameters["client_id"] = options.ClientId!;
parameters["client_secret"] = options.ClientSecret!;
}
request.Content = new FormUrlEncodedContent(parameters);
var client = _httpClientFactory.CreateClient(RancherHubConnectorOptions.HttpClientName);
using var response = await client.SendAsync(request, cancellationToken).ConfigureAwait(false);
if (!response.IsSuccessStatusCode)
{
var payload = await response.Content.ReadAsStringAsync(cancellationToken).ConfigureAwait(false);
throw new InvalidOperationException($"Failed to acquire Rancher hub access token ({response.StatusCode}): {payload}");
}
await using var stream = await response.Content.ReadAsStreamAsync(cancellationToken).ConfigureAwait(false);
using var document = await JsonDocument.ParseAsync(stream, cancellationToken: cancellationToken).ConfigureAwait(false);
var root = document.RootElement;
if (!root.TryGetProperty("access_token", out var accessTokenProperty) || accessTokenProperty.ValueKind is not JsonValueKind.String)
{
throw new InvalidOperationException("Token endpoint response missing access_token.");
}
var token = accessTokenProperty.GetString();
if (string.IsNullOrWhiteSpace(token))
{
throw new InvalidOperationException("Token endpoint response contained an empty access_token.");
}
var tokenType = root.TryGetProperty("token_type", out var tokenTypeElement) && tokenTypeElement.ValueKind is JsonValueKind.String
? tokenTypeElement.GetString() ?? "Bearer"
: "Bearer";
var expires = root.TryGetProperty("expires_in", out var expiresElement) &&
expiresElement.ValueKind is JsonValueKind.Number &&
expiresElement.TryGetInt32(out var expiresSeconds)
? DateTimeOffset.UtcNow + TimeSpan.FromSeconds(Math.Max(30, expiresSeconds))
: DateTimeOffset.UtcNow + TimeSpan.FromMinutes(30);
_logger.LogDebug("Acquired Rancher hub access token (expires {Expires}).", expires);
return new RancherHubAccessToken(token, tokenType, expires);
}
}
public sealed record RancherHubAccessToken(string Value, string TokenType, DateTimeOffset ExpiresAt)
{
public bool IsExpired() => DateTimeOffset.UtcNow >= ExpiresAt - TimeSpan.FromMinutes(1);
}

186
src/StellaOps.Vexer.Connectors.SUSE.RancherVEXHub/Configuration/RancherHubConnectorOptions.cs Normal file
View File

@@ -0,0 +1,186 @@
using System;
using System.Collections.Generic;
using System.IO;
using System.IO.Abstractions;
namespace StellaOps.Vexer.Connectors.SUSE.RancherVEXHub.Configuration;
public sealed class RancherHubConnectorOptions
{
public static readonly Uri DefaultDiscoveryUri = new("https://vexhub.suse.com/.well-known/vex/rancher-hub.json");
/// <summary>
/// HTTP client name registered for the connector.
/// </summary>
public const string HttpClientName = "vexer.connector.suse.rancherhub";
/// <summary>
/// URI for the Rancher VEX hub discovery document.
/// </summary>
public Uri DiscoveryUri { get; set; } = DefaultDiscoveryUri;
/// <summary>
/// Optional OAuth2/OIDC token endpoint used for hub authentication.
/// </summary>
public Uri? TokenEndpoint { get; set; }
/// <summary>
/// Client identifier used when requesting hub access tokens.
/// </summary>
public string? ClientId { get; set; }
/// <summary>
/// Client secret used when requesting hub access tokens.
/// </summary>
public string? ClientSecret { get; set; }
/// <summary>
/// OAuth scopes requested for hub access; defaults align with Rancher hub reader role.
/// </summary>
public IList<string> Scopes { get; } = new List<string> { "hub.read" };
/// <summary>
/// Optional audience claim passed when requesting tokens (client credential grant).
/// </summary>
public string? Audience { get; set; }
/// <summary>
/// Preferred authentication scheme. Supported: client_secret_basic (default) or client_secret_post.
/// </summary>
public string ClientAuthenticationScheme { get; set; } = "client_secret_basic";
/// <summary>
/// Duration to cache discovery metadata before re-fetching.
/// </summary>
public TimeSpan MetadataCacheDuration { get; set; } = TimeSpan.FromMinutes(30);
/// <summary>
/// Optional file path for discovery metadata snapshots.
/// </summary>
public string? OfflineSnapshotPath { get; set; }
/// <summary>
/// When true, the loader prefers the offline snapshot prior to attempting network discovery.
/// </summary>
public bool PreferOfflineSnapshot { get; set; }
/// <summary>
/// Enables persisting freshly fetched discovery documents to <see cref="OfflineSnapshotPath"/>.
/// </summary>
public bool PersistOfflineSnapshot { get; set; } = true;
/// <summary>
/// Weight applied to the provider entry; hubs default below direct vendor feeds.
/// </summary>
public double TrustWeight { get; set; } = 0.6;
/// <summary>
/// Optional Sigstore/Cosign issuer for verifying hub-delivered attestations.
/// </summary>
public string? CosignIssuer { get; set; }
/// <summary>
/// Cosign identity pattern matched against transparency log subjects.
/// </summary>
public string? CosignIdentityPattern { get; set; }
/// <summary>
/// Additional trusted PGP fingerprints declared by the hub.
/// </summary>
public IList<string> PgpFingerprints { get; } = new List<string>();
/// <summary>
/// Allows falling back to unauthenticated discovery requests when credentials are absent.
/// </summary>
public bool AllowAnonymousDiscovery { get; set; }
public void Validate(IFileSystem? fileSystem = null)
{
if (DiscoveryUri is null || !DiscoveryUri.IsAbsoluteUri)
{
throw new InvalidOperationException("DiscoveryUri must be an absolute URI.");
}
if (DiscoveryUri.Scheme is not ("http" or "https"))
{
throw new InvalidOperationException("DiscoveryUri must use HTTP or HTTPS.");
}
if (MetadataCacheDuration <= TimeSpan.Zero)
{
throw new InvalidOperationException("MetadataCacheDuration must be positive.");
}
if (!string.IsNullOrWhiteSpace(OfflineSnapshotPath))
{
var fs = fileSystem ?? new FileSystem();
var directory = Path.GetDirectoryName(OfflineSnapshotPath);
if (!string.IsNullOrWhiteSpace(directory) && !fs.Directory.Exists(directory))
{
fs.Directory.CreateDirectory(directory);
}
}
if (PreferOfflineSnapshot && string.IsNullOrWhiteSpace(OfflineSnapshotPath))
{
throw new InvalidOperationException("OfflineSnapshotPath must be provided when PreferOfflineSnapshot is enabled.");
}
var hasClientId = !string.IsNullOrWhiteSpace(ClientId);
var hasClientSecret = !string.IsNullOrWhiteSpace(ClientSecret);
var hasTokenEndpoint = TokenEndpoint is not null;
if (hasClientId || hasClientSecret || hasTokenEndpoint)
{
if (!(hasClientId && hasClientSecret && hasTokenEndpoint))
{
throw new InvalidOperationException("ClientId, ClientSecret, and TokenEndpoint must be provided together for authenticated discovery.");
}
if (TokenEndpoint is not null && (!TokenEndpoint.IsAbsoluteUri || TokenEndpoint.Scheme is not ("http" or "https")))
{
throw new InvalidOperationException("TokenEndpoint must be an absolute HTTP(S) URI.");
}
}
if (double.IsNaN(TrustWeight) || double.IsInfinity(TrustWeight))
{
TrustWeight = 0.6;
}
else if (TrustWeight <= 0)
{
TrustWeight = 0.1;
}
else if (TrustWeight > 1.0)
{
TrustWeight = 1.0;
}
if (!string.IsNullOrWhiteSpace(CosignIssuer) && string.IsNullOrWhiteSpace(CosignIdentityPattern))
{
throw new InvalidOperationException("CosignIdentityPattern must be provided when CosignIssuer is specified.");
}
if (!string.Equals(ClientAuthenticationScheme, "client_secret_basic", StringComparison.Ordinal) &&
!string.Equals(ClientAuthenticationScheme, "client_secret_post", StringComparison.Ordinal))
{
throw new InvalidOperationException("ClientAuthenticationScheme must be 'client_secret_basic' or 'client_secret_post'.");
}
// Remove any empty scopes to avoid token request issues.
if (Scopes.Count > 0)
{
for (var i = Scopes.Count - 1; i >= 0; i--)
{
if (string.IsNullOrWhiteSpace(Scopes[i]))
{
Scopes.RemoveAt(i);
}
}
}
if (Scopes.Count == 0)
{
Scopes.Add("hub.read");
}
}
}

32
src/StellaOps.Vexer.Connectors.SUSE.RancherVEXHub/Configuration/RancherHubConnectorOptionsValidator.cs Normal file
View File

@@ -0,0 +1,32 @@
using System;
using System.Collections.Generic;
using System.IO.Abstractions;
using StellaOps.Vexer.Connectors.Abstractions;
namespace StellaOps.Vexer.Connectors.SUSE.RancherVEXHub.Configuration;
public sealed class RancherHubConnectorOptionsValidator : IVexConnectorOptionsValidator<RancherHubConnectorOptions>
{
private readonly IFileSystem _fileSystem;
public RancherHubConnectorOptionsValidator(IFileSystem fileSystem)
{
_fileSystem = fileSystem ?? throw new ArgumentNullException(nameof(fileSystem));
}
public void Validate(VexConnectorDescriptor descriptor, RancherHubConnectorOptions options, IList<string> errors)
{
ArgumentNullException.ThrowIfNull(descriptor);
ArgumentNullException.ThrowIfNull(options);
ArgumentNullException.ThrowIfNull(errors);
try
{
options.Validate(_fileSystem);
}
catch (Exception ex)
{
errors.Add(ex.Message);
}
}
}

49
src/StellaOps.Vexer.Connectors.SUSE.RancherVEXHub/DependencyInjection/RancherHubConnectorServiceCollectionExtensions.cs Normal file
View File

@@ -0,0 +1,49 @@
using System;
using System.Net;
using System.Net.Http;
using Microsoft.Extensions.Caching.Memory;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.DependencyInjection.Extensions;
using StellaOps.Vexer.Connectors.Abstractions;
using StellaOps.Vexer.Connectors.SUSE.RancherVEXHub.Authentication;
using StellaOps.Vexer.Connectors.SUSE.RancherVEXHub.Configuration;
using StellaOps.Vexer.Connectors.SUSE.RancherVEXHub.Metadata;
using StellaOps.Vexer.Core;
using System.IO.Abstractions;
namespace StellaOps.Vexer.Connectors.SUSE.RancherVEXHub.DependencyInjection;
public static class RancherHubConnectorServiceCollectionExtensions
{
public static IServiceCollection AddRancherHubConnector(this IServiceCollection services, Action<RancherHubConnectorOptions>? configure = null)
{
ArgumentNullException.ThrowIfNull(services);
services.TryAddSingleton<IMemoryCache, MemoryCache>();
services.TryAddSingleton<IFileSystem, FileSystem>();
services.AddOptions<RancherHubConnectorOptions>()
.Configure(options =>
{
configure?.Invoke(options);
});
services.AddSingleton<IVexConnectorOptionsValidator<RancherHubConnectorOptions>, RancherHubConnectorOptionsValidator>();
services.AddSingleton<RancherHubTokenProvider>();
services.AddSingleton<RancherHubMetadataLoader>();
services.AddSingleton<IVexConnector, RancherHubConnector>();
services.AddHttpClient(RancherHubConnectorOptions.HttpClientName, client =>
{
client.Timeout = TimeSpan.FromSeconds(30);
client.DefaultRequestHeaders.UserAgent.ParseAdd("StellaOps.Vexer.Connectors.SUSE.RancherVEXHub/1.0");
client.DefaultRequestHeaders.Accept.ParseAdd("application/json");
})
.ConfigurePrimaryHttpMessageHandler(() => new HttpClientHandler
{
AutomaticDecompression = DecompressionMethods.All,
});
return services;
}
}

455
src/StellaOps.Vexer.Connectors.SUSE.RancherVEXHub/Metadata/RancherHubMetadataLoader.cs Normal file
View File

@@ -0,0 +1,455 @@
using System;
using System.Collections.Generic;
using System.Collections.Immutable;
using System.IO.Abstractions;
using System.Net;
using System.Net.Http;
using System.Net.Http.Headers;
using System.Text.Json;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.Extensions.Caching.Memory;
using Microsoft.Extensions.Logging;
using StellaOps.Vexer.Connectors.SUSE.RancherVEXHub.Authentication;
using StellaOps.Vexer.Connectors.SUSE.RancherVEXHub.Configuration;
using StellaOps.Vexer.Core;
namespace StellaOps.Vexer.Connectors.SUSE.RancherVEXHub.Metadata;
public sealed class RancherHubMetadataLoader
{
public const string CachePrefix = "StellaOps.Vexer.Connectors.SUSE.RancherVEXHub.Metadata";
private readonly IHttpClientFactory _httpClientFactory;
private readonly IMemoryCache _memoryCache;
private readonly RancherHubTokenProvider _tokenProvider;
private readonly IFileSystem _fileSystem;
private readonly ILogger<RancherHubMetadataLoader> _logger;
private readonly SemaphoreSlim _semaphore = new(1, 1);
private readonly JsonDocumentOptions _documentOptions;
public RancherHubMetadataLoader(
IHttpClientFactory httpClientFactory,
IMemoryCache memoryCache,
RancherHubTokenProvider tokenProvider,
IFileSystem fileSystem,
ILogger<RancherHubMetadataLoader> logger)
{
_httpClientFactory = httpClientFactory ?? throw new ArgumentNullException(nameof(httpClientFactory));
_memoryCache = memoryCache ?? throw new ArgumentNullException(nameof(memoryCache));
_tokenProvider = tokenProvider ?? throw new ArgumentNullException(nameof(tokenProvider));
_fileSystem = fileSystem ?? throw new ArgumentNullException(nameof(fileSystem));
_logger = logger ?? throw new ArgumentNullException(nameof(logger));
_documentOptions = new JsonDocumentOptions
{
CommentHandling = JsonCommentHandling.Skip,
AllowTrailingCommas = true,
};
}
public async Task<RancherHubMetadataResult> LoadAsync(RancherHubConnectorOptions options, CancellationToken cancellationToken)
{
ArgumentNullException.ThrowIfNull(options);
var cacheKey = CreateCacheKey(options);
if (_memoryCache.TryGetValue<CacheEntry>(cacheKey, out var cached) && cached is not null && !cached.IsExpired())
{
_logger.LogDebug("Returning cached Rancher hub metadata (expires {Expires}).", cached.ExpiresAt);
return new RancherHubMetadataResult(cached.Metadata, cached.FetchedAt, FromCache: true, cached.FromOfflineSnapshot);
}
await _semaphore.WaitAsync(cancellationToken).ConfigureAwait(false);
try
{
if (_memoryCache.TryGetValue<CacheEntry>(cacheKey, out cached) && cached is not null && !cached.IsExpired())
{
return new RancherHubMetadataResult(cached.Metadata, cached.FetchedAt, FromCache: true, cached.FromOfflineSnapshot);
}
CacheEntry? previous = cached;
CacheEntry? entry = null;
if (options.PreferOfflineSnapshot)
{
entry = TryLoadFromOffline(options);
if (entry is null)
{
throw new InvalidOperationException("PreferOfflineSnapshot is enabled but no offline discovery snapshot was found.");
}
}
else
{
entry = await TryFetchFromNetworkAsync(options, previous, cancellationToken).ConfigureAwait(false);
if (entry is null)
{
entry = TryLoadFromOffline(options);
}
}
if (entry is null)
{
throw new InvalidOperationException("Unable to load Rancher hub discovery metadata from network or offline snapshot.");
}
_memoryCache.Set(cacheKey, entry, new MemoryCacheEntryOptions
{
AbsoluteExpirationRelativeToNow = options.MetadataCacheDuration,
});
return new RancherHubMetadataResult(entry.Metadata, entry.FetchedAt, FromCache: false, entry.FromOfflineSnapshot);
}
finally
{
_semaphore.Release();
}
}
private async Task<CacheEntry?> TryFetchFromNetworkAsync(RancherHubConnectorOptions options, CacheEntry? previous, CancellationToken cancellationToken)
{
try
{
var client = _httpClientFactory.CreateClient(RancherHubConnectorOptions.HttpClientName);
using var request = new HttpRequestMessage(HttpMethod.Get, options.DiscoveryUri);
request.Headers.Accept.ParseAdd("application/json");
if (!string.IsNullOrWhiteSpace(previous?.ETag) && EntityTagHeaderValue.TryParse(previous.ETag, out var etag))
{
request.Headers.IfNoneMatch.Add(etag);
}
var token = await _tokenProvider.GetAccessTokenAsync(options, cancellationToken).ConfigureAwait(false);
if (token is not null)
{
var scheme = string.IsNullOrWhiteSpace(token.TokenType) ? "Bearer" : token.TokenType;
request.Headers.Authorization = new AuthenticationHeaderValue(scheme, token.Value);
}
using var response = await client.SendAsync(request, HttpCompletionOption.ResponseHeadersRead, cancellationToken).ConfigureAwait(false);
if (response.StatusCode == HttpStatusCode.NotModified && previous is not null)
{
_logger.LogDebug("Rancher hub discovery document not modified (etag {ETag}).", previous.ETag);
return previous with
{
FetchedAt = DateTimeOffset.UtcNow,
ExpiresAt = DateTimeOffset.UtcNow + options.MetadataCacheDuration,
FromOfflineSnapshot = false,
};
}
response.EnsureSuccessStatusCode();
var payload = await response.Content.ReadAsStringAsync(cancellationToken).ConfigureAwait(false);
var metadata = ParseMetadata(payload, options);
var entry = new CacheEntry(
metadata,
DateTimeOffset.UtcNow,
DateTimeOffset.UtcNow + options.MetadataCacheDuration,
response.Headers.ETag?.ToString(),
FromOfflineSnapshot: false,
Payload: payload);
PersistOfflineSnapshot(options, payload);
return entry;
}
catch (Exception ex) when (ex is not OperationCanceledException && !options.PreferOfflineSnapshot)
{
_logger.LogWarning(ex, "Failed to fetch Rancher hub discovery document; attempting offline snapshot fallback.");
return null;
}
}
private CacheEntry? TryLoadFromOffline(RancherHubConnectorOptions options)
{
if (string.IsNullOrWhiteSpace(options.OfflineSnapshotPath))
{
return null;
}
if (!_fileSystem.File.Exists(options.OfflineSnapshotPath))
{
_logger.LogWarning("Rancher hub offline snapshot path {Path} does not exist.", options.OfflineSnapshotPath);
return null;
}
try
{
var payload = _fileSystem.File.ReadAllText(options.OfflineSnapshotPath);
var metadata = ParseMetadata(payload, options);
return new CacheEntry(
metadata,
DateTimeOffset.UtcNow,
DateTimeOffset.UtcNow + options.MetadataCacheDuration,
ETag: null,
FromOfflineSnapshot: true,
Payload: payload);
}
catch (Exception ex)
{
_logger.LogError(ex, "Failed to load Rancher hub discovery metadata from offline snapshot {Path}.", options.OfflineSnapshotPath);
return null;
}
}
private void PersistOfflineSnapshot(RancherHubConnectorOptions options, string payload)
{
if (!options.PersistOfflineSnapshot || string.IsNullOrWhiteSpace(options.OfflineSnapshotPath))
{
return;
}
try
{
var directory = _fileSystem.Path.GetDirectoryName(options.OfflineSnapshotPath);
if (!string.IsNullOrWhiteSpace(directory))
{
_fileSystem.Directory.CreateDirectory(directory);
}
_fileSystem.File.WriteAllText(options.OfflineSnapshotPath, payload);
_logger.LogDebug("Persisted Rancher hub discovery snapshot to {Path}.", options.OfflineSnapshotPath);
}
catch (Exception ex)
{
_logger.LogWarning(ex, "Failed to persist Rancher hub discovery snapshot to {Path}.", options.OfflineSnapshotPath);
}
}
private RancherHubMetadata ParseMetadata(string payload, RancherHubConnectorOptions options)
{
if (string.IsNullOrWhiteSpace(payload))
{
throw new InvalidOperationException("Rancher hub discovery payload was empty.");
}
try
{
using var document = JsonDocument.Parse(payload, _documentOptions);
var root = document.RootElement;
var hubId = ReadString(root, "hubId") ?? "vexer:suse:rancher";
var title = ReadString(root, "title") ?? ReadString(root, "displayName") ?? "SUSE Rancher VEX Hub";
var baseUri = ReadUri(root, "baseUri");
var subscriptionElement = TryGetProperty(root, "subscription");
if (!subscriptionElement.HasValue)
{
throw new InvalidOperationException("Discovery payload missing subscription section.");
}
var subscription = subscriptionElement.Value;
var eventsUri = ReadRequiredUri(subscription, "eventsUri", "eventsUrl", "eventsEndpoint");
var checkpointUri = ReadUri(subscription, "checkpointUri", "checkpointUrl", "checkpointEndpoint");
var channels = ReadStringArray(subscription, "channels", "defaultChannels", "products");
var scopes = ReadStringArray(subscription, "scopes", "defaultScopes");
var requiresAuth = ReadBoolean(subscription, "requiresAuthentication", defaultValue: options.TokenEndpoint is not null);
var authenticationElement = TryGetProperty(root, "authentication");
var tokenEndpointFromMetadata = authenticationElement.HasValue
? ReadUri(authenticationElement.Value, "tokenUri", "tokenEndpoint") ?? options.TokenEndpoint
: options.TokenEndpoint;
var audience = authenticationElement.HasValue
? ReadString(authenticationElement.Value, "audience", "aud") ?? options.Audience
: options.Audience;
var offlineElement = TryGetProperty(root, "offline", "snapshot");
var offlineSnapshot = offlineElement.HasValue
? BuildOfflineSnapshot(offlineElement.Value, options)
: null;
var provider = BuildProvider(hubId, title, baseUri, eventsUri, options);
var subscriptionMetadata = new RancherHubSubscriptionMetadata(eventsUri, checkpointUri, channels, scopes, requiresAuth);
var authenticationMetadata = new RancherHubAuthenticationMetadata(tokenEndpointFromMetadata, audience);
return new RancherHubMetadata(provider, subscriptionMetadata, authenticationMetadata, offlineSnapshot);
}
catch (JsonException ex)
{
throw new InvalidOperationException("Failed to parse Rancher hub discovery payload.", ex);
}
}
private RancherHubOfflineSnapshotMetadata? BuildOfflineSnapshot(JsonElement element, RancherHubConnectorOptions options)
{
var snapshotUri = ReadUri(element, "snapshotUri", "uri", "url");
if (snapshotUri is null)
{
return null;
}
var checksum = ReadString(element, "sha256", "checksum", "digest");
DateTimeOffset? updatedAt = null;
var updatedString = ReadString(element, "updated", "lastModified", "timestamp");
if (!string.IsNullOrWhiteSpace(updatedString) && DateTimeOffset.TryParse(updatedString, out var parsed))
{
updatedAt = parsed;
}
return new RancherHubOfflineSnapshotMetadata(snapshotUri, checksum, updatedAt);
}
private VexProvider BuildProvider(string hubId, string title, Uri? baseUri, Uri eventsUri, RancherHubConnectorOptions options)
{
var baseUris = new List<Uri>();
if (baseUri is not null)
{
baseUris.Add(baseUri);
}
baseUris.Add(eventsUri);
VexCosignTrust? cosign = null;
if (!string.IsNullOrWhiteSpace(options.CosignIssuer) && !string.IsNullOrWhiteSpace(options.CosignIdentityPattern))
{
cosign = new VexCosignTrust(options.CosignIssuer!, options.CosignIdentityPattern!);
}
var trust = new VexProviderTrust(options.TrustWeight, cosign, options.PgpFingerprints);
return new VexProvider(hubId, title, VexProviderKind.Hub, baseUris, new VexProviderDiscovery(options.DiscoveryUri, null), trust);
}
private static string CreateCacheKey(RancherHubConnectorOptions options)
=> $"{CachePrefix}:{options.DiscoveryUri}";
private static JsonElement? TryGetProperty(JsonElement element, params string[] propertyNames)
{
foreach (var name in propertyNames)
{
if (element.TryGetProperty(name, out var value) && value.ValueKind is not JsonValueKind.Null and not JsonValueKind.Undefined)
{
return value;
}
}
return null;
}
private static string? ReadString(JsonElement element, params string[] propertyNames)
{
var property = TryGetProperty(element, propertyNames);
if (property is null || property.Value.ValueKind is not JsonValueKind.String)
{
return null;
}
var value = property.Value.GetString();
return string.IsNullOrWhiteSpace(value) ? null : value;
}
private static bool ReadBoolean(JsonElement element, string propertyName, bool defaultValue)
{
if (!element.TryGetProperty(propertyName, out var property))
{
return defaultValue;
}
return property.ValueKind switch
{
JsonValueKind.True => true,
JsonValueKind.False => false,
JsonValueKind.String when bool.TryParse(property.GetString(), out var parsed) => parsed,
_ => defaultValue,
};
}
private static ImmutableArray<string> ReadStringArray(JsonElement element, params string[] propertyNames)
{
var property = TryGetProperty(element, propertyNames);
if (property is null)
{
return ImmutableArray<string>.Empty;
}
if (property.Value.ValueKind is JsonValueKind.Array)
{
var builder = ImmutableArray.CreateBuilder<string>();
foreach (var item in property.Value.EnumerateArray())
{
if (item.ValueKind is JsonValueKind.String)
{
var value = item.GetString();
if (!string.IsNullOrWhiteSpace(value))
{
builder.Add(value!);
}
}
}
return builder.Count == 0 ? ImmutableArray<string>.Empty : builder.ToImmutable();
}
if (property.Value.ValueKind is JsonValueKind.String)
{
var single = property.Value.GetString();
return string.IsNullOrWhiteSpace(single)
? ImmutableArray<string>.Empty
: ImmutableArray.Create(single!);
}
return ImmutableArray<string>.Empty;
}
private static Uri? ReadUri(JsonElement element, params string[] propertyNames)
{
var value = ReadString(element, propertyNames);
if (string.IsNullOrWhiteSpace(value))
{
return null;
}
if (!Uri.TryCreate(value, UriKind.Absolute, out var uri) || uri.Scheme is not ("http" or "https"))
{
throw new InvalidOperationException($"Discovery field '{string.Join("/", propertyNames)}' must be an absolute HTTP(S) URI.");
}
return uri;
}
private static Uri ReadRequiredUri(JsonElement element, params string[] propertyNames)
{
var uri = ReadUri(element, propertyNames);
if (uri is null)
{
throw new InvalidOperationException($"Discovery payload missing required URI field '{string.Join("/", propertyNames)}'.");
}
return uri;
}
private sealed record CacheEntry(
RancherHubMetadata Metadata,
DateTimeOffset FetchedAt,
DateTimeOffset ExpiresAt,
string? ETag,
bool FromOfflineSnapshot,
string? Payload)
{
public bool IsExpired() => DateTimeOffset.UtcNow >= ExpiresAt;
}
}
public sealed record RancherHubMetadata(
VexProvider Provider,
RancherHubSubscriptionMetadata Subscription,
RancherHubAuthenticationMetadata Authentication,
RancherHubOfflineSnapshotMetadata? OfflineSnapshot);
public sealed record RancherHubSubscriptionMetadata(
Uri EventsUri,
Uri? CheckpointUri,
ImmutableArray<string> Channels,
ImmutableArray<string> Scopes,
bool RequiresAuthentication);
public sealed record RancherHubAuthenticationMetadata(
Uri? TokenEndpoint,
string? Audience);
public sealed record RancherHubOfflineSnapshotMetadata(
Uri SnapshotUri,
string? Sha256,
DateTimeOffset? UpdatedAt);
public sealed record RancherHubMetadataResult(
RancherHubMetadata Metadata,
DateTimeOffset FetchedAt,
bool FromCache,
bool FromOfflineSnapshot);

85
src/StellaOps.Vexer.Connectors.SUSE.RancherVEXHub/RancherHubConnector.cs Normal file
View File

@@ -0,0 +1,85 @@
using System;
using System.Collections.Generic;
using System.Collections.Immutable;
using System.Runtime.CompilerServices;
using Microsoft.Extensions.Logging;
using StellaOps.Vexer.Connectors.Abstractions;
using StellaOps.Vexer.Connectors.SUSE.RancherVEXHub.Configuration;
using StellaOps.Vexer.Connectors.SUSE.RancherVEXHub.Metadata;
using StellaOps.Vexer.Core;
namespace StellaOps.Vexer.Connectors.SUSE.RancherVEXHub;
public sealed class RancherHubConnector : VexConnectorBase
{
private static readonly VexConnectorDescriptor StaticDescriptor = new(
id: "vexer:suse.rancher",
kind: VexProviderKind.Hub,
displayName: "SUSE Rancher VEX Hub")
{
Tags = ImmutableArray.Create("hub", "suse", "offline"),
};
private readonly RancherHubMetadataLoader _metadataLoader;
private readonly IEnumerable<IVexConnectorOptionsValidator<RancherHubConnectorOptions>> _validators;
private RancherHubConnectorOptions? _options;
private RancherHubMetadataResult? _metadata;
public RancherHubConnector(
RancherHubMetadataLoader metadataLoader,
ILogger<RancherHubConnector> logger,
TimeProvider timeProvider,
IEnumerable<IVexConnectorOptionsValidator<RancherHubConnectorOptions>>? validators = null)
: base(StaticDescriptor, logger, timeProvider)
{
_metadataLoader = metadataLoader ?? throw new ArgumentNullException(nameof(metadataLoader));
_validators = validators ?? Array.Empty<IVexConnectorOptionsValidator<RancherHubConnectorOptions>>();
}
public override async ValueTask ValidateAsync(VexConnectorSettings settings, CancellationToken cancellationToken)
{
_options = VexConnectorOptionsBinder.Bind(
Descriptor,
settings,
validators: _validators);
_metadata = await _metadataLoader.LoadAsync(_options, cancellationToken).ConfigureAwait(false);
LogConnectorEvent(LogLevel.Information, "validate", "Rancher hub discovery loaded.", new Dictionary<string, object?>
{
["discoveryUri"] = _options.DiscoveryUri.ToString(),
["subscriptionUri"] = _metadata.Metadata.Subscription.EventsUri.ToString(),
["requiresAuth"] = _metadata.Metadata.Subscription.RequiresAuthentication,
["fromOffline"] = _metadata.FromOfflineSnapshot,
});
}
public override async IAsyncEnumerable<VexRawDocument> FetchAsync(VexConnectorContext context, [EnumeratorCancellation] CancellationToken cancellationToken)
{
ArgumentNullException.ThrowIfNull(context);
if (_options is null)
{
throw new InvalidOperationException("Connector must be validated before fetch operations.");
}
if (_metadata is null)
{
_metadata = await _metadataLoader.LoadAsync(_options, cancellationToken).ConfigureAwait(false);
}
LogConnectorEvent(LogLevel.Debug, "fetch", "Rancher hub connector discovery ready; event ingestion will be implemented in VEXER-CONN-SUSE-01-002.", new Dictionary<string, object?>
{
["since"] = context.Since?.ToString("O"),
["subscriptionUri"] = _metadata.Metadata.Subscription.EventsUri.ToString(),
});
yield break;
}
public override ValueTask<VexClaimBatch> NormalizeAsync(VexRawDocument document, CancellationToken cancellationToken)
=> throw new NotSupportedException("RancherHubConnector relies on format-specific normalizers for CSAF/OpenVEX payloads.");
public RancherHubMetadata? GetCachedMetadata() => _metadata?.Metadata;
}

19
src/StellaOps.Vexer.Connectors.SUSE.RancherVEXHub/StellaOps.Vexer.Connectors.SUSE.RancherVEXHub.csproj Normal file
View File

@@ -0,0 +1,19 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>net10.0</TargetFramework>
<LangVersion>preview</LangVersion>
<Nullable>enable</Nullable>
<ImplicitUsings>enable</ImplicitUsings>
<TreatWarningsAsErrors>true</TreatWarningsAsErrors>
</PropertyGroup>
<ItemGroup>
<ProjectReference Include="..\StellaOps.Vexer.Connectors.Abstractions\StellaOps.Vexer.Connectors.Abstractions.csproj" />
<ProjectReference Include="..\StellaOps.Vexer.Storage.Mongo\StellaOps.Vexer.Storage.Mongo.csproj" />
</ItemGroup>
<ItemGroup>
<PackageReference Include="Microsoft.Extensions.Caching.Memory" Version="8.0.0" />
<PackageReference Include="Microsoft.Extensions.Http" Version="8.0.0" />
<PackageReference Include="Microsoft.Extensions.Options" Version="8.0.0" />
<PackageReference Include="System.IO.Abstractions" Version="20.0.28" />
</ItemGroup>
</Project>

2
src/StellaOps.Vexer.Connectors.SUSE.RancherVEXHub/TASKS.md
View File

@@ -2,6 +2,6 @@ If you are working on this file you need to read docs/ARCHITECTURE_VEXER.md and
# TASKS # TASKS
| Task | Owner(s) | Depends on | Notes | | Task | Owner(s) | Depends on | Notes |
|---|---|---|---| |---|---|---|---|
|VEXER-CONN-SUSE-01-001 – Rancher hub discovery & auth|Team Vexer Connectors – SUSE|VEXER-CONN-ABS-01-001|TODO – Implement hub discovery/subscription setup with credential handling and offline snapshot support.| |VEXER-CONN-SUSE-01-001 – Rancher hub discovery & auth|Team Vexer Connectors – SUSE|VEXER-CONN-ABS-01-001|**DONE (2025-10-17)** – Added Rancher hub options/token provider, discovery metadata loader with offline snapshots + caching, connector shell, DI wiring, and unit tests covering network/offline paths.|
|VEXER-CONN-SUSE-01-002 – Checkpointed event ingestion|Team Vexer Connectors – SUSE|VEXER-CONN-SUSE-01-001, VEXER-STORAGE-01-003|TODO – Process hub events with resume checkpoints, deduplication, and quarantine path for malformed payloads.| |VEXER-CONN-SUSE-01-002 – Checkpointed event ingestion|Team Vexer Connectors – SUSE|VEXER-CONN-SUSE-01-001, VEXER-STORAGE-01-003|TODO – Process hub events with resume checkpoints, deduplication, and quarantine path for malformed payloads.|
|VEXER-CONN-SUSE-01-003 – Trust metadata & policy hints|Team Vexer Connectors – SUSE|VEXER-CONN-SUSE-01-002, VEXER-POLICY-01-001|TODO – Emit provider trust configuration (signers, weight overrides) and attach provenance hints for consensus engine.| |VEXER-CONN-SUSE-01-003 – Trust metadata & policy hints|Team Vexer Connectors – SUSE|VEXER-CONN-SUSE-01-002, VEXER-POLICY-01-001|TODO – Emit provider trust configuration (signers, weight overrides) and attach provenance hints for consensus engine.|

172
src/StellaOps.Vexer.Connectors.Ubuntu.CSAF.Tests/Metadata/UbuntuCatalogLoaderTests.cs Normal file
View File

@@ -0,0 +1,172 @@
using System.Collections.Generic;
using System.Net;
using System.Net.Http;
using System.Text;
using FluentAssertions;
using Microsoft.Extensions.Caching.Memory;
using Microsoft.Extensions.Logging.Abstractions;
using StellaOps.Vexer.Connectors.Ubuntu.CSAF.Configuration;
using StellaOps.Vexer.Connectors.Ubuntu.CSAF.Metadata;
using System.IO.Abstractions.TestingHelpers;
using Xunit;
using System.Threading;
namespace StellaOps.Vexer.Connectors.Ubuntu.CSAF.Tests.Metadata;
public sealed class UbuntuCatalogLoaderTests
{
private const string SampleIndex = """
{
"generated": "2025-10-10T00:00:00Z",
"channels": [
{
"name": "stable",
"catalogUrl": "https://ubuntu.com/security/csaf/stable/catalog.json",
"sha256": "abc",
"lastUpdated": "2025-10-09T10:00:00Z"
},
{
"name": "esm",
"catalogUrl": "https://ubuntu.com/security/csaf/esm/catalog.json",
"sha256": "def",
"lastUpdated": "2025-10-08T10:00:00Z"
}
]
}
""";
[Fact]
public async Task LoadAsync_FetchesAndCachesIndex()
{
var handler = new TestHttpMessageHandler(new Dictionary<Uri, HttpResponseMessage>
{
[new Uri("https://ubuntu.com/security/csaf/index.json")] = CreateResponse(SampleIndex),
});
var client = new HttpClient(handler);
var factory = new SingleClientHttpClientFactory(client);
var cache = new MemoryCache(new MemoryCacheOptions());
var fileSystem = new MockFileSystem();
var loader = new UbuntuCatalogLoader(factory, cache, fileSystem, NullLogger<UbuntuCatalogLoader>.Instance, new AdjustableTimeProvider());
var options = new UbuntuConnectorOptions
{
IndexUri = new Uri("https://ubuntu.com/security/csaf/index.json"),
OfflineSnapshotPath = "/snapshots/ubuntu-index.json",
};
var result = await loader.LoadAsync(options, CancellationToken.None);
result.Metadata.Channels.Should().HaveCount(1);
result.Metadata.Channels[0].Name.Should().Be("stable");
fileSystem.FileExists(options.OfflineSnapshotPath!).Should().BeTrue();
result.FromCache.Should().BeFalse();
handler.ResetInvocationCount();
var cached = await loader.LoadAsync(options, CancellationToken.None);
cached.FromCache.Should().BeTrue();
handler.InvocationCount.Should().Be(0);
}
[Fact]
public async Task LoadAsync_UsesOfflineSnapshotWhenPreferred()
{
var handler = new TestHttpMessageHandler(new Dictionary<Uri, HttpResponseMessage>());
var client = new HttpClient(handler);
var factory = new SingleClientHttpClientFactory(client);
var cache = new MemoryCache(new MemoryCacheOptions());
var fileSystem = new MockFileSystem();
fileSystem.AddFile("/snapshots/ubuntu-index.json", new MockFileData($"{{\"metadata\":{SampleIndex},\"fetchedAt\":\"2025-10-10T00:00:00Z\"}}"));
var loader = new UbuntuCatalogLoader(factory, cache, fileSystem, NullLogger<UbuntuCatalogLoader>.Instance, new AdjustableTimeProvider());
var options = new UbuntuConnectorOptions
{
IndexUri = new Uri("https://ubuntu.com/security/csaf/index.json"),
OfflineSnapshotPath = "/snapshots/ubuntu-index.json",
PreferOfflineSnapshot = true,
Channels = { "stable" }
};
var result = await loader.LoadAsync(options, CancellationToken.None);
result.FromOfflineSnapshot.Should().BeTrue();
result.Metadata.Channels.Should().NotBeEmpty();
}
[Fact]
public async Task LoadAsync_ThrowsWhenNoChannelsMatch()
{
var handler = new TestHttpMessageHandler(new Dictionary<Uri, HttpResponseMessage>
{
[new Uri("https://ubuntu.com/security/csaf/index.json")] = CreateResponse(SampleIndex),
});
var client = new HttpClient(handler);
var factory = new SingleClientHttpClientFactory(client);
var cache = new MemoryCache(new MemoryCacheOptions());
var fileSystem = new MockFileSystem();
var loader = new UbuntuCatalogLoader(factory, cache, fileSystem, NullLogger<UbuntuCatalogLoader>.Instance, new AdjustableTimeProvider());
var options = new UbuntuConnectorOptions
{
IndexUri = new Uri("https://ubuntu.com/security/csaf/index.json"),
};
options.Channels.Clear();
options.Channels.Add("nonexistent");
await Assert.ThrowsAsync<InvalidOperationException>(() => loader.LoadAsync(options, CancellationToken.None));
}
private static HttpResponseMessage CreateResponse(string payload)
=> new(HttpStatusCode.OK)
{
Content = new StringContent(payload, Encoding.UTF8, "application/json"),
};
private sealed class SingleClientHttpClientFactory : IHttpClientFactory
{
private readonly HttpClient _client;
public SingleClientHttpClientFactory(HttpClient client)
{
_client = client;
}
public HttpClient CreateClient(string name) => _client;
}
private sealed class AdjustableTimeProvider : TimeProvider
{
private DateTimeOffset _now = DateTimeOffset.UtcNow;
public override DateTimeOffset GetUtcNow() => _now;
}
private sealed class TestHttpMessageHandler : HttpMessageHandler
{
private readonly Dictionary<Uri, HttpResponseMessage> _responses;
public TestHttpMessageHandler(Dictionary<Uri, HttpResponseMessage> responses)
{
_responses = responses;
}
public int InvocationCount { get; private set; }
public void ResetInvocationCount() => InvocationCount = 0;
protected override async Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
{
InvocationCount++;
if (request.RequestUri is not null && _responses.TryGetValue(request.RequestUri, out var response))
{
var payload = await response.Content.ReadAsStringAsync(cancellationToken).ConfigureAwait(false);
return new HttpResponseMessage(response.StatusCode)
{
Content = new StringContent(payload, Encoding.UTF8, "application/json"),
};
}
return new HttpResponseMessage(HttpStatusCode.InternalServerError)
{
Content = new StringContent("unexpected request"),
};
}
}
}

17
src/StellaOps.Vexer.Connectors.Ubuntu.CSAF.Tests/StellaOps.Vexer.Connectors.Ubuntu.CSAF.Tests.csproj Normal file
View File

@@ -0,0 +1,17 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>net10.0</TargetFramework>
<LangVersion>preview</LangVersion>
<Nullable>enable</Nullable>
<ImplicitUsings>enable</ImplicitUsings>
<TreatWarningsAsErrors>true</TreatWarningsAsErrors>
</PropertyGroup>
<ItemGroup>
<ProjectReference Include="..\StellaOps.Vexer.Connectors.Ubuntu.CSAF\StellaOps.Vexer.Connectors.Ubuntu.CSAF.csproj" />
</ItemGroup>
<ItemGroup>
<PackageReference Include="FluentAssertions" Version="6.12.0" />
<PackageReference Include="Microsoft.Extensions.Logging.Abstractions" Version="8.0.0" />
<PackageReference Include="System.IO.Abstractions.TestingHelpers" Version="20.0.28" />
</ItemGroup>
</Project>

90
src/StellaOps.Vexer.Connectors.Ubuntu.CSAF/Configuration/UbuntuConnectorOptions.cs Normal file
View File

@@ -0,0 +1,90 @@
using System;
using System.Collections.Generic;
using System.IO;
using System.IO.Abstractions;
namespace StellaOps.Vexer.Connectors.Ubuntu.CSAF.Configuration;
public sealed class UbuntuConnectorOptions
{
public const string HttpClientName = "vexer.connector.ubuntu.catalog";
/// <summary>
/// Root index that lists Ubuntu CSAF channels.
/// </summary>
public Uri IndexUri { get; set; } = new("https://ubuntu.com/security/csaf/index.json");
/// <summary>
/// Channels to include (e.g. stable, esm, lts).
/// </summary>
public IList<string> Channels { get; } = new List<string> { "stable" };
/// <summary>
/// Duration to cache discovery metadata.
/// </summary>
public TimeSpan MetadataCacheDuration { get; set; } = TimeSpan.FromHours(4);
/// <summary>
/// Prefer offline snapshot when available.
/// </summary>
public bool PreferOfflineSnapshot { get; set; }
/// <summary>
/// Optional file path for offline index snapshot.
/// </summary>
public string? OfflineSnapshotPath { get; set; }
/// <summary>
/// Controls persistence of network responses to <see cref="OfflineSnapshotPath"/>.
/// </summary>
public bool PersistOfflineSnapshot { get; set; } = true;
public void Validate(IFileSystem? fileSystem = null)
{
if (IndexUri is null || !IndexUri.IsAbsoluteUri)
{
throw new InvalidOperationException("IndexUri must be an absolute URI.");
}
if (IndexUri.Scheme is not ("http" or "https"))
{
throw new InvalidOperationException("IndexUri must use HTTP or HTTPS.");
}
if (Channels.Count == 0)
{
throw new InvalidOperationException("At least one channel must be specified.");
}
for (var i = Channels.Count - 1; i >= 0; i--)
{
if (string.IsNullOrWhiteSpace(Channels[i]))
{
Channels.RemoveAt(i);
}
}
if (Channels.Count == 0)
{
throw new InvalidOperationException("Channel names cannot be empty.");
}
if (MetadataCacheDuration <= TimeSpan.Zero)
{
throw new InvalidOperationException("MetadataCacheDuration must be positive.");
}
if (PreferOfflineSnapshot && string.IsNullOrWhiteSpace(OfflineSnapshotPath))
{
throw new InvalidOperationException("OfflineSnapshotPath must be provided when PreferOfflineSnapshot is enabled.");
}
if (!string.IsNullOrWhiteSpace(OfflineSnapshotPath))
{
var fs = fileSystem ?? new FileSystem();
var directory = Path.GetDirectoryName(OfflineSnapshotPath);
if (!string.IsNullOrWhiteSpace(directory) && !fs.Directory.Exists(directory))
{
fs.Directory.CreateDirectory(directory);
}
}
}
}

32
src/StellaOps.Vexer.Connectors.Ubuntu.CSAF/Configuration/UbuntuConnectorOptionsValidator.cs Normal file
View File

@@ -0,0 +1,32 @@
using System;
using System.Collections.Generic;
using System.IO.Abstractions;
using StellaOps.Vexer.Connectors.Abstractions;
namespace StellaOps.Vexer.Connectors.Ubuntu.CSAF.Configuration;
public sealed class UbuntuConnectorOptionsValidator : IVexConnectorOptionsValidator<UbuntuConnectorOptions>
{
private readonly IFileSystem _fileSystem;
public UbuntuConnectorOptionsValidator(IFileSystem fileSystem)
{
_fileSystem = fileSystem ?? throw new ArgumentNullException(nameof(fileSystem));
}
public void Validate(VexConnectorDescriptor descriptor, UbuntuConnectorOptions options, IList<string> errors)
{
ArgumentNullException.ThrowIfNull(descriptor);
ArgumentNullException.ThrowIfNull(options);
ArgumentNullException.ThrowIfNull(errors);
try
{
options.Validate(_fileSystem);
}
catch (Exception ex)
{
errors.Add(ex.Message);
}
}
}

45
src/StellaOps.Vexer.Connectors.Ubuntu.CSAF/DependencyInjection/UbuntuConnectorServiceCollectionExtensions.cs Normal file
View File

@@ -0,0 +1,45 @@
using System;
using System.Net;
using System.Net.Http;
using Microsoft.Extensions.Caching.Memory;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.DependencyInjection.Extensions;
using StellaOps.Vexer.Connectors.Abstractions;
using StellaOps.Vexer.Connectors.Ubuntu.CSAF.Configuration;
using StellaOps.Vexer.Connectors.Ubuntu.CSAF.Metadata;
using StellaOps.Vexer.Core;
using System.IO.Abstractions;
namespace StellaOps.Vexer.Connectors.Ubuntu.CSAF.DependencyInjection;
public static class UbuntuConnectorServiceCollectionExtensions
{
public static IServiceCollection AddUbuntuCsafConnector(this IServiceCollection services, Action<UbuntuConnectorOptions>? configure = null)
{
ArgumentNullException.ThrowIfNull(services);
services.TryAddSingleton<IMemoryCache, MemoryCache>();
services.TryAddSingleton<IFileSystem, FileSystem>();
services.AddOptions<UbuntuConnectorOptions>()
.Configure(options => configure?.Invoke(options));
services.AddSingleton<IVexConnectorOptionsValidator<UbuntuConnectorOptions>, UbuntuConnectorOptionsValidator>();
services.AddHttpClient(UbuntuConnectorOptions.HttpClientName, client =>
{
client.Timeout = TimeSpan.FromSeconds(60);
client.DefaultRequestHeaders.UserAgent.ParseAdd("StellaOps.Vexer.Connectors.Ubuntu.CSAF/1.0");
client.DefaultRequestHeaders.Accept.ParseAdd("application/json");
})
.ConfigurePrimaryHttpMessageHandler(static () => new HttpClientHandler
{
AutomaticDecompression = DecompressionMethods.All,
});
services.AddSingleton<UbuntuCatalogLoader>();
services.AddSingleton<IVexConnector, UbuntuCsafConnector>();
return services;
}
}

248
src/StellaOps.Vexer.Connectors.Ubuntu.CSAF/Metadata/UbuntuCatalogLoader.cs Normal file
View File

@@ -0,0 +1,248 @@
using System;
using System.Collections.Generic;
using System.Collections.Immutable;
using System.IO.Abstractions;
using System.Net;
using System.Net.Http;
using System.Text.Json;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.Extensions.Caching.Memory;
using Microsoft.Extensions.Logging;
using StellaOps.Vexer.Connectors.Ubuntu.CSAF.Configuration;
namespace StellaOps.Vexer.Connectors.Ubuntu.CSAF.Metadata;
public sealed class UbuntuCatalogLoader
{
public const string CachePrefix = "StellaOps.Vexer.Connectors.Ubuntu.CSAF.Index";
private readonly IHttpClientFactory _httpClientFactory;
private readonly IMemoryCache _memoryCache;
private readonly IFileSystem _fileSystem;
private readonly ILogger<UbuntuCatalogLoader> _logger;
private readonly TimeProvider _timeProvider;
private readonly SemaphoreSlim _semaphore = new(1, 1);
private readonly JsonSerializerOptions _serializerOptions = new(JsonSerializerDefaults.Web);
public UbuntuCatalogLoader(
IHttpClientFactory httpClientFactory,
IMemoryCache memoryCache,
IFileSystem fileSystem,
ILogger<UbuntuCatalogLoader> logger,
TimeProvider? timeProvider = null)
{
_httpClientFactory = httpClientFactory ?? throw new ArgumentNullException(nameof(httpClientFactory));
_memoryCache = memoryCache ?? throw new ArgumentNullException(nameof(memoryCache));
_fileSystem = fileSystem ?? throw new ArgumentNullException(nameof(fileSystem));
_logger = logger ?? throw new ArgumentNullException(nameof(logger));
_timeProvider = timeProvider ?? TimeProvider.System;
}
public async Task<UbuntuCatalogResult> LoadAsync(UbuntuConnectorOptions options, CancellationToken cancellationToken)
{
ArgumentNullException.ThrowIfNull(options);
options.Validate(_fileSystem);
var cacheKey = CreateCacheKey(options);
if (_memoryCache.TryGetValue<CacheEntry>(cacheKey, out var cached) && cached is not null && !cached.IsExpired(_timeProvider.GetUtcNow()))
{
return cached.ToResult(fromCache: true);
}
await _semaphore.WaitAsync(cancellationToken).ConfigureAwait(false);
try
{
if (_memoryCache.TryGetValue<CacheEntry>(cacheKey, out cached) && cached is not null && !cached.IsExpired(_timeProvider.GetUtcNow()))
{
return cached.ToResult(fromCache: true);
}
CacheEntry? entry = null;
if (options.PreferOfflineSnapshot)
{
entry = LoadFromOffline(options);
if (entry is null)
{
throw new InvalidOperationException("PreferOfflineSnapshot is enabled but no offline Ubuntu snapshot was found.");
}
}
else
{
entry = await TryFetchFromNetworkAsync(options, cancellationToken).ConfigureAwait(false)
?? LoadFromOffline(options);
}
if (entry is null)
{
throw new InvalidOperationException("Unable to load Ubuntu CSAF index from network or offline snapshot.");
}
var cacheOptions = new MemoryCacheEntryOptions();
if (entry.MetadataCacheDuration > TimeSpan.Zero)
{
cacheOptions.AbsoluteExpiration = _timeProvider.GetUtcNow().Add(entry.MetadataCacheDuration);
}
_memoryCache.Set(cacheKey, entry with { CachedAt = _timeProvider.GetUtcNow() }, cacheOptions);
return entry.ToResult(fromCache: false);
}
finally
{
_semaphore.Release();
}
}
private async Task<CacheEntry?> TryFetchFromNetworkAsync(UbuntuConnectorOptions options, CancellationToken cancellationToken)
{
try
{
var client = _httpClientFactory.CreateClient(UbuntuConnectorOptions.HttpClientName);
using var response = await client.GetAsync(options.IndexUri, HttpCompletionOption.ResponseHeadersRead, cancellationToken).ConfigureAwait(false);
response.EnsureSuccessStatusCode();
var payload = await response.Content.ReadAsStringAsync(cancellationToken).ConfigureAwait(false);
var metadata = ParseMetadata(payload, options.Channels);
var now = _timeProvider.GetUtcNow();
var entry = new CacheEntry(metadata, now, now, options.MetadataCacheDuration, false);
PersistSnapshotIfNeeded(options, metadata, now);
return entry;
}
catch (Exception ex) when (ex is not OperationCanceledException)
{
_logger.LogWarning(ex, "Failed to fetch Ubuntu CSAF index from {Uri}; attempting offline fallback if available.", options.IndexUri);
return null;
}
}
private CacheEntry? LoadFromOffline(UbuntuConnectorOptions options)
{
if (string.IsNullOrWhiteSpace(options.OfflineSnapshotPath))
{
return null;
}
if (!_fileSystem.File.Exists(options.OfflineSnapshotPath))
{
_logger.LogWarning("Ubuntu offline snapshot path {Path} does not exist.", options.OfflineSnapshotPath);
return null;
}
try
{
var payload = _fileSystem.File.ReadAllText(options.OfflineSnapshotPath);
var snapshot = JsonSerializer.Deserialize<UbuntuCatalogSnapshot>(payload, _serializerOptions);
if (snapshot is null)
{
throw new InvalidOperationException("Offline snapshot payload was empty.");
}
return new CacheEntry(snapshot.Metadata, snapshot.FetchedAt, _timeProvider.GetUtcNow(), options.MetadataCacheDuration, true);
}
catch (Exception ex)
{
_logger.LogError(ex, "Failed to load Ubuntu CSAF index from offline snapshot {Path}.", options.OfflineSnapshotPath);
return null;
}
}
private UbuntuCatalogMetadata ParseMetadata(string payload, IList<string> channels)
{
if (string.IsNullOrWhiteSpace(payload))
{
throw new InvalidOperationException("Ubuntu index payload was empty.");
}
using var document = JsonDocument.Parse(payload);
var root = document.RootElement;
var generatedAt = root.TryGetProperty("generated", out var generatedElement) && generatedElement.ValueKind == JsonValueKind.String && DateTimeOffset.TryParse(generatedElement.GetString(), out var generated)
? generated
: _timeProvider.GetUtcNow();
var channelSet = new HashSet<string>(channels, StringComparer.OrdinalIgnoreCase);
if (!root.TryGetProperty("channels", out var channelsElement) || channelsElement.ValueKind is not JsonValueKind.Array)
{
throw new InvalidOperationException("Ubuntu index did not include a channels array.");
}
var builder = ImmutableArray.CreateBuilder<UbuChannelCatalog>();
foreach (var channelElement in channelsElement.EnumerateArray())
{
var name = channelElement.TryGetProperty("name", out var nameElement) && nameElement.ValueKind == JsonValueKind.String ? nameElement.GetString() : null;
if (string.IsNullOrWhiteSpace(name) || !channelSet.Contains(name))
{
continue;
}
if (!channelElement.TryGetProperty("catalogUrl", out var urlElement) || urlElement.ValueKind != JsonValueKind.String || !Uri.TryCreate(urlElement.GetString(), UriKind.Absolute, out var catalogUri))
{
_logger.LogWarning("Channel {Channel} did not specify a valid catalogUrl.", name);
continue;
}
string? sha256 = null;
if (channelElement.TryGetProperty("sha256", out var shaElement) && shaElement.ValueKind == JsonValueKind.String)
{
sha256 = shaElement.GetString();
}
DateTimeOffset? lastUpdated = null;
if (channelElement.TryGetProperty("lastUpdated", out var updatedElement) && updatedElement.ValueKind == JsonValueKind.String && DateTimeOffset.TryParse(updatedElement.GetString(), out var updated))
{
lastUpdated = updated;
}
builder.Add(new UbuChannelCatalog(name!, catalogUri, sha256, lastUpdated));
}
if (builder.Count == 0)
{
throw new InvalidOperationException("None of the requested Ubuntu channels were present in the index.");
}
return new UbuntuCatalogMetadata(generatedAt, builder.ToImmutable());
}
private void PersistSnapshotIfNeeded(UbuntuConnectorOptions options, UbuntuCatalogMetadata metadata, DateTimeOffset fetchedAt)
{
if (!options.PersistOfflineSnapshot || string.IsNullOrWhiteSpace(options.OfflineSnapshotPath))
{
return;
}
try
{
var snapshot = new UbuntuCatalogSnapshot(metadata, fetchedAt);
var payload = JsonSerializer.Serialize(snapshot, _serializerOptions);
_fileSystem.File.WriteAllText(options.OfflineSnapshotPath, payload);
_logger.LogDebug("Persisted Ubuntu CSAF index snapshot to {Path}.", options.OfflineSnapshotPath);
}
catch (Exception ex)
{
_logger.LogWarning(ex, "Failed to persist Ubuntu CSAF index snapshot to {Path}.", options.OfflineSnapshotPath);
}
}
private static string CreateCacheKey(UbuntuConnectorOptions options)
=> $"{CachePrefix}:{options.IndexUri}:{string.Join(',', options.Channels)}";
private sealed record CacheEntry(UbuntuCatalogMetadata Metadata, DateTimeOffset FetchedAt, DateTimeOffset CachedAt, TimeSpan MetadataCacheDuration, bool FromOfflineSnapshot)
{
public bool IsExpired(DateTimeOffset now)
=> MetadataCacheDuration > TimeSpan.Zero && now >= CachedAt + MetadataCacheDuration;
public UbuntuCatalogResult ToResult(bool fromCache)
=> new(Metadata, FetchedAt, fromCache, FromOfflineSnapshot);
}
private sealed record UbuntuCatalogSnapshot(UbuntuCatalogMetadata Metadata, DateTimeOffset FetchedAt);
}
public sealed record UbuntuCatalogMetadata(DateTimeOffset GeneratedAt, ImmutableArray<UbuChannelCatalog> Channels);
public sealed record UbuChannelCatalog(string Name, Uri CatalogUri, string? Sha256, DateTimeOffset? LastUpdated);
public sealed record UbuntuCatalogResult(UbuntuCatalogMetadata Metadata, DateTimeOffset FetchedAt, bool FromCache, bool FromOfflineSnapshot);

18
src/StellaOps.Vexer.Connectors.Ubuntu.CSAF/StellaOps.Vexer.Connectors.Ubuntu.CSAF.csproj Normal file
View File

@@ -0,0 +1,18 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>net10.0</TargetFramework>
<LangVersion>preview</LangVersion>
<Nullable>enable</Nullable>
<ImplicitUsings>enable</ImplicitUsings>
<TreatWarningsAsErrors>true</TreatWarningsAsErrors>
</PropertyGroup>
<ItemGroup>
<ProjectReference Include="..\StellaOps.Vexer.Connectors.Abstractions\StellaOps.Vexer.Connectors.Abstractions.csproj" />
</ItemGroup>
<ItemGroup>
<PackageReference Include="Microsoft.Extensions.Caching.Memory" Version="8.0.0" />
<PackageReference Include="Microsoft.Extensions.Http" Version="8.0.0" />
<PackageReference Include="Microsoft.Extensions.Options" Version="8.0.0" />
<PackageReference Include="System.IO.Abstractions" Version="20.0.28" />
</ItemGroup>
</Project>

2
src/StellaOps.Vexer.Connectors.Ubuntu.CSAF/TASKS.md
View File

@@ -2,6 +2,6 @@ If you are working on this file you need to read docs/ARCHITECTURE_VEXER.md and
# TASKS # TASKS
| Task | Owner(s) | Depends on | Notes | | Task | Owner(s) | Depends on | Notes |
|---|---|---|---| |---|---|---|---|
|VEXER-CONN-UBUNTU-01-001 – Ubuntu CSAF discovery & channels|Team Vexer Connectors – Ubuntu|VEXER-CONN-ABS-01-001|TODO – Implement discovery of Ubuntu CSAF catalogs, channel selection (stable/LTS), and offline snapshot import.| |VEXER-CONN-UBUNTU-01-001 – Ubuntu CSAF discovery & channels|Team Vexer Connectors – Ubuntu|VEXER-CONN-ABS-01-001|**DONE (2025-10-17)** – Added Ubuntu connector project with configurable channel options, catalog loader (network/offline), DI wiring, and discovery unit tests.|
|VEXER-CONN-UBUNTU-01-002 – Incremental fetch & deduplication|Team Vexer Connectors – Ubuntu|VEXER-CONN-UBUNTU-01-001, VEXER-STORAGE-01-003|TODO – Fetch CSAF bundles with ETag handling, checksum validation, deduplication, and raw persistence.| |VEXER-CONN-UBUNTU-01-002 – Incremental fetch & deduplication|Team Vexer Connectors – Ubuntu|VEXER-CONN-UBUNTU-01-001, VEXER-STORAGE-01-003|TODO – Fetch CSAF bundles with ETag handling, checksum validation, deduplication, and raw persistence.|
|VEXER-CONN-UBUNTU-01-003 – Trust metadata & provenance|Team Vexer Connectors – Ubuntu|VEXER-CONN-UBUNTU-01-002, VEXER-POLICY-01-001|TODO – Emit Ubuntu signing metadata (GPG fingerprints) plus provenance hints for policy weighting and diagnostics.| |VEXER-CONN-UBUNTU-01-003 – Trust metadata & provenance|Team Vexer Connectors – Ubuntu|VEXER-CONN-UBUNTU-01-002, VEXER-POLICY-01-001|TODO – Emit Ubuntu signing metadata (GPG fingerprints) plus provenance hints for policy weighting and diagnostics.|

80
src/StellaOps.Vexer.Connectors.Ubuntu.CSAF/UbuntuCsafConnector.cs Normal file
View File

@@ -0,0 +1,80 @@
using System.Collections.Generic;
using System.Collections.Immutable;
using System.Runtime.CompilerServices;
using Microsoft.Extensions.Logging;
using StellaOps.Vexer.Connectors.Abstractions;
using StellaOps.Vexer.Connectors.Ubuntu.CSAF.Configuration;
using StellaOps.Vexer.Connectors.Ubuntu.CSAF.Metadata;
using StellaOps.Vexer.Core;
namespace StellaOps.Vexer.Connectors.Ubuntu.CSAF;
public sealed class UbuntuCsafConnector : VexConnectorBase
{
private static readonly VexConnectorDescriptor DescriptorInstance = new(
id: "vexer:ubuntu",
kind: VexProviderKind.Distro,
displayName: "Ubuntu CSAF")
{
Tags = ImmutableArray.Create("ubuntu", "csaf", "usn"),
};
private readonly UbuntuCatalogLoader _catalogLoader;
private readonly IEnumerable<IVexConnectorOptionsValidator<UbuntuConnectorOptions>> _validators;
private UbuntuConnectorOptions? _options;
private UbuntuCatalogResult? _catalog;
public UbuntuCsafConnector(
UbuntuCatalogLoader catalogLoader,
IEnumerable<IVexConnectorOptionsValidator<UbuntuConnectorOptions>> validators,
ILogger<UbuntuCsafConnector> logger,
TimeProvider timeProvider)
: base(DescriptorInstance, logger, timeProvider)
{
_catalogLoader = catalogLoader ?? throw new ArgumentNullException(nameof(catalogLoader));
_validators = validators ?? Array.Empty<IVexConnectorOptionsValidator<UbuntuConnectorOptions>>();
}
public override async ValueTask ValidateAsync(VexConnectorSettings settings, CancellationToken cancellationToken)
{
_options = VexConnectorOptionsBinder.Bind(
Descriptor,
settings,
validators: _validators);
_catalog = await _catalogLoader.LoadAsync(_options, cancellationToken).ConfigureAwait(false);
LogConnectorEvent(LogLevel.Information, "validate", "Ubuntu CSAF index loaded.", new Dictionary<string, object?>
{
["channelCount"] = _catalog.Metadata.Channels.Length,
["fromOffline"] = _catalog.FromOfflineSnapshot,
});
}
public override async IAsyncEnumerable<VexRawDocument> FetchAsync(VexConnectorContext context, [EnumeratorCancellation] CancellationToken cancellationToken)
{
ArgumentNullException.ThrowIfNull(context);
if (_options is null)
{
throw new InvalidOperationException("Connector must be validated before fetch operations.");
}
if (_catalog is null)
{
_catalog = await _catalogLoader.LoadAsync(_options, cancellationToken).ConfigureAwait(false);
}
LogConnectorEvent(LogLevel.Debug, "fetch", "Ubuntu CSAF discovery ready; channel catalogs handled in subsequent task.", new Dictionary<string, object?>
{
["since"] = context.Since?.ToString("O"),
});
yield break;
}
public override ValueTask<VexClaimBatch> NormalizeAsync(VexRawDocument document, CancellationToken cancellationToken)
=> throw new NotSupportedException("UbuntuCsafConnector relies on CSAF normalizers for document processing.");
public UbuntuCatalogResult? GetCachedCatalog() => _catalog;
}

60
src/StellaOps.Vexer.Export.Tests/ExportEngineTests.cs
View File

@@ -88,10 +88,46 @@ public sealed class ExportEngineTests
Assert.Equal(1, recorder2.SaveCount); Assert.Equal(1, recorder2.SaveCount);
} }
[Fact]
public async Task ExportAsync_AttachesAttestationMetadata()
{
var store = new InMemoryExportStore();
var evaluator = new StaticPolicyEvaluator("baseline/v1");
var dataSource = new InMemoryExportDataSource();
var exporter = new DummyExporter(VexExportFormat.Json);
var attestation = new RecordingAttestationClient();
var engine = new VexExportEngine(
store,
evaluator,
dataSource,
new[] { exporter },
NullLogger<VexExportEngine>.Instance,
cacheIndex: null,
artifactStores: null,
attestationClient: attestation);
var query = VexQuery.Create(new[] { new VexQueryFilter("vulnId", "CVE-2025-0001") });
var requestedAt = DateTimeOffset.UtcNow;
var context = new VexExportRequestContext(query, VexExportFormat.Json, requestedAt);
var manifest = await engine.ExportAsync(context, CancellationToken.None);
Assert.NotNull(attestation.LastRequest);
Assert.Equal(manifest.ExportId, attestation.LastRequest!.ExportId);
Assert.NotNull(manifest.Attestation);
Assert.Equal(attestation.Response.Attestation.EnvelopeDigest, manifest.Attestation!.EnvelopeDigest);
Assert.Equal(attestation.Response.Attestation.PredicateType, manifest.Attestation.PredicateType);
Assert.NotNull(store.LastSavedManifest);
Assert.Equal(manifest.Attestation, store.LastSavedManifest!.Attestation);
}
private sealed class InMemoryExportStore : IVexExportStore private sealed class InMemoryExportStore : IVexExportStore
{ {
private readonly Dictionary<string, VexExportManifest> _store = new(StringComparer.Ordinal); private readonly Dictionary<string, VexExportManifest> _store = new(StringComparer.Ordinal);
public VexExportManifest? LastSavedManifest { get; private set; }
public ValueTask<VexExportManifest?> FindAsync(VexQuerySignature signature, VexExportFormat format, CancellationToken cancellationToken) public ValueTask<VexExportManifest?> FindAsync(VexQuerySignature signature, VexExportFormat format, CancellationToken cancellationToken)
{ {
var key = CreateKey(signature.Value, format); var key = CreateKey(signature.Value, format);
@@ -103,6 +139,7 @@ public sealed class ExportEngineTests
{ {
var key = CreateKey(manifest.QuerySignature.Value, manifest.Format); var key = CreateKey(manifest.QuerySignature.Value, manifest.Format);
_store[key] = manifest; _store[key] = manifest;
LastSavedManifest = manifest;
return ValueTask.CompletedTask; return ValueTask.CompletedTask;
} }
@@ -110,6 +147,28 @@ public sealed class ExportEngineTests
=> FormattableString.Invariant($"{signature}|{format}"); => FormattableString.Invariant($"{signature}|{format}");
} }
private sealed class RecordingAttestationClient : IVexAttestationClient
{
public VexAttestationRequest? LastRequest { get; private set; }
public VexAttestationResponse Response { get; } = new VexAttestationResponse(
new VexAttestationMetadata(
predicateType: "https://stella-ops.org/attestations/vex-export",
rekor: new VexRekorReference("0.2", "rekor://entry", "123"),
envelopeDigest: "sha256:envelope",
signedAt: DateTimeOffset.UnixEpoch),
ImmutableDictionary<string, string>.Empty);
public ValueTask<VexAttestationResponse> SignAsync(VexAttestationRequest request, CancellationToken cancellationToken)
{
LastRequest = request;
return ValueTask.FromResult(Response);
}
public ValueTask<VexAttestationVerification> VerifyAsync(VexAttestationRequest request, CancellationToken cancellationToken)
=> ValueTask.FromResult(new VexAttestationVerification(true, ImmutableDictionary<string, string>.Empty));
}
private sealed class RecordingCacheIndex : IVexCacheIndex private sealed class RecordingCacheIndex : IVexCacheIndex
{ {
public Dictionary<(string Signature, VexExportFormat Format), bool> RemoveCalls { get; } = new(); public Dictionary<(string Signature, VexExportFormat Format), bool> RemoveCalls { get; } = new();
@@ -213,4 +272,5 @@ public sealed class ExportEngineTests
return ValueTask.FromResult(new VexExportResult(new VexContentAddress("sha256", "deadbeef"), bytes.Length, ImmutableDictionary<string, string>.Empty)); return ValueTask.FromResult(new VexExportResult(new VexContentAddress("sha256", "deadbeef"), bytes.Length, ImmutableDictionary<string, string>.Empty));
} }
} }
} }

39
src/StellaOps.Vexer.Export/ExportEngine.cs
View File

@@ -39,6 +39,7 @@ public sealed class VexExportEngine : IExportEngine
private readonly ILogger<VexExportEngine> _logger; private readonly ILogger<VexExportEngine> _logger;
private readonly IVexCacheIndex? _cacheIndex; private readonly IVexCacheIndex? _cacheIndex;
private readonly IReadOnlyList<IVexArtifactStore> _artifactStores; private readonly IReadOnlyList<IVexArtifactStore> _artifactStores;
private readonly IVexAttestationClient? _attestationClient;
public VexExportEngine( public VexExportEngine(
IVexExportStore exportStore, IVexExportStore exportStore,
@@ -47,7 +48,8 @@ public sealed class VexExportEngine : IExportEngine
IEnumerable<IVexExporter> exporters, IEnumerable<IVexExporter> exporters,
ILogger<VexExportEngine> logger, ILogger<VexExportEngine> logger,
IVexCacheIndex? cacheIndex = null, IVexCacheIndex? cacheIndex = null,
IEnumerable<IVexArtifactStore>? artifactStores = null) IEnumerable<IVexArtifactStore>? artifactStores = null,
IVexAttestationClient? attestationClient = null)
{ {
_exportStore = exportStore ?? throw new ArgumentNullException(nameof(exportStore)); _exportStore = exportStore ?? throw new ArgumentNullException(nameof(exportStore));
_policyEvaluator = policyEvaluator ?? throw new ArgumentNullException(nameof(policyEvaluator)); _policyEvaluator = policyEvaluator ?? throw new ArgumentNullException(nameof(policyEvaluator));
@@ -55,6 +57,7 @@ public sealed class VexExportEngine : IExportEngine
_logger = logger ?? throw new ArgumentNullException(nameof(logger)); _logger = logger ?? throw new ArgumentNullException(nameof(logger));
_cacheIndex = cacheIndex; _cacheIndex = cacheIndex;
_artifactStores = artifactStores?.ToArray() ?? Array.Empty<IVexArtifactStore>(); _artifactStores = artifactStores?.ToArray() ?? Array.Empty<IVexArtifactStore>();
_attestationClient = attestationClient;
if (exporters is null) if (exporters is null)
{ {
@@ -105,6 +108,7 @@ public sealed class VexExportEngine : IExportEngine
context.RequestedAt); context.RequestedAt);
var digest = exporter.Digest(exportRequest); var digest = exporter.Digest(exportRequest);
var exportId = FormattableString.Invariant($"exports/{context.RequestedAt:yyyyMMddTHHmmssfffZ}/{digest.Digest}");
await using var buffer = new MemoryStream(); await using var buffer = new MemoryStream();
var result = await exporter.SerializeAsync(exportRequest, buffer, cancellationToken).ConfigureAwait(false); var result = await exporter.SerializeAsync(exportRequest, buffer, cancellationToken).ConfigureAwait(false);
@@ -134,7 +138,36 @@ public sealed class VexExportEngine : IExportEngine
} }
} }
var exportId = FormattableString.Invariant($"exports/{context.RequestedAt:yyyyMMddTHHmmssfffZ}/{digest.Digest}"); VexAttestationMetadata? attestationMetadata = null;
if (_attestationClient is not null)
{
var attestationRequest = new VexAttestationRequest(
exportId,
signature,
digest,
context.Format,
context.RequestedAt,
dataset.SourceProviders,
result.Metadata);
var response = await _attestationClient.SignAsync(attestationRequest, cancellationToken).ConfigureAwait(false);
attestationMetadata = response.Attestation;
if (!response.Diagnostics.IsEmpty)
{
foreach (var diagnostic in response.Diagnostics)
{
_logger.LogDebug(
"Attestation diagnostic {Key}={Value} for export {ExportId}",
diagnostic.Key,
diagnostic.Value,
exportId);
}
}
_logger.LogInformation("Attestation generated for export {ExportId}", exportId);
}
var manifest = new VexExportManifest( var manifest = new VexExportManifest(
exportId, exportId,
signature, signature,
@@ -145,7 +178,7 @@ public sealed class VexExportEngine : IExportEngine
dataset.SourceProviders, dataset.SourceProviders,
fromCache: false, fromCache: false,
consensusRevision: _policyEvaluator.Version, consensusRevision: _policyEvaluator.Version,
attestation: null, attestation: attestationMetadata,
sizeBytes: result.BytesWritten); sizeBytes: result.BytesWritten);
await _exportStore.SaveAsync(manifest, cancellationToken).ConfigureAwait(false); await _exportStore.SaveAsync(manifest, cancellationToken).ConfigureAwait(false);

2
src/StellaOps.Vexer.Export/TASKS.md
View File

@@ -5,5 +5,5 @@ If you are working on this file you need to read docs/ARCHITECTURE_VEXER.md and
|VEXER-EXPORT-01-001 – Export engine orchestration|Team Vexer Export|VEXER-CORE-01-003|DONE (2025-10-15) – Export engine scaffolding with cache lookup, data source hooks, and deterministic manifest emission.| |VEXER-EXPORT-01-001 – Export engine orchestration|Team Vexer Export|VEXER-CORE-01-003|DONE (2025-10-15) – Export engine scaffolding with cache lookup, data source hooks, and deterministic manifest emission.|
|VEXER-EXPORT-01-002 – Cache index & eviction hooks|Team Vexer Export|VEXER-EXPORT-01-001, VEXER-STORAGE-01-003|**DONE (2025-10-16)** – Export engine now invalidates cache entries on force refresh, cache services expose prune/invalidate APIs, and storage maintenance trims expired/dangling records with Mongo2Go coverage.| |VEXER-EXPORT-01-002 – Cache index & eviction hooks|Team Vexer Export|VEXER-EXPORT-01-001, VEXER-STORAGE-01-003|**DONE (2025-10-16)** – Export engine now invalidates cache entries on force refresh, cache services expose prune/invalidate APIs, and storage maintenance trims expired/dangling records with Mongo2Go coverage.|
|VEXER-EXPORT-01-003 – Artifact store adapters|Team Vexer Export|VEXER-EXPORT-01-001|**DONE (2025-10-16)** – Implemented multi-store pipeline with filesystem, S3-compatible, and offline bundle adapters (hash verification + manifest/zip output) plus unit coverage and DI hooks.| |VEXER-EXPORT-01-003 – Artifact store adapters|Team Vexer Export|VEXER-EXPORT-01-001|**DONE (2025-10-16)** – Implemented multi-store pipeline with filesystem, S3-compatible, and offline bundle adapters (hash verification + manifest/zip output) plus unit coverage and DI hooks.|
|VEXER-EXPORT-01-004 – Attestation handoff integration|Team Vexer Export|VEXER-EXPORT-01-001, VEXER-ATTEST-01-001|TODO – Connect export engine to attestation client, persist Rekor metadata, and reuse cached attestations.| |VEXER-EXPORT-01-004 – Attestation handoff integration|Team Vexer Export|VEXER-EXPORT-01-001, VEXER-ATTEST-01-001|**DONE (2025-10-17)** – Export engine now invokes attestation client, logs diagnostics, and persists Rekor/envelope metadata on manifests; regression coverage added in `ExportEngineTests.ExportAsync_AttachesAttestationMetadata`.|
|VEXER-EXPORT-01-005 – Score & resolve envelope surfaces|Team Vexer Export|VEXER-EXPORT-01-004, VEXER-CORE-02-001|TODO – Emit consensus+score envelopes in export manifests, include policy/scoring digests, and update offline bundle/ORAS layouts to carry signed VEX responses.| |VEXER-EXPORT-01-005 – Score & resolve envelope surfaces|Team Vexer Export|VEXER-EXPORT-01-004, VEXER-CORE-02-001|TODO – Emit consensus+score envelopes in export manifests, include policy/scoring digests, and update offline bundle/ORAS layouts to carry signed VEX responses.|

131
src/StellaOps.Vexer.Formats.CSAF.Tests/CsafNormalizerTests.cs Normal file
View File

@@ -0,0 +1,131 @@
using System.Collections.Immutable;
using System.Linq;
using System.Text;
using System.IO;
using FluentAssertions;
using Microsoft.Extensions.Logging.Abstractions;
using StellaOps.Vexer.Core;
using StellaOps.Vexer.Formats.CSAF;
namespace StellaOps.Vexer.Formats.CSAF.Tests;
public sealed class CsafNormalizerTests
{
[Fact]
public async Task NormalizeAsync_ProducesClaimsPerProductStatus()
{
var json = """
{
"document": {
"tracking": {
"id": "RHSA-2025:0001",
"version": "3",
"revision": "3",
"status": "final",
"initial_release_date": "2025-10-01T00:00:00Z",
"current_release_date": "2025-10-10T00:00:00Z"
},
"publisher": {
"name": "Red Hat Product Security",
"category": "vendor"
}
},
"product_tree": {
"full_product_names": [
{
"product_id": "CSAFPID-0001",
"name": "Red Hat Enterprise Linux 9",
"product_identification_helper": {
"cpe": "cpe:/o:redhat:enterprise_linux:9",
"purl": "pkg:rpm/redhat/enterprise-linux@9"
}
}
]
},
"vulnerabilities": [
{
"cve": "CVE-2025-0001",
"title": "Kernel vulnerability",
"product_status": {
"known_affected": [ "CSAFPID-0001" ]
}
},
{
"id": "VEX-0002",
"title": "Library issue",
"product_status": {
"known_not_affected": [ "CSAFPID-0001" ]
}
}
]
}
""";
var rawDocument = new VexRawDocument(
ProviderId: "vexer:redhat",
VexDocumentFormat.Csaf,
new Uri("https://example.com/csaf/rhsa-2025-0001.json"),
new DateTimeOffset(2025, 10, 11, 0, 0, 0, TimeSpan.Zero),
"sha256:dummydigest",
Encoding.UTF8.GetBytes(json),
ImmutableDictionary<string, string>.Empty);
var provider = new VexProvider("vexer:redhat", "Red Hat CSAF", VexProviderKind.Distro);
var normalizer = new CsafNormalizer(NullLogger<CsafNormalizer>.Instance);
var batch = await normalizer.NormalizeAsync(rawDocument, provider, CancellationToken.None);
batch.Claims.Should().HaveCount(2);
var affectedClaim = batch.Claims.First(c => c.VulnerabilityId == "CVE-2025-0001");
affectedClaim.Status.Should().Be(VexClaimStatus.Affected);
affectedClaim.Product.Key.Should().Be("CSAFPID-0001");
affectedClaim.Product.Purl.Should().Be("pkg:rpm/redhat/enterprise-linux@9");
affectedClaim.Product.Cpe.Should().Be("cpe:/o:redhat:enterprise_linux:9");
affectedClaim.Document.Revision.Should().Be("3");
affectedClaim.FirstSeen.Should().Be(new DateTimeOffset(2025, 10, 1, 0, 0, 0, TimeSpan.Zero));
affectedClaim.LastSeen.Should().Be(new DateTimeOffset(2025, 10, 10, 0, 0, 0, TimeSpan.Zero));
affectedClaim.AdditionalMetadata.Should().ContainKey("csaf.tracking.id");
affectedClaim.AdditionalMetadata["csaf.publisher.name"].Should().Be("Red Hat Product Security");
var notAffectedClaim = batch.Claims.First(c => c.VulnerabilityId == "VEX-0002");
notAffectedClaim.Status.Should().Be(VexClaimStatus.NotAffected);
}
[Fact]
public async Task NormalizeAsync_PreservesRedHatSpecificMetadata()
{
var path = Path.Combine(AppContext.BaseDirectory, "Fixtures", "rhsa-sample.json");
var json = await File.ReadAllTextAsync(path);
var rawDocument = new VexRawDocument(
ProviderId: "vexer:redhat",
VexDocumentFormat.Csaf,
new Uri("https://security.example.com/rhsa-2025-1001.json"),
new DateTimeOffset(2025, 10, 6, 0, 0, 0, TimeSpan.Zero),
"sha256:rhdadigest",
Encoding.UTF8.GetBytes(json),
ImmutableDictionary<string, string>.Empty);
var provider = new VexProvider("vexer:redhat", "Red Hat CSAF", VexProviderKind.Distro);
var normalizer = new CsafNormalizer(NullLogger<CsafNormalizer>.Instance);
var batch = await normalizer.NormalizeAsync(rawDocument, provider, CancellationToken.None);
batch.Claims.Should().ContainSingle();
var claim = batch.Claims[0];
claim.VulnerabilityId.Should().Be("CVE-2025-1234");
claim.Status.Should().Be(VexClaimStatus.Affected);
claim.Product.Key.Should().Be("rh-enterprise-linux-9");
claim.Product.Name.Should().Be("Red Hat Enterprise Linux 9");
claim.Product.Purl.Should().Be("pkg:rpm/redhat/enterprise-linux@9");
claim.Product.Cpe.Should().Be("cpe:/o:redhat:enterprise_linux:9");
claim.FirstSeen.Should().Be(new DateTimeOffset(2025, 10, 1, 12, 0, 0, TimeSpan.Zero));
claim.LastSeen.Should().Be(new DateTimeOffset(2025, 10, 5, 10, 0, 0, TimeSpan.Zero));
claim.AdditionalMetadata.Should().ContainKey("csaf.tracking.id");
claim.AdditionalMetadata["csaf.tracking.id"].Should().Be("RHSA-2025:1001");
claim.AdditionalMetadata["csaf.tracking.status"].Should().Be("final");
claim.AdditionalMetadata["csaf.publisher.name"].Should().Be("Red Hat Product Security");
}
}

47
src/StellaOps.Vexer.Formats.CSAF.Tests/Fixtures/rhsa-sample.json Normal file
View File

@@ -0,0 +1,47 @@
{
"document": {
"publisher": {
"name": "Red Hat Product Security",
"category": "vendor"
},
"tracking": {
"id": "RHSA-2025:1001",
"status": "final",
"version": "3",
"initial_release_date": "2025-10-01T12:00:00Z",
"current_release_date": "2025-10-05T10:00:00Z"
}
},
"product_tree": {
"full_product_names": [
{
"product_id": "rh-enterprise-linux-9",
"name": "Red Hat Enterprise Linux 9",
"product_identification_helper": {
"cpe": "cpe:/o:redhat:enterprise_linux:9",
"purl": "pkg:rpm/redhat/enterprise-linux@9"
}
}
],
"branches": [
{
"name": "Red Hat Enterprise Linux",
"product": {
"product_id": "rh-enterprise-linux-9",
"name": "Red Hat Enterprise Linux 9"
}
}
]
},
"vulnerabilities": [
{
"cve": "CVE-2025-1234",
"title": "Kernel privilege escalation",
"product_status": {
"known_affected": [
"rh-enterprise-linux-9"
]
}
}
]
}

20
src/StellaOps.Vexer.Formats.CSAF.Tests/StellaOps.Vexer.Formats.CSAF.Tests.csproj Normal file
View File

@@ -0,0 +1,20 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>net10.0</TargetFramework>
<Nullable>enable</Nullable>
<ImplicitUsings>enable</ImplicitUsings>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="FluentAssertions" Version="6.12.0" />
</ItemGroup>
<ItemGroup>
<Using Include="Xunit" />
</ItemGroup>
<ItemGroup>
<ProjectReference Include="..\StellaOps.Vexer.Core\StellaOps.Vexer.Core.csproj" />
<ProjectReference Include="..\StellaOps.Vexer.Formats.CSAF\StellaOps.Vexer.Formats.CSAF.csproj" />
</ItemGroup>
<ItemGroup>
<None Include="Fixtures\**\*" CopyToOutputDirectory="Always" />
</ItemGroup>
</Project>

532
src/StellaOps.Vexer.Formats.CSAF/CsafNormalizer.cs Normal file
View File

@@ -0,0 +1,532 @@
using System;
using System.Collections.Generic;
using System.Collections.Immutable;
using System.Linq;
using System.Text.Json;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.Extensions.Logging;
using StellaOps.Vexer.Core;
namespace StellaOps.Vexer.Formats.CSAF;
public sealed class CsafNormalizer : IVexNormalizer
{
private static readonly ImmutableDictionary<VexClaimStatus, int> StatusPrecedence = new Dictionary<VexClaimStatus, int>
{
[VexClaimStatus.UnderInvestigation] = 0,
[VexClaimStatus.Affected] = 1,
[VexClaimStatus.NotAffected] = 2,
[VexClaimStatus.Fixed] = 3,
}.ToImmutableDictionary();
private readonly ILogger<CsafNormalizer> _logger;
public CsafNormalizer(ILogger<CsafNormalizer> logger)
{
_logger = logger ?? throw new ArgumentNullException(nameof(logger));
}
public string Format => VexDocumentFormat.Csaf.ToString().ToLowerInvariant();
public bool CanHandle(VexRawDocument document)
=> document is not null && document.Format == VexDocumentFormat.Csaf;
public ValueTask<VexClaimBatch> NormalizeAsync(VexRawDocument document, VexProvider provider, CancellationToken cancellationToken)
{
ArgumentNullException.ThrowIfNull(document);
ArgumentNullException.ThrowIfNull(provider);
cancellationToken.ThrowIfCancellationRequested();
try
{
var result = CsafParser.Parse(document);
var claims = ImmutableArray.CreateBuilder<VexClaim>(result.Claims.Length);
foreach (var entry in result.Claims)
{
var product = new VexProduct(
entry.Product.ProductId,
entry.Product.Name,
entry.Product.Version,
entry.Product.Purl,
entry.Product.Cpe);
var claimDocument = new VexClaimDocument(
VexDocumentFormat.Csaf,
document.Digest,
document.SourceUri,
result.Revision,
signature: null);
var claim = new VexClaim(
entry.VulnerabilityId,
provider.Id,
product,
entry.Status,
claimDocument,
result.FirstRelease,
result.LastRelease,
justification: null,
detail: entry.Detail,
confidence: null,
additionalMetadata: result.Metadata);
claims.Add(claim);
}
var orderedClaims = claims
.ToImmutable()
.OrderBy(static claim => claim.VulnerabilityId, StringComparer.Ordinal)
.ThenBy(static claim => claim.Product.Key, StringComparer.Ordinal)
.ToImmutableArray();
_logger.LogInformation(
"Normalized CSAF document {Source} into {ClaimCount} claim(s).",
document.SourceUri,
orderedClaims.Length);
return ValueTask.FromResult(new VexClaimBatch(
document,
orderedClaims,
ImmutableDictionary<string, string>.Empty));
}
catch (JsonException ex)
{
_logger.LogError(ex, "Failed to parse CSAF document {SourceUri}", document.SourceUri);
throw;
}
}
private static class CsafParser
{
public static CsafParseResult Parse(VexRawDocument document)
{
using var json = JsonDocument.Parse(document.Content.ToArray());
var root = json.RootElement;
var tracking = root.TryGetProperty("document", out var documentElement) &&
documentElement.ValueKind == JsonValueKind.Object &&
documentElement.TryGetProperty("tracking", out var trackingElement)
? trackingElement
: default;
var firstRelease = ParseDate(tracking, "initial_release_date") ?? document.RetrievedAt;
var lastRelease = ParseDate(tracking, "current_release_date") ?? firstRelease;
if (lastRelease < firstRelease)
{
lastRelease = firstRelease;
}
var metadataBuilder = ImmutableDictionary.CreateBuilder<string, string>(StringComparer.Ordinal);
AddIfPresent(metadataBuilder, tracking, "id", "csaf.tracking.id");
AddIfPresent(metadataBuilder, tracking, "version", "csaf.tracking.version");
AddIfPresent(metadataBuilder, tracking, "status", "csaf.tracking.status");
AddPublisherMetadata(metadataBuilder, documentElement);
var revision = TryGetString(tracking, "revision");
var productCatalog = CollectProducts(root);
var claimsBuilder = ImmutableArray.CreateBuilder<CsafClaimEntry>();
if (root.TryGetProperty("vulnerabilities", out var vulnerabilitiesElement) &&
vulnerabilitiesElement.ValueKind == JsonValueKind.Array)
{
foreach (var vulnerability in vulnerabilitiesElement.EnumerateArray())
{
var vulnerabilityId = ResolveVulnerabilityId(vulnerability);
if (string.IsNullOrWhiteSpace(vulnerabilityId))
{
continue;
}
var detail = ResolveDetail(vulnerability);
var productClaims = BuildClaimsForVulnerability(
vulnerabilityId,
vulnerability,
productCatalog,
detail);
claimsBuilder.AddRange(productClaims);
}
}
return new CsafParseResult(
firstRelease,
lastRelease,
revision,
metadataBuilder.ToImmutable(),
claimsBuilder.ToImmutable());
}
private static IReadOnlyList<CsafClaimEntry> BuildClaimsForVulnerability(
string vulnerabilityId,
JsonElement vulnerability,
IReadOnlyDictionary<string, CsafProductInfo> productCatalog,
string? detail)
{
if (!vulnerability.TryGetProperty("product_status", out var statusElement) ||
statusElement.ValueKind != JsonValueKind.Object)
{
return Array.Empty<CsafClaimEntry>();
}
var claims = new Dictionary<string, CsafClaimEntryBuilder>(StringComparer.OrdinalIgnoreCase);
foreach (var statusProperty in statusElement.EnumerateObject())
{
var status = MapStatus(statusProperty.Name);
if (status is null)
{
continue;
}
if (statusProperty.Value.ValueKind != JsonValueKind.Array)
{
continue;
}
foreach (var productIdElement in statusProperty.Value.EnumerateArray())
{
var productId = productIdElement.GetString();
if (string.IsNullOrWhiteSpace(productId))
{
continue;
}
var product = ResolveProduct(productCatalog, productId);
UpdateClaim(claims, product, status.Value, detail);
}
}
if (claims.Count == 0)
{
return Array.Empty<CsafClaimEntry>();
}
return claims.Values
.Select(builder => new CsafClaimEntry(
vulnerabilityId,
builder.Product,
builder.Status,
builder.Detail))
.ToArray();
}
private static void UpdateClaim(
IDictionary<string, CsafClaimEntryBuilder> claims,
CsafProductInfo product,
VexClaimStatus status,
string? detail)
{
if (!claims.TryGetValue(product.ProductId, out var existing) ||
StatusPrecedence[status] > StatusPrecedence[existing.Status])
{
claims[product.ProductId] = new CsafClaimEntryBuilder(product, status, detail);
}
}
private static CsafProductInfo ResolveProduct(
IReadOnlyDictionary<string, CsafProductInfo> catalog,
string productId)
{
if (catalog.TryGetValue(productId, out var product))
{
return product;
}
return new CsafProductInfo(productId, productId, null, null, null);
}
private static string ResolveVulnerabilityId(JsonElement vulnerability)
{
var id = TryGetString(vulnerability, "cve")
?? TryGetString(vulnerability, "id")
?? TryGetString(vulnerability, "vuln_id");
return string.IsNullOrWhiteSpace(id) ? string.Empty : id.Trim();
}
private static string? ResolveDetail(JsonElement vulnerability)
{
var title = TryGetString(vulnerability, "title");
if (!string.IsNullOrWhiteSpace(title))
{
return title.Trim();
}
if (vulnerability.TryGetProperty("notes", out var notesElement) &&
notesElement.ValueKind == JsonValueKind.Array)
{
foreach (var note in notesElement.EnumerateArray())
{
if (note.ValueKind != JsonValueKind.Object)
{
continue;
}
var category = TryGetString(note, "category");
if (!string.IsNullOrWhiteSpace(category) &&
!string.Equals(category, "description", StringComparison.OrdinalIgnoreCase))
{
continue;
}
var text = TryGetString(note, "text");
if (!string.IsNullOrWhiteSpace(text))
{
return text.Trim();
}
}
}
return null;
}
private static Dictionary<string, CsafProductInfo> CollectProducts(JsonElement root)
{
var products = new Dictionary<string, CsafProductInfo>(StringComparer.OrdinalIgnoreCase);
if (!root.TryGetProperty("product_tree", out var productTree) ||
productTree.ValueKind != JsonValueKind.Object)
{
return products;
}
if (productTree.TryGetProperty("full_product_names", out var fullNames) &&
fullNames.ValueKind == JsonValueKind.Array)
{
foreach (var productEntry in fullNames.EnumerateArray())
{
var product = ParseProduct(productEntry, parentBranchName: null);
if (product is not null)
{
AddOrUpdate(product);
}
}
}
if (productTree.TryGetProperty("branches", out var branches) &&
branches.ValueKind == JsonValueKind.Array)
{
foreach (var branch in branches.EnumerateArray())
{
VisitBranch(branch, parentBranchName: null);
}
}
return products;
void VisitBranch(JsonElement branch, string? parentBranchName)
{
if (branch.ValueKind != JsonValueKind.Object)
{
return;
}
var branchName = TryGetString(branch, "name") ?? parentBranchName;
if (branch.TryGetProperty("product", out var productElement))
{
var product = ParseProduct(productElement, branchName);
if (product is not null)
{
AddOrUpdate(product);
}
}
if (branch.TryGetProperty("branches", out var childBranches) &&
childBranches.ValueKind == JsonValueKind.Array)
{
foreach (var childBranch in childBranches.EnumerateArray())
{
VisitBranch(childBranch, branchName);
}
}
}
void AddOrUpdate(CsafProductInfo product)
{
if (products.TryGetValue(product.ProductId, out var existing))
{
products[product.ProductId] = MergeProducts(existing, product);
}
else
{
products[product.ProductId] = product;
}
}
static CsafProductInfo MergeProducts(CsafProductInfo existing, CsafProductInfo incoming)
{
static string ChooseName(string incoming, string fallback)
=> string.IsNullOrWhiteSpace(incoming) ? fallback : incoming;
static string? ChooseOptional(string? incoming, string? fallback)
=> string.IsNullOrWhiteSpace(incoming) ? fallback : incoming;
return new CsafProductInfo(
existing.ProductId,
ChooseName(incoming.Name, existing.Name),
ChooseOptional(incoming.Version, existing.Version),
ChooseOptional(incoming.Purl, existing.Purl),
ChooseOptional(incoming.Cpe, existing.Cpe));
}
}
private static CsafProductInfo? ParseProduct(JsonElement element, string? parentBranchName)
{
if (element.ValueKind != JsonValueKind.Object)
{
return null;
}
JsonElement productElement = element;
if (!element.TryGetProperty("product_id", out var idElement) &&
element.TryGetProperty("product", out var nestedProduct) &&
nestedProduct.ValueKind == JsonValueKind.Object &&
nestedProduct.TryGetProperty("product_id", out idElement))
{
productElement = nestedProduct;
}
var productId = idElement.GetString();
if (string.IsNullOrWhiteSpace(productId))
{
return null;
}
var name = TryGetString(productElement, "name")
?? TryGetString(element, "name")
?? parentBranchName
?? productId;
var version = TryGetString(productElement, "product_version")
?? TryGetString(productElement, "version")
?? TryGetString(element, "product_version");
string? cpe = null;
string? purl = null;
if (productElement.TryGetProperty("product_identification_helper", out var helper) &&
helper.ValueKind == JsonValueKind.Object)
{
cpe = TryGetString(helper, "cpe");
purl = TryGetString(helper, "purl");
}
return new CsafProductInfo(productId.Trim(), name.Trim(), version?.Trim(), purl?.Trim(), cpe?.Trim());
}
private static VexClaimStatus? MapStatus(string statusName)
{
if (string.IsNullOrWhiteSpace(statusName))
{
return null;
}
return statusName switch
{
"known_affected" or "fixed_after_release" or "first_affected" or "last_affected" => VexClaimStatus.Affected,
"known_not_affected" or "last_not_affected" or "first_not_affected" => VexClaimStatus.NotAffected,
"fixed" or "first_fixed" or "last_fixed" => VexClaimStatus.Fixed,
"under_investigation" or "investigating" => VexClaimStatus.UnderInvestigation,
_ => null,
};
}
private static DateTimeOffset? ParseDate(JsonElement element, string propertyName)
{
if (element.ValueKind != JsonValueKind.Object)
{
return null;
}
if (!element.TryGetProperty(propertyName, out var dateElement))
{
return null;
}
var value = dateElement.GetString();
if (string.IsNullOrWhiteSpace(value))
{
return null;
}
if (DateTimeOffset.TryParse(value, out var parsed))
{
return parsed;
}
return null;
}
private static string? TryGetString(JsonElement element, string propertyName)
{
if (element.ValueKind != JsonValueKind.Object)
{
return null;
}
if (!element.TryGetProperty(propertyName, out var property))
{
return null;
}
return property.ValueKind == JsonValueKind.String ? property.GetString() : null;
}
private static void AddIfPresent(
ImmutableDictionary<string, string>.Builder builder,
JsonElement element,
string propertyName,
string metadataKey)
{
var value = TryGetString(element, propertyName);
if (!string.IsNullOrWhiteSpace(value))
{
builder[metadataKey] = value.Trim();
}
}
private static void AddPublisherMetadata(
ImmutableDictionary<string, string>.Builder builder,
JsonElement documentElement)
{
if (documentElement.ValueKind != JsonValueKind.Object ||
!documentElement.TryGetProperty("publisher", out var publisher) ||
publisher.ValueKind != JsonValueKind.Object)
{
return;
}
AddIfPresent(builder, publisher, "name", "csaf.publisher.name");
AddIfPresent(builder, publisher, "category", "csaf.publisher.category");
}
private readonly record struct CsafClaimEntryBuilder(
CsafProductInfo Product,
VexClaimStatus Status,
string? Detail);
}
private sealed record CsafParseResult(
DateTimeOffset FirstRelease,
DateTimeOffset LastRelease,
string? Revision,
ImmutableDictionary<string, string> Metadata,
ImmutableArray<CsafClaimEntry> Claims);
private sealed record CsafClaimEntry(
string VulnerabilityId,
CsafProductInfo Product,
VexClaimStatus Status,
string? Detail);
private sealed record CsafProductInfo(
string ProductId,
string Name,
string? Version,
string? Purl,
string? Cpe);
}

14
src/StellaOps.Vexer.Formats.CSAF/ServiceCollectionExtensions.cs Normal file
View File

@@ -0,0 +1,14 @@
using Microsoft.Extensions.DependencyInjection;
using StellaOps.Vexer.Core;
namespace StellaOps.Vexer.Formats.CSAF;
public static class CsafFormatsServiceCollectionExtensions
{
public static IServiceCollection AddCsafNormalizer(this IServiceCollection services)
{
ArgumentNullException.ThrowIfNull(services);
services.AddSingleton<IVexNormalizer, CsafNormalizer>();
return services;
}
}

16
src/StellaOps.Vexer.Formats.CSAF/StellaOps.Vexer.Formats.CSAF.csproj Normal file
View File

@@ -0,0 +1,16 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>net10.0</TargetFramework>
<LangVersion>preview</LangVersion>
<Nullable>enable</Nullable>
<ImplicitUsings>enable</ImplicitUsings>
<TreatWarningsAsErrors>true</TreatWarningsAsErrors>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Microsoft.Extensions.DependencyInjection.Abstractions" Version="8.0.0" />
<PackageReference Include="Microsoft.Extensions.Logging.Abstractions" Version="8.0.0" />
</ItemGroup>
<ItemGroup>
<ProjectReference Include="..\StellaOps.Vexer.Core\StellaOps.Vexer.Core.csproj" />
</ItemGroup>
</Project>

2
src/StellaOps.Vexer.Formats.CSAF/TASKS.md
View File

@@ -2,6 +2,6 @@ If you are working on this file you need to read docs/ARCHITECTURE_VEXER.md and
# TASKS # TASKS
| Task | Owner(s) | Depends on | Notes | | Task | Owner(s) | Depends on | Notes |
|---|---|---|---| |---|---|---|---|
|VEXER-FMT-CSAF-01-001 – CSAF normalizer foundation|Team Vexer Formats|VEXER-CORE-01-001|TODO – Implement CSAF parser covering provider metadata, document tracking, and vulnerability/product mapping into `VexClaim`.| |VEXER-FMT-CSAF-01-001 – CSAF normalizer foundation|Team Vexer Formats|VEXER-CORE-01-001|**DONE (2025-10-17)** – Implemented CSAF normalizer + DI hook, parsing tracking metadata, product tree branches/full names, and mapping product statuses into canonical `VexClaim`s with baseline precedence. Regression added in `CsafNormalizerTests`.|
|VEXER-FMT-CSAF-01-002 – Status/justification mapping|Team Vexer Formats|VEXER-FMT-CSAF-01-001, VEXER-POLICY-01-001|TODO – Normalize CSAF `product_status` + `justification` values into policy-aware enums with audit diagnostics for unsupported codes.| |VEXER-FMT-CSAF-01-002 – Status/justification mapping|Team Vexer Formats|VEXER-FMT-CSAF-01-001, VEXER-POLICY-01-001|TODO – Normalize CSAF `product_status` + `justification` values into policy-aware enums with audit diagnostics for unsupported codes.|
|VEXER-FMT-CSAF-01-003 – CSAF export adapter|Team Vexer Formats|VEXER-EXPORT-01-001, VEXER-FMT-CSAF-01-001|TODO – Provide CSAF export writer producing deterministic documents (per vuln/product) and manifest metadata for attestation.| |VEXER-FMT-CSAF-01-003 – CSAF export adapter|Team Vexer Formats|VEXER-EXPORT-01-001, VEXER-FMT-CSAF-01-001|TODO – Provide CSAF export writer producing deterministic documents (per vuln/product) and manifest metadata for attestation.|

93
src/StellaOps.Vexer.Formats.CycloneDX.Tests/CycloneDxNormalizerTests.cs Normal file
View File

@@ -0,0 +1,93 @@
using System.Collections.Immutable;
using System.Linq;
using System.Text;
using FluentAssertions;
using Microsoft.Extensions.Logging.Abstractions;
using StellaOps.Vexer.Core;
using StellaOps.Vexer.Formats.CycloneDX;
namespace StellaOps.Vexer.Formats.CycloneDX.Tests;
public sealed class CycloneDxNormalizerTests
{
[Fact]
public async Task NormalizeAsync_MapsAnalysisStateAndJustification()
{
var json = """
{
"bomFormat": "CycloneDX",
"specVersion": "1.4",
"serialNumber": "urn:uuid:1234",
"version": "7",
"metadata": {
"timestamp": "2025-10-15T12:00:00Z"
},
"components": [
{
"bom-ref": "pkg:npm/acme/lib@1.0.0",
"name": "acme-lib",
"version": "1.0.0",
"purl": "pkg:npm/acme/lib@1.0.0"
}
],
"vulnerabilities": [
{
"id": "CVE-2025-1000",
"detail": "Library issue",
"analysis": {
"state": "not_affected",
"justification": "code_not_present",
"response": [ "can_not_fix", "will_not_fix" ]
},
"affects": [
{ "ref": "pkg:npm/acme/lib@1.0.0" }
]
},
{
"id": "CVE-2025-1001",
"description": "Investigating impact",
"analysis": {
"state": "in_triage"
},
"affects": [
{ "ref": "pkg:npm/missing/component@2.0.0" }
]
}
]
}
""";
var rawDocument = new VexRawDocument(
"vexer:cyclonedx",
VexDocumentFormat.CycloneDx,
new Uri("https://example.org/vex.json"),
new DateTimeOffset(2025, 10, 16, 0, 0, 0, TimeSpan.Zero),
"sha256:dummydigest",
Encoding.UTF8.GetBytes(json),
ImmutableDictionary<string, string>.Empty);
var provider = new VexProvider("vexer:cyclonedx", "CycloneDX Provider", VexProviderKind.Vendor);
var normalizer = new CycloneDxNormalizer(NullLogger<CycloneDxNormalizer>.Instance);
var batch = await normalizer.NormalizeAsync(rawDocument, provider, CancellationToken.None);
batch.Claims.Should().HaveCount(2);
var notAffected = batch.Claims.Single(c => c.VulnerabilityId == "CVE-2025-1000");
notAffected.Status.Should().Be(VexClaimStatus.NotAffected);
notAffected.Justification.Should().Be(VexJustification.CodeNotPresent);
notAffected.Product.Key.Should().Be("pkg:npm/acme/lib@1.0.0");
notAffected.Product.Purl.Should().Be("pkg:npm/acme/lib@1.0.0");
notAffected.Document.Revision.Should().Be("7");
notAffected.AdditionalMetadata["cyclonedx.specVersion"].Should().Be("1.4");
notAffected.AdditionalMetadata["cyclonedx.analysis.state"].Should().Be("not_affected");
notAffected.AdditionalMetadata["cyclonedx.analysis.response"].Should().Be("can_not_fix,will_not_fix");
var investigating = batch.Claims.Single(c => c.VulnerabilityId == "CVE-2025-1001");
investigating.Status.Should().Be(VexClaimStatus.UnderInvestigation);
investigating.Justification.Should().BeNull();
investigating.Product.Key.Should().Be("pkg:npm/missing/component@2.0.0");
investigating.Product.Name.Should().Be("pkg:npm/missing/component@2.0.0");
investigating.AdditionalMetadata.Should().ContainKey("cyclonedx.specVersion");
}
}

17
src/StellaOps.Vexer.Formats.CycloneDX.Tests/StellaOps.Vexer.Formats.CycloneDX.Tests.csproj Normal file
View File

@@ -0,0 +1,17 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>net10.0</TargetFramework>
<Nullable>enable</Nullable>
<ImplicitUsings>enable</ImplicitUsings>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="FluentAssertions" Version="6.12.0" />
</ItemGroup>
<ItemGroup>
<Using Include="Xunit" />
</ItemGroup>
<ItemGroup>
<ProjectReference Include="..\StellaOps.Vexer.Core\StellaOps.Vexer.Core.csproj" />
<ProjectReference Include="..\StellaOps.Vexer.Formats.CycloneDX\StellaOps.Vexer.Formats.CycloneDX.csproj" />
</ItemGroup>
</Project>

459
src/StellaOps.Vexer.Formats.CycloneDX/CycloneDxNormalizer.cs Normal file
View File

@@ -0,0 +1,459 @@
using System;
using System.Collections.Generic;
using System.Collections.Immutable;
using System.Linq;
using System.Text.Json;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.Extensions.Logging;
using StellaOps.Vexer.Core;
namespace StellaOps.Vexer.Formats.CycloneDX;
public sealed class CycloneDxNormalizer : IVexNormalizer
{
private static readonly ImmutableDictionary<string, VexClaimStatus> StateMap = new Dictionary<string, VexClaimStatus>(StringComparer.OrdinalIgnoreCase)
{
["not_affected"] = VexClaimStatus.NotAffected,
["resolved"] = VexClaimStatus.Fixed,
["resolved_with_patches"] = VexClaimStatus.Fixed,
["resolved_no_fix"] = VexClaimStatus.Fixed,
["fixed"] = VexClaimStatus.Fixed,
["affected"] = VexClaimStatus.Affected,
["known_affected"] = VexClaimStatus.Affected,
["exploitable"] = VexClaimStatus.Affected,
["in_triage"] = VexClaimStatus.UnderInvestigation,
["under_investigation"] = VexClaimStatus.UnderInvestigation,
["unknown"] = VexClaimStatus.UnderInvestigation,
}.ToImmutableDictionary(StringComparer.OrdinalIgnoreCase);
private static readonly ImmutableDictionary<string, VexJustification> JustificationMap = new Dictionary<string, VexJustification>(StringComparer.OrdinalIgnoreCase)
{
["code_not_present"] = VexJustification.CodeNotPresent,
["code_not_reachable"] = VexJustification.CodeNotReachable,
["component_not_present"] = VexJustification.ComponentNotPresent,
["component_not_configured"] = VexJustification.ComponentNotConfigured,
["vulnerable_code_not_present"] = VexJustification.VulnerableCodeNotPresent,
["vulnerable_code_not_in_execute_path"] = VexJustification.VulnerableCodeNotInExecutePath,
["vulnerable_code_cannot_be_controlled_by_adversary"] = VexJustification.VulnerableCodeCannotBeControlledByAdversary,
["inline_mitigations_already_exist"] = VexJustification.InlineMitigationsAlreadyExist,
["protected_by_mitigating_control"] = VexJustification.ProtectedByMitigatingControl,
["protected_by_compensating_control"] = VexJustification.ProtectedByCompensatingControl,
["requires_configuration"] = VexJustification.RequiresConfiguration,
["requires_dependency"] = VexJustification.RequiresDependency,
["requires_environment"] = VexJustification.RequiresEnvironment,
}.ToImmutableDictionary(StringComparer.OrdinalIgnoreCase);
private readonly ILogger<CycloneDxNormalizer> _logger;
public CycloneDxNormalizer(ILogger<CycloneDxNormalizer> logger)
{
_logger = logger ?? throw new ArgumentNullException(nameof(logger));
}
public string Format => VexDocumentFormat.CycloneDx.ToString().ToLowerInvariant();
public bool CanHandle(VexRawDocument document)
=> document is not null && document.Format == VexDocumentFormat.CycloneDx;
public ValueTask<VexClaimBatch> NormalizeAsync(VexRawDocument document, VexProvider provider, CancellationToken cancellationToken)
{
ArgumentNullException.ThrowIfNull(document);
ArgumentNullException.ThrowIfNull(provider);
cancellationToken.ThrowIfCancellationRequested();
try
{
var parseResult = CycloneDxParser.Parse(document);
var baseMetadata = parseResult.Metadata;
var claimsBuilder = ImmutableArray.CreateBuilder<VexClaim>();
foreach (var vulnerability in parseResult.Vulnerabilities)
{
cancellationToken.ThrowIfCancellationRequested();
var state = MapState(vulnerability.AnalysisState, out var stateRaw);
var justification = MapJustification(vulnerability.AnalysisJustification);
var responses = vulnerability.AnalysisResponses;
foreach (var affect in vulnerability.Affects)
{
var productInfo = parseResult.ResolveProduct(affect.ComponentRef);
var product = new VexProduct(
productInfo.Key,
productInfo.Name,
productInfo.Version,
productInfo.Purl,
productInfo.Cpe);
var metadata = baseMetadata;
if (!string.IsNullOrWhiteSpace(stateRaw))
{
metadata = metadata.SetItem("cyclonedx.analysis.state", stateRaw);
}
if (!string.IsNullOrWhiteSpace(vulnerability.AnalysisJustification))
{
metadata = metadata.SetItem("cyclonedx.analysis.justification", vulnerability.AnalysisJustification);
}
if (responses.Length > 0)
{
metadata = metadata.SetItem("cyclonedx.analysis.response", string.Join(",", responses));
}
if (!string.IsNullOrWhiteSpace(affect.ComponentRef))
{
metadata = metadata.SetItem("cyclonedx.affects.ref", affect.ComponentRef);
}
var claimDocument = new VexClaimDocument(
VexDocumentFormat.CycloneDx,
document.Digest,
document.SourceUri,
parseResult.BomVersion,
signature: null);
var claim = new VexClaim(
vulnerability.VulnerabilityId,
provider.Id,
product,
state,
claimDocument,
parseResult.FirstObserved,
parseResult.LastObserved,
justification,
vulnerability.Detail,
confidence: null,
additionalMetadata: metadata);
claimsBuilder.Add(claim);
}
}
var orderedClaims = claimsBuilder
.ToImmutable()
.OrderBy(static c => c.VulnerabilityId, StringComparer.Ordinal)
.ThenBy(static c => c.Product.Key, StringComparer.Ordinal)
.ToImmutableArray();
_logger.LogInformation(
"Normalized CycloneDX document {Source} into {ClaimCount} claim(s).",
document.SourceUri,
orderedClaims.Length);
return ValueTask.FromResult(new VexClaimBatch(
document,
orderedClaims,
ImmutableDictionary<string, string>.Empty));
}
catch (JsonException ex)
{
_logger.LogError(ex, "Failed to parse CycloneDX VEX document {SourceUri}", document.SourceUri);
throw;
}
}
private static VexClaimStatus MapState(string? state, out string? raw)
{
raw = state?.Trim();
if (!string.IsNullOrWhiteSpace(state) && StateMap.TryGetValue(state.Trim(), out var mapped))
{
return mapped;
}
return VexClaimStatus.UnderInvestigation;
}
private static VexJustification? MapJustification(string? justification)
{
if (string.IsNullOrWhiteSpace(justification))
{
return null;
}
return JustificationMap.TryGetValue(justification.Trim(), out var mapped)
? mapped
: null;
}
private sealed class CycloneDxParser
{
public static CycloneDxParseResult Parse(VexRawDocument document)
{
using var json = JsonDocument.Parse(document.Content.ToArray());
var root = json.RootElement;
var specVersion = TryGetString(root, "specVersion");
var bomVersion = TryGetString(root, "version");
var serialNumber = TryGetString(root, "serialNumber");
var metadataTimestamp = ParseDate(TryGetProperty(root, "metadata"), "timestamp");
var observedTimestamp = metadataTimestamp ?? document.RetrievedAt;
var metadataBuilder = ImmutableDictionary.CreateBuilder<string, string>(StringComparer.Ordinal);
if (!string.IsNullOrWhiteSpace(specVersion))
{
metadataBuilder["cyclonedx.specVersion"] = specVersion!;
}
if (!string.IsNullOrWhiteSpace(bomVersion))
{
metadataBuilder["cyclonedx.version"] = bomVersion!;
}
if (!string.IsNullOrWhiteSpace(serialNumber))
{
metadataBuilder["cyclonedx.serialNumber"] = serialNumber!;
}
var components = CollectComponents(root);
var vulnerabilities = CollectVulnerabilities(root);
return new CycloneDxParseResult(
metadataBuilder.ToImmutable(),
bomVersion,
observedTimestamp,
observedTimestamp,
components,
vulnerabilities);
}
private static ImmutableDictionary<string, CycloneDxComponent> CollectComponents(JsonElement root)
{
var builder = ImmutableDictionary.CreateBuilder<string, CycloneDxComponent>(StringComparer.Ordinal);
if (root.TryGetProperty("components", out var components) && components.ValueKind == JsonValueKind.Array)
{
foreach (var component in components.EnumerateArray())
{
if (component.ValueKind != JsonValueKind.Object)
{
continue;
}
var reference = TryGetString(component, "bom-ref") ?? TryGetString(component, "bomRef");
if (string.IsNullOrWhiteSpace(reference))
{
continue;
}
var name = TryGetString(component, "name") ?? reference;
var version = TryGetString(component, "version");
var purl = TryGetString(component, "purl");
string? cpe = null;
if (component.TryGetProperty("externalReferences", out var externalRefs) && externalRefs.ValueKind == JsonValueKind.Array)
{
foreach (var referenceEntry in externalRefs.EnumerateArray())
{
if (referenceEntry.ValueKind != JsonValueKind.Object)
{
continue;
}
var type = TryGetString(referenceEntry, "type");
if (!string.Equals(type, "cpe", StringComparison.OrdinalIgnoreCase))
{
continue;
}
if (referenceEntry.TryGetProperty("url", out var url) && url.ValueKind == JsonValueKind.String)
{
cpe = url.GetString();
break;
}
}
}
builder[reference!] = new CycloneDxComponent(reference!, name ?? reference!, version, purl, cpe);
}
}
return builder.ToImmutable();
}
private static ImmutableArray<CycloneDxVulnerability> CollectVulnerabilities(JsonElement root)
{
if (!root.TryGetProperty("vulnerabilities", out var vulnerabilitiesElement) ||
vulnerabilitiesElement.ValueKind != JsonValueKind.Array)
{
return ImmutableArray<CycloneDxVulnerability>.Empty;
}
var builder = ImmutableArray.CreateBuilder<CycloneDxVulnerability>();
foreach (var vulnerability in vulnerabilitiesElement.EnumerateArray())
{
if (vulnerability.ValueKind != JsonValueKind.Object)
{
continue;
}
var vulnerabilityId =
TryGetString(vulnerability, "id") ??
TryGetString(vulnerability, "bom-ref") ??
TryGetString(vulnerability, "bomRef") ??
TryGetString(vulnerability, "cve");
if (string.IsNullOrWhiteSpace(vulnerabilityId))
{
continue;
}
var detail = TryGetString(vulnerability, "detail") ?? TryGetString(vulnerability, "description");
var analysis = TryGetProperty(vulnerability, "analysis");
var analysisState = TryGetString(analysis, "state");
var analysisJustification = TryGetString(analysis, "justification");
var analysisResponses = CollectResponses(analysis);
var affects = CollectAffects(vulnerability);
if (affects.Length == 0)
{
continue;
}
builder.Add(new CycloneDxVulnerability(
vulnerabilityId.Trim(),
detail?.Trim(),
analysisState,
analysisJustification,
analysisResponses,
affects));
}
return builder.ToImmutable();
}
private static ImmutableArray<string> CollectResponses(JsonElement analysis)
{
if (analysis.ValueKind != JsonValueKind.Object ||
!analysis.TryGetProperty("response", out var responseElement) ||
responseElement.ValueKind != JsonValueKind.Array)
{
return ImmutableArray<string>.Empty;
}
var responses = new SortedSet<string>(StringComparer.OrdinalIgnoreCase);
foreach (var response in responseElement.EnumerateArray())
{
if (response.ValueKind == JsonValueKind.String)
{
var value = response.GetString();
if (!string.IsNullOrWhiteSpace(value))
{
responses.Add(value.Trim());
}
}
}
return responses.Count == 0 ? ImmutableArray<string>.Empty : responses.ToImmutableArray();
}
private static ImmutableArray<CycloneDxAffect> CollectAffects(JsonElement vulnerability)
{
if (!vulnerability.TryGetProperty("affects", out var affectsElement) ||
affectsElement.ValueKind != JsonValueKind.Array)
{
return ImmutableArray<CycloneDxAffect>.Empty;
}
var builder = ImmutableArray.CreateBuilder<CycloneDxAffect>();
foreach (var affect in affectsElement.EnumerateArray())
{
if (affect.ValueKind != JsonValueKind.Object)
{
continue;
}
var reference = TryGetString(affect, "ref");
if (string.IsNullOrWhiteSpace(reference))
{
continue;
}
builder.Add(new CycloneDxAffect(reference.Trim()));
}
return builder.ToImmutable();
}
private static JsonElement TryGetProperty(JsonElement element, string propertyName)
=> element.ValueKind == JsonValueKind.Object && element.TryGetProperty(propertyName, out var value)
? value
: default;
private static string? TryGetString(JsonElement element, string propertyName)
{
if (element.ValueKind != JsonValueKind.Object)
{
return null;
}
if (!element.TryGetProperty(propertyName, out var value))
{
return null;
}
return value.ValueKind == JsonValueKind.String ? value.GetString() : null;
}
private static DateTimeOffset? ParseDate(JsonElement element, string propertyName)
{
var value = TryGetString(element, propertyName);
if (string.IsNullOrWhiteSpace(value))
{
return null;
}
return DateTimeOffset.TryParse(value, out var parsed) ? parsed : null;
}
}
private sealed record CycloneDxParseResult(
ImmutableDictionary<string, string> Metadata,
string? BomVersion,
DateTimeOffset FirstObserved,
DateTimeOffset LastObserved,
ImmutableDictionary<string, CycloneDxComponent> Components,
ImmutableArray<CycloneDxVulnerability> Vulnerabilities)
{
public CycloneDxProductInfo ResolveProduct(string? componentRef)
{
if (!string.IsNullOrWhiteSpace(componentRef) &&
Components.TryGetValue(componentRef.Trim(), out var component))
{
return new CycloneDxProductInfo(component.Reference, component.Name, component.Version, component.Purl, component.Cpe);
}
var key = string.IsNullOrWhiteSpace(componentRef) ? "unknown-component" : componentRef.Trim();
return new CycloneDxProductInfo(key, key, null, null, null);
}
}
private sealed record CycloneDxComponent(
string Reference,
string Name,
string? Version,
string? Purl,
string? Cpe);
private sealed record CycloneDxVulnerability(
string VulnerabilityId,
string? Detail,
string? AnalysisState,
string? AnalysisJustification,
ImmutableArray<string> AnalysisResponses,
ImmutableArray<CycloneDxAffect> Affects);
private sealed record CycloneDxAffect(string ComponentRef);
private sealed record CycloneDxProductInfo(
string Key,
string Name,
string? Version,
string? Purl,
string? Cpe);
}

14
src/StellaOps.Vexer.Formats.CycloneDX/ServiceCollectionExtensions.cs Normal file
View File

@@ -0,0 +1,14 @@
using Microsoft.Extensions.DependencyInjection;
using StellaOps.Vexer.Core;
namespace StellaOps.Vexer.Formats.CycloneDX;
public static class CycloneDxFormatsServiceCollectionExtensions
{
public static IServiceCollection AddCycloneDxNormalizer(this IServiceCollection services)
{
ArgumentNullException.ThrowIfNull(services);
services.AddSingleton<IVexNormalizer, CycloneDxNormalizer>();
return services;
}
}

16
src/StellaOps.Vexer.Formats.CycloneDX/StellaOps.Vexer.Formats.CycloneDX.csproj Normal file
View File

@@ -0,0 +1,16 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>net10.0</TargetFramework>
<LangVersion>preview</LangVersion>
<Nullable>enable</Nullable>
<ImplicitUsings>enable</ImplicitUsings>
<TreatWarningsAsErrors>true</TreatWarningsAsErrors>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Microsoft.Extensions.DependencyInjection.Abstractions" Version="8.0.0" />
<PackageReference Include="Microsoft.Extensions.Logging.Abstractions" Version="8.0.0" />
</ItemGroup>
<ItemGroup>
<ProjectReference Include="..\StellaOps.Vexer.Core\StellaOps.Vexer.Core.csproj" />
</ItemGroup>
</Project>

2
src/StellaOps.Vexer.Formats.CycloneDX/TASKS.md
View File

@@ -2,6 +2,6 @@ If you are working on this file you need to read docs/ARCHITECTURE_VEXER.md and
# TASKS # TASKS
| Task | Owner(s) | Depends on | Notes | | Task | Owner(s) | Depends on | Notes |
|---|---|---|---| |---|---|---|---|
|VEXER-FMT-CYCLONE-01-001 – CycloneDX VEX normalizer|Team Vexer Formats|VEXER-CORE-01-001|TODO – Parse CycloneDX `analysis` data into `VexClaim` entries with deterministic component identifiers.| |VEXER-FMT-CYCLONE-01-001 – CycloneDX VEX normalizer|Team Vexer Formats|VEXER-CORE-01-001|**DONE (2025-10-17)** – CycloneDX normalizer parses `analysis` data, resolves component references, and emits canonical `VexClaim`s; regression lives in `CycloneDxNormalizerTests`.|
|VEXER-FMT-CYCLONE-01-002 – Component reference reconciliation|Team Vexer Formats|VEXER-FMT-CYCLONE-01-001|TODO – Implement helpers to reconcile component/service references against policy expectations and emit diagnostics for missing SBOM links.| |VEXER-FMT-CYCLONE-01-002 – Component reference reconciliation|Team Vexer Formats|VEXER-FMT-CYCLONE-01-001|TODO – Implement helpers to reconcile component/service references against policy expectations and emit diagnostics for missing SBOM links.|
|VEXER-FMT-CYCLONE-01-003 – CycloneDX export serializer|Team Vexer Formats|VEXER-EXPORT-01-001, VEXER-FMT-CYCLONE-01-001|TODO – Provide exporters producing CycloneDX VEX output with canonical ordering and hash-stable manifests.| |VEXER-FMT-CYCLONE-01-003 – CycloneDX export serializer|Team Vexer Formats|VEXER-EXPORT-01-001, VEXER-FMT-CYCLONE-01-001|TODO – Provide exporters producing CycloneDX VEX output with canonical ordering and hash-stable manifests.|

87
src/StellaOps.Vexer.Formats.OpenVEX.Tests/OpenVexNormalizerTests.cs Normal file
View File

@@ -0,0 +1,87 @@
using System.Collections.Immutable;
using System.Linq;
using System.Text;
using FluentAssertions;
using Microsoft.Extensions.Logging.Abstractions;
using StellaOps.Vexer.Core;
using StellaOps.Vexer.Formats.OpenVEX;
namespace StellaOps.Vexer.Formats.OpenVEX.Tests;
public sealed class OpenVexNormalizerTests
{
[Fact]
public async Task NormalizeAsync_ProducesClaimsForStatements()
{
var json = """
{
"document": {
"author": "Acme Security",
"version": "1",
"issued": "2025-10-01T00:00:00Z",
"last_updated": "2025-10-05T00:00:00Z"
},
"statements": [
{
"id": "statement-1",
"vulnerability": "CVE-2025-2000",
"status": "not_affected",
"justification": "code_not_present",
"products": [
{
"id": "acme-widget@1.2.3",
"name": "Acme Widget",
"version": "1.2.3",
"purl": "pkg:acme/widget@1.2.3",
"cpe": "cpe:/a:acme:widget:1.2.3"
}
],
"statement": "The vulnerable code was never shipped."
},
{
"id": "statement-2",
"vulnerability": "CVE-2025-2001",
"status": "affected",
"products": [
"pkg:acme/widget@2.0.0"
],
"remediation": "Upgrade to 2.1.0"
}
]
}
""";
var rawDocument = new VexRawDocument(
"vexer:openvex",
VexDocumentFormat.OpenVex,
new Uri("https://example.com/openvex.json"),
new DateTimeOffset(2025, 10, 6, 0, 0, 0, TimeSpan.Zero),
"sha256:dummydigest",
Encoding.UTF8.GetBytes(json),
ImmutableDictionary<string, string>.Empty);
var provider = new VexProvider("vexer:openvex", "OpenVEX Provider", VexProviderKind.Vendor);
var normalizer = new OpenVexNormalizer(NullLogger<OpenVexNormalizer>.Instance);
var batch = await normalizer.NormalizeAsync(rawDocument, provider, CancellationToken.None);
batch.Claims.Should().HaveCount(2);
var notAffected = batch.Claims.Single(c => c.VulnerabilityId == "CVE-2025-2000");
notAffected.Status.Should().Be(VexClaimStatus.NotAffected);
notAffected.Justification.Should().Be(VexJustification.CodeNotPresent);
notAffected.Product.Key.Should().Be("acme-widget@1.2.3");
notAffected.Product.Purl.Should().Be("pkg:acme/widget@1.2.3");
notAffected.Document.Revision.Should().Be("1");
notAffected.AdditionalMetadata["openvex.document.author"].Should().Be("Acme Security");
notAffected.AdditionalMetadata["openvex.statement.status"].Should().Be("not_affected");
notAffected.Detail.Should().Be("The vulnerable code was never shipped.");
var affected = batch.Claims.Single(c => c.VulnerabilityId == "CVE-2025-2001");
affected.Status.Should().Be(VexClaimStatus.Affected);
affected.Justification.Should().BeNull();
affected.Product.Key.Should().Be("pkg:acme/widget@2.0.0");
affected.Product.Name.Should().Be("pkg:acme/widget@2.0.0");
affected.Detail.Should().Be("Upgrade to 2.1.0");
}
}

17
src/StellaOps.Vexer.Formats.OpenVEX.Tests/StellaOps.Vexer.Formats.OpenVEX.Tests.csproj Normal file
View File

@@ -0,0 +1,17 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>net10.0</TargetFramework>
<Nullable>enable</Nullable>
<ImplicitUsings>enable</ImplicitUsings>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="FluentAssertions" Version="6.12.0" />
</ItemGroup>
<ItemGroup>
<Using Include="Xunit" />
</ItemGroup>
<ItemGroup>
<ProjectReference Include="..\StellaOps.Vexer.Core\StellaOps.Vexer.Core.csproj" />
<ProjectReference Include="..\StellaOps.Vexer.Formats.OpenVEX\StellaOps.Vexer.Formats.OpenVEX.csproj" />
</ItemGroup>
</Project>

367
src/StellaOps.Vexer.Formats.OpenVEX/OpenVexNormalizer.cs Normal file
View File

@@ -0,0 +1,367 @@
using System;
using System.Collections.Generic;
using System.Collections.Immutable;
using System.Linq;
using System.Text.Json;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.Extensions.Logging;
using StellaOps.Vexer.Core;
namespace StellaOps.Vexer.Formats.OpenVEX;
public sealed class OpenVexNormalizer : IVexNormalizer
{
private static readonly ImmutableDictionary<string, VexClaimStatus> StatusMap = new Dictionary<string, VexClaimStatus>(StringComparer.OrdinalIgnoreCase)
{
["affected"] = VexClaimStatus.Affected,
["not_affected"] = VexClaimStatus.NotAffected,
["fixed"] = VexClaimStatus.Fixed,
["under_investigation"] = VexClaimStatus.UnderInvestigation,
}.ToImmutableDictionary(StringComparer.OrdinalIgnoreCase);
private static readonly ImmutableDictionary<string, VexJustification> JustificationMap = new Dictionary<string, VexJustification>(StringComparer.OrdinalIgnoreCase)
{
["component_not_present"] = VexJustification.ComponentNotPresent,
["component_not_configured"] = VexJustification.ComponentNotConfigured,
["vulnerable_code_not_present"] = VexJustification.VulnerableCodeNotPresent,
["vulnerable_code_not_in_execute_path"] = VexJustification.VulnerableCodeNotInExecutePath,
["vulnerable_code_cannot_be_controlled_by_adversary"] = VexJustification.VulnerableCodeCannotBeControlledByAdversary,
["inline_mitigations_already_exist"] = VexJustification.InlineMitigationsAlreadyExist,
["protected_by_mitigating_control"] = VexJustification.ProtectedByMitigatingControl,
["protected_by_compensating_control"] = VexJustification.ProtectedByCompensatingControl,
["code_not_present"] = VexJustification.CodeNotPresent,
["code_not_reachable"] = VexJustification.CodeNotReachable,
["requires_configuration"] = VexJustification.RequiresConfiguration,
["requires_dependency"] = VexJustification.RequiresDependency,
["requires_environment"] = VexJustification.RequiresEnvironment,
}.ToImmutableDictionary(StringComparer.OrdinalIgnoreCase);
private readonly ILogger<OpenVexNormalizer> _logger;
public OpenVexNormalizer(ILogger<OpenVexNormalizer> logger)
{
_logger = logger ?? throw new ArgumentNullException(nameof(logger));
}
public string Format => VexDocumentFormat.OpenVex.ToString().ToLowerInvariant();
public bool CanHandle(VexRawDocument document)
=> document is not null && document.Format == VexDocumentFormat.OpenVex;
public ValueTask<VexClaimBatch> NormalizeAsync(VexRawDocument document, VexProvider provider, CancellationToken cancellationToken)
{
ArgumentNullException.ThrowIfNull(document);
ArgumentNullException.ThrowIfNull(provider);
cancellationToken.ThrowIfCancellationRequested();
try
{
var result = OpenVexParser.Parse(document);
var claims = ImmutableArray.CreateBuilder<VexClaim>(result.Statements.Length);
foreach (var statement in result.Statements)
{
cancellationToken.ThrowIfCancellationRequested();
var status = MapStatus(statement.Status);
var justification = MapJustification(statement.Justification);
foreach (var product in statement.Products)
{
var vexProduct = new VexProduct(
product.Key,
product.Name,
product.Version,
product.Purl,
product.Cpe);
var metadata = result.Metadata;
metadata = metadata.SetItem("openvex.statement.id", statement.Id);
if (!string.IsNullOrWhiteSpace(statement.Status))
{
metadata = metadata.SetItem("openvex.statement.status", statement.Status!);
}
if (!string.IsNullOrWhiteSpace(statement.Justification))
{
metadata = metadata.SetItem("openvex.statement.justification", statement.Justification!);
}
if (!string.IsNullOrWhiteSpace(product.OriginalId))
{
metadata = metadata.SetItem("openvex.product.source", product.OriginalId!);
}
var claimDocument = new VexClaimDocument(
VexDocumentFormat.OpenVex,
document.Digest,
document.SourceUri,
result.DocumentVersion,
signature: null);
var claim = new VexClaim(
statement.Vulnerability,
provider.Id,
vexProduct,
status,
claimDocument,
result.FirstObserved,
result.LastObserved,
justification,
statement.Remarks,
confidence: null,
additionalMetadata: metadata);
claims.Add(claim);
}
}
var orderedClaims = claims
.ToImmutable()
.OrderBy(static claim => claim.VulnerabilityId, StringComparer.Ordinal)
.ThenBy(static claim => claim.Product.Key, StringComparer.Ordinal)
.ToImmutableArray();
_logger.LogInformation(
"Normalized OpenVEX document {Source} into {ClaimCount} claim(s).",
document.SourceUri,
orderedClaims.Length);
return ValueTask.FromResult(new VexClaimBatch(
document,
orderedClaims,
ImmutableDictionary<string, string>.Empty));
}
catch (JsonException ex)
{
_logger.LogError(ex, "Failed to parse OpenVEX document {SourceUri}", document.SourceUri);
throw;
}
}
private static VexClaimStatus MapStatus(string? status)
{
if (!string.IsNullOrWhiteSpace(status) && StatusMap.TryGetValue(status.Trim(), out var mapped))
{
return mapped;
}
return VexClaimStatus.UnderInvestigation;
}
private static VexJustification? MapJustification(string? justification)
{
if (string.IsNullOrWhiteSpace(justification))
{
return null;
}
return JustificationMap.TryGetValue(justification.Trim(), out var mapped)
? mapped
: null;
}
private static class OpenVexParser
{
public static OpenVexParseResult Parse(VexRawDocument document)
{
using var json = JsonDocument.Parse(document.Content.ToArray());
var root = json.RootElement;
var metadata = ImmutableDictionary.CreateBuilder<string, string>(StringComparer.Ordinal);
var documentElement = TryGetProperty(root, "document");
var version = TryGetString(documentElement, "version");
var author = TryGetString(documentElement, "author");
if (!string.IsNullOrWhiteSpace(version))
{
metadata["openvex.document.version"] = version!;
}
if (!string.IsNullOrWhiteSpace(author))
{
metadata["openvex.document.author"] = author!;
}
var issued = ParseDate(documentElement, "issued");
var lastUpdated = ParseDate(documentElement, "last_updated") ?? issued ?? document.RetrievedAt;
var effectiveDate = ParseDate(documentElement, "effective_date") ?? issued ?? document.RetrievedAt;
var statements = CollectStatements(root);
return new OpenVexParseResult(
metadata.ToImmutable(),
version,
effectiveDate,
lastUpdated,
statements);
}
private static ImmutableArray<OpenVexStatement> CollectStatements(JsonElement root)
{
if (!root.TryGetProperty("statements", out var statementsElement) ||
statementsElement.ValueKind != JsonValueKind.Array)
{
return ImmutableArray<OpenVexStatement>.Empty;
}
var builder = ImmutableArray.CreateBuilder<OpenVexStatement>();
foreach (var statement in statementsElement.EnumerateArray())
{
if (statement.ValueKind != JsonValueKind.Object)
{
continue;
}
var vulnerability = TryGetString(statement, "vulnerability") ?? TryGetString(statement, "vuln") ?? string.Empty;
if (string.IsNullOrWhiteSpace(vulnerability))
{
continue;
}
var id = TryGetString(statement, "id") ?? Guid.NewGuid().ToString();
var status = TryGetString(statement, "status");
var justification = TryGetString(statement, "justification");
var remarks = TryGetString(statement, "remediation") ?? TryGetString(statement, "statement");
var products = CollectProducts(statement);
if (products.Length == 0)
{
continue;
}
builder.Add(new OpenVexStatement(
id,
vulnerability.Trim(),
status,
justification,
remarks,
products));
}
return builder.ToImmutable();
}
private static ImmutableArray<OpenVexProduct> CollectProducts(JsonElement statement)
{
if (!statement.TryGetProperty("products", out var productsElement) ||
productsElement.ValueKind != JsonValueKind.Array)
{
return ImmutableArray<OpenVexProduct>.Empty;
}
var builder = ImmutableArray.CreateBuilder<OpenVexProduct>();
foreach (var product in productsElement.EnumerateArray())
{
if (product.ValueKind != JsonValueKind.String && product.ValueKind != JsonValueKind.Object)
{
continue;
}
if (product.ValueKind == JsonValueKind.String)
{
var value = product.GetString();
if (string.IsNullOrWhiteSpace(value))
{
continue;
}
builder.Add(OpenVexProduct.FromString(value.Trim()));
continue;
}
var id = TryGetString(product, "id") ?? TryGetString(product, "product_id");
var name = TryGetString(product, "name");
var version = TryGetString(product, "version");
var purl = TryGetString(product, "purl");
var cpe = TryGetString(product, "cpe");
if (string.IsNullOrWhiteSpace(id) && string.IsNullOrWhiteSpace(purl))
{
continue;
}
builder.Add(new OpenVexProduct(
id ?? purl!,
name ?? id ?? purl!,
version,
purl,
cpe,
OriginalId: id));
}
return builder.ToImmutable();
}
private static JsonElement TryGetProperty(JsonElement element, string propertyName)
=> element.ValueKind == JsonValueKind.Object && element.TryGetProperty(propertyName, out var value)
? value
: default;
private static string? TryGetString(JsonElement element, string propertyName)
{
if (element.ValueKind != JsonValueKind.Object)
{
return null;
}
if (!element.TryGetProperty(propertyName, out var value))
{
return null;
}
return value.ValueKind == JsonValueKind.String ? value.GetString() : null;
}
private static DateTimeOffset? ParseDate(JsonElement element, string propertyName)
{
var value = TryGetString(element, propertyName);
if (string.IsNullOrWhiteSpace(value))
{
return null;
}
return DateTimeOffset.TryParse(value, out var parsed) ? parsed : null;
}
}
private sealed record OpenVexParseResult(
ImmutableDictionary<string, string> Metadata,
string? DocumentVersion,
DateTimeOffset FirstObserved,
DateTimeOffset LastObserved,
ImmutableArray<OpenVexStatement> Statements);
private sealed record OpenVexStatement(
string Id,
string Vulnerability,
string? Status,
string? Justification,
string? Remarks,
ImmutableArray<OpenVexProduct> Products);
private sealed record OpenVexProduct(
string Key,
string Name,
string? Version,
string? Purl,
string? Cpe,
string? OriginalId)
{
public static OpenVexProduct FromString(string value)
{
var key = value;
string? purl = null;
string? name = value;
if (value.StartsWith("pkg:", StringComparison.OrdinalIgnoreCase))
{
purl = value;
}
return new OpenVexProduct(key, name, null, purl, null, OriginalId: value);
}
}
}

14
src/StellaOps.Vexer.Formats.OpenVEX/ServiceCollectionExtensions.cs Normal file
View File

@@ -0,0 +1,14 @@
using Microsoft.Extensions.DependencyInjection;
using StellaOps.Vexer.Core;
namespace StellaOps.Vexer.Formats.OpenVEX;
public static class OpenVexFormatsServiceCollectionExtensions
{
public static IServiceCollection AddOpenVexNormalizer(this IServiceCollection services)
{
ArgumentNullException.ThrowIfNull(services);
services.AddSingleton<IVexNormalizer, OpenVexNormalizer>();
return services;
}
}

16
src/StellaOps.Vexer.Formats.OpenVEX/StellaOps.Vexer.Formats.OpenVEX.csproj Normal file
View File

@@ -0,0 +1,16 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>net10.0</TargetFramework>
<LangVersion>preview</LangVersion>
<Nullable>enable</Nullable>
<ImplicitUsings>enable</ImplicitUsings>
<TreatWarningsAsErrors>true</TreatWarningsAsErrors>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Microsoft.Extensions.DependencyInjection.Abstractions" Version="8.0.0" />
<PackageReference Include="Microsoft.Extensions.Logging.Abstractions" Version="8.0.0" />
</ItemGroup>
<ItemGroup>
<ProjectReference Include="..\StellaOps.Vexer.Core\StellaOps.Vexer.Core.csproj" />
</ItemGroup>
</Project>

2
src/StellaOps.Vexer.Formats.OpenVEX/TASKS.md
View File

@@ -2,6 +2,6 @@ If you are working on this file you need to read docs/ARCHITECTURE_VEXER.md and
# TASKS # TASKS
| Task | Owner(s) | Depends on | Notes | | Task | Owner(s) | Depends on | Notes |
|---|---|---|---| |---|---|---|---|
|VEXER-FMT-OPENVEX-01-001 – OpenVEX normalizer|Team Vexer Formats|VEXER-CORE-01-001|TODO – Implement OpenVEX parser covering statements, products, and `status/justification` mapping with provenance metadata.| |VEXER-FMT-OPENVEX-01-001 – OpenVEX normalizer|Team Vexer Formats|VEXER-CORE-01-001|**DONE (2025-10-17)** – OpenVEX normalizer parses statements/products, maps status/justification, and surfaces provenance metadata; coverage in `OpenVexNormalizerTests`.|
|VEXER-FMT-OPENVEX-01-002 – Statement merge utilities|Team Vexer Formats|VEXER-FMT-OPENVEX-01-001|TODO – Add reducers merging multiple OpenVEX statements, resolving conflicts deterministically, and emitting policy diagnostics.| |VEXER-FMT-OPENVEX-01-002 – Statement merge utilities|Team Vexer Formats|VEXER-FMT-OPENVEX-01-001|TODO – Add reducers merging multiple OpenVEX statements, resolving conflicts deterministically, and emitting policy diagnostics.|
|VEXER-FMT-OPENVEX-01-003 – OpenVEX export writer|Team Vexer Formats|VEXER-EXPORT-01-001, VEXER-FMT-OPENVEX-01-001|TODO – Provide export serializer generating canonical OpenVEX documents with optional SBOM references and hash-stable ordering.| |VEXER-FMT-OPENVEX-01-003 – OpenVEX export writer|Team Vexer Formats|VEXER-EXPORT-01-001, VEXER-FMT-OPENVEX-01-001|TODO – Provide export serializer generating canonical OpenVEX documents with optional SBOM references and hash-stable ordering.|

13
src/StellaOps.Vexer.Storage.Mongo/IVexStorageContracts.cs
View File

@@ -1,5 +1,6 @@
using System; using System;
using System.Collections.Generic; using System.Collections.Generic;
using System.Collections.Immutable;
using System.Threading; using System.Threading;
using System.Threading.Tasks; using System.Threading.Tasks;
using StellaOps.Vexer.Core; using StellaOps.Vexer.Core;
@@ -24,6 +25,18 @@ public interface IVexConsensusStore
ValueTask SaveAsync(VexConsensus consensus, CancellationToken cancellationToken); ValueTask SaveAsync(VexConsensus consensus, CancellationToken cancellationToken);
} }
public sealed record VexConnectorState(
string ConnectorId,
DateTimeOffset? LastUpdated,
ImmutableArray<string> DocumentDigests);
public interface IVexConnectorStateRepository
{
ValueTask<VexConnectorState?> GetAsync(string connectorId, CancellationToken cancellationToken);
ValueTask SaveAsync(VexConnectorState state, CancellationToken cancellationToken);
}
public interface IVexCacheIndex public interface IVexCacheIndex
{ {
ValueTask<VexCacheEntry?> FindAsync(VexQuerySignature signature, VexExportFormat format, CancellationToken cancellationToken); ValueTask<VexCacheEntry?> FindAsync(VexQuerySignature signature, VexExportFormat format, CancellationToken cancellationToken);

55
src/StellaOps.Vexer.Storage.Mongo/MongoVexConnectorStateRepository.cs Normal file
View File

@@ -0,0 +1,55 @@
using System;
using System.Collections.Immutable;
using System.Linq;
using System.Threading;
using System.Threading.Tasks;
using MongoDB.Driver;
namespace StellaOps.Vexer.Storage.Mongo;
public sealed class MongoVexConnectorStateRepository : IVexConnectorStateRepository
{
private readonly IMongoCollection<VexConnectorStateDocument> _collection;
public MongoVexConnectorStateRepository(IMongoDatabase database)
{
ArgumentNullException.ThrowIfNull(database);
VexMongoMappingRegistry.Register();
_collection = database.GetCollection<VexConnectorStateDocument>(VexMongoCollectionNames.ConnectorState);
}
public async ValueTask<VexConnectorState?> GetAsync(string connectorId, CancellationToken cancellationToken)
{
ArgumentException.ThrowIfNullOrWhiteSpace(connectorId);
var filter = Builders<VexConnectorStateDocument>.Filter.Eq(x => x.ConnectorId, connectorId.Trim());
var document = await _collection.Find(filter).FirstOrDefaultAsync(cancellationToken).ConfigureAwait(false);
return document?.ToRecord();
}
public async ValueTask SaveAsync(VexConnectorState state, CancellationToken cancellationToken)
{
ArgumentNullException.ThrowIfNull(state);
var document = VexConnectorStateDocument.FromRecord(state.WithNormalizedDigests());
var filter = Builders<VexConnectorStateDocument>.Filter.Eq(x => x.ConnectorId, document.ConnectorId);
await _collection.ReplaceOneAsync(filter, document, new ReplaceOptions { IsUpsert = true }, cancellationToken).ConfigureAwait(false);
}
}
internal static class VexConnectorStateExtensions
{
private const int MaxDigestHistory = 200;
public static VexConnectorState WithNormalizedDigests(this VexConnectorState state)
{
var digests = state.DocumentDigests;
if (digests.Length <= MaxDigestHistory)
{
return state;
}
var trimmed = digests.Skip(digests.Length - MaxDigestHistory).ToImmutableArray();
return state with { DocumentDigests = trimmed };
}
}

1
src/StellaOps.Vexer.Storage.Mongo/ServiceCollectionExtensions.cs
View File

@@ -16,6 +16,7 @@ public static class VexMongoServiceCollectionExtensions
services.AddSingleton<IVexConsensusStore, MongoVexConsensusStore>(); services.AddSingleton<IVexConsensusStore, MongoVexConsensusStore>();
services.AddSingleton<IVexCacheIndex, MongoVexCacheIndex>(); services.AddSingleton<IVexCacheIndex, MongoVexCacheIndex>();
services.AddSingleton<IVexCacheMaintenance, MongoVexCacheMaintenance>(); services.AddSingleton<IVexCacheMaintenance, MongoVexCacheMaintenance>();
services.AddSingleton<IVexConnectorStateRepository, MongoVexConnectorStateRepository>();
services.AddSingleton<IVexMongoMigration, VexInitialIndexMigration>(); services.AddSingleton<IVexMongoMigration, VexInitialIndexMigration>();
services.AddSingleton<VexMongoMigrationRunner>(); services.AddSingleton<VexMongoMigrationRunner>();
services.AddHostedService<VexMongoMigrationHostedService>(); services.AddHostedService<VexMongoMigrationHostedService>();

2
src/StellaOps.Vexer.Storage.Mongo/VexMongoMappingRegistry.cs
View File

@@ -39,6 +39,7 @@ public static class VexMongoMappingRegistry
RegisterClassMap<VexConsensusConflictDocument>(); RegisterClassMap<VexConsensusConflictDocument>();
RegisterClassMap<VexConfidenceDocument>(); RegisterClassMap<VexConfidenceDocument>();
RegisterClassMap<VexCacheEntryRecord>(); RegisterClassMap<VexCacheEntryRecord>();
RegisterClassMap<VexConnectorStateDocument>();
} }
private static void RegisterClassMap<TDocument>() private static void RegisterClassMap<TDocument>()
@@ -66,4 +67,5 @@ public static class VexMongoCollectionNames
public const string Consensus = "vex.consensus"; public const string Consensus = "vex.consensus";
public const string Exports = "vex.exports"; public const string Exports = "vex.exports";
public const string Cache = "vex.cache"; public const string Cache = "vex.cache";
public const string ConnectorState = "vex.connector_state";
} }

32
src/StellaOps.Vexer.Storage.Mongo/VexMongoModels.cs
View File

@@ -570,3 +570,35 @@ internal sealed class VexCacheEntryRecord
expires); expires);
} }
} }
[BsonIgnoreExtraElements]
internal sealed class VexConnectorStateDocument
{
[BsonId]
public string ConnectorId { get; set; } = default!;
public DateTime? LastUpdated { get; set; }
= null;
public List<string> DocumentDigests { get; set; } = new();
public static VexConnectorStateDocument FromRecord(VexConnectorState state)
=> new()
{
ConnectorId = state.ConnectorId,
LastUpdated = state.LastUpdated?.UtcDateTime,
DocumentDigests = state.DocumentDigests.ToList(),
};
public VexConnectorState ToRecord()
{
var lastUpdated = LastUpdated.HasValue
? new DateTimeOffset(DateTime.SpecifyKind(LastUpdated.Value, DateTimeKind.Utc))
: (DateTimeOffset?)null;
return new VexConnectorState(
ConnectorId,
lastUpdated,
DocumentDigests.ToImmutableArray());
}
}

2
src/StellaOps.Vexer.WebService/Program.cs
View File

@@ -7,6 +7,7 @@ using StellaOps.Vexer.Attestation.Transparency;
using StellaOps.Vexer.ArtifactStores.S3.Extensions; using StellaOps.Vexer.ArtifactStores.S3.Extensions;
using StellaOps.Vexer.Export; using StellaOps.Vexer.Export;
using StellaOps.Vexer.Storage.Mongo; using StellaOps.Vexer.Storage.Mongo;
using StellaOps.Vexer.Connectors.RedHat.CSAF.DependencyInjection;
var builder = WebApplication.CreateBuilder(args); var builder = WebApplication.CreateBuilder(args);
var configuration = builder.Configuration; var configuration = builder.Configuration;
@@ -21,6 +22,7 @@ services.AddVexExportEngine();
services.AddVexExportCacheServices(); services.AddVexExportCacheServices();
services.AddVexAttestation(); services.AddVexAttestation();
services.Configure<VexAttestationClientOptions>(configuration.GetSection("Vexer:Attestation:Client")); services.Configure<VexAttestationClientOptions>(configuration.GetSection("Vexer:Attestation:Client"));
services.AddRedHatCsafConnector();
var rekorSection = configuration.GetSection("Vexer:Attestation:Rekor"); var rekorSection = configuration.GetSection("Vexer:Attestation:Rekor");
if (rekorSection.Exists()) if (rekorSection.Exists())

2
src/StellaOps.Vexer.WebService/TASKS.md
View File

@@ -2,7 +2,7 @@ If you are working on this file you need to read docs/ARCHITECTURE_VEXER.md and
# TASKS # TASKS
| Task | Owner(s) | Depends on | Notes | | Task | Owner(s) | Depends on | Notes |
|---|---|---|---| |---|---|---|---|
|VEXER-WEB-01-001 – Minimal API bootstrap & DI|Team Vexer WebService|VEXER-CORE-01-003, VEXER-STORAGE-01-003|TODO – Scaffold ASP.NET host, register connectors/normalizers via plugin loader, bind policy/storage/attestation services, and expose `/vexer/status`.| |VEXER-WEB-01-001 – Minimal API bootstrap & DI|Team Vexer WebService|VEXER-CORE-01-003, VEXER-STORAGE-01-003|**DONE (2025-10-17)** – Minimal API host composes storage/export/attestation/artifact stores, binds Mongo/attestation options, and exposes `/vexer/status` + health endpoints with regression coverage in `StatusEndpointTests`.|
|VEXER-WEB-01-002 – Ingest & reconcile endpoints|Team Vexer WebService|VEXER-WEB-01-001|TODO – Implement `/vexer/init`, `/vexer/ingest/run`, `/vexer/ingest/resume`, `/vexer/reconcile` with token scope enforcement and structured run telemetry.| |VEXER-WEB-01-002 – Ingest & reconcile endpoints|Team Vexer WebService|VEXER-WEB-01-001|TODO – Implement `/vexer/init`, `/vexer/ingest/run`, `/vexer/ingest/resume`, `/vexer/reconcile` with token scope enforcement and structured run telemetry.|
|VEXER-WEB-01-003 – Export & verify endpoints|Team Vexer WebService|VEXER-WEB-01-001, VEXER-EXPORT-01-001, VEXER-ATTEST-01-001|TODO – Add `/vexer/export`, `/vexer/export/{id}`, `/vexer/export/{id}/download`, `/vexer/verify`, returning artifact + attestation metadata with cache awareness.| |VEXER-WEB-01-003 – Export & verify endpoints|Team Vexer WebService|VEXER-WEB-01-001, VEXER-EXPORT-01-001, VEXER-ATTEST-01-001|TODO – Add `/vexer/export`, `/vexer/export/{id}`, `/vexer/export/{id}/download`, `/vexer/verify`, returning artifact + attestation metadata with cache awareness.|
|VEXER-WEB-01-004 – Resolve API & signed responses|Team Vexer WebService|VEXER-WEB-01-001, VEXER-ATTEST-01-002|TODO – Deliver `/vexer/resolve` (subject/context), return consensus + score envelopes, attach cosign/Rekor metadata, and document auth + rate guardrails.| |VEXER-WEB-01-004 – Resolve API & signed responses|Team Vexer WebService|VEXER-WEB-01-001, VEXER-ATTEST-01-002|TODO – Deliver `/vexer/resolve` (subject/context), return consensus + score envelopes, attach cosign/Rekor metadata, and document auth + rate guardrails.|

Some files were not shown because too many files have changed in this diff Show More