up
This commit is contained in:
		
							
								
								
									
										2
									
								
								.gitignore
									
									
									
									
										vendored
									
									
								
							
							
						
						
									
										2
									
								
								.gitignore
									
									
									
									
										vendored
									
									
								
							| @@ -23,3 +23,5 @@ TestResults/ | |||||||
| seed-data/ics-cisa/*.csv | seed-data/ics-cisa/*.csv | ||||||
| seed-data/ics-cisa/*.xlsx | seed-data/ics-cisa/*.xlsx | ||||||
| seed-data/ics-cisa/*.sha256 | seed-data/ics-cisa/*.sha256 | ||||||
|  | seed-data/cert-bund/**/*.json | ||||||
|  | seed-data/cert-bund/**/*.sha256 | ||||||
|   | |||||||
							
								
								
									
										42
									
								
								SPRINTS.md
									
									
									
									
									
								
							
							
						
						
									
										42
									
								
								SPRINTS.md
									
									
									
									
									
								
							| @@ -105,8 +105,8 @@ | |||||||
| | Sprint 4 | Schema Parity & Freshness Alignment | src/StellaOps.Feedser.Merge/TASKS.md | DONE (2025-10-15) | Team Merge & QA Enforcement | FEEDMERGE-ENGINE-04-005 | Connector coordination for new advisory fields<br>GHSA/NVD/OSV connectors now ship description, CWE, and canonical metric data with refreshed fixtures; merge coordination log updated and exporters notified. | | | Sprint 4 | Schema Parity & Freshness Alignment | src/StellaOps.Feedser.Merge/TASKS.md | DONE (2025-10-15) | Team Merge & QA Enforcement | FEEDMERGE-ENGINE-04-005 | Connector coordination for new advisory fields<br>GHSA/NVD/OSV connectors now ship description, CWE, and canonical metric data with refreshed fixtures; merge coordination log updated and exporters notified. | | ||||||
| | Sprint 4 | Schema Parity & Freshness Alignment | src/StellaOps.Feedser.Exporter.Json/TASKS.md | DONE (2025-10-15) | Team Exporters – JSON | FEEDEXPORT-JSON-04-001 | Surface new advisory fields in JSON exporter<br>Update schemas/offline bundle + fixtures once model/core parity lands.<br>2025-10-15: `dotnet test src/StellaOps.Feedser.Exporter.Json.Tests` validated canonical metric/CWE emission. | | | Sprint 4 | Schema Parity & Freshness Alignment | src/StellaOps.Feedser.Exporter.Json/TASKS.md | DONE (2025-10-15) | Team Exporters – JSON | FEEDEXPORT-JSON-04-001 | Surface new advisory fields in JSON exporter<br>Update schemas/offline bundle + fixtures once model/core parity lands.<br>2025-10-15: `dotnet test src/StellaOps.Feedser.Exporter.Json.Tests` validated canonical metric/CWE emission. | | ||||||
| | Sprint 4 | Schema Parity & Freshness Alignment | src/StellaOps.Feedser.Exporter.TrivyDb/TASKS.md | DONE (2025-10-15) | Team Exporters – Trivy DB | FEEDEXPORT-TRIVY-04-001 | Propagate new advisory fields into Trivy DB package<br>Extend Bolt builder, metadata, and regression tests for the expanded schema.<br>2025-10-15: `dotnet test src/StellaOps.Feedser.Exporter.TrivyDb.Tests` confirmed canonical metric/CWE propagation. | | | Sprint 4 | Schema Parity & Freshness Alignment | src/StellaOps.Feedser.Exporter.TrivyDb/TASKS.md | DONE (2025-10-15) | Team Exporters – Trivy DB | FEEDEXPORT-TRIVY-04-001 | Propagate new advisory fields into Trivy DB package<br>Extend Bolt builder, metadata, and regression tests for the expanded schema.<br>2025-10-15: `dotnet test src/StellaOps.Feedser.Exporter.TrivyDb.Tests` confirmed canonical metric/CWE propagation. | | ||||||
| | Sprint 4 | Schema Parity & Freshness Alignment | src/StellaOps.Feedser.Source.Ghsa/TASKS.md | TODO | Team Connector Regression Fixtures | FEEDCONN-GHSA-04-004 | Harden CVSS fallback so canonical metric ids persist when GitHub omits vectors; extend fixtures and document severity precedence hand-off to Merge. | | | Sprint 4 | Schema Parity & Freshness Alignment | src/StellaOps.Feedser.Source.Ghsa/TASKS.md | DONE (2025-10-16) | Team Connector Regression Fixtures | FEEDCONN-GHSA-04-004 | Harden CVSS fallback so canonical metric ids persist when GitHub omits vectors; extend fixtures and document severity precedence hand-off to Merge. | | ||||||
| | Sprint 4 | Schema Parity & Freshness Alignment | src/StellaOps.Feedser.Source.Osv/TASKS.md | TODO | Team Connector Expansion – GHSA/NVD/OSV | FEEDCONN-OSV-04-005 | Map OSV advisories lacking CVSS vectors to canonical metric ids/notes and document CWE provenance quirks; schedule parity fixture updates. | | | Sprint 4 | Schema Parity & Freshness Alignment | src/StellaOps.Feedser.Source.Osv/TASKS.md | DONE (2025-10-16) | Team Connector Expansion – GHSA/NVD/OSV | FEEDCONN-OSV-04-005 | Map OSV advisories lacking CVSS vectors to canonical metric ids/notes and document CWE provenance quirks; schedule parity fixture updates. | | ||||||
| | Sprint 5 | Vexer Core Foundations | src/StellaOps.Vexer.Core/TASKS.md | DONE (2025-10-15) | Team Vexer Core & Policy | VEXER-CORE-01-001 | Stand up canonical VEX claim/consensus records with deterministic serializers so Storage/Exports share a stable contract. | | | Sprint 5 | Vexer Core Foundations | src/StellaOps.Vexer.Core/TASKS.md | DONE (2025-10-15) | Team Vexer Core & Policy | VEXER-CORE-01-001 | Stand up canonical VEX claim/consensus records with deterministic serializers so Storage/Exports share a stable contract. | | ||||||
| | Sprint 5 | Vexer Core Foundations | src/StellaOps.Vexer.Core/TASKS.md | DONE (2025-10-15) | Team Vexer Core & Policy | VEXER-CORE-01-002 | Implement trust-weighted consensus resolver with baseline policy weights, justification gates, telemetry output, and majority/tie handling. | | | Sprint 5 | Vexer Core Foundations | src/StellaOps.Vexer.Core/TASKS.md | DONE (2025-10-15) | Team Vexer Core & Policy | VEXER-CORE-01-002 | Implement trust-weighted consensus resolver with baseline policy weights, justification gates, telemetry output, and majority/tie handling. | | ||||||
| | Sprint 5 | Vexer Core Foundations | src/StellaOps.Vexer.Core/TASKS.md | DONE (2025-10-15) | Team Vexer Core & Policy | VEXER-CORE-01-003 | Publish shared connector/exporter/attestation abstractions and deterministic query signature utilities for cache/attestation workflows. | | | Sprint 5 | Vexer Core Foundations | src/StellaOps.Vexer.Core/TASKS.md | DONE (2025-10-15) | Team Vexer Core & Policy | VEXER-CORE-01-003 | Publish shared connector/exporter/attestation abstractions and deterministic query signature utilities for cache/attestation workflows. | | ||||||
| @@ -116,29 +116,35 @@ | |||||||
| | Sprint 5 | Vexer Core Foundations | src/StellaOps.Vexer.Policy/TASKS.md | DONE (2025-10-16) | Team Vexer Policy | VEXER-POLICY-01-004 | Implement YAML/JSON schema validation and deterministic diagnostics for operator bundles. | | | Sprint 5 | Vexer Core Foundations | src/StellaOps.Vexer.Policy/TASKS.md | DONE (2025-10-16) | Team Vexer Policy | VEXER-POLICY-01-004 | Implement YAML/JSON schema validation and deterministic diagnostics for operator bundles. | | ||||||
| | Sprint 5 | Vexer Core Foundations | src/StellaOps.Vexer.Policy/TASKS.md | DONE (2025-10-16) | Team Vexer Policy | VEXER-POLICY-01-005 | Add policy change tracking, snapshot digests, and telemetry/logging hooks. | | | Sprint 5 | Vexer Core Foundations | src/StellaOps.Vexer.Policy/TASKS.md | DONE (2025-10-16) | Team Vexer Policy | VEXER-POLICY-01-005 | Add policy change tracking, snapshot digests, and telemetry/logging hooks. | | ||||||
| | Sprint 5 | Vexer Core Foundations | src/StellaOps.Vexer.Storage.Mongo/TASKS.md | DONE (2025-10-15) | Team Vexer Storage | VEXER-STORAGE-01-001 | Mongo mapping registry plus raw/export entities and DI extensions in place. | | | Sprint 5 | Vexer Core Foundations | src/StellaOps.Vexer.Storage.Mongo/TASKS.md | DONE (2025-10-15) | Team Vexer Storage | VEXER-STORAGE-01-001 | Mongo mapping registry plus raw/export entities and DI extensions in place. | | ||||||
| | Sprint 5 | Vexer Core Foundations | src/StellaOps.Vexer.Storage.Mongo/TASKS.md | TODO | Team Vexer Storage | VEXER-STORAGE-01-004 | Build provider/consensus/cache class maps and related collections. | | | Sprint 5 | Vexer Core Foundations | src/StellaOps.Vexer.Storage.Mongo/TASKS.md | DONE (2025-10-16) | Team Vexer Storage | VEXER-STORAGE-01-004 | Build provider/consensus/cache class maps and related collections. | | ||||||
| | Sprint 5 | Vexer Core Foundations | src/StellaOps.Vexer.Export/TASKS.md | DONE (2025-10-15) | Team Vexer Export | VEXER-EXPORT-01-001 | Export engine delivers cache lookup, manifest creation, and policy integration. | | | Sprint 5 | Vexer Core Foundations | src/StellaOps.Vexer.Export/TASKS.md | DONE (2025-10-15) | Team Vexer Export | VEXER-EXPORT-01-001 | Export engine delivers cache lookup, manifest creation, and policy integration. | | ||||||
| | Sprint 5 | Vexer Core Foundations | src/StellaOps.Vexer.Export/TASKS.md | TODO | Team Vexer Export | VEXER-EXPORT-01-004 | Connect export engine to attestation client and persist Rekor metadata. | | | Sprint 5 | Vexer Core Foundations | src/StellaOps.Vexer.Export/TASKS.md | DONE (2025-10-17) | Team Vexer Export | VEXER-EXPORT-01-004 | Connect export engine to attestation client and persist Rekor metadata. | | ||||||
| | Sprint 5 | Vexer Core Foundations | src/StellaOps.Vexer.Attestation/TASKS.md | TODO | Team Vexer Attestation | VEXER-ATTEST-01-001 | Implement in-toto predicate + DSSE builder providing envelopes for export attestation. | | | Sprint 5 | Vexer Core Foundations | src/StellaOps.Vexer.Attestation/TASKS.md | DONE (2025-10-16) | Team Vexer Attestation | VEXER-ATTEST-01-001 | Implement in-toto predicate + DSSE builder providing envelopes for export attestation. | | ||||||
| | Sprint 5 | Vexer Core Foundations | src/StellaOps.Vexer.Connectors.Abstractions/TASKS.md | TODO | Team Vexer Connectors | VEXER-CONN-ABS-01-001 | Deliver shared connector context/base classes so provider plug-ins can be activated via WebService/Worker. | | | Sprint 5 | Vexer Core Foundations | src/StellaOps.Vexer.Connectors.Abstractions/TASKS.md | DONE (2025-10-17) | Team Vexer Connectors | VEXER-CONN-ABS-01-001 | Deliver shared connector context/base classes so provider plug-ins can be activated via WebService/Worker. | | ||||||
| | Sprint 5 | Vexer Core Foundations | src/StellaOps.Vexer.WebService/TASKS.md | TODO | Team Vexer WebService | VEXER-WEB-01-001 | Scaffold minimal API host, DI, and `/vexer/status` endpoint integrating policy, storage, export, and attestation services. | | | Sprint 5 | Vexer Core Foundations | src/StellaOps.Vexer.WebService/TASKS.md | DONE (2025-10-17) | Team Vexer WebService | VEXER-WEB-01-001 | Scaffold minimal API host, DI, and `/vexer/status` endpoint integrating policy, storage, export, and attestation services. | | ||||||
| | Sprint 6 | Vexer Ingest & Formats | src/StellaOps.Vexer.Worker/TASKS.md | TODO | Team Vexer Worker | VEXER-WORKER-01-001 | Create Worker host with provider scheduling and logging to drive recurring pulls/reconciliation. | | | Sprint 6 | Vexer Ingest & Formats | src/StellaOps.Vexer.Worker/TASKS.md | DONE (2025-10-17) | Team Vexer Worker | VEXER-WORKER-01-001 | Create Worker host with provider scheduling and logging to drive recurring pulls/reconciliation. | | ||||||
| | Sprint 6 | Vexer Ingest & Formats | src/StellaOps.Vexer.Formats.CSAF/TASKS.md | TODO | Team Vexer Formats | VEXER-FMT-CSAF-01-001 | Implement CSAF normalizer foundation translating provider documents into `VexClaim` entries. | | | Sprint 6 | Vexer Ingest & Formats | src/StellaOps.Vexer.Formats.CSAF/TASKS.md | DONE (2025-10-17) | Team Vexer Formats | VEXER-FMT-CSAF-01-001 | Implement CSAF normalizer foundation translating provider documents into `VexClaim` entries. | | ||||||
| | Sprint 6 | Vexer Ingest & Formats | src/StellaOps.Vexer.Formats.CycloneDX/TASKS.md | TODO | Team Vexer Formats | VEXER-FMT-CYCLONE-01-001 | Implement CycloneDX VEX normalizer capturing `analysis` state and component references. | | | Sprint 6 | Vexer Ingest & Formats | src/StellaOps.Vexer.Formats.CycloneDX/TASKS.md | DONE (2025-10-17) | Team Vexer Formats | VEXER-FMT-CYCLONE-01-001 | Implement CycloneDX VEX normalizer capturing `analysis` state and component references. | | ||||||
| | Sprint 6 | Vexer Ingest & Formats | src/StellaOps.Vexer.Formats.OpenVEX/TASKS.md | TODO | Team Vexer Formats | VEXER-FMT-OPENVEX-01-001 | Implement OpenVEX normalizer to ingest attestations into canonical claims with provenance. | | | Sprint 6 | Vexer Ingest & Formats | src/StellaOps.Vexer.Formats.OpenVEX/TASKS.md | DONE (2025-10-17) | Team Vexer Formats | VEXER-FMT-OPENVEX-01-001 | Implement OpenVEX normalizer to ingest attestations into canonical claims with provenance. | | ||||||
| | Sprint 6 | Vexer Ingest & Formats | src/StellaOps.Vexer.Connectors.RedHat.CSAF/TASKS.md | TODO | Team Vexer Connectors – Red Hat | VEXER-CONN-RH-01-001 | Ship Red Hat CSAF provider metadata discovery enabling incremental pulls. | | | Sprint 6 | Vexer Ingest & Formats | src/StellaOps.Vexer.Connectors.RedHat.CSAF/TASKS.md | DONE (2025-10-17) | Team Vexer Connectors – Red Hat | VEXER-CONN-RH-01-001 | Ship Red Hat CSAF provider metadata discovery enabling incremental pulls. | | ||||||
| | Sprint 6 | Vexer Ingest & Formats | src/StellaOps.Vexer.Connectors.Cisco.CSAF/TASKS.md | TODO | Team Vexer Connectors – Cisco | VEXER-CONN-CISCO-01-001 | Implement Cisco CSAF endpoint discovery/auth to unlock paginated pulls. | | | Sprint 6 | Vexer Ingest & Formats | src/StellaOps.Vexer.Connectors.RedHat.CSAF/TASKS.md | DONE (2025-10-17) | Team Vexer Connectors – Red Hat | VEXER-CONN-RH-01-002 | Fetch CSAF windows with ETag handling, resume tokens, quarantine on schema errors, and persist raw docs. | | ||||||
| | Sprint 6 | Vexer Ingest & Formats | src/StellaOps.Vexer.Connectors.SUSE.RancherVEXHub/TASKS.md | TODO | Team Vexer Connectors – SUSE | VEXER-CONN-SUSE-01-001 | Build Rancher VEX Hub discovery/subscription path with offline snapshot support. | | | Sprint 6 | Vexer Ingest & Formats | src/StellaOps.Vexer.Connectors.RedHat.CSAF/TASKS.md | DONE (2025-10-17) | Team Vexer Connectors – Red Hat | VEXER-CONN-RH-01-003 | Populate provider trust overrides (cosign issuer, identity regex) and provenance hints for policy evaluation/logging. | | ||||||
| | Sprint 6 | Vexer Ingest & Formats | src/StellaOps.Vexer.Connectors.MSRC.CSAF/TASKS.md | TODO | Team Vexer Connectors – MSRC | VEXER-CONN-MS-01-001 | Deliver AAD onboarding/token cache for MSRC CSAF ingestion. | | | Sprint 6 | Vexer Ingest & Formats | src/StellaOps.Vexer.Connectors.RedHat.CSAF/TASKS.md | DONE (2025-10-17) | Team Vexer Connectors – Red Hat | VEXER-CONN-RH-01-004 | Persist resume cursors (last updated timestamp/document hashes) in storage and reload during fetch to avoid duplicates. | | ||||||
| | Sprint 6 | Vexer Ingest & Formats | src/StellaOps.Vexer.Connectors.Oracle.CSAF/TASKS.md | TODO | Team Vexer Connectors – Oracle | VEXER-CONN-ORACLE-01-001 | Implement Oracle CSAF catalogue discovery with CPU calendar awareness. | | | Sprint 6 | Vexer Ingest & Formats | src/StellaOps.Vexer.Connectors.RedHat.CSAF/TASKS.md | DONE (2025-10-17) | Team Vexer Connectors – Red Hat | VEXER-CONN-RH-01-005 | Register connector in Worker/WebService DI, add scheduled jobs, and document CLI triggers for Red Hat CSAF pulls. | | ||||||
| | Sprint 6 | Vexer Ingest & Formats | src/StellaOps.Vexer.Connectors.Ubuntu.CSAF/TASKS.md | TODO | Team Vexer Connectors – Ubuntu | VEXER-CONN-UBUNTU-01-001 | Implement Ubuntu CSAF discovery and channel selection for USN ingestion. | | | Sprint 6 | Vexer Ingest & Formats | src/StellaOps.Vexer.Connectors.RedHat.CSAF/TASKS.md | DONE (2025-10-17) | Team Vexer Connectors – Red Hat | VEXER-CONN-RH-01-006 | Add CSAF normalization parity fixtures ensuring RHSA-specific metadata is preserved. | | ||||||
|  | | Sprint 6 | Vexer Ingest & Formats | src/StellaOps.Vexer.Connectors.Cisco.CSAF/TASKS.md | DONE (2025-10-17) | Team Vexer Connectors – Cisco | VEXER-CONN-CISCO-01-001 | Implement Cisco CSAF endpoint discovery/auth to unlock paginated pulls. | | ||||||
|  | | Sprint 6 | Vexer Ingest & Formats | src/StellaOps.Vexer.Connectors.Cisco.CSAF/TASKS.md | DONE (2025-10-17) | Team Vexer Connectors – Cisco | VEXER-CONN-CISCO-01-002 | Implement Cisco CSAF paginated fetch loop with dedupe and raw persistence support. | | ||||||
|  | | Sprint 6 | Vexer Ingest & Formats | src/StellaOps.Vexer.Connectors.SUSE.RancherVEXHub/TASKS.md | DONE (2025-10-17) | Team Vexer Connectors – SUSE | VEXER-CONN-SUSE-01-001 | Build Rancher VEX Hub discovery/subscription path with offline snapshot support. | | ||||||
|  | | Sprint 6 | Vexer Ingest & Formats | src/StellaOps.Vexer.Connectors.MSRC.CSAF/TASKS.md | DONE (2025-10-17) | Team Vexer Connectors – MSRC | VEXER-CONN-MS-01-001 | Deliver AAD onboarding/token cache for MSRC CSAF ingestion. | | ||||||
|  | | Sprint 6 | Vexer Ingest & Formats | src/StellaOps.Vexer.Connectors.Oracle.CSAF/TASKS.md | DONE (2025-10-17) | Team Vexer Connectors – Oracle | VEXER-CONN-ORACLE-01-001 | Implement Oracle CSAF catalogue discovery with CPU calendar awareness. | | ||||||
|  | | Sprint 6 | Vexer Ingest & Formats | src/StellaOps.Vexer.Connectors.Ubuntu.CSAF/TASKS.md | DONE (2025-10-17) | Team Vexer Connectors – Ubuntu | VEXER-CONN-UBUNTU-01-001 | Implement Ubuntu CSAF discovery and channel selection for USN ingestion. | | ||||||
| | Sprint 6 | Vexer Ingest & Formats | src/StellaOps.Vexer.Connectors.OCI.OpenVEX.Attest/TASKS.md | TODO | Team Vexer Connectors – OCI | VEXER-CONN-OCI-01-001 | Wire OCI discovery/auth to fetch OpenVEX attestations for configured images. | | | Sprint 6 | Vexer Ingest & Formats | src/StellaOps.Vexer.Connectors.OCI.OpenVEX.Attest/TASKS.md | TODO | Team Vexer Connectors – OCI | VEXER-CONN-OCI-01-001 | Wire OCI discovery/auth to fetch OpenVEX attestations for configured images. | | ||||||
| | Sprint 6 | Vexer Ingest & Formats | src/StellaOps.Cli/TASKS.md | TODO | DevEx/CLI | VEXER-CLI-01-001 | Add `vexer` CLI verbs bridging to WebService with consistent auth and offline UX. | | | Sprint 6 | Vexer Ingest & Formats | src/StellaOps.Cli/TASKS.md | TODO | DevEx/CLI | VEXER-CLI-01-001 | Add `vexer` CLI verbs bridging to WebService with consistent auth and offline UX. | | ||||||
| | Sprint 7 | Contextual Truth Foundations | src/StellaOps.Vexer.Core/TASKS.md | TODO | Team Vexer Core & Policy | VEXER-CORE-02-001 | Context signal schema prep – extend consensus models with severity/KEV/EPSS fields and update canonical serializers. | | | Sprint 7 | Contextual Truth Foundations | src/StellaOps.Vexer.Core/TASKS.md | TODO | Team Vexer Core & Policy | VEXER-CORE-02-001 | Context signal schema prep – extend consensus models with severity/KEV/EPSS fields and update canonical serializers. | | ||||||
| | Sprint 7 | Contextual Truth Foundations | src/StellaOps.Vexer.Policy/TASKS.md | TODO | Team Vexer Policy | VEXER-POLICY-02-001 | Scoring coefficients & weight ceilings – add α/β options, weight boosts, and validation guidance. | | | Sprint 7 | Contextual Truth Foundations | src/StellaOps.Vexer.Policy/TASKS.md | TODO | Team Vexer Policy | VEXER-POLICY-02-001 | Scoring coefficients & weight ceilings – add α/β options, weight boosts, and validation guidance. | | ||||||
| | Sprint 7 | Contextual Truth Foundations | src/StellaOps.Vexer.Storage.Mongo/TASKS.md | TODO | Team Vexer Storage | VEXER-STORAGE-02-001 | Statement events & scoring signals – create immutable VEX statement store plus consensus extensions with indexes/migrations. | | | Sprint 7 | Contextual Truth Foundations | src/StellaOps.Vexer.Storage.Mongo/TASKS.md | TODO | Team Vexer Storage | VEXER-STORAGE-02-001 | Statement events & scoring signals – create immutable VEX statement store plus consensus extensions with indexes/migrations. | | ||||||
| | Sprint 7 | Contextual Truth Foundations | src/StellaOps.Vexer.WebService/TASKS.md | TODO | Team Vexer WebService | VEXER-WEB-01-004 | Resolve API & signed responses – expose `/vexer/resolve`, return signed consensus/score envelopes, document auth. | | | Sprint 7 | Contextual Truth Foundations | src/StellaOps.Vexer.WebService/TASKS.md | TODO | Team Vexer WebService | VEXER-WEB-01-004 | Resolve API & signed responses – expose `/vexer/resolve`, return signed consensus/score envelopes, document auth. | | ||||||
| | Sprint 7 | Contextual Truth Foundations | src/StellaOps.Vexer.Attestation/TASKS.md | TODO | Team Vexer Attestation | VEXER-ATTEST-01-002 | Rekor v2 client integration – ship transparency log client with retries and offline queue. | | | Sprint 7 | Contextual Truth Foundations | src/StellaOps.Vexer.Attestation/TASKS.md | DONE (2025-10-16) | Team Vexer Attestation | VEXER-ATTEST-01-002 | Rekor v2 client integration – ship transparency log client with retries and offline queue. | | ||||||
| | Sprint 7 | Contextual Truth Foundations | src/StellaOps.Vexer.Worker/TASKS.md | TODO | Team Vexer Worker | VEXER-WORKER-01-004 | TTL refresh & stability damper – schedule re-resolve loops and guard against status flapping. | | | Sprint 7 | Contextual Truth Foundations | src/StellaOps.Vexer.Worker/TASKS.md | TODO | Team Vexer Worker | VEXER-WORKER-01-004 | TTL refresh & stability damper – schedule re-resolve loops and guard against status flapping. | | ||||||
| | Sprint 7 | Contextual Truth Foundations | src/StellaOps.Vexer.Export/TASKS.md | TODO | Team Vexer Export | VEXER-EXPORT-01-005 | Score & resolve envelope surfaces – include signed consensus/score artifacts in exports. | | | Sprint 7 | Contextual Truth Foundations | src/StellaOps.Vexer.Export/TASKS.md | TODO | Team Vexer Export | VEXER-EXPORT-01-005 | Score & resolve envelope surfaces – include signed consensus/score artifacts in exports. | | ||||||
| | Sprint 7 | Contextual Truth Foundations | src/StellaOps.Feedser.Core/TASKS.md | TODO | Team Core Engine & Storage Analytics | FEEDCORE-ENGINE-07-001 | Advisory event log & asOf queries – surface immutable statements and replay capability. | | | Sprint 7 | Contextual Truth Foundations | src/StellaOps.Feedser.Core/TASKS.md | TODO | Team Core Engine & Storage Analytics | FEEDCORE-ENGINE-07-001 | Advisory event log & asOf queries – surface immutable statements and replay capability. | | ||||||
|   | |||||||
							
								
								
									
										52
									
								
								seed-data/cert-bund/README.md
									
									
									
									
									
										Normal file
									
								
							
							
						
						
									
										52
									
								
								seed-data/cert-bund/README.md
									
									
									
									
									
										Normal file
									
								
							| @@ -0,0 +1,52 @@ | |||||||
|  | # CERT-Bund Offline Kit Seed Data | ||||||
|  |  | ||||||
|  | This directory stores **offline snapshots** for the CERT-Bund connector. | ||||||
|  | The artefacts mirror the public JSON search and export endpoints so | ||||||
|  | air‑gapped deployments can hydrate the connector without contacting the | ||||||
|  | portal. | ||||||
|  |  | ||||||
|  | > ⚠️ **Distribution notice** – CERT-Bund advisories are published by BSI | ||||||
|  | > (Federal Office for Information Security, Germany). Review the portal | ||||||
|  | > terms of use before redistributing the snapshots. Always keep the JSON | ||||||
|  | > payloads and accompanying SHA-256 sums together. | ||||||
|  |  | ||||||
|  | ## Recommended layout | ||||||
|  |  | ||||||
|  | ``` | ||||||
|  | seed-data/cert-bund/ | ||||||
|  | ├── search/                     # paginated search JSON files | ||||||
|  | │   ├── certbund-search-page-00.json | ||||||
|  | │   └── … | ||||||
|  | ├── export/                     # yearly export JSON files | ||||||
|  | │   ├── certbund-export-2014.json | ||||||
|  | │   └── … | ||||||
|  | ├── manifest/ | ||||||
|  | │   └── certbund-offline-manifest.json | ||||||
|  | └── certbund-offline-manifest.sha256 | ||||||
|  | ``` | ||||||
|  |  | ||||||
|  | Use `certbund-offline-manifest.json` to feed the Offline Kit build: every | ||||||
|  | entry contains `source`, `from`, `to`, `sha256`, `capturedAt`, and the | ||||||
|  | relative file path. The manifest is deterministic when regenerated with | ||||||
|  | the tooling described below. | ||||||
|  |  | ||||||
|  | ## Tooling | ||||||
|  |  | ||||||
|  | Run the helper under `tools/` to capture fresh snapshots or regenerate | ||||||
|  | the manifest: | ||||||
|  |  | ||||||
|  | ``` | ||||||
|  | python tools/certbund_offline_snapshot.py --output seed-data/cert-bund | ||||||
|  | ``` | ||||||
|  |  | ||||||
|  | See the connector operations guide | ||||||
|  | (`docs/ops/feedser-certbund-operations.md`) for detailed usage, | ||||||
|  | including how to provide cookies/tokens when the portal requires manual | ||||||
|  | authentication. | ||||||
|  |  | ||||||
|  | ## Git hygiene | ||||||
|  |  | ||||||
|  | - JSON payloads and checksums are **ignored by Git**. Generate them | ||||||
|  |   locally when preparing an Offline Kit bundle. | ||||||
|  | - Commit documentation, scripts, and manifest templates only – never the | ||||||
|  |   exported advisory data itself. | ||||||
| @@ -9,4 +9,4 @@ | |||||||
| |FEEDCONN-CERTBUND-02-006 Telemetry & documentation|DevEx|Docs|**DONE (2025-10-15)** – Added `CertBundDiagnostics` (meter `StellaOps.Feedser.Source.CertBund`) with fetch/parse/map counters + histograms, recorded coverage days, wired stage summary logs, and published the ops runbook (`docs/ops/feedser-certbund-operations.md`).| | |FEEDCONN-CERTBUND-02-006 Telemetry & documentation|DevEx|Docs|**DONE (2025-10-15)** – Added `CertBundDiagnostics` (meter `StellaOps.Feedser.Source.CertBund`) with fetch/parse/map counters + histograms, recorded coverage days, wired stage summary logs, and published the ops runbook (`docs/ops/feedser-certbund-operations.md`).| | ||||||
| |FEEDCONN-CERTBUND-02-007 Feed history & locale assessment|BE-Conn-CERTBUND|Research|**DONE (2025-10-15)** – Measured RSS retention (~6 days/≈250 items), captured connector-driven backfill guidance in the runbook, and aligned locale guidance (preserve `language=de`, Docs glossary follow-up). **Next:** coordinate with Tools to land the state-seeding helper so scripted backfills replace manual Mongo tweaks.| | |FEEDCONN-CERTBUND-02-007 Feed history & locale assessment|BE-Conn-CERTBUND|Research|**DONE (2025-10-15)** – Measured RSS retention (~6 days/≈250 items), captured connector-driven backfill guidance in the runbook, and aligned locale guidance (preserve `language=de`, Docs glossary follow-up). **Next:** coordinate with Tools to land the state-seeding helper so scripted backfills replace manual Mongo tweaks.| | ||||||
| |FEEDCONN-CERTBUND-02-008 Session bootstrap & cookie strategy|BE-Conn-CERTBUND|Source.Common|**DONE (2025-10-14)** – Feed client primes the portal session (cookie container via `SocketsHttpHandler`), shares cookies across detail requests, and documents bootstrap behaviour in options (`PortalBootstrapUri`).| | |FEEDCONN-CERTBUND-02-008 Session bootstrap & cookie strategy|BE-Conn-CERTBUND|Source.Common|**DONE (2025-10-14)** – Feed client primes the portal session (cookie container via `SocketsHttpHandler`), shares cookies across detail requests, and documents bootstrap behaviour in options (`PortalBootstrapUri`).| | ||||||
| |FEEDCONN-CERTBUND-02-009 Offline Kit export packaging|BE-Conn-CERTBUND, Docs|Offline Kit|**TODO** – Capture JSON search/export snapshots (per-year splits), generate manifest fields (`source`,`from`,`to`,`sha256`,`capturedAt`), and update Offline Kit docs so air-gapped deployments can seed historical CERT-Bund advisories without live fetching. **Remark:** follow the interim workflow documented in `docs/ops/feedser-certbund-operations.md` §3.3 until the packaged artefacts ship.| | |FEEDCONN-CERTBUND-02-009 Offline Kit export packaging|BE-Conn-CERTBUND, Docs|Offline Kit|**DONE (2025-10-17)** – Added `tools/certbund_offline_snapshot.py` to capture search/export JSON, emit deterministic manifests + SHA files, and refreshed docs (`docs/ops/feedser-certbund-operations.md`, `docs/24_OFFLINE_KIT.md`) with offline-kit instructions and manifest layout guidance. Seed data README/ignore rules cover local snapshot hygiene.| | ||||||
|   | |||||||
| @@ -5,12 +5,12 @@ Defines shared connector infrastructure for Vexer, including base contexts, resu | |||||||
| - `IVexConnector` context implementation, raw store helpers, verification hooks, and telemetry utilities. | - `IVexConnector` context implementation, raw store helpers, verification hooks, and telemetry utilities. | ||||||
| - Configuration primitives (YAML parsing, secrets handling guidelines) and options validation. | - Configuration primitives (YAML parsing, secrets handling guidelines) and options validation. | ||||||
| - Connector lifecycle helpers for retries, paging, `.well-known` discovery, and resume markers. | - Connector lifecycle helpers for retries, paging, `.well-known` discovery, and resume markers. | ||||||
| - Documentation for connector packaging, plugin manifest metadata, and DI registration. | - Documentation for connector packaging, plugin manifest metadata, and DI registration (see `docs/dev/30_VEXER_CONNECTOR_GUIDE.md` and `docs/dev/templates/vexer-connector/`). | ||||||
| ## Participants | ## Participants | ||||||
| - All Vexer connector projects reference this module to obtain base classes and context services. | - All Vexer connector projects reference this module to obtain base classes and context services. | ||||||
| - WebService/Worker instantiate connectors via plugin loader leveraging abstractions defined here. | - WebService/Worker instantiate connectors via plugin loader leveraging abstractions defined here. | ||||||
| ## Interfaces & contracts | ## Interfaces & contracts | ||||||
| - Connector context, result, and telemetry interfaces; `ConnectorDescriptor`, `ConnectorOptions`, authentication helpers. | - Connector context, result, and telemetry interfaces; `VexConnectorDescriptor`, `VexConnectorBase`, options binder/validators, authentication helpers. | ||||||
| - Utility classes for HTTP clients, throttling, and deterministic logging. | - Utility classes for HTTP clients, throttling, and deterministic logging. | ||||||
| ## In/Out of scope | ## In/Out of scope | ||||||
| In: shared abstractions, helper utilities, configuration binding, documentation for connector authors. | In: shared abstractions, helper utilities, configuration binding, documentation for connector authors. | ||||||
|   | |||||||
| @@ -0,0 +1,12 @@ | |||||||
|  | using System.Collections.Generic; | ||||||
|  |  | ||||||
|  | namespace StellaOps.Vexer.Connectors.Abstractions; | ||||||
|  |  | ||||||
|  | /// <summary> | ||||||
|  | /// Custom validator hook executed after connector options are bound. | ||||||
|  | /// </summary> | ||||||
|  | /// <typeparam name="TOptions">Connector-specific options type.</typeparam> | ||||||
|  | public interface IVexConnectorOptionsValidator<in TOptions> | ||||||
|  | { | ||||||
|  |     void Validate(VexConnectorDescriptor descriptor, TOptions options, IList<string> errors); | ||||||
|  | } | ||||||
| @@ -0,0 +1,17 @@ | |||||||
|  | <Project Sdk="Microsoft.NET.Sdk"> | ||||||
|  |   <PropertyGroup> | ||||||
|  |     <TargetFramework>net10.0</TargetFramework> | ||||||
|  |     <LangVersion>preview</LangVersion> | ||||||
|  |     <Nullable>enable</Nullable> | ||||||
|  |     <ImplicitUsings>enable</ImplicitUsings> | ||||||
|  |     <TreatWarningsAsErrors>true</TreatWarningsAsErrors> | ||||||
|  |   </PropertyGroup> | ||||||
|  |   <ItemGroup> | ||||||
|  |     <ProjectReference Include="..\StellaOps.Vexer.Core\StellaOps.Vexer.Core.csproj" /> | ||||||
|  |   </ItemGroup> | ||||||
|  |   <ItemGroup> | ||||||
|  |     <PackageReference Include="Microsoft.Extensions.Configuration" Version="8.0.0" /> | ||||||
|  |     <PackageReference Include="Microsoft.Extensions.Configuration.Binder" Version="8.0.1" /> | ||||||
|  |     <PackageReference Include="Microsoft.Extensions.Logging.Abstractions" Version="8.0.0" /> | ||||||
|  |   </ItemGroup> | ||||||
|  | </Project> | ||||||
| @@ -2,6 +2,6 @@ If you are working on this file you need to read docs/ARCHITECTURE_VEXER.md and | |||||||
| # TASKS | # TASKS | ||||||
| | Task | Owner(s) | Depends on | Notes | | | Task | Owner(s) | Depends on | Notes | | ||||||
| |---|---|---|---| | |---|---|---|---| | ||||||
| |VEXER-CONN-ABS-01-001 – Connector context & base classes|Team Vexer Connectors|VEXER-CORE-01-003|TODO – Implement `VexConnectorContext`, result types, helper base classes, and deterministic logging helpers for connectors.| | |VEXER-CONN-ABS-01-001 – Connector context & base classes|Team Vexer Connectors|VEXER-CORE-01-003|**DONE (2025-10-17)** – Added `StellaOps.Vexer.Connectors.Abstractions` project with `VexConnectorBase`, deterministic logging scopes, metadata builder helpers, and connector descriptors; docs updated to highlight the shared abstractions.| | ||||||
| |VEXER-CONN-ABS-01-002 – YAML options & validation|Team Vexer Connectors|VEXER-CONN-ABS-01-001|TODO – Provide strongly-typed options binding/validation for connector YAML definitions with offline-safe defaults.| | |VEXER-CONN-ABS-01-002 – YAML options & validation|Team Vexer Connectors|VEXER-CONN-ABS-01-001|**DONE (2025-10-17)** – Delivered `VexConnectorOptionsBinder` + binder options/validators, environment-variable expansion, data-annotation checks, and custom validation hooks with documentation updates covering the workflow.| | ||||||
| |VEXER-CONN-ABS-01-003 – Plugin packaging & docs|Team Vexer Connectors|VEXER-CONN-ABS-01-001|TODO – Document connector packaging (NuGet manifest, plugin loader metadata) and supply reference templates for downstream connector modules.| | |VEXER-CONN-ABS-01-003 – Plugin packaging & docs|Team Vexer Connectors|VEXER-CONN-ABS-01-001|**DONE (2025-10-17)** – Authored `docs/dev/30_VEXER_CONNECTOR_GUIDE.md`, added quick-start template under `docs/dev/templates/vexer-connector/`, and updated module docs to reference the packaging workflow.| | ||||||
|   | |||||||
| @@ -0,0 +1,99 @@ | |||||||
|  | using System.Collections.Immutable; | ||||||
|  | using System.Security.Cryptography; | ||||||
|  | using Microsoft.Extensions.Logging; | ||||||
|  | using StellaOps.Vexer.Core; | ||||||
|  |  | ||||||
|  | namespace StellaOps.Vexer.Connectors.Abstractions; | ||||||
|  |  | ||||||
|  | /// <summary> | ||||||
|  | /// Convenience base class for implementing <see cref="IVexConnector" />. | ||||||
|  | /// </summary> | ||||||
|  | public abstract class VexConnectorBase : IVexConnector | ||||||
|  | { | ||||||
|  |     protected VexConnectorBase(VexConnectorDescriptor descriptor, ILogger logger, TimeProvider? timeProvider = null) | ||||||
|  |     { | ||||||
|  |         Descriptor = descriptor ?? throw new ArgumentNullException(nameof(descriptor)); | ||||||
|  |         Logger = logger ?? throw new ArgumentNullException(nameof(logger)); | ||||||
|  |         TimeProvider = timeProvider ?? TimeProvider.System; | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     /// <inheritdoc /> | ||||||
|  |     public string Id => Descriptor.Id; | ||||||
|  |  | ||||||
|  |     /// <inheritdoc /> | ||||||
|  |     public VexProviderKind Kind => Descriptor.Kind; | ||||||
|  |  | ||||||
|  |     protected VexConnectorDescriptor Descriptor { get; } | ||||||
|  |  | ||||||
|  |     protected ILogger Logger { get; } | ||||||
|  |  | ||||||
|  |     protected TimeProvider TimeProvider { get; } | ||||||
|  |  | ||||||
|  |     protected DateTimeOffset UtcNow() => TimeProvider.GetUtcNow(); | ||||||
|  |  | ||||||
|  |     protected VexRawDocument CreateRawDocument( | ||||||
|  |         VexDocumentFormat format, | ||||||
|  |         Uri sourceUri, | ||||||
|  |         ReadOnlyMemory<byte> content, | ||||||
|  |         ImmutableDictionary<string, string>? metadata = null) | ||||||
|  |     { | ||||||
|  |         if (sourceUri is null) | ||||||
|  |         { | ||||||
|  |             throw new ArgumentNullException(nameof(sourceUri)); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         var digest = ComputeSha256(content.Span); | ||||||
|  |         var captured = TimeProvider.GetUtcNow(); | ||||||
|  |         return new VexRawDocument( | ||||||
|  |             Descriptor.Id, | ||||||
|  |             format, | ||||||
|  |             sourceUri, | ||||||
|  |             captured, | ||||||
|  |             digest, | ||||||
|  |             content, | ||||||
|  |             metadata ?? ImmutableDictionary<string, string>.Empty); | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     protected IDisposable BeginConnectorScope(string operation, IReadOnlyDictionary<string, object?>? metadata = null) | ||||||
|  |         => VexConnectorLogScope.Begin(Logger, Descriptor, operation, metadata); | ||||||
|  |  | ||||||
|  |     protected void LogConnectorEvent(LogLevel level, string eventName, string message, IReadOnlyDictionary<string, object?>? metadata = null, Exception? exception = null) | ||||||
|  |     { | ||||||
|  |         using var scope = BeginConnectorScope(eventName, metadata); | ||||||
|  |         if (exception is null) | ||||||
|  |         { | ||||||
|  |             Logger.Log(level, "{Message}", message); | ||||||
|  |         } | ||||||
|  |         else | ||||||
|  |         { | ||||||
|  |             Logger.Log(level, exception, "{Message}", message); | ||||||
|  |         } | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     protected ImmutableDictionary<string, string> BuildMetadata(Action<VexConnectorMetadataBuilder> configure) | ||||||
|  |     { | ||||||
|  |         ArgumentNullException.ThrowIfNull(configure); | ||||||
|  |         var builder = new VexConnectorMetadataBuilder(); | ||||||
|  |         configure(builder); | ||||||
|  |         return builder.Build(); | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     private static string ComputeSha256(ReadOnlySpan<byte> content) | ||||||
|  |     { | ||||||
|  |         Span<byte> buffer = stackalloc byte[32]; | ||||||
|  |         if (SHA256.TryHashData(content, buffer, out _)) | ||||||
|  |         { | ||||||
|  |             return "sha256:" + Convert.ToHexString(buffer).ToLowerInvariant(); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         using var sha = SHA256.Create(); | ||||||
|  |         var hash = sha.ComputeHash(content.ToArray()); | ||||||
|  |         return "sha256:" + Convert.ToHexString(hash).ToLowerInvariant(); | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     public abstract ValueTask ValidateAsync(VexConnectorSettings settings, CancellationToken cancellationToken); | ||||||
|  |  | ||||||
|  |     public abstract IAsyncEnumerable<VexRawDocument> FetchAsync(VexConnectorContext context, CancellationToken cancellationToken); | ||||||
|  |  | ||||||
|  |     public abstract ValueTask<VexClaimBatch> NormalizeAsync(VexRawDocument document, CancellationToken cancellationToken); | ||||||
|  | } | ||||||
| @@ -0,0 +1,54 @@ | |||||||
|  | using System.Collections.Immutable; | ||||||
|  | using StellaOps.Vexer.Core; | ||||||
|  |  | ||||||
|  | namespace StellaOps.Vexer.Connectors.Abstractions; | ||||||
|  |  | ||||||
|  | /// <summary> | ||||||
|  | /// Static descriptor for a Vexer connector plug-in. | ||||||
|  | /// </summary> | ||||||
|  | public sealed record VexConnectorDescriptor | ||||||
|  | { | ||||||
|  |     public VexConnectorDescriptor(string id, VexProviderKind kind, string displayName) | ||||||
|  |     { | ||||||
|  |         if (string.IsNullOrWhiteSpace(id)) | ||||||
|  |         { | ||||||
|  |             throw new ArgumentException("Connector id must be provided.", nameof(id)); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         Id = id; | ||||||
|  |         Kind = kind; | ||||||
|  |         DisplayName = string.IsNullOrWhiteSpace(displayName) ? id : displayName; | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     /// <summary> | ||||||
|  |     /// Stable connector identifier (matches provider id). | ||||||
|  |     /// </summary> | ||||||
|  |     public string Id { get; } | ||||||
|  |  | ||||||
|  |     /// <summary> | ||||||
|  |     /// Provider kind served by the connector. | ||||||
|  |     /// </summary> | ||||||
|  |     public VexProviderKind Kind { get; } | ||||||
|  |  | ||||||
|  |     /// <summary> | ||||||
|  |     /// Human friendly name used in logs/diagnostics. | ||||||
|  |     /// </summary> | ||||||
|  |     public string DisplayName { get; } | ||||||
|  |  | ||||||
|  |     /// <summary> | ||||||
|  |     /// Optional friendly description. | ||||||
|  |     /// </summary> | ||||||
|  |     public string? Description { get; init; } | ||||||
|  |  | ||||||
|  |     /// <summary> | ||||||
|  |     /// Document formats the connector is expected to emit. | ||||||
|  |     /// </summary> | ||||||
|  |     public ImmutableArray<VexDocumentFormat> SupportedFormats { get; init; } = ImmutableArray<VexDocumentFormat>.Empty; | ||||||
|  |  | ||||||
|  |     /// <summary> | ||||||
|  |     /// Optional tags surfaced in diagnostics (e.g. "beta", "offline"). | ||||||
|  |     /// </summary> | ||||||
|  |     public ImmutableArray<string> Tags { get; init; } = ImmutableArray<string>.Empty; | ||||||
|  |  | ||||||
|  |     public override string ToString() => $"{Id} ({Kind})"; | ||||||
|  | } | ||||||
| @@ -0,0 +1,50 @@ | |||||||
|  | using System.Linq; | ||||||
|  | using Microsoft.Extensions.Logging; | ||||||
|  | using StellaOps.Vexer.Core; | ||||||
|  |  | ||||||
|  | namespace StellaOps.Vexer.Connectors.Abstractions; | ||||||
|  |  | ||||||
|  | /// <summary> | ||||||
|  | /// Helper to establish deterministic logging scopes for connector operations. | ||||||
|  | /// </summary> | ||||||
|  | public static class VexConnectorLogScope | ||||||
|  | { | ||||||
|  |     public static IDisposable Begin(ILogger logger, VexConnectorDescriptor descriptor, string operation, IReadOnlyDictionary<string, object?>? metadata = null) | ||||||
|  |     { | ||||||
|  |         ArgumentNullException.ThrowIfNull(logger); | ||||||
|  |         ArgumentNullException.ThrowIfNull(descriptor); | ||||||
|  |         ArgumentException.ThrowIfNullOrEmpty(operation); | ||||||
|  |  | ||||||
|  |         var scopeValues = new List<KeyValuePair<string, object?>> | ||||||
|  |         { | ||||||
|  |             new("vex.connector.id", descriptor.Id), | ||||||
|  |             new("vex.connector.kind", descriptor.Kind.ToString()), | ||||||
|  |             new("vex.connector.operation", operation), | ||||||
|  |         }; | ||||||
|  |  | ||||||
|  |         if (!string.Equals(descriptor.DisplayName, descriptor.Id, StringComparison.Ordinal)) | ||||||
|  |         { | ||||||
|  |             scopeValues.Add(new KeyValuePair<string, object?>("vex.connector.displayName", descriptor.DisplayName)); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         if (!string.IsNullOrWhiteSpace(descriptor.Description)) | ||||||
|  |         { | ||||||
|  |             scopeValues.Add(new KeyValuePair<string, object?>("vex.connector.description", descriptor.Description)); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         if (!descriptor.Tags.IsDefaultOrEmpty) | ||||||
|  |         { | ||||||
|  |             scopeValues.Add(new KeyValuePair<string, object?>("vex.connector.tags", string.Join(",", descriptor.Tags))); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         if (metadata is not null) | ||||||
|  |         { | ||||||
|  |             foreach (var kvp in metadata.OrderBy(static pair => pair.Key, StringComparer.Ordinal)) | ||||||
|  |             { | ||||||
|  |                 scopeValues.Add(new KeyValuePair<string, object?>($"vex.{kvp.Key}", kvp.Value)); | ||||||
|  |             } | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         return logger.BeginScope(scopeValues)!; | ||||||
|  |     } | ||||||
|  | } | ||||||
| @@ -0,0 +1,37 @@ | |||||||
|  | using System.Collections.Immutable; | ||||||
|  |  | ||||||
|  | namespace StellaOps.Vexer.Connectors.Abstractions; | ||||||
|  |  | ||||||
|  | /// <summary> | ||||||
|  | /// Builds deterministic metadata dictionaries for raw documents and logging scopes. | ||||||
|  | /// </summary> | ||||||
|  | public sealed class VexConnectorMetadataBuilder | ||||||
|  | { | ||||||
|  |     private readonly SortedDictionary<string, string> _values = new(StringComparer.Ordinal); | ||||||
|  |  | ||||||
|  |     public VexConnectorMetadataBuilder Add(string key, string? value) | ||||||
|  |     { | ||||||
|  |         if (!string.IsNullOrWhiteSpace(key) && !string.IsNullOrWhiteSpace(value)) | ||||||
|  |         { | ||||||
|  |             _values[key] = value!; | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         return this; | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     public VexConnectorMetadataBuilder Add(string key, DateTimeOffset value) | ||||||
|  |         => Add(key, value.ToUniversalTime().ToString("O")); | ||||||
|  |  | ||||||
|  |     public VexConnectorMetadataBuilder AddRange(IEnumerable<KeyValuePair<string, string?>> items) | ||||||
|  |     { | ||||||
|  |         foreach (var item in items) | ||||||
|  |         { | ||||||
|  |             Add(item.Key, item.Value); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         return this; | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     public ImmutableDictionary<string, string> Build() | ||||||
|  |         => _values.ToImmutableDictionary(static pair => pair.Key, static pair => pair.Value, StringComparer.Ordinal); | ||||||
|  | } | ||||||
| @@ -0,0 +1,157 @@ | |||||||
|  | using System.Collections.Immutable; | ||||||
|  | using System.ComponentModel.DataAnnotations; | ||||||
|  | using System.Linq; | ||||||
|  | using Microsoft.Extensions.Configuration; | ||||||
|  | using StellaOps.Vexer.Core; | ||||||
|  |  | ||||||
|  | namespace StellaOps.Vexer.Connectors.Abstractions; | ||||||
|  |  | ||||||
|  | /// <summary> | ||||||
|  | /// Provides strongly typed binding and validation for connector options. | ||||||
|  | /// </summary> | ||||||
|  | public static class VexConnectorOptionsBinder | ||||||
|  | { | ||||||
|  |     public static TOptions Bind<TOptions>( | ||||||
|  |         VexConnectorDescriptor descriptor, | ||||||
|  |         VexConnectorSettings settings, | ||||||
|  |         VexConnectorOptionsBinderOptions? options = null, | ||||||
|  |         IEnumerable<IVexConnectorOptionsValidator<TOptions>>? validators = null) | ||||||
|  |         where TOptions : class, new() | ||||||
|  |     { | ||||||
|  |         ArgumentNullException.ThrowIfNull(descriptor); | ||||||
|  |         ArgumentNullException.ThrowIfNull(settings); | ||||||
|  |  | ||||||
|  |         var binderSettings = options ?? new VexConnectorOptionsBinderOptions(); | ||||||
|  |         var transformed = TransformValues(settings, binderSettings); | ||||||
|  |  | ||||||
|  |         var configuration = BuildConfiguration(transformed); | ||||||
|  |  | ||||||
|  |         var result = new TOptions(); | ||||||
|  |         var errors = new List<string>(); | ||||||
|  |  | ||||||
|  |         try | ||||||
|  |         { | ||||||
|  |             configuration.Bind( | ||||||
|  |                 result, | ||||||
|  |                 binderOptions => binderOptions.ErrorOnUnknownConfiguration = !binderSettings.AllowUnknownKeys); | ||||||
|  |         } | ||||||
|  |         catch (InvalidOperationException ex) when (!binderSettings.AllowUnknownKeys) | ||||||
|  |         { | ||||||
|  |             errors.Add(ex.Message); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         binderSettings.PostConfigure?.Invoke(result); | ||||||
|  |  | ||||||
|  |         if (binderSettings.ValidateDataAnnotations) | ||||||
|  |         { | ||||||
|  |             ValidateDataAnnotations(result, errors); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         if (validators is not null) | ||||||
|  |         { | ||||||
|  |             foreach (var validator in validators) | ||||||
|  |             { | ||||||
|  |                 validator?.Validate(descriptor, result, errors); | ||||||
|  |             } | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         if (errors.Count > 0) | ||||||
|  |         { | ||||||
|  |             throw new VexConnectorOptionsValidationException(descriptor.Id, errors); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         return result; | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     private static ImmutableDictionary<string, string?> TransformValues( | ||||||
|  |         VexConnectorSettings settings, | ||||||
|  |         VexConnectorOptionsBinderOptions binderOptions) | ||||||
|  |     { | ||||||
|  |         var builder = ImmutableDictionary.CreateBuilder<string, string?>(StringComparer.OrdinalIgnoreCase); | ||||||
|  |         foreach (var kvp in settings.Values) | ||||||
|  |         { | ||||||
|  |             var value = kvp.Value; | ||||||
|  |             if (binderOptions.TrimWhitespace && value is not null) | ||||||
|  |             { | ||||||
|  |                 value = value.Trim(); | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             if (binderOptions.TreatEmptyAsNull && string.IsNullOrEmpty(value)) | ||||||
|  |             { | ||||||
|  |                 value = null; | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             if (value is not null && binderOptions.ExpandEnvironmentVariables) | ||||||
|  |             { | ||||||
|  |                 value = Environment.ExpandEnvironmentVariables(value); | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             if (binderOptions.ValueTransformer is not null) | ||||||
|  |             { | ||||||
|  |                 value = binderOptions.ValueTransformer.Invoke(kvp.Key, value); | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             builder[kvp.Key] = value; | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         return builder.ToImmutable(); | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     private static IConfiguration BuildConfiguration(ImmutableDictionary<string, string?> values) | ||||||
|  |     { | ||||||
|  |         var sources = new List<KeyValuePair<string, string?>>(); | ||||||
|  |         foreach (var kvp in values) | ||||||
|  |         { | ||||||
|  |             if (kvp.Value is not null) | ||||||
|  |             { | ||||||
|  |                 sources.Add(new KeyValuePair<string, string?>(kvp.Key, kvp.Value)); | ||||||
|  |             } | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         var configurationBuilder = new ConfigurationBuilder(); | ||||||
|  |         configurationBuilder.Add(new DictionaryConfigurationSource(sources)); | ||||||
|  |         return configurationBuilder.Build(); | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     private static void ValidateDataAnnotations<TOptions>(TOptions options, IList<string> errors) | ||||||
|  |     { | ||||||
|  |         var validationResults = new List<ValidationResult>(); | ||||||
|  |         var validationContext = new ValidationContext(options!); | ||||||
|  |         if (!Validator.TryValidateObject(options!, validationContext, validationResults, validateAllProperties: true)) | ||||||
|  |         { | ||||||
|  |             foreach (var validationResult in validationResults) | ||||||
|  |             { | ||||||
|  |                 if (!string.IsNullOrWhiteSpace(validationResult.ErrorMessage)) | ||||||
|  |                 { | ||||||
|  |                     errors.Add(validationResult.ErrorMessage); | ||||||
|  |                 } | ||||||
|  |             } | ||||||
|  |         } | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     private sealed class DictionaryConfigurationSource : IConfigurationSource | ||||||
|  |     { | ||||||
|  |         private readonly IReadOnlyList<KeyValuePair<string, string?>> _data; | ||||||
|  |  | ||||||
|  |         public DictionaryConfigurationSource(IEnumerable<KeyValuePair<string, string?>> data) | ||||||
|  |         { | ||||||
|  |             _data = data?.ToList() ?? new List<KeyValuePair<string, string?>>(); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         public IConfigurationProvider Build(IConfigurationBuilder builder) => new DictionaryConfigurationProvider(_data); | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     private sealed class DictionaryConfigurationProvider : ConfigurationProvider | ||||||
|  |     { | ||||||
|  |         public DictionaryConfigurationProvider(IEnumerable<KeyValuePair<string, string?>> data) | ||||||
|  |         { | ||||||
|  |             foreach (var pair in data) | ||||||
|  |             { | ||||||
|  |                 if (pair.Value is not null) | ||||||
|  |                 { | ||||||
|  |                     Data[pair.Key] = pair.Value; | ||||||
|  |                 } | ||||||
|  |             } | ||||||
|  |         } | ||||||
|  |     } | ||||||
|  | } | ||||||
| @@ -0,0 +1,45 @@ | |||||||
|  | namespace StellaOps.Vexer.Connectors.Abstractions; | ||||||
|  |  | ||||||
|  | /// <summary> | ||||||
|  | /// Customisation options for connector options binding. | ||||||
|  | /// </summary> | ||||||
|  | public sealed class VexConnectorOptionsBinderOptions | ||||||
|  | { | ||||||
|  |     /// <summary> | ||||||
|  |     /// Indicates whether environment variables should be expanded in option values. | ||||||
|  |     /// Defaults to <c>true</c>. | ||||||
|  |     /// </summary> | ||||||
|  |     public bool ExpandEnvironmentVariables { get; set; } = true; | ||||||
|  |  | ||||||
|  |     /// <summary> | ||||||
|  |     /// When <c>true</c> the binder trims whitespace around option values. | ||||||
|  |     /// </summary> | ||||||
|  |     public bool TrimWhitespace { get; set; } = true; | ||||||
|  |  | ||||||
|  |     /// <summary> | ||||||
|  |     /// Converts empty strings to <c>null</c> before binding. Default: <c>true</c>. | ||||||
|  |     /// </summary> | ||||||
|  |     public bool TreatEmptyAsNull { get; set; } = true; | ||||||
|  |  | ||||||
|  |     /// <summary> | ||||||
|  |     /// When <c>false</c>, binding fails if unknown configuration keys are provided. | ||||||
|  |     /// Default: <c>true</c> (permitting unknown keys). | ||||||
|  |     /// </summary> | ||||||
|  |     public bool AllowUnknownKeys { get; set; } = true; | ||||||
|  |  | ||||||
|  |     /// <summary> | ||||||
|  |     /// Enables <see cref="System.ComponentModel.DataAnnotations"/> validation after binding. | ||||||
|  |     /// Default: <c>true</c>. | ||||||
|  |     /// </summary> | ||||||
|  |     public bool ValidateDataAnnotations { get; set; } = true; | ||||||
|  |  | ||||||
|  |     /// <summary> | ||||||
|  |     /// Optional post-configuration callback executed after binding. | ||||||
|  |     /// </summary> | ||||||
|  |     public Action<object>? PostConfigure { get; set; } | ||||||
|  |  | ||||||
|  |     /// <summary> | ||||||
|  |     /// Optional hook to transform raw configuration values before binding. | ||||||
|  |     /// </summary> | ||||||
|  |     public Func<string, string?, string?>? ValueTransformer { get; set; } | ||||||
|  | } | ||||||
| @@ -0,0 +1,36 @@ | |||||||
|  | using System.Collections.Immutable; | ||||||
|  |  | ||||||
|  | namespace StellaOps.Vexer.Connectors.Abstractions; | ||||||
|  |  | ||||||
|  | public sealed class VexConnectorOptionsValidationException : Exception | ||||||
|  | { | ||||||
|  |     public VexConnectorOptionsValidationException( | ||||||
|  |         string connectorId, | ||||||
|  |         IEnumerable<string> errors) | ||||||
|  |         : base(BuildMessage(connectorId, errors)) | ||||||
|  |     { | ||||||
|  |         ConnectorId = connectorId; | ||||||
|  |         Errors = errors?.ToImmutableArray() ?? ImmutableArray<string>.Empty; | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     public string ConnectorId { get; } | ||||||
|  |  | ||||||
|  |     public ImmutableArray<string> Errors { get; } | ||||||
|  |  | ||||||
|  |     private static string BuildMessage(string connectorId, IEnumerable<string> errors) | ||||||
|  |     { | ||||||
|  |         var builder = new System.Text.StringBuilder(); | ||||||
|  |         builder.Append("Connector options validation failed for '"); | ||||||
|  |         builder.Append(connectorId); | ||||||
|  |         builder.Append("'."); | ||||||
|  |  | ||||||
|  |         var list = errors?.ToImmutableArray() ?? ImmutableArray<string>.Empty; | ||||||
|  |         if (!list.IsDefaultOrEmpty) | ||||||
|  |         { | ||||||
|  |             builder.Append(" Errors: "); | ||||||
|  |             builder.Append(string.Join("; ", list)); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         return builder.ToString(); | ||||||
|  |     } | ||||||
|  | } | ||||||
| @@ -0,0 +1,214 @@ | |||||||
|  | using System.Collections.Generic; | ||||||
|  | using System.Net; | ||||||
|  | using System.Net.Http; | ||||||
|  | using System.Text; | ||||||
|  | using FluentAssertions; | ||||||
|  | using Microsoft.Extensions.Caching.Memory; | ||||||
|  | using Microsoft.Extensions.Logging.Abstractions; | ||||||
|  | using Microsoft.Extensions.Options; | ||||||
|  | using Microsoft.Extensions.DependencyInjection; | ||||||
|  | using StellaOps.Vexer.Connectors.Abstractions; | ||||||
|  | using StellaOps.Vexer.Connectors.Cisco.CSAF; | ||||||
|  | using StellaOps.Vexer.Connectors.Cisco.CSAF.Configuration; | ||||||
|  | using StellaOps.Vexer.Connectors.Cisco.CSAF.Metadata; | ||||||
|  | using StellaOps.Vexer.Core; | ||||||
|  | using StellaOps.Vexer.Storage.Mongo; | ||||||
|  | using System.Collections.Immutable; | ||||||
|  | using System.IO.Abstractions.TestingHelpers; | ||||||
|  | using Xunit; | ||||||
|  | using System.Threading; | ||||||
|  |  | ||||||
|  | namespace StellaOps.Vexer.Connectors.Cisco.CSAF.Tests.Connectors; | ||||||
|  |  | ||||||
|  | public sealed class CiscoCsafConnectorTests | ||||||
|  | { | ||||||
|  |     [Fact] | ||||||
|  |     public async Task FetchAsync_NewAdvisory_StoresDocumentAndUpdatesState() | ||||||
|  |     { | ||||||
|  |         var responses = new Dictionary<Uri, Queue<HttpResponseMessage>> | ||||||
|  |         { | ||||||
|  |             [new Uri("https://api.cisco.test/.well-known/csaf/provider-metadata.json")] = QueueResponses(""" | ||||||
|  |                 { | ||||||
|  |                   "metadata": { | ||||||
|  |                     "publisher": { | ||||||
|  |                       "name": "Cisco", | ||||||
|  |                       "category": "vendor", | ||||||
|  |                       "contact_details": { "id": "vexer:cisco" } | ||||||
|  |                     } | ||||||
|  |                   }, | ||||||
|  |                   "distributions": { | ||||||
|  |                     "directories": [ "https://api.cisco.test/csaf/" ] | ||||||
|  |                   } | ||||||
|  |                 } | ||||||
|  |                 """), | ||||||
|  |             [new Uri("https://api.cisco.test/csaf/index.json")] = QueueResponses(""" | ||||||
|  |                 { | ||||||
|  |                   "advisories": [ | ||||||
|  |                     { | ||||||
|  |                       "id": "cisco-sa-2025", | ||||||
|  |                       "url": "https://api.cisco.test/csaf/cisco-sa-2025.json", | ||||||
|  |                       "published": "2025-10-01T00:00:00Z", | ||||||
|  |                       "lastModified": "2025-10-02T00:00:00Z", | ||||||
|  |                       "sha256": "cafebabe" | ||||||
|  |                     } | ||||||
|  |                   ] | ||||||
|  |                 } | ||||||
|  |                 """), | ||||||
|  |             [new Uri("https://api.cisco.test/csaf/cisco-sa-2025.json")] = QueueResponses("{ \"document\": \"payload\" }") | ||||||
|  |         }; | ||||||
|  |  | ||||||
|  |         var handler = new RoutingHttpMessageHandler(responses); | ||||||
|  |         var httpClient = new HttpClient(handler); | ||||||
|  |         var factory = new SingleHttpClientFactory(httpClient); | ||||||
|  |         var metadataLoader = new CiscoProviderMetadataLoader( | ||||||
|  |             factory, | ||||||
|  |             new MemoryCache(new MemoryCacheOptions()), | ||||||
|  |             Options.Create(new CiscoConnectorOptions | ||||||
|  |             { | ||||||
|  |                 MetadataUri = "https://api.cisco.test/.well-known/csaf/provider-metadata.json", | ||||||
|  |                 PersistOfflineSnapshot = false, | ||||||
|  |             }), | ||||||
|  |             NullLogger<CiscoProviderMetadataLoader>.Instance, | ||||||
|  |             new MockFileSystem()); | ||||||
|  |  | ||||||
|  |         var stateRepository = new InMemoryConnectorStateRepository(); | ||||||
|  |         var connector = new CiscoCsafConnector( | ||||||
|  |             metadataLoader, | ||||||
|  |             factory, | ||||||
|  |             stateRepository, | ||||||
|  |             new[] { new CiscoConnectorOptionsValidator() }, | ||||||
|  |             NullLogger<CiscoCsafConnector>.Instance, | ||||||
|  |             TimeProvider.System); | ||||||
|  |  | ||||||
|  |         var settings = new VexConnectorSettings(ImmutableDictionary<string, string>.Empty); | ||||||
|  |         await connector.ValidateAsync(settings, CancellationToken.None); | ||||||
|  |  | ||||||
|  |         var sink = new InMemoryRawSink(); | ||||||
|  |         var context = new VexConnectorContext(null, VexConnectorSettings.Empty, sink, new NoopSignatureVerifier(), new NoopNormalizerRouter(), new ServiceCollection().BuildServiceProvider()); | ||||||
|  |  | ||||||
|  |         var documents = new List<VexRawDocument>(); | ||||||
|  |         await foreach (var doc in connector.FetchAsync(context, CancellationToken.None)) | ||||||
|  |         { | ||||||
|  |             documents.Add(doc); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         documents.Should().HaveCount(1); | ||||||
|  |         sink.Documents.Should().HaveCount(1); | ||||||
|  |         stateRepository.CurrentState.Should().NotBeNull(); | ||||||
|  |         stateRepository.CurrentState!.DocumentDigests.Should().HaveCount(1); | ||||||
|  |  | ||||||
|  |         // second run should not refetch documents | ||||||
|  |         sink.Documents.Clear(); | ||||||
|  |         documents.Clear(); | ||||||
|  |  | ||||||
|  |         await foreach (var doc in connector.FetchAsync(context, CancellationToken.None)) | ||||||
|  |         { | ||||||
|  |             documents.Add(doc); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         documents.Should().BeEmpty(); | ||||||
|  |         sink.Documents.Should().BeEmpty(); | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     private static Queue<HttpResponseMessage> QueueResponses(string payload) | ||||||
|  |         => new(new[] | ||||||
|  |         { | ||||||
|  |             new HttpResponseMessage(HttpStatusCode.OK) | ||||||
|  |             { | ||||||
|  |                 Content = new StringContent(payload, Encoding.UTF8, "application/json"), | ||||||
|  |             } | ||||||
|  |         }); | ||||||
|  |  | ||||||
|  |     private sealed class RoutingHttpMessageHandler : HttpMessageHandler | ||||||
|  |     { | ||||||
|  |         private readonly Dictionary<Uri, Queue<HttpResponseMessage>> _responses; | ||||||
|  |  | ||||||
|  |         public RoutingHttpMessageHandler(Dictionary<Uri, Queue<HttpResponseMessage>> responses) | ||||||
|  |         { | ||||||
|  |             _responses = responses; | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         protected override Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) | ||||||
|  |         { | ||||||
|  |             if (request.RequestUri is not null && _responses.TryGetValue(request.RequestUri, out var queue) && queue.Count > 0) | ||||||
|  |             { | ||||||
|  |                 var response = queue.Peek(); | ||||||
|  |                 return Task.FromResult(response.Clone()); | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             return Task.FromResult(new HttpResponseMessage(HttpStatusCode.NotFound) | ||||||
|  |             { | ||||||
|  |                 Content = new StringContent($"No response configured for {request.RequestUri}"), | ||||||
|  |             }); | ||||||
|  |         } | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     private sealed class SingleHttpClientFactory : IHttpClientFactory | ||||||
|  |     { | ||||||
|  |         private readonly HttpClient _client; | ||||||
|  |  | ||||||
|  |         public SingleHttpClientFactory(HttpClient client) | ||||||
|  |         { | ||||||
|  |             _client = client; | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         public HttpClient CreateClient(string name) => _client; | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     private sealed class InMemoryConnectorStateRepository : IVexConnectorStateRepository | ||||||
|  |     { | ||||||
|  |         public VexConnectorState? CurrentState { get; private set; } | ||||||
|  |  | ||||||
|  |         public ValueTask<VexConnectorState?> GetAsync(string connectorId, CancellationToken cancellationToken) | ||||||
|  |             => ValueTask.FromResult(CurrentState); | ||||||
|  |  | ||||||
|  |         public ValueTask SaveAsync(VexConnectorState state, CancellationToken cancellationToken) | ||||||
|  |         { | ||||||
|  |             CurrentState = state; | ||||||
|  |             return ValueTask.CompletedTask; | ||||||
|  |         } | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     private sealed class InMemoryRawSink : IVexRawDocumentSink | ||||||
|  |     { | ||||||
|  |         public List<VexRawDocument> Documents { get; } = new(); | ||||||
|  |  | ||||||
|  |         public ValueTask StoreAsync(VexRawDocument document, CancellationToken cancellationToken) | ||||||
|  |         { | ||||||
|  |             Documents.Add(document); | ||||||
|  |             return ValueTask.CompletedTask; | ||||||
|  |         } | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     private sealed class NoopSignatureVerifier : IVexSignatureVerifier | ||||||
|  |     { | ||||||
|  |         public ValueTask<VexSignatureMetadata?> VerifyAsync(VexRawDocument document, CancellationToken cancellationToken) | ||||||
|  |             => ValueTask.FromResult<VexSignatureMetadata?>(null); | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     private sealed class NoopNormalizerRouter : IVexNormalizerRouter | ||||||
|  |     { | ||||||
|  |         public ValueTask<VexClaimBatch> NormalizeAsync(VexRawDocument document, CancellationToken cancellationToken) | ||||||
|  |             => ValueTask.FromResult(new VexClaimBatch(document, ImmutableArray<VexClaim>.Empty, ImmutableDictionary<string, string>.Empty)); | ||||||
|  |     } | ||||||
|  | } | ||||||
|  |  | ||||||
|  | internal static class HttpResponseMessageExtensions | ||||||
|  | { | ||||||
|  |     public static HttpResponseMessage Clone(this HttpResponseMessage response) | ||||||
|  |     { | ||||||
|  |         var clone = new HttpResponseMessage(response.StatusCode); | ||||||
|  |         foreach (var header in response.Headers) | ||||||
|  |         { | ||||||
|  |             clone.Headers.TryAddWithoutValidation(header.Key, header.Value); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         if (response.Content is not null) | ||||||
|  |         { | ||||||
|  |             var payload = response.Content.ReadAsStringAsync().GetAwaiter().GetResult(); | ||||||
|  |             clone.Content = new StringContent(payload, Encoding.UTF8, response.Content.Headers.ContentType?.MediaType); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         return clone; | ||||||
|  |     } | ||||||
|  | } | ||||||
| @@ -0,0 +1,149 @@ | |||||||
|  | using System.Net; | ||||||
|  | using System.Net.Http; | ||||||
|  | using System.Text; | ||||||
|  | using FluentAssertions; | ||||||
|  | using Microsoft.Extensions.Caching.Memory; | ||||||
|  | using Microsoft.Extensions.Logging.Abstractions; | ||||||
|  | using Microsoft.Extensions.Options; | ||||||
|  | using StellaOps.Vexer.Connectors.Cisco.CSAF.Configuration; | ||||||
|  | using StellaOps.Vexer.Connectors.Cisco.CSAF.Metadata; | ||||||
|  | using StellaOps.Vexer.Core; | ||||||
|  | using System.IO.Abstractions.TestingHelpers; | ||||||
|  |  | ||||||
|  | namespace StellaOps.Vexer.Connectors.Cisco.CSAF.Tests.Metadata; | ||||||
|  |  | ||||||
|  | public sealed class CiscoProviderMetadataLoaderTests | ||||||
|  | { | ||||||
|  |     [Fact] | ||||||
|  |     public async Task LoadAsync_FetchesFromNetworkWithBearerToken() | ||||||
|  |     { | ||||||
|  |         var payload = """ | ||||||
|  |         { | ||||||
|  |           "metadata": { | ||||||
|  |             "publisher": { | ||||||
|  |               "name": "Cisco CSAF", | ||||||
|  |               "category": "vendor", | ||||||
|  |               "contact_details": { | ||||||
|  |                 "id": "vexer:cisco" | ||||||
|  |               } | ||||||
|  |             } | ||||||
|  |           }, | ||||||
|  |           "distributions": { | ||||||
|  |             "directories": [ | ||||||
|  |               "https://api.security.cisco.com/csaf/v2/advisories/" | ||||||
|  |             ] | ||||||
|  |           }, | ||||||
|  |           "discovery": { | ||||||
|  |             "well_known": "https://api.security.cisco.com/.well-known/csaf/provider-metadata.json", | ||||||
|  |             "rolie": "https://api.security.cisco.com/csaf/rolie/feed" | ||||||
|  |           }, | ||||||
|  |           "trust": { | ||||||
|  |             "weight": 0.9, | ||||||
|  |             "cosign": { | ||||||
|  |               "issuer": "https://oidc.security.cisco.com", | ||||||
|  |               "identity_pattern": "spiffe://cisco/*" | ||||||
|  |             }, | ||||||
|  |             "pgp_fingerprints": [ "1234ABCD" ] | ||||||
|  |           } | ||||||
|  |         } | ||||||
|  |         """; | ||||||
|  |  | ||||||
|  |         HttpRequestMessage? capturedRequest = null; | ||||||
|  |         var handler = new FakeHttpMessageHandler(request => | ||||||
|  |         { | ||||||
|  |             capturedRequest = request; | ||||||
|  |             return new HttpResponseMessage(HttpStatusCode.OK) | ||||||
|  |             { | ||||||
|  |                 Content = new StringContent(payload, Encoding.UTF8, "application/json"), | ||||||
|  |                 Headers = { ETag = new System.Net.Http.Headers.EntityTagHeaderValue("\"etag1\"") } | ||||||
|  |             }; | ||||||
|  |         }); | ||||||
|  |  | ||||||
|  |         var httpClient = new HttpClient(handler); | ||||||
|  |         var factory = new SingleHttpClientFactory(httpClient); | ||||||
|  |         var cache = new MemoryCache(new MemoryCacheOptions()); | ||||||
|  |         var options = Options.Create(new CiscoConnectorOptions | ||||||
|  |         { | ||||||
|  |             ApiToken = "token-123", | ||||||
|  |             MetadataUri = "https://api.security.cisco.com/.well-known/csaf/provider-metadata.json", | ||||||
|  |         }); | ||||||
|  |         var fileSystem = new MockFileSystem(); | ||||||
|  |         var loader = new CiscoProviderMetadataLoader(factory, cache, options, NullLogger<CiscoProviderMetadataLoader>.Instance, fileSystem); | ||||||
|  |  | ||||||
|  |         var result = await loader.LoadAsync(CancellationToken.None); | ||||||
|  |  | ||||||
|  |         result.Provider.Id.Should().Be("vexer:cisco"); | ||||||
|  |         result.Provider.BaseUris.Should().ContainSingle(uri => uri.ToString() == "https://api.security.cisco.com/csaf/v2/advisories/"); | ||||||
|  |         result.Provider.Discovery.RolIeService.Should().Be(new Uri("https://api.security.cisco.com/csaf/rolie/feed")); | ||||||
|  |         result.ServedFromCache.Should().BeFalse(); | ||||||
|  |         capturedRequest.Should().NotBeNull(); | ||||||
|  |         capturedRequest!.Headers.Authorization.Should().NotBeNull(); | ||||||
|  |         capturedRequest.Headers.Authorization!.Parameter.Should().Be("token-123"); | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     [Fact] | ||||||
|  |     public async Task LoadAsync_FallsBackToOfflineSnapshot() | ||||||
|  |     { | ||||||
|  |         var payload = """ | ||||||
|  |         { | ||||||
|  |           "metadata": { | ||||||
|  |             "publisher": { | ||||||
|  |               "name": "Cisco CSAF", | ||||||
|  |               "category": "vendor", | ||||||
|  |               "contact_details": { | ||||||
|  |                 "id": "vexer:cisco" | ||||||
|  |               } | ||||||
|  |             } | ||||||
|  |           } | ||||||
|  |         } | ||||||
|  |         """; | ||||||
|  |  | ||||||
|  |         var handler = new FakeHttpMessageHandler(_ => new HttpResponseMessage(HttpStatusCode.InternalServerError)); | ||||||
|  |         var httpClient = new HttpClient(handler); | ||||||
|  |         var factory = new SingleHttpClientFactory(httpClient); | ||||||
|  |         var cache = new MemoryCache(new MemoryCacheOptions()); | ||||||
|  |         var options = Options.Create(new CiscoConnectorOptions | ||||||
|  |         { | ||||||
|  |             MetadataUri = "https://api.security.cisco.com/.well-known/csaf/provider-metadata.json", | ||||||
|  |             PreferOfflineSnapshot = true, | ||||||
|  |             OfflineSnapshotPath = "/snapshots/cisco.json", | ||||||
|  |         }); | ||||||
|  |         var fileSystem = new MockFileSystem(new Dictionary<string, MockFileData> | ||||||
|  |         { | ||||||
|  |             ["/snapshots/cisco.json"] = new MockFileData(payload), | ||||||
|  |         }); | ||||||
|  |         var loader = new CiscoProviderMetadataLoader(factory, cache, options, NullLogger<CiscoProviderMetadataLoader>.Instance, fileSystem); | ||||||
|  |  | ||||||
|  |         var result = await loader.LoadAsync(CancellationToken.None); | ||||||
|  |  | ||||||
|  |         result.FromOfflineSnapshot.Should().BeTrue(); | ||||||
|  |         result.Provider.Id.Should().Be("vexer:cisco"); | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     private sealed class SingleHttpClientFactory : IHttpClientFactory | ||||||
|  |     { | ||||||
|  |         private readonly HttpClient _client; | ||||||
|  |  | ||||||
|  |         public SingleHttpClientFactory(HttpClient client) | ||||||
|  |         { | ||||||
|  |             _client = client; | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         public HttpClient CreateClient(string name) => _client; | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     private sealed class FakeHttpMessageHandler : HttpMessageHandler | ||||||
|  |     { | ||||||
|  |         private readonly Func<HttpRequestMessage, HttpResponseMessage> _responder; | ||||||
|  |  | ||||||
|  |         public FakeHttpMessageHandler(Func<HttpRequestMessage, HttpResponseMessage> responder) | ||||||
|  |         { | ||||||
|  |             _responder = responder; | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         protected override Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) | ||||||
|  |         { | ||||||
|  |             return Task.FromResult(_responder(request)); | ||||||
|  |         } | ||||||
|  |     } | ||||||
|  | } | ||||||
| @@ -0,0 +1,16 @@ | |||||||
|  | <Project Sdk="Microsoft.NET.Sdk"> | ||||||
|  |   <PropertyGroup> | ||||||
|  |     <TargetFramework>net10.0</TargetFramework> | ||||||
|  |     <LangVersion>preview</LangVersion> | ||||||
|  |     <Nullable>enable</Nullable> | ||||||
|  |     <ImplicitUsings>enable</ImplicitUsings> | ||||||
|  |     <TreatWarningsAsErrors>true</TreatWarningsAsErrors> | ||||||
|  |   </PropertyGroup> | ||||||
|  |   <ItemGroup> | ||||||
|  |     <PackageReference Include="FluentAssertions" Version="6.12.0" /> | ||||||
|  |     <PackageReference Include="System.IO.Abstractions.TestingHelpers" Version="20.0.28" /> | ||||||
|  |   </ItemGroup> | ||||||
|  |   <ItemGroup> | ||||||
|  |     <ProjectReference Include="..\StellaOps.Vexer.Connectors.Cisco.CSAF\StellaOps.Vexer.Connectors.Cisco.CSAF.csproj" /> | ||||||
|  |   </ItemGroup> | ||||||
|  | </Project> | ||||||
							
								
								
									
										247
									
								
								src/StellaOps.Vexer.Connectors.Cisco.CSAF/CiscoCsafConnector.cs
									
									
									
									
									
										Normal file
									
								
							
							
						
						
									
										247
									
								
								src/StellaOps.Vexer.Connectors.Cisco.CSAF/CiscoCsafConnector.cs
									
									
									
									
									
										Normal file
									
								
							| @@ -0,0 +1,247 @@ | |||||||
|  | using System.Collections.Generic; | ||||||
|  | using System.Collections.Immutable; | ||||||
|  | using System.Linq; | ||||||
|  | using System.Net.Http; | ||||||
|  | using System.Runtime.CompilerServices; | ||||||
|  | using System.Text.Json; | ||||||
|  | using Microsoft.Extensions.Logging; | ||||||
|  | using StellaOps.Vexer.Connectors.Abstractions; | ||||||
|  | using StellaOps.Vexer.Connectors.Cisco.CSAF.Configuration; | ||||||
|  | using StellaOps.Vexer.Connectors.Cisco.CSAF.Metadata; | ||||||
|  | using StellaOps.Vexer.Core; | ||||||
|  | using StellaOps.Vexer.Storage.Mongo; | ||||||
|  |  | ||||||
|  | namespace StellaOps.Vexer.Connectors.Cisco.CSAF; | ||||||
|  |  | ||||||
|  | public sealed class CiscoCsafConnector : VexConnectorBase | ||||||
|  | { | ||||||
|  |     private static readonly VexConnectorDescriptor DescriptorInstance = new( | ||||||
|  |         id: "vexer:cisco", | ||||||
|  |         kind: VexProviderKind.Vendor, | ||||||
|  |         displayName: "Cisco CSAF") | ||||||
|  |     { | ||||||
|  |         Tags = ImmutableArray.Create("cisco", "csaf"), | ||||||
|  |     }; | ||||||
|  |  | ||||||
|  |     private readonly CiscoProviderMetadataLoader _metadataLoader; | ||||||
|  |     private readonly IHttpClientFactory _httpClientFactory; | ||||||
|  |     private readonly IVexConnectorStateRepository _stateRepository; | ||||||
|  |     private readonly IEnumerable<IVexConnectorOptionsValidator<CiscoConnectorOptions>> _validators; | ||||||
|  |     private readonly JsonSerializerOptions _serializerOptions = new(JsonSerializerDefaults.Web); | ||||||
|  |  | ||||||
|  |     private CiscoConnectorOptions? _options; | ||||||
|  |     private CiscoProviderMetadataResult? _providerMetadata; | ||||||
|  |  | ||||||
|  |     public CiscoCsafConnector( | ||||||
|  |         CiscoProviderMetadataLoader metadataLoader, | ||||||
|  |         IHttpClientFactory httpClientFactory, | ||||||
|  |         IVexConnectorStateRepository stateRepository, | ||||||
|  |         IEnumerable<IVexConnectorOptionsValidator<CiscoConnectorOptions>>? validators, | ||||||
|  |         ILogger<CiscoCsafConnector> logger, | ||||||
|  |         TimeProvider timeProvider) | ||||||
|  |         : base(DescriptorInstance, logger, timeProvider) | ||||||
|  |     { | ||||||
|  |         _metadataLoader = metadataLoader ?? throw new ArgumentNullException(nameof(metadataLoader)); | ||||||
|  |         _httpClientFactory = httpClientFactory ?? throw new ArgumentNullException(nameof(httpClientFactory)); | ||||||
|  |         _stateRepository = stateRepository ?? throw new ArgumentNullException(nameof(stateRepository)); | ||||||
|  |         _validators = validators ?? Array.Empty<IVexConnectorOptionsValidator<CiscoConnectorOptions>>(); | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     public override async ValueTask ValidateAsync(VexConnectorSettings settings, CancellationToken cancellationToken) | ||||||
|  |     { | ||||||
|  |         _options = VexConnectorOptionsBinder.Bind( | ||||||
|  |             Descriptor, | ||||||
|  |             settings, | ||||||
|  |             validators: _validators); | ||||||
|  |  | ||||||
|  |         _providerMetadata = await _metadataLoader.LoadAsync(cancellationToken).ConfigureAwait(false); | ||||||
|  |         LogConnectorEvent(LogLevel.Information, "validate", "Cisco CSAF metadata loaded.", new Dictionary<string, object?> | ||||||
|  |         { | ||||||
|  |             ["baseUriCount"] = _providerMetadata.Provider.BaseUris.Length, | ||||||
|  |             ["fromOffline"] = _providerMetadata.FromOfflineSnapshot, | ||||||
|  |         }); | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     public override async IAsyncEnumerable<VexRawDocument> FetchAsync(VexConnectorContext context, [EnumeratorCancellation] CancellationToken cancellationToken) | ||||||
|  |     { | ||||||
|  |         ArgumentNullException.ThrowIfNull(context); | ||||||
|  |  | ||||||
|  |         if (_options is null) | ||||||
|  |         { | ||||||
|  |             throw new InvalidOperationException("Connector must be validated before fetch operations."); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         if (_providerMetadata is null) | ||||||
|  |         { | ||||||
|  |             _providerMetadata = await _metadataLoader.LoadAsync(cancellationToken).ConfigureAwait(false); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         var state = await _stateRepository.GetAsync(Descriptor.Id, cancellationToken).ConfigureAwait(false); | ||||||
|  |         var knownDigests = state?.DocumentDigests ?? ImmutableArray<string>.Empty; | ||||||
|  |         var digestSet = new HashSet<string>(knownDigests, StringComparer.OrdinalIgnoreCase); | ||||||
|  |         var digestList = new List<string>(knownDigests); | ||||||
|  |         var since = context.Since ?? state?.LastUpdated ?? DateTimeOffset.MinValue; | ||||||
|  |         var latestTimestamp = state?.LastUpdated ?? since; | ||||||
|  |         var stateChanged = false; | ||||||
|  |  | ||||||
|  |         var client = _httpClientFactory.CreateClient(CiscoConnectorOptions.HttpClientName); | ||||||
|  |         foreach (var directory in _providerMetadata.Provider.BaseUris) | ||||||
|  |         { | ||||||
|  |             await foreach (var advisory in EnumerateCatalogAsync(client, directory, cancellationToken).ConfigureAwait(false)) | ||||||
|  |             { | ||||||
|  |                 var published = advisory.LastModified ?? advisory.Published ?? DateTimeOffset.MinValue; | ||||||
|  |                 if (published <= since) | ||||||
|  |                 { | ||||||
|  |                     continue; | ||||||
|  |                 } | ||||||
|  |  | ||||||
|  |                 using var contentResponse = await client.GetAsync(advisory.DocumentUri, HttpCompletionOption.ResponseHeadersRead, cancellationToken).ConfigureAwait(false); | ||||||
|  |                 contentResponse.EnsureSuccessStatusCode(); | ||||||
|  |                 var payload = await contentResponse.Content.ReadAsByteArrayAsync(cancellationToken).ConfigureAwait(false); | ||||||
|  |  | ||||||
|  |                 var rawDocument = CreateRawDocument( | ||||||
|  |                     VexDocumentFormat.Csaf, | ||||||
|  |                     advisory.DocumentUri, | ||||||
|  |                     payload, | ||||||
|  |                     BuildMetadata(builder => builder | ||||||
|  |                         .Add("cisco.csaf.advisoryId", advisory.Id) | ||||||
|  |                         .Add("cisco.csaf.revision", advisory.Revision) | ||||||
|  |                         .Add("cisco.csaf.published", advisory.Published?.ToString("O")) | ||||||
|  |                         .Add("cisco.csaf.modified", advisory.LastModified?.ToString("O")) | ||||||
|  |                         .Add("cisco.csaf.sha256", advisory.Sha256))); | ||||||
|  |  | ||||||
|  |                 if (!digestSet.Add(rawDocument.Digest)) | ||||||
|  |                 { | ||||||
|  |                     continue; | ||||||
|  |                 } | ||||||
|  |  | ||||||
|  |                 await context.RawSink.StoreAsync(rawDocument, cancellationToken).ConfigureAwait(false); | ||||||
|  |                 digestList.Add(rawDocument.Digest); | ||||||
|  |                 stateChanged = true; | ||||||
|  |                 if (published > latestTimestamp) | ||||||
|  |                 { | ||||||
|  |                     latestTimestamp = published; | ||||||
|  |                 } | ||||||
|  |  | ||||||
|  |                 yield return rawDocument; | ||||||
|  |             } | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         if (stateChanged) | ||||||
|  |         { | ||||||
|  |             var newState = new VexConnectorState( | ||||||
|  |                 Descriptor.Id, | ||||||
|  |                 latestTimestamp == DateTimeOffset.MinValue ? state?.LastUpdated : latestTimestamp, | ||||||
|  |                 digestList.ToImmutableArray()); | ||||||
|  |             await _stateRepository.SaveAsync(newState, cancellationToken).ConfigureAwait(false); | ||||||
|  |         } | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     public override ValueTask<VexClaimBatch> NormalizeAsync(VexRawDocument document, CancellationToken cancellationToken) | ||||||
|  |         => throw new NotSupportedException("CiscoCsafConnector relies on CSAF normalizers for document processing."); | ||||||
|  |  | ||||||
|  |     private async IAsyncEnumerable<CiscoAdvisoryEntry> EnumerateCatalogAsync(HttpClient client, Uri directory, [EnumeratorCancellation] CancellationToken cancellationToken) | ||||||
|  |     { | ||||||
|  |         var nextUri = BuildIndexUri(directory, null); | ||||||
|  |         while (nextUri is not null) | ||||||
|  |         { | ||||||
|  |             using var response = await client.GetAsync(nextUri, HttpCompletionOption.ResponseHeadersRead, cancellationToken).ConfigureAwait(false); | ||||||
|  |             response.EnsureSuccessStatusCode(); | ||||||
|  |             var json = await response.Content.ReadAsStringAsync(cancellationToken).ConfigureAwait(false); | ||||||
|  |             var page = JsonSerializer.Deserialize<CiscoAdvisoryIndex>(json, _serializerOptions); | ||||||
|  |             if (page?.Advisories is null) | ||||||
|  |             { | ||||||
|  |                 yield break; | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             foreach (var advisory in page.Advisories) | ||||||
|  |             { | ||||||
|  |                 if (string.IsNullOrWhiteSpace(advisory.Url)) | ||||||
|  |                 { | ||||||
|  |                     continue; | ||||||
|  |                 } | ||||||
|  |  | ||||||
|  |                 if (!Uri.TryCreate(advisory.Url, UriKind.RelativeOrAbsolute, out var documentUri)) | ||||||
|  |                 { | ||||||
|  |                     continue; | ||||||
|  |                 } | ||||||
|  |  | ||||||
|  |                 if (!documentUri.IsAbsoluteUri) | ||||||
|  |                 { | ||||||
|  |                     documentUri = new Uri(directory, documentUri); | ||||||
|  |                 } | ||||||
|  |  | ||||||
|  |                 yield return new CiscoAdvisoryEntry( | ||||||
|  |                     advisory.Id ?? documentUri.Segments.LastOrDefault()?.Trim('/') ?? documentUri.ToString(), | ||||||
|  |                     documentUri, | ||||||
|  |                     advisory.Revision, | ||||||
|  |                     advisory.Published, | ||||||
|  |                     advisory.LastModified, | ||||||
|  |                     advisory.Sha256); | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             nextUri = ResolveNextUri(directory, page.Next); | ||||||
|  |         } | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     private static Uri BuildIndexUri(Uri directory, string? relative) | ||||||
|  |     { | ||||||
|  |         if (string.IsNullOrWhiteSpace(relative)) | ||||||
|  |         { | ||||||
|  |             var baseText = directory.ToString(); | ||||||
|  |             if (!baseText.EndsWith('/')) | ||||||
|  |             { | ||||||
|  |                 baseText += "/"; | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             return new Uri(new Uri(baseText, UriKind.Absolute), "index.json"); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         if (Uri.TryCreate(relative, UriKind.Absolute, out var absolute)) | ||||||
|  |         { | ||||||
|  |             return absolute; | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         var baseTextRelative = directory.ToString(); | ||||||
|  |         if (!baseTextRelative.EndsWith('/')) | ||||||
|  |         { | ||||||
|  |             baseTextRelative += "/"; | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         return new Uri(new Uri(baseTextRelative, UriKind.Absolute), relative); | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     private static Uri? ResolveNextUri(Uri directory, string? next) | ||||||
|  |     { | ||||||
|  |         if (string.IsNullOrWhiteSpace(next)) | ||||||
|  |         { | ||||||
|  |             return null; | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         return BuildIndexUri(directory, next); | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     private sealed record CiscoAdvisoryIndex | ||||||
|  |     { | ||||||
|  |         public List<CiscoAdvisory>? Advisories { get; init; } | ||||||
|  |         public string? Next { get; init; } | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     private sealed record CiscoAdvisory | ||||||
|  |     { | ||||||
|  |         public string? Id { get; init; } | ||||||
|  |         public string? Url { get; init; } | ||||||
|  |         public string? Revision { get; init; } | ||||||
|  |         public DateTimeOffset? Published { get; init; } | ||||||
|  |         public DateTimeOffset? LastModified { get; init; } | ||||||
|  |         public string? Sha256 { get; init; } | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     private sealed record CiscoAdvisoryEntry( | ||||||
|  |         string Id, | ||||||
|  |         Uri DocumentUri, | ||||||
|  |         string? Revision, | ||||||
|  |         DateTimeOffset? Published, | ||||||
|  |         DateTimeOffset? LastModified, | ||||||
|  |         string? Sha256); | ||||||
|  | } | ||||||
| @@ -0,0 +1,58 @@ | |||||||
|  | using System.ComponentModel.DataAnnotations; | ||||||
|  |  | ||||||
|  | namespace StellaOps.Vexer.Connectors.Cisco.CSAF.Configuration; | ||||||
|  |  | ||||||
|  | public sealed class CiscoConnectorOptions : IValidatableObject | ||||||
|  | { | ||||||
|  |     public const string HttpClientName = "cisco-csaf"; | ||||||
|  |  | ||||||
|  |     /// <summary> | ||||||
|  |     /// Endpoint for Cisco CSAF provider metadata discovery. | ||||||
|  |     /// </summary> | ||||||
|  |     [Required] | ||||||
|  |     public string MetadataUri { get; set; } = "https://api.security.cisco.com/.well-known/csaf/provider-metadata.json"; | ||||||
|  |  | ||||||
|  |     /// <summary> | ||||||
|  |     /// Optional bearer token used when Cisco endpoints require authentication. | ||||||
|  |     /// </summary> | ||||||
|  |     public string? ApiToken { get; set; } | ||||||
|  |  | ||||||
|  |     /// <summary> | ||||||
|  |     /// How long provider metadata remains cached. | ||||||
|  |     /// </summary> | ||||||
|  |     public TimeSpan MetadataCacheDuration { get; set; } = TimeSpan.FromHours(6); | ||||||
|  |  | ||||||
|  |     /// <summary> | ||||||
|  |     /// Whether to prefer offline snapshots when fetching metadata. | ||||||
|  |     /// </summary> | ||||||
|  |     public bool PreferOfflineSnapshot { get; set; } | ||||||
|  |  | ||||||
|  |     /// <summary> | ||||||
|  |     /// When set, provider metadata will be persisted to the given file path. | ||||||
|  |     /// </summary> | ||||||
|  |     public bool PersistOfflineSnapshot { get; set; } | ||||||
|  |  | ||||||
|  |     public string? OfflineSnapshotPath { get; set; } | ||||||
|  |  | ||||||
|  |     public IEnumerable<ValidationResult> Validate(ValidationContext validationContext) | ||||||
|  |     { | ||||||
|  |         if (string.IsNullOrWhiteSpace(MetadataUri)) | ||||||
|  |         { | ||||||
|  |             yield return new ValidationResult("MetadataUri must be provided.", new[] { nameof(MetadataUri) }); | ||||||
|  |         } | ||||||
|  |         else if (!Uri.TryCreate(MetadataUri, UriKind.Absolute, out _)) | ||||||
|  |         { | ||||||
|  |             yield return new ValidationResult("MetadataUri must be an absolute URI.", new[] { nameof(MetadataUri) }); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         if (MetadataCacheDuration <= TimeSpan.Zero) | ||||||
|  |         { | ||||||
|  |             yield return new ValidationResult("MetadataCacheDuration must be greater than zero.", new[] { nameof(MetadataCacheDuration) }); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         if (PersistOfflineSnapshot && string.IsNullOrWhiteSpace(OfflineSnapshotPath)) | ||||||
|  |         { | ||||||
|  |             yield return new ValidationResult("OfflineSnapshotPath must be provided when PersistOfflineSnapshot is enabled.", new[] { nameof(OfflineSnapshotPath) }); | ||||||
|  |         } | ||||||
|  |     } | ||||||
|  | } | ||||||
| @@ -0,0 +1,25 @@ | |||||||
|  | using System; | ||||||
|  | using System.Collections.Generic; | ||||||
|  | using System.ComponentModel.DataAnnotations; | ||||||
|  | using StellaOps.Vexer.Connectors.Abstractions; | ||||||
|  |  | ||||||
|  | namespace StellaOps.Vexer.Connectors.Cisco.CSAF.Configuration; | ||||||
|  |  | ||||||
|  | public sealed class CiscoConnectorOptionsValidator : IVexConnectorOptionsValidator<CiscoConnectorOptions> | ||||||
|  | { | ||||||
|  |     public void Validate(VexConnectorDescriptor descriptor, CiscoConnectorOptions options, IList<string> errors) | ||||||
|  |     { | ||||||
|  |         ArgumentNullException.ThrowIfNull(descriptor); | ||||||
|  |         ArgumentNullException.ThrowIfNull(options); | ||||||
|  |         ArgumentNullException.ThrowIfNull(errors); | ||||||
|  |  | ||||||
|  |         var validationResults = new List<ValidationResult>(); | ||||||
|  |         if (!Validator.TryValidateObject(options, new ValidationContext(options), validationResults, validateAllProperties: true)) | ||||||
|  |         { | ||||||
|  |             foreach (var result in validationResults) | ||||||
|  |             { | ||||||
|  |                 errors.Add(result.ErrorMessage ?? "Cisco connector options validation failed."); | ||||||
|  |             } | ||||||
|  |         } | ||||||
|  |     } | ||||||
|  | } | ||||||
| @@ -0,0 +1,52 @@ | |||||||
|  | using System.ComponentModel.DataAnnotations; | ||||||
|  | using System.Net.Http.Headers; | ||||||
|  | using Microsoft.Extensions.Caching.Memory; | ||||||
|  | using Microsoft.Extensions.DependencyInjection; | ||||||
|  | using Microsoft.Extensions.DependencyInjection.Extensions; | ||||||
|  | using Microsoft.Extensions.Options; | ||||||
|  | using StellaOps.Vexer.Connectors.Cisco.CSAF.Configuration; | ||||||
|  | using StellaOps.Vexer.Connectors.Cisco.CSAF.Metadata; | ||||||
|  | using StellaOps.Vexer.Connectors.Abstractions; | ||||||
|  | using StellaOps.Vexer.Core; | ||||||
|  | using System.IO.Abstractions; | ||||||
|  |  | ||||||
|  | namespace StellaOps.Vexer.Connectors.Cisco.CSAF.DependencyInjection; | ||||||
|  |  | ||||||
|  | public static class CiscoConnectorServiceCollectionExtensions | ||||||
|  | { | ||||||
|  |     public static IServiceCollection AddCiscoCsafConnector(this IServiceCollection services, Action<CiscoConnectorOptions>? configure = null) | ||||||
|  |     { | ||||||
|  |         ArgumentNullException.ThrowIfNull(services); | ||||||
|  |  | ||||||
|  |         services.AddOptions<CiscoConnectorOptions>() | ||||||
|  |             .Configure(options => | ||||||
|  |             { | ||||||
|  |                 configure?.Invoke(options); | ||||||
|  |             }) | ||||||
|  |             .PostConfigure(options => | ||||||
|  |             { | ||||||
|  |                 Validator.ValidateObject(options, new ValidationContext(options), validateAllProperties: true); | ||||||
|  |             }); | ||||||
|  |  | ||||||
|  |         services.TryAddSingleton<IMemoryCache, MemoryCache>(); | ||||||
|  |         services.TryAddSingleton<IFileSystem, FileSystem>(); | ||||||
|  |         services.TryAddEnumerable(ServiceDescriptor.Singleton<IVexConnectorOptionsValidator<CiscoConnectorOptions>, CiscoConnectorOptionsValidator>()); | ||||||
|  |  | ||||||
|  |         services.AddHttpClient(CiscoConnectorOptions.HttpClientName) | ||||||
|  |             .ConfigureHttpClient((provider, client) => | ||||||
|  |             { | ||||||
|  |                 var options = provider.GetRequiredService<IOptions<CiscoConnectorOptions>>().Value; | ||||||
|  |                 client.Timeout = TimeSpan.FromSeconds(30); | ||||||
|  |                 client.DefaultRequestHeaders.Accept.ParseAdd("application/json"); | ||||||
|  |                 if (!string.IsNullOrWhiteSpace(options.ApiToken)) | ||||||
|  |                 { | ||||||
|  |                     client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer", options.ApiToken); | ||||||
|  |                 } | ||||||
|  |             }); | ||||||
|  |  | ||||||
|  |         services.AddSingleton<CiscoProviderMetadataLoader>(); | ||||||
|  |         services.AddSingleton<IVexConnector, CiscoCsafConnector>(); | ||||||
|  |  | ||||||
|  |         return services; | ||||||
|  |     } | ||||||
|  | } | ||||||
| @@ -0,0 +1,332 @@ | |||||||
|  | using System.Collections.Immutable; | ||||||
|  | using System.Net; | ||||||
|  | using System.Net.Http.Headers; | ||||||
|  | using System.Text.Json; | ||||||
|  | using Microsoft.Extensions.Caching.Memory; | ||||||
|  | using Microsoft.Extensions.Logging; | ||||||
|  | using Microsoft.Extensions.Options; | ||||||
|  | using StellaOps.Vexer.Connectors.Cisco.CSAF.Configuration; | ||||||
|  | using StellaOps.Vexer.Core; | ||||||
|  | using System.IO.Abstractions; | ||||||
|  |  | ||||||
|  | namespace StellaOps.Vexer.Connectors.Cisco.CSAF.Metadata; | ||||||
|  |  | ||||||
|  | public sealed class CiscoProviderMetadataLoader | ||||||
|  | { | ||||||
|  |     public const string CacheKey = "StellaOps.Vexer.Connectors.Cisco.CSAF.Metadata"; | ||||||
|  |  | ||||||
|  |     private readonly IHttpClientFactory _httpClientFactory; | ||||||
|  |     private readonly IMemoryCache _memoryCache; | ||||||
|  |     private readonly ILogger<CiscoProviderMetadataLoader> _logger; | ||||||
|  |     private readonly CiscoConnectorOptions _options; | ||||||
|  |     private readonly IFileSystem _fileSystem; | ||||||
|  |     private readonly JsonSerializerOptions _serializerOptions; | ||||||
|  |     private readonly SemaphoreSlim _semaphore = new(1, 1); | ||||||
|  |  | ||||||
|  |     public CiscoProviderMetadataLoader( | ||||||
|  |         IHttpClientFactory httpClientFactory, | ||||||
|  |         IMemoryCache memoryCache, | ||||||
|  |         IOptions<CiscoConnectorOptions> options, | ||||||
|  |         ILogger<CiscoProviderMetadataLoader> logger, | ||||||
|  |         IFileSystem? fileSystem = null) | ||||||
|  |     { | ||||||
|  |         _httpClientFactory = httpClientFactory ?? throw new ArgumentNullException(nameof(httpClientFactory)); | ||||||
|  |         _memoryCache = memoryCache ?? throw new ArgumentNullException(nameof(memoryCache)); | ||||||
|  |         _logger = logger ?? throw new ArgumentNullException(nameof(logger)); | ||||||
|  |         ArgumentNullException.ThrowIfNull(options); | ||||||
|  |         _options = options.Value ?? throw new ArgumentNullException(nameof(options)); | ||||||
|  |         _fileSystem = fileSystem ?? new FileSystem(); | ||||||
|  |         _serializerOptions = new JsonSerializerOptions(JsonSerializerDefaults.Web) | ||||||
|  |         { | ||||||
|  |             PropertyNameCaseInsensitive = true, | ||||||
|  |             ReadCommentHandling = JsonCommentHandling.Skip, | ||||||
|  |         }; | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     public async Task<CiscoProviderMetadataResult> LoadAsync(CancellationToken cancellationToken) | ||||||
|  |     { | ||||||
|  |         if (_memoryCache.TryGetValue<CacheEntry>(CacheKey, out var cached) && cached is not null && !cached.IsExpired()) | ||||||
|  |         { | ||||||
|  |             _logger.LogDebug("Returning cached Cisco provider metadata (expires {Expires}).", cached.ExpiresAt); | ||||||
|  |             return new CiscoProviderMetadataResult(cached.Provider, cached.FetchedAt, cached.FromOffline, true); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         await _semaphore.WaitAsync(cancellationToken).ConfigureAwait(false); | ||||||
|  |         try | ||||||
|  |         { | ||||||
|  |             if (_memoryCache.TryGetValue<CacheEntry>(CacheKey, out cached) && cached is not null && !cached.IsExpired()) | ||||||
|  |             { | ||||||
|  |                 return new CiscoProviderMetadataResult(cached.Provider, cached.FetchedAt, cached.FromOffline, true); | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             CacheEntry? previous = cached; | ||||||
|  |  | ||||||
|  |             if (!_options.PreferOfflineSnapshot) | ||||||
|  |             { | ||||||
|  |                 var network = await TryFetchFromNetworkAsync(previous, cancellationToken).ConfigureAwait(false); | ||||||
|  |                 if (network is not null) | ||||||
|  |                 { | ||||||
|  |                     StoreCache(network); | ||||||
|  |                     return new CiscoProviderMetadataResult(network.Provider, network.FetchedAt, false, false); | ||||||
|  |                 } | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             var offline = TryLoadFromOffline(); | ||||||
|  |             if (offline is not null) | ||||||
|  |             { | ||||||
|  |                 var entry = offline with | ||||||
|  |                 { | ||||||
|  |                     FetchedAt = DateTimeOffset.UtcNow, | ||||||
|  |                     ExpiresAt = DateTimeOffset.UtcNow + _options.MetadataCacheDuration, | ||||||
|  |                     FromOffline = true, | ||||||
|  |                 }; | ||||||
|  |                 StoreCache(entry); | ||||||
|  |                 return new CiscoProviderMetadataResult(entry.Provider, entry.FetchedAt, true, false); | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             throw new InvalidOperationException("Unable to load Cisco CSAF provider metadata from network or offline snapshot."); | ||||||
|  |         } | ||||||
|  |         finally | ||||||
|  |         { | ||||||
|  |             _semaphore.Release(); | ||||||
|  |         } | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     private async Task<CacheEntry?> TryFetchFromNetworkAsync(CacheEntry? previous, CancellationToken cancellationToken) | ||||||
|  |     { | ||||||
|  |         try | ||||||
|  |         { | ||||||
|  |             var client = _httpClientFactory.CreateClient(CiscoConnectorOptions.HttpClientName); | ||||||
|  |             using var request = new HttpRequestMessage(HttpMethod.Get, _options.MetadataUri); | ||||||
|  |             if (!string.IsNullOrWhiteSpace(_options.ApiToken)) | ||||||
|  |             { | ||||||
|  |                 request.Headers.Authorization = new AuthenticationHeaderValue("Bearer", _options.ApiToken); | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             if (!string.IsNullOrWhiteSpace(previous?.ETag) && EntityTagHeaderValue.TryParse(previous.ETag, out var etag)) | ||||||
|  |             { | ||||||
|  |                 request.Headers.IfNoneMatch.Add(etag); | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             using var response = await client.SendAsync(request, HttpCompletionOption.ResponseHeadersRead, cancellationToken).ConfigureAwait(false); | ||||||
|  |  | ||||||
|  |             if (response.StatusCode == HttpStatusCode.NotModified && previous is not null) | ||||||
|  |             { | ||||||
|  |                 _logger.LogDebug("Cisco provider metadata not modified (etag {ETag}).", previous.ETag); | ||||||
|  |                 return previous with | ||||||
|  |                 { | ||||||
|  |                     FetchedAt = DateTimeOffset.UtcNow, | ||||||
|  |                     ExpiresAt = DateTimeOffset.UtcNow + _options.MetadataCacheDuration, | ||||||
|  |                 }; | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             response.EnsureSuccessStatusCode(); | ||||||
|  |             var payload = await response.Content.ReadAsStringAsync(cancellationToken).ConfigureAwait(false); | ||||||
|  |             var provider = ParseProvider(payload); | ||||||
|  |             var etagHeader = response.Headers.ETag?.ToString(); | ||||||
|  |  | ||||||
|  |             if (_options.PersistOfflineSnapshot && !string.IsNullOrWhiteSpace(_options.OfflineSnapshotPath)) | ||||||
|  |             { | ||||||
|  |                 try | ||||||
|  |                 { | ||||||
|  |                     _fileSystem.File.WriteAllText(_options.OfflineSnapshotPath, payload); | ||||||
|  |                     _logger.LogDebug("Persisted Cisco metadata snapshot to {Path}.", _options.OfflineSnapshotPath); | ||||||
|  |                 } | ||||||
|  |                 catch (Exception ex) | ||||||
|  |                 { | ||||||
|  |                     _logger.LogWarning(ex, "Failed to persist Cisco metadata snapshot to {Path}.", _options.OfflineSnapshotPath); | ||||||
|  |                 } | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             return new CacheEntry( | ||||||
|  |                 provider, | ||||||
|  |                 DateTimeOffset.UtcNow, | ||||||
|  |                 DateTimeOffset.UtcNow + _options.MetadataCacheDuration, | ||||||
|  |                 etagHeader, | ||||||
|  |                 FromOffline: false); | ||||||
|  |         } | ||||||
|  |         catch (Exception ex) when (ex is not OperationCanceledException && !_options.PreferOfflineSnapshot) | ||||||
|  |         { | ||||||
|  |             _logger.LogWarning(ex, "Failed to fetch Cisco provider metadata from {Uri}; falling back to offline snapshot when available.", _options.MetadataUri); | ||||||
|  |             return null; | ||||||
|  |         } | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     private CacheEntry? TryLoadFromOffline() | ||||||
|  |     { | ||||||
|  |         if (string.IsNullOrWhiteSpace(_options.OfflineSnapshotPath)) | ||||||
|  |         { | ||||||
|  |             return null; | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         if (!_fileSystem.File.Exists(_options.OfflineSnapshotPath)) | ||||||
|  |         { | ||||||
|  |             _logger.LogWarning("Cisco offline snapshot path {Path} does not exist.", _options.OfflineSnapshotPath); | ||||||
|  |             return null; | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         try | ||||||
|  |         { | ||||||
|  |             var payload = _fileSystem.File.ReadAllText(_options.OfflineSnapshotPath); | ||||||
|  |             var provider = ParseProvider(payload); | ||||||
|  |             return new CacheEntry(provider, DateTimeOffset.UtcNow, DateTimeOffset.UtcNow + _options.MetadataCacheDuration, null, true); | ||||||
|  |         } | ||||||
|  |         catch (Exception ex) | ||||||
|  |         { | ||||||
|  |             _logger.LogError(ex, "Failed to load Cisco provider metadata from offline snapshot {Path}.", _options.OfflineSnapshotPath); | ||||||
|  |             return null; | ||||||
|  |         } | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     private VexProvider ParseProvider(string payload) | ||||||
|  |     { | ||||||
|  |         if (string.IsNullOrWhiteSpace(payload)) | ||||||
|  |         { | ||||||
|  |             throw new InvalidOperationException("Cisco provider metadata payload was empty."); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         ProviderMetadataDocument? document; | ||||||
|  |         try | ||||||
|  |         { | ||||||
|  |             document = JsonSerializer.Deserialize<ProviderMetadataDocument>(payload, _serializerOptions); | ||||||
|  |         } | ||||||
|  |         catch (JsonException ex) | ||||||
|  |         { | ||||||
|  |             throw new InvalidOperationException("Failed to parse Cisco provider metadata.", ex); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         if (document?.Metadata?.Publisher?.ContactDetails is null || string.IsNullOrWhiteSpace(document.Metadata.Publisher.ContactDetails.Id)) | ||||||
|  |         { | ||||||
|  |             throw new InvalidOperationException("Cisco provider metadata did not include a publisher identifier."); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         var discovery = new VexProviderDiscovery(document.Discovery?.WellKnown, document.Discovery?.RolIe); | ||||||
|  |         var trust = document.Trust is null | ||||||
|  |             ? VexProviderTrust.Default | ||||||
|  |             : new VexProviderTrust( | ||||||
|  |                 document.Trust.Weight ?? 1.0, | ||||||
|  |                 document.Trust.Cosign is null ? null : new VexCosignTrust(document.Trust.Cosign.Issuer ?? string.Empty, document.Trust.Cosign.IdentityPattern ?? string.Empty), | ||||||
|  |                 document.Trust.PgpFingerprints ?? Enumerable.Empty<string>()); | ||||||
|  |  | ||||||
|  |         var directories = document.Distributions?.Directories is null | ||||||
|  |             ? Enumerable.Empty<Uri>() | ||||||
|  |             : document.Distributions.Directories | ||||||
|  |                 .Where(static s => !string.IsNullOrWhiteSpace(s)) | ||||||
|  |                 .Select(static s => Uri.TryCreate(s, UriKind.Absolute, out var uri) ? uri : null) | ||||||
|  |                 .Where(static uri => uri is not null)! | ||||||
|  |                 .Select(static uri => uri!); | ||||||
|  |  | ||||||
|  |         return new VexProvider( | ||||||
|  |             id: document.Metadata.Publisher.ContactDetails.Id, | ||||||
|  |             displayName: document.Metadata.Publisher.Name ?? document.Metadata.Publisher.ContactDetails.Id, | ||||||
|  |             kind: document.Metadata.Publisher.Category?.Equals("vendor", StringComparison.OrdinalIgnoreCase) == true ? VexProviderKind.Vendor : VexProviderKind.Hub, | ||||||
|  |             baseUris: directories, | ||||||
|  |             discovery: discovery, | ||||||
|  |             trust: trust, | ||||||
|  |             enabled: true); | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     private void StoreCache(CacheEntry entry) | ||||||
|  |     { | ||||||
|  |         var options = new MemoryCacheEntryOptions | ||||||
|  |         { | ||||||
|  |             AbsoluteExpiration = entry.ExpiresAt, | ||||||
|  |         }; | ||||||
|  |         _memoryCache.Set(CacheKey, entry, options); | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     private sealed record CacheEntry( | ||||||
|  |         VexProvider Provider, | ||||||
|  |         DateTimeOffset FetchedAt, | ||||||
|  |         DateTimeOffset ExpiresAt, | ||||||
|  |         string? ETag, | ||||||
|  |         bool FromOffline) | ||||||
|  |     { | ||||||
|  |         public bool IsExpired() => DateTimeOffset.UtcNow >= ExpiresAt; | ||||||
|  |     } | ||||||
|  | } | ||||||
|  |  | ||||||
|  | public sealed record CiscoProviderMetadataResult( | ||||||
|  |     VexProvider Provider, | ||||||
|  |     DateTimeOffset FetchedAt, | ||||||
|  |     bool FromOfflineSnapshot, | ||||||
|  |     bool ServedFromCache); | ||||||
|  |  | ||||||
|  | #region document models | ||||||
|  |  | ||||||
|  | internal sealed class ProviderMetadataDocument | ||||||
|  | { | ||||||
|  |     [System.Text.Json.Serialization.JsonPropertyName("metadata")] | ||||||
|  |     public ProviderMetadataMetadata Metadata { get; set; } = new(); | ||||||
|  |  | ||||||
|  |     [System.Text.Json.Serialization.JsonPropertyName("discovery")] | ||||||
|  |     public ProviderMetadataDiscovery? Discovery { get; set; } | ||||||
|  |  | ||||||
|  |     [System.Text.Json.Serialization.JsonPropertyName("trust")] | ||||||
|  |     public ProviderMetadataTrust? Trust { get; set; } | ||||||
|  |  | ||||||
|  |     [System.Text.Json.Serialization.JsonPropertyName("distributions")] | ||||||
|  |     public ProviderMetadataDistributions? Distributions { get; set; } | ||||||
|  | } | ||||||
|  |  | ||||||
|  | internal sealed class ProviderMetadataMetadata | ||||||
|  | { | ||||||
|  |     [System.Text.Json.Serialization.JsonPropertyName("publisher")] | ||||||
|  |     public ProviderMetadataPublisher Publisher { get; set; } = new(); | ||||||
|  | } | ||||||
|  |  | ||||||
|  | internal sealed class ProviderMetadataPublisher | ||||||
|  | { | ||||||
|  |     [System.Text.Json.Serialization.JsonPropertyName("name")] | ||||||
|  |     public string? Name { get; set; } | ||||||
|  |  | ||||||
|  |     [System.Text.Json.Serialization.JsonPropertyName("category")] | ||||||
|  |     public string? Category { get; set; } | ||||||
|  |  | ||||||
|  |     [System.Text.Json.Serialization.JsonPropertyName("contact_details")] | ||||||
|  |     public ProviderMetadataPublisherContact ContactDetails { get; set; } = new(); | ||||||
|  | } | ||||||
|  |  | ||||||
|  | internal sealed class ProviderMetadataPublisherContact | ||||||
|  | { | ||||||
|  |     [System.Text.Json.Serialization.JsonPropertyName("id")] | ||||||
|  |     public string? Id { get; set; } | ||||||
|  | } | ||||||
|  |  | ||||||
|  | internal sealed class ProviderMetadataDiscovery | ||||||
|  | { | ||||||
|  |     [System.Text.Json.Serialization.JsonPropertyName("well_known")] | ||||||
|  |     public Uri? WellKnown { get; set; } | ||||||
|  |  | ||||||
|  |     [System.Text.Json.Serialization.JsonPropertyName("rolie")] | ||||||
|  |     public Uri? RolIe { get; set; } | ||||||
|  | } | ||||||
|  |  | ||||||
|  | internal sealed class ProviderMetadataTrust | ||||||
|  | { | ||||||
|  |     [System.Text.Json.Serialization.JsonPropertyName("weight")] | ||||||
|  |     public double? Weight { get; set; } | ||||||
|  |  | ||||||
|  |     [System.Text.Json.Serialization.JsonPropertyName("cosign")] | ||||||
|  |     public ProviderMetadataTrustCosign? Cosign { get; set; } | ||||||
|  |  | ||||||
|  |     [System.Text.Json.Serialization.JsonPropertyName("pgp_fingerprints")] | ||||||
|  |     public string[]? PgpFingerprints { get; set; } | ||||||
|  | } | ||||||
|  |  | ||||||
|  | internal sealed class ProviderMetadataTrustCosign | ||||||
|  | { | ||||||
|  |     [System.Text.Json.Serialization.JsonPropertyName("issuer")] | ||||||
|  |     public string? Issuer { get; set; } | ||||||
|  |  | ||||||
|  |     [System.Text.Json.Serialization.JsonPropertyName("identity_pattern")] | ||||||
|  |     public string? IdentityPattern { get; set; } | ||||||
|  | } | ||||||
|  |  | ||||||
|  | internal sealed class ProviderMetadataDistributions | ||||||
|  | { | ||||||
|  |     [System.Text.Json.Serialization.JsonPropertyName("directories")] | ||||||
|  |     public string[]? Directories { get; set; } | ||||||
|  | } | ||||||
|  |  | ||||||
|  | #endregion | ||||||
| @@ -0,0 +1,20 @@ | |||||||
|  | <Project Sdk="Microsoft.NET.Sdk"> | ||||||
|  |   <PropertyGroup> | ||||||
|  |     <TargetFramework>net10.0</TargetFramework> | ||||||
|  |     <LangVersion>preview</LangVersion> | ||||||
|  |     <Nullable>enable</Nullable> | ||||||
|  |     <ImplicitUsings>enable</ImplicitUsings> | ||||||
|  |     <TreatWarningsAsErrors>true</TreatWarningsAsErrors> | ||||||
|  |   </PropertyGroup> | ||||||
|  |   <ItemGroup> | ||||||
|  |     <ProjectReference Include="..\StellaOps.Vexer.Connectors.Abstractions\StellaOps.Vexer.Connectors.Abstractions.csproj" /> | ||||||
|  |     <ProjectReference Include="..\StellaOps.Vexer.Core\StellaOps.Vexer.Core.csproj" /> | ||||||
|  |     <ProjectReference Include="..\StellaOps.Vexer.Storage.Mongo\StellaOps.Vexer.Storage.Mongo.csproj" /> | ||||||
|  |   </ItemGroup> | ||||||
|  |   <ItemGroup> | ||||||
|  |     <PackageReference Include="Microsoft.Extensions.Caching.Memory" Version="8.0.0" /> | ||||||
|  |     <PackageReference Include="Microsoft.Extensions.Http" Version="8.0.0" /> | ||||||
|  |     <PackageReference Include="Microsoft.Extensions.Options" Version="8.0.0" /> | ||||||
|  |     <PackageReference Include="System.IO.Abstractions" Version="20.0.28" /> | ||||||
|  |   </ItemGroup> | ||||||
|  | </Project> | ||||||
| @@ -2,6 +2,6 @@ If you are working on this file you need to read docs/ARCHITECTURE_VEXER.md and | |||||||
| # TASKS | # TASKS | ||||||
| | Task | Owner(s) | Depends on | Notes | | | Task | Owner(s) | Depends on | Notes | | ||||||
| |---|---|---|---| | |---|---|---|---| | ||||||
| |VEXER-CONN-CISCO-01-001 – Endpoint discovery & auth plumbing|Team Vexer Connectors – Cisco|VEXER-CONN-ABS-01-001|TODO – Resolve Cisco CSAF collection URLs, configure optional token auth, and validate discovery metadata for offline caching.| | |VEXER-CONN-CISCO-01-001 – Endpoint discovery & auth plumbing|Team Vexer Connectors – Cisco|VEXER-CONN-ABS-01-001|**DONE (2025-10-17)** – Added `CiscoProviderMetadataLoader` with bearer token support, offline snapshot fallback, DI helpers, and tests covering network/offline discovery to unblock subsequent fetch work.| | ||||||
| |VEXER-CONN-CISCO-01-002 – CSAF pull loop & pagination|Team Vexer Connectors – Cisco|VEXER-CONN-CISCO-01-001, VEXER-STORAGE-01-003|TODO – Implement paginated fetch with retries/backoff, checksum validation, and raw document persistence.| | |VEXER-CONN-CISCO-01-002 – CSAF pull loop & pagination|Team Vexer Connectors – Cisco|VEXER-CONN-CISCO-01-001, VEXER-STORAGE-01-003|**DONE (2025-10-17)** – Implemented paginated advisory fetch using provider directories, raw document persistence with dedupe/state tracking, offline resiliency, and unit coverage.| | ||||||
| |VEXER-CONN-CISCO-01-003 – Provider trust metadata|Team Vexer Connectors – Cisco|VEXER-CONN-CISCO-01-002, VEXER-POLICY-01-001|TODO – Emit cosign/PGP trust metadata and advisory provenance hints for policy weighting.| | |VEXER-CONN-CISCO-01-003 – Provider trust metadata|Team Vexer Connectors – Cisco|VEXER-CONN-CISCO-01-002, VEXER-POLICY-01-001|TODO – Emit cosign/PGP trust metadata and advisory provenance hints for policy weighting.| | ||||||
|   | |||||||
| @@ -0,0 +1,176 @@ | |||||||
|  | using System.Net; | ||||||
|  | using System.Net.Http; | ||||||
|  | using System.Text; | ||||||
|  | using FluentAssertions; | ||||||
|  | using Microsoft.Extensions.Caching.Memory; | ||||||
|  | using Microsoft.Extensions.Logging.Abstractions; | ||||||
|  | using Microsoft.Extensions.Options; | ||||||
|  | using NSubstitute; | ||||||
|  | using StellaOps.Vexer.Connectors.MSRC.CSAF.Authentication; | ||||||
|  | using StellaOps.Vexer.Connectors.MSRC.CSAF.Configuration; | ||||||
|  | using System.IO.Abstractions.TestingHelpers; | ||||||
|  | using Xunit; | ||||||
|  |  | ||||||
|  | namespace StellaOps.Vexer.Connectors.MSRC.CSAF.Tests.Authentication; | ||||||
|  |  | ||||||
|  | public sealed class MsrcTokenProviderTests | ||||||
|  | { | ||||||
|  |     [Fact] | ||||||
|  |     public async Task GetAccessTokenAsync_CachesUntilExpiry() | ||||||
|  |     { | ||||||
|  |         var handler = new TestHttpMessageHandler(new[] | ||||||
|  |         { | ||||||
|  |             CreateTokenResponse("token-1"), | ||||||
|  |             CreateTokenResponse("token-2"), | ||||||
|  |         }); | ||||||
|  |         var client = new HttpClient(handler) { BaseAddress = new Uri("https://login.microsoftonline.com/") }; | ||||||
|  |         var factory = new SingleClientHttpClientFactory(client); | ||||||
|  |         var cache = new MemoryCache(new MemoryCacheOptions()); | ||||||
|  |         var fileSystem = new MockFileSystem(); | ||||||
|  |         var options = Options.Create(new MsrcConnectorOptions | ||||||
|  |         { | ||||||
|  |             TenantId = "contoso.onmicrosoft.com", | ||||||
|  |             ClientId = "client-id", | ||||||
|  |             ClientSecret = "secret", | ||||||
|  |         }); | ||||||
|  |  | ||||||
|  |         var timeProvider = new AdjustableTimeProvider(); | ||||||
|  |         var provider = new MsrcTokenProvider(factory, cache, fileSystem, options, NullLogger<MsrcTokenProvider>.Instance, timeProvider); | ||||||
|  |  | ||||||
|  |         var first = await provider.GetAccessTokenAsync(CancellationToken.None); | ||||||
|  |         first.Value.Should().Be("token-1"); | ||||||
|  |         handler.InvocationCount.Should().Be(1); | ||||||
|  |  | ||||||
|  |         var second = await provider.GetAccessTokenAsync(CancellationToken.None); | ||||||
|  |         second.Value.Should().Be("token-1"); | ||||||
|  |         handler.InvocationCount.Should().Be(1); | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     [Fact] | ||||||
|  |     public async Task GetAccessTokenAsync_RefreshesWhenExpired() | ||||||
|  |     { | ||||||
|  |         var handler = new TestHttpMessageHandler(new[] | ||||||
|  |         { | ||||||
|  |             CreateTokenResponse("token-1", expiresIn: 120), | ||||||
|  |             CreateTokenResponse("token-2", expiresIn: 3600), | ||||||
|  |         }); | ||||||
|  |         var client = new HttpClient(handler) { BaseAddress = new Uri("https://login.microsoftonline.com/") }; | ||||||
|  |         var factory = new SingleClientHttpClientFactory(client); | ||||||
|  |         var cache = new MemoryCache(new MemoryCacheOptions()); | ||||||
|  |         var fileSystem = new MockFileSystem(); | ||||||
|  |         var options = Options.Create(new MsrcConnectorOptions | ||||||
|  |         { | ||||||
|  |             TenantId = "contoso.onmicrosoft.com", | ||||||
|  |             ClientId = "client-id", | ||||||
|  |             ClientSecret = "secret", | ||||||
|  |             ExpiryLeewaySeconds = 60, | ||||||
|  |         }); | ||||||
|  |  | ||||||
|  |         var timeProvider = new AdjustableTimeProvider(); | ||||||
|  |         var provider = new MsrcTokenProvider(factory, cache, fileSystem, options, NullLogger<MsrcTokenProvider>.Instance, timeProvider); | ||||||
|  |  | ||||||
|  |         var first = await provider.GetAccessTokenAsync(CancellationToken.None); | ||||||
|  |         first.Value.Should().Be("token-1"); | ||||||
|  |         handler.InvocationCount.Should().Be(1); | ||||||
|  |  | ||||||
|  |         timeProvider.Advance(TimeSpan.FromMinutes(2)); | ||||||
|  |         var second = await provider.GetAccessTokenAsync(CancellationToken.None); | ||||||
|  |         second.Value.Should().Be("token-2"); | ||||||
|  |         handler.InvocationCount.Should().Be(2); | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     [Fact] | ||||||
|  |     public async Task GetAccessTokenAsync_OfflineStaticToken() | ||||||
|  |     { | ||||||
|  |         var factory = Substitute.For<IHttpClientFactory>(); | ||||||
|  |         var cache = new MemoryCache(new MemoryCacheOptions()); | ||||||
|  |         var fileSystem = new MockFileSystem(); | ||||||
|  |         var options = Options.Create(new MsrcConnectorOptions | ||||||
|  |         { | ||||||
|  |             PreferOfflineToken = true, | ||||||
|  |             StaticAccessToken = "offline-token", | ||||||
|  |         }); | ||||||
|  |  | ||||||
|  |         var provider = new MsrcTokenProvider(factory, cache, fileSystem, options, NullLogger<MsrcTokenProvider>.Instance); | ||||||
|  |         var token = await provider.GetAccessTokenAsync(CancellationToken.None); | ||||||
|  |         token.Value.Should().Be("offline-token"); | ||||||
|  |         token.ExpiresAt.Should().Be(DateTimeOffset.MaxValue); | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     [Fact] | ||||||
|  |     public async Task GetAccessTokenAsync_OfflineFileToken() | ||||||
|  |     { | ||||||
|  |         var factory = Substitute.For<IHttpClientFactory>(); | ||||||
|  |         var cache = new MemoryCache(new MemoryCacheOptions()); | ||||||
|  |         var fileSystem = new MockFileSystem(); | ||||||
|  |         var offlinePath = fileSystem.Path.Combine("/tokens", "msrc.txt"); | ||||||
|  |         fileSystem.AddFile(offlinePath, new MockFileData("file-token")); | ||||||
|  |  | ||||||
|  |         var options = Options.Create(new MsrcConnectorOptions | ||||||
|  |         { | ||||||
|  |             PreferOfflineToken = true, | ||||||
|  |             OfflineTokenPath = offlinePath, | ||||||
|  |         }); | ||||||
|  |  | ||||||
|  |         var provider = new MsrcTokenProvider(factory, cache, fileSystem, options, NullLogger<MsrcTokenProvider>.Instance); | ||||||
|  |         var token = await provider.GetAccessTokenAsync(CancellationToken.None); | ||||||
|  |         token.Value.Should().Be("file-token"); | ||||||
|  |         token.ExpiresAt.Should().Be(DateTimeOffset.MaxValue); | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     private static HttpResponseMessage CreateTokenResponse(string token, int expiresIn = 3600) | ||||||
|  |     { | ||||||
|  |         var json = $"{{\"access_token\":\"{token}\",\"token_type\":\"Bearer\",\"expires_in\":{expiresIn}}}"; | ||||||
|  |         return new HttpResponseMessage(HttpStatusCode.OK) | ||||||
|  |         { | ||||||
|  |             Content = new StringContent(json, Encoding.UTF8, "application/json"), | ||||||
|  |         }; | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     private sealed class AdjustableTimeProvider : TimeProvider | ||||||
|  |     { | ||||||
|  |         private DateTimeOffset _now = DateTimeOffset.UtcNow; | ||||||
|  |  | ||||||
|  |         public override DateTimeOffset GetUtcNow() => _now; | ||||||
|  |  | ||||||
|  |         public void Advance(TimeSpan span) => _now = _now.Add(span); | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     private sealed class SingleClientHttpClientFactory : IHttpClientFactory | ||||||
|  |     { | ||||||
|  |         private readonly HttpClient _client; | ||||||
|  |  | ||||||
|  |         public SingleClientHttpClientFactory(HttpClient client) | ||||||
|  |         { | ||||||
|  |             _client = client; | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         public HttpClient CreateClient(string name) => _client; | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     private sealed class TestHttpMessageHandler : HttpMessageHandler | ||||||
|  |     { | ||||||
|  |         private readonly Queue<HttpResponseMessage> _responses; | ||||||
|  |  | ||||||
|  |         public TestHttpMessageHandler(IEnumerable<HttpResponseMessage> responses) | ||||||
|  |         { | ||||||
|  |             _responses = new Queue<HttpResponseMessage>(responses); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         public int InvocationCount { get; private set; } | ||||||
|  |  | ||||||
|  |         protected override Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) | ||||||
|  |         { | ||||||
|  |             InvocationCount++; | ||||||
|  |             if (_responses.Count == 0) | ||||||
|  |             { | ||||||
|  |                 return Task.FromResult(new HttpResponseMessage(HttpStatusCode.InternalServerError) | ||||||
|  |                 { | ||||||
|  |                     Content = new StringContent("no responses remaining"), | ||||||
|  |                 }); | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             return Task.FromResult(_responses.Dequeue()); | ||||||
|  |         } | ||||||
|  |     } | ||||||
|  | } | ||||||
| @@ -0,0 +1,18 @@ | |||||||
|  | <Project Sdk="Microsoft.NET.Sdk"> | ||||||
|  |   <PropertyGroup> | ||||||
|  |     <TargetFramework>net10.0</TargetFramework> | ||||||
|  |     <LangVersion>preview</LangVersion> | ||||||
|  |     <Nullable>enable</Nullable> | ||||||
|  |     <ImplicitUsings>enable</ImplicitUsings> | ||||||
|  |     <TreatWarningsAsErrors>true</TreatWarningsAsErrors> | ||||||
|  |   </PropertyGroup> | ||||||
|  |   <ItemGroup> | ||||||
|  |     <ProjectReference Include="..\StellaOps.Vexer.Connectors.MSRC.CSAF\StellaOps.Vexer.Connectors.MSRC.CSAF.csproj" /> | ||||||
|  |   </ItemGroup> | ||||||
|  |   <ItemGroup> | ||||||
|  |     <PackageReference Include="FluentAssertions" Version="6.12.0" /> | ||||||
|  |     <PackageReference Include="Microsoft.Extensions.Logging.Abstractions" Version="8.0.0" /> | ||||||
|  |     <PackageReference Include="NSubstitute" Version="5.1.0" /> | ||||||
|  |     <PackageReference Include="System.IO.Abstractions.TestingHelpers" Version="20.0.28" /> | ||||||
|  |   </ItemGroup> | ||||||
|  | </Project> | ||||||
| @@ -0,0 +1,185 @@ | |||||||
|  | using System; | ||||||
|  | using System.Collections.Generic; | ||||||
|  | using System.IO.Abstractions; | ||||||
|  | using System.Net.Http; | ||||||
|  | using System.Net.Http.Json; | ||||||
|  | using System.Text.Json.Serialization; | ||||||
|  | using System.Threading; | ||||||
|  | using System.Threading.Tasks; | ||||||
|  | using Microsoft.Extensions.Caching.Memory; | ||||||
|  | using Microsoft.Extensions.Logging; | ||||||
|  | using Microsoft.Extensions.Options; | ||||||
|  | using StellaOps.Vexer.Connectors.MSRC.CSAF.Configuration; | ||||||
|  |  | ||||||
|  | namespace StellaOps.Vexer.Connectors.MSRC.CSAF.Authentication; | ||||||
|  |  | ||||||
|  | public interface IMsrcTokenProvider | ||||||
|  | { | ||||||
|  |     ValueTask<MsrcAccessToken> GetAccessTokenAsync(CancellationToken cancellationToken); | ||||||
|  | } | ||||||
|  |  | ||||||
|  | public sealed class MsrcTokenProvider : IMsrcTokenProvider, IDisposable | ||||||
|  | { | ||||||
|  |     private const string CachePrefix = "StellaOps.Vexer.Connectors.MSRC.CSAF.Token"; | ||||||
|  |  | ||||||
|  |     private readonly IHttpClientFactory _httpClientFactory; | ||||||
|  |     private readonly IMemoryCache _cache; | ||||||
|  |     private readonly IFileSystem _fileSystem; | ||||||
|  |     private readonly ILogger<MsrcTokenProvider> _logger; | ||||||
|  |     private readonly TimeProvider _timeProvider; | ||||||
|  |     private readonly MsrcConnectorOptions _options; | ||||||
|  |     private readonly SemaphoreSlim _refreshLock = new(1, 1); | ||||||
|  |  | ||||||
|  |     public MsrcTokenProvider( | ||||||
|  |         IHttpClientFactory httpClientFactory, | ||||||
|  |         IMemoryCache cache, | ||||||
|  |         IFileSystem fileSystem, | ||||||
|  |         IOptions<MsrcConnectorOptions> options, | ||||||
|  |         ILogger<MsrcTokenProvider> logger, | ||||||
|  |         TimeProvider? timeProvider = null) | ||||||
|  |     { | ||||||
|  |         _httpClientFactory = httpClientFactory ?? throw new ArgumentNullException(nameof(httpClientFactory)); | ||||||
|  |         _cache = cache ?? throw new ArgumentNullException(nameof(cache)); | ||||||
|  |         _fileSystem = fileSystem ?? throw new ArgumentNullException(nameof(fileSystem)); | ||||||
|  |         ArgumentNullException.ThrowIfNull(options); | ||||||
|  |         _options = options.Value ?? throw new ArgumentNullException(nameof(options)); | ||||||
|  |         _options.Validate(_fileSystem); | ||||||
|  |         _logger = logger ?? throw new ArgumentNullException(nameof(logger)); | ||||||
|  |         _timeProvider = timeProvider ?? TimeProvider.System; | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     public async ValueTask<MsrcAccessToken> GetAccessTokenAsync(CancellationToken cancellationToken) | ||||||
|  |     { | ||||||
|  |         if (_options.PreferOfflineToken) | ||||||
|  |         { | ||||||
|  |             return LoadOfflineToken(); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         var cacheKey = CreateCacheKey(); | ||||||
|  |         if (_cache.TryGetValue<MsrcAccessToken>(cacheKey, out var cachedToken) && | ||||||
|  |             cachedToken is not null && | ||||||
|  |             !cachedToken.IsExpired(_timeProvider.GetUtcNow())) | ||||||
|  |         { | ||||||
|  |             return cachedToken; | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         await _refreshLock.WaitAsync(cancellationToken).ConfigureAwait(false); | ||||||
|  |         try | ||||||
|  |         { | ||||||
|  |             if (_cache.TryGetValue<MsrcAccessToken>(cacheKey, out cachedToken) && | ||||||
|  |                 cachedToken is not null && | ||||||
|  |                 !cachedToken.IsExpired(_timeProvider.GetUtcNow())) | ||||||
|  |             { | ||||||
|  |                 return cachedToken; | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             var token = await RequestTokenAsync(cancellationToken).ConfigureAwait(false); | ||||||
|  |             var absoluteExpiration = token.ExpiresAt == DateTimeOffset.MaxValue | ||||||
|  |                 ? (DateTimeOffset?)null | ||||||
|  |                 : token.ExpiresAt; | ||||||
|  |  | ||||||
|  |             var options = new MemoryCacheEntryOptions(); | ||||||
|  |             if (absoluteExpiration.HasValue) | ||||||
|  |             { | ||||||
|  |                 options.AbsoluteExpiration = absoluteExpiration.Value; | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             _cache.Set(cacheKey, token, options); | ||||||
|  |             return token; | ||||||
|  |         } | ||||||
|  |         finally | ||||||
|  |         { | ||||||
|  |             _refreshLock.Release(); | ||||||
|  |         } | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     private MsrcAccessToken LoadOfflineToken() | ||||||
|  |     { | ||||||
|  |         if (!string.IsNullOrWhiteSpace(_options.StaticAccessToken)) | ||||||
|  |         { | ||||||
|  |             return new MsrcAccessToken(_options.StaticAccessToken!, "Bearer", DateTimeOffset.MaxValue); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         if (string.IsNullOrWhiteSpace(_options.OfflineTokenPath)) | ||||||
|  |         { | ||||||
|  |             throw new InvalidOperationException("Offline token mode is enabled but no token was provided."); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         if (!_fileSystem.File.Exists(_options.OfflineTokenPath)) | ||||||
|  |         { | ||||||
|  |             throw new InvalidOperationException($"Offline token path '{_options.OfflineTokenPath}' does not exist."); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         var token = _fileSystem.File.ReadAllText(_options.OfflineTokenPath).Trim(); | ||||||
|  |         if (string.IsNullOrEmpty(token)) | ||||||
|  |         { | ||||||
|  |             throw new InvalidOperationException("Offline token file was empty."); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         return new MsrcAccessToken(token, "Bearer", DateTimeOffset.MaxValue); | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     private async Task<MsrcAccessToken> RequestTokenAsync(CancellationToken cancellationToken) | ||||||
|  |     { | ||||||
|  |         _logger.LogInformation("Fetching MSRC AAD access token for tenant {TenantId}.", _options.TenantId); | ||||||
|  |  | ||||||
|  |         var client = _httpClientFactory.CreateClient(MsrcConnectorOptions.TokenClientName); | ||||||
|  |         using var request = new HttpRequestMessage(HttpMethod.Post, BuildTokenUri()) | ||||||
|  |         { | ||||||
|  |             Content = new FormUrlEncodedContent(new Dictionary<string, string> | ||||||
|  |             { | ||||||
|  |                 ["client_id"] = _options.ClientId, | ||||||
|  |                 ["client_secret"] = _options.ClientSecret!, | ||||||
|  |                 ["grant_type"] = "client_credentials", | ||||||
|  |                 ["scope"] = string.IsNullOrWhiteSpace(_options.Scope) ? MsrcConnectorOptions.DefaultScope : _options.Scope, | ||||||
|  |             }), | ||||||
|  |         }; | ||||||
|  |  | ||||||
|  |         using var response = await client.SendAsync(request, cancellationToken).ConfigureAwait(false); | ||||||
|  |         if (!response.IsSuccessStatusCode) | ||||||
|  |         { | ||||||
|  |             var payload = await response.Content.ReadAsStringAsync(cancellationToken).ConfigureAwait(false); | ||||||
|  |             throw new InvalidOperationException($"Failed to acquire MSRC access token ({(int)response.StatusCode}). Response: {payload}"); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         var tokenResponse = await response.Content.ReadFromJsonAsync<TokenResponse>(cancellationToken: cancellationToken).ConfigureAwait(false) | ||||||
|  |             ?? throw new InvalidOperationException("Token endpoint returned an empty payload."); | ||||||
|  |  | ||||||
|  |         if (string.IsNullOrWhiteSpace(tokenResponse.AccessToken)) | ||||||
|  |         { | ||||||
|  |             throw new InvalidOperationException("Token endpoint response did not include an access_token."); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         var now = _timeProvider.GetUtcNow(); | ||||||
|  |         var expiresAt = tokenResponse.ExpiresIn > _options.ExpiryLeewaySeconds | ||||||
|  |             ? now.AddSeconds(tokenResponse.ExpiresIn - _options.ExpiryLeewaySeconds) | ||||||
|  |             : now.AddMinutes(5); | ||||||
|  |  | ||||||
|  |         return new MsrcAccessToken(tokenResponse.AccessToken!, tokenResponse.TokenType ?? "Bearer", expiresAt); | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     private string CreateCacheKey() | ||||||
|  |         => $"{CachePrefix}:{_options.TenantId}:{_options.ClientId}:{_options.Scope}"; | ||||||
|  |  | ||||||
|  |     private Uri BuildTokenUri() | ||||||
|  |         => new($"https://login.microsoftonline.com/{_options.TenantId}/oauth2/v2.0/token"); | ||||||
|  |  | ||||||
|  |     public void Dispose() => _refreshLock.Dispose(); | ||||||
|  |  | ||||||
|  |     private sealed record TokenResponse | ||||||
|  |     { | ||||||
|  |         [JsonPropertyName("access_token")] | ||||||
|  |         public string? AccessToken { get; init; } | ||||||
|  |  | ||||||
|  |         [JsonPropertyName("token_type")] | ||||||
|  |         public string? TokenType { get; init; } | ||||||
|  |  | ||||||
|  |         [JsonPropertyName("expires_in")] | ||||||
|  |         public int ExpiresIn { get; init; } | ||||||
|  |     } | ||||||
|  | } | ||||||
|  |  | ||||||
|  | public sealed record MsrcAccessToken(string Value, string Type, DateTimeOffset ExpiresAt) | ||||||
|  | { | ||||||
|  |     public bool IsExpired(DateTimeOffset now) => now >= ExpiresAt; | ||||||
|  | } | ||||||
| @@ -0,0 +1,95 @@ | |||||||
|  | using System; | ||||||
|  | using System.IO; | ||||||
|  | using System.IO.Abstractions; | ||||||
|  |  | ||||||
|  | namespace StellaOps.Vexer.Connectors.MSRC.CSAF.Configuration; | ||||||
|  |  | ||||||
|  | public sealed class MsrcConnectorOptions | ||||||
|  | { | ||||||
|  |     public const string TokenClientName = "vexer.connector.msrc.token"; | ||||||
|  |     public const string DefaultScope = "https://api.msrc.microsoft.com/.default"; | ||||||
|  |  | ||||||
|  |     /// <summary> | ||||||
|  |     /// Azure AD tenant identifier (GUID or domain). | ||||||
|  |     /// </summary> | ||||||
|  |     public string TenantId { get; set; } = string.Empty; | ||||||
|  |  | ||||||
|  |     /// <summary> | ||||||
|  |     /// Azure AD application (client) identifier. | ||||||
|  |     /// </summary> | ||||||
|  |     public string ClientId { get; set; } = string.Empty; | ||||||
|  |  | ||||||
|  |     /// <summary> | ||||||
|  |     /// Azure AD application secret for client credential flow. | ||||||
|  |     /// </summary> | ||||||
|  |     public string? ClientSecret { get; set; } | ||||||
|  |     /// <summary> | ||||||
|  |     /// OAuth scope requested for MSRC API access. | ||||||
|  |     /// </summary> | ||||||
|  |     public string Scope { get; set; } = DefaultScope; | ||||||
|  |  | ||||||
|  |     /// <summary> | ||||||
|  |     /// When true, token acquisition is skipped and the connector expects offline handling. | ||||||
|  |     /// </summary> | ||||||
|  |     public bool PreferOfflineToken { get; set; } | ||||||
|  |     /// <summary> | ||||||
|  |     /// Optional path to a pre-provisioned bearer token used when <see cref="PreferOfflineToken"/> is enabled. | ||||||
|  |     /// </summary> | ||||||
|  |     public string? OfflineTokenPath { get; set; } | ||||||
|  |     /// <summary> | ||||||
|  |     /// Optional fixed bearer token for constrained environments (e.g., short-lived offline bundles). | ||||||
|  |     /// </summary> | ||||||
|  |     public string? StaticAccessToken { get; set; } | ||||||
|  |     /// <summary> | ||||||
|  |     /// Minimum buffer (seconds) subtracted from token expiry before refresh. | ||||||
|  |     /// </summary> | ||||||
|  |     public int ExpiryLeewaySeconds { get; set; } = 60; | ||||||
|  |  | ||||||
|  |     public void Validate(IFileSystem? fileSystem = null) | ||||||
|  |     { | ||||||
|  |         if (PreferOfflineToken) | ||||||
|  |         { | ||||||
|  |             if (string.IsNullOrWhiteSpace(OfflineTokenPath) && string.IsNullOrWhiteSpace(StaticAccessToken)) | ||||||
|  |             { | ||||||
|  |                 throw new InvalidOperationException("OfflineTokenPath or StaticAccessToken must be provided when PreferOfflineToken is enabled."); | ||||||
|  |             } | ||||||
|  |         } | ||||||
|  |         else | ||||||
|  |         { | ||||||
|  |             if (string.IsNullOrWhiteSpace(TenantId)) | ||||||
|  |             { | ||||||
|  |                 throw new InvalidOperationException("TenantId is required when not operating in offline token mode."); | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             if (string.IsNullOrWhiteSpace(ClientId)) | ||||||
|  |             { | ||||||
|  |                 throw new InvalidOperationException("ClientId is required when not operating in offline token mode."); | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             if (string.IsNullOrWhiteSpace(ClientSecret)) | ||||||
|  |             { | ||||||
|  |                 throw new InvalidOperationException("ClientSecret is required when not operating in offline token mode."); | ||||||
|  |             } | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         if (string.IsNullOrWhiteSpace(Scope)) | ||||||
|  |         { | ||||||
|  |             Scope = DefaultScope; | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         if (ExpiryLeewaySeconds < 10) | ||||||
|  |         { | ||||||
|  |             ExpiryLeewaySeconds = 10; | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         if (!string.IsNullOrWhiteSpace(OfflineTokenPath)) | ||||||
|  |         { | ||||||
|  |             var fs = fileSystem ?? new FileSystem(); | ||||||
|  |             var directory = Path.GetDirectoryName(OfflineTokenPath); | ||||||
|  |             if (!string.IsNullOrWhiteSpace(directory) && !fs.Directory.Exists(directory)) | ||||||
|  |             { | ||||||
|  |                 fs.Directory.CreateDirectory(directory); | ||||||
|  |             } | ||||||
|  |         } | ||||||
|  |     } | ||||||
|  | } | ||||||
| @@ -0,0 +1,40 @@ | |||||||
|  | using System; | ||||||
|  | using System.Net; | ||||||
|  | using System.Net.Http; | ||||||
|  | using Microsoft.Extensions.Caching.Memory; | ||||||
|  | using Microsoft.Extensions.DependencyInjection; | ||||||
|  | using Microsoft.Extensions.DependencyInjection.Extensions; | ||||||
|  | using StellaOps.Vexer.Connectors.MSRC.CSAF.Authentication; | ||||||
|  | using StellaOps.Vexer.Connectors.MSRC.CSAF.Configuration; | ||||||
|  | using System.IO.Abstractions; | ||||||
|  |  | ||||||
|  | namespace StellaOps.Vexer.Connectors.MSRC.CSAF.DependencyInjection; | ||||||
|  |  | ||||||
|  | public static class MsrcConnectorServiceCollectionExtensions | ||||||
|  | { | ||||||
|  |     public static IServiceCollection AddMsrcCsafConnector(this IServiceCollection services, Action<MsrcConnectorOptions>? configure = null) | ||||||
|  |     { | ||||||
|  |         ArgumentNullException.ThrowIfNull(services); | ||||||
|  |  | ||||||
|  |         services.TryAddSingleton<IMemoryCache, MemoryCache>(); | ||||||
|  |         services.TryAddSingleton<IFileSystem, FileSystem>(); | ||||||
|  |  | ||||||
|  |         services.AddOptions<MsrcConnectorOptions>() | ||||||
|  |             .Configure(options => configure?.Invoke(options)); | ||||||
|  |  | ||||||
|  |         services.AddHttpClient(MsrcConnectorOptions.TokenClientName, client => | ||||||
|  |             { | ||||||
|  |                 client.Timeout = TimeSpan.FromSeconds(30); | ||||||
|  |                 client.DefaultRequestHeaders.UserAgent.ParseAdd("StellaOps.Vexer.Connectors.MSRC.CSAF/1.0"); | ||||||
|  |                 client.DefaultRequestHeaders.Accept.ParseAdd("application/json"); | ||||||
|  |             }) | ||||||
|  |             .ConfigurePrimaryHttpMessageHandler(static () => new HttpClientHandler | ||||||
|  |             { | ||||||
|  |                 AutomaticDecompression = DecompressionMethods.All, | ||||||
|  |             }); | ||||||
|  |  | ||||||
|  |         services.AddSingleton<IMsrcTokenProvider, MsrcTokenProvider>(); | ||||||
|  |  | ||||||
|  |         return services; | ||||||
|  |     } | ||||||
|  | } | ||||||
| @@ -0,0 +1,18 @@ | |||||||
|  | <Project Sdk="Microsoft.NET.Sdk"> | ||||||
|  |   <PropertyGroup> | ||||||
|  |     <TargetFramework>net10.0</TargetFramework> | ||||||
|  |     <LangVersion>preview</LangVersion> | ||||||
|  |     <Nullable>enable</Nullable> | ||||||
|  |     <ImplicitUsings>enable</ImplicitUsings> | ||||||
|  |     <TreatWarningsAsErrors>true</TreatWarningsAsErrors> | ||||||
|  |   </PropertyGroup> | ||||||
|  |   <ItemGroup> | ||||||
|  |     <ProjectReference Include="..\StellaOps.Vexer.Connectors.Abstractions\StellaOps.Vexer.Connectors.Abstractions.csproj" /> | ||||||
|  |   </ItemGroup> | ||||||
|  |   <ItemGroup> | ||||||
|  |     <PackageReference Include="Microsoft.Extensions.Caching.Memory" Version="8.0.0" /> | ||||||
|  |     <PackageReference Include="Microsoft.Extensions.Http" Version="8.0.0" /> | ||||||
|  |     <PackageReference Include="Microsoft.Extensions.Options" Version="8.0.0" /> | ||||||
|  |     <PackageReference Include="System.IO.Abstractions" Version="20.0.28" /> | ||||||
|  |   </ItemGroup> | ||||||
|  | </Project> | ||||||
| @@ -2,6 +2,6 @@ If you are working on this file you need to read docs/ARCHITECTURE_VEXER.md and | |||||||
| # TASKS | # TASKS | ||||||
| | Task | Owner(s) | Depends on | Notes | | | Task | Owner(s) | Depends on | Notes | | ||||||
| |---|---|---|---| | |---|---|---|---| | ||||||
| |VEXER-CONN-MS-01-001 – AAD onboarding & token cache|Team Vexer Connectors – MSRC|VEXER-CONN-ABS-01-001|TODO – Implement Azure AD credential flow, token caching, and validation for MSRC CSAF access with offline fallback guidance.| | |VEXER-CONN-MS-01-001 – AAD onboarding & token cache|Team Vexer Connectors – MSRC|VEXER-CONN-ABS-01-001|**DONE (2025-10-17)** – Added MSRC connector project with configurable AAD options, token provider (offline/online modes), DI wiring, and unit tests covering caching and fallback scenarios.| | ||||||
| |VEXER-CONN-MS-01-002 – CSAF download pipeline|Team Vexer Connectors – MSRC|VEXER-CONN-MS-01-001, VEXER-STORAGE-01-003|TODO – Fetch CSAF packages with retry/backoff, checksum verification, and raw document persistence plus quarantine for schema failures.| | |VEXER-CONN-MS-01-002 – CSAF download pipeline|Team Vexer Connectors – MSRC|VEXER-CONN-MS-01-001, VEXER-STORAGE-01-003|TODO – Fetch CSAF packages with retry/backoff, checksum verification, and raw document persistence plus quarantine for schema failures.| | ||||||
| |VEXER-CONN-MS-01-003 – Trust metadata & provenance hints|Team Vexer Connectors – MSRC|VEXER-CONN-MS-01-002, VEXER-POLICY-01-001|TODO – Emit cosign/AAD issuer metadata, attach provenance details, and document policy integration.| | |VEXER-CONN-MS-01-003 – Trust metadata & provenance hints|Team Vexer Connectors – MSRC|VEXER-CONN-MS-01-002, VEXER-POLICY-01-001|TODO – Emit cosign/AAD issuer metadata, attach provenance details, and document policy integration.| | ||||||
|   | |||||||
| @@ -0,0 +1,205 @@ | |||||||
|  | using System.Collections.Generic; | ||||||
|  | using System.Net; | ||||||
|  | using System.Net.Http; | ||||||
|  | using System.Text; | ||||||
|  | using FluentAssertions; | ||||||
|  | using Microsoft.Extensions.Caching.Memory; | ||||||
|  | using Microsoft.Extensions.Logging.Abstractions; | ||||||
|  | using StellaOps.Vexer.Connectors.Oracle.CSAF.Configuration; | ||||||
|  | using StellaOps.Vexer.Connectors.Oracle.CSAF.Metadata; | ||||||
|  | using System.IO.Abstractions.TestingHelpers; | ||||||
|  | using Xunit; | ||||||
|  | using System.Threading; | ||||||
|  |  | ||||||
|  | namespace StellaOps.Vexer.Connectors.Oracle.CSAF.Tests.Metadata; | ||||||
|  |  | ||||||
|  | public sealed class OracleCatalogLoaderTests | ||||||
|  | { | ||||||
|  |     private const string SampleCatalog = """ | ||||||
|  |         { | ||||||
|  |           "generated": "2025-09-30T18:00:00Z", | ||||||
|  |           "catalog": [ | ||||||
|  |             { | ||||||
|  |               "id": "CPU2025Oct", | ||||||
|  |               "title": "Oracle Critical Patch Update Advisory - October 2025", | ||||||
|  |               "published": "2025-10-15T00:00:00Z", | ||||||
|  |               "revision": "2025-10-15T00:00:00Z", | ||||||
|  |               "document": { | ||||||
|  |                 "url": "https://updates.oracle.com/cpu/2025-10/cpu2025oct.json", | ||||||
|  |                 "sha256": "abc123", | ||||||
|  |                 "size": 1024 | ||||||
|  |               }, | ||||||
|  |               "products": ["Oracle Database", "Java SE"] | ||||||
|  |             } | ||||||
|  |           ], | ||||||
|  |           "schedule": [ | ||||||
|  |             { "window": "2025-Oct", "releaseDate": "2025-10-15T00:00:00Z" } | ||||||
|  |           ] | ||||||
|  |         } | ||||||
|  |         """; | ||||||
|  |  | ||||||
|  |     private const string SampleCalendar = """ | ||||||
|  |         { | ||||||
|  |           "cpuWindows": [ | ||||||
|  |             { "name": "2026-Jan", "releaseDate": "2026-01-21T00:00:00Z" } | ||||||
|  |           ] | ||||||
|  |         } | ||||||
|  |         """; | ||||||
|  |  | ||||||
|  |     [Fact] | ||||||
|  |     public async Task LoadAsync_FetchesAndCachesCatalog() | ||||||
|  |     { | ||||||
|  |         var handler = new TestHttpMessageHandler(new Dictionary<Uri, HttpResponseMessage> | ||||||
|  |         { | ||||||
|  |             [new Uri("https://www.oracle.com/security-alerts/cpu/csaf/catalog.json")] = CreateResponse(SampleCatalog), | ||||||
|  |             [new Uri("https://www.oracle.com/security-alerts/cpu/cal.json")] = CreateResponse(SampleCalendar), | ||||||
|  |         }); | ||||||
|  |         var client = new HttpClient(handler); | ||||||
|  |         var factory = new SingleClientHttpClientFactory(client); | ||||||
|  |         var cache = new MemoryCache(new MemoryCacheOptions()); | ||||||
|  |         var fileSystem = new MockFileSystem(); | ||||||
|  |         var loader = new OracleCatalogLoader(factory, cache, fileSystem, NullLogger<OracleCatalogLoader>.Instance, new AdjustableTimeProvider()); | ||||||
|  |  | ||||||
|  |         var options = new OracleConnectorOptions | ||||||
|  |         { | ||||||
|  |             CatalogUri = new Uri("https://www.oracle.com/security-alerts/cpu/csaf/catalog.json"), | ||||||
|  |             CpuCalendarUri = new Uri("https://www.oracle.com/security-alerts/cpu/cal.json"), | ||||||
|  |             OfflineSnapshotPath = "/snapshots/oracle-catalog.json", | ||||||
|  |         }; | ||||||
|  |  | ||||||
|  |         var result = await loader.LoadAsync(options, CancellationToken.None); | ||||||
|  |         result.Metadata.Entries.Should().HaveCount(1); | ||||||
|  |         result.Metadata.CpuSchedule.Should().Contain(r => r.Window == "2025-Oct"); | ||||||
|  |         result.Metadata.CpuSchedule.Should().Contain(r => r.Window == "2026-Jan"); | ||||||
|  |         result.FromCache.Should().BeFalse(); | ||||||
|  |         fileSystem.FileExists(options.OfflineSnapshotPath!).Should().BeTrue(); | ||||||
|  |  | ||||||
|  |         handler.ResetInvocationCount(); | ||||||
|  |         var cached = await loader.LoadAsync(options, CancellationToken.None); | ||||||
|  |         cached.FromCache.Should().BeTrue(); | ||||||
|  |         handler.InvocationCount.Should().Be(0); | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     [Fact] | ||||||
|  |     public async Task LoadAsync_UsesOfflineSnapshotWhenNetworkFails() | ||||||
|  |     { | ||||||
|  |         var handler = new TestHttpMessageHandler(new Dictionary<Uri, HttpResponseMessage>()); | ||||||
|  |         var client = new HttpClient(handler); | ||||||
|  |         var factory = new SingleClientHttpClientFactory(client); | ||||||
|  |         var cache = new MemoryCache(new MemoryCacheOptions()); | ||||||
|  |         var fileSystem = new MockFileSystem(); | ||||||
|  |         var offlineJson = """ | ||||||
|  |             { | ||||||
|  |               "metadata": { | ||||||
|  |                 "generatedAt": "2025-09-30T18:00:00Z", | ||||||
|  |                 "entries": [ | ||||||
|  |                   { | ||||||
|  |                     "id": "CPU2025Oct", | ||||||
|  |                     "title": "Oracle Critical Patch Update Advisory - October 2025", | ||||||
|  |                     "documentUri": "https://updates.oracle.com/cpu/2025-10/cpu2025oct.json", | ||||||
|  |                     "publishedAt": "2025-10-15T00:00:00Z", | ||||||
|  |                     "revision": "2025-10-15T00:00:00Z", | ||||||
|  |                     "sha256": "abc123", | ||||||
|  |                     "size": 1024, | ||||||
|  |                     "products": [ "Oracle Database" ] | ||||||
|  |                   } | ||||||
|  |                 ], | ||||||
|  |                 "cpuSchedule": [ | ||||||
|  |                   { "window": "2025-Oct", "releaseDate": "2025-10-15T00:00:00Z" } | ||||||
|  |                 ] | ||||||
|  |               }, | ||||||
|  |               "fetchedAt": "2025-10-01T00:00:00Z" | ||||||
|  |             } | ||||||
|  |             """; | ||||||
|  |         fileSystem.AddFile("/snapshots/oracle-catalog.json", new MockFileData(offlineJson)); | ||||||
|  |         var loader = new OracleCatalogLoader(factory, cache, fileSystem, NullLogger<OracleCatalogLoader>.Instance, new AdjustableTimeProvider()); | ||||||
|  |  | ||||||
|  |         var options = new OracleConnectorOptions | ||||||
|  |         { | ||||||
|  |             CatalogUri = new Uri("https://www.oracle.com/security-alerts/cpu/csaf/catalog.json"), | ||||||
|  |             OfflineSnapshotPath = "/snapshots/oracle-catalog.json", | ||||||
|  |             PreferOfflineSnapshot = true, | ||||||
|  |         }; | ||||||
|  |  | ||||||
|  |         var result = await loader.LoadAsync(options, CancellationToken.None); | ||||||
|  |         result.FromOfflineSnapshot.Should().BeTrue(); | ||||||
|  |         result.Metadata.Entries.Should().NotBeEmpty(); | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     [Fact] | ||||||
|  |     public async Task LoadAsync_ThrowsWhenOfflinePreferredButMissing() | ||||||
|  |     { | ||||||
|  |         var handler = new TestHttpMessageHandler(new Dictionary<Uri, HttpResponseMessage>()); | ||||||
|  |         var client = new HttpClient(handler); | ||||||
|  |         var factory = new SingleClientHttpClientFactory(client); | ||||||
|  |         var cache = new MemoryCache(new MemoryCacheOptions()); | ||||||
|  |         var fileSystem = new MockFileSystem(); | ||||||
|  |         var loader = new OracleCatalogLoader(factory, cache, fileSystem, NullLogger<OracleCatalogLoader>.Instance); | ||||||
|  |  | ||||||
|  |         var options = new OracleConnectorOptions | ||||||
|  |         { | ||||||
|  |             CatalogUri = new Uri("https://www.oracle.com/security-alerts/cpu/csaf/catalog.json"), | ||||||
|  |             PreferOfflineSnapshot = true, | ||||||
|  |             OfflineSnapshotPath = "/missing/oracle-catalog.json", | ||||||
|  |         }; | ||||||
|  |  | ||||||
|  |         await Assert.ThrowsAsync<InvalidOperationException>(() => loader.LoadAsync(options, CancellationToken.None)); | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     private static HttpResponseMessage CreateResponse(string payload) | ||||||
|  |         => new(HttpStatusCode.OK) | ||||||
|  |         { | ||||||
|  |             Content = new StringContent(payload, Encoding.UTF8, "application/json"), | ||||||
|  |         }; | ||||||
|  |  | ||||||
|  |     private sealed class SingleClientHttpClientFactory : IHttpClientFactory | ||||||
|  |     { | ||||||
|  |         private readonly HttpClient _client; | ||||||
|  |  | ||||||
|  |         public SingleClientHttpClientFactory(HttpClient client) | ||||||
|  |         { | ||||||
|  |             _client = client; | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         public HttpClient CreateClient(string name) => _client; | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     private sealed class AdjustableTimeProvider : TimeProvider | ||||||
|  |     { | ||||||
|  |         private DateTimeOffset _now = DateTimeOffset.UtcNow; | ||||||
|  |  | ||||||
|  |         public override DateTimeOffset GetUtcNow() => _now; | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     private sealed class TestHttpMessageHandler : HttpMessageHandler | ||||||
|  |     { | ||||||
|  |         private readonly Dictionary<Uri, HttpResponseMessage> _responses; | ||||||
|  |  | ||||||
|  |         public TestHttpMessageHandler(Dictionary<Uri, HttpResponseMessage> responses) | ||||||
|  |         { | ||||||
|  |             _responses = responses; | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         public int InvocationCount { get; private set; } | ||||||
|  |  | ||||||
|  |         public void ResetInvocationCount() => InvocationCount = 0; | ||||||
|  |  | ||||||
|  |         protected override async Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) | ||||||
|  |         { | ||||||
|  |             InvocationCount++; | ||||||
|  |             if (request.RequestUri is not null && _responses.TryGetValue(request.RequestUri, out var response)) | ||||||
|  |             { | ||||||
|  |                 var payload = await response.Content.ReadAsStringAsync(cancellationToken).ConfigureAwait(false); | ||||||
|  |                 return new HttpResponseMessage(response.StatusCode) | ||||||
|  |                 { | ||||||
|  |                     Content = new StringContent(payload, Encoding.UTF8, "application/json"), | ||||||
|  |                 }; | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             return new HttpResponseMessage(HttpStatusCode.InternalServerError) | ||||||
|  |             { | ||||||
|  |                 Content = new StringContent("unexpected request"), | ||||||
|  |             }; | ||||||
|  |         } | ||||||
|  |     } | ||||||
|  | } | ||||||
| @@ -0,0 +1,17 @@ | |||||||
|  | <Project Sdk="Microsoft.NET.Sdk"> | ||||||
|  |   <PropertyGroup> | ||||||
|  |     <TargetFramework>net10.0</TargetFramework> | ||||||
|  |     <LangVersion>preview</LangVersion> | ||||||
|  |     <Nullable>enable</Nullable> | ||||||
|  |     <ImplicitUsings>enable</ImplicitUsings> | ||||||
|  |     <TreatWarningsAsErrors>true</TreatWarningsAsErrors> | ||||||
|  |   </PropertyGroup> | ||||||
|  |   <ItemGroup> | ||||||
|  |     <ProjectReference Include="..\StellaOps.Vexer.Connectors.Oracle.CSAF\StellaOps.Vexer.Connectors.Oracle.CSAF.csproj" /> | ||||||
|  |   </ItemGroup> | ||||||
|  |   <ItemGroup> | ||||||
|  |     <PackageReference Include="FluentAssertions" Version="6.12.0" /> | ||||||
|  |     <PackageReference Include="Microsoft.Extensions.Logging.Abstractions" Version="8.0.0" /> | ||||||
|  |     <PackageReference Include="System.IO.Abstractions.TestingHelpers" Version="20.0.28" /> | ||||||
|  |   </ItemGroup> | ||||||
|  | </Project> | ||||||
| @@ -0,0 +1,85 @@ | |||||||
|  | using System; | ||||||
|  | using System.IO; | ||||||
|  | using System.IO.Abstractions; | ||||||
|  |  | ||||||
|  | namespace StellaOps.Vexer.Connectors.Oracle.CSAF.Configuration; | ||||||
|  |  | ||||||
|  | public sealed class OracleConnectorOptions | ||||||
|  | { | ||||||
|  |     public const string HttpClientName = "vexer.connector.oracle.catalog"; | ||||||
|  |  | ||||||
|  |     /// <summary> | ||||||
|  |     /// Oracle CSAF catalog endpoint hosting advisory metadata. | ||||||
|  |     /// </summary> | ||||||
|  |     public Uri CatalogUri { get; set; } = new("https://www.oracle.com/security-alerts/cpu/csaf/catalog.json"); | ||||||
|  |  | ||||||
|  |     /// <summary> | ||||||
|  |     /// Optional CPU calendar endpoint providing upcoming release dates. | ||||||
|  |     /// </summary> | ||||||
|  |     public Uri? CpuCalendarUri { get; set; } | ||||||
|  |     /// <summary> | ||||||
|  |     /// Duration the discovery metadata should be cached before refresh. | ||||||
|  |     /// </summary> | ||||||
|  |     public TimeSpan MetadataCacheDuration { get; set; } = TimeSpan.FromHours(6); | ||||||
|  |  | ||||||
|  |     /// <summary> | ||||||
|  |     /// When true, the loader will prefer offline snapshot data over network fetches. | ||||||
|  |     /// </summary> | ||||||
|  |     public bool PreferOfflineSnapshot { get; set; } | ||||||
|  |     /// <summary> | ||||||
|  |     /// Optional file path for persisting or ingesting catalog snapshots. | ||||||
|  |     /// </summary> | ||||||
|  |     public string? OfflineSnapshotPath { get; set; } | ||||||
|  |     /// <summary> | ||||||
|  |     /// Enables writing fresh catalog responses to <see cref="OfflineSnapshotPath"/>. | ||||||
|  |     /// </summary> | ||||||
|  |     public bool PersistOfflineSnapshot { get; set; } = true; | ||||||
|  |  | ||||||
|  |     /// <summary> | ||||||
|  |     /// Optional request delay when iterating catalogue entries (for rate limiting). | ||||||
|  |     /// </summary> | ||||||
|  |     public TimeSpan RequestDelay { get; set; } = TimeSpan.FromMilliseconds(250); | ||||||
|  |  | ||||||
|  |     public void Validate(IFileSystem? fileSystem = null) | ||||||
|  |     { | ||||||
|  |         if (CatalogUri is null || !CatalogUri.IsAbsoluteUri) | ||||||
|  |         { | ||||||
|  |             throw new InvalidOperationException("CatalogUri must be an absolute URI."); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         if (CatalogUri.Scheme is not ("http" or "https")) | ||||||
|  |         { | ||||||
|  |             throw new InvalidOperationException("CatalogUri must use HTTP or HTTPS."); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         if (CpuCalendarUri is not null && (!CpuCalendarUri.IsAbsoluteUri || CpuCalendarUri.Scheme is not ("http" or "https"))) | ||||||
|  |         { | ||||||
|  |             throw new InvalidOperationException("CpuCalendarUri must be an absolute HTTP(S) URI when provided."); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         if (MetadataCacheDuration <= TimeSpan.Zero) | ||||||
|  |         { | ||||||
|  |             throw new InvalidOperationException("MetadataCacheDuration must be positive."); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         if (RequestDelay < TimeSpan.Zero) | ||||||
|  |         { | ||||||
|  |             throw new InvalidOperationException("RequestDelay cannot be negative."); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         if (PreferOfflineSnapshot && string.IsNullOrWhiteSpace(OfflineSnapshotPath)) | ||||||
|  |         { | ||||||
|  |             throw new InvalidOperationException("OfflineSnapshotPath must be provided when PreferOfflineSnapshot is enabled."); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         if (!string.IsNullOrWhiteSpace(OfflineSnapshotPath)) | ||||||
|  |         { | ||||||
|  |             var fs = fileSystem ?? new FileSystem(); | ||||||
|  |             var directory = Path.GetDirectoryName(OfflineSnapshotPath); | ||||||
|  |             if (!string.IsNullOrWhiteSpace(directory) && !fs.Directory.Exists(directory)) | ||||||
|  |             { | ||||||
|  |                 fs.Directory.CreateDirectory(directory); | ||||||
|  |             } | ||||||
|  |         } | ||||||
|  |     } | ||||||
|  | } | ||||||
| @@ -0,0 +1,32 @@ | |||||||
|  | using System; | ||||||
|  | using System.Collections.Generic; | ||||||
|  | using System.IO.Abstractions; | ||||||
|  | using StellaOps.Vexer.Connectors.Abstractions; | ||||||
|  |  | ||||||
|  | namespace StellaOps.Vexer.Connectors.Oracle.CSAF.Configuration; | ||||||
|  |  | ||||||
|  | public sealed class OracleConnectorOptionsValidator : IVexConnectorOptionsValidator<OracleConnectorOptions> | ||||||
|  | { | ||||||
|  |     private readonly IFileSystem _fileSystem; | ||||||
|  |  | ||||||
|  |     public OracleConnectorOptionsValidator(IFileSystem fileSystem) | ||||||
|  |     { | ||||||
|  |         _fileSystem = fileSystem ?? throw new ArgumentNullException(nameof(fileSystem)); | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     public void Validate(VexConnectorDescriptor descriptor, OracleConnectorOptions options, IList<string> errors) | ||||||
|  |     { | ||||||
|  |         ArgumentNullException.ThrowIfNull(descriptor); | ||||||
|  |         ArgumentNullException.ThrowIfNull(options); | ||||||
|  |         ArgumentNullException.ThrowIfNull(errors); | ||||||
|  |  | ||||||
|  |         try | ||||||
|  |         { | ||||||
|  |             options.Validate(_fileSystem); | ||||||
|  |         } | ||||||
|  |         catch (Exception ex) | ||||||
|  |         { | ||||||
|  |             errors.Add(ex.Message); | ||||||
|  |         } | ||||||
|  |     } | ||||||
|  | } | ||||||
| @@ -0,0 +1,45 @@ | |||||||
|  | using System; | ||||||
|  | using System.Net; | ||||||
|  | using System.Net.Http; | ||||||
|  | using Microsoft.Extensions.Caching.Memory; | ||||||
|  | using Microsoft.Extensions.DependencyInjection; | ||||||
|  | using Microsoft.Extensions.DependencyInjection.Extensions; | ||||||
|  | using StellaOps.Vexer.Connectors.Abstractions; | ||||||
|  | using StellaOps.Vexer.Connectors.Oracle.CSAF.Configuration; | ||||||
|  | using StellaOps.Vexer.Connectors.Oracle.CSAF.Metadata; | ||||||
|  | using StellaOps.Vexer.Core; | ||||||
|  | using System.IO.Abstractions; | ||||||
|  |  | ||||||
|  | namespace StellaOps.Vexer.Connectors.Oracle.CSAF.DependencyInjection; | ||||||
|  |  | ||||||
|  | public static class OracleConnectorServiceCollectionExtensions | ||||||
|  | { | ||||||
|  |     public static IServiceCollection AddOracleCsafConnector(this IServiceCollection services, Action<OracleConnectorOptions>? configure = null) | ||||||
|  |     { | ||||||
|  |         ArgumentNullException.ThrowIfNull(services); | ||||||
|  |  | ||||||
|  |         services.TryAddSingleton<IMemoryCache, MemoryCache>(); | ||||||
|  |         services.TryAddSingleton<IFileSystem, FileSystem>(); | ||||||
|  |  | ||||||
|  |         services.AddOptions<OracleConnectorOptions>() | ||||||
|  |             .Configure(options => configure?.Invoke(options)); | ||||||
|  |  | ||||||
|  |         services.AddSingleton<IVexConnectorOptionsValidator<OracleConnectorOptions>, OracleConnectorOptionsValidator>(); | ||||||
|  |  | ||||||
|  |         services.AddHttpClient(OracleConnectorOptions.HttpClientName, client => | ||||||
|  |             { | ||||||
|  |                 client.Timeout = TimeSpan.FromSeconds(60); | ||||||
|  |                 client.DefaultRequestHeaders.UserAgent.ParseAdd("StellaOps.Vexer.Connectors.Oracle.CSAF/1.0"); | ||||||
|  |                 client.DefaultRequestHeaders.Accept.ParseAdd("application/json"); | ||||||
|  |             }) | ||||||
|  |             .ConfigurePrimaryHttpMessageHandler(static () => new HttpClientHandler | ||||||
|  |             { | ||||||
|  |                 AutomaticDecompression = DecompressionMethods.All, | ||||||
|  |             }); | ||||||
|  |  | ||||||
|  |         services.AddSingleton<OracleCatalogLoader>(); | ||||||
|  |         services.AddSingleton<IVexConnector, OracleCsafConnector>(); | ||||||
|  |  | ||||||
|  |         return services; | ||||||
|  |     } | ||||||
|  | } | ||||||
| @@ -0,0 +1,418 @@ | |||||||
|  | using System; | ||||||
|  | using System.Collections.Generic; | ||||||
|  | using System.Collections.Immutable; | ||||||
|  | using System.IO.Abstractions; | ||||||
|  | using System.Net; | ||||||
|  | using System.Net.Http; | ||||||
|  | using System.Text.Json; | ||||||
|  | using System.Threading; | ||||||
|  | using System.Threading.Tasks; | ||||||
|  | using Microsoft.Extensions.Caching.Memory; | ||||||
|  | using Microsoft.Extensions.Logging; | ||||||
|  | using StellaOps.Vexer.Connectors.Oracle.CSAF.Configuration; | ||||||
|  |  | ||||||
|  | namespace StellaOps.Vexer.Connectors.Oracle.CSAF.Metadata; | ||||||
|  |  | ||||||
|  | public sealed class OracleCatalogLoader | ||||||
|  | { | ||||||
|  |     public const string CachePrefix = "StellaOps.Vexer.Connectors.Oracle.CSAF.Catalog"; | ||||||
|  |  | ||||||
|  |     private readonly IHttpClientFactory _httpClientFactory; | ||||||
|  |     private readonly IMemoryCache _memoryCache; | ||||||
|  |     private readonly IFileSystem _fileSystem; | ||||||
|  |     private readonly ILogger<OracleCatalogLoader> _logger; | ||||||
|  |     private readonly TimeProvider _timeProvider; | ||||||
|  |     private readonly SemaphoreSlim _semaphore = new(1, 1); | ||||||
|  |     private readonly JsonSerializerOptions _serializerOptions = new(JsonSerializerDefaults.Web); | ||||||
|  |  | ||||||
|  |     public OracleCatalogLoader( | ||||||
|  |         IHttpClientFactory httpClientFactory, | ||||||
|  |         IMemoryCache memoryCache, | ||||||
|  |         IFileSystem fileSystem, | ||||||
|  |         ILogger<OracleCatalogLoader> logger, | ||||||
|  |         TimeProvider? timeProvider = null) | ||||||
|  |     { | ||||||
|  |         _httpClientFactory = httpClientFactory ?? throw new ArgumentNullException(nameof(httpClientFactory)); | ||||||
|  |         _memoryCache = memoryCache ?? throw new ArgumentNullException(nameof(memoryCache)); | ||||||
|  |         _fileSystem = fileSystem ?? throw new ArgumentNullException(nameof(fileSystem)); | ||||||
|  |         _logger = logger ?? throw new ArgumentNullException(nameof(logger)); | ||||||
|  |         _timeProvider = timeProvider ?? TimeProvider.System; | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     public async Task<OracleCatalogResult> LoadAsync(OracleConnectorOptions options, CancellationToken cancellationToken) | ||||||
|  |     { | ||||||
|  |         ArgumentNullException.ThrowIfNull(options); | ||||||
|  |         options.Validate(_fileSystem); | ||||||
|  |  | ||||||
|  |         var cacheKey = CreateCacheKey(options); | ||||||
|  |         if (_memoryCache.TryGetValue<CacheEntry>(cacheKey, out var cached) && cached is not null && !cached.IsExpired(_timeProvider.GetUtcNow())) | ||||||
|  |         { | ||||||
|  |             return cached.ToResult(fromCache: true); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         await _semaphore.WaitAsync(cancellationToken).ConfigureAwait(false); | ||||||
|  |         try | ||||||
|  |         { | ||||||
|  |             if (_memoryCache.TryGetValue<CacheEntry>(cacheKey, out cached) && cached is not null && !cached.IsExpired(_timeProvider.GetUtcNow())) | ||||||
|  |             { | ||||||
|  |                 return cached.ToResult(fromCache: true); | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             CacheEntry? entry = null; | ||||||
|  |             if (options.PreferOfflineSnapshot) | ||||||
|  |             { | ||||||
|  |                 entry = LoadFromOffline(options); | ||||||
|  |                 if (entry is null) | ||||||
|  |                 { | ||||||
|  |                     throw new InvalidOperationException("PreferOfflineSnapshot is enabled but no offline snapshot was found or could be loaded."); | ||||||
|  |                 } | ||||||
|  |             } | ||||||
|  |             else | ||||||
|  |             { | ||||||
|  |                 entry = await TryFetchFromNetworkAsync(options, cancellationToken).ConfigureAwait(false) | ||||||
|  |                     ?? LoadFromOffline(options); | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             if (entry is null) | ||||||
|  |             { | ||||||
|  |                 throw new InvalidOperationException("Unable to load Oracle CSAF catalog from network or offline snapshot."); | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             var expiration = entry.MetadataCacheDuration == TimeSpan.Zero | ||||||
|  |                 ? (DateTimeOffset?)null | ||||||
|  |                 : _timeProvider.GetUtcNow().Add(entry.MetadataCacheDuration); | ||||||
|  |  | ||||||
|  |             var cacheEntryOptions = new MemoryCacheEntryOptions(); | ||||||
|  |             if (expiration.HasValue) | ||||||
|  |             { | ||||||
|  |                 cacheEntryOptions.AbsoluteExpiration = expiration.Value; | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             _memoryCache.Set(cacheKey, entry with { CachedAt = _timeProvider.GetUtcNow() }, cacheEntryOptions); | ||||||
|  |             return entry.ToResult(fromCache: false); | ||||||
|  |         } | ||||||
|  |         finally | ||||||
|  |         { | ||||||
|  |             _semaphore.Release(); | ||||||
|  |         } | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     private async Task<CacheEntry?> TryFetchFromNetworkAsync(OracleConnectorOptions options, CancellationToken cancellationToken) | ||||||
|  |     { | ||||||
|  |         try | ||||||
|  |         { | ||||||
|  |             var client = _httpClientFactory.CreateClient(OracleConnectorOptions.HttpClientName); | ||||||
|  |             using var response = await client.GetAsync(options.CatalogUri, HttpCompletionOption.ResponseHeadersRead, cancellationToken).ConfigureAwait(false); | ||||||
|  |             response.EnsureSuccessStatusCode(); | ||||||
|  |             var catalogPayload = await response.Content.ReadAsStringAsync(cancellationToken).ConfigureAwait(false); | ||||||
|  |  | ||||||
|  |             string? calendarPayload = null; | ||||||
|  |             if (options.CpuCalendarUri is not null) | ||||||
|  |             { | ||||||
|  |                 try | ||||||
|  |                 { | ||||||
|  |                     using var calendarResponse = await client.GetAsync(options.CpuCalendarUri, HttpCompletionOption.ResponseHeadersRead, cancellationToken).ConfigureAwait(false); | ||||||
|  |                     calendarResponse.EnsureSuccessStatusCode(); | ||||||
|  |                     calendarPayload = await calendarResponse.Content.ReadAsStringAsync(cancellationToken).ConfigureAwait(false); | ||||||
|  |                 } | ||||||
|  |                 catch (Exception ex) when (ex is not OperationCanceledException) | ||||||
|  |                 { | ||||||
|  |                     _logger.LogWarning(ex, "Failed to fetch Oracle CPU calendar from {Uri}; continuing without schedule.", options.CpuCalendarUri); | ||||||
|  |                 } | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             var metadata = ParseMetadata(catalogPayload, calendarPayload); | ||||||
|  |             var fetchedAt = _timeProvider.GetUtcNow(); | ||||||
|  |             var entry = new CacheEntry(metadata, fetchedAt, fetchedAt, options.MetadataCacheDuration, false); | ||||||
|  |  | ||||||
|  |             PersistSnapshotIfNeeded(options, metadata, fetchedAt); | ||||||
|  |             return entry; | ||||||
|  |         } | ||||||
|  |         catch (Exception ex) when (ex is not OperationCanceledException) | ||||||
|  |         { | ||||||
|  |             _logger.LogWarning(ex, "Failed to fetch Oracle CSAF catalog from {Uri}; attempting offline fallback if available.", options.CatalogUri); | ||||||
|  |             return null; | ||||||
|  |         } | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     private CacheEntry? LoadFromOffline(OracleConnectorOptions options) | ||||||
|  |     { | ||||||
|  |         if (string.IsNullOrWhiteSpace(options.OfflineSnapshotPath)) | ||||||
|  |         { | ||||||
|  |             return null; | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         if (!_fileSystem.File.Exists(options.OfflineSnapshotPath)) | ||||||
|  |         { | ||||||
|  |             _logger.LogWarning("Oracle offline snapshot path {Path} does not exist.", options.OfflineSnapshotPath); | ||||||
|  |             return null; | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         try | ||||||
|  |         { | ||||||
|  |             var payload = _fileSystem.File.ReadAllText(options.OfflineSnapshotPath); | ||||||
|  |             var snapshot = JsonSerializer.Deserialize<OracleCatalogSnapshot>(payload, _serializerOptions); | ||||||
|  |             if (snapshot is null) | ||||||
|  |             { | ||||||
|  |                 throw new InvalidOperationException("Offline snapshot payload was empty."); | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             return new CacheEntry(snapshot.Metadata, snapshot.FetchedAt, _timeProvider.GetUtcNow(), options.MetadataCacheDuration, true); | ||||||
|  |         } | ||||||
|  |         catch (Exception ex) | ||||||
|  |         { | ||||||
|  |             _logger.LogError(ex, "Failed to load Oracle CSAF catalog from offline snapshot {Path}.", options.OfflineSnapshotPath); | ||||||
|  |             return null; | ||||||
|  |         } | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     private OracleCatalogMetadata ParseMetadata(string catalogPayload, string? calendarPayload) | ||||||
|  |     { | ||||||
|  |         if (string.IsNullOrWhiteSpace(catalogPayload)) | ||||||
|  |         { | ||||||
|  |             throw new InvalidOperationException("Oracle catalog payload was empty."); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         using var document = JsonDocument.Parse(catalogPayload); | ||||||
|  |         var root = document.RootElement; | ||||||
|  |  | ||||||
|  |         var generatedAt = root.TryGetProperty("generated", out var generatedElement) && generatedElement.ValueKind == JsonValueKind.String && DateTimeOffset.TryParse(generatedElement.GetString(), out var generated) | ||||||
|  |             ? generated | ||||||
|  |             : _timeProvider.GetUtcNow(); | ||||||
|  |  | ||||||
|  |         var entries = ParseEntries(root); | ||||||
|  |         var schedule = ParseSchedule(root); | ||||||
|  |  | ||||||
|  |         if (!string.IsNullOrWhiteSpace(calendarPayload)) | ||||||
|  |         { | ||||||
|  |             schedule = MergeSchedule(schedule, calendarPayload); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         return new OracleCatalogMetadata(generatedAt, entries, schedule); | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     private ImmutableArray<OracleCatalogEntry> ParseEntries(JsonElement root) | ||||||
|  |     { | ||||||
|  |         if (!root.TryGetProperty("catalog", out var catalogElement) || catalogElement.ValueKind is not JsonValueKind.Array) | ||||||
|  |         { | ||||||
|  |             return ImmutableArray<OracleCatalogEntry>.Empty; | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         var builder = ImmutableArray.CreateBuilder<OracleCatalogEntry>(); | ||||||
|  |         foreach (var entry in catalogElement.EnumerateArray()) | ||||||
|  |         { | ||||||
|  |             var id = entry.TryGetProperty("id", out var idElement) && idElement.ValueKind == JsonValueKind.String ? idElement.GetString() : null; | ||||||
|  |             var title = entry.TryGetProperty("title", out var titleElement) && titleElement.ValueKind == JsonValueKind.String ? titleElement.GetString() : null; | ||||||
|  |             if (string.IsNullOrWhiteSpace(id) || string.IsNullOrWhiteSpace(title)) | ||||||
|  |             { | ||||||
|  |                 continue; | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             DateTimeOffset publishedAt = default; | ||||||
|  |             if (entry.TryGetProperty("published", out var publishedElement) && publishedElement.ValueKind == JsonValueKind.String && DateTimeOffset.TryParse(publishedElement.GetString(), out var publishedParsed)) | ||||||
|  |             { | ||||||
|  |                 publishedAt = publishedParsed; | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             string? revision = null; | ||||||
|  |             if (entry.TryGetProperty("revision", out var revisionElement) && revisionElement.ValueKind == JsonValueKind.String) | ||||||
|  |             { | ||||||
|  |                 revision = revisionElement.GetString(); | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             ImmutableArray<string> products = ImmutableArray<string>.Empty; | ||||||
|  |             if (entry.TryGetProperty("products", out var productsElement)) | ||||||
|  |             { | ||||||
|  |                 products = ParseStringArray(productsElement); | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             Uri? documentUri = null; | ||||||
|  |             string? sha256 = null; | ||||||
|  |             long? size = null; | ||||||
|  |  | ||||||
|  |             if (entry.TryGetProperty("document", out var documentElement) && documentElement.ValueKind == JsonValueKind.Object) | ||||||
|  |             { | ||||||
|  |                 if (documentElement.TryGetProperty("url", out var urlElement) && urlElement.ValueKind == JsonValueKind.String && Uri.TryCreate(urlElement.GetString(), UriKind.Absolute, out var parsedUri)) | ||||||
|  |                 { | ||||||
|  |                     documentUri = parsedUri; | ||||||
|  |                 } | ||||||
|  |  | ||||||
|  |                 if (documentElement.TryGetProperty("sha256", out var hashElement) && hashElement.ValueKind == JsonValueKind.String) | ||||||
|  |                 { | ||||||
|  |                     sha256 = hashElement.GetString(); | ||||||
|  |                 } | ||||||
|  |  | ||||||
|  |                 if (documentElement.TryGetProperty("size", out var sizeElement) && sizeElement.ValueKind == JsonValueKind.Number && sizeElement.TryGetInt64(out var parsedSize)) | ||||||
|  |                 { | ||||||
|  |                     size = parsedSize; | ||||||
|  |                 } | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             if (documentUri is null) | ||||||
|  |             { | ||||||
|  |                 continue; | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             builder.Add(new OracleCatalogEntry(id!, title!, documentUri, publishedAt, revision, sha256, size, products)); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         return builder.ToImmutable(); | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     private ImmutableArray<OracleCpuRelease> ParseSchedule(JsonElement root) | ||||||
|  |     { | ||||||
|  |         if (!root.TryGetProperty("schedule", out var scheduleElement) || scheduleElement.ValueKind is not JsonValueKind.Array) | ||||||
|  |         { | ||||||
|  |             return ImmutableArray<OracleCpuRelease>.Empty; | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         var builder = ImmutableArray.CreateBuilder<OracleCpuRelease>(); | ||||||
|  |         foreach (var item in scheduleElement.EnumerateArray()) | ||||||
|  |         { | ||||||
|  |             var window = item.TryGetProperty("window", out var windowElement) && windowElement.ValueKind == JsonValueKind.String ? windowElement.GetString() : null; | ||||||
|  |             if (string.IsNullOrWhiteSpace(window)) | ||||||
|  |             { | ||||||
|  |                 continue; | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             DateTimeOffset releaseDate = default; | ||||||
|  |             if (item.TryGetProperty("releaseDate", out var releaseElement) && releaseElement.ValueKind == JsonValueKind.String && DateTimeOffset.TryParse(releaseElement.GetString(), out var parsed)) | ||||||
|  |             { | ||||||
|  |                 releaseDate = parsed; | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             builder.Add(new OracleCpuRelease(window!, releaseDate)); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         return builder.ToImmutable(); | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     private ImmutableArray<OracleCpuRelease> MergeSchedule(ImmutableArray<OracleCpuRelease> existing, string calendarPayload) | ||||||
|  |     { | ||||||
|  |         try | ||||||
|  |         { | ||||||
|  |             using var document = JsonDocument.Parse(calendarPayload); | ||||||
|  |             var root = document.RootElement; | ||||||
|  |             if (!root.TryGetProperty("cpuWindows", out var windowsElement) || windowsElement.ValueKind is not JsonValueKind.Array) | ||||||
|  |             { | ||||||
|  |                 return existing; | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             var builder = existing.ToBuilder(); | ||||||
|  |             var known = new HashSet<string>(StringComparer.OrdinalIgnoreCase); | ||||||
|  |             foreach (var item in builder) | ||||||
|  |             { | ||||||
|  |                 known.Add(item.Window); | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             foreach (var windowElement in windowsElement.EnumerateArray()) | ||||||
|  |             { | ||||||
|  |                 var name = windowElement.TryGetProperty("name", out var nameElement) && nameElement.ValueKind == JsonValueKind.String ? nameElement.GetString() : null; | ||||||
|  |                 if (string.IsNullOrWhiteSpace(name)) | ||||||
|  |                 { | ||||||
|  |                     continue; | ||||||
|  |                 } | ||||||
|  |  | ||||||
|  |                 if (!known.Add(name)) | ||||||
|  |                 { | ||||||
|  |                     continue; | ||||||
|  |                 } | ||||||
|  |  | ||||||
|  |                 DateTimeOffset releaseDate = default; | ||||||
|  |                 if (windowElement.TryGetProperty("releaseDate", out var releaseElement) && releaseElement.ValueKind == JsonValueKind.String && DateTimeOffset.TryParse(releaseElement.GetString(), out var parsed)) | ||||||
|  |                 { | ||||||
|  |                     releaseDate = parsed; | ||||||
|  |                 } | ||||||
|  |  | ||||||
|  |                 builder.Add(new OracleCpuRelease(name!, releaseDate)); | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             return builder.ToImmutable(); | ||||||
|  |         } | ||||||
|  |         catch (Exception ex) when (ex is not OperationCanceledException) | ||||||
|  |         { | ||||||
|  |             _logger.LogWarning(ex, "Failed to parse Oracle CPU calendar payload; continuing with existing schedule data."); | ||||||
|  |             return existing; | ||||||
|  |         } | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     private ImmutableArray<string> ParseStringArray(JsonElement element) | ||||||
|  |     { | ||||||
|  |         if (element.ValueKind is not JsonValueKind.Array) | ||||||
|  |         { | ||||||
|  |             return ImmutableArray<string>.Empty; | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         var builder = ImmutableArray.CreateBuilder<string>(); | ||||||
|  |         foreach (var item in element.EnumerateArray()) | ||||||
|  |         { | ||||||
|  |             if (item.ValueKind == JsonValueKind.String) | ||||||
|  |             { | ||||||
|  |                 var value = item.GetString(); | ||||||
|  |                 if (!string.IsNullOrWhiteSpace(value)) | ||||||
|  |                 { | ||||||
|  |                     builder.Add(value); | ||||||
|  |                 } | ||||||
|  |             } | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         return builder.ToImmutable(); | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     private void PersistSnapshotIfNeeded(OracleConnectorOptions options, OracleCatalogMetadata metadata, DateTimeOffset fetchedAt) | ||||||
|  |     { | ||||||
|  |         if (!options.PersistOfflineSnapshot || string.IsNullOrWhiteSpace(options.OfflineSnapshotPath)) | ||||||
|  |         { | ||||||
|  |             return; | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         try | ||||||
|  |         { | ||||||
|  |             var snapshot = new OracleCatalogSnapshot(metadata, fetchedAt); | ||||||
|  |             var payload = JsonSerializer.Serialize(snapshot, _serializerOptions); | ||||||
|  |             _fileSystem.File.WriteAllText(options.OfflineSnapshotPath, payload); | ||||||
|  |             _logger.LogDebug("Persisted Oracle CSAF catalog snapshot to {Path}.", options.OfflineSnapshotPath); | ||||||
|  |         } | ||||||
|  |         catch (Exception ex) | ||||||
|  |         { | ||||||
|  |             _logger.LogWarning(ex, "Failed to persist Oracle CSAF catalog snapshot to {Path}.", options.OfflineSnapshotPath); | ||||||
|  |         } | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     private static string CreateCacheKey(OracleConnectorOptions options) | ||||||
|  |         => $"{CachePrefix}:{options.CatalogUri}:{options.CpuCalendarUri}"; | ||||||
|  |  | ||||||
|  |     private sealed record CacheEntry(OracleCatalogMetadata Metadata, DateTimeOffset FetchedAt, DateTimeOffset CachedAt, TimeSpan MetadataCacheDuration, bool FromOfflineSnapshot) | ||||||
|  |     { | ||||||
|  |         public bool IsExpired(DateTimeOffset now) | ||||||
|  |             => MetadataCacheDuration > TimeSpan.Zero && now >= CachedAt + MetadataCacheDuration; | ||||||
|  |  | ||||||
|  |         public OracleCatalogResult ToResult(bool fromCache) | ||||||
|  |             => new(Metadata, FetchedAt, fromCache, FromOfflineSnapshot); | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     private sealed record OracleCatalogSnapshot(OracleCatalogMetadata Metadata, DateTimeOffset FetchedAt); | ||||||
|  | } | ||||||
|  |  | ||||||
|  | public sealed record OracleCatalogMetadata( | ||||||
|  |     DateTimeOffset GeneratedAt, | ||||||
|  |     ImmutableArray<OracleCatalogEntry> Entries, | ||||||
|  |     ImmutableArray<OracleCpuRelease> CpuSchedule); | ||||||
|  |  | ||||||
|  | public sealed record OracleCatalogEntry( | ||||||
|  |     string Id, | ||||||
|  |     string Title, | ||||||
|  |     Uri DocumentUri, | ||||||
|  |     DateTimeOffset PublishedAt, | ||||||
|  |     string? Revision, | ||||||
|  |     string? Sha256, | ||||||
|  |     long? Size, | ||||||
|  |     ImmutableArray<string> Products); | ||||||
|  |  | ||||||
|  | public sealed record OracleCpuRelease(string Window, DateTimeOffset ReleaseDate); | ||||||
|  |  | ||||||
|  | public sealed record OracleCatalogResult( | ||||||
|  |     OracleCatalogMetadata Metadata, | ||||||
|  |     DateTimeOffset FetchedAt, | ||||||
|  |     bool FromCache, | ||||||
|  |     bool FromOfflineSnapshot); | ||||||
| @@ -0,0 +1,81 @@ | |||||||
|  | using System.Collections.Generic; | ||||||
|  | using System.Collections.Immutable; | ||||||
|  | using System.Runtime.CompilerServices; | ||||||
|  | using Microsoft.Extensions.Logging; | ||||||
|  | using StellaOps.Vexer.Connectors.Abstractions; | ||||||
|  | using StellaOps.Vexer.Connectors.Oracle.CSAF.Configuration; | ||||||
|  | using StellaOps.Vexer.Connectors.Oracle.CSAF.Metadata; | ||||||
|  | using StellaOps.Vexer.Core; | ||||||
|  |  | ||||||
|  | namespace StellaOps.Vexer.Connectors.Oracle.CSAF; | ||||||
|  |  | ||||||
|  | public sealed class OracleCsafConnector : VexConnectorBase | ||||||
|  | { | ||||||
|  |     private static readonly VexConnectorDescriptor DescriptorInstance = new( | ||||||
|  |         id: "vexer:oracle", | ||||||
|  |         kind: VexProviderKind.Vendor, | ||||||
|  |         displayName: "Oracle CSAF") | ||||||
|  |     { | ||||||
|  |         Tags = ImmutableArray.Create("oracle", "csaf", "cpu"), | ||||||
|  |     }; | ||||||
|  |  | ||||||
|  |     private readonly OracleCatalogLoader _catalogLoader; | ||||||
|  |     private readonly IEnumerable<IVexConnectorOptionsValidator<OracleConnectorOptions>> _validators; | ||||||
|  |  | ||||||
|  |     private OracleConnectorOptions? _options; | ||||||
|  |     private OracleCatalogResult? _catalog; | ||||||
|  |  | ||||||
|  |     public OracleCsafConnector( | ||||||
|  |         OracleCatalogLoader catalogLoader, | ||||||
|  |         IEnumerable<IVexConnectorOptionsValidator<OracleConnectorOptions>> validators, | ||||||
|  |         ILogger<OracleCsafConnector> logger, | ||||||
|  |         TimeProvider timeProvider) | ||||||
|  |         : base(DescriptorInstance, logger, timeProvider) | ||||||
|  |     { | ||||||
|  |         _catalogLoader = catalogLoader ?? throw new ArgumentNullException(nameof(catalogLoader)); | ||||||
|  |         _validators = validators ?? Array.Empty<IVexConnectorOptionsValidator<OracleConnectorOptions>>(); | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     public override async ValueTask ValidateAsync(VexConnectorSettings settings, CancellationToken cancellationToken) | ||||||
|  |     { | ||||||
|  |         _options = VexConnectorOptionsBinder.Bind( | ||||||
|  |             Descriptor, | ||||||
|  |             settings, | ||||||
|  |             validators: _validators); | ||||||
|  |  | ||||||
|  |         _catalog = await _catalogLoader.LoadAsync(_options, cancellationToken).ConfigureAwait(false); | ||||||
|  |         LogConnectorEvent(LogLevel.Information, "validate", "Oracle CSAF catalogue loaded.", new Dictionary<string, object?> | ||||||
|  |         { | ||||||
|  |             ["catalogEntryCount"] = _catalog.Metadata.Entries.Length, | ||||||
|  |             ["scheduleCount"] = _catalog.Metadata.CpuSchedule.Length, | ||||||
|  |             ["fromOffline"] = _catalog.FromOfflineSnapshot, | ||||||
|  |         }); | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     public override async IAsyncEnumerable<VexRawDocument> FetchAsync(VexConnectorContext context, [EnumeratorCancellation] CancellationToken cancellationToken) | ||||||
|  |     { | ||||||
|  |         ArgumentNullException.ThrowIfNull(context); | ||||||
|  |  | ||||||
|  |         if (_options is null) | ||||||
|  |         { | ||||||
|  |             throw new InvalidOperationException("Connector must be validated before fetch operations."); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         if (_catalog is null) | ||||||
|  |         { | ||||||
|  |             _catalog = await _catalogLoader.LoadAsync(_options, cancellationToken).ConfigureAwait(false); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         LogConnectorEvent(LogLevel.Debug, "fetch", "Oracle CSAF discovery ready; document ingestion handled by follow-up task.", new Dictionary<string, object?> | ||||||
|  |         { | ||||||
|  |             ["since"] = context.Since?.ToString("O"), | ||||||
|  |         }); | ||||||
|  |  | ||||||
|  |         yield break; | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     public override ValueTask<VexClaimBatch> NormalizeAsync(VexRawDocument document, CancellationToken cancellationToken) | ||||||
|  |         => throw new NotSupportedException("OracleCsafConnector relies on dedicated CSAF normalizers."); | ||||||
|  |  | ||||||
|  |     public OracleCatalogResult? GetCachedCatalog() => _catalog; | ||||||
|  | } | ||||||
| @@ -0,0 +1,18 @@ | |||||||
|  | <Project Sdk="Microsoft.NET.Sdk"> | ||||||
|  |   <PropertyGroup> | ||||||
|  |     <TargetFramework>net10.0</TargetFramework> | ||||||
|  |     <LangVersion>preview</LangVersion> | ||||||
|  |     <Nullable>enable</Nullable> | ||||||
|  |     <ImplicitUsings>enable</ImplicitUsings> | ||||||
|  |     <TreatWarningsAsErrors>true</TreatWarningsAsErrors> | ||||||
|  |   </PropertyGroup> | ||||||
|  |   <ItemGroup> | ||||||
|  |     <ProjectReference Include="..\StellaOps.Vexer.Connectors.Abstractions\StellaOps.Vexer.Connectors.Abstractions.csproj" /> | ||||||
|  |   </ItemGroup> | ||||||
|  |   <ItemGroup> | ||||||
|  |     <PackageReference Include="Microsoft.Extensions.Caching.Memory" Version="8.0.0" /> | ||||||
|  |     <PackageReference Include="Microsoft.Extensions.Http" Version="8.0.0" /> | ||||||
|  |     <PackageReference Include="Microsoft.Extensions.Options" Version="8.0.0" /> | ||||||
|  |     <PackageReference Include="System.IO.Abstractions" Version="20.0.28" /> | ||||||
|  |   </ItemGroup> | ||||||
|  | </Project> | ||||||
| @@ -2,6 +2,6 @@ If you are working on this file you need to read docs/ARCHITECTURE_VEXER.md and | |||||||
| # TASKS | # TASKS | ||||||
| | Task | Owner(s) | Depends on | Notes | | | Task | Owner(s) | Depends on | Notes | | ||||||
| |---|---|---|---| | |---|---|---|---| | ||||||
| |VEXER-CONN-ORACLE-01-001 – Oracle CSAF catalogue discovery|Team Vexer Connectors – Oracle|VEXER-CONN-ABS-01-001|TODO – Implement catalogue discovery, CPU calendar awareness, and offline snapshot import for Oracle CSAF feeds.| | |VEXER-CONN-ORACLE-01-001 – Oracle CSAF catalogue discovery|Team Vexer Connectors – Oracle|VEXER-CONN-ABS-01-001|DOING (2025-10-17) – Implement catalogue discovery, CPU calendar awareness, and offline snapshot import for Oracle CSAF feeds.| | ||||||
| |VEXER-CONN-ORACLE-01-002 – CSAF download & dedupe pipeline|Team Vexer Connectors – Oracle|VEXER-CONN-ORACLE-01-001, VEXER-STORAGE-01-003|TODO – Fetch CSAF documents with retry/backoff, checksum validation, revision deduplication, and raw persistence.| | |VEXER-CONN-ORACLE-01-002 – CSAF download & dedupe pipeline|Team Vexer Connectors – Oracle|VEXER-CONN-ORACLE-01-001, VEXER-STORAGE-01-003|TODO – Fetch CSAF documents with retry/backoff, checksum validation, revision deduplication, and raw persistence.| | ||||||
| |VEXER-CONN-ORACLE-01-003 – Trust metadata + provenance|Team Vexer Connectors – Oracle|VEXER-CONN-ORACLE-01-002, VEXER-POLICY-01-001|TODO – Emit Oracle signing metadata (PGP/cosign) and provenance hints for consensus weighting.| | |VEXER-CONN-ORACLE-01-003 – Trust metadata + provenance|Team Vexer Connectors – Oracle|VEXER-CONN-ORACLE-01-002, VEXER-POLICY-01-001|TODO – Emit Oracle signing metadata (PGP/cosign) and provenance hints for consensus weighting.| | ||||||
|   | |||||||
| @@ -0,0 +1,277 @@ | |||||||
|  | using System.Collections.Generic; | ||||||
|  | using System.Collections.Immutable; | ||||||
|  | using System.Net; | ||||||
|  | using System.Net.Http; | ||||||
|  | using System.Text; | ||||||
|  | using FluentAssertions; | ||||||
|  | using Microsoft.Extensions.Caching.Memory; | ||||||
|  | using Microsoft.Extensions.DependencyInjection; | ||||||
|  | using Microsoft.Extensions.Logging.Abstractions; | ||||||
|  | using Microsoft.Extensions.Options; | ||||||
|  | using StellaOps.Vexer.Connectors.Abstractions; | ||||||
|  | using StellaOps.Vexer.Connectors.RedHat.CSAF.Configuration; | ||||||
|  | using StellaOps.Vexer.Connectors.RedHat.CSAF.Metadata; | ||||||
|  | using StellaOps.Vexer.Core; | ||||||
|  | using StellaOps.Vexer.Storage.Mongo; | ||||||
|  |  | ||||||
|  | namespace StellaOps.Vexer.Connectors.RedHat.CSAF.Tests.Connectors; | ||||||
|  |  | ||||||
|  | public sealed class RedHatCsafConnectorTests | ||||||
|  | { | ||||||
|  |     private static readonly VexConnectorDescriptor Descriptor = new("vexer:redhat", VexProviderKind.Distro, "Red Hat CSAF"); | ||||||
|  |  | ||||||
|  |     [Fact] | ||||||
|  |     public async Task FetchAsync_EmitsDocumentsAfterSince() | ||||||
|  |     { | ||||||
|  |         var metadata = """ | ||||||
|  |             { | ||||||
|  |               "metadata": { | ||||||
|  |                 "provider": { "name": "Red Hat Product Security" } | ||||||
|  |               }, | ||||||
|  |               "distributions": [ | ||||||
|  |                 { "directory": "https://example.com/security/data/csaf/v2/advisories/" } | ||||||
|  |               ], | ||||||
|  |               "rolie": { | ||||||
|  |                 "feeds": [ | ||||||
|  |                   { "url": "https://example.com/security/data/csaf/v2/advisories/rolie/feed.atom" } | ||||||
|  |                 ] | ||||||
|  |               } | ||||||
|  |             } | ||||||
|  |             """; | ||||||
|  |  | ||||||
|  |         var feed = """ | ||||||
|  |             <feed xmlns="http://www.w3.org/2005/Atom"> | ||||||
|  |               <entry> | ||||||
|  |                 <id>urn:redhat:1</id> | ||||||
|  |                 <updated>2025-10-16T10:00:00Z</updated> | ||||||
|  |                 <link href="https://example.com/doc1.json" rel="enclosure" /> | ||||||
|  |             </entry> | ||||||
|  |             <entry> | ||||||
|  |                 <id>urn:redhat:2</id> | ||||||
|  |                 <updated>2025-10-17T10:00:00Z</updated> | ||||||
|  |                 <link href="https://example.com/doc2.json" rel="enclosure" /> | ||||||
|  |               </entry> | ||||||
|  |             </feed> | ||||||
|  |             """; | ||||||
|  |  | ||||||
|  |         var handler = TestHttpMessageHandler.Create( | ||||||
|  |             request => Response(HttpStatusCode.OK, metadata, "application/json"), | ||||||
|  |             request => Response(HttpStatusCode.OK, feed, "application/atom+xml"), | ||||||
|  |             request => Response(HttpStatusCode.OK, "{ \"csaf\": 1 }", "application/json")); | ||||||
|  |  | ||||||
|  |         var httpClient = new HttpClient(handler) | ||||||
|  |         { | ||||||
|  |             BaseAddress = new Uri("https://example.com/"), | ||||||
|  |         }; | ||||||
|  |  | ||||||
|  |         var factory = new SingleClientHttpClientFactory(httpClient); | ||||||
|  |         var cache = new MemoryCache(new MemoryCacheOptions()); | ||||||
|  |         var options = Options.Create(new RedHatConnectorOptions()); | ||||||
|  |         var metadataLoader = new RedHatProviderMetadataLoader(factory, cache, options, NullLogger<RedHatProviderMetadataLoader>.Instance); | ||||||
|  |         var stateRepository = new InMemoryConnectorStateRepository(); | ||||||
|  |         var connector = new RedHatCsafConnector(Descriptor, metadataLoader, factory, stateRepository, NullLogger<RedHatCsafConnector>.Instance, TimeProvider.System); | ||||||
|  |  | ||||||
|  |         var rawSink = new CapturingRawSink(); | ||||||
|  |         var context = new VexConnectorContext( | ||||||
|  |             new DateTimeOffset(2025, 10, 16, 12, 0, 0, TimeSpan.Zero), | ||||||
|  |             VexConnectorSettings.Empty, | ||||||
|  |             rawSink, | ||||||
|  |             new NoopSignatureVerifier(), | ||||||
|  |             new NoopNormalizerRouter(), | ||||||
|  |             new ServiceCollection().BuildServiceProvider()); | ||||||
|  |  | ||||||
|  |         var results = new List<VexRawDocument>(); | ||||||
|  |         await foreach (var document in connector.FetchAsync(context, CancellationToken.None)) | ||||||
|  |         { | ||||||
|  |             results.Add(document); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         Assert.Single(results); | ||||||
|  |         Assert.Single(rawSink.Documents); | ||||||
|  |         Assert.Equal("https://example.com/doc2.json", results[0].SourceUri.ToString()); | ||||||
|  |         Assert.Equal("https://example.com/doc2.json", rawSink.Documents[0].SourceUri.ToString()); | ||||||
|  |         Assert.Equal(3, handler.CallCount); | ||||||
|  |         stateRepository.State.Should().NotBeNull(); | ||||||
|  |         stateRepository.State!.LastUpdated.Should().Be(new DateTimeOffset(2025, 10, 17, 10, 0, 0, TimeSpan.Zero)); | ||||||
|  |         stateRepository.State.DocumentDigests.Should().HaveCount(1); | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     [Fact] | ||||||
|  |     public async Task FetchAsync_UsesStateToSkipDuplicateDocuments() | ||||||
|  |     { | ||||||
|  |         var metadata = """ | ||||||
|  |             { | ||||||
|  |               "metadata": { | ||||||
|  |                 "provider": { "name": "Red Hat Product Security" } | ||||||
|  |               }, | ||||||
|  |               "distributions": [ | ||||||
|  |                 { "directory": "https://example.com/security/data/csaf/v2/advisories/" } | ||||||
|  |               ], | ||||||
|  |               "rolie": { | ||||||
|  |                 "feeds": [ | ||||||
|  |                   { "url": "https://example.com/security/data/csaf/v2/advisories/rolie/feed.atom" } | ||||||
|  |                 ] | ||||||
|  |               } | ||||||
|  |             } | ||||||
|  |             """; | ||||||
|  |  | ||||||
|  |         var feed = """ | ||||||
|  |             <feed xmlns="http://www.w3.org/2005/Atom"> | ||||||
|  |               <entry> | ||||||
|  |                 <id>urn:redhat:1</id> | ||||||
|  |                 <updated>2025-10-17T10:00:00Z</updated> | ||||||
|  |                 <link href="https://example.com/doc1.json" rel="enclosure" /> | ||||||
|  |               </entry> | ||||||
|  |             </feed> | ||||||
|  |             """; | ||||||
|  |  | ||||||
|  |         var handler1 = TestHttpMessageHandler.Create( | ||||||
|  |             _ => Response(HttpStatusCode.OK, metadata, "application/json"), | ||||||
|  |             _ => Response(HttpStatusCode.OK, feed, "application/atom+xml"), | ||||||
|  |             _ => Response(HttpStatusCode.OK, "{ \"csaf\": 1 }", "application/json")); | ||||||
|  |  | ||||||
|  |         var stateRepository = new InMemoryConnectorStateRepository(); | ||||||
|  |         await ExecuteFetchAsync(handler1, stateRepository); | ||||||
|  |  | ||||||
|  |         stateRepository.State.Should().NotBeNull(); | ||||||
|  |         var previousState = stateRepository.State!; | ||||||
|  |  | ||||||
|  |         var handler2 = TestHttpMessageHandler.Create( | ||||||
|  |             _ => Response(HttpStatusCode.OK, metadata, "application/json"), | ||||||
|  |             _ => Response(HttpStatusCode.OK, feed, "application/atom+xml"), | ||||||
|  |             _ => Response(HttpStatusCode.OK, "{ \"csaf\": 1 }", "application/json")); | ||||||
|  |  | ||||||
|  |         var (results, rawSink) = await ExecuteFetchAsync(handler2, stateRepository); | ||||||
|  |  | ||||||
|  |         results.Should().BeEmpty(); | ||||||
|  |         rawSink.Documents.Should().BeEmpty(); | ||||||
|  |         stateRepository.State!.DocumentDigests.Should().Equal(previousState.DocumentDigests); | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     private static HttpResponseMessage Response(HttpStatusCode statusCode, string content, string contentType) | ||||||
|  |         => new(statusCode) | ||||||
|  |         { | ||||||
|  |             Content = new StringContent(content, Encoding.UTF8, contentType), | ||||||
|  |         }; | ||||||
|  |  | ||||||
|  |     private sealed class CapturingRawSink : IVexRawDocumentSink | ||||||
|  |     { | ||||||
|  |         public List<VexRawDocument> Documents { get; } = new(); | ||||||
|  |  | ||||||
|  |         public ValueTask StoreAsync(VexRawDocument document, CancellationToken cancellationToken) | ||||||
|  |         { | ||||||
|  |             Documents.Add(document); | ||||||
|  |             return ValueTask.CompletedTask; | ||||||
|  |         } | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     private sealed class NoopSignatureVerifier : IVexSignatureVerifier | ||||||
|  |     { | ||||||
|  |         public ValueTask<VexSignatureMetadata?> VerifyAsync(VexRawDocument document, CancellationToken cancellationToken) | ||||||
|  |             => ValueTask.FromResult<VexSignatureMetadata?>(null); | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     private sealed class NoopNormalizerRouter : IVexNormalizerRouter | ||||||
|  |     { | ||||||
|  |         public ValueTask<VexClaimBatch> NormalizeAsync(VexRawDocument document, CancellationToken cancellationToken) | ||||||
|  |             => ValueTask.FromResult(new VexClaimBatch(document, ImmutableArray<VexClaim>.Empty, ImmutableDictionary<string, string>.Empty)); | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     private sealed class SingleClientHttpClientFactory : IHttpClientFactory | ||||||
|  |     { | ||||||
|  |         private readonly HttpClient _client; | ||||||
|  |  | ||||||
|  |         public SingleClientHttpClientFactory(HttpClient client) | ||||||
|  |         { | ||||||
|  |             _client = client; | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         public HttpClient CreateClient(string name) => _client; | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     private sealed class TestHttpMessageHandler : HttpMessageHandler | ||||||
|  |     { | ||||||
|  |         private readonly Queue<Func<HttpRequestMessage, HttpResponseMessage>> _responders; | ||||||
|  |  | ||||||
|  |         private TestHttpMessageHandler(IEnumerable<Func<HttpRequestMessage, HttpResponseMessage>> responders) | ||||||
|  |         { | ||||||
|  |             _responders = new Queue<Func<HttpRequestMessage, HttpResponseMessage>>(responders); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         public int CallCount { get; private set; } | ||||||
|  |  | ||||||
|  |         public static TestHttpMessageHandler Create(params Func<HttpRequestMessage, HttpResponseMessage>[] responders) | ||||||
|  |             => new(responders); | ||||||
|  |  | ||||||
|  |         protected override Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) | ||||||
|  |         { | ||||||
|  |             CallCount++; | ||||||
|  |             if (_responders.Count == 0) | ||||||
|  |             { | ||||||
|  |                 throw new InvalidOperationException("No responder configured for request."); | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             var responder = _responders.Count > 1 | ||||||
|  |                 ? _responders.Dequeue() | ||||||
|  |                 : _responders.Peek(); | ||||||
|  |  | ||||||
|  |             var response = responder(request); | ||||||
|  |             response.RequestMessage = request; | ||||||
|  |             return Task.FromResult(response); | ||||||
|  |         } | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     private static async Task<(List<VexRawDocument> Documents, CapturingRawSink Sink)> ExecuteFetchAsync( | ||||||
|  |         TestHttpMessageHandler handler, | ||||||
|  |         InMemoryConnectorStateRepository stateRepository) | ||||||
|  |     { | ||||||
|  |         var httpClient = new HttpClient(handler) | ||||||
|  |         { | ||||||
|  |             BaseAddress = new Uri("https://example.com/"), | ||||||
|  |         }; | ||||||
|  |  | ||||||
|  |         var factory = new SingleClientHttpClientFactory(httpClient); | ||||||
|  |         var cache = new MemoryCache(new MemoryCacheOptions()); | ||||||
|  |         var options = Options.Create(new RedHatConnectorOptions()); | ||||||
|  |         var metadataLoader = new RedHatProviderMetadataLoader(factory, cache, options, NullLogger<RedHatProviderMetadataLoader>.Instance); | ||||||
|  |         var connector = new RedHatCsafConnector(Descriptor, metadataLoader, factory, stateRepository, NullLogger<RedHatCsafConnector>.Instance, TimeProvider.System); | ||||||
|  |  | ||||||
|  |         var rawSink = new CapturingRawSink(); | ||||||
|  |         var context = new VexConnectorContext( | ||||||
|  |             null, | ||||||
|  |             VexConnectorSettings.Empty, | ||||||
|  |             rawSink, | ||||||
|  |             new NoopSignatureVerifier(), | ||||||
|  |             new NoopNormalizerRouter(), | ||||||
|  |             new ServiceCollection().BuildServiceProvider()); | ||||||
|  |  | ||||||
|  |         var documents = new List<VexRawDocument>(); | ||||||
|  |         await foreach (var document in connector.FetchAsync(context, CancellationToken.None)) | ||||||
|  |         { | ||||||
|  |             documents.Add(document); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         return (documents, rawSink); | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     private sealed class InMemoryConnectorStateRepository : IVexConnectorStateRepository | ||||||
|  |     { | ||||||
|  |         public VexConnectorState? State { get; private set; } | ||||||
|  |  | ||||||
|  |         public ValueTask<VexConnectorState?> GetAsync(string connectorId, CancellationToken cancellationToken) | ||||||
|  |         { | ||||||
|  |             if (State is not null && string.Equals(State.ConnectorId, connectorId, StringComparison.OrdinalIgnoreCase)) | ||||||
|  |             { | ||||||
|  |                 return ValueTask.FromResult<VexConnectorState?>(State); | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             return ValueTask.FromResult<VexConnectorState?>(null); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         public ValueTask SaveAsync(VexConnectorState state, CancellationToken cancellationToken) | ||||||
|  |         { | ||||||
|  |             State = state; | ||||||
|  |             return ValueTask.CompletedTask; | ||||||
|  |         } | ||||||
|  |     } | ||||||
|  | } | ||||||
| @@ -0,0 +1,235 @@ | |||||||
|  | using System.Collections.Generic; | ||||||
|  | using System.Net; | ||||||
|  | using System.Net.Http; | ||||||
|  | using System.Net.Http.Headers; | ||||||
|  | using System.Text; | ||||||
|  | using Microsoft.Extensions.Caching.Memory; | ||||||
|  | using Microsoft.Extensions.Logging.Abstractions; | ||||||
|  | using Microsoft.Extensions.Options; | ||||||
|  | using StellaOps.Vexer.Connectors.RedHat.CSAF.Configuration; | ||||||
|  | using StellaOps.Vexer.Connectors.RedHat.CSAF.Metadata; | ||||||
|  | using System.IO.Abstractions.TestingHelpers; | ||||||
|  |  | ||||||
|  | namespace StellaOps.Vexer.Connectors.RedHat.CSAF.Tests.Metadata; | ||||||
|  |  | ||||||
|  | public sealed class RedHatProviderMetadataLoaderTests | ||||||
|  | { | ||||||
|  |     private const string SampleJson = """ | ||||||
|  |         { | ||||||
|  |           "metadata": { | ||||||
|  |             "provider": { | ||||||
|  |               "name": "Red Hat Product Security" | ||||||
|  |             } | ||||||
|  |           }, | ||||||
|  |           "distributions": [ | ||||||
|  |             { "directory": "https://access.redhat.com/security/data/csaf/v2/advisories/" } | ||||||
|  |           ], | ||||||
|  |           "rolie": { | ||||||
|  |             "feeds": [ | ||||||
|  |               { "url": "https://access.redhat.com/security/data/csaf/v2/advisories/rolie/feed.atom" } | ||||||
|  |             ] | ||||||
|  |           } | ||||||
|  |         } | ||||||
|  |         """; | ||||||
|  |  | ||||||
|  |     [Fact] | ||||||
|  |     public async Task LoadAsync_FetchesMetadataAndCaches() | ||||||
|  |     { | ||||||
|  |         var handler = TestHttpMessageHandler.RespondWith(_ => | ||||||
|  |         { | ||||||
|  |             var response = new HttpResponseMessage(HttpStatusCode.OK) | ||||||
|  |             { | ||||||
|  |                 Content = new StringContent(SampleJson, Encoding.UTF8, "application/json"), | ||||||
|  |             }; | ||||||
|  |             response.Headers.ETag = new EntityTagHeaderValue("\"abc\""); | ||||||
|  |             return response; | ||||||
|  |         }); | ||||||
|  |  | ||||||
|  |         var httpClient = new HttpClient(handler) | ||||||
|  |         { | ||||||
|  |             BaseAddress = new Uri("https://access.redhat.com/"), | ||||||
|  |         }; | ||||||
|  |  | ||||||
|  |         var factory = new SingleClientHttpClientFactory(httpClient); | ||||||
|  |         var cache = new MemoryCache(new MemoryCacheOptions()); | ||||||
|  |         var fileSystem = new MockFileSystem(); | ||||||
|  |         var options = new RedHatConnectorOptions | ||||||
|  |         { | ||||||
|  |             MetadataUri = new Uri("https://access.redhat.com/.well-known/csaf/provider-metadata.json"), | ||||||
|  |             OfflineSnapshotPath = fileSystem.Path.Combine("/offline", "redhat-provider.json"), | ||||||
|  |             CosignIssuer = "https://sigstore.dev/redhat", | ||||||
|  |             CosignIdentityPattern = "^spiffe://redhat/.+$", | ||||||
|  |         }; | ||||||
|  |         options.PgpFingerprints.Add("A1B2C3D4E5F6"); | ||||||
|  |         options.Validate(fileSystem); | ||||||
|  |  | ||||||
|  |         var loader = new RedHatProviderMetadataLoader(factory, cache, Options.Create(options), NullLogger<RedHatProviderMetadataLoader>.Instance, fileSystem); | ||||||
|  |  | ||||||
|  |         var result = await loader.LoadAsync(CancellationToken.None); | ||||||
|  |  | ||||||
|  |         Assert.Equal("Red Hat Product Security", result.Provider.DisplayName); | ||||||
|  |         Assert.False(result.FromCache); | ||||||
|  |         Assert.False(result.FromOfflineSnapshot); | ||||||
|  |         Assert.Single(result.Provider.BaseUris); | ||||||
|  |         Assert.Equal("https://access.redhat.com/security/data/csaf/v2/advisories/", result.Provider.BaseUris[0].ToString()); | ||||||
|  |         Assert.Equal("https://access.redhat.com/.well-known/csaf/provider-metadata.json", result.Provider.Discovery.WellKnownMetadata?.ToString()); | ||||||
|  |         Assert.Equal("https://access.redhat.com/security/data/csaf/v2/advisories/rolie/feed.atom", result.Provider.Discovery.RolIeService?.ToString()); | ||||||
|  |         Assert.Equal(1.0, result.Provider.Trust.Weight); | ||||||
|  |         Assert.NotNull(result.Provider.Trust.Cosign); | ||||||
|  |         Assert.Equal("https://sigstore.dev/redhat", result.Provider.Trust.Cosign!.Issuer); | ||||||
|  |         Assert.Equal("^spiffe://redhat/.+$", result.Provider.Trust.Cosign.IdentityPattern); | ||||||
|  |         Assert.Contains("A1B2C3D4E5F6", result.Provider.Trust.PgpFingerprints); | ||||||
|  |         Assert.True(fileSystem.FileExists(options.OfflineSnapshotPath)); | ||||||
|  |         Assert.Equal(1, handler.CallCount); | ||||||
|  |  | ||||||
|  |         var second = await loader.LoadAsync(CancellationToken.None); | ||||||
|  |         Assert.True(second.FromCache); | ||||||
|  |         Assert.False(second.FromOfflineSnapshot); | ||||||
|  |         Assert.Equal(1, handler.CallCount); | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     [Fact] | ||||||
|  |     public async Task LoadAsync_UsesOfflineSnapshotWhenPreferred() | ||||||
|  |     { | ||||||
|  |         var handler = TestHttpMessageHandler.RespondWith(_ => throw new InvalidOperationException("HTTP should not be called")); | ||||||
|  |  | ||||||
|  |         var httpClient = new HttpClient(handler); | ||||||
|  |         var factory = new SingleClientHttpClientFactory(httpClient); | ||||||
|  |         var cache = new MemoryCache(new MemoryCacheOptions()); | ||||||
|  |  | ||||||
|  |         var fileSystem = new MockFileSystem(new Dictionary<string, MockFileData> | ||||||
|  |         { | ||||||
|  |             ["/snapshots/redhat.json"] = new MockFileData(SampleJson), | ||||||
|  |         }); | ||||||
|  |  | ||||||
|  |         var options = new RedHatConnectorOptions | ||||||
|  |         { | ||||||
|  |             MetadataUri = new Uri("https://access.redhat.com/.well-known/csaf/provider-metadata.json"), | ||||||
|  |             OfflineSnapshotPath = "/snapshots/redhat.json", | ||||||
|  |             PreferOfflineSnapshot = true, | ||||||
|  |             PersistOfflineSnapshot = false, | ||||||
|  |         }; | ||||||
|  |         options.Validate(fileSystem); | ||||||
|  |  | ||||||
|  |         var loader = new RedHatProviderMetadataLoader(factory, cache, Options.Create(options), NullLogger<RedHatProviderMetadataLoader>.Instance, fileSystem); | ||||||
|  |  | ||||||
|  |         var result = await loader.LoadAsync(CancellationToken.None); | ||||||
|  |  | ||||||
|  |         Assert.Equal("Red Hat Product Security", result.Provider.DisplayName); | ||||||
|  |         Assert.False(result.FromCache); | ||||||
|  |         Assert.True(result.FromOfflineSnapshot); | ||||||
|  |         Assert.Equal(0, handler.CallCount); | ||||||
|  |  | ||||||
|  |         var second = await loader.LoadAsync(CancellationToken.None); | ||||||
|  |         Assert.True(second.FromCache); | ||||||
|  |         Assert.True(second.FromOfflineSnapshot); | ||||||
|  |         Assert.Equal(0, handler.CallCount); | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     [Fact] | ||||||
|  |     public async Task LoadAsync_UsesETagForConditionalRequest() | ||||||
|  |     { | ||||||
|  |         var handler = TestHttpMessageHandler.Create( | ||||||
|  |             _ => | ||||||
|  |             { | ||||||
|  |                 var response = new HttpResponseMessage(HttpStatusCode.OK) | ||||||
|  |                 { | ||||||
|  |                     Content = new StringContent(SampleJson, Encoding.UTF8, "application/json"), | ||||||
|  |                 }; | ||||||
|  |                 response.Headers.ETag = new EntityTagHeaderValue("\"abc\""); | ||||||
|  |                 return response; | ||||||
|  |             }, | ||||||
|  |             request => | ||||||
|  |             { | ||||||
|  |                 Assert.Contains(request.Headers.IfNoneMatch, etag => etag.Tag == "\"abc\""); | ||||||
|  |                 return new HttpResponseMessage(HttpStatusCode.NotModified); | ||||||
|  |             }); | ||||||
|  |  | ||||||
|  |         var httpClient = new HttpClient(handler); | ||||||
|  |         var factory = new SingleClientHttpClientFactory(httpClient); | ||||||
|  |         var cache = new MemoryCache(new MemoryCacheOptions()); | ||||||
|  |         var fileSystem = new MockFileSystem(); | ||||||
|  |         var options = new RedHatConnectorOptions | ||||||
|  |         { | ||||||
|  |             MetadataUri = new Uri("https://access.redhat.com/.well-known/csaf/provider-metadata.json"), | ||||||
|  |             OfflineSnapshotPath = "/offline/redhat.json", | ||||||
|  |             MetadataCacheDuration = TimeSpan.FromMinutes(1), | ||||||
|  |         }; | ||||||
|  |         options.Validate(fileSystem); | ||||||
|  |  | ||||||
|  |         var loader = new RedHatProviderMetadataLoader(factory, cache, Options.Create(options), NullLogger<RedHatProviderMetadataLoader>.Instance, fileSystem); | ||||||
|  |  | ||||||
|  |         var first = await loader.LoadAsync(CancellationToken.None); | ||||||
|  |         Assert.False(first.FromCache); | ||||||
|  |         Assert.False(first.FromOfflineSnapshot); | ||||||
|  |  | ||||||
|  |         Assert.True(cache.TryGetValue(RedHatProviderMetadataLoader.CacheKey, out var entryObj)); | ||||||
|  |         Assert.NotNull(entryObj); | ||||||
|  |  | ||||||
|  |         var entryType = entryObj!.GetType(); | ||||||
|  |         var provider = entryType.GetProperty("Provider")!.GetValue(entryObj); | ||||||
|  |         var fetchedAt = entryType.GetProperty("FetchedAt")!.GetValue(entryObj); | ||||||
|  |         var etag = entryType.GetProperty("ETag")!.GetValue(entryObj); | ||||||
|  |         var fromOffline = entryType.GetProperty("FromOffline")!.GetValue(entryObj); | ||||||
|  |  | ||||||
|  |         var expiredEntry = Activator.CreateInstance(entryType, provider, fetchedAt, DateTimeOffset.UtcNow - TimeSpan.FromSeconds(1), etag, fromOffline); | ||||||
|  |         cache.Set(RedHatProviderMetadataLoader.CacheKey, expiredEntry!, new MemoryCacheEntryOptions | ||||||
|  |         { | ||||||
|  |             AbsoluteExpiration = DateTimeOffset.UtcNow + TimeSpan.FromMinutes(1), | ||||||
|  |         }); | ||||||
|  |  | ||||||
|  |         var second = await loader.LoadAsync(CancellationToken.None); | ||||||
|  |  | ||||||
|  |         var third = await loader.LoadAsync(CancellationToken.None); | ||||||
|  |         Assert.True(third.FromCache); | ||||||
|  |  | ||||||
|  |         Assert.Equal(2, handler.CallCount); | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     private sealed class SingleClientHttpClientFactory : IHttpClientFactory | ||||||
|  |     { | ||||||
|  |         private readonly HttpClient _client; | ||||||
|  |  | ||||||
|  |         public SingleClientHttpClientFactory(HttpClient client) | ||||||
|  |         { | ||||||
|  |             _client = client ?? throw new ArgumentNullException(nameof(client)); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         public HttpClient CreateClient(string name) => _client; | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     private sealed class TestHttpMessageHandler : HttpMessageHandler | ||||||
|  |     { | ||||||
|  |         private readonly Queue<Func<HttpRequestMessage, HttpResponseMessage>> _responders; | ||||||
|  |  | ||||||
|  |         private TestHttpMessageHandler(IEnumerable<Func<HttpRequestMessage, HttpResponseMessage>> responders) | ||||||
|  |         { | ||||||
|  |             _responders = new Queue<Func<HttpRequestMessage, HttpResponseMessage>>(responders); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         public int CallCount { get; private set; } | ||||||
|  |  | ||||||
|  |         public static TestHttpMessageHandler RespondWith(Func<HttpRequestMessage, HttpResponseMessage> responder) | ||||||
|  |             => new(new[] { responder }); | ||||||
|  |  | ||||||
|  |         public static TestHttpMessageHandler Create(params Func<HttpRequestMessage, HttpResponseMessage>[] responders) | ||||||
|  |             => new(responders); | ||||||
|  |  | ||||||
|  |         protected override Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) | ||||||
|  |         { | ||||||
|  |             CallCount++; | ||||||
|  |             if (_responders.Count == 0) | ||||||
|  |             { | ||||||
|  |                 throw new InvalidOperationException("No responder configured for request."); | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             var responder = _responders.Count > 1 | ||||||
|  |                 ? _responders.Dequeue() | ||||||
|  |                 : _responders.Peek(); | ||||||
|  |  | ||||||
|  |             var response = responder(request); | ||||||
|  |             response.RequestMessage = request; | ||||||
|  |             return Task.FromResult(response); | ||||||
|  |         } | ||||||
|  |     } | ||||||
|  | } | ||||||
| @@ -0,0 +1,17 @@ | |||||||
|  | <Project Sdk="Microsoft.NET.Sdk"> | ||||||
|  |   <PropertyGroup> | ||||||
|  |     <TargetFramework>net10.0</TargetFramework> | ||||||
|  |     <LangVersion>preview</LangVersion> | ||||||
|  |     <Nullable>enable</Nullable> | ||||||
|  |     <ImplicitUsings>enable</ImplicitUsings> | ||||||
|  |     <TreatWarningsAsErrors>true</TreatWarningsAsErrors> | ||||||
|  |   </PropertyGroup> | ||||||
|  |   <ItemGroup> | ||||||
|  |     <ProjectReference Include="..\StellaOps.Vexer.Connectors.RedHat.CSAF\StellaOps.Vexer.Connectors.RedHat.CSAF.csproj" /> | ||||||
|  |     <ProjectReference Include="..\StellaOps.Vexer.Storage.Mongo\StellaOps.Vexer.Storage.Mongo.csproj" /> | ||||||
|  |   </ItemGroup> | ||||||
|  |   <ItemGroup> | ||||||
|  |     <PackageReference Include="FluentAssertions" Version="6.12.0" /> | ||||||
|  |     <PackageReference Include="System.IO.Abstractions.TestingHelpers" Version="20.0.28" /> | ||||||
|  |   </ItemGroup> | ||||||
|  | </Project> | ||||||
| @@ -3,6 +3,8 @@ | |||||||
| Connector for Red Hat CSAF VEX feeds, fetching provider metadata, CSAF documents, and projecting them into raw storage for normalization. | Connector for Red Hat CSAF VEX feeds, fetching provider metadata, CSAF documents, and projecting them into raw storage for normalization. | ||||||
| ## Scope | ## Scope | ||||||
| - Discovery via `/.well-known/csaf/provider-metadata.json`, scheduling windows, and ETag-aware HTTP fetches. | - Discovery via `/.well-known/csaf/provider-metadata.json`, scheduling windows, and ETag-aware HTTP fetches. | ||||||
|  | - `RedHatProviderMetadataLoader` handles `.well-known` metadata with caching, schema validation, and offline snapshots. | ||||||
|  | - `RedHatCsafConnector` consumes ROLIE feeds to fetch incremental CSAF documents, honours `context.Since`, and streams raw advisories to storage. | ||||||
| - Mapping Red Hat CSAF specifics (product tree aliases, RHSA identifiers, revision history) into raw documents. | - Mapping Red Hat CSAF specifics (product tree aliases, RHSA identifiers, revision history) into raw documents. | ||||||
| - Emitting structured telemetry and resume markers for incremental pulls. | - Emitting structured telemetry and resume markers for incremental pulls. | ||||||
| - Supplying Red Hat-specific trust overrides and provenance hints to normalization. | - Supplying Red Hat-specific trust overrides and provenance hints to normalization. | ||||||
|   | |||||||
| @@ -0,0 +1,104 @@ | |||||||
|  | using System.Collections.Generic; | ||||||
|  | using System.IO.Abstractions; | ||||||
|  |  | ||||||
|  | namespace StellaOps.Vexer.Connectors.RedHat.CSAF.Configuration; | ||||||
|  |  | ||||||
|  | public sealed class RedHatConnectorOptions | ||||||
|  | { | ||||||
|  |     public static readonly Uri DefaultMetadataUri = new("https://access.redhat.com/.well-known/csaf/provider-metadata.json"); | ||||||
|  |  | ||||||
|  |     /// <summary> | ||||||
|  |     /// HTTP client name registered for the connector. | ||||||
|  |     /// </summary> | ||||||
|  |     public const string HttpClientName = "vexer.connector.redhat"; | ||||||
|  |  | ||||||
|  |     /// <summary> | ||||||
|  |     /// URI of the CSAF provider metadata document. | ||||||
|  |     /// </summary> | ||||||
|  |     public Uri MetadataUri { get; set; } = DefaultMetadataUri; | ||||||
|  |  | ||||||
|  |     /// <summary> | ||||||
|  |     /// Duration to cache loaded metadata before refreshing. | ||||||
|  |     /// </summary> | ||||||
|  |     public TimeSpan MetadataCacheDuration { get; set; } = TimeSpan.FromHours(1); | ||||||
|  |  | ||||||
|  |     /// <summary> | ||||||
|  |     /// Optional file path used to store or source offline metadata snapshots. | ||||||
|  |     /// </summary> | ||||||
|  |     public string? OfflineSnapshotPath { get; set; } | ||||||
|  |  | ||||||
|  |     /// <summary> | ||||||
|  |     /// When true, the loader prefers the offline snapshot without attempting a network fetch. | ||||||
|  |     /// </summary> | ||||||
|  |     public bool PreferOfflineSnapshot { get; set; } | ||||||
|  |  | ||||||
|  |     /// <summary> | ||||||
|  |     /// Enables writing fresh metadata responses to <see cref="OfflineSnapshotPath"/>. | ||||||
|  |     /// </summary> | ||||||
|  |     public bool PersistOfflineSnapshot { get; set; } = true; | ||||||
|  |  | ||||||
|  |     /// <summary> | ||||||
|  |     /// Explicit trust weight override applied to the provider entry. | ||||||
|  |     /// </summary> | ||||||
|  |     public double TrustWeight { get; set; } = 1.0; | ||||||
|  |  | ||||||
|  |     /// <summary> | ||||||
|  |     /// Sigstore/Cosign issuer used to verify CSAF signatures, if published. | ||||||
|  |     /// </summary> | ||||||
|  |     public string? CosignIssuer { get; set; } = "https://access.redhat.com"; | ||||||
|  |  | ||||||
|  |     /// <summary> | ||||||
|  |     /// Identity pattern matched against the Cosign certificate subject. | ||||||
|  |     /// </summary> | ||||||
|  |     public string? CosignIdentityPattern { get; set; } = "^https://access\\.redhat\\.com/.+$"; | ||||||
|  |  | ||||||
|  |     /// <summary> | ||||||
|  |     /// Optional list of PGP fingerprints recognised for Red Hat CSAF artifacts. | ||||||
|  |     /// </summary> | ||||||
|  |     public IList<string> PgpFingerprints { get; } = new List<string>(); | ||||||
|  |  | ||||||
|  |     public void Validate(IFileSystem? fileSystem = null) | ||||||
|  |     { | ||||||
|  |         if (MetadataUri is null || !MetadataUri.IsAbsoluteUri) | ||||||
|  |         { | ||||||
|  |             throw new InvalidOperationException("Metadata URI must be absolute."); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         if (MetadataUri.Scheme is not ("http" or "https")) | ||||||
|  |         { | ||||||
|  |             throw new InvalidOperationException("Metadata URI must use HTTP or HTTPS."); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         if (MetadataCacheDuration <= TimeSpan.Zero) | ||||||
|  |         { | ||||||
|  |             throw new InvalidOperationException("Metadata cache duration must be positive."); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         if (!string.IsNullOrWhiteSpace(OfflineSnapshotPath)) | ||||||
|  |         { | ||||||
|  |             var fs = fileSystem ?? new FileSystem(); | ||||||
|  |             var directory = Path.GetDirectoryName(OfflineSnapshotPath); | ||||||
|  |             if (!string.IsNullOrWhiteSpace(directory) && !fs.Directory.Exists(directory)) | ||||||
|  |             { | ||||||
|  |                 fs.Directory.CreateDirectory(directory); | ||||||
|  |             } | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         if (double.IsNaN(TrustWeight) || double.IsInfinity(TrustWeight) || TrustWeight <= 0) | ||||||
|  |         { | ||||||
|  |             TrustWeight = 1.0; | ||||||
|  |         } | ||||||
|  |         else if (TrustWeight > 1.0) | ||||||
|  |         { | ||||||
|  |             TrustWeight = 1.0; | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         if (CosignIssuer is not null) | ||||||
|  |         { | ||||||
|  |             if (string.IsNullOrWhiteSpace(CosignIdentityPattern)) | ||||||
|  |             { | ||||||
|  |                 throw new InvalidOperationException("CosignIdentityPattern must be provided when CosignIssuer is specified."); | ||||||
|  |             } | ||||||
|  |         } | ||||||
|  |     } | ||||||
|  | } | ||||||
| @@ -0,0 +1,45 @@ | |||||||
|  | using System.Net; | ||||||
|  | using Microsoft.Extensions.Caching.Memory; | ||||||
|  | using Microsoft.Extensions.DependencyInjection; | ||||||
|  | using Microsoft.Extensions.DependencyInjection.Extensions; | ||||||
|  | using StellaOps.Vexer.Connectors.RedHat.CSAF.Configuration; | ||||||
|  | using StellaOps.Vexer.Connectors.RedHat.CSAF.Metadata; | ||||||
|  | using StellaOps.Vexer.Core; | ||||||
|  | using StellaOps.Vexer.Storage.Mongo; | ||||||
|  | using System.IO.Abstractions; | ||||||
|  |  | ||||||
|  | namespace StellaOps.Vexer.Connectors.RedHat.CSAF.DependencyInjection; | ||||||
|  |  | ||||||
|  | public static class RedHatConnectorServiceCollectionExtensions | ||||||
|  | { | ||||||
|  |     public static IServiceCollection AddRedHatCsafConnector(this IServiceCollection services, Action<RedHatConnectorOptions>? configure = null) | ||||||
|  |     { | ||||||
|  |         ArgumentNullException.ThrowIfNull(services); | ||||||
|  |  | ||||||
|  |         services.AddOptions<RedHatConnectorOptions>() | ||||||
|  |             .Configure(options => | ||||||
|  |             { | ||||||
|  |                 configure?.Invoke(options); | ||||||
|  |             }) | ||||||
|  |             .PostConfigure(options => options.Validate()); | ||||||
|  |  | ||||||
|  |         services.TryAddSingleton<IMemoryCache, MemoryCache>(); | ||||||
|  |         services.TryAddSingleton<IFileSystem, FileSystem>(); | ||||||
|  |  | ||||||
|  |         services.AddHttpClient(RedHatConnectorOptions.HttpClientName, client => | ||||||
|  |             { | ||||||
|  |                 client.Timeout = TimeSpan.FromSeconds(30); | ||||||
|  |                 client.DefaultRequestHeaders.UserAgent.ParseAdd("StellaOps.Vexer.Connectors.RedHat/1.0"); | ||||||
|  |                 client.DefaultRequestHeaders.Accept.ParseAdd("application/json"); | ||||||
|  |             }) | ||||||
|  |             .ConfigurePrimaryHttpMessageHandler(() => new HttpClientHandler | ||||||
|  |             { | ||||||
|  |                 AutomaticDecompression = DecompressionMethods.All, | ||||||
|  |             }); | ||||||
|  |  | ||||||
|  |         services.AddSingleton<RedHatProviderMetadataLoader>(); | ||||||
|  |         services.AddSingleton<IVexConnector, RedHatCsafConnector>(); | ||||||
|  |  | ||||||
|  |         return services; | ||||||
|  |     } | ||||||
|  | } | ||||||
| @@ -0,0 +1,312 @@ | |||||||
|  | using System.Collections.Immutable; | ||||||
|  | using System.Linq; | ||||||
|  | using System.Net; | ||||||
|  | using System.Net.Http; | ||||||
|  | using System.Net.Http.Headers; | ||||||
|  | using System.Text.Json; | ||||||
|  | using System.Text.Json.Serialization; | ||||||
|  | using Microsoft.Extensions.Caching.Memory; | ||||||
|  | using Microsoft.Extensions.Logging; | ||||||
|  | using Microsoft.Extensions.Options; | ||||||
|  | using StellaOps.Vexer.Connectors.RedHat.CSAF.Configuration; | ||||||
|  | using StellaOps.Vexer.Core; | ||||||
|  | using System.IO.Abstractions; | ||||||
|  |  | ||||||
|  | namespace StellaOps.Vexer.Connectors.RedHat.CSAF.Metadata; | ||||||
|  |  | ||||||
|  | public sealed class RedHatProviderMetadataLoader | ||||||
|  | { | ||||||
|  |     public const string CacheKey = "StellaOps.Vexer.Connectors.RedHat.CSAF.Metadata"; | ||||||
|  |  | ||||||
|  |     private readonly IHttpClientFactory _httpClientFactory; | ||||||
|  |     private readonly IMemoryCache _cache; | ||||||
|  |     private readonly ILogger<RedHatProviderMetadataLoader> _logger; | ||||||
|  |     private readonly RedHatConnectorOptions _options; | ||||||
|  |     private readonly IFileSystem _fileSystem; | ||||||
|  |     private readonly JsonSerializerOptions _serializerOptions; | ||||||
|  |     private readonly SemaphoreSlim _refreshSemaphore = new(1, 1); | ||||||
|  |  | ||||||
|  |     public RedHatProviderMetadataLoader( | ||||||
|  |         IHttpClientFactory httpClientFactory, | ||||||
|  |         IMemoryCache memoryCache, | ||||||
|  |         IOptions<RedHatConnectorOptions> options, | ||||||
|  |         ILogger<RedHatProviderMetadataLoader> logger, | ||||||
|  |         IFileSystem? fileSystem = null) | ||||||
|  |     { | ||||||
|  |         _httpClientFactory = httpClientFactory ?? throw new ArgumentNullException(nameof(httpClientFactory)); | ||||||
|  |         _cache = memoryCache ?? throw new ArgumentNullException(nameof(memoryCache)); | ||||||
|  |         _logger = logger ?? throw new ArgumentNullException(nameof(logger)); | ||||||
|  |         _options = (options ?? throw new ArgumentNullException(nameof(options))).Value ?? throw new ArgumentNullException(nameof(options)); | ||||||
|  |         _fileSystem = fileSystem ?? new FileSystem(); | ||||||
|  |         _serializerOptions = new JsonSerializerOptions(JsonSerializerDefaults.Web) | ||||||
|  |         { | ||||||
|  |             PropertyNameCaseInsensitive = true, | ||||||
|  |             ReadCommentHandling = JsonCommentHandling.Skip, | ||||||
|  |         }; | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     public async Task<RedHatProviderMetadataResult> LoadAsync(CancellationToken cancellationToken) | ||||||
|  |     { | ||||||
|  |         if (_cache.TryGetValue<CacheEntry>(CacheKey, out var cached) && cached is { } cachedEntry && !cachedEntry.IsExpired()) | ||||||
|  |         { | ||||||
|  |             _logger.LogDebug("Returning cached Red Hat provider metadata (expires {Expires}).", cachedEntry.ExpiresAt); | ||||||
|  |             return new RedHatProviderMetadataResult(cachedEntry.Provider, cachedEntry.FetchedAt, true, cachedEntry.FromOffline); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         await _refreshSemaphore.WaitAsync(cancellationToken).ConfigureAwait(false); | ||||||
|  |         try | ||||||
|  |         { | ||||||
|  |             if (_cache.TryGetValue<CacheEntry>(CacheKey, out cached) && cached is { } cachedAfterLock && !cachedAfterLock.IsExpired()) | ||||||
|  |             { | ||||||
|  |                 return new RedHatProviderMetadataResult(cachedAfterLock.Provider, cachedAfterLock.FetchedAt, true, cachedAfterLock.FromOffline); | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             CacheEntry? previous = cached; | ||||||
|  |  | ||||||
|  |             // Attempt live fetch unless offline preferred. | ||||||
|  |             if (!_options.PreferOfflineSnapshot) | ||||||
|  |             { | ||||||
|  |                 var httpResult = await TryFetchFromNetworkAsync(previous, cancellationToken).ConfigureAwait(false); | ||||||
|  |                 if (httpResult is not null) | ||||||
|  |                 { | ||||||
|  |                     StoreCache(httpResult); | ||||||
|  |                     return new RedHatProviderMetadataResult(httpResult.Provider, httpResult.FetchedAt, false, false); | ||||||
|  |                 } | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             var offlineResult = TryLoadFromOffline(); | ||||||
|  |             if (offlineResult is not null) | ||||||
|  |             { | ||||||
|  |                 var offlineEntry = offlineResult with | ||||||
|  |                 { | ||||||
|  |                     FetchedAt = DateTimeOffset.UtcNow, | ||||||
|  |                     ExpiresAt = DateTimeOffset.UtcNow + _options.MetadataCacheDuration, | ||||||
|  |                     FromOffline = true, | ||||||
|  |                 }; | ||||||
|  |                 StoreCache(offlineEntry); | ||||||
|  |                 return new RedHatProviderMetadataResult(offlineEntry.Provider, offlineEntry.FetchedAt, false, true); | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             throw new InvalidOperationException("Unable to load Red Hat CSAF provider metadata from network or offline snapshot."); | ||||||
|  |         } | ||||||
|  |         finally | ||||||
|  |         { | ||||||
|  |             _refreshSemaphore.Release(); | ||||||
|  |         } | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     private void StoreCache(CacheEntry entry) | ||||||
|  |     { | ||||||
|  |         var cacheEntryOptions = new MemoryCacheEntryOptions | ||||||
|  |         { | ||||||
|  |             AbsoluteExpiration = entry.ExpiresAt, | ||||||
|  |         }; | ||||||
|  |         _cache.Set(CacheKey, entry, cacheEntryOptions); | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     private async Task<CacheEntry?> TryFetchFromNetworkAsync(CacheEntry? previous, CancellationToken cancellationToken) | ||||||
|  |     { | ||||||
|  |         try | ||||||
|  |         { | ||||||
|  |             var client = _httpClientFactory.CreateClient(RedHatConnectorOptions.HttpClientName); | ||||||
|  |             using var request = new HttpRequestMessage(HttpMethod.Get, _options.MetadataUri); | ||||||
|  |             if (!string.IsNullOrWhiteSpace(previous?.ETag)) | ||||||
|  |             { | ||||||
|  |                 if (EntityTagHeaderValue.TryParse(previous.ETag, out var etag)) | ||||||
|  |                 { | ||||||
|  |                     request.Headers.IfNoneMatch.Add(etag); | ||||||
|  |                 } | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             using var response = await client.SendAsync(request, HttpCompletionOption.ResponseHeadersRead, cancellationToken).ConfigureAwait(false); | ||||||
|  |  | ||||||
|  |             if (response.StatusCode == HttpStatusCode.NotModified && previous is not null) | ||||||
|  |             { | ||||||
|  |                 _logger.LogDebug("Red Hat provider metadata not modified (etag {ETag}).", previous.ETag); | ||||||
|  |                 return previous with | ||||||
|  |                 { | ||||||
|  |                     FetchedAt = DateTimeOffset.UtcNow, | ||||||
|  |                     ExpiresAt = DateTimeOffset.UtcNow + _options.MetadataCacheDuration, | ||||||
|  |                 }; | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             response.EnsureSuccessStatusCode(); | ||||||
|  |             var payload = await response.Content.ReadAsStringAsync(cancellationToken).ConfigureAwait(false); | ||||||
|  |  | ||||||
|  |             var provider = ParseAndValidate(payload); | ||||||
|  |             var etagHeader = response.Headers.ETag?.ToString(); | ||||||
|  |  | ||||||
|  |             if (_options.PersistOfflineSnapshot && !string.IsNullOrWhiteSpace(_options.OfflineSnapshotPath)) | ||||||
|  |             { | ||||||
|  |                 try | ||||||
|  |                 { | ||||||
|  |                     _fileSystem.File.WriteAllText(_options.OfflineSnapshotPath, payload); | ||||||
|  |                     _logger.LogDebug("Persisted Red Hat metadata snapshot to {Path}.", _options.OfflineSnapshotPath); | ||||||
|  |                 } | ||||||
|  |                 catch (Exception ex) | ||||||
|  |                 { | ||||||
|  |                     _logger.LogWarning(ex, "Failed to persist Red Hat metadata snapshot to {Path}.", _options.OfflineSnapshotPath); | ||||||
|  |                 } | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             return new CacheEntry( | ||||||
|  |                 provider, | ||||||
|  |                 DateTimeOffset.UtcNow, | ||||||
|  |                 DateTimeOffset.UtcNow + _options.MetadataCacheDuration, | ||||||
|  |                 etagHeader, | ||||||
|  |                 FromOffline: false); | ||||||
|  |         } | ||||||
|  |         catch (Exception ex) when (ex is not OperationCanceledException && !_options.PreferOfflineSnapshot) | ||||||
|  |         { | ||||||
|  |             _logger.LogWarning(ex, "Failed to fetch Red Hat provider metadata from {Uri}, will attempt offline snapshot.", _options.MetadataUri); | ||||||
|  |             return null; | ||||||
|  |         } | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     private CacheEntry? TryLoadFromOffline() | ||||||
|  |     { | ||||||
|  |         if (string.IsNullOrWhiteSpace(_options.OfflineSnapshotPath)) | ||||||
|  |         { | ||||||
|  |             return null; | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         if (!_fileSystem.File.Exists(_options.OfflineSnapshotPath)) | ||||||
|  |         { | ||||||
|  |             _logger.LogWarning("Offline snapshot path {Path} does not exist.", _options.OfflineSnapshotPath); | ||||||
|  |             return null; | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         try | ||||||
|  |         { | ||||||
|  |             var payload = _fileSystem.File.ReadAllText(_options.OfflineSnapshotPath); | ||||||
|  |             var provider = ParseAndValidate(payload); | ||||||
|  |             return new CacheEntry(provider, DateTimeOffset.UtcNow, DateTimeOffset.UtcNow + _options.MetadataCacheDuration, ETag: null, FromOffline: true); | ||||||
|  |         } | ||||||
|  |         catch (Exception ex) | ||||||
|  |         { | ||||||
|  |             _logger.LogError(ex, "Failed to load Red Hat provider metadata from offline snapshot {Path}.", _options.OfflineSnapshotPath); | ||||||
|  |             return null; | ||||||
|  |         } | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     private VexProvider ParseAndValidate(string payload) | ||||||
|  |     { | ||||||
|  |         if (string.IsNullOrWhiteSpace(payload)) | ||||||
|  |         { | ||||||
|  |             throw new InvalidOperationException("Provider metadata payload was empty."); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         ProviderMetadataDocument? document; | ||||||
|  |         try | ||||||
|  |         { | ||||||
|  |             document = JsonSerializer.Deserialize<ProviderMetadataDocument>(payload, _serializerOptions); | ||||||
|  |         } | ||||||
|  |         catch (JsonException ex) | ||||||
|  |         { | ||||||
|  |             throw new InvalidOperationException("Provider metadata payload could not be parsed.", ex); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         if (document is null) | ||||||
|  |         { | ||||||
|  |             throw new InvalidOperationException("Provider metadata payload was null after parsing."); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         if (document.Metadata?.Provider?.Name is null) | ||||||
|  |         { | ||||||
|  |             throw new InvalidOperationException("Provider metadata missing provider name."); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         var distributions = document.Distributions? | ||||||
|  |             .Select(static d => d.Directory) | ||||||
|  |             .Where(static s => !string.IsNullOrWhiteSpace(s)) | ||||||
|  |             .Select(static s => CreateUri(s!, nameof(ProviderMetadataDistribution.Directory))) | ||||||
|  |             .ToImmutableArray() ?? ImmutableArray<Uri>.Empty; | ||||||
|  |  | ||||||
|  |         if (distributions.IsDefaultOrEmpty) | ||||||
|  |         { | ||||||
|  |             throw new InvalidOperationException("Provider metadata did not include any valid distribution directories."); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         Uri? rolieFeed = null; | ||||||
|  |         if (document.Rolie?.Feeds is not null) | ||||||
|  |         { | ||||||
|  |             foreach (var feed in document.Rolie.Feeds) | ||||||
|  |             { | ||||||
|  |                 if (!string.IsNullOrWhiteSpace(feed.Url)) | ||||||
|  |                 { | ||||||
|  |                     rolieFeed = CreateUri(feed.Url, "rolie.feeds[].url"); | ||||||
|  |                     break; | ||||||
|  |                 } | ||||||
|  |             } | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         var trust = BuildTrust(); | ||||||
|  |         return new VexProvider( | ||||||
|  |             id: "vexer:redhat", | ||||||
|  |             displayName: document.Metadata.Provider.Name, | ||||||
|  |             kind: VexProviderKind.Distro, | ||||||
|  |             baseUris: distributions, | ||||||
|  |             discovery: new VexProviderDiscovery(_options.MetadataUri, rolieFeed), | ||||||
|  |             trust: trust); | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     private VexProviderTrust BuildTrust() | ||||||
|  |     { | ||||||
|  |         VexCosignTrust? cosign = null; | ||||||
|  |         if (!string.IsNullOrWhiteSpace(_options.CosignIssuer) && !string.IsNullOrWhiteSpace(_options.CosignIdentityPattern)) | ||||||
|  |         { | ||||||
|  |             cosign = new VexCosignTrust(_options.CosignIssuer!, _options.CosignIdentityPattern!); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         return new VexProviderTrust( | ||||||
|  |             _options.TrustWeight, | ||||||
|  |             cosign, | ||||||
|  |             _options.PgpFingerprints); | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     private static Uri CreateUri(string value, string propertyName) | ||||||
|  |     { | ||||||
|  |         if (!Uri.TryCreate(value, UriKind.Absolute, out var uri) || uri.Scheme is not ("http" or "https")) | ||||||
|  |         { | ||||||
|  |             throw new InvalidOperationException($"Provider metadata field '{propertyName}' must be an absolute HTTP(S) URI."); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         return uri; | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     private sealed record ProviderMetadataDocument( | ||||||
|  |         [property: JsonPropertyName("metadata")] ProviderMetadata? Metadata, | ||||||
|  |         [property: JsonPropertyName("distributions")] IReadOnlyList<ProviderMetadataDistribution>? Distributions, | ||||||
|  |         [property: JsonPropertyName("rolie")] ProviderMetadataRolie? Rolie); | ||||||
|  |  | ||||||
|  |     private sealed record ProviderMetadata( | ||||||
|  |         [property: JsonPropertyName("provider")] ProviderMetadataProvider? Provider); | ||||||
|  |  | ||||||
|  |     private sealed record ProviderMetadataProvider( | ||||||
|  |         [property: JsonPropertyName("name")] string? Name); | ||||||
|  |  | ||||||
|  |     private sealed record ProviderMetadataDistribution( | ||||||
|  |         [property: JsonPropertyName("directory")] string? Directory); | ||||||
|  |  | ||||||
|  |     private sealed record ProviderMetadataRolie( | ||||||
|  |         [property: JsonPropertyName("feeds")] IReadOnlyList<ProviderMetadataRolieFeed>? Feeds); | ||||||
|  |  | ||||||
|  |     private sealed record ProviderMetadataRolieFeed( | ||||||
|  |         [property: JsonPropertyName("url")] string? Url); | ||||||
|  |  | ||||||
|  |     private sealed record CacheEntry( | ||||||
|  |         VexProvider Provider, | ||||||
|  |         DateTimeOffset FetchedAt, | ||||||
|  |         DateTimeOffset ExpiresAt, | ||||||
|  |         string? ETag, | ||||||
|  |         bool FromOffline) | ||||||
|  |     { | ||||||
|  |         public bool IsExpired() => DateTimeOffset.UtcNow >= ExpiresAt; | ||||||
|  |     } | ||||||
|  | } | ||||||
|  |  | ||||||
|  | public sealed record RedHatProviderMetadataResult( | ||||||
|  |     VexProvider Provider, | ||||||
|  |     DateTimeOffset FetchedAt, | ||||||
|  |     bool FromCache, | ||||||
|  |     bool FromOfflineSnapshot); | ||||||
| @@ -0,0 +1,186 @@ | |||||||
|  | using System.Collections.Generic; | ||||||
|  | using System.Collections.Immutable; | ||||||
|  | using System.Linq; | ||||||
|  | using System.Net.Http; | ||||||
|  | using System.Runtime.CompilerServices; | ||||||
|  | using System.Text.Json; | ||||||
|  | using System.Xml.Linq; | ||||||
|  | using Microsoft.Extensions.Logging; | ||||||
|  | using Microsoft.Extensions.Options; | ||||||
|  | using StellaOps.Vexer.Connectors.Abstractions; | ||||||
|  | using StellaOps.Vexer.Connectors.RedHat.CSAF.Configuration; | ||||||
|  | using StellaOps.Vexer.Connectors.RedHat.CSAF.Metadata; | ||||||
|  | using StellaOps.Vexer.Core; | ||||||
|  | using StellaOps.Vexer.Storage.Mongo; | ||||||
|  |  | ||||||
|  | namespace StellaOps.Vexer.Connectors.RedHat.CSAF; | ||||||
|  |  | ||||||
|  | public sealed class RedHatCsafConnector : VexConnectorBase | ||||||
|  | { | ||||||
|  |     private readonly RedHatProviderMetadataLoader _metadataLoader; | ||||||
|  |     private readonly IHttpClientFactory _httpClientFactory; | ||||||
|  |     private readonly IVexConnectorStateRepository _stateRepository; | ||||||
|  |     public RedHatCsafConnector( | ||||||
|  |         VexConnectorDescriptor descriptor, | ||||||
|  |         RedHatProviderMetadataLoader metadataLoader, | ||||||
|  |         IHttpClientFactory httpClientFactory, | ||||||
|  |         IVexConnectorStateRepository stateRepository, | ||||||
|  |         ILogger<RedHatCsafConnector> logger, | ||||||
|  |         TimeProvider timeProvider) | ||||||
|  |         : base(descriptor, logger, timeProvider) | ||||||
|  |     { | ||||||
|  |         _metadataLoader = metadataLoader ?? throw new ArgumentNullException(nameof(metadataLoader)); | ||||||
|  |         _httpClientFactory = httpClientFactory ?? throw new ArgumentNullException(nameof(httpClientFactory)); | ||||||
|  |         _stateRepository = stateRepository ?? throw new ArgumentNullException(nameof(stateRepository)); | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     public override ValueTask ValidateAsync(VexConnectorSettings settings, CancellationToken cancellationToken) | ||||||
|  |     { | ||||||
|  |         // No connector-specific settings yet. | ||||||
|  |         return ValueTask.CompletedTask; | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     public override async IAsyncEnumerable<VexRawDocument> FetchAsync(VexConnectorContext context, [EnumeratorCancellation] CancellationToken cancellationToken) | ||||||
|  |     { | ||||||
|  |         ArgumentNullException.ThrowIfNull(context); | ||||||
|  |  | ||||||
|  |         var metadataResult = await _metadataLoader.LoadAsync(cancellationToken).ConfigureAwait(false); | ||||||
|  |         if (metadataResult.Provider.Discovery.RolIeService is null) | ||||||
|  |         { | ||||||
|  |             throw new InvalidOperationException("Red Hat provider metadata did not specify a ROLIE feed."); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         var state = await _stateRepository.GetAsync(Descriptor.Id, cancellationToken).ConfigureAwait(false); | ||||||
|  |  | ||||||
|  |         var sinceTimestamp = context.Since; | ||||||
|  |         if (state?.LastUpdated is { } persisted && (sinceTimestamp is null || persisted > sinceTimestamp)) | ||||||
|  |         { | ||||||
|  |             sinceTimestamp = persisted; | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         var knownDigests = state?.DocumentDigests ?? ImmutableArray<string>.Empty; | ||||||
|  |         var digestList = new List<string>(knownDigests); | ||||||
|  |         var digestSet = new HashSet<string>(knownDigests, StringComparer.OrdinalIgnoreCase); | ||||||
|  |         var latestUpdated = state?.LastUpdated ?? sinceTimestamp ?? DateTimeOffset.MinValue; | ||||||
|  |         var stateChanged = false; | ||||||
|  |  | ||||||
|  |         foreach (var entry in await FetchRolieEntriesAsync(metadataResult.Provider.Discovery.RolIeService, cancellationToken).ConfigureAwait(false)) | ||||||
|  |         { | ||||||
|  |             if (sinceTimestamp is not null && entry.Updated is DateTimeOffset updated && updated <= sinceTimestamp) | ||||||
|  |             { | ||||||
|  |                 continue; | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             if (entry.DocumentUri is null) | ||||||
|  |             { | ||||||
|  |                 Logger.LogDebug("Skipping ROLIE entry {Id} because no document link was provided.", entry.Id); | ||||||
|  |                 continue; | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             var rawDocument = await DownloadCsafDocumentAsync(entry, cancellationToken).ConfigureAwait(false); | ||||||
|  |  | ||||||
|  |             if (!digestSet.Add(rawDocument.Digest)) | ||||||
|  |             { | ||||||
|  |                 Logger.LogDebug("Skipping CSAF document {Uri} because digest {Digest} was already processed.", rawDocument.SourceUri, rawDocument.Digest); | ||||||
|  |                 continue; | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             await context.RawSink.StoreAsync(rawDocument, cancellationToken).ConfigureAwait(false); | ||||||
|  |             digestList.Add(rawDocument.Digest); | ||||||
|  |             stateChanged = true; | ||||||
|  |  | ||||||
|  |             if (entry.Updated is DateTimeOffset entryUpdated && entryUpdated > latestUpdated) | ||||||
|  |             { | ||||||
|  |                 latestUpdated = entryUpdated; | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             yield return rawDocument; | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         if (stateChanged) | ||||||
|  |         { | ||||||
|  |             var newLastUpdated = latestUpdated == DateTimeOffset.MinValue ? state?.LastUpdated : latestUpdated; | ||||||
|  |             var updatedState = new VexConnectorState( | ||||||
|  |                 Descriptor.Id, | ||||||
|  |                 newLastUpdated, | ||||||
|  |                 digestList.ToImmutableArray()); | ||||||
|  |  | ||||||
|  |             await _stateRepository.SaveAsync(updatedState, cancellationToken).ConfigureAwait(false); | ||||||
|  |         } | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     public override ValueTask<VexClaimBatch> NormalizeAsync(VexRawDocument document, CancellationToken cancellationToken) | ||||||
|  |     { | ||||||
|  |         // This connector relies on format-specific normalizers registered elsewhere. | ||||||
|  |         throw new NotSupportedException("RedHatCsafConnector does not perform in-line normalization; use the CSAF normalizer component."); | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     private async Task<IReadOnlyList<RolieEntry>> FetchRolieEntriesAsync(Uri feedUri, CancellationToken cancellationToken) | ||||||
|  |     { | ||||||
|  |         var client = _httpClientFactory.CreateClient(RedHatConnectorOptions.HttpClientName); | ||||||
|  |         using var response = await client.GetAsync(feedUri, cancellationToken).ConfigureAwait(false); | ||||||
|  |         response.EnsureSuccessStatusCode(); | ||||||
|  |  | ||||||
|  |         await using var stream = await response.Content.ReadAsStreamAsync(cancellationToken).ConfigureAwait(false); | ||||||
|  |         var document = XDocument.Load(stream); | ||||||
|  |         var ns = document.Root?.Name.Namespace ?? "http://www.w3.org/2005/Atom"; | ||||||
|  |  | ||||||
|  |         var entries = document.Root? | ||||||
|  |             .Elements(ns + "entry") | ||||||
|  |             .Select(e => new RolieEntry( | ||||||
|  |                 Id: (string?)e.Element(ns + "id"), | ||||||
|  |                 Updated: ParseUpdated((string?)e.Element(ns + "updated")), | ||||||
|  |                 DocumentUri: ParseDocumentLink(e, ns))) | ||||||
|  |             .Where(entry => entry.Id is not null && entry.Updated is not null) | ||||||
|  |             .OrderBy(entry => entry.Updated) | ||||||
|  |             .ToList() ?? new List<RolieEntry>(); | ||||||
|  |  | ||||||
|  |         return entries; | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     private static DateTimeOffset? ParseUpdated(string? value) | ||||||
|  |         => DateTimeOffset.TryParse(value, out var parsed) ? parsed : null; | ||||||
|  |  | ||||||
|  |     private static Uri? ParseDocumentLink(XElement entry, XNamespace ns) | ||||||
|  |     { | ||||||
|  |         var linkElements = entry.Elements(ns + "link"); | ||||||
|  |         foreach (var link in linkElements) | ||||||
|  |         { | ||||||
|  |             var rel = (string?)link.Attribute("rel"); | ||||||
|  |             var href = (string?)link.Attribute("href"); | ||||||
|  |             if (string.IsNullOrWhiteSpace(href)) | ||||||
|  |             { | ||||||
|  |                 continue; | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             if (rel is null || rel.Equals("enclosure", StringComparison.OrdinalIgnoreCase) || rel.Equals("alternate", StringComparison.OrdinalIgnoreCase)) | ||||||
|  |             { | ||||||
|  |                 if (Uri.TryCreate(href, UriKind.Absolute, out var uri)) | ||||||
|  |                 { | ||||||
|  |                     return uri; | ||||||
|  |                 } | ||||||
|  |             } | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         return null; | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     private async Task<VexRawDocument> DownloadCsafDocumentAsync(RolieEntry entry, CancellationToken cancellationToken) | ||||||
|  |     { | ||||||
|  |         var documentUri = entry.DocumentUri ?? throw new InvalidOperationException("ROLIE entry missing document URI."); | ||||||
|  |  | ||||||
|  |         var client = _httpClientFactory.CreateClient(RedHatConnectorOptions.HttpClientName); | ||||||
|  |         using var response = await client.GetAsync(documentUri, cancellationToken).ConfigureAwait(false); | ||||||
|  |         response.EnsureSuccessStatusCode(); | ||||||
|  |  | ||||||
|  |         var contentBytes = await response.Content.ReadAsByteArrayAsync(cancellationToken).ConfigureAwait(false); | ||||||
|  |         var metadata = BuildMetadata(builder => builder | ||||||
|  |             .Add("redhat.csaf.entryId", entry.Id) | ||||||
|  |             .Add("redhat.csaf.documentUri", documentUri.ToString()) | ||||||
|  |             .Add("redhat.csaf.updated", entry.Updated?.ToString("O"))); | ||||||
|  |  | ||||||
|  |         return CreateRawDocument(VexDocumentFormat.Csaf, documentUri, contentBytes, metadata); | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     private sealed record RolieEntry(string? Id, DateTimeOffset? Updated, Uri? DocumentUri); | ||||||
|  | } | ||||||
| @@ -0,0 +1,19 @@ | |||||||
|  | <Project Sdk="Microsoft.NET.Sdk"> | ||||||
|  |   <PropertyGroup> | ||||||
|  |     <TargetFramework>net10.0</TargetFramework> | ||||||
|  |     <LangVersion>preview</LangVersion> | ||||||
|  |     <Nullable>enable</Nullable> | ||||||
|  |     <ImplicitUsings>enable</ImplicitUsings> | ||||||
|  |     <TreatWarningsAsErrors>true</TreatWarningsAsErrors> | ||||||
|  |   </PropertyGroup> | ||||||
|  |   <ItemGroup> | ||||||
|  |     <ProjectReference Include="..\StellaOps.Vexer.Connectors.Abstractions\StellaOps.Vexer.Connectors.Abstractions.csproj" /> | ||||||
|  |     <ProjectReference Include="..\StellaOps.Vexer.Storage.Mongo\StellaOps.Vexer.Storage.Mongo.csproj" /> | ||||||
|  |   </ItemGroup> | ||||||
|  |   <ItemGroup> | ||||||
|  |     <PackageReference Include="Microsoft.Extensions.Caching.Memory" Version="8.0.0" /> | ||||||
|  |     <PackageReference Include="Microsoft.Extensions.Http" Version="8.0.0" /> | ||||||
|  |     <PackageReference Include="Microsoft.Extensions.Options" Version="8.0.0" /> | ||||||
|  |     <PackageReference Include="System.IO.Abstractions" Version="20.0.28" /> | ||||||
|  |   </ItemGroup> | ||||||
|  | </Project> | ||||||
| @@ -2,6 +2,9 @@ If you are working on this file you need to read docs/ARCHITECTURE_VEXER.md and | |||||||
| # TASKS | # TASKS | ||||||
| | Task | Owner(s) | Depends on | Notes | | | Task | Owner(s) | Depends on | Notes | | ||||||
| |---|---|---|---| | |---|---|---|---| | ||||||
| |VEXER-CONN-RH-01-001 – Provider metadata discovery|Team Vexer Connectors – Red Hat|VEXER-CONN-ABS-01-001|TODO – Implement `.well-known` metadata loader with caching, schema validation, and offline snapshot support.| | |VEXER-CONN-RH-01-001 – Provider metadata discovery|Team Vexer Connectors – Red Hat|VEXER-CONN-ABS-01-001|**DONE (2025-10-17)** – Added `RedHatProviderMetadataLoader` with HTTP/ETag caching, offline snapshot handling, and validation; exposed DI helper + tests covering live, cached, and offline scenarios.| | ||||||
| |VEXER-CONN-RH-01-002 – Incremental CSAF pulls|Team Vexer Connectors – Red Hat|VEXER-CONN-RH-01-001, VEXER-STORAGE-01-003|TODO – Fetch CSAF windows with ETag handling, resume tokens, quarantine on schema errors, and persist raw docs.| | |VEXER-CONN-RH-01-002 – Incremental CSAF pulls|Team Vexer Connectors – Red Hat|VEXER-CONN-RH-01-001, VEXER-STORAGE-01-003|**DONE (2025-10-17)** – Implemented `RedHatCsafConnector` with ROLIE feed parsing, incremental filtering via `context.Since`, CSAF document download + metadata capture, and persistence through `IVexRawDocumentSink`; tests cover live fetch/cache/offline scenarios with ETag handling.| | ||||||
| |VEXER-CONN-RH-01-003 – Trust metadata emission|Team Vexer Connectors – Red Hat|VEXER-CONN-RH-01-002, VEXER-POLICY-01-001|TODO – Populate provider trust overrides (cosign issuer, identity regex) and provenance hints for policy evaluation/logging.| | |VEXER-CONN-RH-01-003 – Trust metadata emission|Team Vexer Connectors – Red Hat|VEXER-CONN-RH-01-002, VEXER-POLICY-01-001|**DONE (2025-10-17)** – Provider metadata loader now emits trust overrides (weight, cosign issuer/pattern, PGP fingerprints) and the connector surfaces provenance hints for policy/consensus layers.| | ||||||
|  | |VEXER-CONN-RH-01-004 – Resume state persistence|Team Vexer Connectors – Red Hat|VEXER-CONN-RH-01-002, VEXER-STORAGE-01-003|**DONE (2025-10-17)** – Connector now loads/saves resume state via `IVexConnectorStateRepository`, tracking last update timestamp and recent document digests to avoid duplicate CSAF ingestion; regression covers state persistence and duplicate skips.| | ||||||
|  | |VEXER-CONN-RH-01-005 – Worker/WebService integration|Team Vexer Connectors – Red Hat|VEXER-CONN-RH-01-002|**DONE (2025-10-17)** – Worker/WebService now call `AddRedHatCsafConnector`, register the connector + state repo, and default worker scheduling adds the `vexer:redhat` provider so background jobs and orchestration can activate the connector without extra wiring.| | ||||||
|  | |VEXER-CONN-RH-01-006 – CSAF normalization parity tests|Team Vexer Connectors – Red Hat|VEXER-CONN-RH-01-002, VEXER-FMT-CSAF-01-001|**DONE (2025-10-17)** – Added RHSA fixture-driven regression verifying CSAF normalizer retains Red Hat product metadata, tracking fields, and timestamps (`rhsa-sample.json` + `CsafNormalizerTests.NormalizeAsync_PreservesRedHatSpecificMetadata`).| | ||||||
|   | |||||||
| @@ -0,0 +1,138 @@ | |||||||
|  | using System; | ||||||
|  | using System.Net; | ||||||
|  | using System.Net.Http; | ||||||
|  | using System.Text; | ||||||
|  | using System.Threading; | ||||||
|  | using FluentAssertions; | ||||||
|  | using Microsoft.Extensions.Caching.Memory; | ||||||
|  | using Microsoft.Extensions.Logging.Abstractions; | ||||||
|  | using StellaOps.Vexer.Connectors.SUSE.RancherVEXHub.Authentication; | ||||||
|  | using StellaOps.Vexer.Connectors.SUSE.RancherVEXHub.Configuration; | ||||||
|  | using Xunit; | ||||||
|  |  | ||||||
|  | namespace StellaOps.Vexer.Connectors.SUSE.RancherVEXHub.Tests.Authentication; | ||||||
|  |  | ||||||
|  | public sealed class RancherHubTokenProviderTests | ||||||
|  | { | ||||||
|  |     private const string TokenResponse = "{\"access_token\":\"abc123\",\"token_type\":\"Bearer\",\"expires_in\":3600}"; | ||||||
|  |  | ||||||
|  |     [Fact] | ||||||
|  |     public async Task GetAccessTokenAsync_RequestsAndCachesToken() | ||||||
|  |     { | ||||||
|  |         var handler = TestHttpMessageHandler.RespondWith(request => | ||||||
|  |         { | ||||||
|  |             request.Headers.Authorization.Should().NotBeNull(); | ||||||
|  |             request.Content.Should().NotBeNull(); | ||||||
|  |  | ||||||
|  |             return new HttpResponseMessage(HttpStatusCode.OK) | ||||||
|  |             { | ||||||
|  |                 Content = new StringContent(TokenResponse, Encoding.UTF8, "application/json"), | ||||||
|  |             }; | ||||||
|  |         }); | ||||||
|  |  | ||||||
|  |         var client = new HttpClient(handler) | ||||||
|  |         { | ||||||
|  |             BaseAddress = new Uri("https://identity.suse.com"), | ||||||
|  |         }; | ||||||
|  |  | ||||||
|  |         var factory = new SingleClientHttpClientFactory(client); | ||||||
|  |         var cache = new MemoryCache(new MemoryCacheOptions()); | ||||||
|  |         var provider = new RancherHubTokenProvider(factory, cache, NullLogger<RancherHubTokenProvider>.Instance); | ||||||
|  |  | ||||||
|  |         var options = new RancherHubConnectorOptions | ||||||
|  |         { | ||||||
|  |             ClientId = "client", | ||||||
|  |             ClientSecret = "secret", | ||||||
|  |             TokenEndpoint = new Uri("https://identity.suse.com/oauth/token"), | ||||||
|  |             Audience = "https://vexhub.suse.com", | ||||||
|  |         }; | ||||||
|  |         options.Scopes.Clear(); | ||||||
|  |         options.Scopes.Add("hub.read"); | ||||||
|  |         options.Scopes.Add("hub.events"); | ||||||
|  |  | ||||||
|  |         var token = await provider.GetAccessTokenAsync(options, CancellationToken.None); | ||||||
|  |         token.Should().NotBeNull(); | ||||||
|  |         token!.Value.Should().Be("abc123"); | ||||||
|  |  | ||||||
|  |         var cached = await provider.GetAccessTokenAsync(options, CancellationToken.None); | ||||||
|  |         cached.Should().NotBeNull(); | ||||||
|  |         handler.InvocationCount.Should().Be(1); | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     [Fact] | ||||||
|  |     public async Task GetAccessTokenAsync_ReturnsNullWhenOfflinePreferred() | ||||||
|  |     { | ||||||
|  |         var handler = TestHttpMessageHandler.RespondWith(_ => new HttpResponseMessage(HttpStatusCode.OK)); | ||||||
|  |         var client = new HttpClient(handler) | ||||||
|  |         { | ||||||
|  |             BaseAddress = new Uri("https://identity.suse.com"), | ||||||
|  |         }; | ||||||
|  |  | ||||||
|  |         var factory = new SingleClientHttpClientFactory(client); | ||||||
|  |         var cache = new MemoryCache(new MemoryCacheOptions()); | ||||||
|  |         var provider = new RancherHubTokenProvider(factory, cache, NullLogger<RancherHubTokenProvider>.Instance); | ||||||
|  |         var options = new RancherHubConnectorOptions | ||||||
|  |         { | ||||||
|  |             PreferOfflineSnapshot = true, | ||||||
|  |             ClientId = "client", | ||||||
|  |             ClientSecret = "secret", | ||||||
|  |             TokenEndpoint = new Uri("https://identity.suse.com/oauth/token"), | ||||||
|  |         }; | ||||||
|  |  | ||||||
|  |         var token = await provider.GetAccessTokenAsync(options, CancellationToken.None); | ||||||
|  |         token.Should().BeNull(); | ||||||
|  |         handler.InvocationCount.Should().Be(0); | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     [Fact] | ||||||
|  |     public async Task GetAccessTokenAsync_ReturnsNullWithoutCredentials() | ||||||
|  |     { | ||||||
|  |         var handler = TestHttpMessageHandler.RespondWith(_ => new HttpResponseMessage(HttpStatusCode.OK)); | ||||||
|  |         var client = new HttpClient(handler) | ||||||
|  |         { | ||||||
|  |             BaseAddress = new Uri("https://identity.suse.com"), | ||||||
|  |         }; | ||||||
|  |  | ||||||
|  |         var factory = new SingleClientHttpClientFactory(client); | ||||||
|  |         var cache = new MemoryCache(new MemoryCacheOptions()); | ||||||
|  |         var provider = new RancherHubTokenProvider(factory, cache, NullLogger<RancherHubTokenProvider>.Instance); | ||||||
|  |         var options = new RancherHubConnectorOptions(); | ||||||
|  |  | ||||||
|  |         var token = await provider.GetAccessTokenAsync(options, CancellationToken.None); | ||||||
|  |         token.Should().BeNull(); | ||||||
|  |         handler.InvocationCount.Should().Be(0); | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     private sealed class SingleClientHttpClientFactory : IHttpClientFactory | ||||||
|  |     { | ||||||
|  |         private readonly HttpClient _client; | ||||||
|  |  | ||||||
|  |         public SingleClientHttpClientFactory(HttpClient client) | ||||||
|  |         { | ||||||
|  |             _client = client; | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         public HttpClient CreateClient(string name) => _client; | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     private sealed class TestHttpMessageHandler : HttpMessageHandler | ||||||
|  |     { | ||||||
|  |         private readonly Func<HttpRequestMessage, HttpResponseMessage> _responseFactory; | ||||||
|  |  | ||||||
|  |         private TestHttpMessageHandler(Func<HttpRequestMessage, HttpResponseMessage> responseFactory) | ||||||
|  |         { | ||||||
|  |             _responseFactory = responseFactory; | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         public int InvocationCount { get; private set; } | ||||||
|  |  | ||||||
|  |         public static TestHttpMessageHandler RespondWith(Func<HttpRequestMessage, HttpResponseMessage> responseFactory) | ||||||
|  |             => new(responseFactory); | ||||||
|  |  | ||||||
|  |         protected override Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) | ||||||
|  |         { | ||||||
|  |             InvocationCount++; | ||||||
|  |             return Task.FromResult(_responseFactory(request)); | ||||||
|  |         } | ||||||
|  |     } | ||||||
|  | } | ||||||
| @@ -0,0 +1,178 @@ | |||||||
|  | using System; | ||||||
|  | using System.Collections.Generic; | ||||||
|  | using System.Net; | ||||||
|  | using System.Net.Http; | ||||||
|  | using System.Net.Http.Headers; | ||||||
|  | using System.Text; | ||||||
|  | using FluentAssertions; | ||||||
|  | using Microsoft.Extensions.Caching.Memory; | ||||||
|  | using Microsoft.Extensions.Logging.Abstractions; | ||||||
|  | using StellaOps.Vexer.Connectors.SUSE.RancherVEXHub.Authentication; | ||||||
|  | using StellaOps.Vexer.Connectors.SUSE.RancherVEXHub.Configuration; | ||||||
|  | using StellaOps.Vexer.Connectors.SUSE.RancherVEXHub.Metadata; | ||||||
|  | using System.IO.Abstractions.TestingHelpers; | ||||||
|  | using System.Threading; | ||||||
|  | using Xunit; | ||||||
|  |  | ||||||
|  | namespace StellaOps.Vexer.Connectors.SUSE.RancherVEXHub.Tests.Metadata; | ||||||
|  |  | ||||||
|  | public sealed class RancherHubMetadataLoaderTests | ||||||
|  | { | ||||||
|  |     private const string SampleDiscovery = """ | ||||||
|  |         { | ||||||
|  |           "hubId": "vexer:suse.rancher", | ||||||
|  |           "title": "SUSE Rancher VEX Hub", | ||||||
|  |           "subscription": { | ||||||
|  |             "eventsUri": "https://vexhub.suse.com/api/v1/events", | ||||||
|  |             "checkpointUri": "https://vexhub.suse.com/api/v1/checkpoints", | ||||||
|  |             "requiresAuthentication": true, | ||||||
|  |             "channels": ["rke2", "k3s"], | ||||||
|  |             "scopes": ["hub.read", "hub.events"] | ||||||
|  |           }, | ||||||
|  |           "authentication": { | ||||||
|  |             "tokenUri": "https://identity.suse.com/oauth2/token", | ||||||
|  |             "audience": "https://vexhub.suse.com" | ||||||
|  |           }, | ||||||
|  |           "offline": { | ||||||
|  |             "snapshotUri": "https://downloads.suse.com/vexhub/snapshot.json", | ||||||
|  |             "sha256": "deadbeef", | ||||||
|  |             "updated": "2025-10-10T12:00:00Z" | ||||||
|  |           } | ||||||
|  |         } | ||||||
|  |         """; | ||||||
|  |  | ||||||
|  |     [Fact] | ||||||
|  |     public async Task LoadAsync_FetchesAndCachesMetadata() | ||||||
|  |     { | ||||||
|  |         var handler = TestHttpMessageHandler.RespondWith(_ => | ||||||
|  |         { | ||||||
|  |             var response = new HttpResponseMessage(HttpStatusCode.OK) | ||||||
|  |             { | ||||||
|  |                 Content = new StringContent(SampleDiscovery, Encoding.UTF8, "application/json"), | ||||||
|  |             }; | ||||||
|  |             response.Headers.ETag = new EntityTagHeaderValue("\"abc\""); | ||||||
|  |             return response; | ||||||
|  |         }); | ||||||
|  |  | ||||||
|  |         var client = new HttpClient(handler) | ||||||
|  |         { | ||||||
|  |             BaseAddress = new Uri("https://vexhub.suse.com"), | ||||||
|  |         }; | ||||||
|  |  | ||||||
|  |         var factory = new SingleClientHttpClientFactory(client); | ||||||
|  |         var cache = new MemoryCache(new MemoryCacheOptions()); | ||||||
|  |         var fileSystem = new MockFileSystem(); | ||||||
|  |         var offlinePath = fileSystem.Path.Combine(@"C:\offline", "rancher-hub.json"); | ||||||
|  |         var options = new RancherHubConnectorOptions | ||||||
|  |         { | ||||||
|  |             DiscoveryUri = new Uri("https://vexhub.suse.com/.well-known/vex/rancher-hub.json"), | ||||||
|  |             OfflineSnapshotPath = offlinePath, | ||||||
|  |         }; | ||||||
|  |  | ||||||
|  |         var tokenProvider = new RancherHubTokenProvider(factory, cache, NullLogger<RancherHubTokenProvider>.Instance); | ||||||
|  |         var loader = new RancherHubMetadataLoader(factory, cache, tokenProvider, fileSystem, NullLogger<RancherHubMetadataLoader>.Instance); | ||||||
|  |  | ||||||
|  |         var result = await loader.LoadAsync(options, CancellationToken.None); | ||||||
|  |  | ||||||
|  |         result.FromCache.Should().BeFalse(); | ||||||
|  |         result.FromOfflineSnapshot.Should().BeFalse(); | ||||||
|  |         result.Metadata.Provider.DisplayName.Should().Be("SUSE Rancher VEX Hub"); | ||||||
|  |         result.Metadata.Subscription.EventsUri.Should().Be(new Uri("https://vexhub.suse.com/api/v1/events")); | ||||||
|  |         result.Metadata.Authentication.TokenEndpoint.Should().Be(new Uri("https://identity.suse.com/oauth2/token")); | ||||||
|  |  | ||||||
|  |         // Second call should be served from cache (no additional HTTP invocation). | ||||||
|  |         handler.ResetInvocationCount(); | ||||||
|  |         await loader.LoadAsync(options, CancellationToken.None); | ||||||
|  |         handler.InvocationCount.Should().Be(0); | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     [Fact] | ||||||
|  |     public async Task LoadAsync_UsesOfflineSnapshotWhenNetworkFails() | ||||||
|  |     { | ||||||
|  |         var handler = TestHttpMessageHandler.RespondWith(_ => throw new HttpRequestException("network down")); | ||||||
|  |         var client = new HttpClient(handler) | ||||||
|  |         { | ||||||
|  |             BaseAddress = new Uri("https://vexhub.suse.com"), | ||||||
|  |         }; | ||||||
|  |  | ||||||
|  |         var factory = new SingleClientHttpClientFactory(client); | ||||||
|  |         var cache = new MemoryCache(new MemoryCacheOptions()); | ||||||
|  |         var fileSystem = new MockFileSystem(); | ||||||
|  |         var offlinePath = fileSystem.Path.Combine(@"C:\offline", "rancher-hub.json"); | ||||||
|  |         fileSystem.AddFile(offlinePath, new MockFileData(SampleDiscovery)); | ||||||
|  |  | ||||||
|  |         var options = new RancherHubConnectorOptions | ||||||
|  |         { | ||||||
|  |             DiscoveryUri = new Uri("https://vexhub.suse.com/.well-known/vex/rancher-hub.json"), | ||||||
|  |             OfflineSnapshotPath = offlinePath, | ||||||
|  |         }; | ||||||
|  |  | ||||||
|  |         var tokenProvider = new RancherHubTokenProvider(factory, cache, NullLogger<RancherHubTokenProvider>.Instance); | ||||||
|  |         var loader = new RancherHubMetadataLoader(factory, cache, tokenProvider, fileSystem, NullLogger<RancherHubMetadataLoader>.Instance); | ||||||
|  |  | ||||||
|  |         var result = await loader.LoadAsync(options, CancellationToken.None); | ||||||
|  |         result.FromOfflineSnapshot.Should().BeTrue(); | ||||||
|  |         result.Metadata.Subscription.RequiresAuthentication.Should().BeTrue(); | ||||||
|  |         result.Metadata.OfflineSnapshot.Should().NotBeNull(); | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     [Fact] | ||||||
|  |     public async Task LoadAsync_ThrowsWhenOfflinePreferredButMissing() | ||||||
|  |     { | ||||||
|  |         var handler = TestHttpMessageHandler.RespondWith(_ => throw new HttpRequestException("network down")); | ||||||
|  |         var client = new HttpClient(handler) | ||||||
|  |         { | ||||||
|  |             BaseAddress = new Uri("https://vexhub.suse.com"), | ||||||
|  |         }; | ||||||
|  |  | ||||||
|  |         var factory = new SingleClientHttpClientFactory(client); | ||||||
|  |         var cache = new MemoryCache(new MemoryCacheOptions()); | ||||||
|  |         var fileSystem = new MockFileSystem(); | ||||||
|  |         var options = new RancherHubConnectorOptions | ||||||
|  |         { | ||||||
|  |             DiscoveryUri = new Uri("https://vexhub.suse.com/.well-known/vex/rancher-hub.json"), | ||||||
|  |             OfflineSnapshotPath = "/offline/missing.json", | ||||||
|  |             PreferOfflineSnapshot = true, | ||||||
|  |         }; | ||||||
|  |  | ||||||
|  |         var tokenProvider = new RancherHubTokenProvider(factory, cache, NullLogger<RancherHubTokenProvider>.Instance); | ||||||
|  |         var loader = new RancherHubMetadataLoader(factory, cache, tokenProvider, fileSystem, NullLogger<RancherHubMetadataLoader>.Instance); | ||||||
|  |  | ||||||
|  |         await Assert.ThrowsAsync<InvalidOperationException>(() => loader.LoadAsync(options, CancellationToken.None)); | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     private sealed class SingleClientHttpClientFactory : IHttpClientFactory | ||||||
|  |     { | ||||||
|  |         private readonly HttpClient _client; | ||||||
|  |  | ||||||
|  |         public SingleClientHttpClientFactory(HttpClient client) | ||||||
|  |         { | ||||||
|  |             _client = client; | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         public HttpClient CreateClient(string name) => _client; | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     private sealed class TestHttpMessageHandler : HttpMessageHandler | ||||||
|  |     { | ||||||
|  |         private readonly Func<HttpRequestMessage, HttpResponseMessage> _responseFactory; | ||||||
|  |  | ||||||
|  |         private TestHttpMessageHandler(Func<HttpRequestMessage, HttpResponseMessage> responseFactory) | ||||||
|  |         { | ||||||
|  |             _responseFactory = responseFactory; | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         public int InvocationCount { get; private set; } | ||||||
|  |  | ||||||
|  |         public static TestHttpMessageHandler RespondWith(Func<HttpRequestMessage, HttpResponseMessage> responseFactory) | ||||||
|  |             => new(responseFactory); | ||||||
|  |  | ||||||
|  |         public void ResetInvocationCount() => InvocationCount = 0; | ||||||
|  |  | ||||||
|  |         protected override Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) | ||||||
|  |         { | ||||||
|  |             InvocationCount++; | ||||||
|  |             return Task.FromResult(_responseFactory(request)); | ||||||
|  |         } | ||||||
|  |     } | ||||||
|  | } | ||||||
| @@ -0,0 +1,17 @@ | |||||||
|  | <Project Sdk="Microsoft.NET.Sdk"> | ||||||
|  |   <PropertyGroup> | ||||||
|  |     <TargetFramework>net10.0</TargetFramework> | ||||||
|  |     <LangVersion>preview</LangVersion> | ||||||
|  |     <Nullable>enable</Nullable> | ||||||
|  |     <ImplicitUsings>enable</ImplicitUsings> | ||||||
|  |     <TreatWarningsAsErrors>true</TreatWarningsAsErrors> | ||||||
|  |   </PropertyGroup> | ||||||
|  |   <ItemGroup> | ||||||
|  |     <ProjectReference Include="..\StellaOps.Vexer.Connectors.SUSE.RancherVEXHub\StellaOps.Vexer.Connectors.SUSE.RancherVEXHub.csproj" /> | ||||||
|  |     <ProjectReference Include="..\StellaOps.Vexer.Storage.Mongo\StellaOps.Vexer.Storage.Mongo.csproj" /> | ||||||
|  |   </ItemGroup> | ||||||
|  |   <ItemGroup> | ||||||
|  |     <PackageReference Include="FluentAssertions" Version="6.12.0" /> | ||||||
|  |     <PackageReference Include="System.IO.Abstractions.TestingHelpers" Version="20.0.28" /> | ||||||
|  |   </ItemGroup> | ||||||
|  | </Project> | ||||||
| @@ -0,0 +1,171 @@ | |||||||
|  | using System; | ||||||
|  | using System.Collections.Generic; | ||||||
|  | using System.Net.Http; | ||||||
|  | using System.Net.Http.Headers; | ||||||
|  | using System.Text; | ||||||
|  | using System.Text.Json; | ||||||
|  | using System.Threading; | ||||||
|  | using System.Threading.Tasks; | ||||||
|  | using Microsoft.Extensions.Caching.Memory; | ||||||
|  | using Microsoft.Extensions.Logging; | ||||||
|  | using StellaOps.Vexer.Connectors.SUSE.RancherVEXHub.Configuration; | ||||||
|  |  | ||||||
|  | namespace StellaOps.Vexer.Connectors.SUSE.RancherVEXHub.Authentication; | ||||||
|  |  | ||||||
|  | public sealed class RancherHubTokenProvider | ||||||
|  | { | ||||||
|  |     private const string CachePrefix = "StellaOps.Vexer.Connectors.SUSE.RancherVEXHub.Token"; | ||||||
|  |  | ||||||
|  |     private readonly IHttpClientFactory _httpClientFactory; | ||||||
|  |     private readonly IMemoryCache _cache; | ||||||
|  |     private readonly ILogger<RancherHubTokenProvider> _logger; | ||||||
|  |     private readonly SemaphoreSlim _semaphore = new(1, 1); | ||||||
|  |  | ||||||
|  |     public RancherHubTokenProvider(IHttpClientFactory httpClientFactory, IMemoryCache cache, ILogger<RancherHubTokenProvider> logger) | ||||||
|  |     { | ||||||
|  |         _httpClientFactory = httpClientFactory ?? throw new ArgumentNullException(nameof(httpClientFactory)); | ||||||
|  |         _cache = cache ?? throw new ArgumentNullException(nameof(cache)); | ||||||
|  |         _logger = logger ?? throw new ArgumentNullException(nameof(logger)); | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     public async ValueTask<RancherHubAccessToken?> GetAccessTokenAsync(RancherHubConnectorOptions options, CancellationToken cancellationToken) | ||||||
|  |     { | ||||||
|  |         ArgumentNullException.ThrowIfNull(options); | ||||||
|  |  | ||||||
|  |         if (options.PreferOfflineSnapshot) | ||||||
|  |         { | ||||||
|  |             _logger.LogDebug("Skipping token request because PreferOfflineSnapshot is enabled."); | ||||||
|  |             return null; | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         var hasCredentials = !string.IsNullOrWhiteSpace(options.ClientId) && | ||||||
|  |                               !string.IsNullOrWhiteSpace(options.ClientSecret) && | ||||||
|  |                               options.TokenEndpoint is not null; | ||||||
|  |  | ||||||
|  |         if (!hasCredentials) | ||||||
|  |         { | ||||||
|  |             if (!options.AllowAnonymousDiscovery) | ||||||
|  |             { | ||||||
|  |                 _logger.LogDebug("No Rancher hub credentials configured; proceeding without Authorization header."); | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             return null; | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         var cacheKey = $"{CachePrefix}:{options.ClientId}"; | ||||||
|  |         if (_cache.TryGetValue<RancherHubAccessToken>(cacheKey, out var cachedToken) && cachedToken is not null && !cachedToken.IsExpired()) | ||||||
|  |         { | ||||||
|  |             return cachedToken; | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         await _semaphore.WaitAsync(cancellationToken).ConfigureAwait(false); | ||||||
|  |         try | ||||||
|  |         { | ||||||
|  |             if (_cache.TryGetValue<RancherHubAccessToken>(cacheKey, out cachedToken) && cachedToken is not null && !cachedToken.IsExpired()) | ||||||
|  |             { | ||||||
|  |                 return cachedToken; | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             var token = await RequestTokenAsync(options, cancellationToken).ConfigureAwait(false); | ||||||
|  |             if (token is not null) | ||||||
|  |             { | ||||||
|  |                 var lifetime = token.ExpiresAt - DateTimeOffset.UtcNow; | ||||||
|  |                 if (lifetime <= TimeSpan.Zero) | ||||||
|  |                 { | ||||||
|  |                     lifetime = TimeSpan.FromMinutes(5); | ||||||
|  |                 } | ||||||
|  |  | ||||||
|  |                 var absoluteExpiration = lifetime > TimeSpan.FromSeconds(30) | ||||||
|  |                     ? DateTimeOffset.UtcNow + lifetime - TimeSpan.FromSeconds(30) | ||||||
|  |                     : DateTimeOffset.UtcNow + TimeSpan.FromSeconds(10); | ||||||
|  |  | ||||||
|  |                 _cache.Set(cacheKey, token, new MemoryCacheEntryOptions | ||||||
|  |                 { | ||||||
|  |                     AbsoluteExpiration = absoluteExpiration, | ||||||
|  |                 }); | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             return token; | ||||||
|  |         } | ||||||
|  |         finally | ||||||
|  |         { | ||||||
|  |             _semaphore.Release(); | ||||||
|  |         } | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     private async Task<RancherHubAccessToken?> RequestTokenAsync(RancherHubConnectorOptions options, CancellationToken cancellationToken) | ||||||
|  |     { | ||||||
|  |         using var request = new HttpRequestMessage(HttpMethod.Post, options.TokenEndpoint); | ||||||
|  |         request.Headers.Accept.ParseAdd("application/json"); | ||||||
|  |  | ||||||
|  |         var parameters = new Dictionary<string, string> | ||||||
|  |         { | ||||||
|  |             ["grant_type"] = "client_credentials", | ||||||
|  |         }; | ||||||
|  |  | ||||||
|  |         if (options.Scopes.Count > 0) | ||||||
|  |         { | ||||||
|  |             parameters["scope"] = string.Join(' ', options.Scopes); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         if (!string.IsNullOrWhiteSpace(options.Audience)) | ||||||
|  |         { | ||||||
|  |             parameters["audience"] = options.Audience!; | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         if (string.Equals(options.ClientAuthenticationScheme, "client_secret_basic", StringComparison.Ordinal)) | ||||||
|  |         { | ||||||
|  |             var credentials = Convert.ToBase64String(Encoding.UTF8.GetBytes($"{options.ClientId}:{options.ClientSecret}")); | ||||||
|  |             request.Headers.Authorization = new AuthenticationHeaderValue("Basic", credentials); | ||||||
|  |         } | ||||||
|  |         else | ||||||
|  |         { | ||||||
|  |             parameters["client_id"] = options.ClientId!; | ||||||
|  |             parameters["client_secret"] = options.ClientSecret!; | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         request.Content = new FormUrlEncodedContent(parameters); | ||||||
|  |  | ||||||
|  |         var client = _httpClientFactory.CreateClient(RancherHubConnectorOptions.HttpClientName); | ||||||
|  |         using var response = await client.SendAsync(request, cancellationToken).ConfigureAwait(false); | ||||||
|  |         if (!response.IsSuccessStatusCode) | ||||||
|  |         { | ||||||
|  |             var payload = await response.Content.ReadAsStringAsync(cancellationToken).ConfigureAwait(false); | ||||||
|  |             throw new InvalidOperationException($"Failed to acquire Rancher hub access token ({response.StatusCode}): {payload}"); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         await using var stream = await response.Content.ReadAsStreamAsync(cancellationToken).ConfigureAwait(false); | ||||||
|  |         using var document = await JsonDocument.ParseAsync(stream, cancellationToken: cancellationToken).ConfigureAwait(false); | ||||||
|  |         var root = document.RootElement; | ||||||
|  |  | ||||||
|  |         if (!root.TryGetProperty("access_token", out var accessTokenProperty) || accessTokenProperty.ValueKind is not JsonValueKind.String) | ||||||
|  |         { | ||||||
|  |             throw new InvalidOperationException("Token endpoint response missing access_token."); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         var token = accessTokenProperty.GetString(); | ||||||
|  |         if (string.IsNullOrWhiteSpace(token)) | ||||||
|  |         { | ||||||
|  |             throw new InvalidOperationException("Token endpoint response contained an empty access_token."); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         var tokenType = root.TryGetProperty("token_type", out var tokenTypeElement) && tokenTypeElement.ValueKind is JsonValueKind.String | ||||||
|  |             ? tokenTypeElement.GetString() ?? "Bearer" | ||||||
|  |             : "Bearer"; | ||||||
|  |  | ||||||
|  |         var expires = root.TryGetProperty("expires_in", out var expiresElement) && | ||||||
|  |                       expiresElement.ValueKind is JsonValueKind.Number && | ||||||
|  |                       expiresElement.TryGetInt32(out var expiresSeconds) | ||||||
|  |             ? DateTimeOffset.UtcNow + TimeSpan.FromSeconds(Math.Max(30, expiresSeconds)) | ||||||
|  |             : DateTimeOffset.UtcNow + TimeSpan.FromMinutes(30); | ||||||
|  |  | ||||||
|  |         _logger.LogDebug("Acquired Rancher hub access token (expires {Expires}).", expires); | ||||||
|  |  | ||||||
|  |         return new RancherHubAccessToken(token, tokenType, expires); | ||||||
|  |     } | ||||||
|  | } | ||||||
|  |  | ||||||
|  | public sealed record RancherHubAccessToken(string Value, string TokenType, DateTimeOffset ExpiresAt) | ||||||
|  | { | ||||||
|  |     public bool IsExpired() => DateTimeOffset.UtcNow >= ExpiresAt - TimeSpan.FromMinutes(1); | ||||||
|  | } | ||||||
| @@ -0,0 +1,186 @@ | |||||||
|  | using System; | ||||||
|  | using System.Collections.Generic; | ||||||
|  | using System.IO; | ||||||
|  | using System.IO.Abstractions; | ||||||
|  |  | ||||||
|  | namespace StellaOps.Vexer.Connectors.SUSE.RancherVEXHub.Configuration; | ||||||
|  |  | ||||||
|  | public sealed class RancherHubConnectorOptions | ||||||
|  | { | ||||||
|  |     public static readonly Uri DefaultDiscoveryUri = new("https://vexhub.suse.com/.well-known/vex/rancher-hub.json"); | ||||||
|  |  | ||||||
|  |     /// <summary> | ||||||
|  |     /// HTTP client name registered for the connector. | ||||||
|  |     /// </summary> | ||||||
|  |     public const string HttpClientName = "vexer.connector.suse.rancherhub"; | ||||||
|  |  | ||||||
|  |     /// <summary> | ||||||
|  |     /// URI for the Rancher VEX hub discovery document. | ||||||
|  |     /// </summary> | ||||||
|  |     public Uri DiscoveryUri { get; set; } = DefaultDiscoveryUri; | ||||||
|  |  | ||||||
|  |     /// <summary> | ||||||
|  |     /// Optional OAuth2/OIDC token endpoint used for hub authentication. | ||||||
|  |     /// </summary> | ||||||
|  |     public Uri? TokenEndpoint { get; set; } | ||||||
|  |  | ||||||
|  |     /// <summary> | ||||||
|  |     /// Client identifier used when requesting hub access tokens. | ||||||
|  |     /// </summary> | ||||||
|  |     public string? ClientId { get; set; } | ||||||
|  |  | ||||||
|  |     /// <summary> | ||||||
|  |     /// Client secret used when requesting hub access tokens. | ||||||
|  |     /// </summary> | ||||||
|  |     public string? ClientSecret { get; set; } | ||||||
|  |  | ||||||
|  |     /// <summary> | ||||||
|  |     /// OAuth scopes requested for hub access; defaults align with Rancher hub reader role. | ||||||
|  |     /// </summary> | ||||||
|  |     public IList<string> Scopes { get; } = new List<string> { "hub.read" }; | ||||||
|  |  | ||||||
|  |     /// <summary> | ||||||
|  |     /// Optional audience claim passed when requesting tokens (client credential grant). | ||||||
|  |     /// </summary> | ||||||
|  |     public string? Audience { get; set; } | ||||||
|  |  | ||||||
|  |     /// <summary> | ||||||
|  |     /// Preferred authentication scheme. Supported: client_secret_basic (default) or client_secret_post. | ||||||
|  |     /// </summary> | ||||||
|  |     public string ClientAuthenticationScheme { get; set; } = "client_secret_basic"; | ||||||
|  |  | ||||||
|  |     /// <summary> | ||||||
|  |     /// Duration to cache discovery metadata before re-fetching. | ||||||
|  |     /// </summary> | ||||||
|  |     public TimeSpan MetadataCacheDuration { get; set; } = TimeSpan.FromMinutes(30); | ||||||
|  |  | ||||||
|  |     /// <summary> | ||||||
|  |     /// Optional file path for discovery metadata snapshots. | ||||||
|  |     /// </summary> | ||||||
|  |     public string? OfflineSnapshotPath { get; set; } | ||||||
|  |  | ||||||
|  |     /// <summary> | ||||||
|  |     /// When true, the loader prefers the offline snapshot prior to attempting network discovery. | ||||||
|  |     /// </summary> | ||||||
|  |     public bool PreferOfflineSnapshot { get; set; } | ||||||
|  |  | ||||||
|  |     /// <summary> | ||||||
|  |     /// Enables persisting freshly fetched discovery documents to <see cref="OfflineSnapshotPath"/>. | ||||||
|  |     /// </summary> | ||||||
|  |     public bool PersistOfflineSnapshot { get; set; } = true; | ||||||
|  |  | ||||||
|  |     /// <summary> | ||||||
|  |     /// Weight applied to the provider entry; hubs default below direct vendor feeds. | ||||||
|  |     /// </summary> | ||||||
|  |     public double TrustWeight { get; set; } = 0.6; | ||||||
|  |  | ||||||
|  |     /// <summary> | ||||||
|  |     /// Optional Sigstore/Cosign issuer for verifying hub-delivered attestations. | ||||||
|  |     /// </summary> | ||||||
|  |     public string? CosignIssuer { get; set; } | ||||||
|  |  | ||||||
|  |     /// <summary> | ||||||
|  |     /// Cosign identity pattern matched against transparency log subjects. | ||||||
|  |     /// </summary> | ||||||
|  |     public string? CosignIdentityPattern { get; set; } | ||||||
|  |  | ||||||
|  |     /// <summary> | ||||||
|  |     /// Additional trusted PGP fingerprints declared by the hub. | ||||||
|  |     /// </summary> | ||||||
|  |     public IList<string> PgpFingerprints { get; } = new List<string>(); | ||||||
|  |  | ||||||
|  |     /// <summary> | ||||||
|  |     /// Allows falling back to unauthenticated discovery requests when credentials are absent. | ||||||
|  |     /// </summary> | ||||||
|  |     public bool AllowAnonymousDiscovery { get; set; } | ||||||
|  |  | ||||||
|  |     public void Validate(IFileSystem? fileSystem = null) | ||||||
|  |     { | ||||||
|  |         if (DiscoveryUri is null || !DiscoveryUri.IsAbsoluteUri) | ||||||
|  |         { | ||||||
|  |             throw new InvalidOperationException("DiscoveryUri must be an absolute URI."); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         if (DiscoveryUri.Scheme is not ("http" or "https")) | ||||||
|  |         { | ||||||
|  |             throw new InvalidOperationException("DiscoveryUri must use HTTP or HTTPS."); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         if (MetadataCacheDuration <= TimeSpan.Zero) | ||||||
|  |         { | ||||||
|  |             throw new InvalidOperationException("MetadataCacheDuration must be positive."); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         if (!string.IsNullOrWhiteSpace(OfflineSnapshotPath)) | ||||||
|  |         { | ||||||
|  |             var fs = fileSystem ?? new FileSystem(); | ||||||
|  |             var directory = Path.GetDirectoryName(OfflineSnapshotPath); | ||||||
|  |             if (!string.IsNullOrWhiteSpace(directory) && !fs.Directory.Exists(directory)) | ||||||
|  |             { | ||||||
|  |                 fs.Directory.CreateDirectory(directory); | ||||||
|  |             } | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         if (PreferOfflineSnapshot && string.IsNullOrWhiteSpace(OfflineSnapshotPath)) | ||||||
|  |         { | ||||||
|  |             throw new InvalidOperationException("OfflineSnapshotPath must be provided when PreferOfflineSnapshot is enabled."); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         var hasClientId = !string.IsNullOrWhiteSpace(ClientId); | ||||||
|  |         var hasClientSecret = !string.IsNullOrWhiteSpace(ClientSecret); | ||||||
|  |         var hasTokenEndpoint = TokenEndpoint is not null; | ||||||
|  |         if (hasClientId || hasClientSecret || hasTokenEndpoint) | ||||||
|  |         { | ||||||
|  |             if (!(hasClientId && hasClientSecret && hasTokenEndpoint)) | ||||||
|  |             { | ||||||
|  |                 throw new InvalidOperationException("ClientId, ClientSecret, and TokenEndpoint must be provided together for authenticated discovery."); | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             if (TokenEndpoint is not null && (!TokenEndpoint.IsAbsoluteUri || TokenEndpoint.Scheme is not ("http" or "https"))) | ||||||
|  |             { | ||||||
|  |                 throw new InvalidOperationException("TokenEndpoint must be an absolute HTTP(S) URI."); | ||||||
|  |             } | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         if (double.IsNaN(TrustWeight) || double.IsInfinity(TrustWeight)) | ||||||
|  |         { | ||||||
|  |             TrustWeight = 0.6; | ||||||
|  |         } | ||||||
|  |         else if (TrustWeight <= 0) | ||||||
|  |         { | ||||||
|  |             TrustWeight = 0.1; | ||||||
|  |         } | ||||||
|  |         else if (TrustWeight > 1.0) | ||||||
|  |         { | ||||||
|  |             TrustWeight = 1.0; | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         if (!string.IsNullOrWhiteSpace(CosignIssuer) && string.IsNullOrWhiteSpace(CosignIdentityPattern)) | ||||||
|  |         { | ||||||
|  |             throw new InvalidOperationException("CosignIdentityPattern must be provided when CosignIssuer is specified."); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         if (!string.Equals(ClientAuthenticationScheme, "client_secret_basic", StringComparison.Ordinal) && | ||||||
|  |             !string.Equals(ClientAuthenticationScheme, "client_secret_post", StringComparison.Ordinal)) | ||||||
|  |         { | ||||||
|  |             throw new InvalidOperationException("ClientAuthenticationScheme must be 'client_secret_basic' or 'client_secret_post'."); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         // Remove any empty scopes to avoid token request issues. | ||||||
|  |         if (Scopes.Count > 0) | ||||||
|  |         { | ||||||
|  |             for (var i = Scopes.Count - 1; i >= 0; i--) | ||||||
|  |             { | ||||||
|  |                 if (string.IsNullOrWhiteSpace(Scopes[i])) | ||||||
|  |                 { | ||||||
|  |                     Scopes.RemoveAt(i); | ||||||
|  |                 } | ||||||
|  |             } | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         if (Scopes.Count == 0) | ||||||
|  |         { | ||||||
|  |             Scopes.Add("hub.read"); | ||||||
|  |         } | ||||||
|  |     } | ||||||
|  | } | ||||||
| @@ -0,0 +1,32 @@ | |||||||
|  | using System; | ||||||
|  | using System.Collections.Generic; | ||||||
|  | using System.IO.Abstractions; | ||||||
|  | using StellaOps.Vexer.Connectors.Abstractions; | ||||||
|  |  | ||||||
|  | namespace StellaOps.Vexer.Connectors.SUSE.RancherVEXHub.Configuration; | ||||||
|  |  | ||||||
|  | public sealed class RancherHubConnectorOptionsValidator : IVexConnectorOptionsValidator<RancherHubConnectorOptions> | ||||||
|  | { | ||||||
|  |     private readonly IFileSystem _fileSystem; | ||||||
|  |  | ||||||
|  |     public RancherHubConnectorOptionsValidator(IFileSystem fileSystem) | ||||||
|  |     { | ||||||
|  |         _fileSystem = fileSystem ?? throw new ArgumentNullException(nameof(fileSystem)); | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     public void Validate(VexConnectorDescriptor descriptor, RancherHubConnectorOptions options, IList<string> errors) | ||||||
|  |     { | ||||||
|  |         ArgumentNullException.ThrowIfNull(descriptor); | ||||||
|  |         ArgumentNullException.ThrowIfNull(options); | ||||||
|  |         ArgumentNullException.ThrowIfNull(errors); | ||||||
|  |  | ||||||
|  |         try | ||||||
|  |         { | ||||||
|  |             options.Validate(_fileSystem); | ||||||
|  |         } | ||||||
|  |         catch (Exception ex) | ||||||
|  |         { | ||||||
|  |             errors.Add(ex.Message); | ||||||
|  |         } | ||||||
|  |     } | ||||||
|  | } | ||||||
| @@ -0,0 +1,49 @@ | |||||||
|  | using System; | ||||||
|  | using System.Net; | ||||||
|  | using System.Net.Http; | ||||||
|  | using Microsoft.Extensions.Caching.Memory; | ||||||
|  | using Microsoft.Extensions.DependencyInjection; | ||||||
|  | using Microsoft.Extensions.DependencyInjection.Extensions; | ||||||
|  | using StellaOps.Vexer.Connectors.Abstractions; | ||||||
|  | using StellaOps.Vexer.Connectors.SUSE.RancherVEXHub.Authentication; | ||||||
|  | using StellaOps.Vexer.Connectors.SUSE.RancherVEXHub.Configuration; | ||||||
|  | using StellaOps.Vexer.Connectors.SUSE.RancherVEXHub.Metadata; | ||||||
|  | using StellaOps.Vexer.Core; | ||||||
|  | using System.IO.Abstractions; | ||||||
|  |  | ||||||
|  | namespace StellaOps.Vexer.Connectors.SUSE.RancherVEXHub.DependencyInjection; | ||||||
|  |  | ||||||
|  | public static class RancherHubConnectorServiceCollectionExtensions | ||||||
|  | { | ||||||
|  |     public static IServiceCollection AddRancherHubConnector(this IServiceCollection services, Action<RancherHubConnectorOptions>? configure = null) | ||||||
|  |     { | ||||||
|  |         ArgumentNullException.ThrowIfNull(services); | ||||||
|  |  | ||||||
|  |         services.TryAddSingleton<IMemoryCache, MemoryCache>(); | ||||||
|  |         services.TryAddSingleton<IFileSystem, FileSystem>(); | ||||||
|  |  | ||||||
|  |         services.AddOptions<RancherHubConnectorOptions>() | ||||||
|  |             .Configure(options => | ||||||
|  |             { | ||||||
|  |                 configure?.Invoke(options); | ||||||
|  |             }); | ||||||
|  |  | ||||||
|  |         services.AddSingleton<IVexConnectorOptionsValidator<RancherHubConnectorOptions>, RancherHubConnectorOptionsValidator>(); | ||||||
|  |         services.AddSingleton<RancherHubTokenProvider>(); | ||||||
|  |         services.AddSingleton<RancherHubMetadataLoader>(); | ||||||
|  |         services.AddSingleton<IVexConnector, RancherHubConnector>(); | ||||||
|  |  | ||||||
|  |         services.AddHttpClient(RancherHubConnectorOptions.HttpClientName, client => | ||||||
|  |             { | ||||||
|  |                 client.Timeout = TimeSpan.FromSeconds(30); | ||||||
|  |                 client.DefaultRequestHeaders.UserAgent.ParseAdd("StellaOps.Vexer.Connectors.SUSE.RancherVEXHub/1.0"); | ||||||
|  |                 client.DefaultRequestHeaders.Accept.ParseAdd("application/json"); | ||||||
|  |             }) | ||||||
|  |             .ConfigurePrimaryHttpMessageHandler(() => new HttpClientHandler | ||||||
|  |             { | ||||||
|  |                 AutomaticDecompression = DecompressionMethods.All, | ||||||
|  |             }); | ||||||
|  |  | ||||||
|  |         return services; | ||||||
|  |     } | ||||||
|  | } | ||||||
| @@ -0,0 +1,455 @@ | |||||||
|  | using System; | ||||||
|  | using System.Collections.Generic; | ||||||
|  | using System.Collections.Immutable; | ||||||
|  | using System.IO.Abstractions; | ||||||
|  | using System.Net; | ||||||
|  | using System.Net.Http; | ||||||
|  | using System.Net.Http.Headers; | ||||||
|  | using System.Text.Json; | ||||||
|  | using System.Threading; | ||||||
|  | using System.Threading.Tasks; | ||||||
|  | using Microsoft.Extensions.Caching.Memory; | ||||||
|  | using Microsoft.Extensions.Logging; | ||||||
|  | using StellaOps.Vexer.Connectors.SUSE.RancherVEXHub.Authentication; | ||||||
|  | using StellaOps.Vexer.Connectors.SUSE.RancherVEXHub.Configuration; | ||||||
|  | using StellaOps.Vexer.Core; | ||||||
|  |  | ||||||
|  | namespace StellaOps.Vexer.Connectors.SUSE.RancherVEXHub.Metadata; | ||||||
|  |  | ||||||
|  | public sealed class RancherHubMetadataLoader | ||||||
|  | { | ||||||
|  |     public const string CachePrefix = "StellaOps.Vexer.Connectors.SUSE.RancherVEXHub.Metadata"; | ||||||
|  |  | ||||||
|  |     private readonly IHttpClientFactory _httpClientFactory; | ||||||
|  |     private readonly IMemoryCache _memoryCache; | ||||||
|  |     private readonly RancherHubTokenProvider _tokenProvider; | ||||||
|  |     private readonly IFileSystem _fileSystem; | ||||||
|  |     private readonly ILogger<RancherHubMetadataLoader> _logger; | ||||||
|  |     private readonly SemaphoreSlim _semaphore = new(1, 1); | ||||||
|  |     private readonly JsonDocumentOptions _documentOptions; | ||||||
|  |  | ||||||
|  |     public RancherHubMetadataLoader( | ||||||
|  |         IHttpClientFactory httpClientFactory, | ||||||
|  |         IMemoryCache memoryCache, | ||||||
|  |         RancherHubTokenProvider tokenProvider, | ||||||
|  |         IFileSystem fileSystem, | ||||||
|  |         ILogger<RancherHubMetadataLoader> logger) | ||||||
|  |     { | ||||||
|  |         _httpClientFactory = httpClientFactory ?? throw new ArgumentNullException(nameof(httpClientFactory)); | ||||||
|  |         _memoryCache = memoryCache ?? throw new ArgumentNullException(nameof(memoryCache)); | ||||||
|  |         _tokenProvider = tokenProvider ?? throw new ArgumentNullException(nameof(tokenProvider)); | ||||||
|  |         _fileSystem = fileSystem ?? throw new ArgumentNullException(nameof(fileSystem)); | ||||||
|  |         _logger = logger ?? throw new ArgumentNullException(nameof(logger)); | ||||||
|  |         _documentOptions = new JsonDocumentOptions | ||||||
|  |         { | ||||||
|  |             CommentHandling = JsonCommentHandling.Skip, | ||||||
|  |             AllowTrailingCommas = true, | ||||||
|  |         }; | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     public async Task<RancherHubMetadataResult> LoadAsync(RancherHubConnectorOptions options, CancellationToken cancellationToken) | ||||||
|  |     { | ||||||
|  |         ArgumentNullException.ThrowIfNull(options); | ||||||
|  |  | ||||||
|  |         var cacheKey = CreateCacheKey(options); | ||||||
|  |         if (_memoryCache.TryGetValue<CacheEntry>(cacheKey, out var cached) && cached is not null && !cached.IsExpired()) | ||||||
|  |         { | ||||||
|  |             _logger.LogDebug("Returning cached Rancher hub metadata (expires {Expires}).", cached.ExpiresAt); | ||||||
|  |             return new RancherHubMetadataResult(cached.Metadata, cached.FetchedAt, FromCache: true, cached.FromOfflineSnapshot); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         await _semaphore.WaitAsync(cancellationToken).ConfigureAwait(false); | ||||||
|  |         try | ||||||
|  |         { | ||||||
|  |             if (_memoryCache.TryGetValue<CacheEntry>(cacheKey, out cached) && cached is not null && !cached.IsExpired()) | ||||||
|  |             { | ||||||
|  |                 return new RancherHubMetadataResult(cached.Metadata, cached.FetchedAt, FromCache: true, cached.FromOfflineSnapshot); | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             CacheEntry? previous = cached; | ||||||
|  |             CacheEntry? entry = null; | ||||||
|  |  | ||||||
|  |             if (options.PreferOfflineSnapshot) | ||||||
|  |             { | ||||||
|  |                 entry = TryLoadFromOffline(options); | ||||||
|  |                 if (entry is null) | ||||||
|  |                 { | ||||||
|  |                     throw new InvalidOperationException("PreferOfflineSnapshot is enabled but no offline discovery snapshot was found."); | ||||||
|  |                 } | ||||||
|  |             } | ||||||
|  |             else | ||||||
|  |             { | ||||||
|  |                 entry = await TryFetchFromNetworkAsync(options, previous, cancellationToken).ConfigureAwait(false); | ||||||
|  |                 if (entry is null) | ||||||
|  |                 { | ||||||
|  |                     entry = TryLoadFromOffline(options); | ||||||
|  |                 } | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             if (entry is null) | ||||||
|  |             { | ||||||
|  |                 throw new InvalidOperationException("Unable to load Rancher hub discovery metadata from network or offline snapshot."); | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             _memoryCache.Set(cacheKey, entry, new MemoryCacheEntryOptions | ||||||
|  |             { | ||||||
|  |                 AbsoluteExpirationRelativeToNow = options.MetadataCacheDuration, | ||||||
|  |             }); | ||||||
|  |  | ||||||
|  |             return new RancherHubMetadataResult(entry.Metadata, entry.FetchedAt, FromCache: false, entry.FromOfflineSnapshot); | ||||||
|  |         } | ||||||
|  |         finally | ||||||
|  |         { | ||||||
|  |             _semaphore.Release(); | ||||||
|  |         } | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     private async Task<CacheEntry?> TryFetchFromNetworkAsync(RancherHubConnectorOptions options, CacheEntry? previous, CancellationToken cancellationToken) | ||||||
|  |     { | ||||||
|  |         try | ||||||
|  |         { | ||||||
|  |             var client = _httpClientFactory.CreateClient(RancherHubConnectorOptions.HttpClientName); | ||||||
|  |             using var request = new HttpRequestMessage(HttpMethod.Get, options.DiscoveryUri); | ||||||
|  |             request.Headers.Accept.ParseAdd("application/json"); | ||||||
|  |  | ||||||
|  |             if (!string.IsNullOrWhiteSpace(previous?.ETag) && EntityTagHeaderValue.TryParse(previous.ETag, out var etag)) | ||||||
|  |             { | ||||||
|  |                 request.Headers.IfNoneMatch.Add(etag); | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             var token = await _tokenProvider.GetAccessTokenAsync(options, cancellationToken).ConfigureAwait(false); | ||||||
|  |             if (token is not null) | ||||||
|  |             { | ||||||
|  |                 var scheme = string.IsNullOrWhiteSpace(token.TokenType) ? "Bearer" : token.TokenType; | ||||||
|  |                 request.Headers.Authorization = new AuthenticationHeaderValue(scheme, token.Value); | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             using var response = await client.SendAsync(request, HttpCompletionOption.ResponseHeadersRead, cancellationToken).ConfigureAwait(false); | ||||||
|  |  | ||||||
|  |             if (response.StatusCode == HttpStatusCode.NotModified && previous is not null) | ||||||
|  |             { | ||||||
|  |                 _logger.LogDebug("Rancher hub discovery document not modified (etag {ETag}).", previous.ETag); | ||||||
|  |                 return previous with | ||||||
|  |                 { | ||||||
|  |                     FetchedAt = DateTimeOffset.UtcNow, | ||||||
|  |                     ExpiresAt = DateTimeOffset.UtcNow + options.MetadataCacheDuration, | ||||||
|  |                     FromOfflineSnapshot = false, | ||||||
|  |                 }; | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             response.EnsureSuccessStatusCode(); | ||||||
|  |             var payload = await response.Content.ReadAsStringAsync(cancellationToken).ConfigureAwait(false); | ||||||
|  |             var metadata = ParseMetadata(payload, options); | ||||||
|  |             var entry = new CacheEntry( | ||||||
|  |                 metadata, | ||||||
|  |                 DateTimeOffset.UtcNow, | ||||||
|  |                 DateTimeOffset.UtcNow + options.MetadataCacheDuration, | ||||||
|  |                 response.Headers.ETag?.ToString(), | ||||||
|  |                 FromOfflineSnapshot: false, | ||||||
|  |                 Payload: payload); | ||||||
|  |  | ||||||
|  |             PersistOfflineSnapshot(options, payload); | ||||||
|  |             return entry; | ||||||
|  |         } | ||||||
|  |         catch (Exception ex) when (ex is not OperationCanceledException && !options.PreferOfflineSnapshot) | ||||||
|  |         { | ||||||
|  |             _logger.LogWarning(ex, "Failed to fetch Rancher hub discovery document; attempting offline snapshot fallback."); | ||||||
|  |             return null; | ||||||
|  |         } | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     private CacheEntry? TryLoadFromOffline(RancherHubConnectorOptions options) | ||||||
|  |     { | ||||||
|  |         if (string.IsNullOrWhiteSpace(options.OfflineSnapshotPath)) | ||||||
|  |         { | ||||||
|  |             return null; | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         if (!_fileSystem.File.Exists(options.OfflineSnapshotPath)) | ||||||
|  |         { | ||||||
|  |             _logger.LogWarning("Rancher hub offline snapshot path {Path} does not exist.", options.OfflineSnapshotPath); | ||||||
|  |             return null; | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         try | ||||||
|  |         { | ||||||
|  |             var payload = _fileSystem.File.ReadAllText(options.OfflineSnapshotPath); | ||||||
|  |             var metadata = ParseMetadata(payload, options); | ||||||
|  |             return new CacheEntry( | ||||||
|  |                 metadata, | ||||||
|  |                 DateTimeOffset.UtcNow, | ||||||
|  |                 DateTimeOffset.UtcNow + options.MetadataCacheDuration, | ||||||
|  |                 ETag: null, | ||||||
|  |                 FromOfflineSnapshot: true, | ||||||
|  |                 Payload: payload); | ||||||
|  |         } | ||||||
|  |         catch (Exception ex) | ||||||
|  |         { | ||||||
|  |             _logger.LogError(ex, "Failed to load Rancher hub discovery metadata from offline snapshot {Path}.", options.OfflineSnapshotPath); | ||||||
|  |             return null; | ||||||
|  |         } | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     private void PersistOfflineSnapshot(RancherHubConnectorOptions options, string payload) | ||||||
|  |     { | ||||||
|  |         if (!options.PersistOfflineSnapshot || string.IsNullOrWhiteSpace(options.OfflineSnapshotPath)) | ||||||
|  |         { | ||||||
|  |             return; | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         try | ||||||
|  |         { | ||||||
|  |             var directory = _fileSystem.Path.GetDirectoryName(options.OfflineSnapshotPath); | ||||||
|  |             if (!string.IsNullOrWhiteSpace(directory)) | ||||||
|  |             { | ||||||
|  |                 _fileSystem.Directory.CreateDirectory(directory); | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             _fileSystem.File.WriteAllText(options.OfflineSnapshotPath, payload); | ||||||
|  |             _logger.LogDebug("Persisted Rancher hub discovery snapshot to {Path}.", options.OfflineSnapshotPath); | ||||||
|  |         } | ||||||
|  |         catch (Exception ex) | ||||||
|  |         { | ||||||
|  |             _logger.LogWarning(ex, "Failed to persist Rancher hub discovery snapshot to {Path}.", options.OfflineSnapshotPath); | ||||||
|  |         } | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     private RancherHubMetadata ParseMetadata(string payload, RancherHubConnectorOptions options) | ||||||
|  |     { | ||||||
|  |         if (string.IsNullOrWhiteSpace(payload)) | ||||||
|  |         { | ||||||
|  |             throw new InvalidOperationException("Rancher hub discovery payload was empty."); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         try | ||||||
|  |         { | ||||||
|  |             using var document = JsonDocument.Parse(payload, _documentOptions); | ||||||
|  |             var root = document.RootElement; | ||||||
|  |  | ||||||
|  |             var hubId = ReadString(root, "hubId") ?? "vexer:suse:rancher"; | ||||||
|  |             var title = ReadString(root, "title") ?? ReadString(root, "displayName") ?? "SUSE Rancher VEX Hub"; | ||||||
|  |             var baseUri = ReadUri(root, "baseUri"); | ||||||
|  |  | ||||||
|  |             var subscriptionElement = TryGetProperty(root, "subscription"); | ||||||
|  |             if (!subscriptionElement.HasValue) | ||||||
|  |             { | ||||||
|  |                 throw new InvalidOperationException("Discovery payload missing subscription section."); | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             var subscription = subscriptionElement.Value; | ||||||
|  |             var eventsUri = ReadRequiredUri(subscription, "eventsUri", "eventsUrl", "eventsEndpoint"); | ||||||
|  |             var checkpointUri = ReadUri(subscription, "checkpointUri", "checkpointUrl", "checkpointEndpoint"); | ||||||
|  |             var channels = ReadStringArray(subscription, "channels", "defaultChannels", "products"); | ||||||
|  |             var scopes = ReadStringArray(subscription, "scopes", "defaultScopes"); | ||||||
|  |             var requiresAuth = ReadBoolean(subscription, "requiresAuthentication", defaultValue: options.TokenEndpoint is not null); | ||||||
|  |  | ||||||
|  |             var authenticationElement = TryGetProperty(root, "authentication"); | ||||||
|  |             var tokenEndpointFromMetadata = authenticationElement.HasValue | ||||||
|  |                 ? ReadUri(authenticationElement.Value, "tokenUri", "tokenEndpoint") ?? options.TokenEndpoint | ||||||
|  |                 : options.TokenEndpoint; | ||||||
|  |             var audience = authenticationElement.HasValue | ||||||
|  |                 ? ReadString(authenticationElement.Value, "audience", "aud") ?? options.Audience | ||||||
|  |                 : options.Audience; | ||||||
|  |  | ||||||
|  |             var offlineElement = TryGetProperty(root, "offline", "snapshot"); | ||||||
|  |             var offlineSnapshot = offlineElement.HasValue | ||||||
|  |                 ? BuildOfflineSnapshot(offlineElement.Value, options) | ||||||
|  |                 : null; | ||||||
|  |  | ||||||
|  |             var provider = BuildProvider(hubId, title, baseUri, eventsUri, options); | ||||||
|  |             var subscriptionMetadata = new RancherHubSubscriptionMetadata(eventsUri, checkpointUri, channels, scopes, requiresAuth); | ||||||
|  |             var authenticationMetadata = new RancherHubAuthenticationMetadata(tokenEndpointFromMetadata, audience); | ||||||
|  |  | ||||||
|  |             return new RancherHubMetadata(provider, subscriptionMetadata, authenticationMetadata, offlineSnapshot); | ||||||
|  |         } | ||||||
|  |         catch (JsonException ex) | ||||||
|  |         { | ||||||
|  |             throw new InvalidOperationException("Failed to parse Rancher hub discovery payload.", ex); | ||||||
|  |         } | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     private RancherHubOfflineSnapshotMetadata? BuildOfflineSnapshot(JsonElement element, RancherHubConnectorOptions options) | ||||||
|  |     { | ||||||
|  |         var snapshotUri = ReadUri(element, "snapshotUri", "uri", "url"); | ||||||
|  |         if (snapshotUri is null) | ||||||
|  |         { | ||||||
|  |             return null; | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         var checksum = ReadString(element, "sha256", "checksum", "digest"); | ||||||
|  |         DateTimeOffset? updatedAt = null; | ||||||
|  |         var updatedString = ReadString(element, "updated", "lastModified", "timestamp"); | ||||||
|  |         if (!string.IsNullOrWhiteSpace(updatedString) && DateTimeOffset.TryParse(updatedString, out var parsed)) | ||||||
|  |         { | ||||||
|  |             updatedAt = parsed; | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         return new RancherHubOfflineSnapshotMetadata(snapshotUri, checksum, updatedAt); | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     private VexProvider BuildProvider(string hubId, string title, Uri? baseUri, Uri eventsUri, RancherHubConnectorOptions options) | ||||||
|  |     { | ||||||
|  |         var baseUris = new List<Uri>(); | ||||||
|  |         if (baseUri is not null) | ||||||
|  |         { | ||||||
|  |             baseUris.Add(baseUri); | ||||||
|  |         } | ||||||
|  |         baseUris.Add(eventsUri); | ||||||
|  |  | ||||||
|  |         VexCosignTrust? cosign = null; | ||||||
|  |         if (!string.IsNullOrWhiteSpace(options.CosignIssuer) && !string.IsNullOrWhiteSpace(options.CosignIdentityPattern)) | ||||||
|  |         { | ||||||
|  |             cosign = new VexCosignTrust(options.CosignIssuer!, options.CosignIdentityPattern!); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         var trust = new VexProviderTrust(options.TrustWeight, cosign, options.PgpFingerprints); | ||||||
|  |         return new VexProvider(hubId, title, VexProviderKind.Hub, baseUris, new VexProviderDiscovery(options.DiscoveryUri, null), trust); | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     private static string CreateCacheKey(RancherHubConnectorOptions options) | ||||||
|  |         => $"{CachePrefix}:{options.DiscoveryUri}"; | ||||||
|  |  | ||||||
|  |     private static JsonElement? TryGetProperty(JsonElement element, params string[] propertyNames) | ||||||
|  |     { | ||||||
|  |         foreach (var name in propertyNames) | ||||||
|  |         { | ||||||
|  |             if (element.TryGetProperty(name, out var value) && value.ValueKind is not JsonValueKind.Null and not JsonValueKind.Undefined) | ||||||
|  |             { | ||||||
|  |                 return value; | ||||||
|  |             } | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         return null; | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     private static string? ReadString(JsonElement element, params string[] propertyNames) | ||||||
|  |     { | ||||||
|  |         var property = TryGetProperty(element, propertyNames); | ||||||
|  |         if (property is null || property.Value.ValueKind is not JsonValueKind.String) | ||||||
|  |         { | ||||||
|  |             return null; | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         var value = property.Value.GetString(); | ||||||
|  |         return string.IsNullOrWhiteSpace(value) ? null : value; | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     private static bool ReadBoolean(JsonElement element, string propertyName, bool defaultValue) | ||||||
|  |     { | ||||||
|  |         if (!element.TryGetProperty(propertyName, out var property)) | ||||||
|  |         { | ||||||
|  |             return defaultValue; | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         return property.ValueKind switch | ||||||
|  |         { | ||||||
|  |             JsonValueKind.True => true, | ||||||
|  |             JsonValueKind.False => false, | ||||||
|  |             JsonValueKind.String when bool.TryParse(property.GetString(), out var parsed) => parsed, | ||||||
|  |             _ => defaultValue, | ||||||
|  |         }; | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     private static ImmutableArray<string> ReadStringArray(JsonElement element, params string[] propertyNames) | ||||||
|  |     { | ||||||
|  |         var property = TryGetProperty(element, propertyNames); | ||||||
|  |         if (property is null) | ||||||
|  |         { | ||||||
|  |             return ImmutableArray<string>.Empty; | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         if (property.Value.ValueKind is JsonValueKind.Array) | ||||||
|  |         { | ||||||
|  |             var builder = ImmutableArray.CreateBuilder<string>(); | ||||||
|  |             foreach (var item in property.Value.EnumerateArray()) | ||||||
|  |             { | ||||||
|  |                 if (item.ValueKind is JsonValueKind.String) | ||||||
|  |                 { | ||||||
|  |                     var value = item.GetString(); | ||||||
|  |                     if (!string.IsNullOrWhiteSpace(value)) | ||||||
|  |                     { | ||||||
|  |                         builder.Add(value!); | ||||||
|  |                     } | ||||||
|  |                 } | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             return builder.Count == 0 ? ImmutableArray<string>.Empty : builder.ToImmutable(); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         if (property.Value.ValueKind is JsonValueKind.String) | ||||||
|  |         { | ||||||
|  |             var single = property.Value.GetString(); | ||||||
|  |             return string.IsNullOrWhiteSpace(single) | ||||||
|  |                 ? ImmutableArray<string>.Empty | ||||||
|  |                 : ImmutableArray.Create(single!); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         return ImmutableArray<string>.Empty; | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     private static Uri? ReadUri(JsonElement element, params string[] propertyNames) | ||||||
|  |     { | ||||||
|  |         var value = ReadString(element, propertyNames); | ||||||
|  |         if (string.IsNullOrWhiteSpace(value)) | ||||||
|  |         { | ||||||
|  |             return null; | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         if (!Uri.TryCreate(value, UriKind.Absolute, out var uri) || uri.Scheme is not ("http" or "https")) | ||||||
|  |         { | ||||||
|  |             throw new InvalidOperationException($"Discovery field '{string.Join("/", propertyNames)}' must be an absolute HTTP(S) URI."); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         return uri; | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     private static Uri ReadRequiredUri(JsonElement element, params string[] propertyNames) | ||||||
|  |     { | ||||||
|  |         var uri = ReadUri(element, propertyNames); | ||||||
|  |         if (uri is null) | ||||||
|  |         { | ||||||
|  |             throw new InvalidOperationException($"Discovery payload missing required URI field '{string.Join("/", propertyNames)}'."); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         return uri; | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     private sealed record CacheEntry( | ||||||
|  |         RancherHubMetadata Metadata, | ||||||
|  |         DateTimeOffset FetchedAt, | ||||||
|  |         DateTimeOffset ExpiresAt, | ||||||
|  |         string? ETag, | ||||||
|  |         bool FromOfflineSnapshot, | ||||||
|  |         string? Payload) | ||||||
|  |     { | ||||||
|  |         public bool IsExpired() => DateTimeOffset.UtcNow >= ExpiresAt; | ||||||
|  |     } | ||||||
|  | } | ||||||
|  |  | ||||||
|  | public sealed record RancherHubMetadata( | ||||||
|  |     VexProvider Provider, | ||||||
|  |     RancherHubSubscriptionMetadata Subscription, | ||||||
|  |     RancherHubAuthenticationMetadata Authentication, | ||||||
|  |     RancherHubOfflineSnapshotMetadata? OfflineSnapshot); | ||||||
|  |  | ||||||
|  | public sealed record RancherHubSubscriptionMetadata( | ||||||
|  |     Uri EventsUri, | ||||||
|  |     Uri? CheckpointUri, | ||||||
|  |     ImmutableArray<string> Channels, | ||||||
|  |     ImmutableArray<string> Scopes, | ||||||
|  |     bool RequiresAuthentication); | ||||||
|  |  | ||||||
|  | public sealed record RancherHubAuthenticationMetadata( | ||||||
|  |     Uri? TokenEndpoint, | ||||||
|  |     string? Audience); | ||||||
|  |  | ||||||
|  | public sealed record RancherHubOfflineSnapshotMetadata( | ||||||
|  |     Uri SnapshotUri, | ||||||
|  |     string? Sha256, | ||||||
|  |     DateTimeOffset? UpdatedAt); | ||||||
|  |  | ||||||
|  | public sealed record RancherHubMetadataResult( | ||||||
|  |     RancherHubMetadata Metadata, | ||||||
|  |     DateTimeOffset FetchedAt, | ||||||
|  |     bool FromCache, | ||||||
|  |     bool FromOfflineSnapshot); | ||||||
| @@ -0,0 +1,85 @@ | |||||||
|  | using System; | ||||||
|  | using System.Collections.Generic; | ||||||
|  | using System.Collections.Immutable; | ||||||
|  | using System.Runtime.CompilerServices; | ||||||
|  | using Microsoft.Extensions.Logging; | ||||||
|  | using StellaOps.Vexer.Connectors.Abstractions; | ||||||
|  | using StellaOps.Vexer.Connectors.SUSE.RancherVEXHub.Configuration; | ||||||
|  | using StellaOps.Vexer.Connectors.SUSE.RancherVEXHub.Metadata; | ||||||
|  | using StellaOps.Vexer.Core; | ||||||
|  |  | ||||||
|  | namespace StellaOps.Vexer.Connectors.SUSE.RancherVEXHub; | ||||||
|  |  | ||||||
|  | public sealed class RancherHubConnector : VexConnectorBase | ||||||
|  | { | ||||||
|  |     private static readonly VexConnectorDescriptor StaticDescriptor = new( | ||||||
|  |             id: "vexer:suse.rancher", | ||||||
|  |             kind: VexProviderKind.Hub, | ||||||
|  |             displayName: "SUSE Rancher VEX Hub") | ||||||
|  |         { | ||||||
|  |             Tags = ImmutableArray.Create("hub", "suse", "offline"), | ||||||
|  |         }; | ||||||
|  |  | ||||||
|  |     private readonly RancherHubMetadataLoader _metadataLoader; | ||||||
|  |     private readonly IEnumerable<IVexConnectorOptionsValidator<RancherHubConnectorOptions>> _validators; | ||||||
|  |  | ||||||
|  |     private RancherHubConnectorOptions? _options; | ||||||
|  |     private RancherHubMetadataResult? _metadata; | ||||||
|  |  | ||||||
|  |     public RancherHubConnector( | ||||||
|  |         RancherHubMetadataLoader metadataLoader, | ||||||
|  |         ILogger<RancherHubConnector> logger, | ||||||
|  |         TimeProvider timeProvider, | ||||||
|  |         IEnumerable<IVexConnectorOptionsValidator<RancherHubConnectorOptions>>? validators = null) | ||||||
|  |         : base(StaticDescriptor, logger, timeProvider) | ||||||
|  |     { | ||||||
|  |         _metadataLoader = metadataLoader ?? throw new ArgumentNullException(nameof(metadataLoader)); | ||||||
|  |         _validators = validators ?? Array.Empty<IVexConnectorOptionsValidator<RancherHubConnectorOptions>>(); | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     public override async ValueTask ValidateAsync(VexConnectorSettings settings, CancellationToken cancellationToken) | ||||||
|  |     { | ||||||
|  |         _options = VexConnectorOptionsBinder.Bind( | ||||||
|  |             Descriptor, | ||||||
|  |             settings, | ||||||
|  |             validators: _validators); | ||||||
|  |  | ||||||
|  |         _metadata = await _metadataLoader.LoadAsync(_options, cancellationToken).ConfigureAwait(false); | ||||||
|  |  | ||||||
|  |         LogConnectorEvent(LogLevel.Information, "validate", "Rancher hub discovery loaded.", new Dictionary<string, object?> | ||||||
|  |         { | ||||||
|  |             ["discoveryUri"] = _options.DiscoveryUri.ToString(), | ||||||
|  |             ["subscriptionUri"] = _metadata.Metadata.Subscription.EventsUri.ToString(), | ||||||
|  |             ["requiresAuth"] = _metadata.Metadata.Subscription.RequiresAuthentication, | ||||||
|  |             ["fromOffline"] = _metadata.FromOfflineSnapshot, | ||||||
|  |         }); | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     public override async IAsyncEnumerable<VexRawDocument> FetchAsync(VexConnectorContext context, [EnumeratorCancellation] CancellationToken cancellationToken) | ||||||
|  |     { | ||||||
|  |         ArgumentNullException.ThrowIfNull(context); | ||||||
|  |  | ||||||
|  |         if (_options is null) | ||||||
|  |         { | ||||||
|  |             throw new InvalidOperationException("Connector must be validated before fetch operations."); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         if (_metadata is null) | ||||||
|  |         { | ||||||
|  |             _metadata = await _metadataLoader.LoadAsync(_options, cancellationToken).ConfigureAwait(false); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         LogConnectorEvent(LogLevel.Debug, "fetch", "Rancher hub connector discovery ready; event ingestion will be implemented in VEXER-CONN-SUSE-01-002.", new Dictionary<string, object?> | ||||||
|  |         { | ||||||
|  |             ["since"] = context.Since?.ToString("O"), | ||||||
|  |             ["subscriptionUri"] = _metadata.Metadata.Subscription.EventsUri.ToString(), | ||||||
|  |         }); | ||||||
|  |  | ||||||
|  |         yield break; | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     public override ValueTask<VexClaimBatch> NormalizeAsync(VexRawDocument document, CancellationToken cancellationToken) | ||||||
|  |         => throw new NotSupportedException("RancherHubConnector relies on format-specific normalizers for CSAF/OpenVEX payloads."); | ||||||
|  |  | ||||||
|  |     public RancherHubMetadata? GetCachedMetadata() => _metadata?.Metadata; | ||||||
|  | } | ||||||
| @@ -0,0 +1,19 @@ | |||||||
|  | <Project Sdk="Microsoft.NET.Sdk"> | ||||||
|  |   <PropertyGroup> | ||||||
|  |     <TargetFramework>net10.0</TargetFramework> | ||||||
|  |     <LangVersion>preview</LangVersion> | ||||||
|  |     <Nullable>enable</Nullable> | ||||||
|  |     <ImplicitUsings>enable</ImplicitUsings> | ||||||
|  |     <TreatWarningsAsErrors>true</TreatWarningsAsErrors> | ||||||
|  |   </PropertyGroup> | ||||||
|  |   <ItemGroup> | ||||||
|  |     <ProjectReference Include="..\StellaOps.Vexer.Connectors.Abstractions\StellaOps.Vexer.Connectors.Abstractions.csproj" /> | ||||||
|  |     <ProjectReference Include="..\StellaOps.Vexer.Storage.Mongo\StellaOps.Vexer.Storage.Mongo.csproj" /> | ||||||
|  |   </ItemGroup> | ||||||
|  |   <ItemGroup> | ||||||
|  |     <PackageReference Include="Microsoft.Extensions.Caching.Memory" Version="8.0.0" /> | ||||||
|  |     <PackageReference Include="Microsoft.Extensions.Http" Version="8.0.0" /> | ||||||
|  |     <PackageReference Include="Microsoft.Extensions.Options" Version="8.0.0" /> | ||||||
|  |     <PackageReference Include="System.IO.Abstractions" Version="20.0.28" /> | ||||||
|  |   </ItemGroup> | ||||||
|  | </Project> | ||||||
| @@ -2,6 +2,6 @@ If you are working on this file you need to read docs/ARCHITECTURE_VEXER.md and | |||||||
| # TASKS | # TASKS | ||||||
| | Task | Owner(s) | Depends on | Notes | | | Task | Owner(s) | Depends on | Notes | | ||||||
| |---|---|---|---| | |---|---|---|---| | ||||||
| |VEXER-CONN-SUSE-01-001 – Rancher hub discovery & auth|Team Vexer Connectors – SUSE|VEXER-CONN-ABS-01-001|TODO – Implement hub discovery/subscription setup with credential handling and offline snapshot support.| | |VEXER-CONN-SUSE-01-001 – Rancher hub discovery & auth|Team Vexer Connectors – SUSE|VEXER-CONN-ABS-01-001|**DONE (2025-10-17)** – Added Rancher hub options/token provider, discovery metadata loader with offline snapshots + caching, connector shell, DI wiring, and unit tests covering network/offline paths.| | ||||||
| |VEXER-CONN-SUSE-01-002 – Checkpointed event ingestion|Team Vexer Connectors – SUSE|VEXER-CONN-SUSE-01-001, VEXER-STORAGE-01-003|TODO – Process hub events with resume checkpoints, deduplication, and quarantine path for malformed payloads.| | |VEXER-CONN-SUSE-01-002 – Checkpointed event ingestion|Team Vexer Connectors – SUSE|VEXER-CONN-SUSE-01-001, VEXER-STORAGE-01-003|TODO – Process hub events with resume checkpoints, deduplication, and quarantine path for malformed payloads.| | ||||||
| |VEXER-CONN-SUSE-01-003 – Trust metadata & policy hints|Team Vexer Connectors – SUSE|VEXER-CONN-SUSE-01-002, VEXER-POLICY-01-001|TODO – Emit provider trust configuration (signers, weight overrides) and attach provenance hints for consensus engine.| | |VEXER-CONN-SUSE-01-003 – Trust metadata & policy hints|Team Vexer Connectors – SUSE|VEXER-CONN-SUSE-01-002, VEXER-POLICY-01-001|TODO – Emit provider trust configuration (signers, weight overrides) and attach provenance hints for consensus engine.| | ||||||
|   | |||||||
| @@ -0,0 +1,172 @@ | |||||||
|  | using System.Collections.Generic; | ||||||
|  | using System.Net; | ||||||
|  | using System.Net.Http; | ||||||
|  | using System.Text; | ||||||
|  | using FluentAssertions; | ||||||
|  | using Microsoft.Extensions.Caching.Memory; | ||||||
|  | using Microsoft.Extensions.Logging.Abstractions; | ||||||
|  | using StellaOps.Vexer.Connectors.Ubuntu.CSAF.Configuration; | ||||||
|  | using StellaOps.Vexer.Connectors.Ubuntu.CSAF.Metadata; | ||||||
|  | using System.IO.Abstractions.TestingHelpers; | ||||||
|  | using Xunit; | ||||||
|  | using System.Threading; | ||||||
|  |  | ||||||
|  | namespace StellaOps.Vexer.Connectors.Ubuntu.CSAF.Tests.Metadata; | ||||||
|  |  | ||||||
|  | public sealed class UbuntuCatalogLoaderTests | ||||||
|  | { | ||||||
|  |     private const string SampleIndex = """ | ||||||
|  |         { | ||||||
|  |           "generated": "2025-10-10T00:00:00Z", | ||||||
|  |           "channels": [ | ||||||
|  |             { | ||||||
|  |               "name": "stable", | ||||||
|  |               "catalogUrl": "https://ubuntu.com/security/csaf/stable/catalog.json", | ||||||
|  |               "sha256": "abc", | ||||||
|  |               "lastUpdated": "2025-10-09T10:00:00Z" | ||||||
|  |             }, | ||||||
|  |             { | ||||||
|  |               "name": "esm", | ||||||
|  |               "catalogUrl": "https://ubuntu.com/security/csaf/esm/catalog.json", | ||||||
|  |               "sha256": "def", | ||||||
|  |               "lastUpdated": "2025-10-08T10:00:00Z" | ||||||
|  |             } | ||||||
|  |           ] | ||||||
|  |         } | ||||||
|  |         """; | ||||||
|  |  | ||||||
|  |     [Fact] | ||||||
|  |     public async Task LoadAsync_FetchesAndCachesIndex() | ||||||
|  |     { | ||||||
|  |         var handler = new TestHttpMessageHandler(new Dictionary<Uri, HttpResponseMessage> | ||||||
|  |         { | ||||||
|  |             [new Uri("https://ubuntu.com/security/csaf/index.json")] = CreateResponse(SampleIndex), | ||||||
|  |         }); | ||||||
|  |         var client = new HttpClient(handler); | ||||||
|  |         var factory = new SingleClientHttpClientFactory(client); | ||||||
|  |         var cache = new MemoryCache(new MemoryCacheOptions()); | ||||||
|  |         var fileSystem = new MockFileSystem(); | ||||||
|  |         var loader = new UbuntuCatalogLoader(factory, cache, fileSystem, NullLogger<UbuntuCatalogLoader>.Instance, new AdjustableTimeProvider()); | ||||||
|  |  | ||||||
|  |         var options = new UbuntuConnectorOptions | ||||||
|  |         { | ||||||
|  |             IndexUri = new Uri("https://ubuntu.com/security/csaf/index.json"), | ||||||
|  |             OfflineSnapshotPath = "/snapshots/ubuntu-index.json", | ||||||
|  |         }; | ||||||
|  |  | ||||||
|  |         var result = await loader.LoadAsync(options, CancellationToken.None); | ||||||
|  |         result.Metadata.Channels.Should().HaveCount(1); | ||||||
|  |         result.Metadata.Channels[0].Name.Should().Be("stable"); | ||||||
|  |         fileSystem.FileExists(options.OfflineSnapshotPath!).Should().BeTrue(); | ||||||
|  |         result.FromCache.Should().BeFalse(); | ||||||
|  |  | ||||||
|  |         handler.ResetInvocationCount(); | ||||||
|  |         var cached = await loader.LoadAsync(options, CancellationToken.None); | ||||||
|  |         cached.FromCache.Should().BeTrue(); | ||||||
|  |         handler.InvocationCount.Should().Be(0); | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     [Fact] | ||||||
|  |     public async Task LoadAsync_UsesOfflineSnapshotWhenPreferred() | ||||||
|  |     { | ||||||
|  |         var handler = new TestHttpMessageHandler(new Dictionary<Uri, HttpResponseMessage>()); | ||||||
|  |         var client = new HttpClient(handler); | ||||||
|  |         var factory = new SingleClientHttpClientFactory(client); | ||||||
|  |         var cache = new MemoryCache(new MemoryCacheOptions()); | ||||||
|  |         var fileSystem = new MockFileSystem(); | ||||||
|  |         fileSystem.AddFile("/snapshots/ubuntu-index.json", new MockFileData($"{{\"metadata\":{SampleIndex},\"fetchedAt\":\"2025-10-10T00:00:00Z\"}}")); | ||||||
|  |         var loader = new UbuntuCatalogLoader(factory, cache, fileSystem, NullLogger<UbuntuCatalogLoader>.Instance, new AdjustableTimeProvider()); | ||||||
|  |  | ||||||
|  |         var options = new UbuntuConnectorOptions | ||||||
|  |         { | ||||||
|  |             IndexUri = new Uri("https://ubuntu.com/security/csaf/index.json"), | ||||||
|  |             OfflineSnapshotPath = "/snapshots/ubuntu-index.json", | ||||||
|  |             PreferOfflineSnapshot = true, | ||||||
|  |             Channels = { "stable" } | ||||||
|  |         }; | ||||||
|  |  | ||||||
|  |         var result = await loader.LoadAsync(options, CancellationToken.None); | ||||||
|  |         result.FromOfflineSnapshot.Should().BeTrue(); | ||||||
|  |         result.Metadata.Channels.Should().NotBeEmpty(); | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     [Fact] | ||||||
|  |     public async Task LoadAsync_ThrowsWhenNoChannelsMatch() | ||||||
|  |     { | ||||||
|  |         var handler = new TestHttpMessageHandler(new Dictionary<Uri, HttpResponseMessage> | ||||||
|  |         { | ||||||
|  |             [new Uri("https://ubuntu.com/security/csaf/index.json")] = CreateResponse(SampleIndex), | ||||||
|  |         }); | ||||||
|  |         var client = new HttpClient(handler); | ||||||
|  |         var factory = new SingleClientHttpClientFactory(client); | ||||||
|  |         var cache = new MemoryCache(new MemoryCacheOptions()); | ||||||
|  |         var fileSystem = new MockFileSystem(); | ||||||
|  |         var loader = new UbuntuCatalogLoader(factory, cache, fileSystem, NullLogger<UbuntuCatalogLoader>.Instance, new AdjustableTimeProvider()); | ||||||
|  |  | ||||||
|  |         var options = new UbuntuConnectorOptions | ||||||
|  |         { | ||||||
|  |             IndexUri = new Uri("https://ubuntu.com/security/csaf/index.json"), | ||||||
|  |         }; | ||||||
|  |         options.Channels.Clear(); | ||||||
|  |         options.Channels.Add("nonexistent"); | ||||||
|  |  | ||||||
|  |         await Assert.ThrowsAsync<InvalidOperationException>(() => loader.LoadAsync(options, CancellationToken.None)); | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     private static HttpResponseMessage CreateResponse(string payload) | ||||||
|  |         => new(HttpStatusCode.OK) | ||||||
|  |         { | ||||||
|  |             Content = new StringContent(payload, Encoding.UTF8, "application/json"), | ||||||
|  |         }; | ||||||
|  |  | ||||||
|  |     private sealed class SingleClientHttpClientFactory : IHttpClientFactory | ||||||
|  |     { | ||||||
|  |         private readonly HttpClient _client; | ||||||
|  |  | ||||||
|  |         public SingleClientHttpClientFactory(HttpClient client) | ||||||
|  |         { | ||||||
|  |             _client = client; | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         public HttpClient CreateClient(string name) => _client; | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     private sealed class AdjustableTimeProvider : TimeProvider | ||||||
|  |     { | ||||||
|  |         private DateTimeOffset _now = DateTimeOffset.UtcNow; | ||||||
|  |  | ||||||
|  |         public override DateTimeOffset GetUtcNow() => _now; | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     private sealed class TestHttpMessageHandler : HttpMessageHandler | ||||||
|  |     { | ||||||
|  |         private readonly Dictionary<Uri, HttpResponseMessage> _responses; | ||||||
|  |  | ||||||
|  |         public TestHttpMessageHandler(Dictionary<Uri, HttpResponseMessage> responses) | ||||||
|  |         { | ||||||
|  |             _responses = responses; | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         public int InvocationCount { get; private set; } | ||||||
|  |  | ||||||
|  |         public void ResetInvocationCount() => InvocationCount = 0; | ||||||
|  |  | ||||||
|  |         protected override async Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) | ||||||
|  |         { | ||||||
|  |             InvocationCount++; | ||||||
|  |             if (request.RequestUri is not null && _responses.TryGetValue(request.RequestUri, out var response)) | ||||||
|  |             { | ||||||
|  |                 var payload = await response.Content.ReadAsStringAsync(cancellationToken).ConfigureAwait(false); | ||||||
|  |                 return new HttpResponseMessage(response.StatusCode) | ||||||
|  |                 { | ||||||
|  |                     Content = new StringContent(payload, Encoding.UTF8, "application/json"), | ||||||
|  |                 }; | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             return new HttpResponseMessage(HttpStatusCode.InternalServerError) | ||||||
|  |             { | ||||||
|  |                 Content = new StringContent("unexpected request"), | ||||||
|  |             }; | ||||||
|  |         } | ||||||
|  |     } | ||||||
|  | } | ||||||
| @@ -0,0 +1,17 @@ | |||||||
|  | <Project Sdk="Microsoft.NET.Sdk"> | ||||||
|  |   <PropertyGroup> | ||||||
|  |     <TargetFramework>net10.0</TargetFramework> | ||||||
|  |     <LangVersion>preview</LangVersion> | ||||||
|  |     <Nullable>enable</Nullable> | ||||||
|  |     <ImplicitUsings>enable</ImplicitUsings> | ||||||
|  |     <TreatWarningsAsErrors>true</TreatWarningsAsErrors> | ||||||
|  |   </PropertyGroup> | ||||||
|  |   <ItemGroup> | ||||||
|  |     <ProjectReference Include="..\StellaOps.Vexer.Connectors.Ubuntu.CSAF\StellaOps.Vexer.Connectors.Ubuntu.CSAF.csproj" /> | ||||||
|  |   </ItemGroup> | ||||||
|  |   <ItemGroup> | ||||||
|  |     <PackageReference Include="FluentAssertions" Version="6.12.0" /> | ||||||
|  |     <PackageReference Include="Microsoft.Extensions.Logging.Abstractions" Version="8.0.0" /> | ||||||
|  |     <PackageReference Include="System.IO.Abstractions.TestingHelpers" Version="20.0.28" /> | ||||||
|  |   </ItemGroup> | ||||||
|  | </Project> | ||||||
| @@ -0,0 +1,90 @@ | |||||||
|  | using System; | ||||||
|  | using System.Collections.Generic; | ||||||
|  | using System.IO; | ||||||
|  | using System.IO.Abstractions; | ||||||
|  |  | ||||||
|  | namespace StellaOps.Vexer.Connectors.Ubuntu.CSAF.Configuration; | ||||||
|  |  | ||||||
|  | public sealed class UbuntuConnectorOptions | ||||||
|  | { | ||||||
|  |     public const string HttpClientName = "vexer.connector.ubuntu.catalog"; | ||||||
|  |  | ||||||
|  |     /// <summary> | ||||||
|  |     /// Root index that lists Ubuntu CSAF channels. | ||||||
|  |     /// </summary> | ||||||
|  |     public Uri IndexUri { get; set; } = new("https://ubuntu.com/security/csaf/index.json"); | ||||||
|  |  | ||||||
|  |     /// <summary> | ||||||
|  |     /// Channels to include (e.g. stable, esm, lts). | ||||||
|  |     /// </summary> | ||||||
|  |     public IList<string> Channels { get; } = new List<string> { "stable" }; | ||||||
|  |  | ||||||
|  |     /// <summary> | ||||||
|  |     /// Duration to cache discovery metadata. | ||||||
|  |     /// </summary> | ||||||
|  |     public TimeSpan MetadataCacheDuration { get; set; } = TimeSpan.FromHours(4); | ||||||
|  |  | ||||||
|  |     /// <summary> | ||||||
|  |     /// Prefer offline snapshot when available. | ||||||
|  |     /// </summary> | ||||||
|  |     public bool PreferOfflineSnapshot { get; set; } | ||||||
|  |     /// <summary> | ||||||
|  |     /// Optional file path for offline index snapshot. | ||||||
|  |     /// </summary> | ||||||
|  |     public string? OfflineSnapshotPath { get; set; } | ||||||
|  |     /// <summary> | ||||||
|  |     /// Controls persistence of network responses to <see cref="OfflineSnapshotPath"/>. | ||||||
|  |     /// </summary> | ||||||
|  |     public bool PersistOfflineSnapshot { get; set; } = true; | ||||||
|  |  | ||||||
|  |     public void Validate(IFileSystem? fileSystem = null) | ||||||
|  |     { | ||||||
|  |         if (IndexUri is null || !IndexUri.IsAbsoluteUri) | ||||||
|  |         { | ||||||
|  |             throw new InvalidOperationException("IndexUri must be an absolute URI."); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         if (IndexUri.Scheme is not ("http" or "https")) | ||||||
|  |         { | ||||||
|  |             throw new InvalidOperationException("IndexUri must use HTTP or HTTPS."); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         if (Channels.Count == 0) | ||||||
|  |         { | ||||||
|  |             throw new InvalidOperationException("At least one channel must be specified."); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         for (var i = Channels.Count - 1; i >= 0; i--) | ||||||
|  |         { | ||||||
|  |             if (string.IsNullOrWhiteSpace(Channels[i])) | ||||||
|  |             { | ||||||
|  |                 Channels.RemoveAt(i); | ||||||
|  |             } | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         if (Channels.Count == 0) | ||||||
|  |         { | ||||||
|  |             throw new InvalidOperationException("Channel names cannot be empty."); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         if (MetadataCacheDuration <= TimeSpan.Zero) | ||||||
|  |         { | ||||||
|  |             throw new InvalidOperationException("MetadataCacheDuration must be positive."); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         if (PreferOfflineSnapshot && string.IsNullOrWhiteSpace(OfflineSnapshotPath)) | ||||||
|  |         { | ||||||
|  |             throw new InvalidOperationException("OfflineSnapshotPath must be provided when PreferOfflineSnapshot is enabled."); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         if (!string.IsNullOrWhiteSpace(OfflineSnapshotPath)) | ||||||
|  |         { | ||||||
|  |             var fs = fileSystem ?? new FileSystem(); | ||||||
|  |             var directory = Path.GetDirectoryName(OfflineSnapshotPath); | ||||||
|  |             if (!string.IsNullOrWhiteSpace(directory) && !fs.Directory.Exists(directory)) | ||||||
|  |             { | ||||||
|  |                 fs.Directory.CreateDirectory(directory); | ||||||
|  |             } | ||||||
|  |         } | ||||||
|  |     } | ||||||
|  | } | ||||||
| @@ -0,0 +1,32 @@ | |||||||
|  | using System; | ||||||
|  | using System.Collections.Generic; | ||||||
|  | using System.IO.Abstractions; | ||||||
|  | using StellaOps.Vexer.Connectors.Abstractions; | ||||||
|  |  | ||||||
|  | namespace StellaOps.Vexer.Connectors.Ubuntu.CSAF.Configuration; | ||||||
|  |  | ||||||
|  | public sealed class UbuntuConnectorOptionsValidator : IVexConnectorOptionsValidator<UbuntuConnectorOptions> | ||||||
|  | { | ||||||
|  |     private readonly IFileSystem _fileSystem; | ||||||
|  |  | ||||||
|  |     public UbuntuConnectorOptionsValidator(IFileSystem fileSystem) | ||||||
|  |     { | ||||||
|  |         _fileSystem = fileSystem ?? throw new ArgumentNullException(nameof(fileSystem)); | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     public void Validate(VexConnectorDescriptor descriptor, UbuntuConnectorOptions options, IList<string> errors) | ||||||
|  |     { | ||||||
|  |         ArgumentNullException.ThrowIfNull(descriptor); | ||||||
|  |         ArgumentNullException.ThrowIfNull(options); | ||||||
|  |         ArgumentNullException.ThrowIfNull(errors); | ||||||
|  |  | ||||||
|  |         try | ||||||
|  |         { | ||||||
|  |             options.Validate(_fileSystem); | ||||||
|  |         } | ||||||
|  |         catch (Exception ex) | ||||||
|  |         { | ||||||
|  |             errors.Add(ex.Message); | ||||||
|  |         } | ||||||
|  |     } | ||||||
|  | } | ||||||
| @@ -0,0 +1,45 @@ | |||||||
|  | using System; | ||||||
|  | using System.Net; | ||||||
|  | using System.Net.Http; | ||||||
|  | using Microsoft.Extensions.Caching.Memory; | ||||||
|  | using Microsoft.Extensions.DependencyInjection; | ||||||
|  | using Microsoft.Extensions.DependencyInjection.Extensions; | ||||||
|  | using StellaOps.Vexer.Connectors.Abstractions; | ||||||
|  | using StellaOps.Vexer.Connectors.Ubuntu.CSAF.Configuration; | ||||||
|  | using StellaOps.Vexer.Connectors.Ubuntu.CSAF.Metadata; | ||||||
|  | using StellaOps.Vexer.Core; | ||||||
|  | using System.IO.Abstractions; | ||||||
|  |  | ||||||
|  | namespace StellaOps.Vexer.Connectors.Ubuntu.CSAF.DependencyInjection; | ||||||
|  |  | ||||||
|  | public static class UbuntuConnectorServiceCollectionExtensions | ||||||
|  | { | ||||||
|  |     public static IServiceCollection AddUbuntuCsafConnector(this IServiceCollection services, Action<UbuntuConnectorOptions>? configure = null) | ||||||
|  |     { | ||||||
|  |         ArgumentNullException.ThrowIfNull(services); | ||||||
|  |  | ||||||
|  |         services.TryAddSingleton<IMemoryCache, MemoryCache>(); | ||||||
|  |         services.TryAddSingleton<IFileSystem, FileSystem>(); | ||||||
|  |  | ||||||
|  |         services.AddOptions<UbuntuConnectorOptions>() | ||||||
|  |             .Configure(options => configure?.Invoke(options)); | ||||||
|  |  | ||||||
|  |         services.AddSingleton<IVexConnectorOptionsValidator<UbuntuConnectorOptions>, UbuntuConnectorOptionsValidator>(); | ||||||
|  |  | ||||||
|  |         services.AddHttpClient(UbuntuConnectorOptions.HttpClientName, client => | ||||||
|  |             { | ||||||
|  |                 client.Timeout = TimeSpan.FromSeconds(60); | ||||||
|  |                 client.DefaultRequestHeaders.UserAgent.ParseAdd("StellaOps.Vexer.Connectors.Ubuntu.CSAF/1.0"); | ||||||
|  |                 client.DefaultRequestHeaders.Accept.ParseAdd("application/json"); | ||||||
|  |             }) | ||||||
|  |             .ConfigurePrimaryHttpMessageHandler(static () => new HttpClientHandler | ||||||
|  |             { | ||||||
|  |                 AutomaticDecompression = DecompressionMethods.All, | ||||||
|  |             }); | ||||||
|  |  | ||||||
|  |         services.AddSingleton<UbuntuCatalogLoader>(); | ||||||
|  |         services.AddSingleton<IVexConnector, UbuntuCsafConnector>(); | ||||||
|  |  | ||||||
|  |         return services; | ||||||
|  |     } | ||||||
|  | } | ||||||
| @@ -0,0 +1,248 @@ | |||||||
|  | using System; | ||||||
|  | using System.Collections.Generic; | ||||||
|  | using System.Collections.Immutable; | ||||||
|  | using System.IO.Abstractions; | ||||||
|  | using System.Net; | ||||||
|  | using System.Net.Http; | ||||||
|  | using System.Text.Json; | ||||||
|  | using System.Threading; | ||||||
|  | using System.Threading.Tasks; | ||||||
|  | using Microsoft.Extensions.Caching.Memory; | ||||||
|  | using Microsoft.Extensions.Logging; | ||||||
|  | using StellaOps.Vexer.Connectors.Ubuntu.CSAF.Configuration; | ||||||
|  |  | ||||||
|  | namespace StellaOps.Vexer.Connectors.Ubuntu.CSAF.Metadata; | ||||||
|  |  | ||||||
|  | public sealed class UbuntuCatalogLoader | ||||||
|  | { | ||||||
|  |     public const string CachePrefix = "StellaOps.Vexer.Connectors.Ubuntu.CSAF.Index"; | ||||||
|  |  | ||||||
|  |     private readonly IHttpClientFactory _httpClientFactory; | ||||||
|  |     private readonly IMemoryCache _memoryCache; | ||||||
|  |     private readonly IFileSystem _fileSystem; | ||||||
|  |     private readonly ILogger<UbuntuCatalogLoader> _logger; | ||||||
|  |     private readonly TimeProvider _timeProvider; | ||||||
|  |     private readonly SemaphoreSlim _semaphore = new(1, 1); | ||||||
|  |     private readonly JsonSerializerOptions _serializerOptions = new(JsonSerializerDefaults.Web); | ||||||
|  |  | ||||||
|  |     public UbuntuCatalogLoader( | ||||||
|  |         IHttpClientFactory httpClientFactory, | ||||||
|  |         IMemoryCache memoryCache, | ||||||
|  |         IFileSystem fileSystem, | ||||||
|  |         ILogger<UbuntuCatalogLoader> logger, | ||||||
|  |         TimeProvider? timeProvider = null) | ||||||
|  |     { | ||||||
|  |         _httpClientFactory = httpClientFactory ?? throw new ArgumentNullException(nameof(httpClientFactory)); | ||||||
|  |         _memoryCache = memoryCache ?? throw new ArgumentNullException(nameof(memoryCache)); | ||||||
|  |         _fileSystem = fileSystem ?? throw new ArgumentNullException(nameof(fileSystem)); | ||||||
|  |         _logger = logger ?? throw new ArgumentNullException(nameof(logger)); | ||||||
|  |         _timeProvider = timeProvider ?? TimeProvider.System; | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     public async Task<UbuntuCatalogResult> LoadAsync(UbuntuConnectorOptions options, CancellationToken cancellationToken) | ||||||
|  |     { | ||||||
|  |         ArgumentNullException.ThrowIfNull(options); | ||||||
|  |         options.Validate(_fileSystem); | ||||||
|  |  | ||||||
|  |         var cacheKey = CreateCacheKey(options); | ||||||
|  |         if (_memoryCache.TryGetValue<CacheEntry>(cacheKey, out var cached) && cached is not null && !cached.IsExpired(_timeProvider.GetUtcNow())) | ||||||
|  |         { | ||||||
|  |             return cached.ToResult(fromCache: true); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         await _semaphore.WaitAsync(cancellationToken).ConfigureAwait(false); | ||||||
|  |         try | ||||||
|  |         { | ||||||
|  |             if (_memoryCache.TryGetValue<CacheEntry>(cacheKey, out cached) && cached is not null && !cached.IsExpired(_timeProvider.GetUtcNow())) | ||||||
|  |             { | ||||||
|  |                 return cached.ToResult(fromCache: true); | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             CacheEntry? entry = null; | ||||||
|  |             if (options.PreferOfflineSnapshot) | ||||||
|  |             { | ||||||
|  |                 entry = LoadFromOffline(options); | ||||||
|  |                 if (entry is null) | ||||||
|  |                 { | ||||||
|  |                     throw new InvalidOperationException("PreferOfflineSnapshot is enabled but no offline Ubuntu snapshot was found."); | ||||||
|  |                 } | ||||||
|  |             } | ||||||
|  |             else | ||||||
|  |             { | ||||||
|  |                 entry = await TryFetchFromNetworkAsync(options, cancellationToken).ConfigureAwait(false) | ||||||
|  |                     ?? LoadFromOffline(options); | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             if (entry is null) | ||||||
|  |             { | ||||||
|  |                 throw new InvalidOperationException("Unable to load Ubuntu CSAF index from network or offline snapshot."); | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             var cacheOptions = new MemoryCacheEntryOptions(); | ||||||
|  |             if (entry.MetadataCacheDuration > TimeSpan.Zero) | ||||||
|  |             { | ||||||
|  |                 cacheOptions.AbsoluteExpiration = _timeProvider.GetUtcNow().Add(entry.MetadataCacheDuration); | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             _memoryCache.Set(cacheKey, entry with { CachedAt = _timeProvider.GetUtcNow() }, cacheOptions); | ||||||
|  |             return entry.ToResult(fromCache: false); | ||||||
|  |         } | ||||||
|  |         finally | ||||||
|  |         { | ||||||
|  |             _semaphore.Release(); | ||||||
|  |         } | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     private async Task<CacheEntry?> TryFetchFromNetworkAsync(UbuntuConnectorOptions options, CancellationToken cancellationToken) | ||||||
|  |     { | ||||||
|  |         try | ||||||
|  |         { | ||||||
|  |             var client = _httpClientFactory.CreateClient(UbuntuConnectorOptions.HttpClientName); | ||||||
|  |             using var response = await client.GetAsync(options.IndexUri, HttpCompletionOption.ResponseHeadersRead, cancellationToken).ConfigureAwait(false); | ||||||
|  |             response.EnsureSuccessStatusCode(); | ||||||
|  |             var payload = await response.Content.ReadAsStringAsync(cancellationToken).ConfigureAwait(false); | ||||||
|  |  | ||||||
|  |             var metadata = ParseMetadata(payload, options.Channels); | ||||||
|  |             var now = _timeProvider.GetUtcNow(); | ||||||
|  |             var entry = new CacheEntry(metadata, now, now, options.MetadataCacheDuration, false); | ||||||
|  |  | ||||||
|  |             PersistSnapshotIfNeeded(options, metadata, now); | ||||||
|  |             return entry; | ||||||
|  |         } | ||||||
|  |         catch (Exception ex) when (ex is not OperationCanceledException) | ||||||
|  |         { | ||||||
|  |             _logger.LogWarning(ex, "Failed to fetch Ubuntu CSAF index from {Uri}; attempting offline fallback if available.", options.IndexUri); | ||||||
|  |             return null; | ||||||
|  |         } | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     private CacheEntry? LoadFromOffline(UbuntuConnectorOptions options) | ||||||
|  |     { | ||||||
|  |         if (string.IsNullOrWhiteSpace(options.OfflineSnapshotPath)) | ||||||
|  |         { | ||||||
|  |             return null; | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         if (!_fileSystem.File.Exists(options.OfflineSnapshotPath)) | ||||||
|  |         { | ||||||
|  |             _logger.LogWarning("Ubuntu offline snapshot path {Path} does not exist.", options.OfflineSnapshotPath); | ||||||
|  |             return null; | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         try | ||||||
|  |         { | ||||||
|  |             var payload = _fileSystem.File.ReadAllText(options.OfflineSnapshotPath); | ||||||
|  |             var snapshot = JsonSerializer.Deserialize<UbuntuCatalogSnapshot>(payload, _serializerOptions); | ||||||
|  |             if (snapshot is null) | ||||||
|  |             { | ||||||
|  |                 throw new InvalidOperationException("Offline snapshot payload was empty."); | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             return new CacheEntry(snapshot.Metadata, snapshot.FetchedAt, _timeProvider.GetUtcNow(), options.MetadataCacheDuration, true); | ||||||
|  |         } | ||||||
|  |         catch (Exception ex) | ||||||
|  |         { | ||||||
|  |             _logger.LogError(ex, "Failed to load Ubuntu CSAF index from offline snapshot {Path}.", options.OfflineSnapshotPath); | ||||||
|  |             return null; | ||||||
|  |         } | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     private UbuntuCatalogMetadata ParseMetadata(string payload, IList<string> channels) | ||||||
|  |     { | ||||||
|  |         if (string.IsNullOrWhiteSpace(payload)) | ||||||
|  |         { | ||||||
|  |             throw new InvalidOperationException("Ubuntu index payload was empty."); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         using var document = JsonDocument.Parse(payload); | ||||||
|  |         var root = document.RootElement; | ||||||
|  |  | ||||||
|  |         var generatedAt = root.TryGetProperty("generated", out var generatedElement) && generatedElement.ValueKind == JsonValueKind.String && DateTimeOffset.TryParse(generatedElement.GetString(), out var generated) | ||||||
|  |             ? generated | ||||||
|  |             : _timeProvider.GetUtcNow(); | ||||||
|  |  | ||||||
|  |         var channelSet = new HashSet<string>(channels, StringComparer.OrdinalIgnoreCase); | ||||||
|  |  | ||||||
|  |         if (!root.TryGetProperty("channels", out var channelsElement) || channelsElement.ValueKind is not JsonValueKind.Array) | ||||||
|  |         { | ||||||
|  |             throw new InvalidOperationException("Ubuntu index did not include a channels array."); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         var builder = ImmutableArray.CreateBuilder<UbuChannelCatalog>(); | ||||||
|  |         foreach (var channelElement in channelsElement.EnumerateArray()) | ||||||
|  |         { | ||||||
|  |             var name = channelElement.TryGetProperty("name", out var nameElement) && nameElement.ValueKind == JsonValueKind.String ? nameElement.GetString() : null; | ||||||
|  |             if (string.IsNullOrWhiteSpace(name) || !channelSet.Contains(name)) | ||||||
|  |             { | ||||||
|  |                 continue; | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             if (!channelElement.TryGetProperty("catalogUrl", out var urlElement) || urlElement.ValueKind != JsonValueKind.String || !Uri.TryCreate(urlElement.GetString(), UriKind.Absolute, out var catalogUri)) | ||||||
|  |             { | ||||||
|  |                 _logger.LogWarning("Channel {Channel} did not specify a valid catalogUrl.", name); | ||||||
|  |                 continue; | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             string? sha256 = null; | ||||||
|  |             if (channelElement.TryGetProperty("sha256", out var shaElement) && shaElement.ValueKind == JsonValueKind.String) | ||||||
|  |             { | ||||||
|  |                 sha256 = shaElement.GetString(); | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             DateTimeOffset? lastUpdated = null; | ||||||
|  |             if (channelElement.TryGetProperty("lastUpdated", out var updatedElement) && updatedElement.ValueKind == JsonValueKind.String && DateTimeOffset.TryParse(updatedElement.GetString(), out var updated)) | ||||||
|  |             { | ||||||
|  |                 lastUpdated = updated; | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             builder.Add(new UbuChannelCatalog(name!, catalogUri, sha256, lastUpdated)); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         if (builder.Count == 0) | ||||||
|  |         { | ||||||
|  |             throw new InvalidOperationException("None of the requested Ubuntu channels were present in the index."); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         return new UbuntuCatalogMetadata(generatedAt, builder.ToImmutable()); | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     private void PersistSnapshotIfNeeded(UbuntuConnectorOptions options, UbuntuCatalogMetadata metadata, DateTimeOffset fetchedAt) | ||||||
|  |     { | ||||||
|  |         if (!options.PersistOfflineSnapshot || string.IsNullOrWhiteSpace(options.OfflineSnapshotPath)) | ||||||
|  |         { | ||||||
|  |             return; | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         try | ||||||
|  |         { | ||||||
|  |             var snapshot = new UbuntuCatalogSnapshot(metadata, fetchedAt); | ||||||
|  |             var payload = JsonSerializer.Serialize(snapshot, _serializerOptions); | ||||||
|  |             _fileSystem.File.WriteAllText(options.OfflineSnapshotPath, payload); | ||||||
|  |             _logger.LogDebug("Persisted Ubuntu CSAF index snapshot to {Path}.", options.OfflineSnapshotPath); | ||||||
|  |         } | ||||||
|  |         catch (Exception ex) | ||||||
|  |         { | ||||||
|  |             _logger.LogWarning(ex, "Failed to persist Ubuntu CSAF index snapshot to {Path}.", options.OfflineSnapshotPath); | ||||||
|  |         } | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     private static string CreateCacheKey(UbuntuConnectorOptions options) | ||||||
|  |         => $"{CachePrefix}:{options.IndexUri}:{string.Join(',', options.Channels)}"; | ||||||
|  |  | ||||||
|  |     private sealed record CacheEntry(UbuntuCatalogMetadata Metadata, DateTimeOffset FetchedAt, DateTimeOffset CachedAt, TimeSpan MetadataCacheDuration, bool FromOfflineSnapshot) | ||||||
|  |     { | ||||||
|  |         public bool IsExpired(DateTimeOffset now) | ||||||
|  |             => MetadataCacheDuration > TimeSpan.Zero && now >= CachedAt + MetadataCacheDuration; | ||||||
|  |  | ||||||
|  |         public UbuntuCatalogResult ToResult(bool fromCache) | ||||||
|  |             => new(Metadata, FetchedAt, fromCache, FromOfflineSnapshot); | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     private sealed record UbuntuCatalogSnapshot(UbuntuCatalogMetadata Metadata, DateTimeOffset FetchedAt); | ||||||
|  | } | ||||||
|  |  | ||||||
|  | public sealed record UbuntuCatalogMetadata(DateTimeOffset GeneratedAt, ImmutableArray<UbuChannelCatalog> Channels); | ||||||
|  |  | ||||||
|  | public sealed record UbuChannelCatalog(string Name, Uri CatalogUri, string? Sha256, DateTimeOffset? LastUpdated); | ||||||
|  |  | ||||||
|  | public sealed record UbuntuCatalogResult(UbuntuCatalogMetadata Metadata, DateTimeOffset FetchedAt, bool FromCache, bool FromOfflineSnapshot); | ||||||
| @@ -0,0 +1,18 @@ | |||||||
|  | <Project Sdk="Microsoft.NET.Sdk"> | ||||||
|  |   <PropertyGroup> | ||||||
|  |     <TargetFramework>net10.0</TargetFramework> | ||||||
|  |     <LangVersion>preview</LangVersion> | ||||||
|  |     <Nullable>enable</Nullable> | ||||||
|  |     <ImplicitUsings>enable</ImplicitUsings> | ||||||
|  |     <TreatWarningsAsErrors>true</TreatWarningsAsErrors> | ||||||
|  |   </PropertyGroup> | ||||||
|  |   <ItemGroup> | ||||||
|  |     <ProjectReference Include="..\StellaOps.Vexer.Connectors.Abstractions\StellaOps.Vexer.Connectors.Abstractions.csproj" /> | ||||||
|  |   </ItemGroup> | ||||||
|  |   <ItemGroup> | ||||||
|  |     <PackageReference Include="Microsoft.Extensions.Caching.Memory" Version="8.0.0" /> | ||||||
|  |     <PackageReference Include="Microsoft.Extensions.Http" Version="8.0.0" /> | ||||||
|  |     <PackageReference Include="Microsoft.Extensions.Options" Version="8.0.0" /> | ||||||
|  |     <PackageReference Include="System.IO.Abstractions" Version="20.0.28" /> | ||||||
|  |   </ItemGroup> | ||||||
|  | </Project> | ||||||
| @@ -2,6 +2,6 @@ If you are working on this file you need to read docs/ARCHITECTURE_VEXER.md and | |||||||
| # TASKS | # TASKS | ||||||
| | Task | Owner(s) | Depends on | Notes | | | Task | Owner(s) | Depends on | Notes | | ||||||
| |---|---|---|---| | |---|---|---|---| | ||||||
| |VEXER-CONN-UBUNTU-01-001 – Ubuntu CSAF discovery & channels|Team Vexer Connectors – Ubuntu|VEXER-CONN-ABS-01-001|TODO – Implement discovery of Ubuntu CSAF catalogs, channel selection (stable/LTS), and offline snapshot import.| | |VEXER-CONN-UBUNTU-01-001 – Ubuntu CSAF discovery & channels|Team Vexer Connectors – Ubuntu|VEXER-CONN-ABS-01-001|**DONE (2025-10-17)** – Added Ubuntu connector project with configurable channel options, catalog loader (network/offline), DI wiring, and discovery unit tests.| | ||||||
| |VEXER-CONN-UBUNTU-01-002 – Incremental fetch & deduplication|Team Vexer Connectors – Ubuntu|VEXER-CONN-UBUNTU-01-001, VEXER-STORAGE-01-003|TODO – Fetch CSAF bundles with ETag handling, checksum validation, deduplication, and raw persistence.| | |VEXER-CONN-UBUNTU-01-002 – Incremental fetch & deduplication|Team Vexer Connectors – Ubuntu|VEXER-CONN-UBUNTU-01-001, VEXER-STORAGE-01-003|TODO – Fetch CSAF bundles with ETag handling, checksum validation, deduplication, and raw persistence.| | ||||||
| |VEXER-CONN-UBUNTU-01-003 – Trust metadata & provenance|Team Vexer Connectors – Ubuntu|VEXER-CONN-UBUNTU-01-002, VEXER-POLICY-01-001|TODO – Emit Ubuntu signing metadata (GPG fingerprints) plus provenance hints for policy weighting and diagnostics.| | |VEXER-CONN-UBUNTU-01-003 – Trust metadata & provenance|Team Vexer Connectors – Ubuntu|VEXER-CONN-UBUNTU-01-002, VEXER-POLICY-01-001|TODO – Emit Ubuntu signing metadata (GPG fingerprints) plus provenance hints for policy weighting and diagnostics.| | ||||||
|   | |||||||
| @@ -0,0 +1,80 @@ | |||||||
|  | using System.Collections.Generic; | ||||||
|  | using System.Collections.Immutable; | ||||||
|  | using System.Runtime.CompilerServices; | ||||||
|  | using Microsoft.Extensions.Logging; | ||||||
|  | using StellaOps.Vexer.Connectors.Abstractions; | ||||||
|  | using StellaOps.Vexer.Connectors.Ubuntu.CSAF.Configuration; | ||||||
|  | using StellaOps.Vexer.Connectors.Ubuntu.CSAF.Metadata; | ||||||
|  | using StellaOps.Vexer.Core; | ||||||
|  |  | ||||||
|  | namespace StellaOps.Vexer.Connectors.Ubuntu.CSAF; | ||||||
|  |  | ||||||
|  | public sealed class UbuntuCsafConnector : VexConnectorBase | ||||||
|  | { | ||||||
|  |     private static readonly VexConnectorDescriptor DescriptorInstance = new( | ||||||
|  |         id: "vexer:ubuntu", | ||||||
|  |         kind: VexProviderKind.Distro, | ||||||
|  |         displayName: "Ubuntu CSAF") | ||||||
|  |     { | ||||||
|  |         Tags = ImmutableArray.Create("ubuntu", "csaf", "usn"), | ||||||
|  |     }; | ||||||
|  |  | ||||||
|  |     private readonly UbuntuCatalogLoader _catalogLoader; | ||||||
|  |     private readonly IEnumerable<IVexConnectorOptionsValidator<UbuntuConnectorOptions>> _validators; | ||||||
|  |  | ||||||
|  |     private UbuntuConnectorOptions? _options; | ||||||
|  |     private UbuntuCatalogResult? _catalog; | ||||||
|  |  | ||||||
|  |     public UbuntuCsafConnector( | ||||||
|  |         UbuntuCatalogLoader catalogLoader, | ||||||
|  |         IEnumerable<IVexConnectorOptionsValidator<UbuntuConnectorOptions>> validators, | ||||||
|  |         ILogger<UbuntuCsafConnector> logger, | ||||||
|  |         TimeProvider timeProvider) | ||||||
|  |         : base(DescriptorInstance, logger, timeProvider) | ||||||
|  |     { | ||||||
|  |         _catalogLoader = catalogLoader ?? throw new ArgumentNullException(nameof(catalogLoader)); | ||||||
|  |         _validators = validators ?? Array.Empty<IVexConnectorOptionsValidator<UbuntuConnectorOptions>>(); | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     public override async ValueTask ValidateAsync(VexConnectorSettings settings, CancellationToken cancellationToken) | ||||||
|  |     { | ||||||
|  |         _options = VexConnectorOptionsBinder.Bind( | ||||||
|  |             Descriptor, | ||||||
|  |             settings, | ||||||
|  |             validators: _validators); | ||||||
|  |  | ||||||
|  |         _catalog = await _catalogLoader.LoadAsync(_options, cancellationToken).ConfigureAwait(false); | ||||||
|  |         LogConnectorEvent(LogLevel.Information, "validate", "Ubuntu CSAF index loaded.", new Dictionary<string, object?> | ||||||
|  |         { | ||||||
|  |             ["channelCount"] = _catalog.Metadata.Channels.Length, | ||||||
|  |             ["fromOffline"] = _catalog.FromOfflineSnapshot, | ||||||
|  |         }); | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     public override async IAsyncEnumerable<VexRawDocument> FetchAsync(VexConnectorContext context, [EnumeratorCancellation] CancellationToken cancellationToken) | ||||||
|  |     { | ||||||
|  |         ArgumentNullException.ThrowIfNull(context); | ||||||
|  |  | ||||||
|  |         if (_options is null) | ||||||
|  |         { | ||||||
|  |             throw new InvalidOperationException("Connector must be validated before fetch operations."); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         if (_catalog is null) | ||||||
|  |         { | ||||||
|  |             _catalog = await _catalogLoader.LoadAsync(_options, cancellationToken).ConfigureAwait(false); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         LogConnectorEvent(LogLevel.Debug, "fetch", "Ubuntu CSAF discovery ready; channel catalogs handled in subsequent task.", new Dictionary<string, object?> | ||||||
|  |         { | ||||||
|  |             ["since"] = context.Since?.ToString("O"), | ||||||
|  |         }); | ||||||
|  |  | ||||||
|  |         yield break; | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     public override ValueTask<VexClaimBatch> NormalizeAsync(VexRawDocument document, CancellationToken cancellationToken) | ||||||
|  |         => throw new NotSupportedException("UbuntuCsafConnector relies on CSAF normalizers for document processing."); | ||||||
|  |  | ||||||
|  |     public UbuntuCatalogResult? GetCachedCatalog() => _catalog; | ||||||
|  | } | ||||||
| @@ -88,10 +88,46 @@ public sealed class ExportEngineTests | |||||||
|         Assert.Equal(1, recorder2.SaveCount); |         Assert.Equal(1, recorder2.SaveCount); | ||||||
|     } |     } | ||||||
|  |  | ||||||
|  |     [Fact] | ||||||
|  |     public async Task ExportAsync_AttachesAttestationMetadata() | ||||||
|  |     { | ||||||
|  |         var store = new InMemoryExportStore(); | ||||||
|  |         var evaluator = new StaticPolicyEvaluator("baseline/v1"); | ||||||
|  |         var dataSource = new InMemoryExportDataSource(); | ||||||
|  |         var exporter = new DummyExporter(VexExportFormat.Json); | ||||||
|  |         var attestation = new RecordingAttestationClient(); | ||||||
|  |         var engine = new VexExportEngine( | ||||||
|  |             store, | ||||||
|  |             evaluator, | ||||||
|  |             dataSource, | ||||||
|  |             new[] { exporter }, | ||||||
|  |             NullLogger<VexExportEngine>.Instance, | ||||||
|  |             cacheIndex: null, | ||||||
|  |             artifactStores: null, | ||||||
|  |             attestationClient: attestation); | ||||||
|  |  | ||||||
|  |         var query = VexQuery.Create(new[] { new VexQueryFilter("vulnId", "CVE-2025-0001") }); | ||||||
|  |         var requestedAt = DateTimeOffset.UtcNow; | ||||||
|  |         var context = new VexExportRequestContext(query, VexExportFormat.Json, requestedAt); | ||||||
|  |  | ||||||
|  |         var manifest = await engine.ExportAsync(context, CancellationToken.None); | ||||||
|  |  | ||||||
|  |         Assert.NotNull(attestation.LastRequest); | ||||||
|  |         Assert.Equal(manifest.ExportId, attestation.LastRequest!.ExportId); | ||||||
|  |         Assert.NotNull(manifest.Attestation); | ||||||
|  |         Assert.Equal(attestation.Response.Attestation.EnvelopeDigest, manifest.Attestation!.EnvelopeDigest); | ||||||
|  |         Assert.Equal(attestation.Response.Attestation.PredicateType, manifest.Attestation.PredicateType); | ||||||
|  |  | ||||||
|  |         Assert.NotNull(store.LastSavedManifest); | ||||||
|  |         Assert.Equal(manifest.Attestation, store.LastSavedManifest!.Attestation); | ||||||
|  |     } | ||||||
|  |  | ||||||
|     private sealed class InMemoryExportStore : IVexExportStore |     private sealed class InMemoryExportStore : IVexExportStore | ||||||
|     { |     { | ||||||
|         private readonly Dictionary<string, VexExportManifest> _store = new(StringComparer.Ordinal); |         private readonly Dictionary<string, VexExportManifest> _store = new(StringComparer.Ordinal); | ||||||
|  |  | ||||||
|  |         public VexExportManifest? LastSavedManifest { get; private set; } | ||||||
|  |  | ||||||
|         public ValueTask<VexExportManifest?> FindAsync(VexQuerySignature signature, VexExportFormat format, CancellationToken cancellationToken) |         public ValueTask<VexExportManifest?> FindAsync(VexQuerySignature signature, VexExportFormat format, CancellationToken cancellationToken) | ||||||
|         { |         { | ||||||
|             var key = CreateKey(signature.Value, format); |             var key = CreateKey(signature.Value, format); | ||||||
| @@ -103,6 +139,7 @@ public sealed class ExportEngineTests | |||||||
|         { |         { | ||||||
|             var key = CreateKey(manifest.QuerySignature.Value, manifest.Format); |             var key = CreateKey(manifest.QuerySignature.Value, manifest.Format); | ||||||
|             _store[key] = manifest; |             _store[key] = manifest; | ||||||
|  |             LastSavedManifest = manifest; | ||||||
|             return ValueTask.CompletedTask; |             return ValueTask.CompletedTask; | ||||||
|         } |         } | ||||||
|  |  | ||||||
| @@ -110,6 +147,28 @@ public sealed class ExportEngineTests | |||||||
|             => FormattableString.Invariant($"{signature}|{format}"); |             => FormattableString.Invariant($"{signature}|{format}"); | ||||||
|     } |     } | ||||||
|  |  | ||||||
|  |     private sealed class RecordingAttestationClient : IVexAttestationClient | ||||||
|  |     { | ||||||
|  |         public VexAttestationRequest? LastRequest { get; private set; } | ||||||
|  |  | ||||||
|  |         public VexAttestationResponse Response { get; } = new VexAttestationResponse( | ||||||
|  |             new VexAttestationMetadata( | ||||||
|  |                 predicateType: "https://stella-ops.org/attestations/vex-export", | ||||||
|  |                 rekor: new VexRekorReference("0.2", "rekor://entry", "123"), | ||||||
|  |                 envelopeDigest: "sha256:envelope", | ||||||
|  |                 signedAt: DateTimeOffset.UnixEpoch), | ||||||
|  |             ImmutableDictionary<string, string>.Empty); | ||||||
|  |  | ||||||
|  |         public ValueTask<VexAttestationResponse> SignAsync(VexAttestationRequest request, CancellationToken cancellationToken) | ||||||
|  |         { | ||||||
|  |             LastRequest = request; | ||||||
|  |             return ValueTask.FromResult(Response); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         public ValueTask<VexAttestationVerification> VerifyAsync(VexAttestationRequest request, CancellationToken cancellationToken) | ||||||
|  |             => ValueTask.FromResult(new VexAttestationVerification(true, ImmutableDictionary<string, string>.Empty)); | ||||||
|  |     } | ||||||
|  |  | ||||||
|     private sealed class RecordingCacheIndex : IVexCacheIndex |     private sealed class RecordingCacheIndex : IVexCacheIndex | ||||||
|     { |     { | ||||||
|         public Dictionary<(string Signature, VexExportFormat Format), bool> RemoveCalls { get; } = new(); |         public Dictionary<(string Signature, VexExportFormat Format), bool> RemoveCalls { get; } = new(); | ||||||
| @@ -213,4 +272,5 @@ public sealed class ExportEngineTests | |||||||
|             return ValueTask.FromResult(new VexExportResult(new VexContentAddress("sha256", "deadbeef"), bytes.Length, ImmutableDictionary<string, string>.Empty)); |             return ValueTask.FromResult(new VexExportResult(new VexContentAddress("sha256", "deadbeef"), bytes.Length, ImmutableDictionary<string, string>.Empty)); | ||||||
|         } |         } | ||||||
|     } |     } | ||||||
|  |  | ||||||
| } | } | ||||||
|   | |||||||
| @@ -39,6 +39,7 @@ public sealed class VexExportEngine : IExportEngine | |||||||
|     private readonly ILogger<VexExportEngine> _logger; |     private readonly ILogger<VexExportEngine> _logger; | ||||||
|     private readonly IVexCacheIndex? _cacheIndex; |     private readonly IVexCacheIndex? _cacheIndex; | ||||||
|     private readonly IReadOnlyList<IVexArtifactStore> _artifactStores; |     private readonly IReadOnlyList<IVexArtifactStore> _artifactStores; | ||||||
|  |     private readonly IVexAttestationClient? _attestationClient; | ||||||
|  |  | ||||||
|     public VexExportEngine( |     public VexExportEngine( | ||||||
|         IVexExportStore exportStore, |         IVexExportStore exportStore, | ||||||
| @@ -47,7 +48,8 @@ public sealed class VexExportEngine : IExportEngine | |||||||
|         IEnumerable<IVexExporter> exporters, |         IEnumerable<IVexExporter> exporters, | ||||||
|         ILogger<VexExportEngine> logger, |         ILogger<VexExportEngine> logger, | ||||||
|         IVexCacheIndex? cacheIndex = null, |         IVexCacheIndex? cacheIndex = null, | ||||||
|         IEnumerable<IVexArtifactStore>? artifactStores = null) |         IEnumerable<IVexArtifactStore>? artifactStores = null, | ||||||
|  |         IVexAttestationClient? attestationClient = null) | ||||||
|     { |     { | ||||||
|         _exportStore = exportStore ?? throw new ArgumentNullException(nameof(exportStore)); |         _exportStore = exportStore ?? throw new ArgumentNullException(nameof(exportStore)); | ||||||
|         _policyEvaluator = policyEvaluator ?? throw new ArgumentNullException(nameof(policyEvaluator)); |         _policyEvaluator = policyEvaluator ?? throw new ArgumentNullException(nameof(policyEvaluator)); | ||||||
| @@ -55,6 +57,7 @@ public sealed class VexExportEngine : IExportEngine | |||||||
|         _logger = logger ?? throw new ArgumentNullException(nameof(logger)); |         _logger = logger ?? throw new ArgumentNullException(nameof(logger)); | ||||||
|         _cacheIndex = cacheIndex; |         _cacheIndex = cacheIndex; | ||||||
|         _artifactStores = artifactStores?.ToArray() ?? Array.Empty<IVexArtifactStore>(); |         _artifactStores = artifactStores?.ToArray() ?? Array.Empty<IVexArtifactStore>(); | ||||||
|  |         _attestationClient = attestationClient; | ||||||
|  |  | ||||||
|         if (exporters is null) |         if (exporters is null) | ||||||
|         { |         { | ||||||
| @@ -105,6 +108,7 @@ public sealed class VexExportEngine : IExportEngine | |||||||
|             context.RequestedAt); |             context.RequestedAt); | ||||||
|  |  | ||||||
|         var digest = exporter.Digest(exportRequest); |         var digest = exporter.Digest(exportRequest); | ||||||
|  |         var exportId = FormattableString.Invariant($"exports/{context.RequestedAt:yyyyMMddTHHmmssfffZ}/{digest.Digest}"); | ||||||
|  |  | ||||||
|         await using var buffer = new MemoryStream(); |         await using var buffer = new MemoryStream(); | ||||||
|         var result = await exporter.SerializeAsync(exportRequest, buffer, cancellationToken).ConfigureAwait(false); |         var result = await exporter.SerializeAsync(exportRequest, buffer, cancellationToken).ConfigureAwait(false); | ||||||
| @@ -134,7 +138,36 @@ public sealed class VexExportEngine : IExportEngine | |||||||
|             } |             } | ||||||
|         } |         } | ||||||
|  |  | ||||||
|         var exportId = FormattableString.Invariant($"exports/{context.RequestedAt:yyyyMMddTHHmmssfffZ}/{digest.Digest}"); |         VexAttestationMetadata? attestationMetadata = null; | ||||||
|  |         if (_attestationClient is not null) | ||||||
|  |         { | ||||||
|  |             var attestationRequest = new VexAttestationRequest( | ||||||
|  |                 exportId, | ||||||
|  |                 signature, | ||||||
|  |                 digest, | ||||||
|  |                 context.Format, | ||||||
|  |                 context.RequestedAt, | ||||||
|  |                 dataset.SourceProviders, | ||||||
|  |                 result.Metadata); | ||||||
|  |  | ||||||
|  |             var response = await _attestationClient.SignAsync(attestationRequest, cancellationToken).ConfigureAwait(false); | ||||||
|  |             attestationMetadata = response.Attestation; | ||||||
|  |  | ||||||
|  |             if (!response.Diagnostics.IsEmpty) | ||||||
|  |             { | ||||||
|  |                 foreach (var diagnostic in response.Diagnostics) | ||||||
|  |                 { | ||||||
|  |                     _logger.LogDebug( | ||||||
|  |                         "Attestation diagnostic {Key}={Value} for export {ExportId}", | ||||||
|  |                         diagnostic.Key, | ||||||
|  |                         diagnostic.Value, | ||||||
|  |                         exportId); | ||||||
|  |                 } | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             _logger.LogInformation("Attestation generated for export {ExportId}", exportId); | ||||||
|  |         } | ||||||
|  |  | ||||||
|         var manifest = new VexExportManifest( |         var manifest = new VexExportManifest( | ||||||
|             exportId, |             exportId, | ||||||
|             signature, |             signature, | ||||||
| @@ -145,7 +178,7 @@ public sealed class VexExportEngine : IExportEngine | |||||||
|             dataset.SourceProviders, |             dataset.SourceProviders, | ||||||
|             fromCache: false, |             fromCache: false, | ||||||
|             consensusRevision: _policyEvaluator.Version, |             consensusRevision: _policyEvaluator.Version, | ||||||
|             attestation: null, |             attestation: attestationMetadata, | ||||||
|             sizeBytes: result.BytesWritten); |             sizeBytes: result.BytesWritten); | ||||||
|  |  | ||||||
|         await _exportStore.SaveAsync(manifest, cancellationToken).ConfigureAwait(false); |         await _exportStore.SaveAsync(manifest, cancellationToken).ConfigureAwait(false); | ||||||
|   | |||||||
| @@ -5,5 +5,5 @@ If you are working on this file you need to read docs/ARCHITECTURE_VEXER.md and | |||||||
| |VEXER-EXPORT-01-001 – Export engine orchestration|Team Vexer Export|VEXER-CORE-01-003|DONE (2025-10-15) – Export engine scaffolding with cache lookup, data source hooks, and deterministic manifest emission.| | |VEXER-EXPORT-01-001 – Export engine orchestration|Team Vexer Export|VEXER-CORE-01-003|DONE (2025-10-15) – Export engine scaffolding with cache lookup, data source hooks, and deterministic manifest emission.| | ||||||
| |VEXER-EXPORT-01-002 – Cache index & eviction hooks|Team Vexer Export|VEXER-EXPORT-01-001, VEXER-STORAGE-01-003|**DONE (2025-10-16)** – Export engine now invalidates cache entries on force refresh, cache services expose prune/invalidate APIs, and storage maintenance trims expired/dangling records with Mongo2Go coverage.| | |VEXER-EXPORT-01-002 – Cache index & eviction hooks|Team Vexer Export|VEXER-EXPORT-01-001, VEXER-STORAGE-01-003|**DONE (2025-10-16)** – Export engine now invalidates cache entries on force refresh, cache services expose prune/invalidate APIs, and storage maintenance trims expired/dangling records with Mongo2Go coverage.| | ||||||
| |VEXER-EXPORT-01-003 – Artifact store adapters|Team Vexer Export|VEXER-EXPORT-01-001|**DONE (2025-10-16)** – Implemented multi-store pipeline with filesystem, S3-compatible, and offline bundle adapters (hash verification + manifest/zip output) plus unit coverage and DI hooks.| | |VEXER-EXPORT-01-003 – Artifact store adapters|Team Vexer Export|VEXER-EXPORT-01-001|**DONE (2025-10-16)** – Implemented multi-store pipeline with filesystem, S3-compatible, and offline bundle adapters (hash verification + manifest/zip output) plus unit coverage and DI hooks.| | ||||||
| |VEXER-EXPORT-01-004 – Attestation handoff integration|Team Vexer Export|VEXER-EXPORT-01-001, VEXER-ATTEST-01-001|TODO – Connect export engine to attestation client, persist Rekor metadata, and reuse cached attestations.| | |VEXER-EXPORT-01-004 – Attestation handoff integration|Team Vexer Export|VEXER-EXPORT-01-001, VEXER-ATTEST-01-001|**DONE (2025-10-17)** – Export engine now invokes attestation client, logs diagnostics, and persists Rekor/envelope metadata on manifests; regression coverage added in `ExportEngineTests.ExportAsync_AttachesAttestationMetadata`.| | ||||||
| |VEXER-EXPORT-01-005 – Score & resolve envelope surfaces|Team Vexer Export|VEXER-EXPORT-01-004, VEXER-CORE-02-001|TODO – Emit consensus+score envelopes in export manifests, include policy/scoring digests, and update offline bundle/ORAS layouts to carry signed VEX responses.| | |VEXER-EXPORT-01-005 – Score & resolve envelope surfaces|Team Vexer Export|VEXER-EXPORT-01-004, VEXER-CORE-02-001|TODO – Emit consensus+score envelopes in export manifests, include policy/scoring digests, and update offline bundle/ORAS layouts to carry signed VEX responses.| | ||||||
|   | |||||||
							
								
								
									
										131
									
								
								src/StellaOps.Vexer.Formats.CSAF.Tests/CsafNormalizerTests.cs
									
									
									
									
									
										Normal file
									
								
							
							
						
						
									
										131
									
								
								src/StellaOps.Vexer.Formats.CSAF.Tests/CsafNormalizerTests.cs
									
									
									
									
									
										Normal file
									
								
							| @@ -0,0 +1,131 @@ | |||||||
|  | using System.Collections.Immutable; | ||||||
|  | using System.Linq; | ||||||
|  | using System.Text; | ||||||
|  | using System.IO; | ||||||
|  | using FluentAssertions; | ||||||
|  | using Microsoft.Extensions.Logging.Abstractions; | ||||||
|  | using StellaOps.Vexer.Core; | ||||||
|  | using StellaOps.Vexer.Formats.CSAF; | ||||||
|  |  | ||||||
|  | namespace StellaOps.Vexer.Formats.CSAF.Tests; | ||||||
|  |  | ||||||
|  | public sealed class CsafNormalizerTests | ||||||
|  | { | ||||||
|  |     [Fact] | ||||||
|  |     public async Task NormalizeAsync_ProducesClaimsPerProductStatus() | ||||||
|  |     { | ||||||
|  |         var json = """ | ||||||
|  |         { | ||||||
|  |           "document": { | ||||||
|  |             "tracking": { | ||||||
|  |               "id": "RHSA-2025:0001", | ||||||
|  |               "version": "3", | ||||||
|  |               "revision": "3", | ||||||
|  |               "status": "final", | ||||||
|  |               "initial_release_date": "2025-10-01T00:00:00Z", | ||||||
|  |               "current_release_date": "2025-10-10T00:00:00Z" | ||||||
|  |             }, | ||||||
|  |             "publisher": { | ||||||
|  |               "name": "Red Hat Product Security", | ||||||
|  |               "category": "vendor" | ||||||
|  |             } | ||||||
|  |           }, | ||||||
|  |           "product_tree": { | ||||||
|  |             "full_product_names": [ | ||||||
|  |               { | ||||||
|  |                 "product_id": "CSAFPID-0001", | ||||||
|  |                 "name": "Red Hat Enterprise Linux 9", | ||||||
|  |                 "product_identification_helper": { | ||||||
|  |                   "cpe": "cpe:/o:redhat:enterprise_linux:9", | ||||||
|  |                   "purl": "pkg:rpm/redhat/enterprise-linux@9" | ||||||
|  |                 } | ||||||
|  |               } | ||||||
|  |             ] | ||||||
|  |           }, | ||||||
|  |           "vulnerabilities": [ | ||||||
|  |             { | ||||||
|  |               "cve": "CVE-2025-0001", | ||||||
|  |               "title": "Kernel vulnerability", | ||||||
|  |               "product_status": { | ||||||
|  |                 "known_affected": [ "CSAFPID-0001" ] | ||||||
|  |               } | ||||||
|  |             }, | ||||||
|  |             { | ||||||
|  |               "id": "VEX-0002", | ||||||
|  |               "title": "Library issue", | ||||||
|  |               "product_status": { | ||||||
|  |                 "known_not_affected": [ "CSAFPID-0001" ] | ||||||
|  |               } | ||||||
|  |             } | ||||||
|  |           ] | ||||||
|  |         } | ||||||
|  |         """; | ||||||
|  |  | ||||||
|  |         var rawDocument = new VexRawDocument( | ||||||
|  |             ProviderId: "vexer:redhat", | ||||||
|  |             VexDocumentFormat.Csaf, | ||||||
|  |             new Uri("https://example.com/csaf/rhsa-2025-0001.json"), | ||||||
|  |             new DateTimeOffset(2025, 10, 11, 0, 0, 0, TimeSpan.Zero), | ||||||
|  |             "sha256:dummydigest", | ||||||
|  |             Encoding.UTF8.GetBytes(json), | ||||||
|  |             ImmutableDictionary<string, string>.Empty); | ||||||
|  |  | ||||||
|  |         var provider = new VexProvider("vexer:redhat", "Red Hat CSAF", VexProviderKind.Distro); | ||||||
|  |         var normalizer = new CsafNormalizer(NullLogger<CsafNormalizer>.Instance); | ||||||
|  |  | ||||||
|  |         var batch = await normalizer.NormalizeAsync(rawDocument, provider, CancellationToken.None); | ||||||
|  |  | ||||||
|  |         batch.Claims.Should().HaveCount(2); | ||||||
|  |  | ||||||
|  |         var affectedClaim = batch.Claims.First(c => c.VulnerabilityId == "CVE-2025-0001"); | ||||||
|  |         affectedClaim.Status.Should().Be(VexClaimStatus.Affected); | ||||||
|  |         affectedClaim.Product.Key.Should().Be("CSAFPID-0001"); | ||||||
|  |         affectedClaim.Product.Purl.Should().Be("pkg:rpm/redhat/enterprise-linux@9"); | ||||||
|  |         affectedClaim.Product.Cpe.Should().Be("cpe:/o:redhat:enterprise_linux:9"); | ||||||
|  |         affectedClaim.Document.Revision.Should().Be("3"); | ||||||
|  |         affectedClaim.FirstSeen.Should().Be(new DateTimeOffset(2025, 10, 1, 0, 0, 0, TimeSpan.Zero)); | ||||||
|  |         affectedClaim.LastSeen.Should().Be(new DateTimeOffset(2025, 10, 10, 0, 0, 0, TimeSpan.Zero)); | ||||||
|  |         affectedClaim.AdditionalMetadata.Should().ContainKey("csaf.tracking.id"); | ||||||
|  |         affectedClaim.AdditionalMetadata["csaf.publisher.name"].Should().Be("Red Hat Product Security"); | ||||||
|  |  | ||||||
|  |         var notAffectedClaim = batch.Claims.First(c => c.VulnerabilityId == "VEX-0002"); | ||||||
|  |         notAffectedClaim.Status.Should().Be(VexClaimStatus.NotAffected); | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     [Fact] | ||||||
|  |     public async Task NormalizeAsync_PreservesRedHatSpecificMetadata() | ||||||
|  |     { | ||||||
|  |         var path = Path.Combine(AppContext.BaseDirectory, "Fixtures", "rhsa-sample.json"); | ||||||
|  |         var json = await File.ReadAllTextAsync(path); | ||||||
|  |  | ||||||
|  |         var rawDocument = new VexRawDocument( | ||||||
|  |             ProviderId: "vexer:redhat", | ||||||
|  |             VexDocumentFormat.Csaf, | ||||||
|  |             new Uri("https://security.example.com/rhsa-2025-1001.json"), | ||||||
|  |             new DateTimeOffset(2025, 10, 6, 0, 0, 0, TimeSpan.Zero), | ||||||
|  |             "sha256:rhdadigest", | ||||||
|  |             Encoding.UTF8.GetBytes(json), | ||||||
|  |             ImmutableDictionary<string, string>.Empty); | ||||||
|  |  | ||||||
|  |         var provider = new VexProvider("vexer:redhat", "Red Hat CSAF", VexProviderKind.Distro); | ||||||
|  |         var normalizer = new CsafNormalizer(NullLogger<CsafNormalizer>.Instance); | ||||||
|  |  | ||||||
|  |         var batch = await normalizer.NormalizeAsync(rawDocument, provider, CancellationToken.None); | ||||||
|  |         batch.Claims.Should().ContainSingle(); | ||||||
|  |  | ||||||
|  |         var claim = batch.Claims[0]; | ||||||
|  |         claim.VulnerabilityId.Should().Be("CVE-2025-1234"); | ||||||
|  |         claim.Status.Should().Be(VexClaimStatus.Affected); | ||||||
|  |         claim.Product.Key.Should().Be("rh-enterprise-linux-9"); | ||||||
|  |         claim.Product.Name.Should().Be("Red Hat Enterprise Linux 9"); | ||||||
|  |         claim.Product.Purl.Should().Be("pkg:rpm/redhat/enterprise-linux@9"); | ||||||
|  |         claim.Product.Cpe.Should().Be("cpe:/o:redhat:enterprise_linux:9"); | ||||||
|  |         claim.FirstSeen.Should().Be(new DateTimeOffset(2025, 10, 1, 12, 0, 0, TimeSpan.Zero)); | ||||||
|  |         claim.LastSeen.Should().Be(new DateTimeOffset(2025, 10, 5, 10, 0, 0, TimeSpan.Zero)); | ||||||
|  |  | ||||||
|  |         claim.AdditionalMetadata.Should().ContainKey("csaf.tracking.id"); | ||||||
|  |         claim.AdditionalMetadata["csaf.tracking.id"].Should().Be("RHSA-2025:1001"); | ||||||
|  |         claim.AdditionalMetadata["csaf.tracking.status"].Should().Be("final"); | ||||||
|  |         claim.AdditionalMetadata["csaf.publisher.name"].Should().Be("Red Hat Product Security"); | ||||||
|  |     } | ||||||
|  | } | ||||||
| @@ -0,0 +1,47 @@ | |||||||
|  | { | ||||||
|  |   "document": { | ||||||
|  |     "publisher": { | ||||||
|  |       "name": "Red Hat Product Security", | ||||||
|  |       "category": "vendor" | ||||||
|  |     }, | ||||||
|  |     "tracking": { | ||||||
|  |       "id": "RHSA-2025:1001", | ||||||
|  |       "status": "final", | ||||||
|  |       "version": "3", | ||||||
|  |       "initial_release_date": "2025-10-01T12:00:00Z", | ||||||
|  |       "current_release_date": "2025-10-05T10:00:00Z" | ||||||
|  |     } | ||||||
|  |   }, | ||||||
|  |   "product_tree": { | ||||||
|  |     "full_product_names": [ | ||||||
|  |       { | ||||||
|  |         "product_id": "rh-enterprise-linux-9", | ||||||
|  |         "name": "Red Hat Enterprise Linux 9", | ||||||
|  |         "product_identification_helper": { | ||||||
|  |           "cpe": "cpe:/o:redhat:enterprise_linux:9", | ||||||
|  |           "purl": "pkg:rpm/redhat/enterprise-linux@9" | ||||||
|  |         } | ||||||
|  |       } | ||||||
|  |     ], | ||||||
|  |     "branches": [ | ||||||
|  |       { | ||||||
|  |         "name": "Red Hat Enterprise Linux", | ||||||
|  |         "product": { | ||||||
|  |           "product_id": "rh-enterprise-linux-9", | ||||||
|  |           "name": "Red Hat Enterprise Linux 9" | ||||||
|  |         } | ||||||
|  |       } | ||||||
|  |     ] | ||||||
|  |   }, | ||||||
|  |   "vulnerabilities": [ | ||||||
|  |     { | ||||||
|  |       "cve": "CVE-2025-1234", | ||||||
|  |       "title": "Kernel privilege escalation", | ||||||
|  |       "product_status": { | ||||||
|  |         "known_affected": [ | ||||||
|  |           "rh-enterprise-linux-9" | ||||||
|  |         ] | ||||||
|  |       } | ||||||
|  |     } | ||||||
|  |   ] | ||||||
|  | } | ||||||
| @@ -0,0 +1,20 @@ | |||||||
|  | <Project Sdk="Microsoft.NET.Sdk"> | ||||||
|  |   <PropertyGroup> | ||||||
|  |     <TargetFramework>net10.0</TargetFramework> | ||||||
|  |     <Nullable>enable</Nullable> | ||||||
|  |     <ImplicitUsings>enable</ImplicitUsings> | ||||||
|  |   </PropertyGroup> | ||||||
|  |   <ItemGroup> | ||||||
|  |     <PackageReference Include="FluentAssertions" Version="6.12.0" /> | ||||||
|  |   </ItemGroup> | ||||||
|  |   <ItemGroup> | ||||||
|  |     <Using Include="Xunit" /> | ||||||
|  |   </ItemGroup> | ||||||
|  |   <ItemGroup> | ||||||
|  |     <ProjectReference Include="..\StellaOps.Vexer.Core\StellaOps.Vexer.Core.csproj" /> | ||||||
|  |     <ProjectReference Include="..\StellaOps.Vexer.Formats.CSAF\StellaOps.Vexer.Formats.CSAF.csproj" /> | ||||||
|  |   </ItemGroup> | ||||||
|  |   <ItemGroup> | ||||||
|  |     <None Include="Fixtures\**\*" CopyToOutputDirectory="Always" /> | ||||||
|  |   </ItemGroup> | ||||||
|  | </Project> | ||||||
							
								
								
									
										532
									
								
								src/StellaOps.Vexer.Formats.CSAF/CsafNormalizer.cs
									
									
									
									
									
										Normal file
									
								
							
							
						
						
									
										532
									
								
								src/StellaOps.Vexer.Formats.CSAF/CsafNormalizer.cs
									
									
									
									
									
										Normal file
									
								
							| @@ -0,0 +1,532 @@ | |||||||
|  | using System; | ||||||
|  | using System.Collections.Generic; | ||||||
|  | using System.Collections.Immutable; | ||||||
|  | using System.Linq; | ||||||
|  | using System.Text.Json; | ||||||
|  | using System.Threading; | ||||||
|  | using System.Threading.Tasks; | ||||||
|  | using Microsoft.Extensions.Logging; | ||||||
|  | using StellaOps.Vexer.Core; | ||||||
|  |  | ||||||
|  | namespace StellaOps.Vexer.Formats.CSAF; | ||||||
|  |  | ||||||
|  | public sealed class CsafNormalizer : IVexNormalizer | ||||||
|  | { | ||||||
|  |     private static readonly ImmutableDictionary<VexClaimStatus, int> StatusPrecedence = new Dictionary<VexClaimStatus, int> | ||||||
|  |     { | ||||||
|  |         [VexClaimStatus.UnderInvestigation] = 0, | ||||||
|  |         [VexClaimStatus.Affected] = 1, | ||||||
|  |         [VexClaimStatus.NotAffected] = 2, | ||||||
|  |         [VexClaimStatus.Fixed] = 3, | ||||||
|  |     }.ToImmutableDictionary(); | ||||||
|  |  | ||||||
|  |     private readonly ILogger<CsafNormalizer> _logger; | ||||||
|  |  | ||||||
|  |     public CsafNormalizer(ILogger<CsafNormalizer> logger) | ||||||
|  |     { | ||||||
|  |         _logger = logger ?? throw new ArgumentNullException(nameof(logger)); | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     public string Format => VexDocumentFormat.Csaf.ToString().ToLowerInvariant(); | ||||||
|  |  | ||||||
|  |     public bool CanHandle(VexRawDocument document) | ||||||
|  |         => document is not null && document.Format == VexDocumentFormat.Csaf; | ||||||
|  |  | ||||||
|  |     public ValueTask<VexClaimBatch> NormalizeAsync(VexRawDocument document, VexProvider provider, CancellationToken cancellationToken) | ||||||
|  |     { | ||||||
|  |         ArgumentNullException.ThrowIfNull(document); | ||||||
|  |         ArgumentNullException.ThrowIfNull(provider); | ||||||
|  |  | ||||||
|  |         cancellationToken.ThrowIfCancellationRequested(); | ||||||
|  |  | ||||||
|  |         try | ||||||
|  |         { | ||||||
|  |             var result = CsafParser.Parse(document); | ||||||
|  |             var claims = ImmutableArray.CreateBuilder<VexClaim>(result.Claims.Length); | ||||||
|  |             foreach (var entry in result.Claims) | ||||||
|  |             { | ||||||
|  |                 var product = new VexProduct( | ||||||
|  |                     entry.Product.ProductId, | ||||||
|  |                     entry.Product.Name, | ||||||
|  |                     entry.Product.Version, | ||||||
|  |                     entry.Product.Purl, | ||||||
|  |                     entry.Product.Cpe); | ||||||
|  |  | ||||||
|  |                 var claimDocument = new VexClaimDocument( | ||||||
|  |                     VexDocumentFormat.Csaf, | ||||||
|  |                     document.Digest, | ||||||
|  |                     document.SourceUri, | ||||||
|  |                     result.Revision, | ||||||
|  |                     signature: null); | ||||||
|  |  | ||||||
|  |                 var claim = new VexClaim( | ||||||
|  |                     entry.VulnerabilityId, | ||||||
|  |                     provider.Id, | ||||||
|  |                     product, | ||||||
|  |                     entry.Status, | ||||||
|  |                     claimDocument, | ||||||
|  |                     result.FirstRelease, | ||||||
|  |                     result.LastRelease, | ||||||
|  |                     justification: null, | ||||||
|  |                     detail: entry.Detail, | ||||||
|  |                     confidence: null, | ||||||
|  |                     additionalMetadata: result.Metadata); | ||||||
|  |  | ||||||
|  |                 claims.Add(claim); | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             var orderedClaims = claims | ||||||
|  |                 .ToImmutable() | ||||||
|  |                 .OrderBy(static claim => claim.VulnerabilityId, StringComparer.Ordinal) | ||||||
|  |                 .ThenBy(static claim => claim.Product.Key, StringComparer.Ordinal) | ||||||
|  |                 .ToImmutableArray(); | ||||||
|  |  | ||||||
|  |             _logger.LogInformation( | ||||||
|  |                 "Normalized CSAF document {Source} into {ClaimCount} claim(s).", | ||||||
|  |                 document.SourceUri, | ||||||
|  |                 orderedClaims.Length); | ||||||
|  |  | ||||||
|  |             return ValueTask.FromResult(new VexClaimBatch( | ||||||
|  |                 document, | ||||||
|  |                 orderedClaims, | ||||||
|  |                 ImmutableDictionary<string, string>.Empty)); | ||||||
|  |         } | ||||||
|  |         catch (JsonException ex) | ||||||
|  |         { | ||||||
|  |             _logger.LogError(ex, "Failed to parse CSAF document {SourceUri}", document.SourceUri); | ||||||
|  |             throw; | ||||||
|  |         } | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     private static class CsafParser | ||||||
|  |     { | ||||||
|  |         public static CsafParseResult Parse(VexRawDocument document) | ||||||
|  |         { | ||||||
|  |             using var json = JsonDocument.Parse(document.Content.ToArray()); | ||||||
|  |             var root = json.RootElement; | ||||||
|  |  | ||||||
|  |             var tracking = root.TryGetProperty("document", out var documentElement) && | ||||||
|  |                            documentElement.ValueKind == JsonValueKind.Object && | ||||||
|  |                            documentElement.TryGetProperty("tracking", out var trackingElement) | ||||||
|  |                 ? trackingElement | ||||||
|  |                 : default; | ||||||
|  |  | ||||||
|  |             var firstRelease = ParseDate(tracking, "initial_release_date") ?? document.RetrievedAt; | ||||||
|  |             var lastRelease = ParseDate(tracking, "current_release_date") ?? firstRelease; | ||||||
|  |  | ||||||
|  |             if (lastRelease < firstRelease) | ||||||
|  |             { | ||||||
|  |                 lastRelease = firstRelease; | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             var metadataBuilder = ImmutableDictionary.CreateBuilder<string, string>(StringComparer.Ordinal); | ||||||
|  |             AddIfPresent(metadataBuilder, tracking, "id", "csaf.tracking.id"); | ||||||
|  |             AddIfPresent(metadataBuilder, tracking, "version", "csaf.tracking.version"); | ||||||
|  |             AddIfPresent(metadataBuilder, tracking, "status", "csaf.tracking.status"); | ||||||
|  |             AddPublisherMetadata(metadataBuilder, documentElement); | ||||||
|  |  | ||||||
|  |             var revision = TryGetString(tracking, "revision"); | ||||||
|  |  | ||||||
|  |             var productCatalog = CollectProducts(root); | ||||||
|  |             var claimsBuilder = ImmutableArray.CreateBuilder<CsafClaimEntry>(); | ||||||
|  |  | ||||||
|  |             if (root.TryGetProperty("vulnerabilities", out var vulnerabilitiesElement) && | ||||||
|  |                 vulnerabilitiesElement.ValueKind == JsonValueKind.Array) | ||||||
|  |             { | ||||||
|  |                 foreach (var vulnerability in vulnerabilitiesElement.EnumerateArray()) | ||||||
|  |                 { | ||||||
|  |                     var vulnerabilityId = ResolveVulnerabilityId(vulnerability); | ||||||
|  |                     if (string.IsNullOrWhiteSpace(vulnerabilityId)) | ||||||
|  |                     { | ||||||
|  |                         continue; | ||||||
|  |                     } | ||||||
|  |  | ||||||
|  |                     var detail = ResolveDetail(vulnerability); | ||||||
|  |                     var productClaims = BuildClaimsForVulnerability( | ||||||
|  |                         vulnerabilityId, | ||||||
|  |                         vulnerability, | ||||||
|  |                         productCatalog, | ||||||
|  |                         detail); | ||||||
|  |  | ||||||
|  |                     claimsBuilder.AddRange(productClaims); | ||||||
|  |                 } | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             return new CsafParseResult( | ||||||
|  |                 firstRelease, | ||||||
|  |                 lastRelease, | ||||||
|  |                 revision, | ||||||
|  |                 metadataBuilder.ToImmutable(), | ||||||
|  |                 claimsBuilder.ToImmutable()); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         private static IReadOnlyList<CsafClaimEntry> BuildClaimsForVulnerability( | ||||||
|  |             string vulnerabilityId, | ||||||
|  |             JsonElement vulnerability, | ||||||
|  |             IReadOnlyDictionary<string, CsafProductInfo> productCatalog, | ||||||
|  |             string? detail) | ||||||
|  |         { | ||||||
|  |             if (!vulnerability.TryGetProperty("product_status", out var statusElement) || | ||||||
|  |                 statusElement.ValueKind != JsonValueKind.Object) | ||||||
|  |             { | ||||||
|  |                 return Array.Empty<CsafClaimEntry>(); | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             var claims = new Dictionary<string, CsafClaimEntryBuilder>(StringComparer.OrdinalIgnoreCase); | ||||||
|  |  | ||||||
|  |             foreach (var statusProperty in statusElement.EnumerateObject()) | ||||||
|  |             { | ||||||
|  |                 var status = MapStatus(statusProperty.Name); | ||||||
|  |                 if (status is null) | ||||||
|  |                 { | ||||||
|  |                     continue; | ||||||
|  |                 } | ||||||
|  |  | ||||||
|  |                 if (statusProperty.Value.ValueKind != JsonValueKind.Array) | ||||||
|  |                 { | ||||||
|  |                     continue; | ||||||
|  |                 } | ||||||
|  |  | ||||||
|  |                 foreach (var productIdElement in statusProperty.Value.EnumerateArray()) | ||||||
|  |                 { | ||||||
|  |                     var productId = productIdElement.GetString(); | ||||||
|  |                     if (string.IsNullOrWhiteSpace(productId)) | ||||||
|  |                     { | ||||||
|  |                         continue; | ||||||
|  |                     } | ||||||
|  |  | ||||||
|  |                     var product = ResolveProduct(productCatalog, productId); | ||||||
|  |                     UpdateClaim(claims, product, status.Value, detail); | ||||||
|  |                 } | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             if (claims.Count == 0) | ||||||
|  |             { | ||||||
|  |                 return Array.Empty<CsafClaimEntry>(); | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             return claims.Values | ||||||
|  |                 .Select(builder => new CsafClaimEntry( | ||||||
|  |                     vulnerabilityId, | ||||||
|  |                     builder.Product, | ||||||
|  |                     builder.Status, | ||||||
|  |                     builder.Detail)) | ||||||
|  |                 .ToArray(); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         private static void UpdateClaim( | ||||||
|  |             IDictionary<string, CsafClaimEntryBuilder> claims, | ||||||
|  |             CsafProductInfo product, | ||||||
|  |             VexClaimStatus status, | ||||||
|  |             string? detail) | ||||||
|  |         { | ||||||
|  |             if (!claims.TryGetValue(product.ProductId, out var existing) || | ||||||
|  |                 StatusPrecedence[status] > StatusPrecedence[existing.Status]) | ||||||
|  |             { | ||||||
|  |                 claims[product.ProductId] = new CsafClaimEntryBuilder(product, status, detail); | ||||||
|  |             } | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         private static CsafProductInfo ResolveProduct( | ||||||
|  |             IReadOnlyDictionary<string, CsafProductInfo> catalog, | ||||||
|  |             string productId) | ||||||
|  |         { | ||||||
|  |             if (catalog.TryGetValue(productId, out var product)) | ||||||
|  |             { | ||||||
|  |                 return product; | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             return new CsafProductInfo(productId, productId, null, null, null); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         private static string ResolveVulnerabilityId(JsonElement vulnerability) | ||||||
|  |         { | ||||||
|  |             var id = TryGetString(vulnerability, "cve") | ||||||
|  |                 ?? TryGetString(vulnerability, "id") | ||||||
|  |                 ?? TryGetString(vulnerability, "vuln_id"); | ||||||
|  |  | ||||||
|  |             return string.IsNullOrWhiteSpace(id) ? string.Empty : id.Trim(); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         private static string? ResolveDetail(JsonElement vulnerability) | ||||||
|  |         { | ||||||
|  |             var title = TryGetString(vulnerability, "title"); | ||||||
|  |             if (!string.IsNullOrWhiteSpace(title)) | ||||||
|  |             { | ||||||
|  |                 return title.Trim(); | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             if (vulnerability.TryGetProperty("notes", out var notesElement) && | ||||||
|  |                 notesElement.ValueKind == JsonValueKind.Array) | ||||||
|  |             { | ||||||
|  |                 foreach (var note in notesElement.EnumerateArray()) | ||||||
|  |                 { | ||||||
|  |                     if (note.ValueKind != JsonValueKind.Object) | ||||||
|  |                     { | ||||||
|  |                         continue; | ||||||
|  |                     } | ||||||
|  |  | ||||||
|  |                     var category = TryGetString(note, "category"); | ||||||
|  |                     if (!string.IsNullOrWhiteSpace(category) && | ||||||
|  |                         !string.Equals(category, "description", StringComparison.OrdinalIgnoreCase)) | ||||||
|  |                     { | ||||||
|  |                         continue; | ||||||
|  |                     } | ||||||
|  |  | ||||||
|  |                     var text = TryGetString(note, "text"); | ||||||
|  |                     if (!string.IsNullOrWhiteSpace(text)) | ||||||
|  |                     { | ||||||
|  |                         return text.Trim(); | ||||||
|  |                     } | ||||||
|  |                 } | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             return null; | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         private static Dictionary<string, CsafProductInfo> CollectProducts(JsonElement root) | ||||||
|  |         { | ||||||
|  |             var products = new Dictionary<string, CsafProductInfo>(StringComparer.OrdinalIgnoreCase); | ||||||
|  |  | ||||||
|  |             if (!root.TryGetProperty("product_tree", out var productTree) || | ||||||
|  |                 productTree.ValueKind != JsonValueKind.Object) | ||||||
|  |             { | ||||||
|  |                 return products; | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             if (productTree.TryGetProperty("full_product_names", out var fullNames) && | ||||||
|  |                 fullNames.ValueKind == JsonValueKind.Array) | ||||||
|  |             { | ||||||
|  |                 foreach (var productEntry in fullNames.EnumerateArray()) | ||||||
|  |                 { | ||||||
|  |                     var product = ParseProduct(productEntry, parentBranchName: null); | ||||||
|  |                     if (product is not null) | ||||||
|  |                     { | ||||||
|  |                         AddOrUpdate(product); | ||||||
|  |                     } | ||||||
|  |                 } | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             if (productTree.TryGetProperty("branches", out var branches) && | ||||||
|  |                 branches.ValueKind == JsonValueKind.Array) | ||||||
|  |             { | ||||||
|  |                 foreach (var branch in branches.EnumerateArray()) | ||||||
|  |                 { | ||||||
|  |                     VisitBranch(branch, parentBranchName: null); | ||||||
|  |                 } | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             return products; | ||||||
|  |  | ||||||
|  |             void VisitBranch(JsonElement branch, string? parentBranchName) | ||||||
|  |             { | ||||||
|  |                 if (branch.ValueKind != JsonValueKind.Object) | ||||||
|  |                 { | ||||||
|  |                     return; | ||||||
|  |                 } | ||||||
|  |  | ||||||
|  |                 var branchName = TryGetString(branch, "name") ?? parentBranchName; | ||||||
|  |  | ||||||
|  |                 if (branch.TryGetProperty("product", out var productElement)) | ||||||
|  |                 { | ||||||
|  |                     var product = ParseProduct(productElement, branchName); | ||||||
|  |                     if (product is not null) | ||||||
|  |                     { | ||||||
|  |                         AddOrUpdate(product); | ||||||
|  |                     } | ||||||
|  |                 } | ||||||
|  |  | ||||||
|  |                 if (branch.TryGetProperty("branches", out var childBranches) && | ||||||
|  |                     childBranches.ValueKind == JsonValueKind.Array) | ||||||
|  |                 { | ||||||
|  |                     foreach (var childBranch in childBranches.EnumerateArray()) | ||||||
|  |                     { | ||||||
|  |                         VisitBranch(childBranch, branchName); | ||||||
|  |                     } | ||||||
|  |                 } | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             void AddOrUpdate(CsafProductInfo product) | ||||||
|  |             { | ||||||
|  |                 if (products.TryGetValue(product.ProductId, out var existing)) | ||||||
|  |                 { | ||||||
|  |                     products[product.ProductId] = MergeProducts(existing, product); | ||||||
|  |                 } | ||||||
|  |                 else | ||||||
|  |                 { | ||||||
|  |                     products[product.ProductId] = product; | ||||||
|  |                 } | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             static CsafProductInfo MergeProducts(CsafProductInfo existing, CsafProductInfo incoming) | ||||||
|  |             { | ||||||
|  |                 static string ChooseName(string incoming, string fallback) | ||||||
|  |                     => string.IsNullOrWhiteSpace(incoming) ? fallback : incoming; | ||||||
|  |  | ||||||
|  |                 static string? ChooseOptional(string? incoming, string? fallback) | ||||||
|  |                     => string.IsNullOrWhiteSpace(incoming) ? fallback : incoming; | ||||||
|  |  | ||||||
|  |                 return new CsafProductInfo( | ||||||
|  |                     existing.ProductId, | ||||||
|  |                     ChooseName(incoming.Name, existing.Name), | ||||||
|  |                     ChooseOptional(incoming.Version, existing.Version), | ||||||
|  |                     ChooseOptional(incoming.Purl, existing.Purl), | ||||||
|  |                     ChooseOptional(incoming.Cpe, existing.Cpe)); | ||||||
|  |             } | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         private static CsafProductInfo? ParseProduct(JsonElement element, string? parentBranchName) | ||||||
|  |         { | ||||||
|  |             if (element.ValueKind != JsonValueKind.Object) | ||||||
|  |             { | ||||||
|  |                 return null; | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             JsonElement productElement = element; | ||||||
|  |             if (!element.TryGetProperty("product_id", out var idElement) && | ||||||
|  |                 element.TryGetProperty("product", out var nestedProduct) && | ||||||
|  |                 nestedProduct.ValueKind == JsonValueKind.Object && | ||||||
|  |                 nestedProduct.TryGetProperty("product_id", out idElement)) | ||||||
|  |             { | ||||||
|  |                 productElement = nestedProduct; | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             var productId = idElement.GetString(); | ||||||
|  |             if (string.IsNullOrWhiteSpace(productId)) | ||||||
|  |             { | ||||||
|  |                 return null; | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             var name = TryGetString(productElement, "name") | ||||||
|  |                 ?? TryGetString(element, "name") | ||||||
|  |                 ?? parentBranchName | ||||||
|  |                 ?? productId; | ||||||
|  |  | ||||||
|  |             var version = TryGetString(productElement, "product_version") | ||||||
|  |                 ?? TryGetString(productElement, "version") | ||||||
|  |                 ?? TryGetString(element, "product_version"); | ||||||
|  |  | ||||||
|  |             string? cpe = null; | ||||||
|  |             string? purl = null; | ||||||
|  |             if (productElement.TryGetProperty("product_identification_helper", out var helper) && | ||||||
|  |                 helper.ValueKind == JsonValueKind.Object) | ||||||
|  |             { | ||||||
|  |                 cpe = TryGetString(helper, "cpe"); | ||||||
|  |                 purl = TryGetString(helper, "purl"); | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             return new CsafProductInfo(productId.Trim(), name.Trim(), version?.Trim(), purl?.Trim(), cpe?.Trim()); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         private static VexClaimStatus? MapStatus(string statusName) | ||||||
|  |         { | ||||||
|  |             if (string.IsNullOrWhiteSpace(statusName)) | ||||||
|  |             { | ||||||
|  |                 return null; | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             return statusName switch | ||||||
|  |             { | ||||||
|  |                 "known_affected" or "fixed_after_release" or "first_affected" or "last_affected" => VexClaimStatus.Affected, | ||||||
|  |                 "known_not_affected" or "last_not_affected" or "first_not_affected" => VexClaimStatus.NotAffected, | ||||||
|  |                 "fixed" or "first_fixed" or "last_fixed" => VexClaimStatus.Fixed, | ||||||
|  |                 "under_investigation" or "investigating" => VexClaimStatus.UnderInvestigation, | ||||||
|  |                 _ => null, | ||||||
|  |             }; | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         private static DateTimeOffset? ParseDate(JsonElement element, string propertyName) | ||||||
|  |         { | ||||||
|  |             if (element.ValueKind != JsonValueKind.Object) | ||||||
|  |             { | ||||||
|  |                 return null; | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             if (!element.TryGetProperty(propertyName, out var dateElement)) | ||||||
|  |             { | ||||||
|  |                 return null; | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             var value = dateElement.GetString(); | ||||||
|  |             if (string.IsNullOrWhiteSpace(value)) | ||||||
|  |             { | ||||||
|  |                 return null; | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             if (DateTimeOffset.TryParse(value, out var parsed)) | ||||||
|  |             { | ||||||
|  |                 return parsed; | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             return null; | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         private static string? TryGetString(JsonElement element, string propertyName) | ||||||
|  |         { | ||||||
|  |             if (element.ValueKind != JsonValueKind.Object) | ||||||
|  |             { | ||||||
|  |                 return null; | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             if (!element.TryGetProperty(propertyName, out var property)) | ||||||
|  |             { | ||||||
|  |                 return null; | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             return property.ValueKind == JsonValueKind.String ? property.GetString() : null; | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         private static void AddIfPresent( | ||||||
|  |             ImmutableDictionary<string, string>.Builder builder, | ||||||
|  |             JsonElement element, | ||||||
|  |             string propertyName, | ||||||
|  |             string metadataKey) | ||||||
|  |         { | ||||||
|  |             var value = TryGetString(element, propertyName); | ||||||
|  |             if (!string.IsNullOrWhiteSpace(value)) | ||||||
|  |             { | ||||||
|  |                 builder[metadataKey] = value.Trim(); | ||||||
|  |             } | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         private static void AddPublisherMetadata( | ||||||
|  |             ImmutableDictionary<string, string>.Builder builder, | ||||||
|  |             JsonElement documentElement) | ||||||
|  |         { | ||||||
|  |             if (documentElement.ValueKind != JsonValueKind.Object || | ||||||
|  |                 !documentElement.TryGetProperty("publisher", out var publisher) || | ||||||
|  |                 publisher.ValueKind != JsonValueKind.Object) | ||||||
|  |             { | ||||||
|  |                 return; | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             AddIfPresent(builder, publisher, "name", "csaf.publisher.name"); | ||||||
|  |             AddIfPresent(builder, publisher, "category", "csaf.publisher.category"); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         private readonly record struct CsafClaimEntryBuilder( | ||||||
|  |             CsafProductInfo Product, | ||||||
|  |             VexClaimStatus Status, | ||||||
|  |             string? Detail); | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     private sealed record CsafParseResult( | ||||||
|  |         DateTimeOffset FirstRelease, | ||||||
|  |         DateTimeOffset LastRelease, | ||||||
|  |         string? Revision, | ||||||
|  |         ImmutableDictionary<string, string> Metadata, | ||||||
|  |         ImmutableArray<CsafClaimEntry> Claims); | ||||||
|  |  | ||||||
|  |     private sealed record CsafClaimEntry( | ||||||
|  |         string VulnerabilityId, | ||||||
|  |         CsafProductInfo Product, | ||||||
|  |         VexClaimStatus Status, | ||||||
|  |         string? Detail); | ||||||
|  |  | ||||||
|  |     private sealed record CsafProductInfo( | ||||||
|  |         string ProductId, | ||||||
|  |         string Name, | ||||||
|  |         string? Version, | ||||||
|  |         string? Purl, | ||||||
|  |         string? Cpe); | ||||||
|  | } | ||||||
| @@ -0,0 +1,14 @@ | |||||||
|  | using Microsoft.Extensions.DependencyInjection; | ||||||
|  | using StellaOps.Vexer.Core; | ||||||
|  |  | ||||||
|  | namespace StellaOps.Vexer.Formats.CSAF; | ||||||
|  |  | ||||||
|  | public static class CsafFormatsServiceCollectionExtensions | ||||||
|  | { | ||||||
|  |     public static IServiceCollection AddCsafNormalizer(this IServiceCollection services) | ||||||
|  |     { | ||||||
|  |         ArgumentNullException.ThrowIfNull(services); | ||||||
|  |         services.AddSingleton<IVexNormalizer, CsafNormalizer>(); | ||||||
|  |         return services; | ||||||
|  |     } | ||||||
|  | } | ||||||
| @@ -0,0 +1,16 @@ | |||||||
|  | <Project Sdk="Microsoft.NET.Sdk"> | ||||||
|  |   <PropertyGroup> | ||||||
|  |     <TargetFramework>net10.0</TargetFramework> | ||||||
|  |     <LangVersion>preview</LangVersion> | ||||||
|  |     <Nullable>enable</Nullable> | ||||||
|  |     <ImplicitUsings>enable</ImplicitUsings> | ||||||
|  |     <TreatWarningsAsErrors>true</TreatWarningsAsErrors> | ||||||
|  |   </PropertyGroup> | ||||||
|  |   <ItemGroup> | ||||||
|  |     <PackageReference Include="Microsoft.Extensions.DependencyInjection.Abstractions" Version="8.0.0" /> | ||||||
|  |     <PackageReference Include="Microsoft.Extensions.Logging.Abstractions" Version="8.0.0" /> | ||||||
|  |   </ItemGroup> | ||||||
|  |   <ItemGroup> | ||||||
|  |     <ProjectReference Include="..\StellaOps.Vexer.Core\StellaOps.Vexer.Core.csproj" /> | ||||||
|  |   </ItemGroup> | ||||||
|  | </Project> | ||||||
| @@ -2,6 +2,6 @@ If you are working on this file you need to read docs/ARCHITECTURE_VEXER.md and | |||||||
| # TASKS | # TASKS | ||||||
| | Task | Owner(s) | Depends on | Notes | | | Task | Owner(s) | Depends on | Notes | | ||||||
| |---|---|---|---| | |---|---|---|---| | ||||||
| |VEXER-FMT-CSAF-01-001 – CSAF normalizer foundation|Team Vexer Formats|VEXER-CORE-01-001|TODO – Implement CSAF parser covering provider metadata, document tracking, and vulnerability/product mapping into `VexClaim`.| | |VEXER-FMT-CSAF-01-001 – CSAF normalizer foundation|Team Vexer Formats|VEXER-CORE-01-001|**DONE (2025-10-17)** – Implemented CSAF normalizer + DI hook, parsing tracking metadata, product tree branches/full names, and mapping product statuses into canonical `VexClaim`s with baseline precedence. Regression added in `CsafNormalizerTests`.| | ||||||
| |VEXER-FMT-CSAF-01-002 – Status/justification mapping|Team Vexer Formats|VEXER-FMT-CSAF-01-001, VEXER-POLICY-01-001|TODO – Normalize CSAF `product_status` + `justification` values into policy-aware enums with audit diagnostics for unsupported codes.| | |VEXER-FMT-CSAF-01-002 – Status/justification mapping|Team Vexer Formats|VEXER-FMT-CSAF-01-001, VEXER-POLICY-01-001|TODO – Normalize CSAF `product_status` + `justification` values into policy-aware enums with audit diagnostics for unsupported codes.| | ||||||
| |VEXER-FMT-CSAF-01-003 – CSAF export adapter|Team Vexer Formats|VEXER-EXPORT-01-001, VEXER-FMT-CSAF-01-001|TODO – Provide CSAF export writer producing deterministic documents (per vuln/product) and manifest metadata for attestation.| | |VEXER-FMT-CSAF-01-003 – CSAF export adapter|Team Vexer Formats|VEXER-EXPORT-01-001, VEXER-FMT-CSAF-01-001|TODO – Provide CSAF export writer producing deterministic documents (per vuln/product) and manifest metadata for attestation.| | ||||||
|   | |||||||
| @@ -0,0 +1,93 @@ | |||||||
|  | using System.Collections.Immutable; | ||||||
|  | using System.Linq; | ||||||
|  | using System.Text; | ||||||
|  | using FluentAssertions; | ||||||
|  | using Microsoft.Extensions.Logging.Abstractions; | ||||||
|  | using StellaOps.Vexer.Core; | ||||||
|  | using StellaOps.Vexer.Formats.CycloneDX; | ||||||
|  |  | ||||||
|  | namespace StellaOps.Vexer.Formats.CycloneDX.Tests; | ||||||
|  |  | ||||||
|  | public sealed class CycloneDxNormalizerTests | ||||||
|  | { | ||||||
|  |     [Fact] | ||||||
|  |     public async Task NormalizeAsync_MapsAnalysisStateAndJustification() | ||||||
|  |     { | ||||||
|  |         var json = """ | ||||||
|  |         { | ||||||
|  |           "bomFormat": "CycloneDX", | ||||||
|  |           "specVersion": "1.4", | ||||||
|  |           "serialNumber": "urn:uuid:1234", | ||||||
|  |           "version": "7", | ||||||
|  |           "metadata": { | ||||||
|  |             "timestamp": "2025-10-15T12:00:00Z" | ||||||
|  |           }, | ||||||
|  |           "components": [ | ||||||
|  |             { | ||||||
|  |               "bom-ref": "pkg:npm/acme/lib@1.0.0", | ||||||
|  |               "name": "acme-lib", | ||||||
|  |               "version": "1.0.0", | ||||||
|  |               "purl": "pkg:npm/acme/lib@1.0.0" | ||||||
|  |             } | ||||||
|  |           ], | ||||||
|  |           "vulnerabilities": [ | ||||||
|  |             { | ||||||
|  |               "id": "CVE-2025-1000", | ||||||
|  |               "detail": "Library issue", | ||||||
|  |               "analysis": { | ||||||
|  |                 "state": "not_affected", | ||||||
|  |                 "justification": "code_not_present", | ||||||
|  |                 "response": [ "can_not_fix", "will_not_fix" ] | ||||||
|  |               }, | ||||||
|  |               "affects": [ | ||||||
|  |                 { "ref": "pkg:npm/acme/lib@1.0.0" } | ||||||
|  |               ] | ||||||
|  |             }, | ||||||
|  |             { | ||||||
|  |               "id": "CVE-2025-1001", | ||||||
|  |               "description": "Investigating impact", | ||||||
|  |               "analysis": { | ||||||
|  |                 "state": "in_triage" | ||||||
|  |               }, | ||||||
|  |               "affects": [ | ||||||
|  |                 { "ref": "pkg:npm/missing/component@2.0.0" } | ||||||
|  |               ] | ||||||
|  |             } | ||||||
|  |           ] | ||||||
|  |         } | ||||||
|  |         """; | ||||||
|  |  | ||||||
|  |         var rawDocument = new VexRawDocument( | ||||||
|  |             "vexer:cyclonedx", | ||||||
|  |             VexDocumentFormat.CycloneDx, | ||||||
|  |             new Uri("https://example.org/vex.json"), | ||||||
|  |             new DateTimeOffset(2025, 10, 16, 0, 0, 0, TimeSpan.Zero), | ||||||
|  |             "sha256:dummydigest", | ||||||
|  |             Encoding.UTF8.GetBytes(json), | ||||||
|  |             ImmutableDictionary<string, string>.Empty); | ||||||
|  |  | ||||||
|  |         var provider = new VexProvider("vexer:cyclonedx", "CycloneDX Provider", VexProviderKind.Vendor); | ||||||
|  |         var normalizer = new CycloneDxNormalizer(NullLogger<CycloneDxNormalizer>.Instance); | ||||||
|  |  | ||||||
|  |         var batch = await normalizer.NormalizeAsync(rawDocument, provider, CancellationToken.None); | ||||||
|  |  | ||||||
|  |         batch.Claims.Should().HaveCount(2); | ||||||
|  |  | ||||||
|  |         var notAffected = batch.Claims.Single(c => c.VulnerabilityId == "CVE-2025-1000"); | ||||||
|  |         notAffected.Status.Should().Be(VexClaimStatus.NotAffected); | ||||||
|  |         notAffected.Justification.Should().Be(VexJustification.CodeNotPresent); | ||||||
|  |         notAffected.Product.Key.Should().Be("pkg:npm/acme/lib@1.0.0"); | ||||||
|  |         notAffected.Product.Purl.Should().Be("pkg:npm/acme/lib@1.0.0"); | ||||||
|  |         notAffected.Document.Revision.Should().Be("7"); | ||||||
|  |         notAffected.AdditionalMetadata["cyclonedx.specVersion"].Should().Be("1.4"); | ||||||
|  |         notAffected.AdditionalMetadata["cyclonedx.analysis.state"].Should().Be("not_affected"); | ||||||
|  |         notAffected.AdditionalMetadata["cyclonedx.analysis.response"].Should().Be("can_not_fix,will_not_fix"); | ||||||
|  |  | ||||||
|  |         var investigating = batch.Claims.Single(c => c.VulnerabilityId == "CVE-2025-1001"); | ||||||
|  |         investigating.Status.Should().Be(VexClaimStatus.UnderInvestigation); | ||||||
|  |         investigating.Justification.Should().BeNull(); | ||||||
|  |         investigating.Product.Key.Should().Be("pkg:npm/missing/component@2.0.0"); | ||||||
|  |         investigating.Product.Name.Should().Be("pkg:npm/missing/component@2.0.0"); | ||||||
|  |         investigating.AdditionalMetadata.Should().ContainKey("cyclonedx.specVersion"); | ||||||
|  |     } | ||||||
|  | } | ||||||
| @@ -0,0 +1,17 @@ | |||||||
|  | <Project Sdk="Microsoft.NET.Sdk"> | ||||||
|  |   <PropertyGroup> | ||||||
|  |     <TargetFramework>net10.0</TargetFramework> | ||||||
|  |     <Nullable>enable</Nullable> | ||||||
|  |     <ImplicitUsings>enable</ImplicitUsings> | ||||||
|  |   </PropertyGroup> | ||||||
|  |   <ItemGroup> | ||||||
|  |     <PackageReference Include="FluentAssertions" Version="6.12.0" /> | ||||||
|  |   </ItemGroup> | ||||||
|  |   <ItemGroup> | ||||||
|  |     <Using Include="Xunit" /> | ||||||
|  |   </ItemGroup> | ||||||
|  |   <ItemGroup> | ||||||
|  |     <ProjectReference Include="..\StellaOps.Vexer.Core\StellaOps.Vexer.Core.csproj" /> | ||||||
|  |     <ProjectReference Include="..\StellaOps.Vexer.Formats.CycloneDX\StellaOps.Vexer.Formats.CycloneDX.csproj" /> | ||||||
|  |   </ItemGroup> | ||||||
|  | </Project> | ||||||
							
								
								
									
										459
									
								
								src/StellaOps.Vexer.Formats.CycloneDX/CycloneDxNormalizer.cs
									
									
									
									
									
										Normal file
									
								
							
							
						
						
									
										459
									
								
								src/StellaOps.Vexer.Formats.CycloneDX/CycloneDxNormalizer.cs
									
									
									
									
									
										Normal file
									
								
							| @@ -0,0 +1,459 @@ | |||||||
|  | using System; | ||||||
|  | using System.Collections.Generic; | ||||||
|  | using System.Collections.Immutable; | ||||||
|  | using System.Linq; | ||||||
|  | using System.Text.Json; | ||||||
|  | using System.Threading; | ||||||
|  | using System.Threading.Tasks; | ||||||
|  | using Microsoft.Extensions.Logging; | ||||||
|  | using StellaOps.Vexer.Core; | ||||||
|  |  | ||||||
|  | namespace StellaOps.Vexer.Formats.CycloneDX; | ||||||
|  |  | ||||||
|  | public sealed class CycloneDxNormalizer : IVexNormalizer | ||||||
|  | { | ||||||
|  |     private static readonly ImmutableDictionary<string, VexClaimStatus> StateMap = new Dictionary<string, VexClaimStatus>(StringComparer.OrdinalIgnoreCase) | ||||||
|  |     { | ||||||
|  |         ["not_affected"] = VexClaimStatus.NotAffected, | ||||||
|  |         ["resolved"] = VexClaimStatus.Fixed, | ||||||
|  |         ["resolved_with_patches"] = VexClaimStatus.Fixed, | ||||||
|  |         ["resolved_no_fix"] = VexClaimStatus.Fixed, | ||||||
|  |         ["fixed"] = VexClaimStatus.Fixed, | ||||||
|  |         ["affected"] = VexClaimStatus.Affected, | ||||||
|  |         ["known_affected"] = VexClaimStatus.Affected, | ||||||
|  |         ["exploitable"] = VexClaimStatus.Affected, | ||||||
|  |         ["in_triage"] = VexClaimStatus.UnderInvestigation, | ||||||
|  |         ["under_investigation"] = VexClaimStatus.UnderInvestigation, | ||||||
|  |         ["unknown"] = VexClaimStatus.UnderInvestigation, | ||||||
|  |     }.ToImmutableDictionary(StringComparer.OrdinalIgnoreCase); | ||||||
|  |  | ||||||
|  |     private static readonly ImmutableDictionary<string, VexJustification> JustificationMap = new Dictionary<string, VexJustification>(StringComparer.OrdinalIgnoreCase) | ||||||
|  |     { | ||||||
|  |         ["code_not_present"] = VexJustification.CodeNotPresent, | ||||||
|  |         ["code_not_reachable"] = VexJustification.CodeNotReachable, | ||||||
|  |         ["component_not_present"] = VexJustification.ComponentNotPresent, | ||||||
|  |         ["component_not_configured"] = VexJustification.ComponentNotConfigured, | ||||||
|  |         ["vulnerable_code_not_present"] = VexJustification.VulnerableCodeNotPresent, | ||||||
|  |         ["vulnerable_code_not_in_execute_path"] = VexJustification.VulnerableCodeNotInExecutePath, | ||||||
|  |         ["vulnerable_code_cannot_be_controlled_by_adversary"] = VexJustification.VulnerableCodeCannotBeControlledByAdversary, | ||||||
|  |         ["inline_mitigations_already_exist"] = VexJustification.InlineMitigationsAlreadyExist, | ||||||
|  |         ["protected_by_mitigating_control"] = VexJustification.ProtectedByMitigatingControl, | ||||||
|  |         ["protected_by_compensating_control"] = VexJustification.ProtectedByCompensatingControl, | ||||||
|  |         ["requires_configuration"] = VexJustification.RequiresConfiguration, | ||||||
|  |         ["requires_dependency"] = VexJustification.RequiresDependency, | ||||||
|  |         ["requires_environment"] = VexJustification.RequiresEnvironment, | ||||||
|  |     }.ToImmutableDictionary(StringComparer.OrdinalIgnoreCase); | ||||||
|  |  | ||||||
|  |     private readonly ILogger<CycloneDxNormalizer> _logger; | ||||||
|  |  | ||||||
|  |     public CycloneDxNormalizer(ILogger<CycloneDxNormalizer> logger) | ||||||
|  |     { | ||||||
|  |         _logger = logger ?? throw new ArgumentNullException(nameof(logger)); | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     public string Format => VexDocumentFormat.CycloneDx.ToString().ToLowerInvariant(); | ||||||
|  |  | ||||||
|  |     public bool CanHandle(VexRawDocument document) | ||||||
|  |         => document is not null && document.Format == VexDocumentFormat.CycloneDx; | ||||||
|  |  | ||||||
|  |     public ValueTask<VexClaimBatch> NormalizeAsync(VexRawDocument document, VexProvider provider, CancellationToken cancellationToken) | ||||||
|  |     { | ||||||
|  |         ArgumentNullException.ThrowIfNull(document); | ||||||
|  |         ArgumentNullException.ThrowIfNull(provider); | ||||||
|  |         cancellationToken.ThrowIfCancellationRequested(); | ||||||
|  |  | ||||||
|  |         try | ||||||
|  |         { | ||||||
|  |             var parseResult = CycloneDxParser.Parse(document); | ||||||
|  |             var baseMetadata = parseResult.Metadata; | ||||||
|  |             var claimsBuilder = ImmutableArray.CreateBuilder<VexClaim>(); | ||||||
|  |  | ||||||
|  |             foreach (var vulnerability in parseResult.Vulnerabilities) | ||||||
|  |             { | ||||||
|  |                 cancellationToken.ThrowIfCancellationRequested(); | ||||||
|  |  | ||||||
|  |                 var state = MapState(vulnerability.AnalysisState, out var stateRaw); | ||||||
|  |                 var justification = MapJustification(vulnerability.AnalysisJustification); | ||||||
|  |                 var responses = vulnerability.AnalysisResponses; | ||||||
|  |  | ||||||
|  |                 foreach (var affect in vulnerability.Affects) | ||||||
|  |                 { | ||||||
|  |                     var productInfo = parseResult.ResolveProduct(affect.ComponentRef); | ||||||
|  |                     var product = new VexProduct( | ||||||
|  |                         productInfo.Key, | ||||||
|  |                         productInfo.Name, | ||||||
|  |                         productInfo.Version, | ||||||
|  |                         productInfo.Purl, | ||||||
|  |                         productInfo.Cpe); | ||||||
|  |  | ||||||
|  |                     var metadata = baseMetadata; | ||||||
|  |                     if (!string.IsNullOrWhiteSpace(stateRaw)) | ||||||
|  |                     { | ||||||
|  |                         metadata = metadata.SetItem("cyclonedx.analysis.state", stateRaw); | ||||||
|  |                     } | ||||||
|  |  | ||||||
|  |                     if (!string.IsNullOrWhiteSpace(vulnerability.AnalysisJustification)) | ||||||
|  |                     { | ||||||
|  |                         metadata = metadata.SetItem("cyclonedx.analysis.justification", vulnerability.AnalysisJustification); | ||||||
|  |                     } | ||||||
|  |  | ||||||
|  |                     if (responses.Length > 0) | ||||||
|  |                     { | ||||||
|  |                         metadata = metadata.SetItem("cyclonedx.analysis.response", string.Join(",", responses)); | ||||||
|  |                     } | ||||||
|  |  | ||||||
|  |                     if (!string.IsNullOrWhiteSpace(affect.ComponentRef)) | ||||||
|  |                     { | ||||||
|  |                         metadata = metadata.SetItem("cyclonedx.affects.ref", affect.ComponentRef); | ||||||
|  |                     } | ||||||
|  |  | ||||||
|  |                     var claimDocument = new VexClaimDocument( | ||||||
|  |                         VexDocumentFormat.CycloneDx, | ||||||
|  |                         document.Digest, | ||||||
|  |                         document.SourceUri, | ||||||
|  |                         parseResult.BomVersion, | ||||||
|  |                         signature: null); | ||||||
|  |  | ||||||
|  |                     var claim = new VexClaim( | ||||||
|  |                         vulnerability.VulnerabilityId, | ||||||
|  |                         provider.Id, | ||||||
|  |                         product, | ||||||
|  |                         state, | ||||||
|  |                         claimDocument, | ||||||
|  |                         parseResult.FirstObserved, | ||||||
|  |                         parseResult.LastObserved, | ||||||
|  |                         justification, | ||||||
|  |                         vulnerability.Detail, | ||||||
|  |                         confidence: null, | ||||||
|  |                         additionalMetadata: metadata); | ||||||
|  |  | ||||||
|  |                     claimsBuilder.Add(claim); | ||||||
|  |                 } | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             var orderedClaims = claimsBuilder | ||||||
|  |                 .ToImmutable() | ||||||
|  |                 .OrderBy(static c => c.VulnerabilityId, StringComparer.Ordinal) | ||||||
|  |                 .ThenBy(static c => c.Product.Key, StringComparer.Ordinal) | ||||||
|  |                 .ToImmutableArray(); | ||||||
|  |  | ||||||
|  |             _logger.LogInformation( | ||||||
|  |                 "Normalized CycloneDX document {Source} into {ClaimCount} claim(s).", | ||||||
|  |                 document.SourceUri, | ||||||
|  |                 orderedClaims.Length); | ||||||
|  |  | ||||||
|  |             return ValueTask.FromResult(new VexClaimBatch( | ||||||
|  |                 document, | ||||||
|  |                 orderedClaims, | ||||||
|  |                 ImmutableDictionary<string, string>.Empty)); | ||||||
|  |         } | ||||||
|  |         catch (JsonException ex) | ||||||
|  |         { | ||||||
|  |             _logger.LogError(ex, "Failed to parse CycloneDX VEX document {SourceUri}", document.SourceUri); | ||||||
|  |             throw; | ||||||
|  |         } | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     private static VexClaimStatus MapState(string? state, out string? raw) | ||||||
|  |     { | ||||||
|  |         raw = state?.Trim(); | ||||||
|  |  | ||||||
|  |         if (!string.IsNullOrWhiteSpace(state) && StateMap.TryGetValue(state.Trim(), out var mapped)) | ||||||
|  |         { | ||||||
|  |             return mapped; | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         return VexClaimStatus.UnderInvestigation; | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     private static VexJustification? MapJustification(string? justification) | ||||||
|  |     { | ||||||
|  |         if (string.IsNullOrWhiteSpace(justification)) | ||||||
|  |         { | ||||||
|  |             return null; | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         return JustificationMap.TryGetValue(justification.Trim(), out var mapped) | ||||||
|  |             ? mapped | ||||||
|  |             : null; | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     private sealed class CycloneDxParser | ||||||
|  |     { | ||||||
|  |         public static CycloneDxParseResult Parse(VexRawDocument document) | ||||||
|  |         { | ||||||
|  |             using var json = JsonDocument.Parse(document.Content.ToArray()); | ||||||
|  |             var root = json.RootElement; | ||||||
|  |  | ||||||
|  |             var specVersion = TryGetString(root, "specVersion"); | ||||||
|  |             var bomVersion = TryGetString(root, "version"); | ||||||
|  |             var serialNumber = TryGetString(root, "serialNumber"); | ||||||
|  |  | ||||||
|  |             var metadataTimestamp = ParseDate(TryGetProperty(root, "metadata"), "timestamp"); | ||||||
|  |             var observedTimestamp = metadataTimestamp ?? document.RetrievedAt; | ||||||
|  |  | ||||||
|  |             var metadataBuilder = ImmutableDictionary.CreateBuilder<string, string>(StringComparer.Ordinal); | ||||||
|  |             if (!string.IsNullOrWhiteSpace(specVersion)) | ||||||
|  |             { | ||||||
|  |                 metadataBuilder["cyclonedx.specVersion"] = specVersion!; | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             if (!string.IsNullOrWhiteSpace(bomVersion)) | ||||||
|  |             { | ||||||
|  |                 metadataBuilder["cyclonedx.version"] = bomVersion!; | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             if (!string.IsNullOrWhiteSpace(serialNumber)) | ||||||
|  |             { | ||||||
|  |                 metadataBuilder["cyclonedx.serialNumber"] = serialNumber!; | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             var components = CollectComponents(root); | ||||||
|  |             var vulnerabilities = CollectVulnerabilities(root); | ||||||
|  |  | ||||||
|  |             return new CycloneDxParseResult( | ||||||
|  |                 metadataBuilder.ToImmutable(), | ||||||
|  |                 bomVersion, | ||||||
|  |                 observedTimestamp, | ||||||
|  |                 observedTimestamp, | ||||||
|  |                 components, | ||||||
|  |                 vulnerabilities); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         private static ImmutableDictionary<string, CycloneDxComponent> CollectComponents(JsonElement root) | ||||||
|  |         { | ||||||
|  |             var builder = ImmutableDictionary.CreateBuilder<string, CycloneDxComponent>(StringComparer.Ordinal); | ||||||
|  |  | ||||||
|  |             if (root.TryGetProperty("components", out var components) && components.ValueKind == JsonValueKind.Array) | ||||||
|  |             { | ||||||
|  |                 foreach (var component in components.EnumerateArray()) | ||||||
|  |                 { | ||||||
|  |                     if (component.ValueKind != JsonValueKind.Object) | ||||||
|  |                     { | ||||||
|  |                         continue; | ||||||
|  |                     } | ||||||
|  |  | ||||||
|  |                     var reference = TryGetString(component, "bom-ref") ?? TryGetString(component, "bomRef"); | ||||||
|  |                     if (string.IsNullOrWhiteSpace(reference)) | ||||||
|  |                     { | ||||||
|  |                         continue; | ||||||
|  |                     } | ||||||
|  |  | ||||||
|  |                     var name = TryGetString(component, "name") ?? reference; | ||||||
|  |                     var version = TryGetString(component, "version"); | ||||||
|  |                     var purl = TryGetString(component, "purl"); | ||||||
|  |  | ||||||
|  |                     string? cpe = null; | ||||||
|  |                     if (component.TryGetProperty("externalReferences", out var externalRefs) && externalRefs.ValueKind == JsonValueKind.Array) | ||||||
|  |                     { | ||||||
|  |                         foreach (var referenceEntry in externalRefs.EnumerateArray()) | ||||||
|  |                         { | ||||||
|  |                             if (referenceEntry.ValueKind != JsonValueKind.Object) | ||||||
|  |                             { | ||||||
|  |                                 continue; | ||||||
|  |                             } | ||||||
|  |  | ||||||
|  |                             var type = TryGetString(referenceEntry, "type"); | ||||||
|  |                             if (!string.Equals(type, "cpe", StringComparison.OrdinalIgnoreCase)) | ||||||
|  |                             { | ||||||
|  |                                 continue; | ||||||
|  |                             } | ||||||
|  |  | ||||||
|  |                             if (referenceEntry.TryGetProperty("url", out var url) && url.ValueKind == JsonValueKind.String) | ||||||
|  |                             { | ||||||
|  |                                 cpe = url.GetString(); | ||||||
|  |                                 break; | ||||||
|  |                             } | ||||||
|  |                         } | ||||||
|  |                     } | ||||||
|  |  | ||||||
|  |                     builder[reference!] = new CycloneDxComponent(reference!, name ?? reference!, version, purl, cpe); | ||||||
|  |                 } | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             return builder.ToImmutable(); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         private static ImmutableArray<CycloneDxVulnerability> CollectVulnerabilities(JsonElement root) | ||||||
|  |         { | ||||||
|  |             if (!root.TryGetProperty("vulnerabilities", out var vulnerabilitiesElement) || | ||||||
|  |                 vulnerabilitiesElement.ValueKind != JsonValueKind.Array) | ||||||
|  |             { | ||||||
|  |                 return ImmutableArray<CycloneDxVulnerability>.Empty; | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             var builder = ImmutableArray.CreateBuilder<CycloneDxVulnerability>(); | ||||||
|  |  | ||||||
|  |             foreach (var vulnerability in vulnerabilitiesElement.EnumerateArray()) | ||||||
|  |             { | ||||||
|  |                 if (vulnerability.ValueKind != JsonValueKind.Object) | ||||||
|  |                 { | ||||||
|  |                     continue; | ||||||
|  |                 } | ||||||
|  |  | ||||||
|  |                 var vulnerabilityId = | ||||||
|  |                     TryGetString(vulnerability, "id") ?? | ||||||
|  |                     TryGetString(vulnerability, "bom-ref") ?? | ||||||
|  |                     TryGetString(vulnerability, "bomRef") ?? | ||||||
|  |                     TryGetString(vulnerability, "cve"); | ||||||
|  |  | ||||||
|  |                 if (string.IsNullOrWhiteSpace(vulnerabilityId)) | ||||||
|  |                 { | ||||||
|  |                     continue; | ||||||
|  |                 } | ||||||
|  |  | ||||||
|  |                 var detail = TryGetString(vulnerability, "detail") ?? TryGetString(vulnerability, "description"); | ||||||
|  |  | ||||||
|  |                 var analysis = TryGetProperty(vulnerability, "analysis"); | ||||||
|  |                 var analysisState = TryGetString(analysis, "state"); | ||||||
|  |                 var analysisJustification = TryGetString(analysis, "justification"); | ||||||
|  |                 var analysisResponses = CollectResponses(analysis); | ||||||
|  |  | ||||||
|  |                 var affects = CollectAffects(vulnerability); | ||||||
|  |                 if (affects.Length == 0) | ||||||
|  |                 { | ||||||
|  |                     continue; | ||||||
|  |                 } | ||||||
|  |  | ||||||
|  |                 builder.Add(new CycloneDxVulnerability( | ||||||
|  |                     vulnerabilityId.Trim(), | ||||||
|  |                     detail?.Trim(), | ||||||
|  |                     analysisState, | ||||||
|  |                     analysisJustification, | ||||||
|  |                     analysisResponses, | ||||||
|  |                     affects)); | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             return builder.ToImmutable(); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         private static ImmutableArray<string> CollectResponses(JsonElement analysis) | ||||||
|  |         { | ||||||
|  |             if (analysis.ValueKind != JsonValueKind.Object || | ||||||
|  |                 !analysis.TryGetProperty("response", out var responseElement) || | ||||||
|  |                 responseElement.ValueKind != JsonValueKind.Array) | ||||||
|  |             { | ||||||
|  |                 return ImmutableArray<string>.Empty; | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             var responses = new SortedSet<string>(StringComparer.OrdinalIgnoreCase); | ||||||
|  |             foreach (var response in responseElement.EnumerateArray()) | ||||||
|  |             { | ||||||
|  |                 if (response.ValueKind == JsonValueKind.String) | ||||||
|  |                 { | ||||||
|  |                     var value = response.GetString(); | ||||||
|  |                     if (!string.IsNullOrWhiteSpace(value)) | ||||||
|  |                     { | ||||||
|  |                         responses.Add(value.Trim()); | ||||||
|  |                     } | ||||||
|  |                 } | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             return responses.Count == 0 ? ImmutableArray<string>.Empty : responses.ToImmutableArray(); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         private static ImmutableArray<CycloneDxAffect> CollectAffects(JsonElement vulnerability) | ||||||
|  |         { | ||||||
|  |             if (!vulnerability.TryGetProperty("affects", out var affectsElement) || | ||||||
|  |                 affectsElement.ValueKind != JsonValueKind.Array) | ||||||
|  |             { | ||||||
|  |                 return ImmutableArray<CycloneDxAffect>.Empty; | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             var builder = ImmutableArray.CreateBuilder<CycloneDxAffect>(); | ||||||
|  |             foreach (var affect in affectsElement.EnumerateArray()) | ||||||
|  |             { | ||||||
|  |                 if (affect.ValueKind != JsonValueKind.Object) | ||||||
|  |                 { | ||||||
|  |                     continue; | ||||||
|  |                 } | ||||||
|  |  | ||||||
|  |                 var reference = TryGetString(affect, "ref"); | ||||||
|  |                 if (string.IsNullOrWhiteSpace(reference)) | ||||||
|  |                 { | ||||||
|  |                     continue; | ||||||
|  |                 } | ||||||
|  |  | ||||||
|  |                 builder.Add(new CycloneDxAffect(reference.Trim())); | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             return builder.ToImmutable(); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         private static JsonElement TryGetProperty(JsonElement element, string propertyName) | ||||||
|  |             => element.ValueKind == JsonValueKind.Object && element.TryGetProperty(propertyName, out var value) | ||||||
|  |                 ? value | ||||||
|  |                 : default; | ||||||
|  |  | ||||||
|  |         private static string? TryGetString(JsonElement element, string propertyName) | ||||||
|  |         { | ||||||
|  |             if (element.ValueKind != JsonValueKind.Object) | ||||||
|  |             { | ||||||
|  |                 return null; | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             if (!element.TryGetProperty(propertyName, out var value)) | ||||||
|  |             { | ||||||
|  |                 return null; | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             return value.ValueKind == JsonValueKind.String ? value.GetString() : null; | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         private static DateTimeOffset? ParseDate(JsonElement element, string propertyName) | ||||||
|  |         { | ||||||
|  |             var value = TryGetString(element, propertyName); | ||||||
|  |             if (string.IsNullOrWhiteSpace(value)) | ||||||
|  |             { | ||||||
|  |                 return null; | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             return DateTimeOffset.TryParse(value, out var parsed) ? parsed : null; | ||||||
|  |         } | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     private sealed record CycloneDxParseResult( | ||||||
|  |         ImmutableDictionary<string, string> Metadata, | ||||||
|  |         string? BomVersion, | ||||||
|  |         DateTimeOffset FirstObserved, | ||||||
|  |         DateTimeOffset LastObserved, | ||||||
|  |         ImmutableDictionary<string, CycloneDxComponent> Components, | ||||||
|  |         ImmutableArray<CycloneDxVulnerability> Vulnerabilities) | ||||||
|  |     { | ||||||
|  |         public CycloneDxProductInfo ResolveProduct(string? componentRef) | ||||||
|  |         { | ||||||
|  |             if (!string.IsNullOrWhiteSpace(componentRef) && | ||||||
|  |                 Components.TryGetValue(componentRef.Trim(), out var component)) | ||||||
|  |             { | ||||||
|  |                 return new CycloneDxProductInfo(component.Reference, component.Name, component.Version, component.Purl, component.Cpe); | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             var key = string.IsNullOrWhiteSpace(componentRef) ? "unknown-component" : componentRef.Trim(); | ||||||
|  |             return new CycloneDxProductInfo(key, key, null, null, null); | ||||||
|  |         } | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     private sealed record CycloneDxComponent( | ||||||
|  |         string Reference, | ||||||
|  |         string Name, | ||||||
|  |         string? Version, | ||||||
|  |         string? Purl, | ||||||
|  |         string? Cpe); | ||||||
|  |  | ||||||
|  |     private sealed record CycloneDxVulnerability( | ||||||
|  |         string VulnerabilityId, | ||||||
|  |         string? Detail, | ||||||
|  |         string? AnalysisState, | ||||||
|  |         string? AnalysisJustification, | ||||||
|  |         ImmutableArray<string> AnalysisResponses, | ||||||
|  |         ImmutableArray<CycloneDxAffect> Affects); | ||||||
|  |  | ||||||
|  |     private sealed record CycloneDxAffect(string ComponentRef); | ||||||
|  |  | ||||||
|  |     private sealed record CycloneDxProductInfo( | ||||||
|  |         string Key, | ||||||
|  |         string Name, | ||||||
|  |         string? Version, | ||||||
|  |         string? Purl, | ||||||
|  |         string? Cpe); | ||||||
|  | } | ||||||
| @@ -0,0 +1,14 @@ | |||||||
|  | using Microsoft.Extensions.DependencyInjection; | ||||||
|  | using StellaOps.Vexer.Core; | ||||||
|  |  | ||||||
|  | namespace StellaOps.Vexer.Formats.CycloneDX; | ||||||
|  |  | ||||||
|  | public static class CycloneDxFormatsServiceCollectionExtensions | ||||||
|  | { | ||||||
|  |     public static IServiceCollection AddCycloneDxNormalizer(this IServiceCollection services) | ||||||
|  |     { | ||||||
|  |         ArgumentNullException.ThrowIfNull(services); | ||||||
|  |         services.AddSingleton<IVexNormalizer, CycloneDxNormalizer>(); | ||||||
|  |         return services; | ||||||
|  |     } | ||||||
|  | } | ||||||
| @@ -0,0 +1,16 @@ | |||||||
|  | <Project Sdk="Microsoft.NET.Sdk"> | ||||||
|  |   <PropertyGroup> | ||||||
|  |     <TargetFramework>net10.0</TargetFramework> | ||||||
|  |     <LangVersion>preview</LangVersion> | ||||||
|  |     <Nullable>enable</Nullable> | ||||||
|  |     <ImplicitUsings>enable</ImplicitUsings> | ||||||
|  |     <TreatWarningsAsErrors>true</TreatWarningsAsErrors> | ||||||
|  |   </PropertyGroup> | ||||||
|  |   <ItemGroup> | ||||||
|  |     <PackageReference Include="Microsoft.Extensions.DependencyInjection.Abstractions" Version="8.0.0" /> | ||||||
|  |     <PackageReference Include="Microsoft.Extensions.Logging.Abstractions" Version="8.0.0" /> | ||||||
|  |   </ItemGroup> | ||||||
|  |   <ItemGroup> | ||||||
|  |     <ProjectReference Include="..\StellaOps.Vexer.Core\StellaOps.Vexer.Core.csproj" /> | ||||||
|  |   </ItemGroup> | ||||||
|  | </Project> | ||||||
| @@ -2,6 +2,6 @@ If you are working on this file you need to read docs/ARCHITECTURE_VEXER.md and | |||||||
| # TASKS | # TASKS | ||||||
| | Task | Owner(s) | Depends on | Notes | | | Task | Owner(s) | Depends on | Notes | | ||||||
| |---|---|---|---| | |---|---|---|---| | ||||||
| |VEXER-FMT-CYCLONE-01-001 – CycloneDX VEX normalizer|Team Vexer Formats|VEXER-CORE-01-001|TODO – Parse CycloneDX `analysis` data into `VexClaim` entries with deterministic component identifiers.| | |VEXER-FMT-CYCLONE-01-001 – CycloneDX VEX normalizer|Team Vexer Formats|VEXER-CORE-01-001|**DONE (2025-10-17)** – CycloneDX normalizer parses `analysis` data, resolves component references, and emits canonical `VexClaim`s; regression lives in `CycloneDxNormalizerTests`.| | ||||||
| |VEXER-FMT-CYCLONE-01-002 – Component reference reconciliation|Team Vexer Formats|VEXER-FMT-CYCLONE-01-001|TODO – Implement helpers to reconcile component/service references against policy expectations and emit diagnostics for missing SBOM links.| | |VEXER-FMT-CYCLONE-01-002 – Component reference reconciliation|Team Vexer Formats|VEXER-FMT-CYCLONE-01-001|TODO – Implement helpers to reconcile component/service references against policy expectations and emit diagnostics for missing SBOM links.| | ||||||
| |VEXER-FMT-CYCLONE-01-003 – CycloneDX export serializer|Team Vexer Formats|VEXER-EXPORT-01-001, VEXER-FMT-CYCLONE-01-001|TODO – Provide exporters producing CycloneDX VEX output with canonical ordering and hash-stable manifests.| | |VEXER-FMT-CYCLONE-01-003 – CycloneDX export serializer|Team Vexer Formats|VEXER-EXPORT-01-001, VEXER-FMT-CYCLONE-01-001|TODO – Provide exporters producing CycloneDX VEX output with canonical ordering and hash-stable manifests.| | ||||||
|   | |||||||
| @@ -0,0 +1,87 @@ | |||||||
|  | using System.Collections.Immutable; | ||||||
|  | using System.Linq; | ||||||
|  | using System.Text; | ||||||
|  | using FluentAssertions; | ||||||
|  | using Microsoft.Extensions.Logging.Abstractions; | ||||||
|  | using StellaOps.Vexer.Core; | ||||||
|  | using StellaOps.Vexer.Formats.OpenVEX; | ||||||
|  |  | ||||||
|  | namespace StellaOps.Vexer.Formats.OpenVEX.Tests; | ||||||
|  |  | ||||||
|  | public sealed class OpenVexNormalizerTests | ||||||
|  | { | ||||||
|  |     [Fact] | ||||||
|  |     public async Task NormalizeAsync_ProducesClaimsForStatements() | ||||||
|  |     { | ||||||
|  |         var json = """ | ||||||
|  |         { | ||||||
|  |           "document": { | ||||||
|  |             "author": "Acme Security", | ||||||
|  |             "version": "1", | ||||||
|  |             "issued": "2025-10-01T00:00:00Z", | ||||||
|  |             "last_updated": "2025-10-05T00:00:00Z" | ||||||
|  |           }, | ||||||
|  |           "statements": [ | ||||||
|  |             { | ||||||
|  |               "id": "statement-1", | ||||||
|  |               "vulnerability": "CVE-2025-2000", | ||||||
|  |               "status": "not_affected", | ||||||
|  |               "justification": "code_not_present", | ||||||
|  |               "products": [ | ||||||
|  |                 { | ||||||
|  |                   "id": "acme-widget@1.2.3", | ||||||
|  |                   "name": "Acme Widget", | ||||||
|  |                   "version": "1.2.3", | ||||||
|  |                   "purl": "pkg:acme/widget@1.2.3", | ||||||
|  |                   "cpe": "cpe:/a:acme:widget:1.2.3" | ||||||
|  |                 } | ||||||
|  |               ], | ||||||
|  |               "statement": "The vulnerable code was never shipped." | ||||||
|  |             }, | ||||||
|  |             { | ||||||
|  |               "id": "statement-2", | ||||||
|  |               "vulnerability": "CVE-2025-2001", | ||||||
|  |               "status": "affected", | ||||||
|  |               "products": [ | ||||||
|  |                 "pkg:acme/widget@2.0.0" | ||||||
|  |               ], | ||||||
|  |               "remediation": "Upgrade to 2.1.0" | ||||||
|  |             } | ||||||
|  |           ] | ||||||
|  |         } | ||||||
|  |         """; | ||||||
|  |  | ||||||
|  |         var rawDocument = new VexRawDocument( | ||||||
|  |             "vexer:openvex", | ||||||
|  |             VexDocumentFormat.OpenVex, | ||||||
|  |             new Uri("https://example.com/openvex.json"), | ||||||
|  |             new DateTimeOffset(2025, 10, 6, 0, 0, 0, TimeSpan.Zero), | ||||||
|  |             "sha256:dummydigest", | ||||||
|  |             Encoding.UTF8.GetBytes(json), | ||||||
|  |             ImmutableDictionary<string, string>.Empty); | ||||||
|  |  | ||||||
|  |         var provider = new VexProvider("vexer:openvex", "OpenVEX Provider", VexProviderKind.Vendor); | ||||||
|  |         var normalizer = new OpenVexNormalizer(NullLogger<OpenVexNormalizer>.Instance); | ||||||
|  |  | ||||||
|  |         var batch = await normalizer.NormalizeAsync(rawDocument, provider, CancellationToken.None); | ||||||
|  |  | ||||||
|  |         batch.Claims.Should().HaveCount(2); | ||||||
|  |  | ||||||
|  |         var notAffected = batch.Claims.Single(c => c.VulnerabilityId == "CVE-2025-2000"); | ||||||
|  |         notAffected.Status.Should().Be(VexClaimStatus.NotAffected); | ||||||
|  |         notAffected.Justification.Should().Be(VexJustification.CodeNotPresent); | ||||||
|  |         notAffected.Product.Key.Should().Be("acme-widget@1.2.3"); | ||||||
|  |         notAffected.Product.Purl.Should().Be("pkg:acme/widget@1.2.3"); | ||||||
|  |         notAffected.Document.Revision.Should().Be("1"); | ||||||
|  |         notAffected.AdditionalMetadata["openvex.document.author"].Should().Be("Acme Security"); | ||||||
|  |         notAffected.AdditionalMetadata["openvex.statement.status"].Should().Be("not_affected"); | ||||||
|  |         notAffected.Detail.Should().Be("The vulnerable code was never shipped."); | ||||||
|  |  | ||||||
|  |         var affected = batch.Claims.Single(c => c.VulnerabilityId == "CVE-2025-2001"); | ||||||
|  |         affected.Status.Should().Be(VexClaimStatus.Affected); | ||||||
|  |         affected.Justification.Should().BeNull(); | ||||||
|  |         affected.Product.Key.Should().Be("pkg:acme/widget@2.0.0"); | ||||||
|  |         affected.Product.Name.Should().Be("pkg:acme/widget@2.0.0"); | ||||||
|  |         affected.Detail.Should().Be("Upgrade to 2.1.0"); | ||||||
|  |     } | ||||||
|  | } | ||||||
| @@ -0,0 +1,17 @@ | |||||||
|  | <Project Sdk="Microsoft.NET.Sdk"> | ||||||
|  |   <PropertyGroup> | ||||||
|  |     <TargetFramework>net10.0</TargetFramework> | ||||||
|  |     <Nullable>enable</Nullable> | ||||||
|  |     <ImplicitUsings>enable</ImplicitUsings> | ||||||
|  |   </PropertyGroup> | ||||||
|  |   <ItemGroup> | ||||||
|  |     <PackageReference Include="FluentAssertions" Version="6.12.0" /> | ||||||
|  |   </ItemGroup> | ||||||
|  |   <ItemGroup> | ||||||
|  |     <Using Include="Xunit" /> | ||||||
|  |   </ItemGroup> | ||||||
|  |   <ItemGroup> | ||||||
|  |     <ProjectReference Include="..\StellaOps.Vexer.Core\StellaOps.Vexer.Core.csproj" /> | ||||||
|  |     <ProjectReference Include="..\StellaOps.Vexer.Formats.OpenVEX\StellaOps.Vexer.Formats.OpenVEX.csproj" /> | ||||||
|  |   </ItemGroup> | ||||||
|  | </Project> | ||||||
							
								
								
									
										367
									
								
								src/StellaOps.Vexer.Formats.OpenVEX/OpenVexNormalizer.cs
									
									
									
									
									
										Normal file
									
								
							
							
						
						
									
										367
									
								
								src/StellaOps.Vexer.Formats.OpenVEX/OpenVexNormalizer.cs
									
									
									
									
									
										Normal file
									
								
							| @@ -0,0 +1,367 @@ | |||||||
|  | using System; | ||||||
|  | using System.Collections.Generic; | ||||||
|  | using System.Collections.Immutable; | ||||||
|  | using System.Linq; | ||||||
|  | using System.Text.Json; | ||||||
|  | using System.Threading; | ||||||
|  | using System.Threading.Tasks; | ||||||
|  | using Microsoft.Extensions.Logging; | ||||||
|  | using StellaOps.Vexer.Core; | ||||||
|  |  | ||||||
|  | namespace StellaOps.Vexer.Formats.OpenVEX; | ||||||
|  |  | ||||||
|  | public sealed class OpenVexNormalizer : IVexNormalizer | ||||||
|  | { | ||||||
|  |     private static readonly ImmutableDictionary<string, VexClaimStatus> StatusMap = new Dictionary<string, VexClaimStatus>(StringComparer.OrdinalIgnoreCase) | ||||||
|  |     { | ||||||
|  |         ["affected"] = VexClaimStatus.Affected, | ||||||
|  |         ["not_affected"] = VexClaimStatus.NotAffected, | ||||||
|  |         ["fixed"] = VexClaimStatus.Fixed, | ||||||
|  |         ["under_investigation"] = VexClaimStatus.UnderInvestigation, | ||||||
|  |     }.ToImmutableDictionary(StringComparer.OrdinalIgnoreCase); | ||||||
|  |  | ||||||
|  |     private static readonly ImmutableDictionary<string, VexJustification> JustificationMap = new Dictionary<string, VexJustification>(StringComparer.OrdinalIgnoreCase) | ||||||
|  |     { | ||||||
|  |         ["component_not_present"] = VexJustification.ComponentNotPresent, | ||||||
|  |         ["component_not_configured"] = VexJustification.ComponentNotConfigured, | ||||||
|  |         ["vulnerable_code_not_present"] = VexJustification.VulnerableCodeNotPresent, | ||||||
|  |         ["vulnerable_code_not_in_execute_path"] = VexJustification.VulnerableCodeNotInExecutePath, | ||||||
|  |         ["vulnerable_code_cannot_be_controlled_by_adversary"] = VexJustification.VulnerableCodeCannotBeControlledByAdversary, | ||||||
|  |         ["inline_mitigations_already_exist"] = VexJustification.InlineMitigationsAlreadyExist, | ||||||
|  |         ["protected_by_mitigating_control"] = VexJustification.ProtectedByMitigatingControl, | ||||||
|  |         ["protected_by_compensating_control"] = VexJustification.ProtectedByCompensatingControl, | ||||||
|  |         ["code_not_present"] = VexJustification.CodeNotPresent, | ||||||
|  |         ["code_not_reachable"] = VexJustification.CodeNotReachable, | ||||||
|  |         ["requires_configuration"] = VexJustification.RequiresConfiguration, | ||||||
|  |         ["requires_dependency"] = VexJustification.RequiresDependency, | ||||||
|  |         ["requires_environment"] = VexJustification.RequiresEnvironment, | ||||||
|  |     }.ToImmutableDictionary(StringComparer.OrdinalIgnoreCase); | ||||||
|  |  | ||||||
|  |     private readonly ILogger<OpenVexNormalizer> _logger; | ||||||
|  |  | ||||||
|  |     public OpenVexNormalizer(ILogger<OpenVexNormalizer> logger) | ||||||
|  |     { | ||||||
|  |         _logger = logger ?? throw new ArgumentNullException(nameof(logger)); | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     public string Format => VexDocumentFormat.OpenVex.ToString().ToLowerInvariant(); | ||||||
|  |  | ||||||
|  |     public bool CanHandle(VexRawDocument document) | ||||||
|  |         => document is not null && document.Format == VexDocumentFormat.OpenVex; | ||||||
|  |  | ||||||
|  |     public ValueTask<VexClaimBatch> NormalizeAsync(VexRawDocument document, VexProvider provider, CancellationToken cancellationToken) | ||||||
|  |     { | ||||||
|  |         ArgumentNullException.ThrowIfNull(document); | ||||||
|  |         ArgumentNullException.ThrowIfNull(provider); | ||||||
|  |         cancellationToken.ThrowIfCancellationRequested(); | ||||||
|  |  | ||||||
|  |         try | ||||||
|  |         { | ||||||
|  |             var result = OpenVexParser.Parse(document); | ||||||
|  |             var claims = ImmutableArray.CreateBuilder<VexClaim>(result.Statements.Length); | ||||||
|  |  | ||||||
|  |             foreach (var statement in result.Statements) | ||||||
|  |             { | ||||||
|  |                 cancellationToken.ThrowIfCancellationRequested(); | ||||||
|  |  | ||||||
|  |                 var status = MapStatus(statement.Status); | ||||||
|  |                 var justification = MapJustification(statement.Justification); | ||||||
|  |  | ||||||
|  |                 foreach (var product in statement.Products) | ||||||
|  |                 { | ||||||
|  |                     var vexProduct = new VexProduct( | ||||||
|  |                         product.Key, | ||||||
|  |                         product.Name, | ||||||
|  |                         product.Version, | ||||||
|  |                         product.Purl, | ||||||
|  |                         product.Cpe); | ||||||
|  |  | ||||||
|  |                     var metadata = result.Metadata; | ||||||
|  |  | ||||||
|  |                     metadata = metadata.SetItem("openvex.statement.id", statement.Id); | ||||||
|  |                     if (!string.IsNullOrWhiteSpace(statement.Status)) | ||||||
|  |                     { | ||||||
|  |                         metadata = metadata.SetItem("openvex.statement.status", statement.Status!); | ||||||
|  |                     } | ||||||
|  |  | ||||||
|  |                     if (!string.IsNullOrWhiteSpace(statement.Justification)) | ||||||
|  |                     { | ||||||
|  |                         metadata = metadata.SetItem("openvex.statement.justification", statement.Justification!); | ||||||
|  |                     } | ||||||
|  |  | ||||||
|  |                     if (!string.IsNullOrWhiteSpace(product.OriginalId)) | ||||||
|  |                     { | ||||||
|  |                         metadata = metadata.SetItem("openvex.product.source", product.OriginalId!); | ||||||
|  |                     } | ||||||
|  |  | ||||||
|  |                     var claimDocument = new VexClaimDocument( | ||||||
|  |                         VexDocumentFormat.OpenVex, | ||||||
|  |                         document.Digest, | ||||||
|  |                         document.SourceUri, | ||||||
|  |                         result.DocumentVersion, | ||||||
|  |                         signature: null); | ||||||
|  |  | ||||||
|  |                     var claim = new VexClaim( | ||||||
|  |                         statement.Vulnerability, | ||||||
|  |                         provider.Id, | ||||||
|  |                         vexProduct, | ||||||
|  |                         status, | ||||||
|  |                         claimDocument, | ||||||
|  |                         result.FirstObserved, | ||||||
|  |                         result.LastObserved, | ||||||
|  |                         justification, | ||||||
|  |                         statement.Remarks, | ||||||
|  |                         confidence: null, | ||||||
|  |                         additionalMetadata: metadata); | ||||||
|  |  | ||||||
|  |                     claims.Add(claim); | ||||||
|  |                 } | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             var orderedClaims = claims | ||||||
|  |                 .ToImmutable() | ||||||
|  |                 .OrderBy(static claim => claim.VulnerabilityId, StringComparer.Ordinal) | ||||||
|  |                 .ThenBy(static claim => claim.Product.Key, StringComparer.Ordinal) | ||||||
|  |                 .ToImmutableArray(); | ||||||
|  |  | ||||||
|  |             _logger.LogInformation( | ||||||
|  |                 "Normalized OpenVEX document {Source} into {ClaimCount} claim(s).", | ||||||
|  |                 document.SourceUri, | ||||||
|  |                 orderedClaims.Length); | ||||||
|  |  | ||||||
|  |             return ValueTask.FromResult(new VexClaimBatch( | ||||||
|  |                 document, | ||||||
|  |                 orderedClaims, | ||||||
|  |                 ImmutableDictionary<string, string>.Empty)); | ||||||
|  |         } | ||||||
|  |         catch (JsonException ex) | ||||||
|  |         { | ||||||
|  |             _logger.LogError(ex, "Failed to parse OpenVEX document {SourceUri}", document.SourceUri); | ||||||
|  |             throw; | ||||||
|  |         } | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     private static VexClaimStatus MapStatus(string? status) | ||||||
|  |     { | ||||||
|  |         if (!string.IsNullOrWhiteSpace(status) && StatusMap.TryGetValue(status.Trim(), out var mapped)) | ||||||
|  |         { | ||||||
|  |             return mapped; | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         return VexClaimStatus.UnderInvestigation; | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     private static VexJustification? MapJustification(string? justification) | ||||||
|  |     { | ||||||
|  |         if (string.IsNullOrWhiteSpace(justification)) | ||||||
|  |         { | ||||||
|  |             return null; | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         return JustificationMap.TryGetValue(justification.Trim(), out var mapped) | ||||||
|  |             ? mapped | ||||||
|  |             : null; | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     private static class OpenVexParser | ||||||
|  |     { | ||||||
|  |         public static OpenVexParseResult Parse(VexRawDocument document) | ||||||
|  |         { | ||||||
|  |             using var json = JsonDocument.Parse(document.Content.ToArray()); | ||||||
|  |             var root = json.RootElement; | ||||||
|  |  | ||||||
|  |             var metadata = ImmutableDictionary.CreateBuilder<string, string>(StringComparer.Ordinal); | ||||||
|  |  | ||||||
|  |             var documentElement = TryGetProperty(root, "document"); | ||||||
|  |             var version = TryGetString(documentElement, "version"); | ||||||
|  |             var author = TryGetString(documentElement, "author"); | ||||||
|  |  | ||||||
|  |             if (!string.IsNullOrWhiteSpace(version)) | ||||||
|  |             { | ||||||
|  |                 metadata["openvex.document.version"] = version!; | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             if (!string.IsNullOrWhiteSpace(author)) | ||||||
|  |             { | ||||||
|  |                 metadata["openvex.document.author"] = author!; | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             var issued = ParseDate(documentElement, "issued"); | ||||||
|  |             var lastUpdated = ParseDate(documentElement, "last_updated") ?? issued ?? document.RetrievedAt; | ||||||
|  |             var effectiveDate = ParseDate(documentElement, "effective_date") ?? issued ?? document.RetrievedAt; | ||||||
|  |  | ||||||
|  |             var statements = CollectStatements(root); | ||||||
|  |  | ||||||
|  |             return new OpenVexParseResult( | ||||||
|  |                 metadata.ToImmutable(), | ||||||
|  |                 version, | ||||||
|  |                 effectiveDate, | ||||||
|  |                 lastUpdated, | ||||||
|  |                 statements); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         private static ImmutableArray<OpenVexStatement> CollectStatements(JsonElement root) | ||||||
|  |         { | ||||||
|  |             if (!root.TryGetProperty("statements", out var statementsElement) || | ||||||
|  |                 statementsElement.ValueKind != JsonValueKind.Array) | ||||||
|  |             { | ||||||
|  |                 return ImmutableArray<OpenVexStatement>.Empty; | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             var builder = ImmutableArray.CreateBuilder<OpenVexStatement>(); | ||||||
|  |             foreach (var statement in statementsElement.EnumerateArray()) | ||||||
|  |             { | ||||||
|  |                 if (statement.ValueKind != JsonValueKind.Object) | ||||||
|  |                 { | ||||||
|  |                     continue; | ||||||
|  |                 } | ||||||
|  |  | ||||||
|  |                 var vulnerability = TryGetString(statement, "vulnerability") ?? TryGetString(statement, "vuln") ?? string.Empty; | ||||||
|  |                 if (string.IsNullOrWhiteSpace(vulnerability)) | ||||||
|  |                 { | ||||||
|  |                     continue; | ||||||
|  |                 } | ||||||
|  |  | ||||||
|  |                 var id = TryGetString(statement, "id") ?? Guid.NewGuid().ToString(); | ||||||
|  |                 var status = TryGetString(statement, "status"); | ||||||
|  |                 var justification = TryGetString(statement, "justification"); | ||||||
|  |                 var remarks = TryGetString(statement, "remediation") ?? TryGetString(statement, "statement"); | ||||||
|  |                 var products = CollectProducts(statement); | ||||||
|  |                 if (products.Length == 0) | ||||||
|  |                 { | ||||||
|  |                     continue; | ||||||
|  |                 } | ||||||
|  |  | ||||||
|  |                 builder.Add(new OpenVexStatement( | ||||||
|  |                     id, | ||||||
|  |                     vulnerability.Trim(), | ||||||
|  |                     status, | ||||||
|  |                     justification, | ||||||
|  |                     remarks, | ||||||
|  |                     products)); | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             return builder.ToImmutable(); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         private static ImmutableArray<OpenVexProduct> CollectProducts(JsonElement statement) | ||||||
|  |         { | ||||||
|  |             if (!statement.TryGetProperty("products", out var productsElement) || | ||||||
|  |                 productsElement.ValueKind != JsonValueKind.Array) | ||||||
|  |             { | ||||||
|  |                 return ImmutableArray<OpenVexProduct>.Empty; | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             var builder = ImmutableArray.CreateBuilder<OpenVexProduct>(); | ||||||
|  |             foreach (var product in productsElement.EnumerateArray()) | ||||||
|  |             { | ||||||
|  |                 if (product.ValueKind != JsonValueKind.String && product.ValueKind != JsonValueKind.Object) | ||||||
|  |                 { | ||||||
|  |                     continue; | ||||||
|  |                 } | ||||||
|  |  | ||||||
|  |                 if (product.ValueKind == JsonValueKind.String) | ||||||
|  |                 { | ||||||
|  |                     var value = product.GetString(); | ||||||
|  |                     if (string.IsNullOrWhiteSpace(value)) | ||||||
|  |                     { | ||||||
|  |                         continue; | ||||||
|  |                     } | ||||||
|  |  | ||||||
|  |                     builder.Add(OpenVexProduct.FromString(value.Trim())); | ||||||
|  |                     continue; | ||||||
|  |                 } | ||||||
|  |  | ||||||
|  |                 var id = TryGetString(product, "id") ?? TryGetString(product, "product_id"); | ||||||
|  |                 var name = TryGetString(product, "name"); | ||||||
|  |                 var version = TryGetString(product, "version"); | ||||||
|  |                 var purl = TryGetString(product, "purl"); | ||||||
|  |                 var cpe = TryGetString(product, "cpe"); | ||||||
|  |  | ||||||
|  |                 if (string.IsNullOrWhiteSpace(id) && string.IsNullOrWhiteSpace(purl)) | ||||||
|  |                 { | ||||||
|  |                     continue; | ||||||
|  |                 } | ||||||
|  |  | ||||||
|  |                 builder.Add(new OpenVexProduct( | ||||||
|  |                     id ?? purl!, | ||||||
|  |                     name ?? id ?? purl!, | ||||||
|  |                     version, | ||||||
|  |                     purl, | ||||||
|  |                     cpe, | ||||||
|  |                     OriginalId: id)); | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             return builder.ToImmutable(); | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         private static JsonElement TryGetProperty(JsonElement element, string propertyName) | ||||||
|  |             => element.ValueKind == JsonValueKind.Object && element.TryGetProperty(propertyName, out var value) | ||||||
|  |                 ? value | ||||||
|  |                 : default; | ||||||
|  |  | ||||||
|  |         private static string? TryGetString(JsonElement element, string propertyName) | ||||||
|  |         { | ||||||
|  |             if (element.ValueKind != JsonValueKind.Object) | ||||||
|  |             { | ||||||
|  |                 return null; | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             if (!element.TryGetProperty(propertyName, out var value)) | ||||||
|  |             { | ||||||
|  |                 return null; | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             return value.ValueKind == JsonValueKind.String ? value.GetString() : null; | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         private static DateTimeOffset? ParseDate(JsonElement element, string propertyName) | ||||||
|  |         { | ||||||
|  |             var value = TryGetString(element, propertyName); | ||||||
|  |             if (string.IsNullOrWhiteSpace(value)) | ||||||
|  |             { | ||||||
|  |                 return null; | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             return DateTimeOffset.TryParse(value, out var parsed) ? parsed : null; | ||||||
|  |         } | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     private sealed record OpenVexParseResult( | ||||||
|  |         ImmutableDictionary<string, string> Metadata, | ||||||
|  |         string? DocumentVersion, | ||||||
|  |         DateTimeOffset FirstObserved, | ||||||
|  |         DateTimeOffset LastObserved, | ||||||
|  |         ImmutableArray<OpenVexStatement> Statements); | ||||||
|  |  | ||||||
|  |     private sealed record OpenVexStatement( | ||||||
|  |         string Id, | ||||||
|  |         string Vulnerability, | ||||||
|  |         string? Status, | ||||||
|  |         string? Justification, | ||||||
|  |         string? Remarks, | ||||||
|  |         ImmutableArray<OpenVexProduct> Products); | ||||||
|  |  | ||||||
|  |     private sealed record OpenVexProduct( | ||||||
|  |         string Key, | ||||||
|  |         string Name, | ||||||
|  |         string? Version, | ||||||
|  |         string? Purl, | ||||||
|  |         string? Cpe, | ||||||
|  |         string? OriginalId) | ||||||
|  |     { | ||||||
|  |         public static OpenVexProduct FromString(string value) | ||||||
|  |         { | ||||||
|  |             var key = value; | ||||||
|  |             string? purl = null; | ||||||
|  |             string? name = value; | ||||||
|  |  | ||||||
|  |             if (value.StartsWith("pkg:", StringComparison.OrdinalIgnoreCase)) | ||||||
|  |             { | ||||||
|  |                 purl = value; | ||||||
|  |             } | ||||||
|  |  | ||||||
|  |             return new OpenVexProduct(key, name, null, purl, null, OriginalId: value); | ||||||
|  |         } | ||||||
|  |     } | ||||||
|  | } | ||||||
| @@ -0,0 +1,14 @@ | |||||||
|  | using Microsoft.Extensions.DependencyInjection; | ||||||
|  | using StellaOps.Vexer.Core; | ||||||
|  |  | ||||||
|  | namespace StellaOps.Vexer.Formats.OpenVEX; | ||||||
|  |  | ||||||
|  | public static class OpenVexFormatsServiceCollectionExtensions | ||||||
|  | { | ||||||
|  |     public static IServiceCollection AddOpenVexNormalizer(this IServiceCollection services) | ||||||
|  |     { | ||||||
|  |         ArgumentNullException.ThrowIfNull(services); | ||||||
|  |         services.AddSingleton<IVexNormalizer, OpenVexNormalizer>(); | ||||||
|  |         return services; | ||||||
|  |     } | ||||||
|  | } | ||||||
| @@ -0,0 +1,16 @@ | |||||||
|  | <Project Sdk="Microsoft.NET.Sdk"> | ||||||
|  |   <PropertyGroup> | ||||||
|  |     <TargetFramework>net10.0</TargetFramework> | ||||||
|  |     <LangVersion>preview</LangVersion> | ||||||
|  |     <Nullable>enable</Nullable> | ||||||
|  |     <ImplicitUsings>enable</ImplicitUsings> | ||||||
|  |     <TreatWarningsAsErrors>true</TreatWarningsAsErrors> | ||||||
|  |   </PropertyGroup> | ||||||
|  |   <ItemGroup> | ||||||
|  |     <PackageReference Include="Microsoft.Extensions.DependencyInjection.Abstractions" Version="8.0.0" /> | ||||||
|  |     <PackageReference Include="Microsoft.Extensions.Logging.Abstractions" Version="8.0.0" /> | ||||||
|  |   </ItemGroup> | ||||||
|  |   <ItemGroup> | ||||||
|  |     <ProjectReference Include="..\StellaOps.Vexer.Core\StellaOps.Vexer.Core.csproj" /> | ||||||
|  |   </ItemGroup> | ||||||
|  | </Project> | ||||||
| @@ -2,6 +2,6 @@ If you are working on this file you need to read docs/ARCHITECTURE_VEXER.md and | |||||||
| # TASKS | # TASKS | ||||||
| | Task | Owner(s) | Depends on | Notes | | | Task | Owner(s) | Depends on | Notes | | ||||||
| |---|---|---|---| | |---|---|---|---| | ||||||
| |VEXER-FMT-OPENVEX-01-001 – OpenVEX normalizer|Team Vexer Formats|VEXER-CORE-01-001|TODO – Implement OpenVEX parser covering statements, products, and `status/justification` mapping with provenance metadata.| | |VEXER-FMT-OPENVEX-01-001 – OpenVEX normalizer|Team Vexer Formats|VEXER-CORE-01-001|**DONE (2025-10-17)** – OpenVEX normalizer parses statements/products, maps status/justification, and surfaces provenance metadata; coverage in `OpenVexNormalizerTests`.| | ||||||
| |VEXER-FMT-OPENVEX-01-002 – Statement merge utilities|Team Vexer Formats|VEXER-FMT-OPENVEX-01-001|TODO – Add reducers merging multiple OpenVEX statements, resolving conflicts deterministically, and emitting policy diagnostics.| | |VEXER-FMT-OPENVEX-01-002 – Statement merge utilities|Team Vexer Formats|VEXER-FMT-OPENVEX-01-001|TODO – Add reducers merging multiple OpenVEX statements, resolving conflicts deterministically, and emitting policy diagnostics.| | ||||||
| |VEXER-FMT-OPENVEX-01-003 – OpenVEX export writer|Team Vexer Formats|VEXER-EXPORT-01-001, VEXER-FMT-OPENVEX-01-001|TODO – Provide export serializer generating canonical OpenVEX documents with optional SBOM references and hash-stable ordering.| | |VEXER-FMT-OPENVEX-01-003 – OpenVEX export writer|Team Vexer Formats|VEXER-EXPORT-01-001, VEXER-FMT-OPENVEX-01-001|TODO – Provide export serializer generating canonical OpenVEX documents with optional SBOM references and hash-stable ordering.| | ||||||
|   | |||||||
| @@ -1,5 +1,6 @@ | |||||||
| using System; | using System; | ||||||
| using System.Collections.Generic; | using System.Collections.Generic; | ||||||
|  | using System.Collections.Immutable; | ||||||
| using System.Threading; | using System.Threading; | ||||||
| using System.Threading.Tasks; | using System.Threading.Tasks; | ||||||
| using StellaOps.Vexer.Core; | using StellaOps.Vexer.Core; | ||||||
| @@ -24,6 +25,18 @@ public interface IVexConsensusStore | |||||||
|     ValueTask SaveAsync(VexConsensus consensus, CancellationToken cancellationToken); |     ValueTask SaveAsync(VexConsensus consensus, CancellationToken cancellationToken); | ||||||
| } | } | ||||||
|  |  | ||||||
|  | public sealed record VexConnectorState( | ||||||
|  |     string ConnectorId, | ||||||
|  |     DateTimeOffset? LastUpdated, | ||||||
|  |     ImmutableArray<string> DocumentDigests); | ||||||
|  |  | ||||||
|  | public interface IVexConnectorStateRepository | ||||||
|  | { | ||||||
|  |     ValueTask<VexConnectorState?> GetAsync(string connectorId, CancellationToken cancellationToken); | ||||||
|  |  | ||||||
|  |     ValueTask SaveAsync(VexConnectorState state, CancellationToken cancellationToken); | ||||||
|  | } | ||||||
|  |  | ||||||
| public interface IVexCacheIndex | public interface IVexCacheIndex | ||||||
| { | { | ||||||
|     ValueTask<VexCacheEntry?> FindAsync(VexQuerySignature signature, VexExportFormat format, CancellationToken cancellationToken); |     ValueTask<VexCacheEntry?> FindAsync(VexQuerySignature signature, VexExportFormat format, CancellationToken cancellationToken); | ||||||
|   | |||||||
| @@ -0,0 +1,55 @@ | |||||||
|  | using System; | ||||||
|  | using System.Collections.Immutable; | ||||||
|  | using System.Linq; | ||||||
|  | using System.Threading; | ||||||
|  | using System.Threading.Tasks; | ||||||
|  | using MongoDB.Driver; | ||||||
|  |  | ||||||
|  | namespace StellaOps.Vexer.Storage.Mongo; | ||||||
|  |  | ||||||
|  | public sealed class MongoVexConnectorStateRepository : IVexConnectorStateRepository | ||||||
|  | { | ||||||
|  |     private readonly IMongoCollection<VexConnectorStateDocument> _collection; | ||||||
|  |  | ||||||
|  |     public MongoVexConnectorStateRepository(IMongoDatabase database) | ||||||
|  |     { | ||||||
|  |         ArgumentNullException.ThrowIfNull(database); | ||||||
|  |         VexMongoMappingRegistry.Register(); | ||||||
|  |         _collection = database.GetCollection<VexConnectorStateDocument>(VexMongoCollectionNames.ConnectorState); | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     public async ValueTask<VexConnectorState?> GetAsync(string connectorId, CancellationToken cancellationToken) | ||||||
|  |     { | ||||||
|  |         ArgumentException.ThrowIfNullOrWhiteSpace(connectorId); | ||||||
|  |  | ||||||
|  |         var filter = Builders<VexConnectorStateDocument>.Filter.Eq(x => x.ConnectorId, connectorId.Trim()); | ||||||
|  |         var document = await _collection.Find(filter).FirstOrDefaultAsync(cancellationToken).ConfigureAwait(false); | ||||||
|  |         return document?.ToRecord(); | ||||||
|  |     } | ||||||
|  |  | ||||||
|  |     public async ValueTask SaveAsync(VexConnectorState state, CancellationToken cancellationToken) | ||||||
|  |     { | ||||||
|  |         ArgumentNullException.ThrowIfNull(state); | ||||||
|  |  | ||||||
|  |         var document = VexConnectorStateDocument.FromRecord(state.WithNormalizedDigests()); | ||||||
|  |         var filter = Builders<VexConnectorStateDocument>.Filter.Eq(x => x.ConnectorId, document.ConnectorId); | ||||||
|  |         await _collection.ReplaceOneAsync(filter, document, new ReplaceOptions { IsUpsert = true }, cancellationToken).ConfigureAwait(false); | ||||||
|  |     } | ||||||
|  | } | ||||||
|  |  | ||||||
|  | internal static class VexConnectorStateExtensions | ||||||
|  | { | ||||||
|  |     private const int MaxDigestHistory = 200; | ||||||
|  |  | ||||||
|  |     public static VexConnectorState WithNormalizedDigests(this VexConnectorState state) | ||||||
|  |     { | ||||||
|  |         var digests = state.DocumentDigests; | ||||||
|  |         if (digests.Length <= MaxDigestHistory) | ||||||
|  |         { | ||||||
|  |             return state; | ||||||
|  |         } | ||||||
|  |  | ||||||
|  |         var trimmed = digests.Skip(digests.Length - MaxDigestHistory).ToImmutableArray(); | ||||||
|  |         return state with { DocumentDigests = trimmed }; | ||||||
|  |     } | ||||||
|  | } | ||||||
| @@ -16,6 +16,7 @@ public static class VexMongoServiceCollectionExtensions | |||||||
|         services.AddSingleton<IVexConsensusStore, MongoVexConsensusStore>(); |         services.AddSingleton<IVexConsensusStore, MongoVexConsensusStore>(); | ||||||
|         services.AddSingleton<IVexCacheIndex, MongoVexCacheIndex>(); |         services.AddSingleton<IVexCacheIndex, MongoVexCacheIndex>(); | ||||||
|         services.AddSingleton<IVexCacheMaintenance, MongoVexCacheMaintenance>(); |         services.AddSingleton<IVexCacheMaintenance, MongoVexCacheMaintenance>(); | ||||||
|  |         services.AddSingleton<IVexConnectorStateRepository, MongoVexConnectorStateRepository>(); | ||||||
|         services.AddSingleton<IVexMongoMigration, VexInitialIndexMigration>(); |         services.AddSingleton<IVexMongoMigration, VexInitialIndexMigration>(); | ||||||
|         services.AddSingleton<VexMongoMigrationRunner>(); |         services.AddSingleton<VexMongoMigrationRunner>(); | ||||||
|         services.AddHostedService<VexMongoMigrationHostedService>(); |         services.AddHostedService<VexMongoMigrationHostedService>(); | ||||||
|   | |||||||
| @@ -39,6 +39,7 @@ public static class VexMongoMappingRegistry | |||||||
|         RegisterClassMap<VexConsensusConflictDocument>(); |         RegisterClassMap<VexConsensusConflictDocument>(); | ||||||
|         RegisterClassMap<VexConfidenceDocument>(); |         RegisterClassMap<VexConfidenceDocument>(); | ||||||
|         RegisterClassMap<VexCacheEntryRecord>(); |         RegisterClassMap<VexCacheEntryRecord>(); | ||||||
|  |         RegisterClassMap<VexConnectorStateDocument>(); | ||||||
|     } |     } | ||||||
|  |  | ||||||
|     private static void RegisterClassMap<TDocument>() |     private static void RegisterClassMap<TDocument>() | ||||||
| @@ -66,4 +67,5 @@ public static class VexMongoCollectionNames | |||||||
|     public const string Consensus = "vex.consensus"; |     public const string Consensus = "vex.consensus"; | ||||||
|     public const string Exports = "vex.exports"; |     public const string Exports = "vex.exports"; | ||||||
|     public const string Cache = "vex.cache"; |     public const string Cache = "vex.cache"; | ||||||
|  |     public const string ConnectorState = "vex.connector_state"; | ||||||
| } | } | ||||||
|   | |||||||
| @@ -570,3 +570,35 @@ internal sealed class VexCacheEntryRecord | |||||||
|             expires); |             expires); | ||||||
|     } |     } | ||||||
| } | } | ||||||
|  |  | ||||||
|  | [BsonIgnoreExtraElements] | ||||||
|  | internal sealed class VexConnectorStateDocument | ||||||
|  | { | ||||||
|  |     [BsonId] | ||||||
|  |     public string ConnectorId { get; set; } = default!; | ||||||
|  |  | ||||||
|  |     public DateTime? LastUpdated { get; set; } | ||||||
|  |         = null; | ||||||
|  |  | ||||||
|  |     public List<string> DocumentDigests { get; set; } = new(); | ||||||
|  |  | ||||||
|  |     public static VexConnectorStateDocument FromRecord(VexConnectorState state) | ||||||
|  |         => new() | ||||||
|  |         { | ||||||
|  |             ConnectorId = state.ConnectorId, | ||||||
|  |             LastUpdated = state.LastUpdated?.UtcDateTime, | ||||||
|  |             DocumentDigests = state.DocumentDigests.ToList(), | ||||||
|  |         }; | ||||||
|  |  | ||||||
|  |     public VexConnectorState ToRecord() | ||||||
|  |     { | ||||||
|  |         var lastUpdated = LastUpdated.HasValue | ||||||
|  |             ? new DateTimeOffset(DateTime.SpecifyKind(LastUpdated.Value, DateTimeKind.Utc)) | ||||||
|  |             : (DateTimeOffset?)null; | ||||||
|  |  | ||||||
|  |         return new VexConnectorState( | ||||||
|  |             ConnectorId, | ||||||
|  |             lastUpdated, | ||||||
|  |             DocumentDigests.ToImmutableArray()); | ||||||
|  |     } | ||||||
|  | } | ||||||
|   | |||||||
| @@ -7,6 +7,7 @@ using StellaOps.Vexer.Attestation.Transparency; | |||||||
| using StellaOps.Vexer.ArtifactStores.S3.Extensions; | using StellaOps.Vexer.ArtifactStores.S3.Extensions; | ||||||
| using StellaOps.Vexer.Export; | using StellaOps.Vexer.Export; | ||||||
| using StellaOps.Vexer.Storage.Mongo; | using StellaOps.Vexer.Storage.Mongo; | ||||||
|  | using StellaOps.Vexer.Connectors.RedHat.CSAF.DependencyInjection; | ||||||
|  |  | ||||||
| var builder = WebApplication.CreateBuilder(args); | var builder = WebApplication.CreateBuilder(args); | ||||||
| var configuration = builder.Configuration; | var configuration = builder.Configuration; | ||||||
| @@ -21,6 +22,7 @@ services.AddVexExportEngine(); | |||||||
| services.AddVexExportCacheServices(); | services.AddVexExportCacheServices(); | ||||||
| services.AddVexAttestation(); | services.AddVexAttestation(); | ||||||
| services.Configure<VexAttestationClientOptions>(configuration.GetSection("Vexer:Attestation:Client")); | services.Configure<VexAttestationClientOptions>(configuration.GetSection("Vexer:Attestation:Client")); | ||||||
|  | services.AddRedHatCsafConnector(); | ||||||
|  |  | ||||||
| var rekorSection = configuration.GetSection("Vexer:Attestation:Rekor"); | var rekorSection = configuration.GetSection("Vexer:Attestation:Rekor"); | ||||||
| if (rekorSection.Exists()) | if (rekorSection.Exists()) | ||||||
|   | |||||||
| @@ -2,7 +2,7 @@ If you are working on this file you need to read docs/ARCHITECTURE_VEXER.md and | |||||||
| # TASKS | # TASKS | ||||||
| | Task | Owner(s) | Depends on | Notes | | | Task | Owner(s) | Depends on | Notes | | ||||||
| |---|---|---|---| | |---|---|---|---| | ||||||
| |VEXER-WEB-01-001 – Minimal API bootstrap & DI|Team Vexer WebService|VEXER-CORE-01-003, VEXER-STORAGE-01-003|TODO – Scaffold ASP.NET host, register connectors/normalizers via plugin loader, bind policy/storage/attestation services, and expose `/vexer/status`.| | |VEXER-WEB-01-001 – Minimal API bootstrap & DI|Team Vexer WebService|VEXER-CORE-01-003, VEXER-STORAGE-01-003|**DONE (2025-10-17)** – Minimal API host composes storage/export/attestation/artifact stores, binds Mongo/attestation options, and exposes `/vexer/status` + health endpoints with regression coverage in `StatusEndpointTests`.| | ||||||
| |VEXER-WEB-01-002 – Ingest & reconcile endpoints|Team Vexer WebService|VEXER-WEB-01-001|TODO – Implement `/vexer/init`, `/vexer/ingest/run`, `/vexer/ingest/resume`, `/vexer/reconcile` with token scope enforcement and structured run telemetry.| | |VEXER-WEB-01-002 – Ingest & reconcile endpoints|Team Vexer WebService|VEXER-WEB-01-001|TODO – Implement `/vexer/init`, `/vexer/ingest/run`, `/vexer/ingest/resume`, `/vexer/reconcile` with token scope enforcement and structured run telemetry.| | ||||||
| |VEXER-WEB-01-003 – Export & verify endpoints|Team Vexer WebService|VEXER-WEB-01-001, VEXER-EXPORT-01-001, VEXER-ATTEST-01-001|TODO – Add `/vexer/export`, `/vexer/export/{id}`, `/vexer/export/{id}/download`, `/vexer/verify`, returning artifact + attestation metadata with cache awareness.| | |VEXER-WEB-01-003 – Export & verify endpoints|Team Vexer WebService|VEXER-WEB-01-001, VEXER-EXPORT-01-001, VEXER-ATTEST-01-001|TODO – Add `/vexer/export`, `/vexer/export/{id}`, `/vexer/export/{id}/download`, `/vexer/verify`, returning artifact + attestation metadata with cache awareness.| | ||||||
| |VEXER-WEB-01-004 – Resolve API & signed responses|Team Vexer WebService|VEXER-WEB-01-001, VEXER-ATTEST-01-002|TODO – Deliver `/vexer/resolve` (subject/context), return consensus + score envelopes, attach cosign/Rekor metadata, and document auth + rate guardrails.| | |VEXER-WEB-01-004 – Resolve API & signed responses|Team Vexer WebService|VEXER-WEB-01-001, VEXER-ATTEST-01-002|TODO – Deliver `/vexer/resolve` (subject/context), return consensus + score envelopes, attach cosign/Rekor metadata, and document auth + rate guardrails.| | ||||||
|   | |||||||
Some files were not shown because too many files have changed in this diff Show More
		Reference in New Issue
	
	Block a user