Compare commits

..

8 Commits

Author SHA1 Message Date
root
44ad31591c Restore vendor connector internals and configure offline packages
Some checks failed
Docs CI / lint-and-preview (push) Has been cancelled
2025-10-20 15:34:24 +03:00
root
09b6a28172 Resolve Concelier/Excititor merge conflicts 2025-10-20 14:19:25 +03:00
Vladimir Moushkov
67d581d2e8 up
Some checks failed
Build Test Deploy / build-test (push) Has been cancelled
Build Test Deploy / authority-container (push) Has been cancelled
Build Test Deploy / docs (push) Has been cancelled
Build Test Deploy / deploy (push) Has been cancelled
Docs CI / lint-and-preview (push) Has been cancelled
2025-10-17 19:34:43 +03:00
Vladimir Moushkov
c8c05abb3d up
Some checks failed
Build Test Deploy / build-test (push) Has been cancelled
Build Test Deploy / authority-container (push) Has been cancelled
Build Test Deploy / docs (push) Has been cancelled
Build Test Deploy / deploy (push) Has been cancelled
2025-10-17 19:17:27 +03:00
Vladimir Moushkov
c65061602b commit
Some checks failed
Build Test Deploy / build-test (push) Has been cancelled
Build Test Deploy / authority-container (push) Has been cancelled
Build Test Deploy / docs (push) Has been cancelled
Build Test Deploy / deploy (push) Has been cancelled
Docs CI / lint-and-preview (push) Has been cancelled
2025-10-16 19:44:10 +03:00
root
0d8233dfb4 up
Some checks failed
Build Test Deploy / authority-container (push) Has been cancelled
Build Test Deploy / docs (push) Has been cancelled
Build Test Deploy / deploy (push) Has been cancelled
Build Test Deploy / build-test (push) Has been cancelled
Docs CI / lint-and-preview (push) Has been cancelled
2025-10-15 19:20:13 +03:00
Vladimir Moushkov
8d153522b0 .
Some checks failed
Build Test Deploy / build-test (push) Has been cancelled
Build Test Deploy / authority-container (push) Has been cancelled
Build Test Deploy / docs (push) Has been cancelled
Build Test Deploy / deploy (push) Has been cancelled
Docs CI / lint-and-preview (push) Has been cancelled
2025-10-15 10:05:17 +03:00
Vladimir Moushkov
ea1106ce7c up 2025-10-15 10:03:56 +03:00
1824 changed files with 200676 additions and 199636 deletions

17
NuGet.config Normal file
View File

@@ -0,0 +1,17 @@
<?xml version="1.0" encoding="utf-8"?>
<configuration>
<packageSources>
<clear />
<add key="local" value="local-nuget" />
<add key="mirror" value="https://mirrors.ablera.dev/nuget/nuget-mirror/v3/index.json" />
</packageSources>
<packageSourceMapping>
<packageSource key="local">
<package pattern="Mongo2Go" />
<package pattern="Microsoft.Extensions.Http.Polly" />
</packageSource>
<packageSource key="mirror">
<package pattern="*" />
</packageSource>
</packageSourceMapping>
</configuration>

2
SPRINTS_VEXER.md Normal file
View File

@@ -0,0 +1,2 @@
| Sprint | Theme | Tasks File Path | Status | Type of Specialist | Task ID | Task Description |
| --- | --- | --- | --- | --- | --- | --- |

463
docs/ARCHITECTURE_VEXER.md Normal file
View File

@@ -0,0 +1,463 @@
# component_architecture_vexer.md — **StellaOps Vexer** (2025Q4)
> **Scope.** This document specifies the **Vexer** service: its purpose, trust model, data structures, APIs, plugin contracts, storage schema, normalization/consensus algorithms, performance budgets, testing matrix, and how it integrates with Scanner, Policy, Feedser, and the attestation chain. It is implementationready.
---
## 0) Mission & role in the platform
**Mission.** Convert heterogeneous **VEX** statements (OpenVEX, CSAF VEX, CycloneDX VEX; vendor/distro/platform sources) into **canonical, queryable claims**; compute **deterministic consensus** per *(vuln, product)*; preserve **conflicts with provenance**; publish **stable, attestable exports** that the backend uses to suppress nonexploitable findings, prioritize remaining risk, and explain decisions.
**Boundaries.**
* Vexer **does not** decide PASS/FAIL. It supplies **evidence** (statuses + justifications + provenance weights).
* Vexer preserves **conflicting claims** unchanged; consensus encodes how we would pick, but the raw set is always exportable.
* VEX consumption is **backendonly**: Scanner never applies VEX. The backends **Policy Engine** asks Vexer for status evidence and then decides what to show.
---
## 1) Inputs, outputs & canonical domain
### 1.1 Accepted input formats (ingest)
* **OpenVEX** JSON documents (attested or raw).
* **CSAF VEX** 2.x (vendor PSIRTs and distros commonly publish CSAF).
* **CycloneDX VEX** 1.4+ (standalone VEX or embedded VEX blocks).
* **OCIattached attestations** (VEX statements shipped as OCI referrers) — optional connectors.
All connectors register **source metadata**: provider identity, trust tier, signature expectations (PGP/cosign/PKI), fetch windows, rate limits, and time anchors.
### 1.2 Canonical model (normalized)
Every incoming statement becomes a set of **VexClaim** records:
```
VexClaim
- providerId // 'redhat', 'suse', 'ubuntu', 'github', 'vendorX'
- vulnId // 'CVE-2025-12345', 'GHSA-xxxx', canonicalized
- productKey // canonical product identity (see §2.2)
- status // affected | not_affected | fixed | under_investigation
- justification? // for 'not_affected'/'affected' where provided
- introducedVersion? // semantics per provider (range or exact)
- fixedVersion? // where provided (range or exact)
- lastObserved // timestamp from source or fetch time
- provenance // doc digest, signature status, fetch URI, line/offset anchors
- evidence[] // raw source snippets for explainability
- supersedes? // optional cross-doc chain (docDigest → docDigest)
```
### 1.3 Exports (consumption)
* **VexConsensus** per `(vulnId, productKey)` with:
* `rollupStatus` (after policy weights/justification gates),
* `sources[]` (winning + losing claims with weights & reasons),
* `policyRevisionId` (identifier of the Vexer policy used),
* `consensusDigest` (stable SHA256 over canonical JSON).
* **Raw claims** export for auditing (unchanged, with provenance).
* **Provider snapshots** (per source, last N days) for operator debugging.
* **Index** optimized for backend joins: `(productKey, vulnId) → (status, confidence, sourceSet)`.
All exports are **deterministic**, and (optionally) **attested** via DSSE and logged to Rekor v2.
---
## 2) Identity model — products & joins
### 2.1 Vuln identity
* Accepts **CVE**, **GHSA**, vendor IDs (MSRC, RHSA…), distro IDs (DSA/USN/RHSA…) — normalized to `vulnId` with alias sets.
* **Alias graph** maintained (from Feedser) to map vendor/distro IDs → CVE (primary) and to **GHSA** where applicable.
### 2.2 Product identity (`productKey`)
* **Primary:** `purl` (Package URL).
* **Secondary links:** `cpe`, **OS package NVRA/EVR**, NuGet/Maven/Golang identity, and **OS package name** when purl unavailable.
* **Fallback:** `oci:<registry>/<repo>@<digest>` for imagelevel VEX.
* **Special cases:** kernel modules, firmware, platforms → providerspecific mapping helpers (connector captures providers product taxonomy → canonical `productKey`).
> Vexer does not invent identities. If a provider cannot be mapped to purl/CPE/NVRA deterministically, we keep the native **product string** and mark the claim as **nonjoinable**; the backend will ignore it unless a policy explicitly whitelists that provider mapping.
---
## 3) Storage schema (MongoDB)
Database: `vexer`
### 3.1 Collections
**`vex.providers`**
```
_id: providerId
name, homepage, contact
trustTier: enum {vendor, distro, platform, hub, attestation}
signaturePolicy: { type: pgp|cosign|x509|none, keys[], certs[], cosignKeylessRoots[] }
fetch: { baseUrl, kind: http|oci|file, rateLimit, etagSupport, windowDays }
enabled: bool
createdAt, modifiedAt
```
**`vex.raw`** (immutable raw documents)
```
_id: sha256(doc bytes)
providerId
uri
ingestedAt
contentType
sig: { verified: bool, method: pgp|cosign|x509|none, keyId|certSubject, bundle? }
payload: GridFS pointer (if large)
disposition: kept|replaced|superseded
correlation: { replaces?: sha256, replacedBy?: sha256 }
```
**`vex.claims`** (normalized rows; dedupe on providerId+vulnId+productKey+docDigest)
```
_id
providerId
vulnId
productKey
status
justification?
introducedVersion?
fixedVersion?
lastObserved
docDigest
provenance { uri, line?, pointer?, signatureState }
evidence[] { key, value, locator }
indices:
- {vulnId:1, productKey:1}
- {providerId:1, lastObserved:-1}
- {status:1}
- text index (optional) on evidence.value for debugging
```
**`vex.consensus`** (rollups)
```
_id: sha256(canonical(vulnId, productKey, policyRevision))
vulnId
productKey
rollupStatus
sources[]: [
{ providerId, status, justification?, weight, lastObserved, accepted:bool, reason }
]
policyRevisionId
evaluatedAt
consensusDigest // same as _id
indices:
- {vulnId:1, productKey:1}
- {policyRevisionId:1, evaluatedAt:-1}
```
**`vex.exports`** (manifest of emitted artifacts)
```
_id
querySignature
format: raw|consensus|index
artifactSha256
rekor { uuid, index, url }?
createdAt
policyRevisionId
cacheable: bool
```
**`vex.cache`**
```
querySignature -> exportId (for fast reuse)
ttl, hits
```
**`vex.migrations`**
* ordered migrations applied at bootstrap to ensure indexes.
### 3.2 Indexing strategy
* Hot path queries use exact `(vulnId, productKey)` and timebounded windows; compound indexes cover both.
* Providers list view by `lastObserved` for monitoring staleness.
* `vex.consensus` keyed by `(vulnId, productKey, policyRevision)` for deterministic reuse.
---
## 4) Ingestion pipeline
### 4.1 Connector contract
```csharp
public interface IVexConnector
{
string ProviderId { get; }
Task FetchAsync(VexConnectorContext ctx, CancellationToken ct); // raw docs
Task NormalizeAsync(VexConnectorContext ctx, CancellationToken ct); // raw -> VexClaim[]
}
```
* **Fetch** must implement: window scheduling, conditional GET (ETag/IfModifiedSince), rate limiting, retry/backoff.
* **Normalize** parses the format, validates schema, maps product identities deterministically, emits `VexClaim` records with **provenance**.
### 4.2 Signature verification (per provider)
* **cosign (keyless or keyful)** for OCI referrers or HTTPserved JSON with Sigstore bundles.
* **PGP** (provider keyrings) for distro/vendor feeds that sign docs.
* **x509** (mutual TLS / providerpinned certs) where applicable.
* Signature state is stored on **vex.raw.sig** and copied into **provenance.signatureState** on claims.
> Claims from sources failing signature policy are marked `"signatureState.verified=false"` and **policy** can downweight or ignore them.
### 4.3 Time discipline
* For each doc, prefer **providers document timestamp**; if absent, use fetch time.
* Claims carry `lastObserved` which drives **tiebreaking** within equal weight tiers.
---
## 5) Normalization: product & status semantics
### 5.1 Product mapping
* **purl** first; **cpe** second; OS package NVRA/EVR mapping helpers (distro connectors) produce purls via canonical tables (e.g., rpm→purl:rpm, deb→purl:deb).
* Where a provider publishes **platformlevel** VEX (e.g., “RHEL 9 not affected”), connectors expand to known product inventory rules (e.g., map to sets of packages/components shipped in the platform). Expansion tables are versioned and kept per provider; every expansion emits **evidence** indicating the rule applied.
* If expansion would be speculative, the claim remains **platformscoped** with `productKey="platform:redhat:rhel:9"` and is flagged **nonjoinable**; backend can decide to use platform VEX only when Scanner proves the platform runtime.
### 5.2 Status + justification mapping
* Canonical **status**: `affected | not_affected | fixed | under_investigation`.
* **Justifications** normalized to a controlled vocabulary (CISAaligned), e.g.:
* `component_not_present`
* `vulnerable_code_not_in_execute_path`
* `vulnerable_configuration_unused`
* `inline_mitigation_applied`
* `fix_available` (with `fixedVersion`)
* `under_investigation`
* Providers with freetext justifications are mapped by deterministic tables; raw text preserved as `evidence`.
---
## 6) Consensus algorithm
**Goal:** produce a **stable**, explainable `rollupStatus` per `(vulnId, productKey)` given possibly conflicting claims.
### 6.1 Inputs
* Set **S** of `VexClaim` for the key.
* **Vexer policy snapshot**:
* **weights** per provider tier and per provider overrides.
* **justification gates** (e.g., require justification for `not_affected` to be acceptable).
* **minEvidence** rules (e.g., `not_affected` must come from ≥1 vendor or 2 distros).
* **signature requirements** (e.g., require verified signature for fixed to be considered).
### 6.2 Steps
1. **Filter invalid** claims by signature policy & justification gates → set `S'`.
2. **Score** each claim:
`score = weight(provider) * freshnessFactor(lastObserved)` where freshnessFactor ∈ [0.8, 1.0] for staleness decay (configurable; small effect).
3. **Aggregate** scores per status: `W(status) = Σ score(claims with that status)`.
4. **Pick** `rollupStatus = argmax_status W(status)`.
5. **Tiebreakers** (in order):
* Higher **max single** provider score wins (vendor > distro > platform > hub).
* More **recent** lastObserved wins.
* Deterministic lexicographic order of status (`fixed` > `not_affected` > `under_investigation` > `affected`) as final tiebreaker.
6. **Explain**: mark accepted sources (`accepted=true; reason="weight"`/`"freshness"`), mark rejected sources with explicit `reason` (`"insufficient_justification"`, `"signature_unverified"`, `"lower_weight"`).
> The algorithm is **pure** given S and policy snapshot; result is reproducible and hashed into `consensusDigest`.
---
## 7) Query & export APIs
All endpoints are versioned under `/api/v1/vex`.
### 7.1 Query (online)
```
POST /claims/search
body: { vulnIds?: string[], productKeys?: string[], providers?: string[], since?: timestamp, limit?: int, pageToken?: string }
→ { claims[], nextPageToken? }
POST /consensus/search
body: { vulnIds?: string[], productKeys?: string[], policyRevisionId?: string, since?: timestamp, limit?: int, pageToken?: string }
→ { entries[], nextPageToken? }
POST /resolve
body: { purls: string[], vulnIds: string[], policyRevisionId?: string }
→ { results: [ { vulnId, productKey, rollupStatus, sources[] } ] }
```
### 7.2 Exports (cacheable snapshots)
```
POST /exports
body: { signature: { vulnFilter?, productFilter?, providers?, since? }, format: raw|consensus|index, policyRevisionId?: string, force?: bool }
→ { exportId, artifactSha256, rekor? }
GET /exports/{exportId} → bytes (application/json or binary index)
GET /exports/{exportId}/meta → { signature, policyRevisionId, createdAt, artifactSha256, rekor? }
```
### 7.3 Provider operations
```
GET /providers → provider list & signature policy
POST /providers/{id}/refresh → trigger fetch/normalize window
GET /providers/{id}/status → last fetch, doc counts, signature stats
```
**Auth:** servicetoservice via Authority tokens; operator operations via UI/CLI with RBAC.
---
## 8) Attestation integration
* Exports can be **DSSEsigned** via **Signer** and logged to **Rekor v2** via **Attestor** (optional but recommended for regulated pipelines).
* `vex.exports.rekor` stores `{uuid, index, url}` when present.
* **Predicate type**: `https://stella-ops.org/attestations/vex-export/1` with fields:
* `querySignature`, `policyRevisionId`, `artifactSha256`, `createdAt`.
---
## 9) Configuration (YAML)
```yaml
vexer:
mongo: { uri: "mongodb://mongo/vexer" }
s3:
endpoint: http://minio:9000
bucket: stellaops
policy:
weights:
vendor: 1.0
distro: 0.9
platform: 0.7
hub: 0.5
attestation: 0.6
providerOverrides:
redhat: 1.0
suse: 0.95
requireJustificationForNotAffected: true
signatureRequiredForFixed: true
minEvidence:
not_affected:
vendorOrTwoDistros: true
connectors:
- providerId: redhat
kind: csaf
baseUrl: https://access.redhat.com/security/data/csaf/v2/
signaturePolicy: { type: pgp, keys: [ "…redhat-pgp-key…" ] }
windowDays: 7
- providerId: suse
kind: csaf
baseUrl: https://ftp.suse.com/pub/projects/security/csaf/
signaturePolicy: { type: pgp, keys: [ "…suse-pgp-key…" ] }
- providerId: ubuntu
kind: openvex
baseUrl: https://…/vex/
signaturePolicy: { type: none }
- providerId: vendorX
kind: cyclonedx-vex
ociRef: ghcr.io/vendorx/vex@sha256:…
signaturePolicy: { type: cosign, cosignKeylessRoots: [ "sigstore-root" ] }
```
---
## 10) Security model
* **Input signature verification** enforced per provider policy (PGP, cosign, x509).
* **Connector allowlists**: outbound fetch constrained to configured domains.
* **Tenant isolation**: pertenant DB prefixes or separate DBs; pertenant S3 prefixes; pertenant policies.
* **AuthN/Z**: Authorityissued OpToks; RBAC roles (`vex.read`, `vex.admin`, `vex.export`).
* **No secrets in logs**; deterministic logging contexts include providerId, docDigest, claim keys.
---
## 11) Performance & scale
* **Targets:**
* Normalize 10k VEX claims/minute/core.
* Consensus compute ≤50ms for 1k unique `(vuln, product)` pairs in hot cache.
* Export (consensus) 1M rows in ≤60s on 8 cores with streaming writer.
* **Scaling:**
* WebService handles control APIs; **Worker** background services (same image) execute fetch/normalize in parallel with ratelimits; Mongo writes batched; upserts by natural keys.
* Exports stream straight to S3 (MinIO) with rolling buffers.
* **Caching:**
* `vex.cache` maps query signatures → export; TTL to avoid stampedes; optimistic reuse unless `force`.
---
## 12) Observability
* **Metrics:**
* `vex.ingest.docs_total{provider}`
* `vex.normalize.claims_total{provider}`
* `vex.signature.failures_total{provider,method}`
* `vex.consensus.conflicts_total{vulnId}`
* `vex.exports.bytes{format}` / `vex.exports.latency_seconds`
* **Tracing:** spans for fetch, verify, parse, map, consensus, export.
* **Dashboards:** provider staleness, top conflicting vulns/components, signature posture, export cache hitrate.
---
## 13) Testing matrix
* **Connectors:** golden raw docs → deterministic claims (fixtures per provider/format).
* **Signature policies:** valid/invalid PGP/cosign/x509 samples; ensure rejects are recorded but not accepted.
* **Normalization edge cases:** platformonly claims, freetext justifications, nonpurl products.
* **Consensus:** conflict scenarios across tiers; check tiebreakers; justification gates.
* **Performance:** 1Mrow export timing; memory ceilings; stream correctness.
* **Determinism:** same inputs + policy → identical `consensusDigest` and export bytes.
* **API contract tests:** pagination, filters, RBAC, rate limits.
---
## 14) Integration points
* **Backend Policy Engine** (in Scanner.WebService): calls `POST /resolve` with batched `(purl, vulnId)` pairs to fetch `rollupStatus + sources`.
* **Feedser**: provides alias graph (CVE↔vendor IDs) and may supply VEXadjacent metadata (e.g., KEV flag) for policy escalation.
* **UI**: VEX explorer screens use `/claims/search` and `/consensus/search`; show conflicts & provenance.
* **CLI**: `stellaops vex export --consensus --since 7d --out vex.json` for audits.
---
## 15) Failure modes & fallback
* **Provider unreachable:** stale thresholds trigger warnings; policy can downweight stale providers automatically (freshness factor).
* **Signature outage:** continue to ingest but mark `signatureState.verified=false`; consensus will likely exclude or downweight per policy.
* **Schema drift:** unknown fields are preserved as `evidence`; normalization rejects only on **invalid identity** or **status**.
---
## 16) Rollout plan (incremental)
1. **MVP**: OpenVEX + CSAF connectors for 3 major providers (e.g., Red Hat/SUSE/Ubuntu), normalization + consensus + `/resolve`.
2. **Signature policies**: PGP for distros; cosign for OCI.
3. **Exports + optional attestation**.
4. **CycloneDX VEX** connectors; platform claim expansion tables; UI explorer.
5. **Scale hardening**: export indexes; conflict analytics.
---
## 17) Appendix — canonical JSON (stable ordering)
All exports and consensus entries are serialized via `VexCanonicalJsonSerializer`:
* UTF8 without BOM;
* keys sorted (ASCII);
* arrays sorted by `(providerId, vulnId, productKey, lastObserved)` unless semantic order mandated;
* timestamps in `YYYYMMDDThh:mm:ssZ`;
* no insignificant whitespace.

83
docs/VEXER_SCORRING.md Normal file
View File

@@ -0,0 +1,83 @@
## Status
This document tracks the future-looking risk scoring model for Vexer. The calculation below is not active yet; Sprint 7 work will add the required schema fields, policy controls, and services. Until that ships, Vexer emits consensus statuses without numeric scores.
## Scoring model (target state)
**S = Gate(VEX_status) × W_trust(source) × [Severity_base × (1 + α·KEV + β·EPSS)]**
* **Gate(VEX_status)**: `affected`/`under_investigation` → 1, `not_affected`/`fixed` → 0. A trusted “not affected” or “fixed” still zeroes the score.
* **W_trust(source)**: normalized policy weight (baseline 01). Policies may opt into >1 boosts for signed vendor feeds once Phase 1 closes.
* **Severity_base**: canonical numeric severity from Feedser (CVSS or org-defined scale).
* **KEV flag**: 0/1 boost when CISA Known Exploited Vulnerabilities applies.
* **EPSS**: probability [0,1]; bounded multiplier.
* **α, β**: configurable coefficients (default α=0.25, β=0.5) stored in policy.
Safeguards: freeze boosts when product identity is unknown, clamp outputs ≥0, and log every factor in the audit trail.
## Implementation roadmap
| Phase | Scope | Artifacts |
| --- | --- | --- |
| **Phase 1 Schema foundations** | Extend Vexer consensus/claims and Feedser canonical advisories with severity, KEV, EPSS, and expose α/β + weight ceilings in policy. | Sprint 7 tasks `VEXER-CORE-02-001`, `VEXER-POLICY-02-001`, `VEXER-STORAGE-02-001`, `FEEDCORE-ENGINE-07-001`. |
| **Phase 2 Deterministic score engine** | Implement a scoring component that executes alongside consensus and persists score envelopes with hashes. | Planned task `VEXER-CORE-02-002` (backlog). |
| **Phase 3 Surfacing & enforcement** | Expose scores via WebService/CLI, integrate with Feedser noise priors, and enforce policy-based suppressions. | To be scheduled after Phase 2. |
## Data model (after Phase 1)
```json
{
"vulnerabilityId": "CVE-2025-12345",
"product": "pkg:name@version",
"consensus": {
"status": "affected",
"policyRevisionId": "rev-12",
"policyDigest": "0D9AEC…"
},
"signals": {
"severity": {"scheme": "CVSS:3.1", "score": 7.5},
"kev": true,
"epss": 0.40
},
"policy": {
"weight": 1.15,
"alpha": 0.25,
"beta": 0.5
},
"score": {
"value": 10.8,
"generatedAt": "2025-11-05T14:12:30Z",
"audit": [
"gate:affected",
"weight:1.15",
"severity:7.5",
"kev:1",
"epss:0.40"
]
}
}
```
## Operational guidance
* **Inputs**: Feedser delivers severity/KEV/EPSS via the advisory event log; Vexer connectors load VEX statements. Policy owns trust tiers and coefficients.
* **Processing**: the scoring engine (Phase 2) runs next to consensus, storing results with deterministic hashes so exports and attestations can reference them.
* **Consumption**: WebService/CLI will return consensus plus score; scanners may suppress findings only when policy-authorized VEX gating and signed score envelopes agree.
## Pseudocode (Phase 2 preview)
```python
def risk_score(gate, weight, severity, kev, epss, alpha, beta, freeze_boosts=False):
if gate == 0:
return 0
if freeze_boosts:
kev, epss = 0, 0
boost = 1 + alpha * kev + beta * epss
return max(0, weight * severity * boost)
```
## FAQ
* **Can operators opt out?** Set α=β=0 or keep weights ≤1.0 via policy.
* **What about missing signals?** Treat them as zero and log the omission.
* **When will this ship?** Phase 1 is planned for Sprint 7; later phases depend on connector coverage and attestation delivery.

View File

@@ -0,0 +1,220 @@
# Vexer Connector Packaging Guide
> **Audience:** teams implementing new Vexer provider plugins (CSAF feeds,
> OpenVEX attestations, etc.)
> **Prerequisites:** read `docs/ARCHITECTURE_VEXER.md` and the module
> `AGENTS.md` in `src/StellaOps.Vexer.Connectors.Abstractions/`.
The Vexer connector SDK gives you:
- `VexConnectorBase` deterministic logging, SHA256 helpers, time provider.
- `VexConnectorOptionsBinder` strongly typed YAML/JSON configuration binding.
- `IVexConnectorOptionsValidator<T>` custom validation hooks (offline defaults, auth invariants).
- `VexConnectorDescriptor` & metadata helpers for consistent telemetry.
This guide explains how to package a connector so the Vexer Worker/WebService
can load it via the plugin host.
---
## 1. Project layout
Start from the template under
`docs/dev/templates/vexer-connector/`. It contains:
```
Vexer.MyConnector/
├── src/
│ ├── Vexer.MyConnector.csproj
│ ├── MyConnectorOptions.cs
│ ├── MyConnector.cs
│ └── MyConnectorPlugin.cs
└── manifest/
└── connector.manifest.yaml
```
Key points:
- Target `net10.0`, enable `TreatWarningsAsErrors`, reference the
`StellaOps.Vexer.Connectors.Abstractions` project (or NuGet once published).
- Keep project ID prefix `StellaOps.Vexer.Connectors.<Provider>` so the
plugin loader can discover it with the default search pattern.
### 1.1 csproj snippet
```xml
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>net10.0</TargetFramework>
<Nullable>enable</Nullable>
<ImplicitUsings>enable</ImplicitUsings>
<TreatWarningsAsErrors>true</TreatWarningsAsErrors>
</PropertyGroup>
<ItemGroup>
<ProjectReference Include="..\..\..\src\StellaOps.Vexer.Connectors.Abstractions\StellaOps.Vexer.Connectors.Abstractions.csproj" />
</ItemGroup>
</Project>
```
Adjust the `ProjectReference` for your checkout (or switch to a NuGet package
once published).
---
## 2. Implement the connector
1. **Options model** create an options POCO with data-annotation attributes.
Bind it via `VexConnectorOptionsBinder.Bind<TOptions>` in your connector
constructor or `ValidateAsync`.
2. **Validator** implement `IVexConnectorOptionsValidator<TOptions>` to add
complex checks (e.g., ensure both `clientId` and `clientSecret` are present).
3. **Connector** inherit from `VexConnectorBase`. Implement:
- `ValidateAsync` run binder/validators, log configuration summary.
- `FetchAsync` stream raw documents to `context.RawSink`.
- `NormalizeAsync` convert raw documents into `VexClaimBatch` via
format-specific normalizers (`context.Normalizers`).
4. **Plugin adapter** expose the connector via a plugin entry point so the
host can instantiate it.
### 2.1 Options binding example
```csharp
public sealed class MyConnectorOptions
{
[Required]
[Url]
public string CatalogUri { get; set; } = default!;
[Required]
public string ApiKey { get; set; } = default!;
[Range(1, 64)]
public int MaxParallelRequests { get; set; } = 4;
}
public sealed class MyConnectorOptionsValidator : IVexConnectorOptionsValidator<MyConnectorOptions>
{
public void Validate(VexConnectorDescriptor descriptor, MyConnectorOptions options, IList<string> errors)
{
if (!options.CatalogUri.StartsWith("https://", StringComparison.OrdinalIgnoreCase))
{
errors.Add("CatalogUri must use HTTPS.");
}
}
}
```
Bind inside the connector:
```csharp
private readonly MyConnectorOptions _options;
public MyConnector(VexConnectorDescriptor descriptor, ILogger<MyConnector> logger, TimeProvider timeProvider)
: base(descriptor, logger, timeProvider)
{
// `settings` comes from the orchestrator; validators registered via DI.
_options = VexConnectorOptionsBinder.Bind<MyConnectorOptions>(
descriptor,
VexConnectorSettings.Empty,
validators: new[] { new MyConnectorOptionsValidator() });
}
```
Replace `VexConnectorSettings.Empty` with the actual settings from context
inside `ValidateAsync`.
---
## 3. Plugin adapter & manifest
Create a simple plugin class that implements
`StellaOps.Plugin.IConnectorPlugin`. The Worker/WebService plugin host uses
this contract today.
```csharp
public sealed class MyConnectorPlugin : IConnectorPlugin
{
private static readonly VexConnectorDescriptor Descriptor =
new("vexer:my-provider", VexProviderKind.Vendor, "My Provider VEX");
public string Name => Descriptor.DisplayName;
public bool IsAvailable(IServiceProvider services) => true; // inject feature flags if needed
public IFeedConnector Create(IServiceProvider services)
{
var logger = services.GetRequiredService<ILogger<MyConnector>>();
var timeProvider = services.GetRequiredService<TimeProvider>();
return new MyConnector(Descriptor, logger, timeProvider);
}
}
```
> **Note:** the Vexer Worker currently instantiates connectors through the
> shared `IConnectorPlugin` contract. Once a dedicated Vexer plugin interface
> lands you simply swap the base interface; the descriptor/connector code
> remains unchanged.
Provide a manifest describing the assembly for operational tooling:
```yaml
# manifest/connector.manifest.yaml
id: vexer-my-provider
assembly: StellaOps.Vexer.Connectors.MyProvider.dll
entryPoint: StellaOps.Vexer.Connectors.MyProvider.MyConnectorPlugin
description: >
Official VEX feed for ExampleCorp products (CSAF JSON, daily updates).
tags:
- vexer
- csaf
- vendor
```
Store manifests under `/opt/stella/vexer/plugins/<connector>/manifest/` in
production so the deployment tooling can inventory and verify plugins.
---
## 4. Packaging workflow
1. `dotnet publish -c Release` → copy the published DLLs to
`/opt/stella/vexer/plugins/<Provider>/`.
2. Place `connector.manifest.yaml` next to the binaries.
3. Restart the Vexer Worker or WebService (hot reload not supported yet).
4. Verify logs: `VEX-ConnectorLoader` should list the connector descriptor.
### 4.1 Offline kits
- Add the connector folder (binaries + manifest) to the Offline Kit bundle.
- Include a `settings.sample.yaml` demonstrating offline-friendly defaults.
- Document any external dependencies (e.g., SHA mirrors) in the manifest `notes`
field.
---
## 5. Testing checklist
- Unit tests around options binding & validators.
- Integration tests (future `StellaOps.Vexer.Connectors.Abstractions.Tests`)
verifying deterministic logging scopes:
`logger.BeginScope` should produce `vex.connector.id`, `vex.connector.kind`,
and `vex.connector.operation`.
- Deterministic SHA tests: repeated `CreateRawDocument` calls with identical
content must return the same digest.
---
## 6. Reference template
See `docs/dev/templates/vexer-connector/` for the full quickstart including:
- Sample options class + validator.
- Connector implementation inheriting from `VexConnectorBase`.
- Plugin adapter + manifest.
Copy the directory, rename namespaces/IDs, then iterate on provider-specific
logic.
---
*Last updated: 2025-10-17*

View File

@@ -0,0 +1,8 @@
id: vexer-my-provider
assembly: StellaOps.Vexer.Connectors.MyProvider.dll
entryPoint: StellaOps.Vexer.Connectors.MyProvider.MyConnectorPlugin
description: |
Example connector template. Replace metadata before shipping.
tags:
- vexer
- template

View File

@@ -0,0 +1,72 @@
using System.Collections.Generic;
using System.Collections.Immutable;
using System.Runtime.CompilerServices;
using Microsoft.Extensions.Logging;
using StellaOps.Vexer.Connectors.Abstractions;
using StellaOps.Vexer.Core;
namespace StellaOps.Vexer.Connectors.MyProvider;
public sealed class MyConnector : VexConnectorBase
{
private readonly IEnumerable<IVexConnectorOptionsValidator<MyConnectorOptions>> _validators;
private MyConnectorOptions? _options;
public MyConnector(VexConnectorDescriptor descriptor, ILogger<MyConnector> logger, TimeProvider timeProvider, IEnumerable<IVexConnectorOptionsValidator<MyConnectorOptions>> validators)
: base(descriptor, logger, timeProvider)
{
_validators = validators;
}
public override ValueTask ValidateAsync(VexConnectorSettings settings, CancellationToken cancellationToken)
{
_options = VexConnectorOptionsBinder.Bind(
Descriptor,
settings,
validators: _validators);
LogConnectorEvent(LogLevel.Information, "validate", "MyConnector configuration loaded.",
new Dictionary<string, object?>
{
["catalogUri"] = _options.CatalogUri,
["maxParallelRequests"] = _options.MaxParallelRequests,
});
return ValueTask.CompletedTask;
}
public override IAsyncEnumerable<VexRawDocument> FetchAsync(VexConnectorContext context, CancellationToken cancellationToken)
{
if (_options is null)
{
throw new InvalidOperationException("Connector not validated.");
}
return FetchInternalAsync(context, cancellationToken);
}
private async IAsyncEnumerable<VexRawDocument> FetchInternalAsync(VexConnectorContext context, [EnumeratorCancellation] CancellationToken cancellationToken)
{
LogConnectorEvent(LogLevel.Information, "fetch", "Fetching catalog window...");
// Replace with real HTTP logic.
await Task.Delay(10, cancellationToken);
var metadata = BuildMetadata(builder => builder
.Add("sourceUri", _options!.CatalogUri)
.Add("window", context.Since?.ToString("O") ?? "full"));
yield return CreateRawDocument(
VexDocumentFormat.CsafJson,
new Uri($"{_options.CatalogUri.TrimEnd('/')}/sample.json"),
new byte[] { 0x7B, 0x7D },
metadata);
}
public override ValueTask<VexClaimBatch> NormalizeAsync(VexRawDocument document, CancellationToken cancellationToken)
{
var claims = ImmutableArray<VexClaim>.Empty;
var diagnostics = ImmutableDictionary<string, string>.Empty;
return ValueTask.FromResult(new VexClaimBatch(document, claims, diagnostics));
}
}

View File

@@ -0,0 +1,16 @@
using System.ComponentModel.DataAnnotations;
namespace StellaOps.Vexer.Connectors.MyProvider;
public sealed class MyConnectorOptions
{
[Required]
[Url]
public string CatalogUri { get; set; } = default!;
[Required]
public string ApiKey { get; set; } = default!;
[Range(1, 32)]
public int MaxParallelRequests { get; set; } = 4;
}

View File

@@ -0,0 +1,15 @@
using System.Collections.Generic;
using StellaOps.Vexer.Connectors.Abstractions;
namespace StellaOps.Vexer.Connectors.MyProvider;
public sealed class MyConnectorOptionsValidator : IVexConnectorOptionsValidator<MyConnectorOptions>
{
public void Validate(VexConnectorDescriptor descriptor, MyConnectorOptions options, IList<string> errors)
{
if (!options.CatalogUri.StartsWith("https://", StringComparison.OrdinalIgnoreCase))
{
errors.Add("CatalogUri must use HTTPS.");
}
}
}

View File

@@ -0,0 +1,27 @@
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Logging;
using StellaOps.Plugin;
using StellaOps.Vexer.Connectors.Abstractions;
using StellaOps.Vexer.Core;
namespace StellaOps.Vexer.Connectors.MyProvider;
public sealed class MyConnectorPlugin : IConnectorPlugin
{
private static readonly VexConnectorDescriptor Descriptor = new(
id: "vexer:my-provider",
kind: VexProviderKind.Vendor,
displayName: "My Provider VEX");
public string Name => Descriptor.DisplayName;
public bool IsAvailable(IServiceProvider services) => true;
public IFeedConnector Create(IServiceProvider services)
{
var logger = services.GetRequiredService<ILogger<MyConnector>>();
var timeProvider = services.GetRequiredService<TimeProvider>();
var validators = services.GetServices<IVexConnectorOptionsValidator<MyConnectorOptions>>();
return new MyConnector(Descriptor, logger, timeProvider, validators);
}
}

View File

@@ -0,0 +1,12 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>net10.0</TargetFramework>
<Nullable>enable</Nullable>
<ImplicitUsings>enable</ImplicitUsings>
<TreatWarningsAsErrors>true</TreatWarningsAsErrors>
</PropertyGroup>
<ItemGroup>
<!-- Adjust the relative path when copying this template into a repo -->
<ProjectReference Include="..\..\..\..\src\StellaOps.Vexer.Connectors.Abstractions\StellaOps.Vexer.Connectors.Abstractions.csproj" />
</ItemGroup>
</Project>

Binary file not shown.

View File

@@ -64,12 +64,13 @@ public class StandardClientProvisioningStoreTests
var result = await provisioning.CreateOrUpdateAsync(registration, CancellationToken.None);
Assert.True(result.Succeeded);
var document = Assert.Contains("signer", store.Documents);
Assert.Equal("attestor signer", document.Value.Properties[AuthorityClientMetadataKeys.Audiences]);
Assert.True(store.Documents.TryGetValue("signer", out var document));
Assert.NotNull(document);
Assert.Equal("attestor signer", document!.Properties[AuthorityClientMetadataKeys.Audiences]);
var descriptor = await provisioning.FindByClientIdAsync("signer", CancellationToken.None);
Assert.NotNull(descriptor);
Assert.Equal(new[] { "attestor", "signer" }, descriptor!.Audiences.OrderBy(value => value, StringComparer.Ordinal));
Assert.Equal(new[] { "attestor", "signer" }, descriptor!.AllowedAudiences.OrderBy(value => value, StringComparer.Ordinal));
}
[Fact]
@@ -101,8 +102,9 @@ public class StandardClientProvisioningStoreTests
await provisioning.CreateOrUpdateAsync(registration, CancellationToken.None);
var document = Assert.Contains("mtls-client", store.Documents).Value;
var binding = Assert.Single(document.CertificateBindings);
Assert.True(store.Documents.TryGetValue("mtls-client", out var document));
Assert.NotNull(document);
var binding = Assert.Single(document!.CertificateBindings);
Assert.Equal("AABBCCDD", binding.Thumbprint);
Assert.Equal("01ff", binding.SerialNumber);
Assert.Equal("CN=mtls-client", binding.Subject);

View File

@@ -103,15 +103,9 @@ internal sealed record CertCcCursor(
var results = new List<Guid>(array.Count);
foreach (var element in array)
{
if (element is BsonString bsonString && Guid.TryParse(bsonString.AsString, out var parsed))
if (TryReadGuid(element, out var parsed))
{
results.Add(parsed);
continue;
}
if (element is BsonBinaryData binary && binary.GuidRepresentation != GuidRepresentation.Unspecified)
{
results.Add(binary.ToGuid());
}
}
@@ -148,6 +142,37 @@ internal sealed record CertCcCursor(
.ToArray();
}
private static bool TryReadGuid(BsonValue value, out Guid guid)
{
if (value is BsonString bsonString && Guid.TryParse(bsonString.AsString, out guid))
{
return true;
}
if (value is BsonBinaryData binary)
{
try
{
guid = binary.ToGuid();
return true;
}
catch (FormatException)
{
// ignore and fall back to byte array parsing
}
var bytes = binary.AsByteArray;
if (bytes.Length == 16)
{
guid = new Guid(bytes);
return true;
}
}
guid = default;
return false;
}
private static Guid[] NormalizeGuidSet(IEnumerable<Guid>? ids)
=> ids?.Where(static id => id != Guid.Empty).Distinct().ToArray() ?? EmptyGuidArray;

View File

@@ -33,6 +33,12 @@
>
> Remark (2025-10-13, SEC5.B): Split follow-up into SEC5.B1 (libsodium provider) and SEC5.B2 (CLI verification) after scoping registry integration; work not yet started.
> Remark (2025-10-13, SEC2.B): Coordinated with Authority Core — audit sinks now receive `/token` success/failure events; awaiting host test suite once signing fixture lands.
>
> Remark (2025-10-13, SEC3.B): Pinged Docs & Plugin guilds — rate limit guidance published in `docs/security/rate-limits.md` and flagged for PLG6.DOC copy lift.
>
> Remark (2025-10-13, SEC5.B): Split follow-up into SEC5.B1 (libsodium provider) and SEC5.B2 (CLI verification) after scoping registry integration; work not yet started.
## Notes
- Target Argon2 parameters follow OWASP Cheat Sheet (memory ≈ 19MiB, iterations 2, parallelism 1). Allow overrides via configuration.
- When CORE8 lands, pair with Team 2 to expose request context information required by the rate limiter (client_id enrichment).

View File

@@ -17,6 +17,7 @@ using System.Collections.Immutable;
using System.IO.Abstractions.TestingHelpers;
using Xunit;
using System.Threading;
using MongoDB.Driver;
namespace StellaOps.Excititor.Connectors.Cisco.CSAF.Tests.Connectors;
@@ -159,10 +160,10 @@ public sealed class CiscoCsafConnectorTests
{
public VexConnectorState? CurrentState { get; private set; }
public ValueTask<VexConnectorState?> GetAsync(string connectorId, CancellationToken cancellationToken)
public ValueTask<VexConnectorState?> GetAsync(string connectorId, CancellationToken cancellationToken, IClientSessionHandle? session = null)
=> ValueTask.FromResult(CurrentState);
public ValueTask SaveAsync(VexConnectorState state, CancellationToken cancellationToken)
public ValueTask SaveAsync(VexConnectorState state, CancellationToken cancellationToken, IClientSessionHandle? session = null)
{
CurrentState = state;
return ValueTask.CompletedTask;

View File

@@ -15,6 +15,7 @@ using StellaOps.Excititor.Connectors.MSRC.CSAF.Configuration;
using StellaOps.Excititor.Core;
using StellaOps.Excititor.Storage.Mongo;
using Xunit;
using MongoDB.Driver;
namespace StellaOps.Excititor.Connectors.MSRC.CSAF.Tests.Connectors;
@@ -316,10 +317,10 @@ public sealed class MsrcCsafConnectorTests
{
public VexConnectorState? State { get; private set; }
public ValueTask<VexConnectorState?> GetAsync(string connectorId, CancellationToken cancellationToken)
public ValueTask<VexConnectorState?> GetAsync(string connectorId, CancellationToken cancellationToken, IClientSessionHandle? session = null)
=> ValueTask.FromResult(State);
public ValueTask SaveAsync(VexConnectorState state, CancellationToken cancellationToken)
public ValueTask SaveAsync(VexConnectorState state, CancellationToken cancellationToken, IClientSessionHandle? session = null)
{
State = state;
return ValueTask.CompletedTask;

View File

@@ -330,7 +330,7 @@ public sealed class MsrcCsafConnector : VexConnectorBase
lastError = ex;
LogConnectorEvent(LogLevel.Warning, "retry", $"Retrying MSRC request (attempt {attempt}/{options.MaxRetryAttempts}).", exception: ex);
}
catch (Exception ex)
catch (Exception)
{
response?.Dispose();
throw;
@@ -492,7 +492,7 @@ public sealed class MsrcCsafConnector : VexConnectorBase
return CsafValidationResult.Valid("gzip");
}
using var jsonDocument = JsonDocument.Parse(payload.Span);
using var jsonDocument = JsonDocument.Parse(payload);
return CsafValidationResult.Valid("json");
}
catch (JsonException ex)

View File

@@ -20,6 +20,7 @@ using StellaOps.Excititor.Core;
using StellaOps.Excititor.Storage.Mongo;
using System.IO.Abstractions.TestingHelpers;
using Xunit;
using MongoDB.Driver;
namespace StellaOps.Excititor.Connectors.Oracle.CSAF.Tests.Connectors;
@@ -254,10 +255,10 @@ public sealed class OracleCsafConnectorTests
{
public VexConnectorState? State { get; private set; }
public ValueTask<VexConnectorState?> GetAsync(string connectorId, CancellationToken cancellationToken)
public ValueTask<VexConnectorState?> GetAsync(string connectorId, CancellationToken cancellationToken, IClientSessionHandle? session = null)
=> ValueTask.FromResult(State);
public ValueTask SaveAsync(VexConnectorState state, CancellationToken cancellationToken)
public ValueTask SaveAsync(VexConnectorState state, CancellationToken cancellationToken, IClientSessionHandle? session = null)
{
State = state;
return ValueTask.CompletedTask;

View File

@@ -13,6 +13,7 @@ using StellaOps.Excititor.Connectors.RedHat.CSAF.Configuration;
using StellaOps.Excititor.Connectors.RedHat.CSAF.Metadata;
using StellaOps.Excititor.Core;
using StellaOps.Excititor.Storage.Mongo;
using MongoDB.Driver;
namespace StellaOps.Excititor.Connectors.RedHat.CSAF.Tests.Connectors;
@@ -258,7 +259,7 @@ public sealed class RedHatCsafConnectorTests
{
public VexConnectorState? State { get; private set; }
public ValueTask<VexConnectorState?> GetAsync(string connectorId, CancellationToken cancellationToken)
public ValueTask<VexConnectorState?> GetAsync(string connectorId, CancellationToken cancellationToken, IClientSessionHandle? session = null)
{
if (State is not null && string.Equals(State.ConnectorId, connectorId, StringComparison.OrdinalIgnoreCase))
{
@@ -268,7 +269,7 @@ public sealed class RedHatCsafConnectorTests
return ValueTask.FromResult<VexConnectorState?>(null);
}
public ValueTask SaveAsync(VexConnectorState state, CancellationToken cancellationToken)
public ValueTask SaveAsync(VexConnectorState state, CancellationToken cancellationToken, IClientSessionHandle? session = null)
{
State = state;
return ValueTask.CompletedTask;

View File

@@ -16,7 +16,7 @@ using StellaOps.Excititor.Connectors.SUSE.RancherVEXHub.Metadata;
namespace StellaOps.Excititor.Connectors.SUSE.RancherVEXHub.Events;
internal sealed class RancherHubEventClient
public sealed class RancherHubEventClient
{
private readonly IHttpClientFactory _httpClientFactory;
private readonly RancherHubTokenProvider _tokenProvider;

View File

@@ -3,7 +3,7 @@ using System.Collections.Immutable;
namespace StellaOps.Excititor.Connectors.SUSE.RancherVEXHub.Events;
internal sealed record RancherHubEventRecord(
public sealed record RancherHubEventRecord(
string RawJson,
string? Id,
string? Type,
@@ -13,7 +13,7 @@ internal sealed record RancherHubEventRecord(
string? DocumentDigest,
string? DocumentFormat);
internal sealed record RancherHubEventBatch(
public sealed record RancherHubEventBatch(
string? Cursor,
string? NextCursor,
ImmutableArray<RancherHubEventRecord> Events,

View File

@@ -8,6 +8,7 @@ using System.Security.Cryptography;
using System.Text;
using System.Threading;
using System.Threading.Tasks;
using System.Runtime.CompilerServices;
using Microsoft.Extensions.Logging;
using StellaOps.Excititor.Connectors.Abstractions;
using StellaOps.Excititor.Connectors.SUSE.RancherVEXHub.Authentication;

View File

@@ -8,13 +8,13 @@ using StellaOps.Excititor.Storage.Mongo;
namespace StellaOps.Excititor.Connectors.SUSE.RancherVEXHub.State;
internal sealed record RancherHubCheckpointState(
public sealed record RancherHubCheckpointState(
string? Cursor,
DateTimeOffset? LastPublishedAt,
DateTimeOffset? EffectiveSince,
ImmutableArray<string> Digests);
internal sealed class RancherHubCheckpointManager
public sealed class RancherHubCheckpointManager
{
private const string CheckpointPrefix = "checkpoint:";
private readonly IVexConnectorStateRepository _repository;

View File

@@ -19,6 +19,7 @@ using StellaOps.Excititor.Core;
using StellaOps.Excititor.Storage.Mongo;
using System.IO.Abstractions.TestingHelpers;
using Xunit;
using MongoDB.Driver;
namespace StellaOps.Excititor.Connectors.Ubuntu.CSAF.Tests.Connectors;
@@ -146,7 +147,7 @@ public sealed class UbuntuCsafConnectorTests
private static (string IndexJson, string CatalogJson) CreateTestManifest(Uri advisoryUri, string advisoryId, string timestamp)
{
var indexJson = $$"""
var indexJson = """
{
"generated": "2025-10-18T00:00:00Z",
"channels": [
@@ -159,7 +160,7 @@ public sealed class UbuntuCsafConnectorTests
}
""";
var catalogJson = $$"""
var catalogJson = """
{
"resources": [
{
@@ -274,10 +275,10 @@ public sealed class UbuntuCsafConnectorTests
{
public VexConnectorState? CurrentState { get; private set; }
public ValueTask<VexConnectorState?> GetAsync(string connectorId, CancellationToken cancellationToken)
public ValueTask<VexConnectorState?> GetAsync(string connectorId, CancellationToken cancellationToken, IClientSessionHandle? session = null)
=> ValueTask.FromResult(CurrentState);
public ValueTask SaveAsync(VexConnectorState state, CancellationToken cancellationToken)
public ValueTask SaveAsync(VexConnectorState state, CancellationToken cancellationToken, IClientSessionHandle? session = null)
{
CurrentState = state;
return ValueTask.CompletedTask;

View File

@@ -201,6 +201,8 @@ public sealed class UbuntuCsafConnector : VexConnectorBase
{
var client = _httpClientFactory.CreateClient(UbuntuConnectorOptions.HttpClientName);
HttpResponseMessage? response = null;
List<UbuntuCatalogEntry>? entries = null;
try
{
response = await client.GetAsync(channel.CatalogUri, HttpCompletionOption.ResponseHeadersRead, cancellationToken).ConfigureAwait(false);
@@ -219,6 +221,7 @@ public sealed class UbuntuCsafConnector : VexConnectorBase
yield break;
}
entries = new List<UbuntuCatalogEntry>(resourcesElement.GetArrayLength());
foreach (var resource in resourcesElement.EnumerateArray())
{
var type = GetString(resource, "type");
@@ -247,7 +250,7 @@ public sealed class UbuntuCsafConnector : VexConnectorBase
var version = GetString(resource, "version");
var advisoryId = GetString(resource, "id") ?? ExtractAdvisoryId(documentUri, title);
yield return new UbuntuCatalogEntry(
entries.Add(new UbuntuCatalogEntry(
channel.Name,
advisoryId,
documentUri,
@@ -255,7 +258,7 @@ public sealed class UbuntuCsafConnector : VexConnectorBase
etag,
lastModified,
title,
version);
version));
}
}
catch (Exception ex) when (ex is not OperationCanceledException)
@@ -270,6 +273,17 @@ public sealed class UbuntuCsafConnector : VexConnectorBase
{
response?.Dispose();
}
if (entries is null)
{
yield break;
}
foreach (var entry in entries)
{
cancellationToken.ThrowIfCancellationRequested();
yield return entry;
}
}
private async Task<DownloadResult?> DownloadDocumentAsync(UbuntuCatalogEntry entry, string? knownEtag, CancellationToken cancellationToken)

View File

@@ -174,13 +174,13 @@ public sealed class ExportEngineTests
{
public Dictionary<(string Signature, VexExportFormat Format), bool> RemoveCalls { get; } = new();
public ValueTask<VexCacheEntry?> FindAsync(VexQuerySignature signature, VexExportFormat format, CancellationToken cancellationToken)
public ValueTask<VexCacheEntry?> FindAsync(VexQuerySignature signature, VexExportFormat format, CancellationToken cancellationToken, IClientSessionHandle? session = null)
=> ValueTask.FromResult<VexCacheEntry?>(null);
public ValueTask SaveAsync(VexCacheEntry entry, CancellationToken cancellationToken)
public ValueTask SaveAsync(VexCacheEntry entry, CancellationToken cancellationToken, IClientSessionHandle? session = null)
=> ValueTask.CompletedTask;
public ValueTask RemoveAsync(VexQuerySignature signature, VexExportFormat format, CancellationToken cancellationToken)
public ValueTask RemoveAsync(VexQuerySignature signature, VexExportFormat format, CancellationToken cancellationToken, IClientSessionHandle? session = null)
{
RemoveCalls[(signature.Value, format)] = true;
return ValueTask.CompletedTask;

View File

@@ -53,13 +53,13 @@ public sealed class VexExportCacheServiceTests
public VexExportFormat LastFormat { get; private set; }
public int RemoveCalls { get; private set; }
public ValueTask<VexCacheEntry?> FindAsync(VexQuerySignature signature, VexExportFormat format, CancellationToken cancellationToken)
public ValueTask<VexCacheEntry?> FindAsync(VexQuerySignature signature, VexExportFormat format, CancellationToken cancellationToken, IClientSessionHandle? session = null)
=> ValueTask.FromResult<VexCacheEntry?>(null);
public ValueTask SaveAsync(VexCacheEntry entry, CancellationToken cancellationToken)
public ValueTask SaveAsync(VexCacheEntry entry, CancellationToken cancellationToken, IClientSessionHandle? session = null)
=> ValueTask.CompletedTask;
public ValueTask RemoveAsync(VexQuerySignature signature, VexExportFormat format, CancellationToken cancellationToken)
public ValueTask RemoveAsync(VexQuerySignature signature, VexExportFormat format, CancellationToken cancellationToken, IClientSessionHandle? session = null)
{
LastSignature = signature;
LastFormat = format;

View File

@@ -1,4 +1,6 @@
using System;
using System.Collections.Immutable;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using Microsoft.Extensions.DependencyInjection;
@@ -88,6 +90,10 @@ public sealed class VexExportEngine : IExportEngine
cached.SourceProviders,
fromCache: true,
cached.ConsensusRevision,
cached.PolicyRevisionId,
cached.PolicyDigest,
cached.ConsensusDigest,
cached.ScoreDigest,
cached.Attestation,
cached.SizeBytes);
}
@@ -100,6 +106,7 @@ public sealed class VexExportEngine : IExportEngine
var dataset = await _dataSource.FetchAsync(context.Query, cancellationToken).ConfigureAwait(false);
var exporter = ResolveExporter(context.Format);
var policySnapshot = _policyEvaluator.Snapshot;
var exportRequest = new VexExportRequest(
context.Query,
@@ -168,6 +175,9 @@ public sealed class VexExportEngine : IExportEngine
_logger.LogInformation("Attestation generated for export {ExportId}", exportId);
}
var consensusDigestAddress = TryGetContentAddress(result.Metadata, "consensusDigest");
var scoreDigestAddress = TryGetContentAddress(result.Metadata, "scoreDigest");
var manifest = new VexExportManifest(
exportId,
signature,
@@ -177,7 +187,11 @@ public sealed class VexExportEngine : IExportEngine
dataset.Claims.Length,
dataset.SourceProviders,
fromCache: false,
consensusRevision: _policyEvaluator.Version,
consensusRevision: policySnapshot.Version,
policyRevisionId: policySnapshot.RevisionId,
policyDigest: policySnapshot.Digest,
consensusDigest: consensusDigestAddress,
scoreDigest: scoreDigestAddress,
attestation: attestationMetadata,
sizeBytes: result.BytesWritten);
@@ -192,6 +206,27 @@ public sealed class VexExportEngine : IExportEngine
return manifest;
}
private static VexContentAddress? TryGetContentAddress(IReadOnlyDictionary<string, string> metadata, string key)
{
if (metadata is null)
{
return null;
}
if (!metadata.TryGetValue(key, out var value) || string.IsNullOrWhiteSpace(value))
{
return null;
}
var parts = value.Split(':', 2, StringSplitOptions.TrimEntries);
if (parts.Length != 2)
{
return null;
}
return new VexContentAddress(parts[0], parts[1]);
}
private IVexExporter ResolveExporter(VexExportFormat format)
=> _exporters.TryGetValue(format, out var exporter)
? exporter

View File

@@ -857,6 +857,10 @@ public sealed class CsafNormalizer : IVexNormalizer
ImmutableArray<string> UnsupportedJustifications,
ImmutableArray<string> ConflictingJustifications);
private sealed record CsafJustificationInfo(
string RawValue,
VexJustification? Normalized);
private sealed record CsafClaimEntry(
string VulnerabilityId,
CsafProductInfo Product,

View File

@@ -4,6 +4,7 @@ using System.Linq;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Logging;
using StellaOps.Plugin;
using StellaOps.Excititor.Connectors.Abstractions;
using StellaOps.Excititor.Core;
using StellaOps.Excititor.Storage.Mongo;
@@ -28,41 +29,43 @@ internal sealed class DefaultVexProviderRunner : IVexProviderRunner
_timeProvider = timeProvider ?? throw new ArgumentNullException(nameof(timeProvider));
}
public async ValueTask RunAsync(string providerId, CancellationToken cancellationToken)
public async ValueTask RunAsync(VexWorkerSchedule schedule, CancellationToken cancellationToken)
{
ArgumentException.ThrowIfNullOrWhiteSpace(providerId);
ArgumentNullException.ThrowIfNull(schedule);
ArgumentException.ThrowIfNullOrWhiteSpace(schedule.ProviderId);
using var scope = _serviceProvider.CreateScope();
var availablePlugins = _pluginCatalog.GetAvailableConnectorPlugins(scope.ServiceProvider);
var matched = availablePlugins.FirstOrDefault(plugin =>
string.Equals(plugin.Name, providerId, StringComparison.OrdinalIgnoreCase));
string.Equals(plugin.Name, schedule.ProviderId, StringComparison.OrdinalIgnoreCase));
if (matched is not null)
{
_logger.LogInformation(
"Connector plugin {PluginName} ({ProviderId}) is available. Execution hooks will be added in subsequent tasks.",
matched.Name,
providerId);
schedule.ProviderId);
}
else
{
_logger.LogInformation("No legacy connector plugin registered for provider {ProviderId}; falling back to DI-managed connectors.", providerId);
_logger.LogInformation("No legacy connector plugin registered for provider {ProviderId}; falling back to DI-managed connectors.", schedule.ProviderId);
}
var connectors = scope.ServiceProvider.GetServices<IVexConnector>();
var connector = connectors.FirstOrDefault(c => string.Equals(c.Id, providerId, StringComparison.OrdinalIgnoreCase));
var connector = connectors.FirstOrDefault(c => string.Equals(c.Id, schedule.ProviderId, StringComparison.OrdinalIgnoreCase));
if (connector is null)
{
_logger.LogWarning("No IVexConnector implementation registered for provider {ProviderId}; skipping run.", providerId);
_logger.LogWarning("No IVexConnector implementation registered for provider {ProviderId}; skipping run.", schedule.ProviderId);
return;
}
await ExecuteConnectorAsync(scope.ServiceProvider, connector, cancellationToken).ConfigureAwait(false);
await ExecuteConnectorAsync(scope.ServiceProvider, connector, schedule.Settings, cancellationToken).ConfigureAwait(false);
}
private async Task ExecuteConnectorAsync(IServiceProvider scopeProvider, IVexConnector connector, CancellationToken cancellationToken)
private async Task ExecuteConnectorAsync(IServiceProvider scopeProvider, IVexConnector connector, VexConnectorSettings settings, CancellationToken cancellationToken)
{
var effectiveSettings = settings ?? VexConnectorSettings.Empty;
var rawStore = scopeProvider.GetRequiredService<IVexRawStore>();
var claimStore = scopeProvider.GetRequiredService<IVexClaimStore>();
var providerStore = scopeProvider.GetRequiredService<IVexProviderStore>();
@@ -82,11 +85,11 @@ internal sealed class DefaultVexProviderRunner : IVexProviderRunner
await providerStore.SaveAsync(provider, cancellationToken, session).ConfigureAwait(false);
await connector.ValidateAsync(VexConnectorSettings.Empty, cancellationToken).ConfigureAwait(false);
await connector.ValidateAsync(effectiveSettings, cancellationToken).ConfigureAwait(false);
var context = new VexConnectorContext(
Since: null,
Settings: VexConnectorSettings.Empty,
Settings: effectiveSettings,
RawSink: rawStore,
SignatureVerifier: signatureVerifier,
Normalizers: normalizerRouter,

View File

@@ -1,4 +1,5 @@
using System.Text;
using System.Security.Cryptography;
using StellaOps.Zastava.Core.Contracts;
using StellaOps.Zastava.Core.Hashing;
using StellaOps.Zastava.Core.Serialization;

View File

@@ -105,7 +105,7 @@ public static class ZastavaContractVersions
/// Canonical string representation (schema@vMajor.Minor).
/// </summary>
public override string ToString()
=> $"{Schema}@v{Version.ToString(2, CultureInfo.InvariantCulture)}";
=> $"{Schema}@v{Version.ToString(2)}";
/// <summary>
/// Determines whether a remote contract is compatible with the local definition.

View File

@@ -1,3 +1,12 @@
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text.Encodings.Web;
using System.Text.Json;
using System.Text.Json.Serialization;
using System.Text.Json.Serialization.Metadata;
using StellaOps.Zastava.Core.Contracts;
namespace StellaOps.Zastava.Core.Serialization;
/// <summary>