Introduce Vexer platform scaffolding and enrich Concelier merge
This commit is contained in:
root
@@ -234,6 +234,11 @@ See `docs/dev/32_AUTH_CLIENT_GUIDE.md` for recommended profiles (online vs. air-
|
||||
|
||||
When running on an interactive terminal without explicit override flags, the CLI uses Spectre.Console prompts to let you choose per-run ORAS/offline bundle behaviour.
|
||||
|
||||
**Startup diagnostics**
|
||||
|
||||
- `stellaops-cli` now loads Authority plug-in manifests during startup (respecting `Authority:Plugins:*`) and surfaces analyzer warnings when a plug-in weakens the baseline password policy (minimum length **12** and all character classes required).
|
||||
- Follow the log entry’s config path and raise `passwordPolicy.minimumLength` to at least 12 while keeping `requireUppercase`, `requireLowercase`, `requireDigit`, and `requireSymbol` set to `true` to clear the warning; weakened overrides are treated as actionable security deviations.
|
||||
|
||||
**Logging & exit codes**
|
||||
|
||||
- Structured logging via `Microsoft.Extensions.Logging` with single-line console output (timestamps in UTC).
|
||||
|
||||
72
docs/ARCHITECTURE_VEXER.md
Normal file
72
docs/ARCHITECTURE_VEXER.md
Normal file
@@ -0,0 +1,72 @@
|
||||
# StellaOps Vexer Architecture
|
||||
|
||||
Vexer is StellaOps' vulnerability-exploitability (VEX) platform. It ingests VEX statements from multiple providers, normalizes them into canonical claims, projects trust-weighted consensus, and delivers deterministic export artifacts with signed attestations. This document summarizes the target architecture and how the current implementation maps to those goals.
|
||||
|
||||
## 1. Solution topology
|
||||
|
||||
| Module | Purpose | Key contracts |
|
||||
| --- | --- | --- |
|
||||
| `StellaOps.Vexer.Core` | Domain models (`VexClaim`, `VexConsensus`, `VexExportManifest`), deterministic JSON helpers, shared abstractions (connectors, exporters, attestations). | `IVexConnector`, `IVexExporter`, `IVexAttestationClient`, `VexCanonicalJsonSerializer` |
|
||||
| `StellaOps.Vexer.Policy` | Loads operator policy (weights, overrides, justification gates) and exposes snapshots for consensus. | `IVexPolicyProvider`, `IVexPolicyEvaluator`, `VexPolicyOptions` |
|
||||
| `StellaOps.Vexer.Storage.Mongo` | Persistence layer for providers, raw docs, claims, consensus, exports, cache. | `IVexRawStore`, `IVexExportStore`, Mongo class maps |
|
||||
| `StellaOps.Vexer.Export` | Orchestrates export pipeline (query signature → cache lookup → snapshot build → attestation handoff). | `IExportEngine`, `IVexExportDataSource` |
|
||||
| `StellaOps.Vexer.Attestation` *(planned)* | Builds in-toto/DSSE envelopes and communicates with Sigstore/Rekor. | `IVexAttestationClient` |
|
||||
| `StellaOps.Vexer.WebService` *(planned)* | Minimal API host for ingest/export endpoints. | `AddVexerWebService()` |
|
||||
| `StellaOps.Vexer.Worker` *(planned)* | Background executor for scheduled pulls, verification, reconciliation, cache GC. | Hosted services |
|
||||
|
||||
All modules target .NET 10 preview and follow the same deterministic logging and serialization conventions as Feedser.
|
||||
|
||||
## 2. Data model
|
||||
|
||||
MongoDB acts as the canonical store; collections (with logical responsibilities) are:
|
||||
|
||||
- `vex.providers` – provider metadata, trust tiers, discovery endpoints, and cosign/PGP details.
|
||||
- `vex.raw` – immutable raw documents (CSAF, CycloneDX VEX, OpenVEX, OCI attestations) with digests, retrieval metadata, and signature state.
|
||||
- `vex.claims` – normalized `VexClaim` rows; deduped on `(providerId, vulnId, productKey, docDigest)`.
|
||||
- `vex.consensus` – consensus projections per `(vulnId, productKey)` capturing rollup status, source weights, conflicts, and policy revision.
|
||||
- `vex.exports` – export manifests containing artifact digests, cache metadata, and attestation pointers.
|
||||
- `vex.cache` – index from `querySignature`/`format` to export digest for fast reuse.
|
||||
|
||||
GridFS is used for large raw payloads when necessary, and artifact stores (S3/MinIO/file) hold serialized exports referenced by `vex.exports`.
|
||||
|
||||
## 3. Ingestion and reconciliation flow
|
||||
|
||||
1. **Discovery & configuration** – connectors load YAML/JSON settings via `StellaOps.Vexer.Policy` (provider enablement, trust overrides).
|
||||
2. **Fetch** – each `IVexConnector` pulls source windows, writing raw documents through `IVexRawDocumentSink` (Mongo-backed) with dedupe on digest.
|
||||
3. **Verification** – signatures/attestations validated through `IVexSignatureVerifier`; metadata stored alongside raw records.
|
||||
4. **Normalization** – format-specific `IVexNormalizer` instances translate raw payloads to canonical `VexClaim` batches.
|
||||
5. **Consensus** – `VexConsensusResolver` (Core) consumes claims with policy weights supplied by `IVexPolicyEvaluator`, producing deterministic consensus entries and conflict annotations.
|
||||
6. **Export** – query requests pass through `VexExportEngine`, generating `VexExportManifest` instances, caching by `VexQuerySignature`, and emitting artifacts for attestation/signature.
|
||||
7. **Attestation & transparency** *(planned)* – `IVexAttestationClient` signs exports (in-toto/DSSE) and records bundles in Rekor v2.
|
||||
|
||||
The Worker coordinates the long-running steps (fetch/verify/normalize/export), while the WebService exposes synchronous APIs for on-demand operations and status lookups.
|
||||
|
||||
## 4. Policy semantics
|
||||
|
||||
- **Weights** – default tiers (`vendor=1.0`, `distro=0.9`, `platform=0.7`, `hub=0.5`, `attestation=0.6`) loaded via `VexPolicyOptions.Weights`, with per-provider overrides.
|
||||
- **Justification gates** – policy enforces that `not_affected` claims must provide a recognized justification; rejected claims are preserved as conflicts with reason metadata.
|
||||
- **Diagnostics** – policy snapshots carry structured issues for misconfigurations (out-of-range weights, empty overrides) surfaced to operators via logs and future CLI/Web endpoints.
|
||||
|
||||
Policy snapshots are immutable and versioned so consensus records capture the policy revision used during evaluation.
|
||||
|
||||
## 5. Determinism & caching
|
||||
|
||||
- JSON serialization uses `VexCanonicalJsonSerializer`, enforcing property ordering and camelCase naming for reproducible snapshots and test fixtures.
|
||||
- `VexQuerySignature` produces canonical filter/order strings and SHA-256 digests, enabling cache keys shared across services.
|
||||
- Export manifests reuse cached artifacts when the same signature/format is requested unless `ForceRefresh` is explicitly set.
|
||||
|
||||
## 6. Observability & offline posture
|
||||
|
||||
- Structured logs (`ILogger`) capture correlation IDs, query signatures, provider IDs, and policy revisions. Metrics/OTel instrumentation will mirror Feedser once tracing hooks are added.
|
||||
- Offline-first: connectors, policy bundles, and export caches can be bundled inside the Offline Kit; no mandatory outbound calls beyond configured provider allowlists.
|
||||
- Operator tooling (CLI/WebService) will expose diagnostics (policy issues, verification failures, cache status) so air-gapped deployments maintain visibility without external telemetry.
|
||||
|
||||
## 7. Roadmap highlights
|
||||
|
||||
- Complete storage mappings for providers/consensus/cache and add migrations/indices per collection.
|
||||
- Implement Rekor/in-toto attestation clients and wire export engine to produce signed bundles.
|
||||
- Build WebService endpoints (`/vexer/status`, `/vexer/claims`, `/vexer/exports`) plus CLI verbs mirroring Feedser patterns.
|
||||
- Provide CSAF, CycloneDX VEX, and OpenVEX normalizers along with vendor-specific connectors (Red Hat, Cisco, SUSE, MSRC, Oracle, Ubuntu, OCI attestation).
|
||||
- Extend policy diagnostics with schema validation, change tracking, and operator-facing diff reports.
|
||||
|
||||
This architecture keeps Vexer aligned with StellaOps' deterministic, offline-operable design while layering VEX-specific consensus and attestation capabilities on top of the Feedser foundations.
|
||||
@@ -42,6 +42,10 @@ Legend: ✅ complete, ⚠️ in progress/partial, ❌ not started.
|
||||
- Merge now emits `feedser.merge.normalized_rules` (tags: `package_type`, `scheme`) and `feedser.merge.normalized_rules_missing` (tags: `package_type`). Track these counters to confirm normalized arrays land as connectors roll out.
|
||||
- Expect `normalized_rules_missing` to trend toward zero as each connector flips on normalized output. Investigate any sustained counts by checking the corresponding module `TASKS.md`.
|
||||
|
||||
## Implementation tips
|
||||
|
||||
- When a connector only needs to populate `AffectedPackage.NormalizedVersions` (without reusing range primitives), call `SemVerRangeRuleBuilder.BuildNormalizedRules(rawRange, patchedVersion, note)` to project the normalized rule list directly. This avoids re-wrapping `SemVerRangeBuildResult` instances and keeps provenance notes consistent with the shared builder.
|
||||
|
||||
## How to use this dashboard
|
||||
|
||||
1. Before opening a connector PR, update the module `TASKS.md` entry and drop a short bullet here (status + timestamp).
|
||||
|
||||
Reference in New Issue
Block a user