docs consolidation, big sln build fixes, new advisories and sprints/tasks

2026-01-05 18:37:04 +02:00
parent d0a7b88398
commit d7bdca6d97
175 changed files with 10322 additions and 307 deletions
--- a/docs/modules/vex-hub/README.md
+++ b/docs/modules/vex-hub/README.md
@@ -0,0 +1,59 @@
+# VexHub
+
+**Status:** Implemented
+**Source:** `src/VexHub/`
+**Owner:** VEX Guild
+
+## Purpose
+
+VexHub aggregates, validates, and distributes VEX statements from multiple upstream sources. Provides the canonical VEX distribution layer with deterministic outputs, conflict detection, and offline-friendly operation for internal services and external tools (Trivy/Grype).
+
+## Components
+
+**Services:**
+- `StellaOps.VexHub.WebService` - Minimal API host for distribution endpoints and admin controls
+- `StellaOps.VexHub.Worker` - Background workers for scheduled ingestion and validation (planned)
+
+**Libraries:**
+- Normalization Pipeline - Canonicalizes statements to OpenVEX-compatible structures
+- Validation Pipeline - Schema validation (OpenVEX/CycloneDX/CSAF) and signature checks
+
+## Configuration
+
+Configuration is typically embedded in module settings.
+
+Key settings:
+- PostgreSQL connection (schema: `vexhub`)
+- Ingestion source connectors
+- Validation policies (schema, signature)
+- Export snapshot settings
+- Conflict detection rules
+
+## API Surface
+
+- `GET /api/v1/vex/cve/{cve-id}` - Query VEX statements by CVE
+- `GET /api/v1/vex/package/{purl}` - Query VEX statements by PURL
+- `GET /api/v1/vex/source/{source-id}` - Query statements from specific source
+- `GET /api/v1/vex/export` - Bulk OpenVEX feed (deterministic)
+- `GET /api/v1/vex/index` - VEX index metadata (vex-index.json)
+
+All responses use deterministic ordering: `timestamp DESC`, then `source_id ASC`, then `statement_hash ASC`.
+
+## Dependencies
+
+- PostgreSQL (schema: `vexhub` with tables: `statement`, `provenance`, `conflict`, `export_cursor`)
+- Authority (authentication)
+- IssuerDirectory (issuer trust registry)
+- VexLens (consumes VexHub data for consensus)
+- Excititor (VEX ingestion source)
+
+## Related Documentation
+
+- Architecture: `./architecture.md`
+- VexLens: `../vex-lens/`
+- IssuerDirectory: `../issuer-directory/`
+- Excititor: `../excititor/`
+
+## Current Status
+
+WebService implemented with distribution API. Normalization and validation pipelines in place. Supports conflict detection and deterministic exports. Worker component planned for scheduled ingestion.
--- a/docs/modules/vex-hub/architecture.md
+++ b/docs/modules/vex-hub/architecture.md
@@ -0,0 +1,72 @@
+# VexHub Architecture
+
+> **Scope.** Architecture and operational contract for the VexHub aggregation service that normalizes, validates, and distributes VEX statements with deterministic, offline-friendly outputs.
+
+## 1) Purpose
+VexHub collects VEX statements from multiple upstream sources, validates and normalizes them, detects conflicts, and exposes a distribution API for internal services and external tools (Trivy/Grype). It is the canonical aggregation layer that feeds VexLens trust scoring and Policy Engine decisioning.
+
+## 2) Responsibilities
+- Scheduled ingestion of upstream VEX sources (connectors + mirrored feeds).
+- Canonical normalization to OpenVEX-compatible structures.
+- Validation pipeline (schema + signature/provenance checks).
+- Conflict detection and provenance capture.
+- Distribution API for CVE/PURL/source queries and bulk exports.
+
+Non-goals: policy decisioning (Policy Engine), consensus computation (VexLens), raw ingestion guardrails (Excititor AOC).
+
+## 3) Component Model
+- **VexHub.WebService**: Minimal API host for distribution endpoints and admin controls.
+- **VexHub.Worker**: Background workers for ingestion schedules and validation pipelines.
+- **Normalization Pipeline**: Canonicalizes statements, deduplicates, and links provenance.
+- **Validation Pipeline**: Schema validation (OpenVEX/CycloneDX/CSAF) and signature checks.
+- **Storage**: PostgreSQL schema `vexhub` for normalized statements, provenance, conflicts, and export cursors.
+
+## 4) Data Model (Draft)
+- `vexhub.statement`
+  - `id`, `source_id`, `vuln_id`, `product_key`, `status`, `justification`, `timestamp`, `statement_hash`
+- `vexhub.provenance`
+  - `statement_id`, `issuer`, `signature_valid`, `signature_ref`, `source_uri`, `ingested_at`
+- `vexhub.conflict`
+  - `vuln_id`, `product_key`, `statement_ids[]`, `detected_at`, `reason`
+- `vexhub.export_cursor`
+  - `source_id`, `last_exported_at`, `snapshot_hash`
+
+All tables must include `tenant_id`, UTC timestamps, and deterministic ordering keys.
+
+## 5) API Surface (Draft)
+- `GET /api/v1/vex/cve/{cve-id}`
+- `GET /api/v1/vex/package/{purl}`
+- `GET /api/v1/vex/source/{source-id}`
+- `GET /api/v1/vex/export` (bulk OpenVEX feed)
+- `GET /api/v1/vex/index` (vex-index.json)
+
+Responses are deterministic: stable ordering by `timestamp DESC`, then `source_id ASC`, then `statement_hash ASC`.
+
+## 6) Determinism & Offline Posture
+- Ingestion runs against frozen snapshots where possible; all outputs include `snapshot_hash`.
+- Canonical JSON serialization with stable key ordering.
+- No network egress outside configured connectors (sealed mode supported).
+- Bulk exports are immutable and content-addressed.
+
+## 7) Security & Auth
+- API access requires Authority scopes (`vexhub.read`, `vexhub.admin`).
+- Signature verification follows issuer registry rules; failures are surfaced as metadata, not silent drops.
+- Rate limiting enforced at API gateway and per-client tokens.
+
+## 8) Observability
+- Metrics: `vexhub_ingest_total`, `vexhub_validation_failures_total`, `vexhub_conflicts_total`, `vexhub_export_duration_seconds`.
+- Logs: include `tenant_id`, `source_id`, `statement_hash`, and `trace_id`.
+- Traces: spans for ingestion, normalization, validation, export.
+
+## 9) Integration Points
+- **Excititor**: upstream connectors provide source payloads and trust hints.
+- **VexLens**: consumes normalized statements and provenance for trust scoring and consensus.
+- **Policy Engine**: reads VexLens consensus results; VexHub provides external distribution.
+- **UI**: VEX conflict studio consumes conflict API once available.
+
+## 10) Testing Strategy
+- Unit tests for normalization and validation pipelines.
+- Integration tests with Postgres for ingestion and API outputs.
+- Determinism tests comparing repeated exports with identical inputs.
+
+*Last updated: 2025-12-22.*
--- a/docs/modules/vex-hub/integration-guide.md
+++ b/docs/modules/vex-hub/integration-guide.md
@@ -0,0 +1,447 @@
+# VexHub Integration Guide
+
+> **Scope.** Integration instructions for consuming VEX statements from VexHub with Trivy, Grype, and other vulnerability scanning tools.
+
+## 1) Overview
+
+VexHub provides VEX (Vulnerability Exploitability eXchange) statements in OpenVEX format that can be consumed by vulnerability scanners to suppress false positives and reduce noise in scan results. This guide covers integration with:
+
+- **Trivy** (Aqua Security)
+- **Grype** (Anchore)
+- **Direct API consumption**
+
+## 2) Prerequisites
+
+- VexHub service running and accessible (default: `http://localhost:5200`)
+- Network access from the scanning tool to VexHub
+- (Optional) API key for authenticated access with higher rate limits
+
+## 3) VexHub Endpoints
+
+### Index Manifest
+
+```
+GET /api/v1/vex/index
+```
+
+Returns the VEX index manifest with available sources and statistics:
+
+```json
+{
+  "version": "1.0",
+  "lastUpdated": "2025-12-22T12:00:00Z",
+  "sources": ["redhat-csaf", "cisco-csaf", "ubuntu-csaf"],
+  "totalStatements": 45678,
+  "endpoints": {
+    "byCve": "/api/v1/vex/cve/{cve}",
+    "byPackage": "/api/v1/vex/package/{purl}",
+    "bulk": "/api/v1/vex/export"
+  }
+}
+```
+
+### Bulk Export (OpenVEX)
+
+```
+GET /api/v1/vex/export
+Accept: application/vnd.openvex+json
+```
+
+Returns all VEX statements in OpenVEX format. Supports pagination:
+
+```
+GET /api/v1/vex/export?pageSize=1000&pageToken=abc123
+```
+
+### Query by CVE
+
+```
+GET /api/v1/vex/cve/{cve-id}
+Accept: application/vnd.openvex+json
+```
+
+Example: `GET /api/v1/vex/cve/CVE-2024-1234`
+
+### Query by Package (PURL)
+
+```
+GET /api/v1/vex/package/{purl}
+Accept: application/vnd.openvex+json
+```
+
+Example: `GET /api/v1/vex/package/pkg%3Anpm%2Fexpress%404.17.1`
+
+Note: PURL must be URL-encoded.
+
+## 4) Trivy Integration
+
+### Option A: VEX URL (Recommended)
+
+Trivy 0.48.0+ supports fetching VEX from a URL with the `--vex` flag:
+
+```bash
+# Scan container image with VexHub VEX
+trivy image --vex https://vexhub.example.com/api/v1/vex/export alpine:3.18
+
+# Scan filesystem with VexHub VEX
+trivy fs --vex https://vexhub.example.com/api/v1/vex/export /app
+```
+
+### Option B: Local VEX File
+
+Download VEX statements and use locally:
+
+```bash
+# Download VEX statements
+curl -H "Accept: application/vnd.openvex+json" \
+  https://vexhub.example.com/api/v1/vex/export > vexhub.openvex.json
+
+# Scan with local VEX file
+trivy image --vex vexhub.openvex.json alpine:3.18
+```
+
+### Option C: VEX Repository
+
+Configure Trivy to use VexHub as a VEX repository in `trivy.yaml`:
+
+```yaml
+# ~/.trivy.yaml or ./trivy.yaml
+vex:
+  - repository:
+      url: https://vexhub.example.com/api/v1/vex
+```
+
+### Trivy VEX Filtering Behavior
+
+When a VEX statement matches a vulnerability:
+
+| VEX Status | Trivy Behavior |
+|------------|----------------|
+| `not_affected` | Vulnerability suppressed from results |
+| `fixed` | Vulnerability shown with fix information |
+| `under_investigation` | Vulnerability shown, marked as under investigation |
+| `affected` | Vulnerability shown as confirmed affected |
+
+### Authentication with Trivy
+
+For authenticated access, use environment variables or headers:
+
+```bash
+# Using environment variable
+export TRIVY_VEX_AUTH_HEADER="X-Api-Key: your-api-key-here"
+trivy image --vex https://vexhub.example.com/api/v1/vex/export alpine:3.18
+
+# Or download with authentication
+curl -H "X-Api-Key: your-api-key-here" \
+  -H "Accept: application/vnd.openvex+json" \
+  https://vexhub.example.com/api/v1/vex/export > vexhub.openvex.json
+```
+
+## 5) Grype Integration
+
+### Option A: VEX File
+
+Grype supports VEX via the `--vex` flag (OpenVEX format):
+
+```bash
+# Download VEX statements
+curl -H "Accept: application/vnd.openvex+json" \
+  https://vexhub.example.com/api/v1/vex/export > vexhub.openvex.json
+
+# Scan with VEX
+grype alpine:3.18 --vex vexhub.openvex.json
+```
+
+### Option B: Multiple VEX Files
+
+Grype supports multiple VEX files:
+
+```bash
+# Download VEX by source
+curl "https://vexhub.example.com/api/v1/vex/source/redhat-csaf" > redhat.openvex.json
+curl "https://vexhub.example.com/api/v1/vex/source/ubuntu-csaf" > ubuntu.openvex.json
+
+# Scan with multiple VEX files
+grype alpine:3.18 --vex redhat.openvex.json --vex ubuntu.openvex.json
+```
+
+### Grype VEX Matching
+
+Grype matches VEX statements by:
+1. CVE ID
+2. Product identifier (PURL)
+3. VEX status and justification
+
+When matched, vulnerabilities with `not_affected` status are filtered from results.
+
+### Automated VEX Updates for Grype
+
+Create a script to refresh VEX before scans:
+
+```bash
+#!/bin/bash
+# refresh-vex.sh
+
+VEX_URL="https://vexhub.example.com/api/v1/vex/export"
+VEX_FILE="/var/lib/grype/vexhub.openvex.json"
+API_KEY="${VEXHUB_API_KEY:-}"
+
+HEADERS=(-H "Accept: application/vnd.openvex+json")
+if [ -n "$API_KEY" ]; then
+  HEADERS+=(-H "X-Api-Key: $API_KEY")
+fi
+
+curl -s "${HEADERS[@]}" "$VEX_URL" > "$VEX_FILE.tmp" && \
+  mv "$VEX_FILE.tmp" "$VEX_FILE"
+
+echo "VEX file updated: $(jq '.statements | length' "$VEX_FILE") statements"
+```
+
+## 6) API Authentication
+
+VexHub supports API key authentication for increased rate limits and access control.
+
+### Rate Limits
+
+| Client Type | Rate Limit (per minute) |
+|-------------|-------------------------|
+| Anonymous (by IP) | Configured default (e.g., 60) |
+| Authenticated (API key) | 2x default (e.g., 120) |
+| Custom (per-key config) | As configured |
+
+### Passing API Key
+
+**Header (recommended):**
+```bash
+curl -H "X-Api-Key: your-api-key-here" https://vexhub.example.com/api/v1/vex/export
+```
+
+**Query parameter:**
+```bash
+curl "https://vexhub.example.com/api/v1/vex/export?api_key=your-api-key-here"
+```
+
+### Rate Limit Headers
+
+Responses include rate limit information:
+
+```http
+X-RateLimit-Limit: 120
+X-RateLimit-Remaining: 115
+X-RateLimit-Reset: 1703260800
+```
+
+When rate limited, the response is `429 Too Many Requests` with `Retry-After` header.
+
+## 7) CI/CD Integration
+
+### GitHub Actions
+
+```yaml
+name: Security Scan with VEX
+on: [push, pull_request]
+
+jobs:
+  scan:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Download VEX statements
+        run: |
+          curl -H "Accept: application/vnd.openvex+json" \
+            -H "X-Api-Key: ${{ secrets.VEXHUB_API_KEY }}" \
+            https://vexhub.example.com/api/v1/vex/export > vexhub.openvex.json
+
+      - name: Run Trivy scan
+        uses: aquasecurity/trivy-action@master
+        with:
+          image-ref: 'my-app:${{ github.sha }}'
+          vex: 'vexhub.openvex.json'
+          exit-code: '1'
+          severity: 'CRITICAL,HIGH'
+```
+
+### GitLab CI
+
+```yaml
+security_scan:
+  stage: test
+  image: aquasec/trivy:latest
+  script:
+    - curl -H "Accept: application/vnd.openvex+json"
+        -H "X-Api-Key: $VEXHUB_API_KEY"
+        https://vexhub.example.com/api/v1/vex/export > vexhub.openvex.json
+    - trivy image --vex vexhub.openvex.json --exit-code 1 $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA
+  variables:
+    TRIVY_SEVERITY: "CRITICAL,HIGH"
+```
+
+### Jenkins Pipeline
+
+```groovy
+pipeline {
+    agent any
+    environment {
+        VEXHUB_URL = 'https://vexhub.example.com/api/v1/vex/export'
+    }
+    stages {
+        stage('Download VEX') {
+            steps {
+                withCredentials([string(credentialsId: 'vexhub-api-key', variable: 'API_KEY')]) {
+                    sh '''
+                        curl -H "Accept: application/vnd.openvex+json" \
+                             -H "X-Api-Key: $API_KEY" \
+                             $VEXHUB_URL > vexhub.openvex.json
+                    '''
+                }
+            }
+        }
+        stage('Security Scan') {
+            steps {
+                sh 'trivy image --vex vexhub.openvex.json --exit-code 1 my-app:latest'
+            }
+        }
+    }
+}
+```
+
+## 8) Webhooks for Real-Time Updates
+
+VexHub supports webhooks to notify when new VEX statements are available.
+
+### Subscribing to Updates
+
+```bash
+curl -X POST https://vexhub.example.com/api/v1/webhooks/subscribe \
+  -H "Content-Type: application/json" \
+  -H "X-Api-Key: your-api-key" \
+  -d '{
+    "url": "https://your-service.example.com/webhook",
+    "events": ["vex.statement.created", "vex.statement.updated"],
+    "secret": "your-webhook-secret"
+  }'
+```
+
+### Webhook Payload
+
+```json
+{
+  "event": "vex.statement.created",
+  "timestamp": "2025-12-22T12:00:00Z",
+  "data": {
+    "statementId": "550e8400-e29b-41d4-a716-446655440000",
+    "vulnerabilityId": "CVE-2024-1234",
+    "status": "not_affected",
+    "source": "redhat-csaf"
+  }
+}
+```
+
+### Webhook Signature Verification
+
+Webhooks include HMAC-SHA256 signature in `X-VexHub-Signature` header:
+
+```python
+import hmac
+import hashlib
+
+def verify_webhook(payload: bytes, signature: str, secret: str) -> bool:
+    expected = hmac.new(
+        secret.encode(),
+        payload,
+        hashlib.sha256
+    ).hexdigest()
+    return hmac.compare_digest(f"sha256={expected}", signature)
+```
+
+## 9) Troubleshooting
+
+### Common Issues
+
+**VEX not applied to vulnerabilities:**
+- Verify PURL format matches exactly
+- Check VEX statement `products` field includes your package
+- Ensure VEX document format is valid OpenVEX
+
+**Rate limit exceeded:**
+- Use API key authentication for higher limits
+- Cache VEX locally and refresh periodically
+- Check `Retry-After` header for wait time
+
+**Authentication failures:**
+- Verify API key is correct
+- Check key has required scopes (`vexhub.read`)
+- Ensure key hasn't expired
+
+### Debug Mode
+
+Enable verbose output to troubleshoot:
+
+```bash
+# Trivy
+trivy image --debug --vex https://vexhub.example.com/api/v1/vex/export alpine:3.18
+
+# Grype
+GRYPE_LOG_LEVEL=debug grype alpine:3.18 --vex vexhub.openvex.json
+```
+
+### Validating VEX Format
+
+Verify VEX document is valid:
+
+```bash
+curl -s https://vexhub.example.com/api/v1/vex/export | jq '.["@context"]'
+# Should output: "https://openvex.dev/ns/v0.2.0"
+```
+
+## 10) OpenVEX Format Reference
+
+VexHub exports in OpenVEX format. Key fields:
+
+```json
+{
+  "@context": "https://openvex.dev/ns/v0.2.0",
+  "@id": "https://vexhub.example.com/vex/550e8400",
+  "author": "StellaOps VexHub",
+  "timestamp": "2025-12-22T12:00:00Z",
+  "version": 1,
+  "statements": [
+    {
+      "vulnerability": {
+        "@id": "https://nvd.nist.gov/vuln/detail/CVE-2024-1234",
+        "name": "CVE-2024-1234"
+      },
+      "products": [
+        {
+          "@id": "pkg:npm/express@4.17.1"
+        }
+      ],
+      "status": "not_affected",
+      "justification": "vulnerable_code_not_present",
+      "statement": "The vulnerable code path is not included in this package."
+    }
+  ]
+}
+```
+
+### Status Values
+
+| Status | Description |
+|--------|-------------|
+| `not_affected` | Product not affected by vulnerability |
+| `affected` | Product is affected |
+| `fixed` | Vulnerability has been fixed in this version |
+| `under_investigation` | Impact is being investigated |
+
+### Justification Values (for `not_affected`)
+
+| Justification | Description |
+|---------------|-------------|
+| `component_not_present` | Vulnerable component not in product |
+| `vulnerable_code_not_present` | Vulnerable code path not included |
+| `vulnerable_code_not_in_execute_path` | Code present but not reachable |
+| `vulnerable_code_cannot_be_controlled_by_adversary` | Attack vector not possible |
+| `inline_mitigations_already_exist` | Mitigations prevent exploitation |
+
+*Last updated: 2025-12-22.*