7503c19b8f
- Implemented comprehensive tests for verdict artifact generation to ensure deterministic outputs across various scenarios, including identical inputs, parallel execution, and change ordering. - Created helper methods for generating sample verdict inputs and computing canonical hashes. - Added tests to validate the stability of canonical hashes, proof spine ordering, and summary statistics. - Introduced a new PowerShell script to update SHA256 sums for files, ensuring accurate hash generation and file integrity checks.
3.5 KiB
3.5 KiB
Request and data flows (detailed)
This document describes the canonical end-to-end flows at a level useful for debugging and auditing. Exact endpoints and payloads are defined by each module dossier under docs/modules/.
1) Scan execution (happy path)
- Client -> Gateway: submit scan request (authenticated; tenant-scoped).
- Gateway -> Scanner.WebService: route request after auth/rate-limit checks.
- Scanner.WebService -> PostgreSQL: persist scan manifest and initial status.
- Scanner.WebService -> queue/stream: enqueue a scan job (Valkey streams by default; optional alternative transports exist).
- Scanner.Worker -> queue/stream: claim job, pull image, extract layers, run analyzers.
- Scanner.Worker -> RustFS/S3: write SBOM fragments, composed SBOMs, and other scan artifacts.
- Scanner.Worker -> Concelier: query linksets / observations needed for evaluation (deployment-dependent).
- Scanner.Worker -> Scanner.WebService: heartbeat and completion callbacks.
- Scanner.WebService -> Policy: request verdict evaluation using SBOM + advisory + VEX + policy inputs.
- Scanner.WebService -> Signer / Attestor (optional): create DSSE/in-toto evidence bundles and (optionally) attach transparency receipts.
- Scanner.WebService -> events stream: publish completion events for notifications and downstream consumers.
- Notify.WebService/Worker -> channels: render and deliver notifications with idempotency tracking.
Offline note: for air-gapped deployments, step 6 writes to local object storage and step 7 relies on offline mirrors/bundles rather than public feeds. See docs/24_OFFLINE_KIT.md and docs/airgap/overview.md.
2) Advisory ingestion (delta-driven)
- Concelier.Worker fetches advisories from configured sources (mirrors first; no hidden outbound calls in air-gap profiles).
- Concelier validates and normalizes advisories, producing canonical observations and linksets.
- Concelier -> PostgreSQL (
vuln) persists immutable raw documents (append-only patterns where required) plus derived linksets. - Concelier -> Scheduler notifies about deltas (new/updated advisories) via webhook/event.
- Scheduler schedules impacted re-scans or evaluations based on the delta.
3) VEX ingestion and consensus
- Excititor.Worker fetches VEX statements from configured sources (mirrors/bundles for offline).
- Excititor verifies signatures where required and normalizes statements into a canonical shape.
- Excititor -> PostgreSQL (
vex) persists immutable raw statements and consensus outcomes. - Excititor -> Scheduler / Policy emits deltas so verdicts can be recomputed deterministically.
4) Policy evaluation (decision trace)
- Caller (Scanner/UI/CLI) -> Policy.Gateway submits evaluation request.
- Policy.Gateway loads exception objects and policy snapshots from its own store.
- Policy Engine consumes advisory/VEX observations (by read model, replication, or API depending on deployment) and applies deterministic precedence/lattice rules.
- Policy.Gateway -> caller returns a verdict plus a trace/explain payload suitable for audits.
5) Notification delivery
- Notify.WebService consumes platform events (scan completed, advisory delta, etc.).
- Notify.WebService -> queue/stream enqueues delivery tasks with idempotency keys.
- Notify.Worker -> channels delivers (email/chat/webhook), records results, and retries with deterministic backoff rules.