7503c19b8f
- Implemented comprehensive tests for verdict artifact generation to ensure deterministic outputs across various scenarios, including identical inputs, parallel execution, and change ordering. - Created helper methods for generating sample verdict inputs and computing canonical hashes. - Added tests to validate the stability of canonical hashes, proof spine ordering, and summary statistics. - Introduced a new PowerShell script to update SHA256 sums for files, ensuring accurate hash generation and file integrity checks.
49 lines
3.5 KiB
Markdown
49 lines
3.5 KiB
Markdown
# Request and data flows (detailed)
|
|
|
|
This document describes the canonical end-to-end flows at a level useful for debugging and auditing. Exact endpoints and payloads are defined by each module dossier under `docs/modules/`.
|
|
|
|
## 1) Scan execution (happy path)
|
|
|
|
1. **Client -> Gateway**: submit scan request (authenticated; tenant-scoped).
|
|
2. **Gateway -> Scanner.WebService**: route request after auth/rate-limit checks.
|
|
3. **Scanner.WebService -> PostgreSQL**: persist scan manifest and initial status.
|
|
4. **Scanner.WebService -> queue/stream**: enqueue a scan job (Valkey streams by default; optional alternative transports exist).
|
|
5. **Scanner.Worker -> queue/stream**: claim job, pull image, extract layers, run analyzers.
|
|
6. **Scanner.Worker -> RustFS/S3**: write SBOM fragments, composed SBOMs, and other scan artifacts.
|
|
7. **Scanner.Worker -> Concelier**: query linksets / observations needed for evaluation (deployment-dependent).
|
|
8. **Scanner.Worker -> Scanner.WebService**: heartbeat and completion callbacks.
|
|
9. **Scanner.WebService -> Policy**: request verdict evaluation using SBOM + advisory + VEX + policy inputs.
|
|
10. **Scanner.WebService -> Signer / Attestor (optional)**: create DSSE/in-toto evidence bundles and (optionally) attach transparency receipts.
|
|
11. **Scanner.WebService -> events stream**: publish completion events for notifications and downstream consumers.
|
|
12. **Notify.WebService/Worker -> channels**: render and deliver notifications with idempotency tracking.
|
|
|
|
Offline note: for air-gapped deployments, step 6 writes to local object storage and step 7 relies on offline mirrors/bundles rather than public feeds. See `docs/24_OFFLINE_KIT.md` and `docs/airgap/overview.md`.
|
|
|
|
## 2) Advisory ingestion (delta-driven)
|
|
|
|
1. **Concelier.Worker** fetches advisories from configured sources (mirrors first; no hidden outbound calls in air-gap profiles).
|
|
2. **Concelier** validates and normalizes advisories, producing canonical observations and linksets.
|
|
3. **Concelier -> PostgreSQL (`vuln`)** persists immutable raw documents (append-only patterns where required) plus derived linksets.
|
|
4. **Concelier -> Scheduler** notifies about deltas (new/updated advisories) via webhook/event.
|
|
5. **Scheduler** schedules impacted re-scans or evaluations based on the delta.
|
|
|
|
## 3) VEX ingestion and consensus
|
|
|
|
1. **Excititor.Worker** fetches VEX statements from configured sources (mirrors/bundles for offline).
|
|
2. **Excititor** verifies signatures where required and normalizes statements into a canonical shape.
|
|
3. **Excititor -> PostgreSQL (`vex`)** persists immutable raw statements and consensus outcomes.
|
|
4. **Excititor -> Scheduler / Policy** emits deltas so verdicts can be recomputed deterministically.
|
|
|
|
## 4) Policy evaluation (decision trace)
|
|
|
|
1. **Caller (Scanner/UI/CLI) -> Policy.Gateway** submits evaluation request.
|
|
2. **Policy.Gateway** loads exception objects and policy snapshots from its own store.
|
|
3. **Policy Engine** consumes advisory/VEX observations (by read model, replication, or API depending on deployment) and applies deterministic precedence/lattice rules.
|
|
4. **Policy.Gateway -> caller** returns a verdict plus a trace/explain payload suitable for audits.
|
|
|
|
## 5) Notification delivery
|
|
|
|
1. **Notify.WebService** consumes platform events (scan completed, advisory delta, etc.).
|
|
2. **Notify.WebService -> queue/stream** enqueues delivery tasks with idempotency keys.
|
|
3. **Notify.Worker -> channels** delivers (email/chat/webhook), records results, and retries with deterministic backoff rules.
|