412 lines
15 KiB
Markdown
412 lines
15 KiB
Markdown
# Scan Submission Flow
|
|
|
|
## Overview
|
|
|
|
The Scan Submission Flow describes the complete lifecycle of a container image scan from initial submission through SBOM generation, vulnerability matching, policy evaluation, and result storage. This is the core workflow that produces security verdicts for container images.
|
|
|
|
**Business Value**: Automated, deterministic scanning ensures every container image is evaluated against the latest advisories and policies before deployment.
|
|
|
|
## Actors
|
|
|
|
| Actor | Type | Role |
|
|
|-------|------|------|
|
|
| Developer/CI | Human/System | Submits scan request |
|
|
| CLI / API Client | System | Initiates scan via API |
|
|
| Gateway | Service | Routes and authenticates |
|
|
| Scheduler | Service | Queues and dispatches work |
|
|
| Scanner | Service | Analyzes image and generates SBOM |
|
|
| Concelier | Service | Provides advisory data |
|
|
| VexLens | Service | Provides VEX statements |
|
|
| Policy | Service | Evaluates policy rules |
|
|
| Attestor | Service | Signs scan results |
|
|
| Notify | Service | Sends notifications |
|
|
|
|
## Prerequisites
|
|
|
|
- Valid API credentials (JWT or API key)
|
|
- Container registry accessible (or image pulled locally)
|
|
- Registry credentials configured (if private registry)
|
|
|
|
## Flow Diagram
|
|
|
|
```
|
|
┌─────────────────────────────────────────────────────────────────────────────────┐
|
|
│ Scan Submission Flow │
|
|
└─────────────────────────────────────────────────────────────────────────────────┘
|
|
|
|
┌───────┐ ┌─────────┐ ┌─────────┐ ┌───────────┐ ┌─────────┐
|
|
│ CLI │ │ Gateway │ │Scheduler│ │ Scanner │ │ Policy │
|
|
└───┬───┘ └────┬────┘ └────┬────┘ └─────┬─────┘ └────┬────┘
|
|
│ │ │ │ │
|
|
│ POST /api/v1/scans │ │ │
|
|
│ {image: "..."} │ │ │
|
|
│───────────>│ │ │ │
|
|
│ │ │ │ │
|
|
│ │ Validate │ │ │
|
|
│ │ + Enqueue │ │ │
|
|
│ │────────────>│ │ │
|
|
│ │ │ │ │
|
|
│ 202 Accepted │ │ │
|
|
│ {scan_id: "..."} │ │ │
|
|
│<───────────│ │ │ │
|
|
│ │ │ │ │
|
|
│ │ │ Dispatch │ │
|
|
│ │ │─────────────>│ │
|
|
│ │ │ │ │
|
|
│ │ │ │ Pull image │
|
|
│ │ │ │──────┐ │
|
|
│ │ │ │ │ │
|
|
│ │ │ │<─────┘ │
|
|
│ │ │ │ │
|
|
│ │ │ │ Extract │
|
|
│ │ │ │ layers │
|
|
│ │ │ │──────┐ │
|
|
│ │ │ │ │ │
|
|
│ │ │ │<─────┘ │
|
|
│ │ │ │ │
|
|
│ │ │ │ Run 11 │
|
|
│ │ │ │ analyzers │
|
|
│ │ │ │──────┐ │
|
|
│ │ │ │ │ │
|
|
│ │ │ │<─────┘ │
|
|
│ │ │ │ │
|
|
│ │ │ │ Generate │
|
|
│ │ │ │ SBOM │
|
|
│ │ │ │──────┐ │
|
|
│ │ │ │ │ │
|
|
│ │ │ │<─────┘ │
|
|
│ │ │ │ │
|
|
│ │ │ │ ┌──────────┐│
|
|
│ │ │ │ │Concelier ││
|
|
│ │ │ │ └────┬─────┘│
|
|
│ │ │ │ │ │
|
|
│ │ │ │ Match vulns │
|
|
│ │ │ │──────>│ │
|
|
│ │ │ │ │ │
|
|
│ │ │ │<──────│ │
|
|
│ │ │ │ │
|
|
│ │ │ │ ┌─────────┐ │
|
|
│ │ │ │ │ VexLens │ │
|
|
│ │ │ │ └────┬────┘ │
|
|
│ │ │ │ │ │
|
|
│ │ │ │ Get VEX │
|
|
│ │ │ │──────>│ │
|
|
│ │ │ │ │ │
|
|
│ │ │ │<──────│ │
|
|
│ │ │ │ │
|
|
│ │ │ │ Request │
|
|
│ │ │ │ verdict │
|
|
│ │ │ │─────────────>│
|
|
│ │ │ │ │
|
|
│ │ │ │ K4 lattice │
|
|
│ │ │ │ evaluation │
|
|
│ │ │ │<─────────────│
|
|
│ │ │ │ │
|
|
│ │ │ │ ┌─────────┐ │
|
|
│ │ │ │ │ Attestor│ │
|
|
│ │ │ │ └────┬────┘ │
|
|
│ │ │ │ │ │
|
|
│ │ │ │ Sign SBOM │
|
|
│ │ │ │──────>│ │
|
|
│ │ │ │ │ │
|
|
│ │ │ │<──────│ │
|
|
│ │ │ │ │
|
|
│ │ │ Complete │ │
|
|
│ │ │<─────────────│ │
|
|
│ │ │ │ │
|
|
│ │ │ ┌────────┐ │ │
|
|
│ │ │ │ Notify │ │ │
|
|
│ │ │ └───┬────┘ │ │
|
|
│ │ │ │ │ │
|
|
│ │ │ Send │ │ │
|
|
│ │ │──────> │ │
|
|
│ │ │ │ │
|
|
```
|
|
|
|
## Step-by-Step
|
|
|
|
### 1. Scan Request Submission
|
|
|
|
**CLI Command:**
|
|
```bash
|
|
stellaops scan docker.io/library/nginx:1.25
|
|
```
|
|
|
|
**API Request:**
|
|
```http
|
|
POST /api/v1/scans HTTP/1.1
|
|
Host: gateway.stellaops.local
|
|
Authorization: Bearer {jwt}
|
|
X-Tenant-Id: acme-corp
|
|
Content-Type: application/json
|
|
|
|
```
|
|
|
|
### 2. Gateway Processing
|
|
|
|
- Validates JWT and extracts claims
|
|
- Applies rate limiting
|
|
- Validates request schema
|
|
- Forwards to Scheduler
|
|
|
|
### 3. Scheduler Queuing
|
|
|
|
- Creates scan job record in `scheduler.jobs` table
|
|
- Assigns priority based on tenant tier
|
|
- Returns scan ID immediately (async pattern)
|
|
|
|
**Response:**
|
|
```json
|
|
{
|
|
"scan_id": "scan-7f3a9b2c-1234-5678-abcd-ef0123456789",
|
|
"status": "queued",
|
|
"estimated_wait": "PT30S",
|
|
"status_url": "/api/v1/scans/scan-7f3a9b2c-1234-5678-abcd-ef0123456789"
|
|
}
|
|
```
|
|
|
|
### 4. Scanner Dispatch
|
|
|
|
Scheduler dispatches to available Scanner worker:
|
|
|
|
```json
|
|
{
|
|
"job_id": "scan-7f3a9b2c-...",
|
|
"image_ref": "docker.io/library/nginx:1.25",
|
|
"resolved_digest": "sha256:abc123...",
|
|
"analyzers": ["os", "dotnet", "java", "node", "python", "go", "rust", "php", "ruby", "deno", "binary"],
|
|
"options": {...}
|
|
}
|
|
```
|
|
|
|
### 5. Image Analysis
|
|
|
|
Scanner performs multi-stage analysis:
|
|
|
|
| Stage | Duration | Description |
|
|
|-------|----------|-------------|
|
|
| Pull | 5-30s | Fetch image manifest and layers |
|
|
| Extract | 2-10s | Unpack layer tarballs |
|
|
| OS Detection | <1s | Identify base OS (Alpine, Debian, etc.) |
|
|
| Analyzer Fan-out | 10-60s | Run 11 parallel language analyzers |
|
|
| Dependency Resolution | 5-20s | Resolve transitive dependencies |
|
|
| SBOM Assembly | 1-5s | Merge results into unified SBOM |
|
|
|
|
### 6. Vulnerability Matching
|
|
|
|
Scanner queries Concelier for matching advisories:
|
|
|
|
```http
|
|
POST /internal/match HTTP/1.1
|
|
Content-Type: application/json
|
|
|
|
```
|
|
|
|
Response includes matched CVEs with affected version ranges.
|
|
|
|
### 7. VEX Application
|
|
|
|
Scanner queries VexLens for applicable VEX statements:
|
|
|
|
```http
|
|
POST /internal/vex/apply HTTP/1.1
|
|
Content-Type: application/json
|
|
|
|
```
|
|
|
|
VEX statements modify vulnerability status (e.g., `not_affected`, `fixed`).
|
|
|
|
### 8. Policy Evaluation
|
|
|
|
Scanner requests policy verdict from Policy engine:
|
|
|
|
```http
|
|
POST /internal/evaluate HTTP/1.1
|
|
Content-Type: application/json
|
|
|
|
```
|
|
|
|
Policy engine applies K4 lattice logic and returns verdict:
|
|
|
|
```json
|
|
{
|
|
"verdict": "FAIL",
|
|
"confidence": 0.92,
|
|
"violations": [
|
|
{
|
|
"rule": "no-critical-reachable",
|
|
"cve": "CVE-2024-1234",
|
|
"message": "Critical CVE with reachable code path"
|
|
}
|
|
]
|
|
}
|
|
```
|
|
|
|
### 9. Attestation
|
|
|
|
Scanner requests DSSE attestation from Attestor:
|
|
|
|
```http
|
|
POST /internal/attest HTTP/1.1
|
|
Content-Type: application/json
|
|
|
|
```
|
|
|
|
### 10. Result Storage
|
|
|
|
Scanner stores results:
|
|
|
|
| Store | Data |
|
|
|-------|------|
|
|
| PostgreSQL `scanner.scans` | Scan metadata and verdict |
|
|
| PostgreSQL `scanner.findings` | Individual vulnerability findings |
|
|
| RustFS `blobs/{sha256}/` | SBOM document |
|
|
| RustFS `attestations/{sha256}/` | DSSE envelope |
|
|
| Valkey `scan:{digest}` | Cache for quick lookup |
|
|
|
|
### 11. Notification
|
|
|
|
Scheduler triggers Notify service for configured channels:
|
|
|
|
```json
|
|
{
|
|
"event": "scan.complete",
|
|
"scan_id": "scan-7f3a9b2c-...",
|
|
"verdict": "FAIL",
|
|
"channels": ["slack", "webhook"]
|
|
}
|
|
```
|
|
|
|
## Data Contracts
|
|
|
|
### Scan Request Schema
|
|
|
|
```typescript
|
|
interface ScanRequest {
|
|
image: string; // Container image reference
|
|
options?: {
|
|
analyzers?: string[]; // List or "all"
|
|
sbom_format?: 'spdx-3.0' | 'cyclonedx-1.6';
|
|
policy_set?: string; // Policy set name
|
|
attestation?: boolean;
|
|
labels?: Record<string, string>;
|
|
};
|
|
}
|
|
```
|
|
|
|
### Scan Result Schema
|
|
|
|
```typescript
|
|
interface ScanResult {
|
|
scan_id: string;
|
|
image: string;
|
|
digest: string;
|
|
status: 'queued' | 'running' | 'completed' | 'failed';
|
|
started_at: string;
|
|
completed_at?: string;
|
|
verdict: 'PASS' | 'FAIL' | 'WARN' | 'PENDING';
|
|
confidence: number;
|
|
summary: {
|
|
critical: number;
|
|
high: number;
|
|
medium: number;
|
|
low: number;
|
|
unknown: number;
|
|
};
|
|
sbom_url: string;
|
|
attestation_url?: string;
|
|
findings_url: string;
|
|
}
|
|
```
|
|
|
|
## Error Handling
|
|
|
|
| Error | HTTP Status | Recovery |
|
|
|-------|-------------|----------|
|
|
| Image not found | 404 | Verify image reference and credentials |
|
|
| Registry auth failed | 401 | Re-configure registry credentials |
|
|
| Analyzer timeout | 504 | Retry with increased timeout |
|
|
| Policy set not found | 400 | Verify policy set exists |
|
|
| Attestation signing failed | 500 | Check Signer service health |
|
|
|
|
## Observability
|
|
|
|
### Metrics
|
|
|
|
| Metric | Type | Labels |
|
|
|--------|------|--------|
|
|
| `scan_submitted_total` | Counter | `tenant` |
|
|
| `scan_completed_total` | Counter | `tenant`, `verdict` |
|
|
| `scan_duration_seconds` | Histogram | `tenant`, `image_size` |
|
|
| `scan_analyzer_duration_seconds` | Histogram | `analyzer` |
|
|
| `scan_findings_total` | Counter | `severity` |
|
|
|
|
### Trace Context
|
|
|
|
```
|
|
scan-submission
|
|
├── gateway-auth
|
|
├── scheduler-enqueue
|
|
└── scanner-execute
|
|
├── image-pull
|
|
├── layer-extract
|
|
├── analyzer-dotnet
|
|
├── analyzer-java
|
|
├── analyzer-node
|
|
├── ...
|
|
├── concelier-match
|
|
├── vexlens-apply
|
|
├── policy-evaluate
|
|
└── attestor-sign
|
|
```
|
|
|
|
## Related Flows
|
|
|
|
- [SBOM Generation Flow](03-sbom-generation-flow.md) - Detailed SBOM creation
|
|
- [Policy Evaluation Flow](04-policy-evaluation-flow.md) - K4 lattice details
|
|
- [CI/CD Gate Flow](10-cicd-gate-flow.md) - Pipeline integration
|