42 KiB
42 KiB
User Flow Architecture
This document provides detailed UML-style diagrams showing how users interact with StellaOps from the UI/CLI through the Gateway to backend services. All flows are documented from the user's perspective.
Table of Contents
- 1. Architecture Overview
- 2. Dashboard Data Flow
- 3. Scan Submission Flow
- 4. SBOM Generation Flow
- 5. Policy Evaluation Flow
- 6. Policy Setup Flow
- 7. Notification Flow
- 8. Export Flow
1. Architecture Overview
Layered Architecture Diagram
+-----------------------------------------------------------------------------------------------------------+
| USER EXPERIENCE LAYER |
+-----------------------------------------------------------------------------------------------------------+
| |
| +----------------+ +----------------+ +----------------+ |
| | Web UI | | CLI | | CI/CD | |
| | (Angular 17) | | (Native AOT) | | (BuildX) | |
| +-------+--------+ +-------+--------+ +-------+--------+ |
| | | | |
| +----------------------+----------------------+ |
| | HTTPS + DPoP/mTLS |
| v |
| +-------------------------------------------------------------------------------------------+ |
| | GATEWAY (API Router) | |
| | +---------------+ +---------------+ +---------------+ +---------------+ +-------------+ | |
| | |CorrelationId |>| DPoP/mTLS |>|IdentityHeader |>| Authorization |>|Routing | | |
| | | Middleware | | Validation | | Policy | | Middleware | |Decision | | |
| | +---------------+ +---------------+ +---------------+ +---------------+ +-------------+ | |
| | | | |
| | +------------+------------+ | |
| | v v | |
| | Token Validation Binary Frame Protocol | |
| | via Authority (TCP/TLS/Valkey) | |
| +-------------------------------------------------------------------------------------------+ |
| |
+-----------------------------------------------------------------------------------------------------------+
|
+------------------+------------------+------------------+------------------+
v v v v v
+-----------------------------------------------------------------------------------------------------------+
| MICROSERVICES LAYER |
+-----------------------------------------------------------------------------------------------------------+
| |
| +------------------+ +------------------+ +------------------+ +------------------+ +--------------+ |
| | AUTHORITY | | SCANNER | | POLICY | | CONCELIER | | EXCITITOR | |
| | (OAuth/OIDC) | | (SBOM Gen) | | (Decisions) | | (Advisories) | | (VEX Ingest) | |
| +------------------+ +------------------+ +------------------+ +------------------+ +--------------+ |
| | * DPoP tokens | | * WebService | | * K4 Lattice | | * NVD/OSV/GHSA | | * OpenVEX | |
| | * mTLS certs | | * Worker pool | | * Confidence | | * CSAF sources | | * CSAF VEX | |
| | * Scopes/RBAC | | * 11 analyzers | | * Gates | | * Linksets | | * Consensus | |
| | * Tenant mgmt | | * Call graphs | | * VEX emission | | * AOC enforced | | * Signatures | |
| +--------+---------+ +--------+---------+ +--------+---------+ +--------+---------+ +------+-------+ |
| | | | | | |
| | +---------------+--------------------+--------------------+----------------------+ |
| | | | | | |
| v v v v v |
| +--------------------------------------------------------------------------------------------+ |
| | EVENT BUS (Valkey Streams / NATS JetStream) | |
| | scanner:events | concelier:drift | policy:evaluated | notify:delivery | scheduler:jobs | |
| +--------------------------------------------------------------------------------------------+ |
| | | | | |
| v v v v |
| +------------------+ +------------------+ +------------------+ +------------------+ |
| | SCHEDULER | | NOTIFY | | ATTESTOR | | EXPORTCENTER | |
| | (Job Orch) | | (Notifications) | | (DSSE/in-toto) | | (SARIF/SBOM) | |
| +------------------+ +------------------+ +------------------+ +------------------+ |
| | * Cron jobs | | * Slack/Teams | | * DSSE envelope | | * Format conv | |
| | * Delta rescan | | * Email/Webhook | | * Rekor v2 log | | * Batch export | |
| | * Lease/backoff | | * Templates | | * Verify chain | | * Evidence zip | |
| +------------------+ +------------------+ +--------+---------+ +------------------+ |
| | |
| v |
| +------------------+ |
| | SIGNER | |
| | (Crypto Ops) | |
| +------------------+ |
| | * PKIX signing | |
| | * FIPS/eIDAS | |
| | * GOST/SM/PQ | |
| +------------------+ |
| |
+-----------------------------------------------------------------------------------------------------------+
|
v
+-----------------------------------------------------------------------------------------------------------+
| PERSISTENCE LAYER |
+-----------------------------------------------------------------------------------------------------------+
| |
| +----------------------------------------------+ +----------------------------------------------+ |
| | PostgreSQL v16+ | | RustFS (S3 API) | |
| | | | | |
| | Schema: authority (identity, clients, keys) | | blobs/{sha256}/sbom.json | |
| | Schema: scanner (manifests, triage) | | blobs/{sha256}/sbom.cdx.pb | |
| | Schema: vuln (advisory_raw, linksets) | | blobs/{sha256}/attestation.dsse | |
| | Schema: vex (vex_raw, consensus) | | blobs/{sha256}/evidence.bundle | |
| | Schema: policy (exceptions, findings) | | | |
| | Schema: scheduler (jobs, runs) | | | |
| | Schema: notify (rules, channels, history) | | | |
| | Schema: orchestrator (workflows) | | | |
| +----------------------------------------------+ +----------------------------------------------+ |
| |
| +----------------------------------------------+ |
| | Valkey v8.0 (Cache/Queue) | |
| | | |
| | scan:{digest} -> Last scan result | |
| | layers:{digest} -> Delta cache (90d TTL) | |
| | dpop:{jti} -> DPoP nonce (5m TTL) | |
| | queue:* -> Task streams | |
| | events:* -> Event streams (7d TTL) | |
| +----------------------------------------------+ |
| |
+-----------------------------------------------------------------------------------------------------------+
2. Dashboard Data Flow
When a user opens the Web UI dashboard, the following data flow occurs:
User opens Web UI (Angular)
|
v
+--------------------------------------------------------------------------------------------+
| HomeDashboardComponent (src/Web/StellaOps.Web/src/app/features/home/) |
| |
| +--------------------------------------------------------------------------------------+ |
| | forkJoin([ | |
| | ConsoleVulnApi.getFacets() -> Severity breakdown (Critical/High/Med/Low) | |
| | RiskApi.stats() -> Risk score & trend (improving/worsening) | |
| | ReachabilityApi.getSummary() -> Reachable/Unreachable/Uncertain counts | |
| | ]) | |
| +--------------------------------------------------------------------------------------+ |
| | |
| | HTTPS |
| v |
| +--------------------------------------------------------------------------------------+ |
| | GATEWAY (routes by path) | |
| | | |
| | /api/console/vuln/facets -> Scanner.WebService | |
| | /api/risk/stats -> RiskEngine | |
| | /api/reachability/summary-> ReachGraph | |
| +--------------------------------------------------------------------------------------+ |
| | |
| v |
| +--------------------------------------------------------------------------------------+ |
| | Dashboard Widgets Rendered: | |
| | +------------+ +------------+ +------------+ +------------+ | |
| | | Severity | | Risk Score | |Reachability| | VEX Impact | | |
| | | Breakdown | | & Trend | | Donut | | Rate | | |
| | +------------+ +------------+ +------------+ +------------+ | |
| +--------------------------------------------------------------------------------------+ |
+--------------------------------------------------------------------------------------------+
Data Sources (PostgreSQL schemas):
- scanner.scan_results -> Aggregated severity counts
- policy.effective_finding -> Finding counts by status
- vuln.advisory_raw -> CVE metadata
- vex.vex_raw -> VEX suppression stats
Dashboard API Endpoints
| Widget | API Endpoint | Module | Schema |
|---|---|---|---|
| Severity Breakdown | GET /api/console/vuln/facets |
Scanner | scanner |
| Risk Score & Trend | GET /api/risk/stats |
RiskEngine | policy |
| Reachability Donut | GET /api/reachability/summary |
ReachGraph | scanner |
| VEX Impact Rate | GET /api/vex/suppression-rate |
Excititor | vex |
3. Scan Submission Flow
CLI/API Scan Submission
User: stellaops scan --image registry.example.com/app:v1.0
|
v
+--------------------------------------------------------------------------------------------+
| POST /api/v1/scans/ |
| { |
| "image": { "reference": "registry.example.com/app:v1.0" }, |
| "metadata": { "tenant": "alpha" } |
| } |
+--------------------------------------------------------------------------------------------+
|
v
+--------------------------------------------------------------------------------------------+
| Scanner.WebService |
| 1. Validate request (ref OR digest required) |
| 2. Normalize ScanTarget |
| 3. Create ScanManifest in PostgreSQL (scanner schema) |
| 4. Enqueue job to Valkey Streams (scanner:jobs) |
| 5. Return 202 Accepted + scanId |
+--------------------------------------------------------------------------------------------+
|
v
+--------------------------------------------------------------------------------------------+
| Scanner.Worker (N replicas, queue-driven) |
| |
| 1. Consume job from Valkey Stream (with lease) |
| 2. Pull OCI image layers |
| 3. Check delta cache (Valkey layers:{digest}) |
| - If cached: stitch existing SBOM fragments (20ms fast path) |
| - If new: run full analysis |
| |
| 4. Execute 11 Language Analyzers: |
| +------------------------------------------------------------------------+ |
| | OS: Apk (Alpine), Dpkg (Debian), Rpm (RHEL) | |
| | Lang: Java, Node, Python, Go, .NET, Rust, Ruby, PHP, Bun, Deno | |
| | Native: ELF (Linux), PE (Windows M2), MachO (macOS M2) | |
| +------------------------------------------------------------------------+ |
| |
| 5. Extract call graphs (for reachability) |
| 6. Generate SBOM: |
| - Inventory view (all components) |
| - Usage view (entrypoint closure) |
| - Formats: CycloneDX 1.6 (JSON/Protobuf), SPDX 3.0.1 |
| |
| 7. Upload to RustFS: blobs/{sha256}/sbom.cdx.json |
| 8. Update PostgreSQL: scanner.artifacts, scanner.scan_status |
| 9. Publish event: scanner:events (scan.completed) |
+--------------------------------------------------------------------------------------------+
Scan Status Polling
GET /api/v1/scans/{scanId}
|
v
+--------------------------------------------------------------------------------------------+
| Response: |
| { |
| "scanId": "scan-abc123", |
| "imageDigest": "sha256:...", |
| "state": "running|completed|failed", |
| "progress": { |
| "currentStage": "analyzing-java", |
| "percentage": 45, |
| "layersCurrent": 5, |
| "layersTotal": 12 |
| }, |
| "startedAt": "2025-01-02T15:30:00Z", |
| "completedAt": null, |
| "artifacts": { |
| "sbom": { "format": "cdx-json", "digest": "sha256:..." } |
| } |
| } |
+--------------------------------------------------------------------------------------------+
4. SBOM Generation Flow
Docker BuildX Plugin Integration
docker buildx build --sbom=true ...
|
v
+--------------------------------------------------------------------------------------------+
| StellaOps.Scanner.Sbomer.BuildXPlugin (src/Scanner/...BuildXPlugin/) |
| |
| For each layer: |
| 1. Extract files |
| 2. Run analyzers (Node, Java, Python, Go, .NET, Rust, Ruby, PHP, Bun, Deno) |
| 3. Generate layer SBOM fragment |
| 4. Upload to local CAS (content-addressable store) |
| 5. Emit OCI annotation (referrer) |
| |
| CI overhead: <=300ms per layer |
+--------------------------------------------------------------------------------------------+
|
v
Image pushed to registry with SBOM referrer annotation
BYOS (Bring Your Own SBOM) Upload
POST /api/v1/sboms/upload
{
"artifactRef": "app:v1.0",
"artifactDigest": "sha256:...",
"document": { /* CycloneDX or SPDX */ },
"format": "cdx-json"
}
|
v
+--------------------------------------------------------------------------------------------+
| SbomByosUploadService: |
| 1. Format detection + schema validation |
| 2. Component normalization (PURL, sort) |
| 3. Quality scoring + warnings |
| 4. Digest computation |
| 5. Register in artifact catalog |
| 6. Trigger scan coordination (same downstream flow) |
+--------------------------------------------------------------------------------------------+
5. Policy Evaluation Flow
Scan Completion Triggers Policy
scan.completed event published to Valkey Streams
|
v
+--------------------------------------------------------------------------------------------+
| Event Subscribers (consume scanner:events) |
| |
| +----------------+ +----------------+ +----------------+ +----------------+ |
| | Policy.Engine | | Notify.Worker | |TimelineIndexer | | ExportCenter | |
| | (evaluate) | | (alert) | | (audit) | | (SARIF export) | |
| +-------+--------+ +----------------+ +----------------+ +----------------+ |
| | |
| v |
| +--------------------------------------------------------------------------------------+ |
| | Policy Engine Evaluation Pipeline | |
| | | |
| | Input Sources (immutable reads only - AOC enforced): | |
| | * SBOM from SbomService (RustFS) | |
| | * Advisory raw from PostgreSQL (vuln.advisory_raw) | |
| | * VEX raw from PostgreSQL (vex.vex_raw) | |
| | * Reachability lattice from Scanner call-graph | |
| | * Policy definition (stella-dsl@1) | |
| | | |
| | +--------------------------------------------------------------------------------+ | |
| | | K4 LATTICE LOGIC (Four-Valued Truth) | | |
| | | | | |
| | | Values: Unknown (bot) | True (T) | False (F) | Conflict (top) | | |
| | | | | |
| | | Knowledge Ordering: bot <= {T,F} <= top | | |
| | | | | |
| | | Conflict Detection: T join F = top | | |
| | | (Both evidence for and against -> conflict preserved, not collapsed) | | |
| | +--------------------------------------------------------------------------------+ | |
| | | |
| | Processing Steps: | |
| | 1. Load Policy IR (cached by policyId+version hash) | |
| | 2. Batch join: SBOM <-> Advisory <-> VEX (deterministic ordering) | |
| | 3. For each (component, vulnerability): | |
| | a. Compute Evidence-Weighted Score | |
| | b. Execute policy rules (first-match) | |
| | c. Apply exceptions (specificity-ranked) | |
| | d. Check unknown budget | |
| | e. Calculate confidence (5 factors): | |
| | +-----------------------------------------------------------------------+ | |
| | | Reachability (0.85 unreachable -> 0.1 confirmed reachable) | | |
| | | Runtime (0.9 supports -> 0.2 contradicts) | | |
| | | VEX (trust score x status weight) | | |
| | | Provenance (SBOM completeness) | | |
| | | Policy (exception coverage) | | |
| | | | | |
| | | Final = Clamp01(Sum of Weight x RawValue) | | |
| | +-----------------------------------------------------------------------+ | |
| | f. Emit OpenVEX decision (if verdict change) | |
| | | |
| | 4. Upsert effective findings: | |
| | - policy.effective_finding_{policyId} (current snapshot) | |
| | - policy.effective_finding_{policyId}_history (audit trail) | |
| | | |
| | 5. Compute determinism hash for replay verification | |
| | | |
| | Output: | |
| | verdict: PASS | BLOCK | WARN | FAIL | |
| | confidence: 0.0-1.0 with tier (VeryHigh/High/Medium/Low/VeryLow) | |
| | explain_trace: [rule hits + factor breakdown] | |
| | evidence_ids: [advisory_raw_ids, vex_raw_ids, sbom_digest] | |
| +--------------------------------------------------------------------------------------+ |
+--------------------------------------------------------------------------------------------+
|
v
+--------------------------------------------------------------------------------------------+
| Optional: DSSE Signing + Transparency |
| |
| Policy Engine -> Signer (DSSE envelope) |
| | |
| v |
| Attestor (in-toto predicate: stella.ops/vexDecision@v1) |
| | |
| v |
| Rekor v2 transparency log (optional) |
+--------------------------------------------------------------------------------------------+
6. Policy Setup Flow
User navigates to Policy Studio (/policy-studio)
|
v
+--------------------------------------------------------------------------------------------+
| Web UI: PolicyWorkspaceComponent |
| |
| +--------------------------------------------------------------------------------------+ |
| | Policy Studio Sub-Features: | |
| | | |
| | 1. Policy Editor (Monaco YAML) | |
| | - Visual rule builder | |
| | - stella-dsl@1 syntax | |
| | - Live validation | |
| | | |
| | 2. Policy Simulation | |
| | - Test against historical scan data | |
| | - What-if analysis | |
| | - Impact preview | |
| | | |
| | 3. Approval Workflow | |
| | - G0 (auto) -> G1 (peer) -> G2 (owner) -> G3 (manager) -> G4 (CISO) | |
| | - Multi-stage review | |
| | | |
| | 4. Policy Dashboard | |
| | - Execution history | |
| | - Metrics & trends | |
| | - Explain traces | |
| +--------------------------------------------------------------------------------------+ |
| | |
| v |
| +--------------------------------------------------------------------------------------+ |
| | Policy Schema (YAML v1.0) | |
| | | |
| | version: "1.0" | |
| | rules: | |
| | - name: Block Critical | |
| | severity: [Critical] | |
| | action: block | |
| | | |
| | - name: Ignore Low Dev | |
| | severity: [Low, None] | |
| | environments: [dev, staging] | |
| | action: ignore | |
| | expires: "2026-01-01" | |
| | | |
| | - name: Escalate Regional High | |
| | sources: [NVD, CNNVD, ENISA] | |
| | severity: [High, Critical] | |
| | action: escalate | |
| +--------------------------------------------------------------------------------------+ |
| | |
| v |
| POST /api/v1/policies/ -> Gateway -> Policy.Engine |
| | |
| v |
| +--------------------------------------------------------------------------------------+ |
| | Policy Engine Processing | |
| | | |
| | 1. Validate schema (JSON-Schema at src/Policy/__Libraries/.../Schemas/) | |
| | 2. Compile to IR (cached by policyId+version hash) | |
| | 3. Store in PostgreSQL (policy.policies) | |
| | 4. Set lifecycle state: DRAFT -> SHADOW -> ACTIVE -> ENFORCING | |
| | | |
| | Storage Tables: | |
| | * policy.policies - Policy versions, lifecycle states | |
| | * policy.policy_runs - Run records with cursors | |
| | * policy.exceptions - Approved exceptions (suppress/defer/downgrade) | |
| | * policy.exception_approval_audit - Approval chain | |
| +--------------------------------------------------------------------------------------+ |
+--------------------------------------------------------------------------------------------+
7. Notification Flow
Event published (e.g., scan.completed, advisory.delta)
|
v
+--------------------------------------------------------------------------------------------+
| Valkey Stream: scanner:events / concelier:drift / etc. |
+--------------------------------------------------------------------------------------------+
|
v
+--------------------------------------------------------------------------------------------+
| Notify.Worker consumes stream |
| 1. Evaluates notification rules (Notify.Rule) |
| 2. Matches event kind (scanner.report.ready, scheduler.rescan.delta) |
| 3. Queries user notification preferences |
| 4. Renders template (configurable per channel + locale) |
| 5. Sends via configured channels: |
| +-- Slack: POST to hooks.slack.com |
| +-- Teams: POST to graph.microsoft.com |
| +-- Email: SMTP send |
| +-- Webhook: POST to custom endpoint |
| 6. Records delivery receipt |
| 7. Idempotency tracking (prevents duplicate sends) |
| 8. Retry with deterministic backoff |
+--------------------------------------------------------------------------------------------+
8. Export Flow
User requests export
|
v
+--------------------------------------------------------------------------------------------+
| GET /api/v1/scans/{scan_id}/export?format=spdx |
+--------------------------------------------------------------------------------------------+
|
v
+--------------------------------------------------------------------------------------------+
| Scanner.Web or ExportCenter |
| 1. Queries scan metadata from PostgreSQL |
| 2. Retrieves SBOM artifact from RustFS |
| 3. Signs SBOM with Signer service (DSSE envelope) |
| 4. Creates in-toto attestation with Attestor |
| 5. Stores final bundle to RustFS |
| 6. Returns signed bundle (application/vnd.in-toto+json) |
+--------------------------------------------------------------------------------------------+
|
v
+--------------------------------------------------------------------------------------------+
| For air-gap export: |
| ExportCenter bundles: |
| - SBOMs (SPDX/CycloneDX) |
| - Offline Kit (vulnerability feeds, trust roots, signatures) |
| - Evidence bundles (DSSE-signed artifacts) |
| Generates mirror artefacts |
| Packages for portable/USB distribution |
+--------------------------------------------------------------------------------------------+
API Endpoint Quick Reference
| User Action | Endpoint | Module | Schema |
|---|---|---|---|
| View Dashboard | GET /api/console/vuln/facets |
Scanner | scanner |
GET /api/risk/stats |
RiskEngine | policy |
|
GET /api/reachability/summary |
ReachGraph | scanner |
|
| Submit Scan | POST /api/v1/scans/ |
Scanner | scanner |
| Check Status | GET /api/v1/scans/{id} |
Scanner | scanner |
| Download SBOM | GET /api/v1/scans/{id}/sbom |
Scanner | RustFS |
| Upload SBOM | POST /api/v1/sboms/upload |
Scanner | scanner |
| View Findings | GET /api/v1/findings |
Policy | policy |
| Create Policy | POST /api/v1/policies/ |
Policy | policy |
| Evaluate Policy | POST /api/v1/policies/evaluate |
Policy | policy + vuln + vex |
| Manage Exceptions | POST /api/v1/exceptions/ |
Policy | policy |
| View VEX | GET /api/vex/statements |
Excititor | vex |
| View Advisories | GET /api/advisories |
Concelier | vuln |
| Export Evidence | GET /api/v1/export/bundle |
ExportCenter | RustFS |