# User Flow Architecture This document provides detailed UML-style diagrams showing how users interact with StellaOps from the UI/CLI through the Gateway to backend services. All flows are documented from the user's perspective. ## Table of Contents - [1. Architecture Overview](#1-architecture-overview) - [2. Dashboard Data Flow](#2-dashboard-data-flow) - [3. Scan Submission Flow](#3-scan-submission-flow) - [4. SBOM Generation Flow](#4-sbom-generation-flow) - [5. Policy Evaluation Flow](#5-policy-evaluation-flow) - [6. Policy Setup Flow](#6-policy-setup-flow) - [7. Notification Flow](#7-notification-flow) - [8. Export Flow](#8-export-flow) --- ## 1. Architecture Overview ### Layered Architecture Diagram ``` +-----------------------------------------------------------------------------------------------------------+ | USER EXPERIENCE LAYER | +-----------------------------------------------------------------------------------------------------------+ | | | +----------------+ +----------------+ +----------------+ | | | Web UI | | CLI | | CI/CD | | | | (Angular 17) | | (Native AOT) | | (BuildX) | | | +-------+--------+ +-------+--------+ +-------+--------+ | | | | | | | +----------------------+----------------------+ | | | HTTPS + DPoP/mTLS | | v | | +-------------------------------------------------------------------------------------------+ | | | GATEWAY (API Router) | | | | +---------------+ +---------------+ +---------------+ +---------------+ +-------------+ | | | | |CorrelationId |>| DPoP/mTLS |>|IdentityHeader |>| Authorization |>|Routing | | | | | | Middleware | | Validation | | Policy | | Middleware | |Decision | | | | | +---------------+ +---------------+ +---------------+ +---------------+ +-------------+ | | | | | | | | | +------------+------------+ | | | | v v | | | | Token Validation Binary Frame Protocol | | | | via Authority (TCP/TLS/Valkey) | | | +-------------------------------------------------------------------------------------------+ | | | +-----------------------------------------------------------------------------------------------------------+ | +------------------+------------------+------------------+------------------+ v v v v v +-----------------------------------------------------------------------------------------------------------+ | MICROSERVICES LAYER | +-----------------------------------------------------------------------------------------------------------+ | | | +------------------+ +------------------+ +------------------+ +------------------+ +--------------+ | | | AUTHORITY | | SCANNER | | POLICY | | CONCELIER | | EXCITITOR | | | | (OAuth/OIDC) | | (SBOM Gen) | | (Decisions) | | (Advisories) | | (VEX Ingest) | | | +------------------+ +------------------+ +------------------+ +------------------+ +--------------+ | | | * DPoP tokens | | * WebService | | * K4 Lattice | | * NVD/OSV/GHSA | | * OpenVEX | | | | * mTLS certs | | * Worker pool | | * Confidence | | * CSAF sources | | * CSAF VEX | | | | * Scopes/RBAC | | * 11 analyzers | | * Gates | | * Linksets | | * Consensus | | | | * Tenant mgmt | | * Call graphs | | * VEX emission | | * AOC enforced | | * Signatures | | | +--------+---------+ +--------+---------+ +--------+---------+ +--------+---------+ +------+-------+ | | | | | | | | | | +---------------+--------------------+--------------------+----------------------+ | | | | | | | | | v v v v v | | +--------------------------------------------------------------------------------------------+ | | | EVENT BUS (Valkey Streams / NATS JetStream) | | | | scanner:events | concelier:drift | policy:evaluated | notify:delivery | scheduler:jobs | | | +--------------------------------------------------------------------------------------------+ | | | | | | | | v v v v | | +------------------+ +------------------+ +------------------+ +------------------+ | | | SCHEDULER | | NOTIFY | | ATTESTOR | | EXPORTCENTER | | | | (Job Orch) | | (Notifications) | | (DSSE/in-toto) | | (SARIF/SBOM) | | | +------------------+ +------------------+ +------------------+ +------------------+ | | | * Cron jobs | | * Slack/Teams | | * DSSE envelope | | * Format conv | | | | * Delta rescan | | * Email/Webhook | | * Rekor v2 log | | * Batch export | | | | * Lease/backoff | | * Templates | | * Verify chain | | * Evidence zip | | | +------------------+ +------------------+ +--------+---------+ +------------------+ | | | | | v | | +------------------+ | | | SIGNER | | | | (Crypto Ops) | | | +------------------+ | | | * PKIX signing | | | | * FIPS/eIDAS | | | | * GOST/SM/PQ | | | +------------------+ | | | +-----------------------------------------------------------------------------------------------------------+ | v +-----------------------------------------------------------------------------------------------------------+ | PERSISTENCE LAYER | +-----------------------------------------------------------------------------------------------------------+ | | | +----------------------------------------------+ +----------------------------------------------+ | | | PostgreSQL v16+ | | RustFS (S3 API) | | | | | | | | | | Schema: authority (identity, clients, keys) | | blobs/{sha256}/sbom.json | | | | Schema: scanner (manifests, triage) | | blobs/{sha256}/sbom.cdx.pb | | | | Schema: vuln (advisory_raw, linksets) | | blobs/{sha256}/attestation.dsse | | | | Schema: vex (vex_raw, consensus) | | blobs/{sha256}/evidence.bundle | | | | Schema: policy (exceptions, findings) | | | | | | Schema: scheduler (jobs, runs) | | | | | | Schema: notify (rules, channels, history) | | | | | | Schema: orchestrator (workflows) | | | | | +----------------------------------------------+ +----------------------------------------------+ | | | | +----------------------------------------------+ | | | Valkey v8.0 (Cache/Queue) | | | | | | | | scan:{digest} -> Last scan result | | | | layers:{digest} -> Delta cache (90d TTL) | | | | dpop:{jti} -> DPoP nonce (5m TTL) | | | | queue:* -> Task streams | | | | events:* -> Event streams (7d TTL) | | | +----------------------------------------------+ | | | +-----------------------------------------------------------------------------------------------------------+ ``` --- ## 2. Dashboard Data Flow When a user opens the Web UI dashboard, the following data flow occurs: ``` User opens Web UI (Angular) | v +--------------------------------------------------------------------------------------------+ | HomeDashboardComponent (src/Web/StellaOps.Web/src/app/features/home/) | | | | +--------------------------------------------------------------------------------------+ | | | forkJoin([ | | | | ConsoleVulnApi.getFacets() -> Severity breakdown (Critical/High/Med/Low) | | | | RiskApi.stats() -> Risk score & trend (improving/worsening) | | | | ReachabilityApi.getSummary() -> Reachable/Unreachable/Uncertain counts | | | | ]) | | | +--------------------------------------------------------------------------------------+ | | | | | | HTTPS | | v | | +--------------------------------------------------------------------------------------+ | | | GATEWAY (routes by path) | | | | | | | | /api/console/vuln/facets -> Scanner.WebService | | | | /api/risk/stats -> RiskEngine | | | | /api/reachability/summary-> ReachGraph | | | +--------------------------------------------------------------------------------------+ | | | | | v | | +--------------------------------------------------------------------------------------+ | | | Dashboard Widgets Rendered: | | | | +------------+ +------------+ +------------+ +------------+ | | | | | Severity | | Risk Score | |Reachability| | VEX Impact | | | | | | Breakdown | | & Trend | | Donut | | Rate | | | | | +------------+ +------------+ +------------+ +------------+ | | | +--------------------------------------------------------------------------------------+ | +--------------------------------------------------------------------------------------------+ Data Sources (PostgreSQL schemas): - scanner.scan_results -> Aggregated severity counts - policy.effective_finding -> Finding counts by status - vuln.advisory_raw -> CVE metadata - vex.vex_raw -> VEX suppression stats ``` ### Dashboard API Endpoints | Widget | API Endpoint | Module | Schema | |--------|--------------|--------|--------| | Severity Breakdown | `GET /api/console/vuln/facets` | Scanner | `scanner` | | Risk Score & Trend | `GET /api/risk/stats` | RiskEngine | `policy` | | Reachability Donut | `GET /api/reachability/summary` | ReachGraph | `scanner` | | VEX Impact Rate | `GET /api/vex/suppression-rate` | Excititor | `vex` | --- ## 3. Scan Submission Flow ### CLI/API Scan Submission ``` User: stellaops scan --image registry.example.com/app:v1.0 | v +--------------------------------------------------------------------------------------------+ | POST /api/v1/scans/ | | { | | "image": { "reference": "registry.example.com/app:v1.0" }, | | "metadata": { "tenant": "alpha" } | | } | +--------------------------------------------------------------------------------------------+ | v +--------------------------------------------------------------------------------------------+ | Scanner.WebService | | 1. Validate request (ref OR digest required) | | 2. Normalize ScanTarget | | 3. Create ScanManifest in PostgreSQL (scanner schema) | | 4. Enqueue job to Valkey Streams (scanner:jobs) | | 5. Return 202 Accepted + scanId | +--------------------------------------------------------------------------------------------+ | v +--------------------------------------------------------------------------------------------+ | Scanner.Worker (N replicas, queue-driven) | | | | 1. Consume job from Valkey Stream (with lease) | | 2. Pull OCI image layers | | 3. Check delta cache (Valkey layers:{digest}) | | - If cached: stitch existing SBOM fragments (20ms fast path) | | - If new: run full analysis | | | | 4. Execute 11 Language Analyzers: | | +------------------------------------------------------------------------+ | | | OS: Apk (Alpine), Dpkg (Debian), Rpm (RHEL) | | | | Lang: Java, Node, Python, Go, .NET, Rust, Ruby, PHP, Bun, Deno | | | | Native: ELF (Linux), PE (Windows M2), MachO (macOS M2) | | | +------------------------------------------------------------------------+ | | | | 5. Extract call graphs (for reachability) | | 6. Generate SBOM: | | - Inventory view (all components) | | - Usage view (entrypoint closure) | | - Formats: CycloneDX 1.6 (JSON/Protobuf), SPDX 3.0.1 | | | | 7. Upload to RustFS: blobs/{sha256}/sbom.cdx.json | | 8. Update PostgreSQL: scanner.artifacts, scanner.scan_status | | 9. Publish event: scanner:events (scan.completed) | +--------------------------------------------------------------------------------------------+ ``` ### Scan Status Polling ``` GET /api/v1/scans/{scanId} | v +--------------------------------------------------------------------------------------------+ | Response: | | { | | "scanId": "scan-abc123", | | "imageDigest": "sha256:...", | | "state": "running|completed|failed", | | "progress": { | | "currentStage": "analyzing-java", | | "percentage": 45, | | "layersCurrent": 5, | | "layersTotal": 12 | | }, | | "startedAt": "2025-01-02T15:30:00Z", | | "completedAt": null, | | "artifacts": { | | "sbom": { "format": "cdx-json", "digest": "sha256:..." } | | } | | } | +--------------------------------------------------------------------------------------------+ ``` --- ## 4. SBOM Generation Flow ### Docker BuildX Plugin Integration ``` docker buildx build --sbom=true ... | v +--------------------------------------------------------------------------------------------+ | StellaOps.Scanner.Sbomer.BuildXPlugin (src/Scanner/...BuildXPlugin/) | | | | For each layer: | | 1. Extract files | | 2. Run analyzers (Node, Java, Python, Go, .NET, Rust, Ruby, PHP, Bun, Deno) | | 3. Generate layer SBOM fragment | | 4. Upload to local CAS (content-addressable store) | | 5. Emit OCI annotation (referrer) | | | | CI overhead: <=300ms per layer | +--------------------------------------------------------------------------------------------+ | v Image pushed to registry with SBOM referrer annotation ``` ### BYOS (Bring Your Own SBOM) Upload ``` POST /api/v1/sboms/upload { "artifactRef": "app:v1.0", "artifactDigest": "sha256:...", "document": { /* CycloneDX or SPDX */ }, "format": "cdx-json" } | v +--------------------------------------------------------------------------------------------+ | SbomByosUploadService: | | 1. Format detection + schema validation | | 2. Component normalization (PURL, sort) | | 3. Quality scoring + warnings | | 4. Digest computation | | 5. Register in artifact catalog | | 6. Trigger scan coordination (same downstream flow) | +--------------------------------------------------------------------------------------------+ ``` --- ## 5. Policy Evaluation Flow ### Scan Completion Triggers Policy ``` scan.completed event published to Valkey Streams | v +--------------------------------------------------------------------------------------------+ | Event Subscribers (consume scanner:events) | | | | +----------------+ +----------------+ +----------------+ +----------------+ | | | Policy.Engine | | Notify.Worker | |TimelineIndexer | | ExportCenter | | | | (evaluate) | | (alert) | | (audit) | | (SARIF export) | | | +-------+--------+ +----------------+ +----------------+ +----------------+ | | | | | v | | +--------------------------------------------------------------------------------------+ | | | Policy Engine Evaluation Pipeline | | | | | | | | Input Sources (immutable reads only - AOC enforced): | | | | * SBOM from SbomService (RustFS) | | | | * Advisory raw from PostgreSQL (vuln.advisory_raw) | | | | * VEX raw from PostgreSQL (vex.vex_raw) | | | | * Reachability lattice from Scanner call-graph | | | | * Policy definition (stella-dsl@1) | | | | | | | | +--------------------------------------------------------------------------------+ | | | | | K4 LATTICE LOGIC (Four-Valued Truth) | | | | | | | | | | | | Values: Unknown (bot) | True (T) | False (F) | Conflict (top) | | | | | | | | | | | | Knowledge Ordering: bot <= {T,F} <= top | | | | | | | | | | | | Conflict Detection: T join F = top | | | | | | (Both evidence for and against -> conflict preserved, not collapsed) | | | | | +--------------------------------------------------------------------------------+ | | | | | | | | Processing Steps: | | | | 1. Load Policy IR (cached by policyId+version hash) | | | | 2. Batch join: SBOM <-> Advisory <-> VEX (deterministic ordering) | | | | 3. For each (component, vulnerability): | | | | a. Compute Evidence-Weighted Score | | | | b. Execute policy rules (first-match) | | | | c. Apply exceptions (specificity-ranked) | | | | d. Check unknown budget | | | | e. Calculate confidence (5 factors): | | | | +-----------------------------------------------------------------------+ | | | | | Reachability (0.85 unreachable -> 0.1 confirmed reachable) | | | | | | Runtime (0.9 supports -> 0.2 contradicts) | | | | | | VEX (trust score x status weight) | | | | | | Provenance (SBOM completeness) | | | | | | Policy (exception coverage) | | | | | | | | | | | | Final = Clamp01(Sum of Weight x RawValue) | | | | | +-----------------------------------------------------------------------+ | | | | f. Emit OpenVEX decision (if verdict change) | | | | | | | | 4. Upsert effective findings: | | | | - policy.effective_finding_{policyId} (current snapshot) | | | | - policy.effective_finding_{policyId}_history (audit trail) | | | | | | | | 5. Compute determinism hash for replay verification | | | | | | | | Output: | | | | verdict: PASS | BLOCK | WARN | FAIL | | | | confidence: 0.0-1.0 with tier (VeryHigh/High/Medium/Low/VeryLow) | | | | explain_trace: [rule hits + factor breakdown] | | | | evidence_ids: [advisory_raw_ids, vex_raw_ids, sbom_digest] | | | +--------------------------------------------------------------------------------------+ | +--------------------------------------------------------------------------------------------+ | v +--------------------------------------------------------------------------------------------+ | Optional: DSSE Signing + Transparency | | | | Policy Engine -> Signer (DSSE envelope) | | | | | v | | Attestor (in-toto predicate: stella.ops/vexDecision@v1) | | | | | v | | Rekor v2 transparency log (optional) | +--------------------------------------------------------------------------------------------+ ``` --- ## 6. Policy Setup Flow ``` User navigates to Policy Studio (/policy-studio) | v +--------------------------------------------------------------------------------------------+ | Web UI: PolicyWorkspaceComponent | | | | +--------------------------------------------------------------------------------------+ | | | Policy Studio Sub-Features: | | | | | | | | 1. Policy Editor (Monaco YAML) | | | | - Visual rule builder | | | | - stella-dsl@1 syntax | | | | - Live validation | | | | | | | | 2. Policy Simulation | | | | - Test against historical scan data | | | | - What-if analysis | | | | - Impact preview | | | | | | | | 3. Approval Workflow | | | | - G0 (auto) -> G1 (peer) -> G2 (owner) -> G3 (manager) -> G4 (CISO) | | | | - Multi-stage review | | | | | | | | 4. Policy Dashboard | | | | - Execution history | | | | - Metrics & trends | | | | - Explain traces | | | +--------------------------------------------------------------------------------------+ | | | | | v | | +--------------------------------------------------------------------------------------+ | | | Policy Schema (YAML v1.0) | | | | | | | | version: "1.0" | | | | rules: | | | | - name: Block Critical | | | | severity: [Critical] | | | | action: block | | | | | | | | - name: Ignore Low Dev | | | | severity: [Low, None] | | | | environments: [dev, staging] | | | | action: ignore | | | | expires: "2026-01-01" | | | | | | | | - name: Escalate Regional High | | | | sources: [NVD, CNNVD, ENISA] | | | | severity: [High, Critical] | | | | action: escalate | | | +--------------------------------------------------------------------------------------+ | | | | | v | | POST /api/v1/policies/ -> Gateway -> Policy.Engine | | | | | v | | +--------------------------------------------------------------------------------------+ | | | Policy Engine Processing | | | | | | | | 1. Validate schema (JSON-Schema at src/Policy/__Libraries/.../Schemas/) | | | | 2. Compile to IR (cached by policyId+version hash) | | | | 3. Store in PostgreSQL (policy.policies) | | | | 4. Set lifecycle state: DRAFT -> SHADOW -> ACTIVE -> ENFORCING | | | | | | | | Storage Tables: | | | | * policy.policies - Policy versions, lifecycle states | | | | * policy.policy_runs - Run records with cursors | | | | * policy.exceptions - Approved exceptions (suppress/defer/downgrade) | | | | * policy.exception_approval_audit - Approval chain | | | +--------------------------------------------------------------------------------------+ | +--------------------------------------------------------------------------------------------+ ``` --- ## 7. Notification Flow ``` Event published (e.g., scan.completed, advisory.delta) | v +--------------------------------------------------------------------------------------------+ | Valkey Stream: scanner:events / concelier:drift / etc. | +--------------------------------------------------------------------------------------------+ | v +--------------------------------------------------------------------------------------------+ | Notify.Worker consumes stream | | 1. Evaluates notification rules (Notify.Rule) | | 2. Matches event kind (scanner.report.ready, scheduler.rescan.delta) | | 3. Queries user notification preferences | | 4. Renders template (configurable per channel + locale) | | 5. Sends via configured channels: | | +-- Slack: POST to hooks.slack.com | | +-- Teams: POST to graph.microsoft.com | | +-- Email: SMTP send | | +-- Webhook: POST to custom endpoint | | 6. Records delivery receipt | | 7. Idempotency tracking (prevents duplicate sends) | | 8. Retry with deterministic backoff | +--------------------------------------------------------------------------------------------+ ``` --- ## 8. Export Flow ``` User requests export | v +--------------------------------------------------------------------------------------------+ | GET /api/v1/scans/{scan_id}/export?format=spdx | +--------------------------------------------------------------------------------------------+ | v +--------------------------------------------------------------------------------------------+ | Scanner.Web or ExportCenter | | 1. Queries scan metadata from PostgreSQL | | 2. Retrieves SBOM artifact from RustFS | | 3. Signs SBOM with Signer service (DSSE envelope) | | 4. Creates in-toto attestation with Attestor | | 5. Stores final bundle to RustFS | | 6. Returns signed bundle (application/vnd.in-toto+json) | +--------------------------------------------------------------------------------------------+ | v +--------------------------------------------------------------------------------------------+ | For air-gap export: | | ExportCenter bundles: | | - SBOMs (SPDX/CycloneDX) | | - Offline Kit (vulnerability feeds, trust roots, signatures) | | - Evidence bundles (DSSE-signed artifacts) | | Generates mirror artefacts | | Packages for portable/USB distribution | +--------------------------------------------------------------------------------------------+ ``` --- ## API Endpoint Quick Reference | User Action | Endpoint | Module | Schema | |-------------|----------|--------|--------| | **View Dashboard** | `GET /api/console/vuln/facets` | Scanner | `scanner` | | | `GET /api/risk/stats` | RiskEngine | `policy` | | | `GET /api/reachability/summary` | ReachGraph | `scanner` | | **Submit Scan** | `POST /api/v1/scans/` | Scanner | `scanner` | | **Check Status** | `GET /api/v1/scans/{id}` | Scanner | `scanner` | | **Download SBOM** | `GET /api/v1/scans/{id}/sbom` | Scanner | RustFS | | **Upload SBOM** | `POST /api/v1/sboms/upload` | Scanner | `scanner` | | **View Findings** | `GET /api/v1/findings` | Policy | `policy` | | **Create Policy** | `POST /api/v1/policies/` | Policy | `policy` | | **Evaluate Policy** | `POST /api/v1/policies/evaluate` | Policy | `policy` + `vuln` + `vex` | | **Manage Exceptions** | `POST /api/v1/exceptions/` | Policy | `policy` | | **View VEX** | `GET /api/vex/statements` | Excititor | `vex` | | **View Advisories** | `GET /api/advisories` | Concelier | `vuln` | | **Export Evidence** | `GET /api/v1/export/bundle` | ExportCenter | RustFS | --- ## Related Documentation - [Architecture Overview](../../ARCHITECTURE_OVERVIEW.md) - [High-Level Architecture](../../ARCHITECTURE_OVERVIEW.md) - [Data Flows](data-flows.md) - [Schema Mapping](schema-mapping.md) - [Module Matrix](module-matrix.md)