stella-ops.org/git.stella-ops.org
1
0
Fork 0
Code Issues Pull Requests Actions 2 Releases Wiki Activity

Deep security engineer journey: UX findings and product assessment

Browse Source 
Walked through the full vulnerability triage workflow as a security
engineer. Found the artifact workspace — the product's killer feature
(evidence-grade findings with reachability, attestations, policy gating,
delta comparison, deterministic replay, VEX decisions). Recorded a VEX
decision for CVE-2023-38545.

Critical UX findings:
- UX-D1: No "Scan" entry point anywhere in the UI — scanner exists
  (2 containers) but has no discoverable trigger from the console
- UX-D2: Triage workspace (best feature) hidden under "Triage" label —
  security engineers look for "Vulnerabilities" or "Findings"
- UX-D3: Record Decision dialog unreachable on smaller viewports —
  needs proper modal overlay instead of in-page drawer
- UX-D4: Security Posture shows 0 findings while Triage has 1 active
  HIGH finding — different data sources

Assessment: The triage artifact workspace is 10/10 UX. The discoverability
is 2/10. Three changes would transform the security engineer experience.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
master
2026-03-16 10:59:55 +02:00
parent 9586006404
commit a650020311
1 changed files with 113 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

113
docs/qa/DEEP_JOURNEY_UX_FINDINGS_20260316.md Normal file
View File

@@ -0,0 +1,113 @@
# Deep Journey UX Findings — Security Engineer Perspective
**Date**: 2026-03-16
**Persona**: DevOps/Security engineer, first time user
**Goal**: Scan containers for vulnerabilities, audit deployments, gate releases
---
## What Works Exceptionally Well
### Triage Artifact Workspace — 10/10
The artifact detail page (`/triage/artifacts/{name}`) is the best vulnerability triage UX I've seen:
- **Evidence strip**: 7/7 evidence types with checkmarks (Reachability, Call-stack, Provenance, VEX, DSSE, Rekor, SBOM) — instant trust
- **Reachability score**: "Vulnerable code is **reachable** from application entry points. Score: 0.78" — this alone justifies the product
- **Record Decision drawer**: VEX status (Affected/Not Affected/Under Investigation) + reason dropdown + audit summary (CVE, evidence hash, policy version) — proper audit trail
- **Policy gating**: Shows per-gate verdicts (CI Build, Registry Admission, Runtime Admission) with WARN/FAIL
- **Delta comparison**: "+1 new, -0 resolved, ~1 changed" with diff from previous scan
- **Attestations**: Signed DSSE attestation with Rekor transparency log verification
- **Replay command**: `stella triage replay --finding-id <id>` — deterministic reproducibility
- **Download Evidence Bundle** — one-click audit export
- **AI Code Guard**: "Review, 1 finding" status badge — AI assistance built in
- **Explainability panels**: Reason / AI / Provenance / History — decision aids
**This is the product's killer feature. The problem is nobody finds it.**
### Policy Gating Per Finding — 9/10
Each finding shows which gates it passes/fails:
- Vulnerability gate: WARN
- Admission gate: FAIL
- Runtime gate: WARN
Plus the policy explains WHY: "Policy 'runtime-risk-budget-v2' dismissed this finding: low exploitability in the current environment."
This is exactly what a security engineer needs for audit proof.
---
## Critical UX Issues (What's Broken)
### UX-D1: No "Scan" Entry Point — The #1 Missing Feature (CRITICAL)
**What happened**: Searched for "scan" in command palette — 0 results. Looked through all sidebar items — no "Scan" or "Scanner" link. Security Posture page has no scan button. The word "scan" doesn't appear on the Security Posture page.
**Impact**: A security engineer's primary action ("scan my container for vulnerabilities") has no discoverable entry point. The product has a full scanner service (63 containers include `stellaops-scanner-web` and `stellaops-scanner-worker`), but there's no way to trigger it from the UI.
**Solution**:
1. Add "Scan Image" to the command palette quick actions (type `>scan`)
2. Add a "Scan" button on the Security Posture page
3. Add "Scanner" as a sidebar item under Security & Audit
4. On the Integrations Hub (after connecting a registry), show "Scan your first image" CTA
### UX-D2: Triage Is Named Wrong — Security Engineers Look for "Vulnerabilities" (HIGH)
**What happened**: The artifact workspace (the product's best feature) is hidden under "Triage" in the sidebar. A security engineer looks for "Vulnerabilities", "Findings", "Scan Results", or "CVEs" — not "Triage".
**Impact**: The best feature in the product is undiscoverable for the primary persona.
**Solution**:
1. Rename "Triage" to "Vulnerabilities" or "Findings" in the sidebar
2. Keep the internal workflow concept of "triage" but use user-facing language
3. Add "Vulnerabilities" as a search keyword for the command palette
4. Link from Security Posture "Risk Posture" card directly to the artifact workspace
### UX-D3: Record Decision Dialog Can't Be Scrolled To (HIGH)
**What happened**: The "Record Decision" dialog opens at the bottom of the page but Playwright couldn't scroll to it. The dialog is positioned outside the viewport and the page doesn't scroll to it.
**Impact**: On smaller screens or when the evidence panel is expanded, the decision dialog is unreachable.
**Solution**: Make the Record Decision dialog a proper modal overlay (fixed position) instead of an in-page drawer at the bottom.
### UX-D4: Security Posture Shows All Zeros But Triage Has Data (MEDIUM)
**What happened**: Security Posture page shows "Risk Posture: GUARDED, 0 findings" but the Triage page shows 1 active artifact with a HIGH finding (CVE-2023-38545).
**Root cause**: Security Posture reads from a different data path than Triage. The seed data populates the triage workspace but not the security posture read model.
**Impact**: A security engineer sees "0 findings" on the main security page and thinks the system is empty, when it actually has actionable findings in Triage.
**Solution**: Security Posture should aggregate from the same data source as Triage, or at minimum link to Triage with a "1 active finding in triage workspace" indicator.
### UX-D5: No Way to Push an Image to the Built-in Registry (LOW)
**What happened**: The compose stack includes a Zot registry (`stellaops-registry` at `registry.stella-ops.local`) but there's no documentation or UI for pushing an image to it and triggering a scan.
**Impact**: The user can't complete the basic workflow: push image → scan → see vulnerabilities → triage.
**Solution**: Add to quickstart: "Push a test image and scan it" section with:
```bash
docker tag alpine:3.19 registry.stella-ops.local/test/alpine:3.19
docker push registry.stella-ops.local/test/alpine:3.19
# UI: Integrations → Add Registry → Connect to registry.stella-ops.local
# Then trigger scan from Security → Scan Image
```
---
## Summary: What a Security Engineer Experiences
**The Good**: Stella Ops has the deepest vulnerability triage UX I've ever seen. Evidence-grade findings with reachability scores, signed attestations, policy explanations, delta comparisons, and deterministic replay. The Record Decision workflow with VEX status + audit summary is audit-ready.
**The Gap**: None of this is discoverable. The scanner has no UI entry point. The triage workspace is hidden under a workflow term. The security posture page shows 0 when findings exist. A security engineer would leave thinking the product is empty.
**The Fix**: Three changes would transform the experience:
1. Add "Scan Image" button/link to Security Posture and command palette
2. Rename "Triage" to "Vulnerabilities" in the sidebar
3. Connect Security Posture numbers to the same data source as Triage
**Time to value without these fixes**: "I don't know how to scan anything"
**Time to value with these fixes**: "I pushed an image, scanned it, found CVE-2023-38545 with reachability proof, and recorded a VEX decision in 5 minutes"