Repair first-time identity and trust operator journeys
This commit is contained in:
master
313
docs/qa/FIRST_TIME_USER_UX_AUDIT_20260315.md
Normal file
313
docs/qa/FIRST_TIME_USER_UX_AUDIT_20260315.md
Normal file
@@ -0,0 +1,313 @@
|
||||
# First-Time User UX Audit - 2026-03-15
|
||||
|
||||
**Auditor**: AI agent acting as first-time platform administrator
|
||||
**Stack**: Live local (stella-ops.local), logged in as admin/Admin@Stella2026!
|
||||
**Scope**: Every sidebar route and sub-tab, exercising create/edit/detail flows
|
||||
**Sprint context**: SPRINT_20260315_003 (Identity/Trust) and SPRINT_20260315_004 (Integrations)
|
||||
|
||||
---
|
||||
|
||||
## Summary
|
||||
|
||||
- **Total routes inspected**: 40+
|
||||
- **P0 (critical UX blockers)**: 6
|
||||
- **P1 (significant UX gaps)**: 14
|
||||
- **P2 (moderate issues)**: 22
|
||||
- **P3 (minor polish)**: 12
|
||||
- **Cross-cutting patterns**: 10
|
||||
|
||||
---
|
||||
|
||||
## P0 - Critical UX Blockers
|
||||
|
||||
### P0-1. Role permissions are free-text with no discoverability
|
||||
**Route**: `/setup/identity-access` > Roles tab > + Create Role
|
||||
**Detail**: The "Permissions" field is a plain `<textarea>` with placeholder `findings:read, vex:read, vuln:investigate`. A first-time user has zero way to discover what scopes are available. No autocomplete, no dropdown, no checkbox list, no catalog, no link to docs. The system has 149 scope permissions but none are surfaced in the UI.
|
||||
**Impact**: Role creation is impossible without internal knowledge or reading source code.
|
||||
**Recommendation**: Replace with a grouped checkbox/chip picker organized by domain (Security, Releases, Evidence, Ops, Setup) with descriptions per scope.
|
||||
|
||||
### P0-2. No role detail view exists
|
||||
**Route**: `/setup/identity-access` > Roles tab
|
||||
**Detail**: Clicking any role row (admin, operator, viewer) does nothing. No detail panel, no side drawer, no navigation. The only visible info is Name, Description ("Full platform access"), Users count, and Built-in flag. You cannot see what permissions any role actually grants.
|
||||
**Impact**: An operator cannot audit or understand existing role configurations.
|
||||
|
||||
### P0-3. No scope catalog or reference anywhere in the app
|
||||
**Route**: Global
|
||||
**Detail**: No "Available Permissions" page, no docs link, no in-app reference. The scope naming convention (module:action) is never explained. A new operator is completely blind.
|
||||
|
||||
### P0-4. "Create Release" header button leads to blank page
|
||||
**Route**: `/releases/versions/new`
|
||||
**Detail**: The primary "Create Release" CTA in the global header navigates to a page with a completely empty `<main>` element. No form, no heading, no description. This is the single most important action button in the product.
|
||||
**Impact**: The most prominent action in the UI is a dead end.
|
||||
|
||||
### P0-5. Promotions page is completely blank
|
||||
**Route**: `/releases/promotions`
|
||||
**Detail**: The `<main>` element has zero children. No heading, no description, no empty state, no guidance. Additionally, the global context bar shows "No regions defined" and "No env defined yet" (disabled buttons), and "Events: DEGRADED".
|
||||
**Impact**: A core release workflow page is a dead end.
|
||||
|
||||
### P0-6. Operations landing page is completely blank
|
||||
**Route**: `/ops/operations`
|
||||
**Detail**: The `<main>` element has zero children. This is the parent landing page for 7 sub-pages (Scheduled Jobs, Signals, Offline Kit, etc.).
|
||||
**Impact**: Users navigating from the sidebar hit a dead end.
|
||||
|
||||
---
|
||||
|
||||
## P1 - Significant UX Gaps
|
||||
|
||||
### P1-1. Role descriptions too vague to be actionable
|
||||
**Route**: `/setup/identity-access` > Roles tab
|
||||
**Detail**: Built-in roles show only one-line descriptions: "Full platform access" (admin), "Release and deployment operations" (operator), "Read-only access" (viewer). These tell you nothing about specific capabilities. Can an operator approve promotions? Can a viewer see security findings? No way to know.
|
||||
|
||||
### P1-2. Role dropdown defaults to admin (most privileged)
|
||||
**Route**: `/setup/identity-access` > Users tab > + Add User
|
||||
**Detail**: The Role `<select>` pre-selects "admin". Security-by-default should pre-select "viewer" (least privilege).
|
||||
|
||||
### P1-3. No role descriptions in the user creation dropdown
|
||||
**Route**: `/setup/identity-access` > Users tab > + Add User
|
||||
**Detail**: The role dropdown shows bare names (admin, operator, viewer, qa-role-...) with no hint about what each grants.
|
||||
|
||||
### P1-4. No delete/deactivate for users
|
||||
**Route**: `/setup/identity-access` > Users tab
|
||||
**Detail**: No delete, deactivate, or disable button on user rows. No row actions at all. No right-click context menu.
|
||||
|
||||
### P1-5. No delete for roles
|
||||
**Route**: `/setup/identity-access` > Roles tab
|
||||
**Detail**: No delete button on role rows. Custom roles with 0 users cannot be removed.
|
||||
|
||||
### P1-6. No delete/archive for tenants
|
||||
**Route**: `/setup/identity-access` > Tenants tab
|
||||
**Detail**: No delete or archive button. Tenants with 0 users are stuck permanently.
|
||||
|
||||
### P1-7. No edit action on any Identity entity
|
||||
**Route**: `/setup/identity-access` (all tabs)
|
||||
**Detail**: Cannot change a user's role after creation. Cannot edit a role's permissions. Cannot rename a tenant. Click-through on any row does nothing. The only workflow is create-and-forget.
|
||||
|
||||
### P1-8. Signing key Rotate/Revoke uses raw browser prompt()
|
||||
**Route**: `/setup/trust-signing/keys`
|
||||
**Detail**: Clicking "Rotate" shows a native browser `window.prompt("Rotation reason for...")` dialog. Same for "Revoke". These should be proper in-app modal dialogs with confirmation, reason field, and impact warning.
|
||||
**Impact**: Destructive actions on cryptographic keys should never use browser prompts.
|
||||
|
||||
### P1-9. Trust Analytics tab is broken (API errors)
|
||||
**Route**: `/setup/trust-signing/analytics`
|
||||
**Detail**: Shows "Failed to load analytics data. Please try again." Console errors: `/api/v1/trust/analytics/summary` and `/api/v1/trust/analytics/verification` return server errors.
|
||||
|
||||
### P1-10. No setup onboarding wizard or getting-started flow
|
||||
**Route**: Global / Setup section
|
||||
**Detail**: No guided first-time setup flow. A new operator lands on the dashboard and must discover Setup sections manually from the sidebar. Should have a checklist: 1) Configure identity, 2) Set up trust, 3) Add integrations, 4) Review topology.
|
||||
|
||||
### P1-11. Security page naming chaos (3 different names)
|
||||
**Route**: `/security`
|
||||
**Detail**: Sidebar says "Vulnerabilities", breadcrumb says "Risk Overview", H1 says "Security / Posture". Three different names for the same page creates significant confusion.
|
||||
|
||||
### P1-12. /security and /security/posture are near-duplicate pages
|
||||
**Route**: `/security` and `/security/posture`
|
||||
**Detail**: Both render with H1 "Security / Posture", same structure. Two sidebar items leading to essentially the same page. Additionally, `/security/posture` appears stuck in "Loading security overview..." state.
|
||||
|
||||
### P1-13. Trusted Issuers have no actions at all
|
||||
**Route**: `/setup/trust-signing/issuers`
|
||||
**Detail**: Unlike Signing Keys (which have View/Rotate/Revoke), the Issuers table has zero actions. The only issuer ("Demo Prod Root CA") shows trust level "Untrusted" but there is no way to change it, promote it, block it, or remove it.
|
||||
|
||||
### P1-14. No password/credential explanation in Add User
|
||||
**Route**: `/setup/identity-access` > Users tab > + Add User
|
||||
**Detail**: Form has Username, Email, Display Name, Role. No password field. No explanation of how the new user gets their credentials (email invite? default password? OIDC redirect only?). A first-time admin would be confused.
|
||||
|
||||
---
|
||||
|
||||
## P2 - Moderate Issues
|
||||
|
||||
### P2-1. No search or filter on Identity tables
|
||||
**Route**: `/setup/identity-access` (Users, Roles, Tenants tabs)
|
||||
**Detail**: No search box, no filter dropdowns on any of the three tables.
|
||||
|
||||
### P2-2. No sorting controls on Identity tables
|
||||
**Route**: `/setup/identity-access` (all tabs)
|
||||
**Detail**: Column headers are not sortable. Built-in roles are alphabetically mixed with custom roles.
|
||||
|
||||
### P2-3. Tenant "Lifecycle" column shows identical boilerplate
|
||||
**Route**: `/setup/identity-access` > Tenants tab
|
||||
**Detail**: Every tenant shows the same text: "Branding and policies are managed from the canonical setup surfaces." This is not lifecycle information.
|
||||
|
||||
### P2-4. Tenant Isolation Mode not explained
|
||||
**Route**: `/setup/identity-access` > Tenants > + Add Tenant
|
||||
**Detail**: "Shared" / "Dedicated" dropdown with zero tooltip or description.
|
||||
|
||||
### P2-5. OAuth Clients tab is a dead end
|
||||
**Route**: `/setup/identity-access` > OAuth Clients tab
|
||||
**Detail**: Shows disclaimer: "registration and secret rotation remain outside this setup tab until the full guided flow is shipped" but doesn't tell the user WHERE to do these things.
|
||||
|
||||
### P2-6. API Tokens tab is a dead end
|
||||
**Route**: `/setup/identity-access` > API Tokens tab
|
||||
**Detail**: Says "Token issuance and revocation are not exposed on this setup route yet" with no alternative path.
|
||||
|
||||
### P2-7. Certificates show raw UUIDs for Issuer/Key References
|
||||
**Route**: `/setup/trust-signing/certificates`
|
||||
**Detail**: Shows `4ac7e1d4-7a2e-4b4d-9e12-5d42e3168a91` instead of human-readable names. Operator must mentally cross-reference.
|
||||
|
||||
### P2-8. Certificate "Expiring Soon" with no action
|
||||
**Route**: `/setup/trust-signing/certificates`
|
||||
**Detail**: Certificate shows "Expiring Soon" (Mar 18, 3 days away) but has no Renew/Rotate button or urgency banner.
|
||||
|
||||
### P2-9. Two notification surfaces with no cross-link
|
||||
**Route**: `/setup/notifications` and `/ops/operations/notifications`
|
||||
**Detail**: Setup Notifications (rich admin with Rules/Channels/Templates/Simulator/Config) vs Ops Notifications (inline forms). No explanation of which is canonical. Sidebar places "Notifications" under Operations, not under Setup.
|
||||
|
||||
### P2-10. Ops Notifications: "Refresh data" button permanently disabled
|
||||
**Route**: `/ops/operations/notifications`
|
||||
**Detail**: Header refresh button is disabled with no tooltip explanation.
|
||||
|
||||
### P2-11. Ops Notifications: Channel dropdown in Rules is empty
|
||||
**Route**: `/ops/operations/notifications`
|
||||
**Detail**: The "Channel" combobox in rule creation shows no options because no channels exist, but there's no "create a channel first" guidance.
|
||||
|
||||
### P2-12. Branding page has no tenant selector
|
||||
**Route**: `/setup/tenant-branding`
|
||||
**Detail**: Branding edits apply to globally selected tenant (from header bar), but this is never stated.
|
||||
|
||||
### P2-13. Branding "Border Colors" and "Status Colors" sections empty
|
||||
**Route**: `/setup/tenant-branding`
|
||||
**Detail**: Section headers appear but no input fields beneath them (unlike Background/Text/Brand).
|
||||
|
||||
### P2-14. All 10 topology environments show "degraded" with no explanation
|
||||
**Route**: `/setup/topology/overview`
|
||||
**Detail**: Overview shows "Healthy 0 / Degraded 10 / Unhealthy 0". Every environment is degraded but no explanation of why or what to fix.
|
||||
|
||||
### P2-15. All topology environments show "0 targets"
|
||||
**Route**: `/setup/topology/overview` and `/setup/topology/targets`
|
||||
**Detail**: Every environment lists 0 targets, yet the dashboard shows active deployments. Mismatch confuses operators.
|
||||
|
||||
### P2-16. System health falsely claims "All systems operational"
|
||||
**Route**: `/setup/system`
|
||||
**Detail**: The Health Check card says "All systems operational" without running any checks. False-positive confidence signal.
|
||||
|
||||
### P2-17. Integrations "Activity stream is coming soon" placeholder
|
||||
**Route**: `/setup/integrations` > Hub tab
|
||||
**Detail**: Recent Activity section shows a "coming soon" stub.
|
||||
|
||||
### P2-18. Failed integration has no visible error message in list
|
||||
**Route**: `/setup/integrations/registries`
|
||||
**Detail**: "QA Harbor" shows Status: Failed, Health: Unhealthy but no inline error message or troubleshooting link in the table row.
|
||||
|
||||
### P2-19. /security/unknowns swallows API errors silently
|
||||
**Route**: `/security/unknowns`
|
||||
**Detail**: 4 console errors (500s from `/api/v1/scanner/unknowns` and `/api/v1/scanner/unknowns/stats`) but UI shows clean empty table.
|
||||
|
||||
### P2-20. Security Reports: Risk Report tab shows embedded triage view
|
||||
**Route**: `/security/reports`
|
||||
**Detail**: The "Risk Report" tab renders the Artifact Triage workspace inside Reports instead of an actual report. Confusing overlap.
|
||||
|
||||
### P2-21. Offline Kit: activity contradicts stats
|
||||
**Route**: `/ops/operations/offline-kit`
|
||||
**Detail**: Recent Activity shows "Loaded offline bundle v2025.01.15" and "Verified 45 assets" but stats show 0 bundles loaded and 0 assets verified.
|
||||
|
||||
### P2-22. JobEngine: console DENY contradicts displayed "Granted"
|
||||
**Route**: `/ops/operations/jobengine`
|
||||
**Detail**: Console logs `[TenantAuth] DENY: jobengine:operate` but the UI shows "Operate Jobs: Granted".
|
||||
|
||||
---
|
||||
|
||||
## P3 - Minor Polish
|
||||
|
||||
### P3-1. Header action button changes contextually without explanation
|
||||
**Route**: Global
|
||||
**Detail**: Top-right button alternates between "ADD TARGET", "Add Integration", "Create Release", "Create Hotfix", "Export Report" depending on page. No visual cue explaining the context.
|
||||
|
||||
### P3-2. Trust overview cards show developer jargon
|
||||
**Route**: `/setup/trust-signing`
|
||||
**Detail**: Subtitles like "Administration inventory projection" and "Routed from live administration projection" are technical jargon, not helpful to operators.
|
||||
|
||||
### P3-3. Trust disclaimer banners read as developer notes
|
||||
**Route**: `/setup/trust-signing` (multiple tabs)
|
||||
**Detail**: "Usage statistics, fingerprint material, and expiry policy are not part of the current administration contract..." should be hidden or rephrased.
|
||||
|
||||
### P3-4. No "Add" buttons for trust entities
|
||||
**Route**: `/setup/trust-signing` (Signing Keys, Issuers, Certificates tabs)
|
||||
**Detail**: No way to add a new signing key, issuer, or certificate from the UI. Only Watchlist has a create action.
|
||||
|
||||
### P3-5. Inconsistent page heading capitalization
|
||||
**Route**: Multiple
|
||||
**Detail**: "Artifact workspace" (lowercase w), "Audit bundles" (lowercase b) vs "Security Reports", "Export Center" (title case).
|
||||
|
||||
### P3-6. Replay & Verify naming mismatch
|
||||
**Route**: `/evidence/verify-replay`
|
||||
**Detail**: Sidebar says "Replay & Verify", page title says "Verify & Replay", H1 says "Verdict Replay". Three names.
|
||||
|
||||
### P3-7. Health page naming mismatch
|
||||
**Route**: `/releases/health`
|
||||
**Detail**: Sidebar says "Health", H1 says "Environment Posture". Description text "Environment . region" looks like a template placeholder.
|
||||
|
||||
### P3-8. Topology map labels truncated
|
||||
**Route**: `/setup/topology/map`
|
||||
**Detail**: Labels like "Production US E..." and "Production EU W..." cut off without tooltips.
|
||||
|
||||
### P3-9. Decision Capsules uses H2 instead of H1
|
||||
**Route**: `/evidence/capsules`
|
||||
**Detail**: Uses H2 for main heading, inconsistent with all other pages that use H1.
|
||||
|
||||
### P3-10. Audit Bundles timestamps in raw ISO format
|
||||
**Route**: `/triage/audit-bundles`
|
||||
**Detail**: Timestamps not human-friendly. Full SHA-256 hashes shown without truncation or copy button.
|
||||
|
||||
### P3-11. Signals: 60% error rate with no alert styling
|
||||
**Route**: `/ops/operations/signals`
|
||||
**Detail**: Metrics show "Error rate: 60%" in normal styling. Should use warning/error visual treatment.
|
||||
|
||||
### P3-12. Export button disabled before diagnostics run
|
||||
**Route**: `/ops/operations/doctor`
|
||||
**Detail**: No tooltip explaining "Run a check first to enable export".
|
||||
|
||||
---
|
||||
|
||||
## Cross-Cutting Patterns
|
||||
|
||||
### CC-1. Naming inconsistency is the single worst pattern
|
||||
At least 4 routes have different names in sidebar, breadcrumb, page title, and H1. Examples:
|
||||
- `/security`: sidebar="Vulnerabilities", breadcrumb="Risk Overview", H1="Security / Posture"
|
||||
- `/evidence/verify-replay`: sidebar="Replay & Verify", title="Verify & Replay", H1="Verdict Replay"
|
||||
- `/releases/health`: sidebar="Health", H1="Environment Posture"
|
||||
|
||||
### CC-2. Three completely blank pages
|
||||
`/releases/promotions`, `/releases/versions/new`, `/ops/operations` render empty `<main>` elements with zero content.
|
||||
|
||||
### CC-3. Context bar inconsistency
|
||||
Some routes show "4 regions" / "All environments" (working), others show "No regions defined" / "No env defined yet" (disabled). Depends on whether URL includes `?regions=...` query params.
|
||||
|
||||
### CC-4. Missing empty-state guidance
|
||||
When tables are empty, most show "No X found" but don't explain how to populate them (how to ingest SBOMs, create capsules, generate audit events).
|
||||
|
||||
### CC-5. Events stream status flickers
|
||||
Some pages show "Events: CONNECTED", others "Events: DEGRADED" in the same session.
|
||||
|
||||
### CC-6. Stale timestamps with no visual warning
|
||||
Triage artifact last scan from 2024, Signal probe data from 6 days ago -- no staleness indicators.
|
||||
|
||||
### CC-7. Developer tooling exposed to end users
|
||||
Evidence Overview has "State mode" buttons (Normal/Degraded/Empty) that appear to be developer/demo tooling.
|
||||
|
||||
### CC-8. Duplicate pages in sidebar
|
||||
`/security` and `/security/posture` are near-duplicates with the same H1.
|
||||
|
||||
### CC-9. Operator/Admin toggle unexplained
|
||||
Evidence Overview and Export Center have an Operator/Admin mode toggle with no explanation of what changes.
|
||||
|
||||
### CC-10. Console errors silently swallowed
|
||||
Multiple pages have backend API errors (500s) but show clean empty tables instead of error states.
|
||||
|
||||
---
|
||||
|
||||
## Priority Matrix
|
||||
|
||||
| Priority | Count | Key Theme |
|
||||
|----------|-------|-----------|
|
||||
| P0 | 6 | Blank pages, scope discoverability |
|
||||
| P1 | 14 | Missing CRUD, broken features, no guidance |
|
||||
| P2 | 22 | Dead ends, contradictions, missing explanations |
|
||||
| P3 | 12 | Naming, polish, minor inconsistencies |
|
||||
| Cross-cutting | 10 | Naming chaos, empty states, context bar |
|
||||
|
||||
## Top 5 Actions for Maximum Self-Serve Impact
|
||||
|
||||
1. **Scope picker for role creation** - Replace free-text permissions with grouped checkbox picker + descriptions
|
||||
2. **Fix the 3 blank pages** - `/releases/promotions`, `/releases/versions/new`, `/ops/operations`
|
||||
3. **Add role detail view** - Click-through on role rows showing all assigned scopes
|
||||
4. **Add edit/delete on Identity entities** - Users, Roles, Tenants need full CRUD
|
||||
5. **Unify naming** - Each page should have ONE name used consistently in sidebar, breadcrumb, title, and H1
|
||||
Reference in New Issue
Block a user