First-time user experience fixes and platform contract repairs
FTUX fixes (Sprint 316-001): - Remove all hardcoded fake data from dashboard — fresh installs show honest setup guide instead of fake crisis data (5 fake criticals gone) - Curate advisory source defaults: 32 sources disabled by default (ecosystem, geo-restricted, exploit, hardware, mirror). ~43 core sources remain enabled. StellaOps Mirror no longer enabled at priority 1. - Filter Mirror-category sources from Create Domain wizard to prevent circular mirror-from-mirror chains - Add 404 catch-all route — unknown URLs show "Page Not Found" instead of silently rendering the dashboard - Fix arrow characters in release target path dropdown (? → →) - Add login credentials to quickstart documentation - Update Feature Matrix: 14 release orchestration features marked as shipped (was marked planned) Platform contract repairs (from prior session): - Add /api/v1/jobengine/quotas/summary endpoint on Platform - Fix gateway route prefix matching for /policy/shadow/* and /policy/simulations/* (regex routes instead of exact match) - Fix VexHub PostgresVexSourceRepository missing interface method - Fix advisory-vex-sources sweep text expectation - Fix mirror operator journey auth (session storage token extraction) Verified: 110/111 canonical routes passing (1 unrelated stale approval ref) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
master
232
docs/qa/FIRST_TIME_USER_SERIES_20260316.md
Normal file
232
docs/qa/FIRST_TIME_USER_SERIES_20260316.md
Normal file
@@ -0,0 +1,232 @@
|
||||
# First-Time User Audit Series — Stella Ops
|
||||
|
||||
**Date**: 2026-03-16
|
||||
**Method**: Interactive Playwright walkthrough + source code analysis + product deep-dive
|
||||
**Stack**: 63 containers live at `https://stella-ops.local`, demo-prod tenant
|
||||
|
||||
---
|
||||
|
||||
## Part 1: The Product Mental Model
|
||||
|
||||
Stella Ops is a **release control plane** for non-Kubernetes container estates. Every operator interaction answers one of three questions:
|
||||
|
||||
1. **Is it safe?** — Vulnerability findings, SBOM health, reachability evidence, VEX dispositions, advisory feed freshness
|
||||
2. **Is it approved?** — Policy gates passed, human approvals given, evidence sealed
|
||||
3. **Is it working?** — Services healthy, feeds syncing, jobs running, no dead letters
|
||||
|
||||
The UI, documentation, and workflows must be organized around these three pillars.
|
||||
|
||||
---
|
||||
|
||||
## Part 2: Findings (Prioritized)
|
||||
|
||||
### CRITICAL
|
||||
|
||||
#### C1. Dashboard is 100% Hardcoded — Zero Real API Data
|
||||
**Source file**: `dashboard-v3.component.ts:966-1200`
|
||||
|
||||
Every number on the dashboard is fake:
|
||||
- `summary` signal: `{ activePromotions: 3, blockedPromotions: 1 }` — literal
|
||||
- `resolveStatusSeed()`: generates metrics by env type (dev=healthy, staging=degraded, prod+us-east=blocked)
|
||||
- `reachabilityStats`: `{ bCoverage: 72, iCoverage: 88, rCoverage: 61 }` — literal
|
||||
- `nightlyOpsSignals`: 4 hardcoded items
|
||||
- Alerts: 3 hardcoded `<li>` HTML elements
|
||||
- Activity: 3 hardcoded cards
|
||||
- **Not a single API call to any backend service**
|
||||
|
||||
The dashboard tells you "5 critical findings, blocked" but security posture shows 0 findings. This destroys trust immediately.
|
||||
|
||||
**Solution**: [See Part 3 — Dashboard Redesign]
|
||||
|
||||
#### C2. StellaOps Mirror Source Enabled at Priority 1 on Every Fresh Install
|
||||
**Source file**: `SourceDefinitions.cs:1351-1366`
|
||||
|
||||
The `StellaMirror` source definition:
|
||||
```csharp
|
||||
BaseEndpoint = "https://mirror.stella-ops.org/api/v1" // nonexistent external URL
|
||||
DefaultPriority = 1 // HIGHEST priority — overrides all other sources
|
||||
EnabledByDefault = true // inherited, always on
|
||||
```
|
||||
|
||||
On every fresh self-hosted install, the highest-priority advisory source points to a URL that doesn't exist. Health check returns "SSL/TLS error." The system is in "Direct" mode but its top-priority source is a mirror.
|
||||
|
||||
**Solution**: Set `EnabledByDefault = false` on StellaMirror. Only enable when mirror consumer wizard is completed.
|
||||
|
||||
#### C3. Mirror Source, Mirror Mode, and Mirror Consumer Wizard Are Disconnected
|
||||
Three independent systems that should be one:
|
||||
1. **Catalog toggle**: Enable "StellaOps Mirror" → hardcoded to `mirror.stella-ops.org`
|
||||
2. **Mode badge**: Shows "Direct" regardless of which sources are enabled
|
||||
3. **Consumer wizard**: "Connect to Mirror" → 4-step wizard with custom URL
|
||||
|
||||
Enabling the catalog source doesn't invoke the wizard. Completing the wizard doesn't update the catalog source. The mode badge doesn't reflect reality.
|
||||
|
||||
**Solution**: Unify — catalog mirror toggle should redirect to consumer wizard. Wizard completion should update source endpoint + enable source + set mode.
|
||||
|
||||
#### C4. 74 of 75 Advisory Sources Enabled by Default — 18 Fail Immediately
|
||||
`EnabledByDefault = true` in the base `SourceDefinition` class. Every source — including geo-restricted (FSTEC Russia, NKCKI Russia), ecosystem-specific (npm, PyPI, Maven, etc.), and the broken mirror — is enabled on fresh install.
|
||||
|
||||
After "Check All": 55 healthy, 18 failed. Stats bar shows confusing "55 enabled" (doesn't match 74 actually enabled).
|
||||
|
||||
**Solution**: Curate defaults. Enable ~25 core sources (Primary + major vendors + major distributions). Disable ecosystem, geo-restricted, hardware, and mirror by default.
|
||||
|
||||
### HIGH
|
||||
|
||||
#### H1. Login Credentials Not Documented Anywhere
|
||||
Quickstart says "Open https://stella-ops.local" — never mentions username/password. User must find `S001_demo_seed.sql` to discover `admin / Admin@Stella2026!`.
|
||||
|
||||
**Solution**: Add credentials to quickstart.md step 5. Add hint to the Welcome page.
|
||||
|
||||
#### H2. Registry Search Returns Mock Results Despite API Failures
|
||||
In release creation wizard, searching "nginx" returns "nginx-service" at `registry.internal/nginx-service` with fake digest `sha256:nginx1234567...`. Console shows 4 errors: `Failed to load resource: /api/registry/images/search`. The mock fallback silently produces results from a nonexistent registry.
|
||||
|
||||
**Solution**: When registry API fails, show error — not mock results. Don't allow sealing releases with mock artifacts.
|
||||
|
||||
#### H3. Create Mirror Domain Wizard Allows Selecting Mirror Source
|
||||
The source picker in Create Domain includes "StellaOps Mirror" under Mirror (1) category. This creates a mirror domain that sources from a nonexistent upstream mirror — circular/broken dependency.
|
||||
|
||||
**Solution**: Filter out `category === 'Mirror'` from the Create Domain source picker.
|
||||
|
||||
#### H4. AuthRef URI Required — No Plain Credentials for Evaluation
|
||||
Harbor registry wizard requires `authref://vault/path#secret`. No vault exists on a fresh install. No way to enter username:password for evaluation.
|
||||
|
||||
**Solution**: Allow `authref://env/VARIABLE_NAME` or `authref://inline/base64` for dev/eval. Document authref format with examples.
|
||||
|
||||
### MEDIUM
|
||||
|
||||
#### M1. Bundle Version Shows UUID Instead of Release Name
|
||||
After sealing "API Gateway v2.1", the page shows `a4451892-b6b2-41f6-8539-7b9c6d33d140 v1` as the heading.
|
||||
|
||||
**Solution**: Show human-readable name as primary heading, UUID as metadata.
|
||||
|
||||
#### M2. "Created by" Shows User ID Hash, Not Username
|
||||
Bundle version shows `b08639745d6549348d843aa311c98958` instead of "admin".
|
||||
|
||||
**Solution**: Resolve user IDs to display names in read-model projections.
|
||||
|
||||
#### M3. Context Controls Race Condition on First Load
|
||||
Topbar shows "No regions defined" for 2-3 seconds, then self-heals to "4 regions". Dashboard below shows environment cards immediately (from seed data).
|
||||
|
||||
**Solution**: Show loading skeleton for context controls until inventory loads.
|
||||
|
||||
#### M4. Doctor Quick Check Skips Database Checks
|
||||
`check.db.connection` and `check.db.latency` show "Skipped — not applicable" despite Postgres being the primary database.
|
||||
|
||||
**Solution**: Always include DB connection check in Quick Check.
|
||||
|
||||
#### M5. Doctor Stream Endpoint Returns Console Error
|
||||
After Quick Check, console shows `Failed to load resource: /api/v1/doctor/run/.../stream`. Results still render via fallback.
|
||||
|
||||
**Solution**: Fix streaming endpoint or suppress console error when fallback succeeds.
|
||||
|
||||
#### M6. Events Status Flickers "DEGRADED" → "CONNECTED" on Login
|
||||
First 2-3 seconds show "Events: DEGRADED" (yellow), then settles to "CONNECTED" (green).
|
||||
|
||||
**Solution**: Show "Connecting..." during initial establishment, not "DEGRADED".
|
||||
|
||||
#### M7. Advisory Sources Show "stale" on Fresh Install (Never Synced)
|
||||
Security Posture: "GitHub Advisory Database — stale", "NVD — stale". These have never been synced.
|
||||
|
||||
**Solution**: Show "Not synced" for sources that have never fetched. Reserve "stale" for sources whose last sync exceeds freshness threshold.
|
||||
|
||||
#### M8. Feature Matrix Shows Release Orchestration as "Planned" (⏳) — But It's Built
|
||||
`FEATURE_MATRIX.md` lists Environment CRUD, Release Bundles, Promotion Workflows as ⏳. The live UI has working releases, approvals, promotions. A buyer would think releases don't work.
|
||||
|
||||
**Solution**: Update FEATURE_MATRIX.md to reflect current implementation status.
|
||||
|
||||
#### M9. Stats Bar Count Mismatch
|
||||
After Check All, stats show "55 enabled" but 74 sources are actually enabled (toggles ON). The "enabled" count in the stats bar doesn't match the source state.
|
||||
|
||||
**Solution**: Fix stats computation to count actual enabled sources, not only those with successful health checks.
|
||||
|
||||
### LOW
|
||||
|
||||
#### L1. Arrow Character Not Rendering in Target Path
|
||||
Release wizard dropdown shows "Dev ? Stage ? Prod" instead of "Dev → Stage → Prod".
|
||||
|
||||
**Solution**: Use `\u2192` or HTML entity `→`.
|
||||
|
||||
#### L2. No 404 Page — Unknown Routes Show Dashboard
|
||||
`/nonexistent-route-12345` renders the Dashboard instead of "Page Not Found".
|
||||
|
||||
**Solution**: Add wildcard catch-all route with a proper 404 component.
|
||||
|
||||
---
|
||||
|
||||
## Part 3: Dashboard Redesign
|
||||
|
||||
### Layout: 3-Column Mission Board
|
||||
|
||||
```
|
||||
┌──────────────────────────────────────────────────────────────┐
|
||||
│ Mission Board — [Demo Production] [↻ Refresh] │
|
||||
│ Last updated: 15 Mar 2026, 23:15 UTC │
|
||||
├──────────────────┬───────────────────────────────────────────┤
|
||||
│ │ │
|
||||
│ SECURITY POSTURE │ ENVIRONMENTS & ACTIONS (2/3) │
|
||||
│ (1/3) │ │
|
||||
│ │ ┌───────────────────────────────────────┐ │
|
||||
│ ┌──────────────┐ │ │ PROMOTION PIPELINE │ │
|
||||
│ │ Vuln Summary │ │ │ [env cards: Dev→Stage→Prod by region] │ │
|
||||
│ │ C:12 H:34 │ │ │ blocked envs highlighted at top │ │
|
||||
│ │ M:89 L:156 │ │ └───────────────────────────────────────┘ │
|
||||
│ │ [donut chart] │ │ │
|
||||
│ │ Reachable: 9 │ │ ┌───────────────────────────────────────┐ │
|
||||
│ │ Unknown: 23 │ │ │ NEEDS ATTENTION │ │
|
||||
│ └──────────────┘ │ │ ⚠ 3 approvals blocked │ │
|
||||
│ │ │ ⚠ 2 waivers expiring │ │
|
||||
│ ┌──────────────┐ │ │ 🔴 Feed freshness degraded │ │
|
||||
│ │ SBOM Health │ │ └───────────────────────────────────────┘ │
|
||||
│ │ 247 comps │ │ │
|
||||
│ │ 231 fresh │ │ ┌───────────────────────────────────────┐ │
|
||||
│ │ B:72% I:88% │ │ │ LIVE ACTIVITY │ │
|
||||
│ │ R:61% │ │ │ • admin sealed "API Gateway v2.1" │ │
|
||||
│ └──────────────┘ │ │ • NVD synced (142 new advisories) │ │
|
||||
│ │ │ • Doctor: 7 pass, 1 warn, 1 fail │ │
|
||||
│ ┌──────────────┐ │ └───────────────────────────────────────┘ │
|
||||
│ │ Feed Status │ │ │
|
||||
│ │ 55/75 active │ │ │
|
||||
│ │ 18 failed │ │ │
|
||||
│ └──────────────┘ │ │
|
||||
├──────────────────┴───────────────────────────────────────────┤
|
||||
│ PLATFORM: 63 svcs ✓ │ DB:OK │ Events:ON │ Doctor:7/1/1 │
|
||||
│ Feed:Live │ Evidence:ON │ Offline:OK │ DLQ:3 pending │
|
||||
└──────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
### Empty State (Fresh Install)
|
||||
|
||||
When no real data exists, show setup guidance — not fake numbers:
|
||||
|
||||
```
|
||||
Welcome to Stella Ops — Release Control Plane
|
||||
|
||||
① Connect a registry [Setup Integrations →]
|
||||
② Define your topology [Topology Wizard →]
|
||||
③ Scan your first image [Start Scan →]
|
||||
④ Create a release [Create Release →]
|
||||
|
||||
Platform Health: 63/63 services ✓
|
||||
Advisory Sources: 55/75 healthy [Configure →]
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Part 4: Solutions Implementation Map
|
||||
|
||||
| # | Fix | Effort | Files |
|
||||
|---|-----|--------|-------|
|
||||
| C2 | Mirror source `EnabledByDefault = false` | S | `SourceDefinitions.cs` |
|
||||
| C4 | Curate default sources (~25 instead of 74) | S | `SourceDefinitions.cs` |
|
||||
| H1 | Add credentials to quickstart | S | `docs/quickstart.md` |
|
||||
| H3 | Filter mirror sources from domain builder | S | `mirror-domain-builder.component.ts` |
|
||||
| M8 | Update Feature Matrix status markers | S | `docs/FEATURE_MATRIX.md` |
|
||||
| L1 | Fix arrow character encoding | S | Release version form component |
|
||||
| L2 | Add 404 catch-all route | S | `app.routes.ts` |
|
||||
| M3 | Loading skeleton for context controls | S | `app-topbar.component.ts` |
|
||||
| M6 | "Connecting..." instead of "DEGRADED" | S | topbar events status |
|
||||
| M7 | "Not synced" vs "stale" for fresh sources | S | `advisory-source-catalog.component.ts` |
|
||||
| M9 | Fix stats bar enabled count | S | `advisory-source-catalog.component.ts` |
|
||||
| C1/Phase1 | Dashboard honest empty state | M | `dashboard-v3.component.ts` |
|
||||
| C3 | Unify mirror toggle → consumer wizard | M | catalog + wizard components |
|
||||
| H2 | Registry search error instead of mock | M | release version wizard |
|
||||
| C1/Phase2 | Dashboard 3-column layout with real APIs | L | `dashboard-v3.component.ts` + API clients |
|
||||
Reference in New Issue
Block a user