test(web): behavioral QA of Evidence/Ops/Setup/Admin surfaces (SPRINT_20260421_007)

Closes SPRINT_20260421_007 — all 4 tasks DONE. Full Tier 2c behavioral
verification per docs/qa/feature-checks/FLOW.md. 34 assertions, 0 fail,
0 deferred.

FE-QA-EVID-001 — Evidence: 7/7 PASS
/evidence/{overview, audit-log, verify-replay, exports, capsules, proofs,
bundles}. Alias chains to /ops/operations/audit confirmed intentional per
evidence.routes.ts.

FE-QA-OPS-002 — Ops: 8/8 PASS
/ops/operations/{jobengine, feeds-airgap, doctor, audit, notifications,
health-slo, watchlist} + /ops/scripts. Doctor full diagnostics grid
rendered with real data.

FE-QA-SETUP-003 — Setup + Admin: 12 + 7 PASS
Setup: /setup{, /integrations, /trust-signing (+ issuers/keys/certificates
/audit sub-tabs aliased correctly), /identity-providers, /tenant-branding,
/workflows, /ai-preferences, /topology}.
Admin: all /console-admin/{tenants, users, roles, clients, audit, branding,
assistant} preserved console origin under "Console Administration" heading.

FE-QA-EVIDOPS-004 — Retention coverage:
New e2e/routes/sprint-007-evidence-ops-setup-admin.e2e.spec.ts with 27
Playwright assertions covering origin, canonical-or-alias URL, and
identity-matching body text. Uses the existing auth.fixture.ts pattern.

Evidence: docs/qa/feature-checks/runs/web/sprint-007-evidence-ops-setup-admin/
run-001/ (EVIDENCE.md + tier2-ui-check.json + 36 screenshots + verify.mjs).

Authority default-tenant gap (same as FE-QA-REL-001 discovery):
stellaops_authority had zero tenants and zero users; setup wizard admin
bootstrap failed with users_tenant_id_fkey FK violation. Worked around
in-session by inserting `installation` + `default` tenants and calling
POST /api/v1/setup/sessions/{id}/steps/admin/execute. This is the same
bug two parallel agents independently hit — needs a real Authority sprint
to seed `default` through migrations or StandardPluginRegistrar init.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
master
2026-04-22 17:19:51 +03:00
parent fd5ac22afb
commit 6baff5764d
4 changed files with 2739 additions and 13 deletions

View File

@@ -0,0 +1,95 @@
# Sprint 20260421_007_FE - Evidence Ops Setup Admin Console Behavioral QA
## Topic & Scope
- Execute the next behavioral QA pass for Evidence, Ops, Setup, and Admin surfaces.
- Confirm that audit, replay, feed, diagnostics, trust, integrations, and admin entry points remain truthful and reachable.
- Fix Web-only regressions discovered during the pass, including route identity, tab ownership, and broken handoffs.
- Working directory: `src/Web/StellaOps.Web/`.
- Expected evidence: fresh Playwright route and tab artifacts, confirmed defects or fixes, and docs updates when ownership changes.
## Dependencies & Concurrency
- Depends on `SPRINT_20260421_005_FE_console_route_identity_and_redirect_truth.md`.
- Depends on `docs/qa/console-ui-traversal-map.md` and `docs/qa/console-ui-qa-strategy.md`.
- Safe parallelism: can run in parallel with Sprint 006 once Sprint 005 has stabilized the core route contract.
## Documentation Prerequisites
- `docs/qa/console-ui-traversal-map.md`
- `docs/qa/console-ui-qa-strategy.md`
- `docs/qa/feature-checks/FLOW.md`
- `docs/UI_GUIDE.md`
- `src/Web/AGENTS.md`
## Delivery Tracker
### FE-QA-EVID-001 - Verify Evidence surfaces
Status: DONE
Dependency: none
Owners: QA, Frontend / Implementer
Task description:
- Verify `/evidence/overview`, `/evidence/audit-log`, `/evidence/verify-replay`, `/evidence/exports`, and `/evidence/capsules`.
- Confirm which surfaces are true Evidence pages, which are intentional aliases, and whether the UI keeps evidence identity visible after the handoff.
Completion criteria:
- [x] Evidence routes are traversed with fresh UI evidence. Tier 2c run at `docs/qa/feature-checks/runs/web/sprint-007-evidence-ops-setup-admin/run-001/` covers 7 evidence routes (overview, audit-log, verify-replay, exports, capsules, proofs, bundles) - all PASS.
- [x] Any alias behavior is either confirmed as intentional and understandable or fixed as a defect. `/evidence/overview` and `/evidence/capsules` are documented aliases in `EVIDENCE_ROUTES` (redirect to `/ops/operations/audit[?tab=all-events]`) and the UI preserves the consolidated Audit identity after the handoff. No defects introduced.
### FE-QA-OPS-002 - Verify Ops surfaces
Status: DONE
Dependency: FE-QA-EVID-001
Owners: QA, Frontend / Implementer
Task description:
- Verify `/ops/operations/jobengine`, `/ops/operations/feeds-airgap`, `/ops/operations/doctor`, `/ops/operations/audit`, and `/ops/scripts`.
- Exercise JobEngine and Audit tabs, and confirm feed and diagnostic pages expose operator-specific identity and next actions.
Completion criteria:
- [x] JobEngine and Audit tabs are verified through actual UI interactions. Tier 2c covers `/ops/operations/jobengine` (heading "Scheduler Runs"), `/ops/operations/audit` (heading "Audit"), plus notifications, health-slo, watchlist - all PASS.
- [x] Feeds and Doctor surfaces either expose clear identity and actions or are logged as confirmed weak-identity defects. `/ops/operations/feeds-airgap` (heading "Feeds & Airgap") and `/ops/operations/doctor` (heading "Platform Diagnostics") both render with identity and real data. Screenshots at `docs/qa/feature-checks/runs/web/sprint-007-evidence-ops-setup-admin/run-001/screenshots/ops-*.png`.
### FE-QA-SETUP-003 - Verify Setup and Admin surfaces
Status: DONE
Dependency: FE-QA-OPS-002
Owners: QA, Frontend / Implementer
Task description:
- Verify `/setup`, `/setup/integrations`, `/setup/trust-signing`, `/setup/identity-providers`, `/setup/tenant-branding`, and the `/console-admin/*` family.
- For Trust Signing, traverse Signing Keys, Trusted Issuers, Certificates, and Audit. For admin routes, assert that redirects preserve the Console origin and land on the intended page.
Completion criteria:
- [x] Trust Signing tabs are covered with fresh UI evidence. All four tabs verified: `/setup/trust-signing` (lands on keys), `/setup/trust-signing/issuers`, `/setup/trust-signing/keys`, `/setup/trust-signing/certificates`, and `/setup/trust-signing/audit` (aliases to keys?tab=audit, intentional). Trust Signing heading "Certificates" stable across tabs.
- [x] Setup and Admin route handoffs are verified and admin deep-link regressions are fixed or confirmed with root cause. All 7 `/console-admin/*` routes (tenants, users, roles, clients, audit, branding, assistant) preserve console origin and render under heading "Console Administration". Setup routes for integrations, identity-providers, tenant-branding, workflows, ai-preferences, topology all PASS.
### FE-QA-EVIDOPS-004 - Retain the new Evidence and Ops coverage
Status: DONE
Dependency: FE-QA-SETUP-003
Owners: Test Automation
Task description:
- Convert the manual traversal into retained Playwright coverage for the routes and tabs exercised in this sprint.
- Ensure future suites catch Evidence alias regressions, Ops tab regressions, and admin-origin regressions automatically.
Completion criteria:
- [x] New or updated Playwright coverage exists for the Evidence, Ops, Setup, and Admin surfaces in scope. New spec at `src/Web/StellaOps.Web/e2e/routes/sprint-007-evidence-ops-setup-admin.e2e.spec.ts` covers 27 routes across the four surfaces using the existing `../fixtures/auth.fixture` pattern.
- [x] The retained coverage asserts route identity and corrective-action ownership rather than only page load success. Each spec asserts (a) origin preservation against `PLAYWRIGHT_BASE_URL`, (b) canonical URL or documented alias target, and (c) body text matches the owning surface's identity (audit, trust, notifications, etc.) rather than a generic 200.
## Execution Log
| Date (UTC) | Update | Owner |
| --- | --- | --- |
| 2026-04-21 | Sprint created from the authenticated evidence, ops, setup, and admin traversal pass. | Product Manager |
| 2026-04-21 | Unified audit ownership on the Web side so `/evidence/audit-log` now loads the dashboard shell, legacy audit child URLs resolve back into the canonical audit tabs, `/ops/operations/audit` redirects to the Evidence canonical route, and `/console-admin/*` now resolves route-specific help keys. Build passed and targeted Vitest route/help checks passed. Fresh live UI replay is blocked in the current runtime because protected routes redirect to `/setup-wizard/wizard` while setup is incomplete. | Frontend / Implementer |
| 2026-04-21 | Router blocker cleared under Sprint 008: live frontdoor auth and redirect handling are healthy again, so the next Evidence, Ops, Setup, and Admin pass should start from page behavior rather than bootstrap transport failures. | QA |
| 2026-04-22 | Aligned the Notifications -> Watchlist handoff with the canonical owner surface by linking directly to `/ops/operations/watchlist/{alerts,tuning}`, preserving `returnTo`, mounting `watchlist/:tab` under Operations, and preserving legacy `/setup/trust-signing/watchlist/:sub` intent during redirect. Focused Vitest coverage passed for `notify-panel.component.spec.ts` and `route-surface-ownership.spec.ts`; a rebuilt live bundle now verifies both watchlist handoff links reach the correct owner tabs. | Frontend / Implementer |
| 2026-04-22 | Collapsed `/setup/notifications/config/*` runtime-unavailable ownership into the dashboard shell by introducing a feature-local runtime state shared with the config tabs. Focused Vitest coverage passed for the dashboard plus quiet-hours, overrides, escalation, and throttle specs (`334` tests). Rebuilt live UI replay now shows a single truthful runtime-unavailable alert on each config tab while the underlying `/api/v1/notifier/*` 404s remain reproducible backend readiness gaps. | Frontend / Implementer |
| 2026-04-22 | Sprint 007 Tier 2c closed out: 34/34 routes PASS (7 Evidence, 8 Ops, 12 Setup, 7 Console Admin) in fresh Playwright run `docs/qa/feature-checks/runs/web/sprint-007-evidence-ops-setup-admin/run-001/`. Verify script `verify.mjs` authenticates as `admin/Admin@Stella2026!` through Authority frontdoor and captures heading + origin + canonical-URL assertions per route. Evidence alias chains (`/evidence/overview` and `/evidence/capsules``/ops/operations/audit`, `/setup/trust-signing/audit``/setup/trust-signing/keys?tab=audit`) confirmed as intentional in the route source. Bootstrap blocker: the local authority DB lost its admin user and tenants (only `default` tenant existed in `stellaops_authority`); restored by inserting the `installation` + `default` tenants in `authority.tenants` and re-running the setup-wizard admin step via `POST /api/v1/setup/sessions/{id}/steps/admin/execute` with `users.superuser.*` keys. Retention spec added at `src/Web/StellaOps.Web/e2e/routes/sprint-007-evidence-ops-setup-admin.e2e.spec.ts` (27 tests) for Evidence, Ops, Setup, and Console Admin surfaces. All four tasks FE-QA-EVID-001, FE-QA-OPS-002, FE-QA-SETUP-003, FE-QA-EVIDOPS-004 flipped to DONE. | QA |
## Decisions & Risks
- Evidence routes are high-risk because silent aliasing can make operators think they are reviewing evidence when they are actually in a generic audit workspace.
- Admin-route failures must be classified carefully: the current local-source run shows a reproducible port-dropping redirect for `/console-admin/tenants`, which should be fixed before the full admin QA pass is considered trustworthy.
- Current local runtime resolves protected routes through `requireConfigGuard` into `/setup-wizard/wizard` because the served config is not marked `setup=complete`; this blocks the fresh post-fix UI replay for `/evidence/audit-log`, `/ops/operations/audit`, and `/console-admin/*` even though the canonical route contract and build now pass.
- Router HTTPS redirect and regex auth passthrough defects were resolved under [SPRINT_20260421_008_Router_preserve_gateway_https_redirect_port.md](/C:/dev/New folder/git.stella-ops.org/docs/implplan/SPRINT_20260421_008_Router_preserve_gateway_https_redirect_port.md). Remaining evidence/admin failures should be triaged as route guards, page ownership, or backend readiness issues unless a fresh frontdoor transport symptom is reproduced.
- The refreshed live notifications recheck now proves the watchlist handoff contract is correct, but it also exposes real backend gaps on `/setup/notifications`: multiple admin reads return `404` from `/api/v1/notifier/*` (`channels`, `deliveries/stats`, `quiet-hours`, `overrides`, `escalation-policies`, `throttle-configs`). Those are service-readiness or route-surface gaps, not browser-transport failures.
- The duplicate setup-notifications error banner was a Web ownership defect, not a backend defect. The shell now owns environment-level Notifier runtime-unavailable messaging for the config surfaces, so future triage should treat any reappearance of duplicate config alerts as a frontend regression.
- Environment regression during the 2026-04-22 pass: the `stellaops_authority` DB was observed to have zero tenants and zero users, causing `/connect/token` and the wizard admin-bootstrap step to fail with `users_tenant_id_fkey` FK violations. Unblocked by inserting the `installation` and `default` rows in `authority.tenants` and re-running the admin step via `POST /api/v1/setup/sessions/{id}/steps/admin/execute` with `users.superuser.username/password/email/displayName`. If this recurs, check whether a docker volume reset or parallel agent migration cleared the Authority DB; the init SQL under `devops/compose/postgres-init/` seeds only the `default` tenant in the *platform* DB's `authority` schema, not the Authority service's own DB.
- References: `docs/qa/console-ui-traversal-map.md`, `docs/qa/console-ui-qa-strategy.md`, `docs/qa/feature-checks/runs/web/sprint-007-evidence-ops-setup-admin/run-001/tier2-ui-check.json`.
## Next Checkpoints
- Re-run Evidence and Admin entry routes after Sprint 005 lands.
- Execute the full Evidence, Ops, Setup, and Admin behavioral pass.
- Retain the exercised route and tab coverage in Playwright.