Close scratch iteration 008 and enforce full surface audits

This commit is contained in:
master
2026-03-13 11:00:12 +02:00
parent fe35801cc5
commit c9a30331ce
11 changed files with 534 additions and 32 deletions

View File

@@ -0,0 +1,80 @@
# Sprint 20260313_003 - Platform Scratch Iteration 008 Full Route Action Audit
## Topic & Scope
- Wipe Stella-owned runtime state again and rerun the documented setup path from zero state.
- Re-enter the application as a first-time user after bootstrap and rerun the full route, page-load, and page-action audit with Playwright.
- Recheck changed or newly discovered surfaces and convert any new manual findings into retained Playwright scenarios before the iteration is considered complete.
- Group any newly exposed defects before fixing so the next commit closes a full iteration rather than a single page slice.
- Working directory: `.`.
- Expected evidence: wipe proof, setup convergence proof, fresh Playwright route/page/action evidence, retained scenario coverage for new findings, grouped defect list, fixes, and retest results.
## Dependencies & Concurrency
- Depends on local commit `fe35801cc` as the clean baseline for the next scratch cycle.
- Safe parallelism: none during wipe/setup because the environment reset is global to the machine.
## Documentation Prerequisites
- `AGENTS.md`
- `docs/INSTALL_GUIDE.md`
- `docs/dev/DEV_ENVIRONMENT_SETUP.md`
- `docs/qa/feature-checks/FLOW.md`
## Delivery Tracker
### PLATFORM-SCRATCH-ITER8-001 - Rebuild from zero Stella runtime state
Status: DONE
Dependency: none
Owners: QA, 3rd line support
Task description:
- Remove Stella-only containers, images, volumes, and the frontdoor network, then rerun the documented setup entrypoint from zero Stella state.
Completion criteria:
- [x] Stella-only Docker state is removed.
- [x] scripts/setup.ps1 is rerun from zero state.
- [x] The first setup outcome is captured before UI verification starts.
### PLATFORM-SCRATCH-ITER8-002 - Re-run the first-user full route/page/action audit
Status: DONE
Dependency: PLATFORM-SCRATCH-ITER8-001
Owners: QA
Task description:
- After scratch setup converges, rerun the canonical route sweep plus the full route/page/action audit suite, including changed-surface and route-ownership checks, and enumerate every newly exposed issue before repair work begins.
Completion criteria:
- [x] Fresh route sweep evidence is captured on the rebuilt stack.
- [x] Fresh route/page/action evidence is captured across the full aggregate suite, including changed-surface and ownership checks.
- [x] Newly exposed defects are grouped and any new manual findings are queued into retained Playwright scenarios before any fix commit is prepared.
### PLATFORM-SCRATCH-ITER8-003 - Repair the grouped defects exposed by the fresh audit
Status: DONE
Dependency: PLATFORM-SCRATCH-ITER8-002
Owners: 3rd line support, Architect, Developer
Task description:
- Diagnose the grouped failures exposed by the fresh audit, choose the clean product/architecture-conformant fix, implement it, add retained Playwright coverage for the new behavior when needed, and rerun the affected verification slices plus the aggregate audit before committing.
Completion criteria:
- [x] Root causes are recorded for the grouped failures.
- [x] Fixes land with focused regression coverage and retained Playwright scenario updates where practical.
- [x] The rebuilt stack is retested before the iteration commit.
## Execution Log
| Date (UTC) | Update | Owner |
| --- | --- | --- |
| 2026-03-13 | Sprint created for the next scratch iteration after local commit `fe35801cc` closed the previous clean baseline. | QA |
| 2026-03-13 | Removed Stella-only containers, `stellaops/*:dev` images, Stella compose volumes, and the `stellaops` / `stellaops_frontdoor` networks to return the machine to zero Stella runtime state for the new iteration. | QA / 3rd line support |
| 2026-03-13 | The zero-state setup rerun completed cleanly on the rebuilt stack, but the documented PowerShell setup entrypoint returned a stale nonzero native exit code after printing `Setup complete!`. Root cause was a missing explicit success exit; `scripts/setup.ps1` and `scripts/setup.sh` now terminate with `exit 0` on the green path, and `setup.ps1 -SkipBuild -SkipImages` reverified cleanly on the converged stack. | QA / 3rd line support / Developer |
| 2026-03-13 | Added retained promotion refresh coverage before re-entering the live audit: focused Angular `src/tests/releases/release-promotions-cutover.spec.ts` passed `7/7`, and focused Playwright `tests/e2e/release-promotions-cutover.spec.ts` passed `2/2`. | QA / Developer |
| 2026-03-13 | Fresh canonical frontdoor route sweep completed on the rebuilt stack with `111/111` passed routes and `0` failed routes. Full aggregate route/page/action audit is now running against the same scratch environment. | QA |
| 2026-03-13 | Root-caused the promotion wizard failure to a stale gate-preview response that could regress the active step after the user advanced; guarded progression while preview loading is active, ignored stale preview responses, and retained both Angular and Playwright coverage for the late-refresh path. | 3rd line support / Architect / Developer |
| 2026-03-13 | The rebuilt scratch stack completed the full retained aggregate route/page/action audit cleanly: `22/22` suites passed, `0` failed, `0` retried, and `0` stabilized-after-retry, including route ownership, changed surfaces, user-reported admin/trust flows, search result actions, uncovered surfaces, and release/evidence workflows. | QA |
## Decisions & Risks
- Decision: each scratch iteration remains a full wipe -> setup -> route/page/action audit -> grouped remediation loop; if the audit comes back clean, that still counts as a completed iteration because the full loop was executed.
- Decision: changed or newly discovered user flows must be converted into retained Playwright coverage before the next scratch iteration starts so the audit surface expands instead of rediscovering the same gaps manually.
- Decision: the documented setup entrypoints must exit `0` explicitly on the success path so the scratch loop can trust setup status instead of inheriting stale native exit codes from earlier Docker commands.
- Decision: a scratch iteration is only considered clean when the first-pass aggregate finishes without retries or stabilized reruns; any suite that only passes after retry is treated as a defect signal to fix, not a clean close-out.
- Risk: scratch rebuilds remain expensive, so verification stays Playwright-first with focused test/build slices rather than indiscriminate full-solution test runs.
## Next Checkpoints
- Start iteration 009 from a fresh Stella wipe using the corrected setup entrypoints and retained aggregate gate set.
- Continue expanding retained Playwright coverage whenever new manual paths or regressions are discovered in later scratch iterations.

View File

@@ -0,0 +1,72 @@
# Sprint 20260313_003 - Platform Scratch Iteration Coverage Enforcement
## Topic & Scope
- Harden the scratch iteration runner so it survives long setup/build streams without misclassifying native stderr as a hard runner failure.
- Expand the enforced Playwright aggregate so route ownership and changed-surface checks become mandatory iteration gates rather than optional probes.
- Sync the QA workflow docs so full UI verification explicitly means every route, page-load state, and visible action, with new manual findings converted into retained Playwright scenarios.
- Working directory: `.`.
- Expected evidence: runner fix, expanded aggregate audit coverage, updated QA workflow documentation, and a clean local verification pass for the touched harnesses.
## Dependencies & Concurrency
- Depends on the current scratch iteration tooling under `scripts/` and `src/Web/StellaOps.Web/scripts/`.
- Safe parallelism: avoid concurrent edits to the scratch runner or aggregate audit while this sprint is in progress.
## Documentation Prerequisites
- `AGENTS.md`
- `docs/qa/feature-checks/FLOW.md`
- `docs/INSTALL_GUIDE.md`
- `docs/dev/DEV_ENVIRONMENT_SETUP.md`
## Delivery Tracker
### PLATFORM-COVERAGE-001 - Harden scratch runner process execution
Status: DONE
Dependency: none
Owners: 3rd line support, Developer
Task description:
- Replace the fragile in-process/native stream invocation pattern in the scratch runner with a subprocess execution path that preserves logs and exit codes during long setup and rebuild phases.
Completion criteria:
- [x] `scripts/run-clean-scratch-iterations.ps1` no longer fails on native stderr progress output alone.
- [x] Setup/build logs remain visible during long-running child processes.
- [x] The runner still returns structured JSON data for route and aggregate audit steps.
### PLATFORM-COVERAGE-002 - Enforce retained changed-surface and ownership audits
Status: DONE
Dependency: PLATFORM-COVERAGE-001
Owners: QA, Developer
Task description:
- Make changed-surface and route-ownership verification first-class aggregate audit gates, and ensure changed-surface coverage fails truthfully when headings, visible actions, search probes, or runtime health regress.
Completion criteria:
- [x] The aggregate audit includes route ownership and changed-surface suites.
- [x] Changed-surface verification reports explicit failures and exits non-zero when retained expectations break.
- [x] Aggregate audit progress is persisted while the suite is running.
### PLATFORM-COVERAGE-003 - Sync workflow docs with the enforced UI QA standard
Status: DONE
Dependency: PLATFORM-COVERAGE-002
Owners: QA, Documentation
Task description:
- Update the QA flow so UI verification explicitly requires every route, page-load state, and visible action, and so new manual findings must be converted into retained Playwright scenarios before the loop closes.
Completion criteria:
- [x] `docs/qa/feature-checks/FLOW.md` reflects the stricter UI verification standard.
- [x] The updated docs align with the runner and aggregate audit behavior.
## Execution Log
| Date (UTC) | Update | Owner |
| --- | --- | --- |
| 2026-03-13 | Sprint created after the scratch loop exposed a runner stream bug and a gap between the required UI QA standard and the currently enforced Playwright aggregate. | QA / Developer |
| 2026-03-13 | Fixed the scratch runner subprocess argument handling so repo paths containing spaces no longer break `powershell.exe -File ...` child execution, preserved streamed setup/build logs, and added a dirty-worktree guard so clean scratch iterations do not auto-commit sprint-only changes on top of pre-existing modifications. | 3rd line support / Developer |
| 2026-03-13 | Promoted route ownership and changed-surface checks into enforced aggregate gates, kept progress persisted to `live-full-core-audit.json` while the audit runs, and reverified the expanded aggregate cleanly on the rebuilt scratch stack with `22/22` suites passed and `0` retries. | QA / Developer |
| 2026-03-13 | Tightened `docs/qa/feature-checks/FLOW.md` so UI verification now explicitly means every route, page-load state, and visible action, and every newly discovered manual path must become retained Playwright coverage before an iteration closes. | QA / Documentation |
## Decisions & Risks
- Decision: treat route ownership and changed-surface verification as required iteration gates, not best-effort diagnostics.
- Decision: newly discovered manual UI paths must be retained in Playwright before a scratch iteration can be considered structurally complete.
- Risk: stricter aggregate coverage increases iteration time, but that is preferable to rediscovering missing surfaces manually in later cycles.
## Next Checkpoints
- Keep the corrected runner driving later scratch iterations from clean Stella state.
- Expand retained Playwright coverage further whenever later iterations expose new manual-only paths.

View File

@@ -229,6 +229,8 @@ A Tier 2 run is valid only if ALL of the following are true:
3. It includes user-surface interactions (HTTP requests, CLI invocations, or UI interactions), not only library test counts.
4. It verifies both positive and negative behavior paths when the feature has error semantics.
5. For rechecks, at least one new user transaction per feature is captured in the new run.
6. For UI features, the run covers the full visible user surface for that feature: every route, every page-load state, and every visible action that can be exercised in the local environment.
7. Any newly discovered manual UI flow or regression path is converted into retained Playwright coverage before the feature may be considered stably verified.
The following are forbidden and invalidate Tier 2:
- Copying a previous run directory and only editing `runId`, timestamps, or summary text.
@@ -321,10 +323,12 @@ echo $? # Verify exit code 0
**Process**:
1. Ensure the Angular app is running (`ng serve` or docker)
2. Use Playwright CLI or MCP to navigate to the feature's UI route
3. Follow E2E Test Plan steps: verify elements render, interactions work, data displays
4. If the feature fails only through transient network/runtime noise, rerun the failing UI transaction in a fresh page or fresh authenticated browser context before declaring the feature failed. Preserve both the first failure evidence and the recheck outcome.
5. Capture screenshots as evidence
6. Test accessibility (keyboard navigation, ARIA labels) if listed in E2E plan
3. Traverse the full visible UI surface for the feature: every route, every page-load state, and every visible action that can be exercised locally.
4. Follow E2E Test Plan steps: verify elements render, interactions work, and data displays with the correct state and routing.
5. If the feature fails only through transient network/runtime noise, rerun the failing UI transaction in a fresh page or fresh authenticated browser context before declaring the feature failed. Preserve both the first failure evidence and the recheck outcome.
6. Capture screenshots as evidence
7. Test accessibility (keyboard navigation, ARIA labels) if listed in E2E plan
8. If manual exploration exposes a route/action not already covered by the retained Playwright suite, add or extend a Playwright scenario before closing the verification loop so the next run replays that path automatically.
**Example for `pipeline-run-centric-view`**:
```bash