From c9a30331cef2863545c815773f63b91825d97a0d Mon Sep 17 00:00:00 2001 From: master <> Date: Fri, 13 Mar 2026 11:00:12 +0200 Subject: [PATCH] Close scratch iteration 008 and enforce full surface audits --- ...h_iteration_008_full_route_action_audit.md | 80 +++++++++ ..._scratch_iteration_coverage_enforcement.md | 72 +++++++++ docs/qa/feature-checks/FLOW.md | 12 +- scripts/run-clean-scratch-iterations.ps1 | 130 +++++++++++++-- scripts/setup.ps1 | 1 + scripts/setup.sh | 1 + .../live-frontdoor-changed-surfaces.mjs | 58 +++++++ .../scripts/live-full-core-audit.mjs | 19 ++- .../promotions/create-promotion.component.ts | 32 ++-- .../release-promotions-cutover.spec.ts | 153 +++++++++++++++++- .../e2e/release-promotions-cutover.spec.ts | 8 + 11 files changed, 534 insertions(+), 32 deletions(-) create mode 100644 docs/implplan/SPRINT_20260313_003_Platform_scratch_iteration_008_full_route_action_audit.md create mode 100644 docs/implplan/SPRINT_20260313_003_Platform_scratch_iteration_coverage_enforcement.md diff --git a/docs/implplan/SPRINT_20260313_003_Platform_scratch_iteration_008_full_route_action_audit.md b/docs/implplan/SPRINT_20260313_003_Platform_scratch_iteration_008_full_route_action_audit.md new file mode 100644 index 000000000..65b985765 --- /dev/null +++ b/docs/implplan/SPRINT_20260313_003_Platform_scratch_iteration_008_full_route_action_audit.md @@ -0,0 +1,80 @@ +# Sprint 20260313_003 - Platform Scratch Iteration 008 Full Route Action Audit + +## Topic & Scope +- Wipe Stella-owned runtime state again and rerun the documented setup path from zero state. +- Re-enter the application as a first-time user after bootstrap and rerun the full route, page-load, and page-action audit with Playwright. +- Recheck changed or newly discovered surfaces and convert any new manual findings into retained Playwright scenarios before the iteration is considered complete. +- Group any newly exposed defects before fixing so the next commit closes a full iteration rather than a single page slice. +- Working directory: `.`. +- Expected evidence: wipe proof, setup convergence proof, fresh Playwright route/page/action evidence, retained scenario coverage for new findings, grouped defect list, fixes, and retest results. + +## Dependencies & Concurrency +- Depends on local commit `fe35801cc` as the clean baseline for the next scratch cycle. +- Safe parallelism: none during wipe/setup because the environment reset is global to the machine. + +## Documentation Prerequisites +- `AGENTS.md` +- `docs/INSTALL_GUIDE.md` +- `docs/dev/DEV_ENVIRONMENT_SETUP.md` +- `docs/qa/feature-checks/FLOW.md` + +## Delivery Tracker + +### PLATFORM-SCRATCH-ITER8-001 - Rebuild from zero Stella runtime state +Status: DONE +Dependency: none +Owners: QA, 3rd line support +Task description: +- Remove Stella-only containers, images, volumes, and the frontdoor network, then rerun the documented setup entrypoint from zero Stella state. + +Completion criteria: +- [x] Stella-only Docker state is removed. +- [x] scripts/setup.ps1 is rerun from zero state. +- [x] The first setup outcome is captured before UI verification starts. + +### PLATFORM-SCRATCH-ITER8-002 - Re-run the first-user full route/page/action audit +Status: DONE +Dependency: PLATFORM-SCRATCH-ITER8-001 +Owners: QA +Task description: +- After scratch setup converges, rerun the canonical route sweep plus the full route/page/action audit suite, including changed-surface and route-ownership checks, and enumerate every newly exposed issue before repair work begins. + +Completion criteria: +- [x] Fresh route sweep evidence is captured on the rebuilt stack. +- [x] Fresh route/page/action evidence is captured across the full aggregate suite, including changed-surface and ownership checks. +- [x] Newly exposed defects are grouped and any new manual findings are queued into retained Playwright scenarios before any fix commit is prepared. + +### PLATFORM-SCRATCH-ITER8-003 - Repair the grouped defects exposed by the fresh audit +Status: DONE +Dependency: PLATFORM-SCRATCH-ITER8-002 +Owners: 3rd line support, Architect, Developer +Task description: +- Diagnose the grouped failures exposed by the fresh audit, choose the clean product/architecture-conformant fix, implement it, add retained Playwright coverage for the new behavior when needed, and rerun the affected verification slices plus the aggregate audit before committing. + +Completion criteria: +- [x] Root causes are recorded for the grouped failures. +- [x] Fixes land with focused regression coverage and retained Playwright scenario updates where practical. +- [x] The rebuilt stack is retested before the iteration commit. + +## Execution Log +| Date (UTC) | Update | Owner | +| --- | --- | --- | +| 2026-03-13 | Sprint created for the next scratch iteration after local commit `fe35801cc` closed the previous clean baseline. | QA | +| 2026-03-13 | Removed Stella-only containers, `stellaops/*:dev` images, Stella compose volumes, and the `stellaops` / `stellaops_frontdoor` networks to return the machine to zero Stella runtime state for the new iteration. | QA / 3rd line support | +| 2026-03-13 | The zero-state setup rerun completed cleanly on the rebuilt stack, but the documented PowerShell setup entrypoint returned a stale nonzero native exit code after printing `Setup complete!`. Root cause was a missing explicit success exit; `scripts/setup.ps1` and `scripts/setup.sh` now terminate with `exit 0` on the green path, and `setup.ps1 -SkipBuild -SkipImages` reverified cleanly on the converged stack. | QA / 3rd line support / Developer | +| 2026-03-13 | Added retained promotion refresh coverage before re-entering the live audit: focused Angular `src/tests/releases/release-promotions-cutover.spec.ts` passed `7/7`, and focused Playwright `tests/e2e/release-promotions-cutover.spec.ts` passed `2/2`. | QA / Developer | +| 2026-03-13 | Fresh canonical frontdoor route sweep completed on the rebuilt stack with `111/111` passed routes and `0` failed routes. Full aggregate route/page/action audit is now running against the same scratch environment. | QA | +| 2026-03-13 | Root-caused the promotion wizard failure to a stale gate-preview response that could regress the active step after the user advanced; guarded progression while preview loading is active, ignored stale preview responses, and retained both Angular and Playwright coverage for the late-refresh path. | 3rd line support / Architect / Developer | +| 2026-03-13 | The rebuilt scratch stack completed the full retained aggregate route/page/action audit cleanly: `22/22` suites passed, `0` failed, `0` retried, and `0` stabilized-after-retry, including route ownership, changed surfaces, user-reported admin/trust flows, search result actions, uncovered surfaces, and release/evidence workflows. | QA | + +## Decisions & Risks +- Decision: each scratch iteration remains a full wipe -> setup -> route/page/action audit -> grouped remediation loop; if the audit comes back clean, that still counts as a completed iteration because the full loop was executed. +- Decision: changed or newly discovered user flows must be converted into retained Playwright coverage before the next scratch iteration starts so the audit surface expands instead of rediscovering the same gaps manually. +- Decision: the documented setup entrypoints must exit `0` explicitly on the success path so the scratch loop can trust setup status instead of inheriting stale native exit codes from earlier Docker commands. +- Decision: a scratch iteration is only considered clean when the first-pass aggregate finishes without retries or stabilized reruns; any suite that only passes after retry is treated as a defect signal to fix, not a clean close-out. +- Risk: scratch rebuilds remain expensive, so verification stays Playwright-first with focused test/build slices rather than indiscriminate full-solution test runs. + +## Next Checkpoints +- Start iteration 009 from a fresh Stella wipe using the corrected setup entrypoints and retained aggregate gate set. +- Continue expanding retained Playwright coverage whenever new manual paths or regressions are discovered in later scratch iterations. + diff --git a/docs/implplan/SPRINT_20260313_003_Platform_scratch_iteration_coverage_enforcement.md b/docs/implplan/SPRINT_20260313_003_Platform_scratch_iteration_coverage_enforcement.md new file mode 100644 index 000000000..ce6b66cf4 --- /dev/null +++ b/docs/implplan/SPRINT_20260313_003_Platform_scratch_iteration_coverage_enforcement.md @@ -0,0 +1,72 @@ +# Sprint 20260313_003 - Platform Scratch Iteration Coverage Enforcement + +## Topic & Scope +- Harden the scratch iteration runner so it survives long setup/build streams without misclassifying native stderr as a hard runner failure. +- Expand the enforced Playwright aggregate so route ownership and changed-surface checks become mandatory iteration gates rather than optional probes. +- Sync the QA workflow docs so full UI verification explicitly means every route, page-load state, and visible action, with new manual findings converted into retained Playwright scenarios. +- Working directory: `.`. +- Expected evidence: runner fix, expanded aggregate audit coverage, updated QA workflow documentation, and a clean local verification pass for the touched harnesses. + +## Dependencies & Concurrency +- Depends on the current scratch iteration tooling under `scripts/` and `src/Web/StellaOps.Web/scripts/`. +- Safe parallelism: avoid concurrent edits to the scratch runner or aggregate audit while this sprint is in progress. + +## Documentation Prerequisites +- `AGENTS.md` +- `docs/qa/feature-checks/FLOW.md` +- `docs/INSTALL_GUIDE.md` +- `docs/dev/DEV_ENVIRONMENT_SETUP.md` + +## Delivery Tracker + +### PLATFORM-COVERAGE-001 - Harden scratch runner process execution +Status: DONE +Dependency: none +Owners: 3rd line support, Developer +Task description: +- Replace the fragile in-process/native stream invocation pattern in the scratch runner with a subprocess execution path that preserves logs and exit codes during long setup and rebuild phases. + +Completion criteria: +- [x] `scripts/run-clean-scratch-iterations.ps1` no longer fails on native stderr progress output alone. +- [x] Setup/build logs remain visible during long-running child processes. +- [x] The runner still returns structured JSON data for route and aggregate audit steps. + +### PLATFORM-COVERAGE-002 - Enforce retained changed-surface and ownership audits +Status: DONE +Dependency: PLATFORM-COVERAGE-001 +Owners: QA, Developer +Task description: +- Make changed-surface and route-ownership verification first-class aggregate audit gates, and ensure changed-surface coverage fails truthfully when headings, visible actions, search probes, or runtime health regress. + +Completion criteria: +- [x] The aggregate audit includes route ownership and changed-surface suites. +- [x] Changed-surface verification reports explicit failures and exits non-zero when retained expectations break. +- [x] Aggregate audit progress is persisted while the suite is running. + +### PLATFORM-COVERAGE-003 - Sync workflow docs with the enforced UI QA standard +Status: DONE +Dependency: PLATFORM-COVERAGE-002 +Owners: QA, Documentation +Task description: +- Update the QA flow so UI verification explicitly requires every route, page-load state, and visible action, and so new manual findings must be converted into retained Playwright scenarios before the loop closes. + +Completion criteria: +- [x] `docs/qa/feature-checks/FLOW.md` reflects the stricter UI verification standard. +- [x] The updated docs align with the runner and aggregate audit behavior. + +## Execution Log +| Date (UTC) | Update | Owner | +| --- | --- | --- | +| 2026-03-13 | Sprint created after the scratch loop exposed a runner stream bug and a gap between the required UI QA standard and the currently enforced Playwright aggregate. | QA / Developer | +| 2026-03-13 | Fixed the scratch runner subprocess argument handling so repo paths containing spaces no longer break `powershell.exe -File ...` child execution, preserved streamed setup/build logs, and added a dirty-worktree guard so clean scratch iterations do not auto-commit sprint-only changes on top of pre-existing modifications. | 3rd line support / Developer | +| 2026-03-13 | Promoted route ownership and changed-surface checks into enforced aggregate gates, kept progress persisted to `live-full-core-audit.json` while the audit runs, and reverified the expanded aggregate cleanly on the rebuilt scratch stack with `22/22` suites passed and `0` retries. | QA / Developer | +| 2026-03-13 | Tightened `docs/qa/feature-checks/FLOW.md` so UI verification now explicitly means every route, page-load state, and visible action, and every newly discovered manual path must become retained Playwright coverage before an iteration closes. | QA / Documentation | + +## Decisions & Risks +- Decision: treat route ownership and changed-surface verification as required iteration gates, not best-effort diagnostics. +- Decision: newly discovered manual UI paths must be retained in Playwright before a scratch iteration can be considered structurally complete. +- Risk: stricter aggregate coverage increases iteration time, but that is preferable to rediscovering missing surfaces manually in later cycles. + +## Next Checkpoints +- Keep the corrected runner driving later scratch iterations from clean Stella state. +- Expand retained Playwright coverage further whenever later iterations expose new manual-only paths. diff --git a/docs/qa/feature-checks/FLOW.md b/docs/qa/feature-checks/FLOW.md index 01bb29c44..0a6054f47 100644 --- a/docs/qa/feature-checks/FLOW.md +++ b/docs/qa/feature-checks/FLOW.md @@ -229,6 +229,8 @@ A Tier 2 run is valid only if ALL of the following are true: 3. It includes user-surface interactions (HTTP requests, CLI invocations, or UI interactions), not only library test counts. 4. It verifies both positive and negative behavior paths when the feature has error semantics. 5. For rechecks, at least one new user transaction per feature is captured in the new run. +6. For UI features, the run covers the full visible user surface for that feature: every route, every page-load state, and every visible action that can be exercised in the local environment. +7. Any newly discovered manual UI flow or regression path is converted into retained Playwright coverage before the feature may be considered stably verified. The following are forbidden and invalidate Tier 2: - Copying a previous run directory and only editing `runId`, timestamps, or summary text. @@ -321,10 +323,12 @@ echo $? # Verify exit code 0 **Process**: 1. Ensure the Angular app is running (`ng serve` or docker) 2. Use Playwright CLI or MCP to navigate to the feature's UI route -3. Follow E2E Test Plan steps: verify elements render, interactions work, data displays -4. If the feature fails only through transient network/runtime noise, rerun the failing UI transaction in a fresh page or fresh authenticated browser context before declaring the feature failed. Preserve both the first failure evidence and the recheck outcome. -5. Capture screenshots as evidence -6. Test accessibility (keyboard navigation, ARIA labels) if listed in E2E plan +3. Traverse the full visible UI surface for the feature: every route, every page-load state, and every visible action that can be exercised locally. +4. Follow E2E Test Plan steps: verify elements render, interactions work, and data displays with the correct state and routing. +5. If the feature fails only through transient network/runtime noise, rerun the failing UI transaction in a fresh page or fresh authenticated browser context before declaring the feature failed. Preserve both the first failure evidence and the recheck outcome. +6. Capture screenshots as evidence +7. Test accessibility (keyboard navigation, ARIA labels) if listed in E2E plan +8. If manual exploration exposes a route/action not already covered by the retained Playwright suite, add or extend a Playwright scenario before closing the verification loop so the next run replays that path automatically. **Example for `pipeline-run-centric-view`**: ```bash diff --git a/scripts/run-clean-scratch-iterations.ps1 b/scripts/run-clean-scratch-iterations.ps1 index 71804849c..8f2d2a45b 100644 --- a/scripts/run-clean-scratch-iterations.ps1 +++ b/scripts/run-clean-scratch-iterations.ps1 @@ -19,6 +19,54 @@ $webRoot = Join-Path $repoRoot 'src/Web/StellaOps.Web' $sprintRoot = Join-Path $repoRoot 'docs/implplan' $outputRoot = Join-Path $webRoot 'output' +function New-LogCapturePath { + param( + [Parameter(Mandatory = $true)] + [string]$Suffix + ) + + return Join-Path ([System.IO.Path]::GetTempPath()) ("stellaops-{0}{1}" -f ([System.Guid]::NewGuid().ToString('N')), $Suffix) +} + +function Write-LogCaptureDelta { + param( + [Parameter(Mandatory = $true)] + [string]$Path, + + [Parameter(Mandatory = $true)] + [ref]$LineIndex + ) + + if (-not (Test-Path $Path)) { + return + } + + $lines = @(Get-Content $Path -ErrorAction SilentlyContinue) + if ($lines.Count -le $LineIndex.Value) { + return + } + + for ($i = $LineIndex.Value; $i -lt $lines.Count; $i++) { + $lines[$i] | Out-Host + } + + $LineIndex.Value = $lines.Count +} + +function Format-ProcessArgument { + param( + [Parameter(Mandatory = $true)] + [AllowEmptyString()] + [string]$Value + ) + + if ($Value -notmatch '[\s"]') { + return $Value + } + + return '"' + ($Value -replace '(\\*)"', '$1$1\"' -replace '(\\+)$', '$1$1') + '"' +} + function Invoke-External { param( [Parameter(Mandatory = $true)] @@ -30,25 +78,57 @@ function Invoke-External { ) Push-Location $WorkingDirectory + $stdoutPath = New-LogCapturePath -Suffix '.stdout.log' + $stderrPath = New-LogCapturePath -Suffix '.stderr.log' try { $resolvedFilePath = $FilePath if (Test-Path $FilePath) { $resolvedFilePath = (Resolve-Path $FilePath).Path } + $startFilePath = $resolvedFilePath + $startArgumentList = @($ArgumentList) + if ([System.IO.Path]::GetExtension($resolvedFilePath).Equals('.ps1', [System.StringComparison]::OrdinalIgnoreCase)) { $powershellPath = (Get-Command powershell.exe -ErrorAction Stop).Source - & $powershellPath -NoLogo -NoProfile -ExecutionPolicy Bypass -File $resolvedFilePath @ArgumentList 2>&1 | Out-Host - } - else { - & $resolvedFilePath @ArgumentList 2>&1 | Out-Host + $startFilePath = $powershellPath + $startArgumentList = @('-NoLogo', '-NoProfile', '-ExecutionPolicy', 'Bypass', '-File', $resolvedFilePath) + @($ArgumentList) } - if ($LASTEXITCODE -ne 0) { + $formattedArgumentList = @($startArgumentList | ForEach-Object { Format-ProcessArgument -Value $_ }) + + $process = Start-Process ` + -FilePath $startFilePath ` + -ArgumentList $formattedArgumentList ` + -WorkingDirectory $WorkingDirectory ` + -RedirectStandardOutput $stdoutPath ` + -RedirectStandardError $stderrPath ` + -PassThru + + $stdoutIndex = 0 + $stderrIndex = 0 + + while (-not $process.HasExited) { + Write-LogCaptureDelta -Path $stdoutPath -LineIndex ([ref]$stdoutIndex) + Write-LogCaptureDelta -Path $stderrPath -LineIndex ([ref]$stderrIndex) + Start-Sleep -Milliseconds 500 + $process.Refresh() + } + + Write-LogCaptureDelta -Path $stdoutPath -LineIndex ([ref]$stdoutIndex) + Write-LogCaptureDelta -Path $stderrPath -LineIndex ([ref]$stderrIndex) + + if ($process.ExitCode -ne 0) { throw "Command failed: $resolvedFilePath $($ArgumentList -join ' ')" } } finally { + foreach ($capturePath in @($stdoutPath, $stderrPath)) { + if (Test-Path $capturePath) { + Remove-Item $capturePath -Force -ErrorAction SilentlyContinue + } + } + Pop-Location } } @@ -198,10 +278,11 @@ function Render-SprintFile { $null = $lines.Add('') $null = $lines.Add('## Topic & Scope') $null = $lines.Add('- Wipe Stella-owned runtime state again and rerun the documented setup path from zero state.') - $null = $lines.Add('- Re-enter the application as a first-time user after bootstrap and rerun the full route, page, and page-action audit with Playwright.') + $null = $lines.Add('- Re-enter the application as a first-time user after bootstrap and rerun the full route, page-load, and page-action audit with Playwright.') + $null = $lines.Add('- Recheck changed or newly discovered surfaces and convert any new manual findings into retained Playwright scenarios before the iteration is considered complete.') $null = $lines.Add('- Group any newly exposed defects before fixing so the next commit closes a full iteration rather than a single page slice.') $null = $lines.Add('- Working directory: `.`.') - $null = $lines.Add('- Expected evidence: wipe proof, setup convergence proof, fresh Playwright route/action evidence, grouped defect list, fixes, and retest results.') + $null = $lines.Add('- Expected evidence: wipe proof, setup convergence proof, fresh Playwright route/page/action evidence, retained scenario coverage for new findings, grouped defect list, fixes, and retest results.') $null = $lines.Add('') $null = $lines.Add('## Dependencies & Concurrency') $null = $lines.Add("- Depends on local commit ``$($State.BaselineCommit)`` as the clean baseline for the next scratch cycle.") @@ -232,23 +313,23 @@ function Render-SprintFile { $null = $lines.Add("Dependency: PLATFORM-SCRATCH-ITER$($State.Iteration)-001") $null = $lines.Add('Owners: QA') $null = $lines.Add('Task description:') - $null = $lines.Add('- After scratch setup converges, rerun the canonical route sweep plus the full action audit suite and enumerate every newly exposed issue before repair work begins.') + $null = $lines.Add('- After scratch setup converges, rerun the canonical route sweep plus the full route/page/action audit suite, including changed-surface and route-ownership checks, and enumerate every newly exposed issue before repair work begins.') $null = $lines.Add('') $null = $lines.Add('Completion criteria:') $null = $lines.Add("- [$(Get-CheckboxMark -Value $State.Criteria21)] Fresh route sweep evidence is captured on the rebuilt stack.") - $null = $lines.Add("- [$(Get-CheckboxMark -Value $State.Criteria22)] Fresh action sweep evidence is captured across the current aggregate suite.") - $null = $lines.Add("- [$(Get-CheckboxMark -Value $State.Criteria23)] Newly exposed defects are grouped before any fix commit is prepared.") + $null = $lines.Add("- [$(Get-CheckboxMark -Value $State.Criteria22)] Fresh route/page/action evidence is captured across the full aggregate suite, including changed-surface and ownership checks.") + $null = $lines.Add("- [$(Get-CheckboxMark -Value $State.Criteria23)] Newly exposed defects are grouped and any new manual findings are queued into retained Playwright scenarios before any fix commit is prepared.") $null = $lines.Add('') $null = $lines.Add("### PLATFORM-SCRATCH-ITER$($State.Iteration)-003 - Repair the grouped defects exposed by the fresh audit") $null = $lines.Add("Status: $($State.Status3)") $null = $lines.Add("Dependency: PLATFORM-SCRATCH-ITER$($State.Iteration)-002") $null = $lines.Add('Owners: 3rd line support, Architect, Developer') $null = $lines.Add('Task description:') - $null = $lines.Add('- Diagnose the grouped failures exposed by the fresh audit, choose the clean product/architecture-conformant fix, implement it, and rerun the affected verification slices plus the aggregate audit before committing.') + $null = $lines.Add('- Diagnose the grouped failures exposed by the fresh audit, choose the clean product/architecture-conformant fix, implement it, add retained Playwright coverage for the new behavior when needed, and rerun the affected verification slices plus the aggregate audit before committing.') $null = $lines.Add('') $null = $lines.Add('Completion criteria:') $null = $lines.Add("- [$(Get-CheckboxMark -Value $State.Criteria31)] Root causes are recorded for the grouped failures.") - $null = $lines.Add("- [$(Get-CheckboxMark -Value $State.Criteria32)] Fixes land with focused regression coverage where practical.") + $null = $lines.Add("- [$(Get-CheckboxMark -Value $State.Criteria32)] Fixes land with focused regression coverage and retained Playwright scenario updates where practical.") $null = $lines.Add("- [$(Get-CheckboxMark -Value $State.Criteria33)] The rebuilt stack is retested before the iteration commit.") $null = $lines.Add('') $null = $lines.Add('## Execution Log') @@ -300,18 +381,27 @@ function Remove-PlaywrightOutput { } } +function Get-WorktreeChanges { + return @((git status --short --untracked-files=all) | Where-Object { $_ }) +} + $batchId = if ($StartingBatchId -gt 0) { $StartingBatchId } else { Get-HighestBatchId -SprintImplId $ImplId } for ($iteration = $StartIteration; $iteration -le $EndIteration; $iteration++) { $batchId++ + $preExistingWorktreeChanges = Get-WorktreeChanges $baselineCommit = (git rev-parse --short=9 HEAD).Trim() $sprintPath = Join-Path $sprintRoot ("SPRINT_{0}_{1:D3}_Platform_scratch_iteration_{2:D3}_full_route_action_audit.md" -f $ImplId, $batchId, $iteration) $state = New-SprintState -Iteration $iteration -BatchId $batchId -BaselineCommit $baselineCommit - $state.Decisions.Add('Decision: each scratch iteration remains a full wipe -> setup -> route/action audit -> grouped remediation loop; if the audit comes back clean, that still counts as a completed iteration because the full loop was executed.') + $state.Decisions.Add('Decision: each scratch iteration remains a full wipe -> setup -> route/page/action audit -> grouped remediation loop; if the audit comes back clean, that still counts as a completed iteration because the full loop was executed.') + $state.Decisions.Add('Decision: changed or newly discovered user flows must be converted into retained Playwright coverage before the next scratch iteration starts so the audit surface expands instead of rediscovering the same gaps manually.') $state.Decisions.Add('Risk: scratch rebuilds remain expensive, so verification stays Playwright-first with focused test/build slices rather than indiscriminate full-solution test runs.') $state.NextCheckpoints.Add('Finish the Stella-only wipe and capture the next zero-state setup outcome.') - $state.NextCheckpoints.Add('Run the full Playwright audit on the rebuilt stack before diagnosing any new fixes.') + $state.NextCheckpoints.Add('Run the full Playwright route/page/action audit, including changed-surface and ownership checks, on the rebuilt stack before diagnosing any new fixes.') Add-SprintLog -State $state -Update "Sprint created for the next scratch iteration after local commit ``$baselineCommit`` closed the previous clean baseline." -Owner 'QA' + if ($preExistingWorktreeChanges.Count -gt 0) { + Add-SprintLog -State $state -Update "Detected ``$($preExistingWorktreeChanges.Count)`` pre-existing worktree change(s) before the iteration started. Automatic sprint-only commit will be skipped so the final commit can stay grouped with the active fixes." -Owner 'QA / Developer' + } Write-SprintState -State $state -Path $sprintPath Remove-StellaRuntime @@ -351,7 +441,7 @@ for ($iteration = $StartIteration; $iteration -le $EndIteration; $iteration++) { $auditOk = ($passedSuites -eq [int]$auditReport.suiteCount) -and ($failedSuites -eq 0) -and ([int]$auditReport.retriedSuiteCount -eq 0) -and ([int]$auditReport.stabilizedAfterRetryCount -eq 0) $state.Criteria22 = $true $state.Criteria23 = $true - Add-SprintLog -State $state -Update "The aggregate audit finished with ``$passedSuites/$($auditReport.suiteCount)`` suites passed, ``$failedSuites`` failed suites, ``$($auditReport.retriedSuiteCount)`` retried suites, and ``$($auditReport.stabilizedAfterRetryCount)`` stabilized-after-retry suites." -Owner 'QA' + Add-SprintLog -State $state -Update "The aggregate audit finished with ``$passedSuites/$($auditReport.suiteCount)`` suites passed, ``$failedSuites`` failed suites, ``$($auditReport.retriedSuiteCount)`` retried suites, and ``$($auditReport.stabilizedAfterRetryCount)`` stabilized-after-retry suites across the retained route/page/action coverage set." -Owner 'QA' if (-not $auditOk) { $state.Status2 = 'DONE' $state.Status3 = 'DOING' @@ -372,13 +462,19 @@ for ($iteration = $StartIteration; $iteration -le $EndIteration; $iteration++) { $state.Criteria31 = $true $state.Criteria32 = $true $state.Criteria33 = $true - Add-SprintLog -State $state -Update 'No stable product defects surfaced on the rebuilt stack. The route/page/action audit completed cleanly without retries, so the iteration closes as a fully verified clean scratch pass.' -Owner 'QA / Architect / Developer' + Add-SprintLog -State $state -Update 'No stable product defects surfaced on the rebuilt stack. The retained route/page/action audit completed cleanly without retries, so the iteration closes as a fully verified clean scratch pass.' -Owner 'QA / Architect / Developer' $state.NextCheckpoints.Clear() $state.NextCheckpoints.Add('Start the next scratch iteration from another Stella-only wipe and documented setup rerun.') - $state.NextCheckpoints.Add('Repeat the full Playwright route/page/action audit on the next rebuilt stack before considering any new fixes.') + $state.NextCheckpoints.Add('Repeat the full Playwright route/page/action audit, including changed-surface and retained-scenario coverage, on the next rebuilt stack before considering any new fixes.') Write-SprintState -State $state -Path $sprintPath Remove-PlaywrightOutput + if ($preExistingWorktreeChanges.Count -gt 0) { + Add-SprintLog -State $state -Update 'Skipped the automatic sprint-only commit because the worktree was already dirty before this scratch iteration began. Manual grouped commit is required.' -Owner 'QA / Developer' + Write-SprintState -State $state -Path $sprintPath + continue + } + Invoke-External -FilePath 'git' -ArgumentList @('add', '--', $sprintPath) -WorkingDirectory $repoRoot Invoke-External -FilePath 'git' -ArgumentList @('commit', '-m', ("Record clean scratch setup iteration {0:D3}" -f $iteration)) -WorkingDirectory $repoRoot } diff --git a/scripts/setup.ps1 b/scripts/setup.ps1 index c06d8136a..68b5bc268 100644 --- a/scripts/setup.ps1 +++ b/scripts/setup.ps1 @@ -931,3 +931,4 @@ Write-Host ' Setup complete!' -ForegroundColor Green Write-Host ' Platform: https://stella-ops.local' -ForegroundColor Green Write-Host ' Docs: docs/dev/DEV_ENVIRONMENT_SETUP.md' -ForegroundColor Green Write-Host '=============================================' -ForegroundColor Green +exit 0 diff --git a/scripts/setup.sh b/scripts/setup.sh index b178459cd..0ecea10df 100644 --- a/scripts/setup.sh +++ b/scripts/setup.sh @@ -709,3 +709,4 @@ echo ' Setup complete!' echo ' Platform: https://stella-ops.local' echo ' Docs: docs/dev/DEV_ENVIRONMENT_SETUP.md' echo '=============================================' +exit 0 diff --git a/src/Web/StellaOps.Web/scripts/live-frontdoor-changed-surfaces.mjs b/src/Web/StellaOps.Web/scripts/live-frontdoor-changed-surfaces.mjs index 277c4d139..c48440f8a 100644 --- a/src/Web/StellaOps.Web/scripts/live-frontdoor-changed-surfaces.mjs +++ b/src/Web/StellaOps.Web/scripts/live-frontdoor-changed-surfaces.mjs @@ -435,6 +435,54 @@ async function verifySurfaceActions(context, surface) { return results; } +function collectSurfaceIssues(surface, record) { + const issues = []; + + if (!record.headingMatched) { + issues.push(`heading-mismatch:${surface.key}:${record.headingText || ''}`); + } + + for (const problemText of record.problemTexts) { + issues.push(`problem-text:${surface.key}:${problemText}`); + } + + for (const errorText of record.consoleErrors) { + issues.push(`console:${surface.key}:${errorText}`); + } + + for (const errorText of record.pageErrors) { + issues.push(`pageerror:${surface.key}:${errorText}`); + } + + for (const failure of record.requestFailures) { + issues.push(`requestfailed:${surface.key}:${failure.method} ${failure.url} ${failure.error}`); + } + + for (const failure of record.responseErrors) { + issues.push(`response:${surface.key}:${failure.status} ${failure.method} ${failure.url}`); + } + + if (surface.searchQuery) { + if (!record.search?.available) { + issues.push(`search-unavailable:${surface.key}`); + } else if ( + !record.search.resultsVisible + && record.search.suggestionCount === 0 + && (record.search.resultsText || '').length === 0 + ) { + issues.push(`search-empty:${surface.key}:${surface.searchQuery}`); + } + } + + for (const actionResult of record.actions) { + if (!actionResult.ok) { + issues.push(`action-failed:${surface.key}:${actionResult.key}:${actionResult.reason ?? actionResult.finalUrl ?? 'unknown'}`); + } + } + + return issues; +} + async function main() { mkdirSync(outputDirectory, { recursive: true }); const authReport = await authenticateFrontdoor({ @@ -452,17 +500,27 @@ async function main() { generatedAtUtc: new Date().toISOString(), baseUrl, surfaces: [], + issues: [], }; for (const surface of surfaceConfigs) { const surfaceReport = await inspectSurface(context, surface); surfaceReport.actions = await verifySurfaceActions(context, surface); + surfaceReport.issues = collectSurfaceIssues(surface, surfaceReport); + surfaceReport.ok = surfaceReport.issues.length === 0; report.surfaces.push(surfaceReport); + report.issues.push(...surfaceReport.issues); } await browser.close(); + report.failedSurfaceCount = report.surfaces.filter((surface) => !surface.ok).length; + report.runtimeIssueCount = report.issues.length; writeFileSync(outputPath, `${JSON.stringify(report, null, 2)}\n`, 'utf8'); process.stdout.write(`${JSON.stringify(report, null, 2)}\n`); + + if (report.failedSurfaceCount > 0 || report.runtimeIssueCount > 0) { + process.exit(1); + } } if (process.argv[1] && path.resolve(process.argv[1]) === __filename) { diff --git a/src/Web/StellaOps.Web/scripts/live-full-core-audit.mjs b/src/Web/StellaOps.Web/scripts/live-full-core-audit.mjs index f5005fe7e..21028ff52 100644 --- a/src/Web/StellaOps.Web/scripts/live-full-core-audit.mjs +++ b/src/Web/StellaOps.Web/scripts/live-full-core-audit.mjs @@ -12,6 +12,11 @@ const outputDir = path.join(webRoot, 'output', 'playwright'); const resultPath = path.join(outputDir, 'live-full-core-audit.json'); const suites = [ + { + name: 'route-surface-ownership-check', + script: 'live-route-surface-ownership-check.mjs', + reportPath: path.join(outputDir, 'live-route-surface-ownership-check.json'), + }, { name: 'frontdoor-canonical-route-sweep', script: 'live-frontdoor-canonical-route-sweep.mjs', @@ -47,6 +52,11 @@ const suites = [ script: 'live-user-reported-admin-trust-check.mjs', reportPath: path.join(outputDir, 'live-user-reported-admin-trust-check.json'), }, + { + name: 'changed-surfaces', + script: 'live-frontdoor-changed-surfaces.mjs', + reportPath: path.join(outputDir, 'live-frontdoor-changed-surfaces.json'), + }, { name: 'jobs-queues-action-sweep', script: 'live-jobs-queues-action-sweep.mjs', @@ -122,6 +132,7 @@ const failureCountKeys = new Set([ 'failedCount', 'failureCount', 'errorCount', + 'failedSurfaceCount', 'runtimeIssueCount', 'issueCount', 'unexpectedErrorCount', @@ -204,6 +215,10 @@ async function readReport(reportPath) { } } +async function persistSummary(summary) { + await writeFile(resultPath, `${JSON.stringify(summary, null, 2)}\n`, 'utf8'); +} + function runSuite({ name, script }) { return new Promise((resolve) => { const startedAt = Date.now(); @@ -236,6 +251,7 @@ async function main() { retriedSuiteCount: 0, stabilizedAfterRetryCount: 0, }; + await persistSummary(summary); for (const suite of suites) { process.stdout.write(`[live-full-core-audit] START ${suite.name}\n`); @@ -292,6 +308,7 @@ async function main() { }; summary.suites.push(result); + await persistSummary(summary); process.stdout.write( `[live-full-core-audit] DONE ${suite.name} ok=${ok} exitCode=${execution.exitCode ?? 'null'} ` + `signals=${failureSignals.length} durationMs=${execution.durationMs}` + @@ -313,7 +330,7 @@ async function main() { stabilizedAfterRetry: suite.stabilizedAfterRetry, })); - await writeFile(resultPath, `${JSON.stringify(summary, null, 2)}\n`, 'utf8'); + await persistSummary(summary); if (summary.failedSuiteCount > 0) { process.exitCode = 1; diff --git a/src/Web/StellaOps.Web/src/app/features/promotions/create-promotion.component.ts b/src/Web/StellaOps.Web/src/app/features/promotions/create-promotion.component.ts index 2cbba0f2f..8665421dd 100644 --- a/src/Web/StellaOps.Web/src/app/features/promotions/create-promotion.component.ts +++ b/src/Web/StellaOps.Web/src/app/features/promotions/create-promotion.component.ts @@ -648,7 +648,7 @@ export class CreatePromotionComponent implements OnInit { case 3: return this.releaseId().trim().length > 0 && this.targetEnvironmentId().length > 0; case 4: - return this.preview() !== null; + return this.preview() !== null && !this.loadingPreview(); case 5: return this.justification().trim().length >= 10; default: @@ -660,7 +660,8 @@ export class CreatePromotionComponent implements OnInit { return ( this.releaseId().trim().length > 0 && this.targetEnvironmentId().length > 0 && - this.justification().trim().length >= 10 + this.justification().trim().length >= 10 && + !this.loadingPreview() ); } @@ -684,12 +685,12 @@ export class CreatePromotionComponent implements OnInit { this.environments.set(items); this.loadingEnvironments.set(false); if (items.length > 0) { - this.activeStep.set(2); + this.promoteActiveStep(2); } if (preferredTargetEnvironmentId && items.some((item) => item.id === preferredTargetEnvironmentId)) { this.targetEnvironmentId.set(preferredTargetEnvironmentId); - this.activeStep.set(4); + this.promoteActiveStep(4); this.loadPreview(); } }); @@ -704,7 +705,10 @@ export class CreatePromotionComponent implements OnInit { } loadPreview(): void { - if (!this.releaseId().trim() || !this.targetEnvironmentId()) { + const releaseId = this.releaseId().trim(); + const targetEnvironmentId = this.targetEnvironmentId(); + + if (!releaseId || !targetEnvironmentId) { return; } @@ -712,7 +716,7 @@ export class CreatePromotionComponent implements OnInit { this.error.set(null); this.api - .getPromotionPreview(this.releaseId().trim(), this.targetEnvironmentId()) + .getPromotionPreview(releaseId, targetEnvironmentId) .pipe( catchError(() => { this.error.set('Failed to load gate preview.'); @@ -720,11 +724,17 @@ export class CreatePromotionComponent implements OnInit { }) ) .subscribe((preview) => { - this.preview.set(preview); this.loadingPreview.set(false); - if (preview) { - this.activeStep.set(4); + if (!preview) { + return; } + + if (this.releaseId().trim() !== releaseId || this.targetEnvironmentId() !== targetEnvironmentId) { + return; + } + + this.preview.set(preview); + this.promoteActiveStep(4); }); } @@ -793,4 +803,8 @@ export class CreatePromotionComponent implements OnInit { } }); } + + private promoteActiveStep(step: Step): void { + this.activeStep.update((current) => (current < step ? step : current)); + } } diff --git a/src/Web/StellaOps.Web/src/tests/releases/release-promotions-cutover.spec.ts b/src/Web/StellaOps.Web/src/tests/releases/release-promotions-cutover.spec.ts index ba6f26fce..62beb917e 100644 --- a/src/Web/StellaOps.Web/src/tests/releases/release-promotions-cutover.spec.ts +++ b/src/Web/StellaOps.Web/src/tests/releases/release-promotions-cutover.spec.ts @@ -1,6 +1,6 @@ import { TestBed } from '@angular/core/testing'; import { ActivatedRoute, Route, Router, convertToParamMap, provideRouter } from '@angular/router'; -import { of } from 'rxjs'; +import { of, Subject } from 'rxjs'; import { routes } from '../../app/app.routes'; import { APPROVAL_API } from '../../app/core/api/approval.client'; @@ -267,4 +267,155 @@ describe('CreatePromotionComponent release-context handoff', () => { }, ); }); + + it('blocks launch progression while gate preview refresh is still in flight', async () => { + const previewRefresh = new Subject<{ + releaseId: string; + releaseName: string; + sourceEnvironment: string; + targetEnvironment: string; + gateResults: []; + allGatesPassed: boolean; + requiredApprovers: number; + estimatedDeployTime: number; + warnings: string[]; + }>(); + const approvalApi = { + getAvailableEnvironments: jasmine.createSpy('getAvailableEnvironments').and.returnValue(of([])), + getPromotionPreview: jasmine.createSpy('getPromotionPreview').and.returnValue(previewRefresh.asObservable()), + submitPromotionRequest: jasmine.createSpy('submitPromotionRequest').and.returnValue(of(null)), + }; + + await TestBed.configureTestingModule({ + imports: [CreatePromotionComponent], + providers: [ + provideRouter([]), + { + provide: ActivatedRoute, + useValue: { + snapshot: { + queryParamMap: convertToParamMap({}), + }, + }, + }, + { provide: APPROVAL_API, useValue: approvalApi }, + ], + }).compileComponents(); + + const fixture = TestBed.createComponent(CreatePromotionComponent); + const component = fixture.componentInstance; + + component.releaseId.set('rel-123'); + component.targetEnvironmentId.set('env-production'); + component.preview.set({ + releaseId: 'rel-123', + releaseName: 'API Gateway', + sourceEnvironment: 'stage', + targetEnvironment: 'production', + gateResults: [], + allGatesPassed: true, + requiredApprovers: 2, + estimatedDeployTime: 120, + warnings: [], + }); + component.activeStep.set(4); + + component.loadPreview(); + + expect(component.loadingPreview()).toBeTrue(); + expect(component.canAdvance(4)).toBeFalse(); + + component.nextStep(); + + expect(component.activeStep()).toBe(4); + + previewRefresh.next({ + releaseId: 'rel-123', + releaseName: 'API Gateway', + sourceEnvironment: 'stage', + targetEnvironment: 'production', + gateResults: [], + allGatesPassed: true, + requiredApprovers: 2, + estimatedDeployTime: 120, + warnings: [], + }); + previewRefresh.complete(); + + expect(component.loadingPreview()).toBeFalse(); + expect(component.canAdvance(4)).toBeTrue(); + expect(component.activeStep()).toBe(4); + }); + + it('keeps the launch step visible when a late gate preview refresh resolves', async () => { + const previewRefresh = new Subject<{ + releaseId: string; + releaseName: string; + sourceEnvironment: string; + targetEnvironment: string; + gateResults: []; + allGatesPassed: boolean; + requiredApprovers: number; + estimatedDeployTime: number; + warnings: string[]; + }>(); + const approvalApi = { + getAvailableEnvironments: jasmine.createSpy('getAvailableEnvironments').and.returnValue(of([])), + getPromotionPreview: jasmine.createSpy('getPromotionPreview').and.returnValue(previewRefresh.asObservable()), + submitPromotionRequest: jasmine.createSpy('submitPromotionRequest').and.returnValue(of(null)), + }; + + await TestBed.configureTestingModule({ + imports: [CreatePromotionComponent], + providers: [ + provideRouter([]), + { + provide: ActivatedRoute, + useValue: { + snapshot: { + queryParamMap: convertToParamMap({}), + }, + }, + }, + { provide: APPROVAL_API, useValue: approvalApi }, + ], + }).compileComponents(); + + const fixture = TestBed.createComponent(CreatePromotionComponent); + const component = fixture.componentInstance; + + component.releaseId.set('rel-123'); + component.targetEnvironmentId.set('env-production'); + component.preview.set({ + releaseId: 'rel-123', + releaseName: 'API Gateway', + sourceEnvironment: 'stage', + targetEnvironment: 'production', + gateResults: [], + allGatesPassed: true, + requiredApprovers: 2, + estimatedDeployTime: 120, + warnings: [], + }); + component.justification.set('Release approval path validated end to end.'); + component.activeStep.set(6); + + component.loadPreview(); + + previewRefresh.next({ + releaseId: 'rel-123', + releaseName: 'API Gateway', + sourceEnvironment: 'stage', + targetEnvironment: 'production', + gateResults: [], + allGatesPassed: true, + requiredApprovers: 2, + estimatedDeployTime: 120, + warnings: [], + }); + previewRefresh.complete(); + + expect(component.activeStep()).toBe(6); + expect(component.canSubmit()).toBeTrue(); + }); }); diff --git a/src/Web/StellaOps.Web/tests/e2e/release-promotions-cutover.spec.ts b/src/Web/StellaOps.Web/tests/e2e/release-promotions-cutover.spec.ts index df1679bdc..67b77c2bd 100644 --- a/src/Web/StellaOps.Web/tests/e2e/release-promotions-cutover.spec.ts +++ b/src/Web/StellaOps.Web/tests/e2e/release-promotions-cutover.spec.ts @@ -259,6 +259,14 @@ test('legacy promotions create alias lands on the canonical wizard and submits a await page.locator('#target-env').selectOption('env-production'); await expect(page.getByRole('heading', { name: 'Gate Preview' })).toBeVisible(); await expect(page.getByText('All gates passed')).toBeVisible(); + await Promise.all([ + page.waitForResponse((response) => + response.request().method() === 'GET' && + response.url().includes('/api/v1/release-orchestrator/releases/rel-001/promotion-preview'), + ), + page.getByRole('button', { name: 'Refresh Gate Preview' }).click(), + ]); + await expect(page.getByText('All gates passed')).toBeVisible(); await page.getByRole('button', { name: 'Next ->' }).click(); await expect(page.getByRole('heading', { name: 'Approval Context' })).toBeVisible();