Add live search readiness and telemetry-off e2e coverage

This commit is contained in:
master
2026-03-07 21:49:41 +02:00
parent bbfa27ca39
commit e11c0a6b59
4 changed files with 86 additions and 15 deletions

View File

@@ -20,7 +20,7 @@
## Delivery Tracker
### QA-ZL2-001 - Expand deterministic Playwright for the simplified surface
Status: TODO
Status: DONE
Dependency: none
Owners: Test Automation
Task description:
@@ -33,7 +33,7 @@ Completion criteria:
- [ ] Covered starter chips always land on a meaningful result or explicit grounded fallback.
### QA-ZL2-002 - Run live ingestion-backed suggestion and readiness matrix
Status: TODO
Status: DONE
Dependency: QA-ZL2-001
Owners: Test Automation
Task description:
@@ -46,7 +46,7 @@ Completion criteria:
- [ ] Execution logs record the exact rebuild and Playwright commands.
### QA-ZL2-003 - Verify telemetry-off search flows
Status: TODO
Status: DONE
Dependency: QA-ZL2-002
Owners: Test Automation
Task description:
@@ -62,6 +62,7 @@ Completion criteria:
| Date (UTC) | Update | Owner |
| --- | --- | --- |
| 2026-03-07 | Sprint created to keep live-ingested suggestion correctness and telemetry-off behavior as explicit release gates. | Project Manager |
| 2026-03-07 | Added grounded-only live suggestion assertions, telemetry-off deterministic coverage, and reran the combined Playwright gate with live corpus preflight. Evidence: `.artifacts/stella-cli/StellaOps.Cli.exe advisoryai sources prepare --json`; `POST /v1/advisory-ai/index/rebuild`; `POST /v1/search/index/rebuild`; `npx playwright test tests/e2e/unified-search-experience-quality.e2e.spec.ts --config playwright.config.ts`; `LIVE_ADVISORYAI_SEARCH_BASE_URL=http://127.0.0.1:10451 npx playwright test tests/e2e/unified-search-contextual-suggestions.live.e2e.spec.ts --config playwright.config.ts`; final combined gate `20/20` passed. | Test Automation |
## Decisions & Risks
- Decision: live suggestion correctness is a product requirement; deterministic mocks alone are insufficient evidence.