Plan consolidated search corrective phases

This commit is contained in:
master
2026-03-07 18:59:17 +02:00
parent 820fb4ec25
commit 3d036a144e
6 changed files with 402 additions and 8 deletions

View File

@@ -0,0 +1,76 @@
# Sprint 20260307-025 - Search Consolidation Corrective Phases
## Topic & Scope
- Translate operator feedback on search and AdvisoryAI into explicit corrective product rules.
- Replace "incremental polish" planning with a phased execution plan that removes competing entry points and search-specific jargon.
- Document why the next phases prioritize shell behavior, zero-learning UI, ranking/overflow correctness, and live ingestion-backed reliability.
- Working directory: `docs/`.
- Expected evidence: updated strategy doc, phased sprint files, and a scoped docs-only commit.
## Dependencies & Concurrency
- Builds on `SPRINT_20260307_017_DOCS_search_zero_learning_consolidation_strategy.md`.
- Builds on implementation evidence from `SPRINT_20260307_019_AdvisoryAI_implicit_scope_weighting_and_suggestion_viability.md`, `SPRINT_20260307_020_FE_overflow_answer_consumption_and_chip_viability.md`, and `SPRINT_20260307_021_FE_live_search_suggestion_reliability_matrix.md`.
- Safe parallelism: implementation teams can proceed in `src/Web/StellaOps.Web/**` and `src/AdvisoryAI/**` after the corrective rules are published.
## Documentation Prerequisites
- `docs/modules/ui/search-zero-learning-primary-entry.md`
- `docs/modules/advisory-ai/knowledge-search.md`
- `docs/modules/advisory-ai/unified-search-architecture.md`
## Delivery Tracker
### DOCS-SC-001 - Publish corrective product rules
Status: DONE
Dependency: none
Owners: Project Manager, Documentation author
Task description:
- Capture the specific operator objections that remain valid after the earlier search work: split search/chat entry, mode-driven assistant framing, overly educational empty states, scope-mechanism exposition, and dead suggestions when ingestion is absent.
- Convert those objections into non-ambiguous product rules so later implementation does not drift back toward "teach the operator how to search."
Completion criteria:
- [x] The strategy doc records the remaining UX failures explicitly.
- [x] The product rules state search-first / assistant-second behavior and optional telemetry.
- [x] The strategy doc distinguishes process health from corpus readiness.
### DOCS-SC-002 - Define implementation phases and ownership
Status: DONE
Dependency: DOCS-SC-001
Owners: Project Manager
Task description:
- Break the corrective work into concrete implementation phases with single owning directories, dependencies, and acceptance criteria.
- Ensure the plan covers shell behavior, search surface simplification, backend ranking/telemetry rules, and live E2E verification.
Completion criteria:
- [x] Separate sprint files exist for the frontend shell work, frontend surface work, backend search work, and E2E verification.
- [x] Each sprint has actionable completion criteria rather than product prose.
- [x] Dependencies between the phases are explicit.
### DOCS-SC-003 - Publish why/how documentation
Status: DONE
Dependency: DOCS-SC-002
Owners: Documentation author
Task description:
- Update the "why and how" documentation so future contributors understand not only what to build, but also why certain UI patterns are forbidden.
- Keep the document focused on operator behavior and self-serve usability rather than internal implementation details.
Completion criteria:
- [x] The primary search strategy doc reflects the corrective phases.
- [x] The strategy doc explains why search cannot route-jump into another feature to open chat.
- [x] The sprint Decisions & Risks reference the updated strategy doc.
## Execution Log
| Date (UTC) | Update | Owner |
| --- | --- | --- |
| 2026-03-07 | Sprint created to convert operator feedback into corrective execution phases for consolidated search and assistant behavior. | Project Manager |
| 2026-03-07 | Published the corrective rules and the phased sprint stack after auditing the implemented search flow, the live Doctor ingestion lane, and the remaining route-jump/mode-driven chat behavior. | Project Manager |
## Decisions & Risks
- Decision: search remains the primary operator entry and AdvisoryAI becomes a secondary deep-dive from that same surface, not a competing route flow.
- Decision: telemetry stays optional and must not influence ranking, suggestions, history, or assistant availability.
- Risk: previous partial implementations can create false confidence because some controls were already removed while route-jump and mode-driven assistant behavior remained.
- Mitigation: split the corrective work into shell, surface, backend, and E2E phases with explicit completion criteria.
- Reference: `docs/modules/ui/search-zero-learning-primary-entry.md`
## Next Checkpoints
- 2026-03-08: complete Phase A shell unification and remove route-jump chat opens.
- 2026-03-09: complete Phase B surface collapse and validate with Playwright.

View File

@@ -0,0 +1,72 @@
# Sprint 20260307-026 - FE Global Assistant Drawer Unification
## Topic & Scope
- Make search the single primary entry by opening AdvisoryAI from a shell-level drawer instead of navigating into `/security/triage`.
- Remove visible mode switching from the assistant and stop mode-driven prompt generation in search-to-chat handoff.
- Reuse one global assistant entry from the top bar, answer panel, and result cards.
- Working directory: `src/Web/StellaOps.Web/`.
- Expected evidence: frontend implementation, unit tests, Playwright coverage, and a scoped commit.
## Dependencies & Concurrency
- Depends on `SPRINT_20260307_025_DOCS_search_consolidation_corrective_phases.md`.
- Builds on existing search-to-chat context plumbing and `stellaops-chat`.
- Safe parallelism: backend ranking work can proceed in `src/AdvisoryAI/**` if it does not change the Web contract during this sprint.
## Documentation Prerequisites
- `docs/modules/ui/search-zero-learning-primary-entry.md`
- `src/Web/StellaOps.Web/AGENTS.md`
## Delivery Tracker
### FE-SC-001 - Host AdvisoryAI at shell scope
Status: TODO
Dependency: none
Owners: Developer
Task description:
- Introduce a shell-level assistant host/drawer so search-originated chat can open on any page without route changes.
- Keep the drawer keyboard-accessible and reusable by page-level assistant affordances.
Completion criteria:
- [ ] The application shell hosts the assistant drawer.
- [ ] Search-originated chat opens on the current page without navigating to `/security/triage`.
- [ ] Existing triage-specific assistant affordances either reuse the same host or are removed if redundant.
### FE-SC-002 - Remove visible search/assistant mode framing
Status: TODO
Dependency: FE-SC-001
Owners: Developer
Task description:
- Remove `Find / Explain / Act` controls and mode labels from the assistant surface.
- Replace mode-driven prompt text with prompt composition based on current page, query, citations, and answer evidence.
Completion criteria:
- [ ] Chat no longer displays mode buttons or mode badges.
- [ ] Search-to-chat handoff prompts are built from query/evidence context rather than a persisted mode.
- [ ] The result still gives useful next-step suggestions without requiring mode selection.
### FE-SC-003 - Verify search-first assistant flows
Status: TODO
Dependency: FE-SC-002
Owners: Test Automation
Task description:
- Add unit and Playwright coverage proving that search is the primary entry and the assistant is a secondary deep-dive that stays on-page.
Completion criteria:
- [ ] Unit tests cover the new shell-level assistant host behavior.
- [ ] Playwright covers top-bar chat launch, answer-panel Ask-AdvisoryAI, and result-card Ask-AdvisoryAI without route jumps.
- [ ] Search context inheritance remains intact in the assistant.
## Execution Log
| Date (UTC) | Update | Owner |
| --- | --- | --- |
| 2026-03-07 | Sprint created from operator feedback on the split search/chat experience and the remaining route-jump behavior. | Project Manager |
## Decisions & Risks
- Decision: the global top-bar search is the canonical assistant launcher; route navigation to open chat is no longer acceptable for the primary flow.
- Risk: the current triage host owns both a page and an assistant drawer, so shell-level hosting could create duplicate drawers if not unified carefully.
- Mitigation: centralize assistant open/close state behind a single shell-level service/host and update search handoff tests.
- Reference: `docs/modules/ui/search-zero-learning-primary-entry.md`
## Next Checkpoints
- 2026-03-08: land shell-level drawer host and remove search route jumps.
- 2026-03-08: land assistant UI simplification and targeted E2E.

View File

@@ -0,0 +1,71 @@
# Sprint 20260307-027 - FE Zero-Learning Search Surface Collapse
## Topic & Scope
- Strip the search surface down to the operator essentials: query, correction, answer, primary cards, overflow when needed, viable suggestions, and successful history.
- Remove or minimize UI elements that teach Stella structure or expose ranking mechanics.
- Working directory: `src/Web/StellaOps.Web/`.
- Expected evidence: simplified search UI implementation, regression tests, Playwright behavior coverage, and a scoped commit.
## Dependencies & Concurrency
- Depends on `SPRINT_20260307_026_FE_global_assistant_drawer_unification.md`.
- Safe parallelism: backend weighting work can proceed in `src/AdvisoryAI/**` if the unified response contract remains backward-compatible.
## Documentation Prerequisites
- `docs/modules/ui/search-zero-learning-primary-entry.md`
- `src/Web/StellaOps.Web/AGENTS.md`
## Delivery Tracker
### FE-SC-101 - Collapse empty-state education
Status: TODO
Dependency: none
Owners: Developer
Task description:
- Remove large domain guides, quick-action teaching panels, and other empty-state elements that require the operator to understand Stella taxonomy before asking a question.
- Replace them with a small set of viable, page-aware search starts only when useful.
Completion criteria:
- [ ] Empty-state UI does not present domain cards or "learn Stella" quick links as the main action.
- [ ] Suggestions shown in the empty state remain executable and page-aware.
- [ ] Search history remains successful-only and is visually low-emphasis.
### FE-SC-102 - Simplify in-result cues
Status: TODO
Dependency: FE-SC-101
Owners: Developer
Task description:
- Keep `Did you mean` directly under the input and make overflow presentation subtle.
- Stop explaining scope weighting mechanics in the main flow; show the better in-scope answer first, then overflow only when needed.
Completion criteria:
- [ ] `Did you mean` is visually attached to the input.
- [ ] Scope weighting hints are removed or translated into plain operator-facing result labels.
- [ ] Overflow only appears when present and is visually secondary to the best in-scope answer.
### FE-SC-103 - Harden suggestion and history behavior
Status: TODO
Dependency: FE-SC-102
Owners: Developer, Test Automation
Task description:
- Ensure no-result searches do not enter local/server history rendering and that dead suggestions remain suppressed.
- Exercise user flows that previously felt broken from the empty state.
Completion criteria:
- [ ] No-result queries do not reappear in rendered history.
- [ ] Suggestion clicks from the empty state remain non-dead-end flows.
- [ ] Playwright covers history, suggestions, did-you-mean placement, and overflow presentation.
## Execution Log
| Date (UTC) | Update | Owner |
| --- | --- | --- |
| 2026-03-07 | Sprint created from operator feedback that the search surface still teaches too much of Stella instead of simply helping. | Project Manager |
## Decisions & Risks
- Decision: the empty state should help the operator start, not explain Stella's information architecture.
- Risk: removing guidance too aggressively could leave brand-new users with a blank surface if viable suggestions are unavailable.
- Mitigation: preserve small, executable page-aware starters while removing large educational panels and ranking jargon.
- Reference: `docs/modules/ui/search-zero-learning-primary-entry.md`
## Next Checkpoints
- 2026-03-09: land the empty-state collapse and plain-language overflow cues.
- 2026-03-09: capture Playwright evidence for the no-learning search surface.

View File

@@ -0,0 +1,75 @@
# Sprint 20260307-028 - AdvisoryAI Consolidated Ranking, Blending, and Optional Telemetry
## Topic & Scope
- Refine backend unified search so current-page/entity/action relevance dominates when sensible and cross-scope overflow appears only when the in-scope answer is weak.
- Tighten answer composition so blended summaries appear only when the leading evidence cluster is genuinely close.
- Make analytics/feedback telemetry explicitly optional at the contract and client-consumption level.
- Working directory: `src/AdvisoryAI/`.
- Expected evidence: backend implementation, targeted tests, updated docs, and a scoped commit.
## Dependencies & Concurrency
- Depends on `SPRINT_20260307_025_DOCS_search_consolidation_corrective_phases.md`.
- Informs `SPRINT_20260307_027_FE_zero_learning_search_surface_collapse.md` and `SPRINT_20260307_029_FE_live_consolidated_search_journeys.md`.
- Safe parallelism: frontend shell work can proceed in `src/Web/StellaOps.Web/**` if the contract remains additive.
## Documentation Prerequisites
- `docs/modules/ui/search-zero-learning-primary-entry.md`
- `docs/modules/advisory-ai/knowledge-search.md`
- `docs/modules/advisory-ai/unified-search-architecture.md`
- `src/AdvisoryAI/AGENTS.md`
## Delivery Tracker
### AI-SC-001 - Refine current-scope dominance and overflow thresholds
Status: TODO
Dependency: none
Owners: Developer
Task description:
- Improve weighting from current route, visible entities, and recent actions so the backend more consistently returns the best in-scope answer first.
- Only emit overflow when in-scope evidence is materially weaker or absent.
Completion criteria:
- [ ] Tests cover stronger current-scope/entity/action weighting.
- [ ] Overflow appears only when the in-scope answer is weak enough to justify it.
- [ ] Coverage metadata still explains the winning scope without requiring frontend heuristics.
### AI-SC-002 - Tighten answer dominance vs blend behavior
Status: TODO
Dependency: AI-SC-001
Owners: Developer
Task description:
- Adjust the answer composer so a single dominant result yields a decisive answer while genuinely close clusters yield a blended summary with citations.
Completion criteria:
- [ ] Service tests cover decisive-answer and blended-answer cases separately.
- [ ] The answer status and citations remain deterministic.
- [ ] Clarify and insufficient fallbacks remain explicit.
### AI-SC-003 - Keep telemetry optional
Status: TODO
Dependency: AI-SC-002
Owners: Developer
Task description:
- Ensure analytics, feedback, and quality telemetry can be disabled by configuration without affecting retrieval, suggestions, history rendering, or chat handoff.
- Document the configuration and fallback behavior.
Completion criteria:
- [ ] Search behavior remains identical with telemetry disabled.
- [ ] Tests cover telemetry-disabled behavior.
- [ ] Docs state clearly that telemetry is optional and not a functional dependency.
## Execution Log
| Date (UTC) | Update | Owner |
| --- | --- | --- |
| 2026-03-07 | Sprint created to finish the backend half of the consolidated search UX after operator feedback on weighting, blending, and telemetry. | Project Manager |
## Decisions & Risks
- Decision: ranking/explanation behavior must be driven by corpus evidence and page context, not by a user-selected mode.
- Decision: telemetry is optional infrastructure, not part of the search correctness path.
- Risk: over-weighting current scope could hide clearly better out-of-scope answers.
- Mitigation: keep additive coverage metadata and explicit overflow criteria in tests.
- Reference: `docs/modules/ui/search-zero-learning-primary-entry.md`
## Next Checkpoints
- 2026-03-09: land stronger current-scope thresholds and answer-blending tests.
- 2026-03-10: land telemetry-off verification and doc updates.

View File

@@ -0,0 +1,71 @@
# Sprint 20260307-029 - FE Live Consolidated Search Journeys
## Topic & Scope
- Prove the consolidated search experience end to end with both deterministic mocks and live ingested corpora.
- Cover the operator journeys that matter after the redesign: empty-state suggestions, did-you-mean, grounded answer, overflow fallback, search history hygiene, and assistant deep-dive.
- Working directory: `src/Web/StellaOps.Web/tests/e2e/`.
- Expected evidence: Playwright suites, exact setup commands, execution logs, and scoped commits.
## Dependencies & Concurrency
- Depends on `SPRINT_20260307_026_FE_global_assistant_drawer_unification.md`.
- Depends on `SPRINT_20260307_027_FE_zero_learning_search_surface_collapse.md`.
- Depends on `SPRINT_20260307_028_AdvisoryAI_consolidated_ranking_blending_and_optional_telemetry.md`.
- Safe parallelism: live suites may run in parallel only when they use isolated services or a read-only shared corpus.
## Documentation Prerequisites
- `docs/modules/ui/search-zero-learning-primary-entry.md`
- `docs/modules/advisory-ai/knowledge-search.md`
- `src/AdvisoryAI/__Tests/INFRASTRUCTURE.md`
## Delivery Tracker
### QA-SC-001 - Expand deterministic Playwright coverage
Status: TODO
Dependency: none
Owners: Test Automation
Task description:
- Extend mock-backed Playwright coverage to the redesigned search/assistant flows so UI regressions are caught quickly.
Completion criteria:
- [ ] Deterministic E2E covers top-bar assistant open, did-you-mean placement, history hygiene, overflow rendering, and no-dead-end suggestions.
- [ ] No route-jump to `/security/triage` remains in the search-first flows.
### QA-SC-002 - Expand live ingested search coverage
Status: TODO
Dependency: QA-SC-001
Owners: Test Automation
Task description:
- Keep live Doctor coverage and expand to additional supported domains when ingestion parity exists.
- Fail fast on empty corpus/setup errors before UI assertions begin.
Completion criteria:
- [ ] Live E2E keeps corpus readiness preflight.
- [ ] Every surfaced live suggestion on covered pages resolves to a non-dead-end state.
- [ ] Execution log records exact live commands and pass counts.
### QA-SC-003 - Verify telemetry-off behavior
Status: TODO
Dependency: QA-SC-002
Owners: Test Automation
Task description:
- Add test coverage for telemetry-disabled search usage so optional analytics does not regress the operator flow.
Completion criteria:
- [ ] Search remains usable when telemetry endpoints/config are disabled.
- [ ] The assistant handoff still works with telemetry disabled.
- [ ] The sprint log records the telemetry-off evidence path.
## Execution Log
| Date (UTC) | Update | Owner |
| --- | --- | --- |
| 2026-03-07 | Sprint created to prove the consolidated search experience from all relevant angles after the redesign phases land. | Project Manager |
## Decisions & Risks
- Decision: live E2E remains mandatory because mocked suggestions cannot prove corpus viability.
- Risk: findings/policy/VEX live coverage may lag until ingestion parity exists for those domains.
- Mitigation: keep deterministic E2E broad, keep Doctor live coverage strict, and expand live domains only when ingestion parity is confirmed.
- Reference: `docs/modules/ui/search-zero-learning-primary-entry.md`
## Next Checkpoints
- 2026-03-10: rerun the live Doctor matrix on the consolidated UX.
- 2026-03-11: add the next supported live domain after ingestion parity verification.

View File

@@ -5,20 +5,29 @@
- The current model asks the user to understand too many concepts before they can succeed: `Find/Explain/Act`, explicit scope switching, recovery actions, and a separate AdvisoryAI destination.
- Live verification also shows a real reliability gap: suggestion chips can render even when the active corpus cannot answer them. On 2026-03-07, the running AdvisoryAI service at `http://127.1.0.44` rebuilt with `documentCount=0`, `chunkCount=0`, and `doctorProjectionCount=0`, while unified overlays rebuilt with only `chunkCount=14`. Querying `database connectivity` then returned zero cards. The current UX still offered that suggestion.
## What still fails after the first pass
- The product still exposes two entry mental models: global search and a route-jumping AdvisoryAI experience. Search-originated chat actions currently navigate into `/security/triage`, which breaks page continuity and makes AdvisoryAI feel like a separate destination.
- The assistant still exposes `Find / Explain / Act` framing in chat. That pushes the user to learn the system's internal reasoning model instead of simply asking for help.
- Empty-state search still carries too much product education: domain guides, quick actions, context exposition, and explanatory labels. That is useful for experts but too loud for a self-serve default.
- Current-scope behavior is correct as a ranking concept, but the UI still talks about the mechanism instead of simply showing the best in-scope answer first and only then out-of-scope overflow.
- Suggestion reliability is fixed only when the active corpus is actually ingested. A healthy process with an empty corpus is still a bad operator experience unless the UI suppresses or fails those paths explicitly.
## Product rules
1. Search is the primary entry point.
2. AdvisoryAI is a secondary deep-dive opened from search, not a competing starting point.
2. AdvisoryAI is a secondary deep-dive opened from search, not a competing starting point or route jump.
3. Search should infer relevance and intent; the user should not need to choose a mode.
4. Search should bias to the current page context automatically; the user should not need to toggle scope.
5. Search should never advertise a suggestion that cannot execute against the active corpus.
6. Search history should contain only successful searches.
7. Telemetry remains optional. Search behavior must not depend on analytics emission.
8. Search UI must avoid teaching Stella terminology or search mechanics before the operator has even started.
## Target interaction model
### Entry
- Keep one primary search field in the top bar.
- Add a compact AdvisoryAI launcher icon beside the search input.
- Opening chat from that icon inherits the current page, current query, and top evidence when available.
- Opening chat must stay on the current page through a global drawer or equivalent shell-level host. Search should not navigate the operator into a different feature route just to ask follow-up questions.
### Query understanding
- Infer likely user intent from the query and visible page context.
@@ -33,6 +42,7 @@
- Remove the recovery panel.
- If top results are close in score, compose one short summary across them.
- If one result is clearly dominant, present that answer first and then cards.
- Prefer concise operator-facing labels over implementation-facing labels. The UI should show "best match here" and "also relevant elsewhere", not expose ranking jargon.
### Suggestions
- Page suggestions become executable search intents, not static chips.
@@ -44,6 +54,7 @@
### History
- Persist and render only searches that returned results.
- Keep history clear as a low-emphasis icon action, not a labeled call-to-action.
- Existing history entries with zero results should be ignored on load so prior failed experiments do not pollute the empty state.
## Runtime rules
### Implicit scope weighting
@@ -63,10 +74,28 @@
- Live Playwright coverage must assert that every surfaced suggestion returns visible results.
- A service health check alone is not enough. On 2026-03-07, `http://127.1.0.44/health` returned `200` while the live knowledge rebuild returned `documentCount=0`; the product still surfaced dead chips. Corpus readiness is the gate, not process liveness.
## Phase map
- Phase 1: FE primary-entry consolidation and removal of explicit search controls.
- Phase 2: AdvisoryAI implicit scope weighting, answer blending, and suggestion viability.
- Phase 3: FE consumption of overflow results and executable suggestion contracts.
- Implemented on 2026-03-07: backend `contextAnswer` is now preferred over frontend heuristics, overflow renders as a secondary result section, and suggestion viability preflight suppresses dead chips before they are shown.
- Phase 4: Live Playwright reliability matrix with corpus preflight and chip-success guarantees.
- Implemented on 2026-03-07: the live suite now rebuilds the active corpus, fails fast on empty knowledge projections, iterates every surfaced Doctor suggestion, and verifies Ask-AdvisoryAI inherits the live search context.
## Corrective execution phases
### Phase A - Search-first and assistant-second shell behavior
- Replace route-jumping chat opens with a global assistant drawer hosted by the shell.
- Remove mode switching from the assistant UI and prompt generation.
- Make the top-bar search icon the canonical assistant entry from any page.
### Phase B - Zero-learning search surface collapse
- Remove or minimize empty-state product education that teaches Stella structure instead of solving the current question.
- Keep `Did you mean` immediately under the input and keep overflow presentation subtle.
- Keep suggestions executable, short, and page-aware without extra explanation burden.
### Phase C - Ranking, answer blending, and optional telemetry
- Refine current-page/entity/action weighting so in-scope answers dominate when sensible.
- Blend answers only when the top evidence cluster is genuinely close; otherwise show a decisive primary answer.
- Ensure analytics, feedback, and quality telemetry can be disabled without breaking search, suggestions, history, or chat handoff.
### Phase D - Live ingestion-backed reliability matrix
- Keep mock-backed E2E for deterministic UI shape.
- Add live ingested E2E for the actual suggestions and answer experience on supported domains.
- Fail fast when corpus rebuild/readiness is missing so dead suggestions are treated as setup failures, not flaky UI tests.
## Current state
- Implemented before the corrective phases: explicit scope/mode/recovery controls were removed from the main search flow, implicit current-scope weighting and overflow contracts were added, and suggestion viability preflight now suppresses dead chips before render.
- Implemented before the corrective phases: the live Doctor suggestion suite now rebuilds the active corpus, fails on empty knowledge projections, iterates every surfaced suggestion, and verifies Ask-AdvisoryAI inherits the live search context.
- Still pending from the corrective phases: shell-level assistant unification, assistant de-mode-ing, empty-state collapse, broader live-page matrices, and explicit client-side telemetry opt-out.