Refine unified search answer shaping and viability

This commit is contained in:
master
2026-03-07 21:49:10 +02:00
parent 8f43378317
commit bbfa27ca39
15 changed files with 719 additions and 35 deletions

View File

@@ -123,9 +123,10 @@ Implemented in `src/AdvisoryAI/StellaOps.AdvisoryAI/KnowledgeSearch/KnowledgeSea
- Global search emits ambient context with each unified query: `currentRoute`, `visibleEntityKeys`, `recentSearches`, `sessionId`, and optional `lastAction` (`action`, `source`, `queryHint`, `domain`, `entityKey`, `route`, `occurredAt`).
- Contract remains backward-compatible: if an API deployment does not yet consume `lastAction`, unknown ambient fields are ignored and base search behavior remains unchanged.
- UI suggestion behavior now combines obvious route defaults with one strategic non-obvious suggestion and action-aware variants (for example, policy/VEX impact and incident timeline pivots).
- Search and AdvisoryAI also share a persisted operator mode (`Find`, `Explain`, `Act`); the UI uses the same mode to rank chips, compose Ask-AI prompts, and label assistant return flows, while backend query contracts remain backward-compatible.
- Unified search now also returns optional `contextAnswer` metadata with `status`, `code`, `summary`, `reason`, `evidence`, bounded `citations`, and bounded follow-up `questions`.
- `contextAnswer.status` is deterministic and must be one of `grounded`, `clarify`, or `insufficient`.
- Suggestion viability now returns additive readiness detail: `viabilityState` (`grounded`, `needs_clarification`, `no_match`, `scope_unready`, `corpus_unready`) plus `scopeReady`.
- Starter chips and page-owned questions must only be treated as executable when viability is `grounded`; broad clarify-only suggestions are intentionally hidden.
- Unified index lifecycle:
- Manual rebuild endpoint: `POST /v1/search/index/rebuild`.
- Optional background refresh loop is available via `KnowledgeSearchOptions` (`UnifiedAutoIndexEnabled`, `UnifiedAutoIndexOnStartup`, `UnifiedIndexRefreshIntervalSeconds`).
@@ -181,6 +182,10 @@ Global search now consumes AKS and supports:
- An answer-first search experience: every non-empty search renders a visible answer state (`grounded`, `clarify`, `insufficient`) before raw cards.
- Preferred source is backend `contextAnswer`.
- FE shell composition remains only as a backward-compatible fallback for older API deployments that do not emit `contextAnswer`.
- Answer shaping is query-driven:
- compare-like queries widen the blended-answer score band so close evidence clusters are summarized together
- scoped troubleshoot-like queries narrow the band so the dominant current-scope answer stays decisive
- empty or unready corpora return explicit `insufficient` codes instead of misleading `clarify` states
- Page-owned self-serve questions and clarifiers, defined in `docs/modules/ui/search-self-serve-contract.md`, so search can offer "Common questions" and recovery prompts without per-page conditionals in the component.
- Zero-result rescue actions that keep the current query visible while broadening scope, trying a related pivot, retrying with page context, or opening AdvisoryAI reformulation.
- AdvisoryAI evidence-first next-step cards that can return search pivots (`chat_next_step_search`, `chat_next_step_policy`) back into global search or open cited evidence/context directly.
@@ -352,8 +357,8 @@ docker compose -f devops/compose/docker-compose.advisoryai-knowledge-test.yml up
docker compose -f devops/compose/docker-compose.advisoryai-knowledge-test.yml ps
# Start the local AdvisoryAI service against that database
export ADVISORYAI__AdvisoryAI__KnowledgeSearch__ConnectionString="Host=localhost;Port=55432;Database=advisoryai_knowledge_test;Username=stellaops_knowledge;Password=stellaops_knowledge"
export ADVISORYAI__AdvisoryAI__KnowledgeSearch__RepositoryRoot="$(pwd)"
export AdvisoryAI__KnowledgeSearch__ConnectionString="Host=localhost;Port=55432;Database=advisoryai_knowledge_test;Username=stellaops_knowledge;Password=stellaops_knowledge"
export AdvisoryAI__KnowledgeSearch__RepositoryRoot="$(pwd)"
dotnet run --project "src/AdvisoryAI/StellaOps.AdvisoryAI.WebService/StellaOps.AdvisoryAI.WebService.csproj" --no-launch-profile
# In a second shell, rebuild the live corpus in the required order
@@ -404,7 +409,8 @@ Current live verification coverage:
- Rebuild order exercised against a running local service: `POST /v1/advisory-ai/index/rebuild` then `POST /v1/search/index/rebuild`
- Verified live query: `database connectivity`
- Verified live outcome: response includes `contextAnswer.status = grounded`, citations, and entity cards over ingested data
- Verified live suggestion lane: `src/Web/StellaOps.Web/tests/e2e/unified-search-contextual-suggestions.live.e2e.spec.ts` now preflights corpus readiness, validates suggestion viability, executes every surfaced Doctor suggestion, asserts grounded-or-clarify answer states, verifies follow-up chips after result open, and verifies Ask-AdvisoryAI inherits the live query context
- Verified live suggestion lane: `src/Web/StellaOps.Web/tests/e2e/unified-search-contextual-suggestions.live.e2e.spec.ts` now preflights corpus readiness, validates suggestion viability, executes every surfaced Doctor suggestion, asserts grounded answer states for surfaced live suggestions, verifies follow-up chips after result open, and verifies Ask-AdvisoryAI inherits the live query context
- Verified combined browser gate on 2026-03-07: `20/20` Playwright tests passed across deterministic UX, telemetry-off search flows, self-serve answer panel, and the live suggestion lane against the ingested local corpus
- Verified local corpus baseline on 2026-03-07 after `advisoryai sources prepare`: `documentCount = 470`, `chunkCount = 9050`, `apiOperationCount = 2190`, `doctorProjectionCount = 8`
- Other routes still rely on deterministic mock-backed Playwright coverage until their ingestion parity is explicitly verified

View File

@@ -122,6 +122,7 @@ sequenceDiagram
`POST /v1/search/query` response notes:
- Entity cards remain the primary retrieval payload.
- `contextAnswer` is the preferred answer-first surface for Web self-serve UX when present.
- `contextAnswer` is query-driven rather than mode-driven: compare-like requests may blend close evidence clusters, while scoped troubleshoot requests prefer a single decisive answer when one result clearly leads.
- `overflow` is additive and bounded so FE can show "outside the current page, but likely relevant" results without reintroducing a scope toggle.
- `overflow` is intentionally narrow: it is suppressed when the current-scope winner has a clear lead, so FE can trust the primary section as the best local answer.
- `coverage` is additive and bounded so FE can suppress misleading suggestions when the active corpus has no sensible candidates for that domain.
@@ -132,6 +133,10 @@ sequenceDiagram
`POST /v1/search/suggestions/evaluate` response notes:
- Intended for proactive suggestion chips and page-owned prompts before the user commits a search.
- Returns per-query viability plus bounded domain coverage.
- Additive fields:
- `viabilityState`: `grounded` | `needs_clarification` | `no_match` | `scope_unready` | `corpus_unready`
- `scopeReady`: `true` when the current route scope has indexed corpus behind it
- `viable=true` is reserved for suggestions that already have grounded evidence; clarify-only suggestions are not considered executable.
- Does not require telemetry and does not record answer-frame analytics.
OpenAPI contract presence is validated by integration test:

View File

@@ -125,6 +125,7 @@
- Return stricter suggestion viability signals so FE can suppress dead suggestions when the corpus is empty, stale, or outside the current route's supported domains.
- Keep out-of-scope overflow secondary and only when it materially improves the answer.
- Keep telemetry optional and separate from retrieval, ranking, suggestion gating, and history.
- Treat clarify-only suggestion candidates as non-executable; only grounded suggestions should survive preflight into the visible chip lane.
### Phase 3 - Live ingestion-backed readiness and regression gate
- Run deterministic Playwright against the simplified surface on every change.