Refine unified search answer shaping and viability
This commit is contained in:
@@ -21,7 +21,7 @@
|
|||||||
## Delivery Tracker
|
## Delivery Tracker
|
||||||
|
|
||||||
### AI-ZL2-001 - Remove mode-shaped answer assumptions from unified search
|
### AI-ZL2-001 - Remove mode-shaped answer assumptions from unified search
|
||||||
Status: TODO
|
Status: DONE
|
||||||
Dependency: none
|
Dependency: none
|
||||||
Owners: Developer
|
Owners: Developer
|
||||||
Task description:
|
Task description:
|
||||||
@@ -34,7 +34,7 @@ Completion criteria:
|
|||||||
- [ ] Clarify and insufficient fallbacks remain deterministic.
|
- [ ] Clarify and insufficient fallbacks remain deterministic.
|
||||||
|
|
||||||
### AI-ZL2-002 - Strengthen suggestion viability and corpus readiness signals
|
### AI-ZL2-002 - Strengthen suggestion viability and corpus readiness signals
|
||||||
Status: TODO
|
Status: DONE
|
||||||
Dependency: AI-ZL2-001
|
Dependency: AI-ZL2-001
|
||||||
Owners: Developer
|
Owners: Developer
|
||||||
Task description:
|
Task description:
|
||||||
@@ -47,7 +47,7 @@ Completion criteria:
|
|||||||
- [ ] Integration tests cover live-readiness and empty-corpus behavior.
|
- [ ] Integration tests cover live-readiness and empty-corpus behavior.
|
||||||
|
|
||||||
### AI-ZL2-003 - Keep telemetry optional and non-blocking
|
### AI-ZL2-003 - Keep telemetry optional and non-blocking
|
||||||
Status: TODO
|
Status: DONE
|
||||||
Dependency: AI-ZL2-002
|
Dependency: AI-ZL2-002
|
||||||
Owners: Developer
|
Owners: Developer
|
||||||
Task description:
|
Task description:
|
||||||
@@ -63,6 +63,8 @@ Completion criteria:
|
|||||||
| Date (UTC) | Update | Owner |
|
| Date (UTC) | Update | Owner |
|
||||||
| --- | --- | --- |
|
| --- | --- | --- |
|
||||||
| 2026-03-07 | Sprint created for the backend half of the operator correction pass on unified search. | Project Manager |
|
| 2026-03-07 | Sprint created for the backend half of the operator correction pass on unified search. | Project Manager |
|
||||||
|
| 2026-03-07 | Implementation started: tightening query-driven answer shaping, corpus-readiness-aware suggestion viability, and telemetry-disabled parity before rerunning backend and Playwright suggestion suites. | Developer |
|
||||||
|
| 2026-03-07 | Completed backend query-driven answer shaping, grounded-only suggestion viability, and telemetry-off parity. Evidence: `dotnet build src/AdvisoryAI/__Tests/StellaOps.AdvisoryAI.Tests/StellaOps.AdvisoryAI.Tests.csproj -v minimal -p:BuildInParallel=false -p:UseSharedCompilation=false`; `StellaOps.AdvisoryAI.Tests.exe -class StellaOps.AdvisoryAI.Tests.UnifiedSearch.UnifiedSearchServiceTests`; `StellaOps.AdvisoryAI.Tests.exe -class StellaOps.AdvisoryAI.Tests.Integration.UnifiedSearchEndpointsIntegrationTests`; `StellaOps.AdvisoryAI.Tests.exe -class StellaOps.AdvisoryAI.Tests.UnifiedSearch.SearchAnalyticsServiceTests`; `StellaOps.AdvisoryAI.Tests.exe -class StellaOps.AdvisoryAI.Tests.UnifiedSearch.SearchQualityMonitorTests`; live query `database connectivity` returned `contextAnswer.status = grounded`. | Developer |
|
||||||
|
|
||||||
## Decisions & Risks
|
## Decisions & Risks
|
||||||
- Decision: FE should not be responsible for choosing answer mode or rescuing dead suggestions.
|
- Decision: FE should not be responsible for choosing answer mode or rescuing dead suggestions.
|
||||||
|
|||||||
@@ -123,9 +123,10 @@ Implemented in `src/AdvisoryAI/StellaOps.AdvisoryAI/KnowledgeSearch/KnowledgeSea
|
|||||||
- Global search emits ambient context with each unified query: `currentRoute`, `visibleEntityKeys`, `recentSearches`, `sessionId`, and optional `lastAction` (`action`, `source`, `queryHint`, `domain`, `entityKey`, `route`, `occurredAt`).
|
- Global search emits ambient context with each unified query: `currentRoute`, `visibleEntityKeys`, `recentSearches`, `sessionId`, and optional `lastAction` (`action`, `source`, `queryHint`, `domain`, `entityKey`, `route`, `occurredAt`).
|
||||||
- Contract remains backward-compatible: if an API deployment does not yet consume `lastAction`, unknown ambient fields are ignored and base search behavior remains unchanged.
|
- Contract remains backward-compatible: if an API deployment does not yet consume `lastAction`, unknown ambient fields are ignored and base search behavior remains unchanged.
|
||||||
- UI suggestion behavior now combines obvious route defaults with one strategic non-obvious suggestion and action-aware variants (for example, policy/VEX impact and incident timeline pivots).
|
- UI suggestion behavior now combines obvious route defaults with one strategic non-obvious suggestion and action-aware variants (for example, policy/VEX impact and incident timeline pivots).
|
||||||
- Search and AdvisoryAI also share a persisted operator mode (`Find`, `Explain`, `Act`); the UI uses the same mode to rank chips, compose Ask-AI prompts, and label assistant return flows, while backend query contracts remain backward-compatible.
|
|
||||||
- Unified search now also returns optional `contextAnswer` metadata with `status`, `code`, `summary`, `reason`, `evidence`, bounded `citations`, and bounded follow-up `questions`.
|
- Unified search now also returns optional `contextAnswer` metadata with `status`, `code`, `summary`, `reason`, `evidence`, bounded `citations`, and bounded follow-up `questions`.
|
||||||
- `contextAnswer.status` is deterministic and must be one of `grounded`, `clarify`, or `insufficient`.
|
- `contextAnswer.status` is deterministic and must be one of `grounded`, `clarify`, or `insufficient`.
|
||||||
|
- Suggestion viability now returns additive readiness detail: `viabilityState` (`grounded`, `needs_clarification`, `no_match`, `scope_unready`, `corpus_unready`) plus `scopeReady`.
|
||||||
|
- Starter chips and page-owned questions must only be treated as executable when viability is `grounded`; broad clarify-only suggestions are intentionally hidden.
|
||||||
- Unified index lifecycle:
|
- Unified index lifecycle:
|
||||||
- Manual rebuild endpoint: `POST /v1/search/index/rebuild`.
|
- Manual rebuild endpoint: `POST /v1/search/index/rebuild`.
|
||||||
- Optional background refresh loop is available via `KnowledgeSearchOptions` (`UnifiedAutoIndexEnabled`, `UnifiedAutoIndexOnStartup`, `UnifiedIndexRefreshIntervalSeconds`).
|
- Optional background refresh loop is available via `KnowledgeSearchOptions` (`UnifiedAutoIndexEnabled`, `UnifiedAutoIndexOnStartup`, `UnifiedIndexRefreshIntervalSeconds`).
|
||||||
@@ -181,6 +182,10 @@ Global search now consumes AKS and supports:
|
|||||||
- An answer-first search experience: every non-empty search renders a visible answer state (`grounded`, `clarify`, `insufficient`) before raw cards.
|
- An answer-first search experience: every non-empty search renders a visible answer state (`grounded`, `clarify`, `insufficient`) before raw cards.
|
||||||
- Preferred source is backend `contextAnswer`.
|
- Preferred source is backend `contextAnswer`.
|
||||||
- FE shell composition remains only as a backward-compatible fallback for older API deployments that do not emit `contextAnswer`.
|
- FE shell composition remains only as a backward-compatible fallback for older API deployments that do not emit `contextAnswer`.
|
||||||
|
- Answer shaping is query-driven:
|
||||||
|
- compare-like queries widen the blended-answer score band so close evidence clusters are summarized together
|
||||||
|
- scoped troubleshoot-like queries narrow the band so the dominant current-scope answer stays decisive
|
||||||
|
- empty or unready corpora return explicit `insufficient` codes instead of misleading `clarify` states
|
||||||
- Page-owned self-serve questions and clarifiers, defined in `docs/modules/ui/search-self-serve-contract.md`, so search can offer "Common questions" and recovery prompts without per-page conditionals in the component.
|
- Page-owned self-serve questions and clarifiers, defined in `docs/modules/ui/search-self-serve-contract.md`, so search can offer "Common questions" and recovery prompts without per-page conditionals in the component.
|
||||||
- Zero-result rescue actions that keep the current query visible while broadening scope, trying a related pivot, retrying with page context, or opening AdvisoryAI reformulation.
|
- Zero-result rescue actions that keep the current query visible while broadening scope, trying a related pivot, retrying with page context, or opening AdvisoryAI reformulation.
|
||||||
- AdvisoryAI evidence-first next-step cards that can return search pivots (`chat_next_step_search`, `chat_next_step_policy`) back into global search or open cited evidence/context directly.
|
- AdvisoryAI evidence-first next-step cards that can return search pivots (`chat_next_step_search`, `chat_next_step_policy`) back into global search or open cited evidence/context directly.
|
||||||
@@ -352,8 +357,8 @@ docker compose -f devops/compose/docker-compose.advisoryai-knowledge-test.yml up
|
|||||||
docker compose -f devops/compose/docker-compose.advisoryai-knowledge-test.yml ps
|
docker compose -f devops/compose/docker-compose.advisoryai-knowledge-test.yml ps
|
||||||
|
|
||||||
# Start the local AdvisoryAI service against that database
|
# Start the local AdvisoryAI service against that database
|
||||||
export ADVISORYAI__AdvisoryAI__KnowledgeSearch__ConnectionString="Host=localhost;Port=55432;Database=advisoryai_knowledge_test;Username=stellaops_knowledge;Password=stellaops_knowledge"
|
export AdvisoryAI__KnowledgeSearch__ConnectionString="Host=localhost;Port=55432;Database=advisoryai_knowledge_test;Username=stellaops_knowledge;Password=stellaops_knowledge"
|
||||||
export ADVISORYAI__AdvisoryAI__KnowledgeSearch__RepositoryRoot="$(pwd)"
|
export AdvisoryAI__KnowledgeSearch__RepositoryRoot="$(pwd)"
|
||||||
dotnet run --project "src/AdvisoryAI/StellaOps.AdvisoryAI.WebService/StellaOps.AdvisoryAI.WebService.csproj" --no-launch-profile
|
dotnet run --project "src/AdvisoryAI/StellaOps.AdvisoryAI.WebService/StellaOps.AdvisoryAI.WebService.csproj" --no-launch-profile
|
||||||
|
|
||||||
# In a second shell, rebuild the live corpus in the required order
|
# In a second shell, rebuild the live corpus in the required order
|
||||||
@@ -404,7 +409,8 @@ Current live verification coverage:
|
|||||||
- Rebuild order exercised against a running local service: `POST /v1/advisory-ai/index/rebuild` then `POST /v1/search/index/rebuild`
|
- Rebuild order exercised against a running local service: `POST /v1/advisory-ai/index/rebuild` then `POST /v1/search/index/rebuild`
|
||||||
- Verified live query: `database connectivity`
|
- Verified live query: `database connectivity`
|
||||||
- Verified live outcome: response includes `contextAnswer.status = grounded`, citations, and entity cards over ingested data
|
- Verified live outcome: response includes `contextAnswer.status = grounded`, citations, and entity cards over ingested data
|
||||||
- Verified live suggestion lane: `src/Web/StellaOps.Web/tests/e2e/unified-search-contextual-suggestions.live.e2e.spec.ts` now preflights corpus readiness, validates suggestion viability, executes every surfaced Doctor suggestion, asserts grounded-or-clarify answer states, verifies follow-up chips after result open, and verifies Ask-AdvisoryAI inherits the live query context
|
- Verified live suggestion lane: `src/Web/StellaOps.Web/tests/e2e/unified-search-contextual-suggestions.live.e2e.spec.ts` now preflights corpus readiness, validates suggestion viability, executes every surfaced Doctor suggestion, asserts grounded answer states for surfaced live suggestions, verifies follow-up chips after result open, and verifies Ask-AdvisoryAI inherits the live query context
|
||||||
|
- Verified combined browser gate on 2026-03-07: `20/20` Playwright tests passed across deterministic UX, telemetry-off search flows, self-serve answer panel, and the live suggestion lane against the ingested local corpus
|
||||||
- Verified local corpus baseline on 2026-03-07 after `advisoryai sources prepare`: `documentCount = 470`, `chunkCount = 9050`, `apiOperationCount = 2190`, `doctorProjectionCount = 8`
|
- Verified local corpus baseline on 2026-03-07 after `advisoryai sources prepare`: `documentCount = 470`, `chunkCount = 9050`, `apiOperationCount = 2190`, `doctorProjectionCount = 8`
|
||||||
- Other routes still rely on deterministic mock-backed Playwright coverage until their ingestion parity is explicitly verified
|
- Other routes still rely on deterministic mock-backed Playwright coverage until their ingestion parity is explicitly verified
|
||||||
|
|
||||||
|
|||||||
@@ -122,6 +122,7 @@ sequenceDiagram
|
|||||||
`POST /v1/search/query` response notes:
|
`POST /v1/search/query` response notes:
|
||||||
- Entity cards remain the primary retrieval payload.
|
- Entity cards remain the primary retrieval payload.
|
||||||
- `contextAnswer` is the preferred answer-first surface for Web self-serve UX when present.
|
- `contextAnswer` is the preferred answer-first surface for Web self-serve UX when present.
|
||||||
|
- `contextAnswer` is query-driven rather than mode-driven: compare-like requests may blend close evidence clusters, while scoped troubleshoot requests prefer a single decisive answer when one result clearly leads.
|
||||||
- `overflow` is additive and bounded so FE can show "outside the current page, but likely relevant" results without reintroducing a scope toggle.
|
- `overflow` is additive and bounded so FE can show "outside the current page, but likely relevant" results without reintroducing a scope toggle.
|
||||||
- `overflow` is intentionally narrow: it is suppressed when the current-scope winner has a clear lead, so FE can trust the primary section as the best local answer.
|
- `overflow` is intentionally narrow: it is suppressed when the current-scope winner has a clear lead, so FE can trust the primary section as the best local answer.
|
||||||
- `coverage` is additive and bounded so FE can suppress misleading suggestions when the active corpus has no sensible candidates for that domain.
|
- `coverage` is additive and bounded so FE can suppress misleading suggestions when the active corpus has no sensible candidates for that domain.
|
||||||
@@ -132,6 +133,10 @@ sequenceDiagram
|
|||||||
`POST /v1/search/suggestions/evaluate` response notes:
|
`POST /v1/search/suggestions/evaluate` response notes:
|
||||||
- Intended for proactive suggestion chips and page-owned prompts before the user commits a search.
|
- Intended for proactive suggestion chips and page-owned prompts before the user commits a search.
|
||||||
- Returns per-query viability plus bounded domain coverage.
|
- Returns per-query viability plus bounded domain coverage.
|
||||||
|
- Additive fields:
|
||||||
|
- `viabilityState`: `grounded` | `needs_clarification` | `no_match` | `scope_unready` | `corpus_unready`
|
||||||
|
- `scopeReady`: `true` when the current route scope has indexed corpus behind it
|
||||||
|
- `viable=true` is reserved for suggestions that already have grounded evidence; clarify-only suggestions are not considered executable.
|
||||||
- Does not require telemetry and does not record answer-frame analytics.
|
- Does not require telemetry and does not record answer-frame analytics.
|
||||||
|
|
||||||
OpenAPI contract presence is validated by integration test:
|
OpenAPI contract presence is validated by integration test:
|
||||||
|
|||||||
@@ -125,6 +125,7 @@
|
|||||||
- Return stricter suggestion viability signals so FE can suppress dead suggestions when the corpus is empty, stale, or outside the current route's supported domains.
|
- Return stricter suggestion viability signals so FE can suppress dead suggestions when the corpus is empty, stale, or outside the current route's supported domains.
|
||||||
- Keep out-of-scope overflow secondary and only when it materially improves the answer.
|
- Keep out-of-scope overflow secondary and only when it materially improves the answer.
|
||||||
- Keep telemetry optional and separate from retrieval, ranking, suggestion gating, and history.
|
- Keep telemetry optional and separate from retrieval, ranking, suggestion gating, and history.
|
||||||
|
- Treat clarify-only suggestion candidates as non-executable; only grounded suggestions should survive preflight into the visible chip lane.
|
||||||
|
|
||||||
### Phase 3 - Live ingestion-backed readiness and regression gate
|
### Phase 3 - Live ingestion-backed readiness and regression gate
|
||||||
- Run deterministic Playwright against the simplified surface on every change.
|
- Run deterministic Playwright against the simplified surface on every change.
|
||||||
|
|||||||
@@ -647,7 +647,9 @@ public static class UnifiedSearchEndpoints
|
|||||||
Code = suggestion.Code,
|
Code = suggestion.Code,
|
||||||
CardCount = suggestion.CardCount,
|
CardCount = suggestion.CardCount,
|
||||||
LeadingDomain = suggestion.LeadingDomain,
|
LeadingDomain = suggestion.LeadingDomain,
|
||||||
Reason = suggestion.Reason
|
Reason = suggestion.Reason,
|
||||||
|
ViabilityState = suggestion.ViabilityState,
|
||||||
|
ScopeReady = suggestion.ScopeReady
|
||||||
}).ToArray(),
|
}).ToArray(),
|
||||||
Coverage = response.Coverage is null
|
Coverage = response.Coverage is null
|
||||||
? null
|
? null
|
||||||
@@ -990,6 +992,10 @@ public sealed record UnifiedSearchSuggestionViabilityApiResult
|
|||||||
public string? LeadingDomain { get; init; }
|
public string? LeadingDomain { get; init; }
|
||||||
|
|
||||||
public string Reason { get; init; } = string.Empty;
|
public string Reason { get; init; } = string.Empty;
|
||||||
|
|
||||||
|
public string ViabilityState { get; init; } = "no_match";
|
||||||
|
|
||||||
|
public bool ScopeReady { get; init; }
|
||||||
}
|
}
|
||||||
|
|
||||||
public sealed record UnifiedSearchApiCard
|
public sealed record UnifiedSearchApiCard
|
||||||
|
|||||||
@@ -0,0 +1,12 @@
|
|||||||
|
namespace StellaOps.AdvisoryAI.KnowledgeSearch;
|
||||||
|
|
||||||
|
internal interface IKnowledgeSearchCorpusAvailabilityStore
|
||||||
|
{
|
||||||
|
Task<IReadOnlyList<KnowledgeSearchDomainCorpusAvailability>> GetDomainCorpusAvailabilityAsync(
|
||||||
|
KnowledgeSearchFilter? filters,
|
||||||
|
CancellationToken cancellationToken);
|
||||||
|
}
|
||||||
|
|
||||||
|
internal sealed record KnowledgeSearchDomainCorpusAvailability(
|
||||||
|
string Domain,
|
||||||
|
int ChunkCount);
|
||||||
@@ -100,6 +100,13 @@ public sealed class KnowledgeSearchOptions
|
|||||||
/// </summary>
|
/// </summary>
|
||||||
public bool SearchQualityMonitorEnabled { get; set; } = true;
|
public bool SearchQualityMonitorEnabled { get; set; } = true;
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Enables optional analytics/feedback telemetry for search quality and ranking feedback loops.
|
||||||
|
/// When false, search retrieval, suggestions, and history remain functional but telemetry events
|
||||||
|
/// and feedback persistence are skipped.
|
||||||
|
/// </summary>
|
||||||
|
public bool SearchTelemetryEnabled { get; set; } = true;
|
||||||
|
|
||||||
/// <summary>
|
/// <summary>
|
||||||
/// Interval in seconds for quality-monitor refresh.
|
/// Interval in seconds for quality-monitor refresh.
|
||||||
/// </summary>
|
/// </summary>
|
||||||
|
|||||||
@@ -7,7 +7,7 @@ using System.Text.Json;
|
|||||||
|
|
||||||
namespace StellaOps.AdvisoryAI.KnowledgeSearch;
|
namespace StellaOps.AdvisoryAI.KnowledgeSearch;
|
||||||
|
|
||||||
internal sealed class PostgresKnowledgeSearchStore : IKnowledgeSearchStore, IAsyncDisposable
|
internal sealed class PostgresKnowledgeSearchStore : IKnowledgeSearchStore, IKnowledgeSearchCorpusAvailabilityStore, IAsyncDisposable
|
||||||
{
|
{
|
||||||
private static readonly JsonDocument EmptyJsonDocument = JsonDocument.Parse("{}");
|
private static readonly JsonDocument EmptyJsonDocument = JsonDocument.Parse("{}");
|
||||||
|
|
||||||
@@ -399,6 +399,75 @@ internal sealed class PostgresKnowledgeSearchStore : IKnowledgeSearchStore, IAsy
|
|||||||
return await ReadChunkRowsAsync(command, cancellationToken).ConfigureAwait(false);
|
return await ReadChunkRowsAsync(command, cancellationToken).ConfigureAwait(false);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
public async Task<IReadOnlyList<KnowledgeSearchDomainCorpusAvailability>> GetDomainCorpusAvailabilityAsync(
|
||||||
|
KnowledgeSearchFilter? filters,
|
||||||
|
CancellationToken cancellationToken)
|
||||||
|
{
|
||||||
|
if (!IsConfigured())
|
||||||
|
{
|
||||||
|
return [];
|
||||||
|
}
|
||||||
|
|
||||||
|
var kinds = ResolveKinds(filters);
|
||||||
|
var tags = ResolveTags(filters);
|
||||||
|
var normalizedProduct = NormalizeOptional(filters?.Product);
|
||||||
|
var normalizedVersion = NormalizeOptional(filters?.Version);
|
||||||
|
var normalizedService = NormalizeOptional(filters?.Service);
|
||||||
|
var normalizedTenant = NormalizeOptional(filters?.Tenant);
|
||||||
|
|
||||||
|
const string sql = """
|
||||||
|
SELECT
|
||||||
|
COALESCE(NULLIF(lower(c.metadata->>'domain'), ''), 'knowledge') AS domain,
|
||||||
|
COUNT(*)::integer AS chunk_count
|
||||||
|
FROM advisoryai.kb_chunk AS c
|
||||||
|
INNER JOIN advisoryai.kb_doc AS d
|
||||||
|
ON d.doc_id = c.doc_id
|
||||||
|
WHERE (@kind_count = 0 OR c.kind = ANY(@kinds))
|
||||||
|
AND (@tag_count = 0 OR EXISTS (
|
||||||
|
SELECT 1
|
||||||
|
FROM jsonb_array_elements_text(COALESCE(c.metadata->'tags', '[]'::jsonb)) AS tag(value)
|
||||||
|
WHERE lower(tag.value) = ANY(@tags)
|
||||||
|
))
|
||||||
|
AND (@product = '' OR lower(d.product) = lower(@product))
|
||||||
|
AND (@version = '' OR lower(d.version) = lower(@version))
|
||||||
|
AND (@service = '' OR lower(COALESCE(c.metadata->>'service', '')) = lower(@service))
|
||||||
|
AND (
|
||||||
|
@tenant = ''
|
||||||
|
OR lower(COALESCE(c.metadata->>'tenant', 'global')) = lower(@tenant)
|
||||||
|
OR lower(COALESCE(c.metadata->>'tenant', 'global')) = 'global'
|
||||||
|
)
|
||||||
|
GROUP BY 1
|
||||||
|
ORDER BY chunk_count DESC, domain ASC;
|
||||||
|
""";
|
||||||
|
|
||||||
|
await using var command = CreateCommand(sql, TimeSpan.FromSeconds(2));
|
||||||
|
command.Parameters.AddWithValue("kind_count", kinds.Length);
|
||||||
|
command.Parameters.AddWithValue(
|
||||||
|
"kinds",
|
||||||
|
NpgsqlDbType.Array | NpgsqlDbType.Text,
|
||||||
|
kinds.Length == 0 ? Array.Empty<string>() : kinds);
|
||||||
|
command.Parameters.AddWithValue("tag_count", tags.Length);
|
||||||
|
command.Parameters.AddWithValue(
|
||||||
|
"tags",
|
||||||
|
NpgsqlDbType.Array | NpgsqlDbType.Text,
|
||||||
|
tags.Length == 0 ? Array.Empty<string>() : tags);
|
||||||
|
command.Parameters.AddWithValue("product", normalizedProduct);
|
||||||
|
command.Parameters.AddWithValue("version", normalizedVersion);
|
||||||
|
command.Parameters.AddWithValue("service", normalizedService);
|
||||||
|
command.Parameters.AddWithValue("tenant", normalizedTenant);
|
||||||
|
|
||||||
|
var domains = new List<KnowledgeSearchDomainCorpusAvailability>();
|
||||||
|
await using var reader = await command.ExecuteReaderAsync(cancellationToken).ConfigureAwait(false);
|
||||||
|
while (await reader.ReadAsync(cancellationToken).ConfigureAwait(false))
|
||||||
|
{
|
||||||
|
domains.Add(new KnowledgeSearchDomainCorpusAvailability(
|
||||||
|
reader.GetString(0),
|
||||||
|
reader.GetInt32(1)));
|
||||||
|
}
|
||||||
|
|
||||||
|
return domains;
|
||||||
|
}
|
||||||
|
|
||||||
public async ValueTask DisposeAsync()
|
public async ValueTask DisposeAsync()
|
||||||
{
|
{
|
||||||
if (_dataSource.IsValueCreated && _dataSource.Value is not null)
|
if (_dataSource.IsValueCreated && _dataSource.Value is not null)
|
||||||
|
|||||||
@@ -16,6 +16,7 @@ Source of truth: `docs/implplan/SPRINT_20260113_005_ADVISORYAI_controlled_conver
|
|||||||
| AI-SELF-004 | DONE | Optional self-serve telemetry now captures answer frames, reformulations, rescue actions, and abandonment signals via hashed session ids; quality metrics and alerts expose the gaps operationally. |
|
| AI-SELF-004 | DONE | Optional self-serve telemetry now captures answer frames, reformulations, rescue actions, and abandonment signals via hashed session ids; quality metrics and alerts expose the gaps operationally. |
|
||||||
| AI-SELF-006 | DONE | Live ingestion-backed answer verification succeeded on the Doctor/knowledge route after local rebuild. |
|
| AI-SELF-006 | DONE | Live ingestion-backed answer verification succeeded on the Doctor/knowledge route after local rebuild. |
|
||||||
| SPRINT_20260307_019-AI-ZL | DONE | Unified search now applies implicit current-scope weighting, emits additive `overflow`/`coverage`, blends close top answers, and evaluates suggestion viability without requiring telemetry. |
|
| SPRINT_20260307_019-AI-ZL | DONE | Unified search now applies implicit current-scope weighting, emits additive `overflow`/`coverage`, blends close top answers, and evaluates suggestion viability without requiring telemetry. |
|
||||||
|
| SPRINT_20260307_033-AI-ZL | DONE | Unified search now derives answer blending from query/context, exposes grounded-only suggestion viability with corpus-readiness states, and keeps analytics/feedback telemetry fully optional. |
|
||||||
| SPRINT_20260222_051-AKS-INGEST | DONE | Added deterministic AKS ingestion controls: markdown allow-list manifest loading, OpenAPI aggregate source path support, and doctor control projection integration for search chunks, including fallback doctor metadata hydration from controls projection fields. |
|
| SPRINT_20260222_051-AKS-INGEST | DONE | Added deterministic AKS ingestion controls: markdown allow-list manifest loading, OpenAPI aggregate source path support, and doctor control projection integration for search chunks, including fallback doctor metadata hydration from controls projection fields. |
|
||||||
| AUDIT-0017-M | DONE | Maintainability audit for StellaOps.AdvisoryAI. |
|
| AUDIT-0017-M | DONE | Maintainability audit for StellaOps.AdvisoryAI. |
|
||||||
| AUDIT-0017-T | DONE | Test coverage audit for StellaOps.AdvisoryAI. |
|
| AUDIT-0017-T | DONE | Test coverage audit for StellaOps.AdvisoryAI. |
|
||||||
|
|||||||
@@ -23,6 +23,11 @@ internal sealed class SearchAnalyticsService
|
|||||||
|
|
||||||
public async Task RecordEventAsync(SearchAnalyticsEvent evt, CancellationToken ct = default)
|
public async Task RecordEventAsync(SearchAnalyticsEvent evt, CancellationToken ct = default)
|
||||||
{
|
{
|
||||||
|
if (!_options.SearchTelemetryEnabled)
|
||||||
|
{
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
var recordedAt = DateTimeOffset.UtcNow;
|
var recordedAt = DateTimeOffset.UtcNow;
|
||||||
var persistedEvent = SanitizeEvent(evt);
|
var persistedEvent = SanitizeEvent(evt);
|
||||||
if (string.IsNullOrWhiteSpace(_options.ConnectionString))
|
if (string.IsNullOrWhiteSpace(_options.ConnectionString))
|
||||||
@@ -67,7 +72,7 @@ internal sealed class SearchAnalyticsService
|
|||||||
|
|
||||||
public async Task RecordEventsAsync(IReadOnlyList<SearchAnalyticsEvent> events, CancellationToken ct = default)
|
public async Task RecordEventsAsync(IReadOnlyList<SearchAnalyticsEvent> events, CancellationToken ct = default)
|
||||||
{
|
{
|
||||||
if (events.Count == 0)
|
if (events.Count == 0 || !_options.SearchTelemetryEnabled)
|
||||||
{
|
{
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
@@ -127,6 +132,11 @@ internal sealed class SearchAnalyticsService
|
|||||||
|
|
||||||
public async Task<IReadOnlyDictionary<string, int>> GetPopularityMapAsync(string tenantId, int days = 30, CancellationToken ct = default)
|
public async Task<IReadOnlyDictionary<string, int>> GetPopularityMapAsync(string tenantId, int days = 30, CancellationToken ct = default)
|
||||||
{
|
{
|
||||||
|
if (!_options.SearchTelemetryEnabled)
|
||||||
|
{
|
||||||
|
return new Dictionary<string, int>(StringComparer.Ordinal);
|
||||||
|
}
|
||||||
|
|
||||||
if (string.IsNullOrWhiteSpace(_options.ConnectionString))
|
if (string.IsNullOrWhiteSpace(_options.ConnectionString))
|
||||||
{
|
{
|
||||||
return BuildFallbackPopularityMap(tenantId, days);
|
return BuildFallbackPopularityMap(tenantId, days);
|
||||||
|
|||||||
@@ -44,6 +44,11 @@ internal sealed class SearchQualityMonitor
|
|||||||
|
|
||||||
public async Task StoreFeedbackAsync(SearchFeedbackEntry entry, CancellationToken ct = default)
|
public async Task StoreFeedbackAsync(SearchFeedbackEntry entry, CancellationToken ct = default)
|
||||||
{
|
{
|
||||||
|
if (!_options.SearchTelemetryEnabled)
|
||||||
|
{
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
var createdAt = DateTimeOffset.UtcNow;
|
var createdAt = DateTimeOffset.UtcNow;
|
||||||
var persistedEntry = SanitizeFeedbackEntry(entry);
|
var persistedEntry = SanitizeFeedbackEntry(entry);
|
||||||
if (string.IsNullOrWhiteSpace(_options.ConnectionString))
|
if (string.IsNullOrWhiteSpace(_options.ConnectionString))
|
||||||
|
|||||||
@@ -167,7 +167,9 @@ public sealed record SearchSuggestionViabilityResult(
|
|||||||
string Code,
|
string Code,
|
||||||
int CardCount,
|
int CardCount,
|
||||||
string? LeadingDomain,
|
string? LeadingDomain,
|
||||||
string Reason);
|
string Reason,
|
||||||
|
string ViabilityState = "no_match",
|
||||||
|
bool ScopeReady = false);
|
||||||
|
|
||||||
public sealed record EntityCard
|
public sealed record EntityCard
|
||||||
{
|
{
|
||||||
|
|||||||
@@ -26,6 +26,8 @@ internal sealed class UnifiedSearchService : IUnifiedSearchService
|
|||||||
private const int CoverageCandidateWindow = 24;
|
private const int CoverageCandidateWindow = 24;
|
||||||
private const double OverflowScoreBandRatio = 0.04d;
|
private const double OverflowScoreBandRatio = 0.04d;
|
||||||
private const double BlendedAnswerScoreBandRatio = 0.025d;
|
private const double BlendedAnswerScoreBandRatio = 0.025d;
|
||||||
|
private const double CompareBlendedAnswerScoreBandRatio = 0.06d;
|
||||||
|
private const double ScopedTroubleshootBlendedAnswerScoreBandRatio = 0.018d;
|
||||||
private readonly KnowledgeSearchOptions _options;
|
private readonly KnowledgeSearchOptions _options;
|
||||||
private readonly UnifiedSearchOptions _unifiedOptions;
|
private readonly UnifiedSearchOptions _unifiedOptions;
|
||||||
private readonly IKnowledgeSearchStore _store;
|
private readonly IKnowledgeSearchStore _store;
|
||||||
@@ -68,6 +70,22 @@ internal sealed class UnifiedSearchService : IUnifiedSearchService
|
|||||||
SuggestionPreflight
|
SuggestionPreflight
|
||||||
}
|
}
|
||||||
|
|
||||||
|
private sealed record CorpusAvailabilitySnapshot(
|
||||||
|
bool Known,
|
||||||
|
string? CurrentScopeDomain,
|
||||||
|
int CurrentScopeChunkCount,
|
||||||
|
int TotalChunkCount,
|
||||||
|
IReadOnlyDictionary<string, int> DomainChunkCounts)
|
||||||
|
{
|
||||||
|
public static CorpusAvailabilitySnapshot Unknown(string? currentScopeDomain) =>
|
||||||
|
new(
|
||||||
|
Known: false,
|
||||||
|
CurrentScopeDomain: currentScopeDomain,
|
||||||
|
CurrentScopeChunkCount: 0,
|
||||||
|
TotalChunkCount: 0,
|
||||||
|
DomainChunkCounts: new Dictionary<string, int>(StringComparer.OrdinalIgnoreCase));
|
||||||
|
}
|
||||||
|
|
||||||
public UnifiedSearchService(
|
public UnifiedSearchService(
|
||||||
IOptions<KnowledgeSearchOptions> options,
|
IOptions<KnowledgeSearchOptions> options,
|
||||||
IKnowledgeSearchStore store,
|
IKnowledgeSearchStore store,
|
||||||
@@ -136,6 +154,11 @@ internal sealed class UnifiedSearchService : IUnifiedSearchService
|
|||||||
return new SearchSuggestionViabilityResponse([], fallbackCoverage);
|
return new SearchSuggestionViabilityResponse([], fallbackCoverage);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
var storeFilter = BuildStoreFilter(request.Filters);
|
||||||
|
var corpusAvailability = await LoadCorpusAvailabilityAsync(
|
||||||
|
storeFilter,
|
||||||
|
request.Ambient,
|
||||||
|
cancellationToken).ConfigureAwait(false);
|
||||||
var results = new List<SearchSuggestionViabilityResult>(normalizedQueries.Length);
|
var results = new List<SearchSuggestionViabilityResult>(normalizedQueries.Length);
|
||||||
UnifiedSearchCoverage? aggregateCoverage = null;
|
UnifiedSearchCoverage? aggregateCoverage = null;
|
||||||
|
|
||||||
@@ -149,16 +172,18 @@ internal sealed class UnifiedSearchService : IUnifiedSearchService
|
|||||||
IncludeSynthesis: false,
|
IncludeSynthesis: false,
|
||||||
IncludeDebug: false,
|
IncludeDebug: false,
|
||||||
Ambient: request.Ambient),
|
Ambient: request.Ambient),
|
||||||
cancellationToken,
|
cancellationToken,
|
||||||
SearchExecutionKind.SuggestionPreflight).ConfigureAwait(false);
|
SearchExecutionKind.SuggestionPreflight,
|
||||||
|
corpusAvailability).ConfigureAwait(false);
|
||||||
|
|
||||||
aggregateCoverage = MergeCoverage(aggregateCoverage, response.Coverage);
|
aggregateCoverage = MergeCoverage(aggregateCoverage, response.Coverage);
|
||||||
var cardCount = response.Cards.Count + (response.Overflow?.Cards.Count ?? 0);
|
var cardCount = response.Cards.Count + (response.Overflow?.Cards.Count ?? 0);
|
||||||
var answer = response.ContextAnswer;
|
var answer = response.ContextAnswer;
|
||||||
|
var viabilityState = DetermineSuggestionViabilityState(cardCount, answer, corpusAvailability);
|
||||||
|
|
||||||
results.Add(new SearchSuggestionViabilityResult(
|
results.Add(new SearchSuggestionViabilityResult(
|
||||||
Query: query,
|
Query: query,
|
||||||
Viable: cardCount > 0 || string.Equals(answer?.Status, "clarify", StringComparison.OrdinalIgnoreCase),
|
Viable: IsSuggestionViable(viabilityState),
|
||||||
Status: answer?.Status ?? "insufficient",
|
Status: answer?.Status ?? "insufficient",
|
||||||
Code: answer?.Code ?? "no_grounded_evidence",
|
Code: answer?.Code ?? "no_grounded_evidence",
|
||||||
CardCount: cardCount,
|
CardCount: cardCount,
|
||||||
@@ -166,7 +191,9 @@ internal sealed class UnifiedSearchService : IUnifiedSearchService
|
|||||||
?? response.Overflow?.Cards.FirstOrDefault()?.Domain
|
?? response.Overflow?.Cards.FirstOrDefault()?.Domain
|
||||||
?? response.Coverage?.Domains.FirstOrDefault(static domain => domain.HasVisibleResults)?.Domain
|
?? response.Coverage?.Domains.FirstOrDefault(static domain => domain.HasVisibleResults)?.Domain
|
||||||
?? response.Coverage?.CurrentScopeDomain,
|
?? response.Coverage?.CurrentScopeDomain,
|
||||||
Reason: answer?.Reason ?? "No grounded evidence matched the suggestion in the active corpus."));
|
Reason: BuildSuggestionViabilityReason(answer, viabilityState, corpusAvailability),
|
||||||
|
ViabilityState: viabilityState,
|
||||||
|
ScopeReady: IsCurrentScopeReady(corpusAvailability)));
|
||||||
}
|
}
|
||||||
|
|
||||||
return new SearchSuggestionViabilityResponse(
|
return new SearchSuggestionViabilityResponse(
|
||||||
@@ -177,7 +204,8 @@ internal sealed class UnifiedSearchService : IUnifiedSearchService
|
|||||||
private async Task<UnifiedSearchResponse> SearchAsyncInternal(
|
private async Task<UnifiedSearchResponse> SearchAsyncInternal(
|
||||||
UnifiedSearchRequest request,
|
UnifiedSearchRequest request,
|
||||||
CancellationToken cancellationToken,
|
CancellationToken cancellationToken,
|
||||||
SearchExecutionKind executionKind)
|
SearchExecutionKind executionKind,
|
||||||
|
CorpusAvailabilitySnapshot? corpusAvailabilityOverride = null)
|
||||||
{
|
{
|
||||||
ArgumentNullException.ThrowIfNull(request);
|
ArgumentNullException.ThrowIfNull(request);
|
||||||
|
|
||||||
@@ -241,12 +269,14 @@ internal sealed class UnifiedSearchService : IUnifiedSearchService
|
|||||||
ContextEntityBoosts = contextEntityBoosts
|
ContextEntityBoosts = contextEntityBoosts
|
||||||
};
|
};
|
||||||
|
|
||||||
|
var storeFilter = BuildStoreFilter(request.Filters);
|
||||||
|
var currentScopeDomain = ResolveAmbientScopeDomain(request.Ambient);
|
||||||
|
var corpusAvailability = corpusAvailabilityOverride
|
||||||
|
?? await LoadCorpusAvailabilityAsync(storeFilter, request.Ambient, cancellationToken).ConfigureAwait(false);
|
||||||
|
|
||||||
var topK = ResolveTopK(request.K);
|
var topK = ResolveTopK(request.K);
|
||||||
var timeout = TimeSpan.FromMilliseconds(Math.Max(250, _options.QueryTimeoutMs));
|
var timeout = TimeSpan.FromMilliseconds(Math.Max(250, _options.QueryTimeoutMs));
|
||||||
|
|
||||||
// Build domain-aware filter for the store query
|
|
||||||
var storeFilter = BuildStoreFilter(request.Filters);
|
|
||||||
|
|
||||||
var ftsRows = await _store.SearchFtsAsync(
|
var ftsRows = await _store.SearchFtsAsync(
|
||||||
query,
|
query,
|
||||||
storeFilter,
|
storeFilter,
|
||||||
@@ -364,7 +394,6 @@ internal sealed class UnifiedSearchService : IUnifiedSearchService
|
|||||||
_timeProvider.GetUtcNow());
|
_timeProvider.GetUtcNow());
|
||||||
}
|
}
|
||||||
|
|
||||||
var currentScopeDomain = ResolveAmbientScopeDomain(request.Ambient);
|
|
||||||
var (primaryCards, overflow) = PartitionCardsByScope(cards, currentScopeDomain);
|
var (primaryCards, overflow) = PartitionCardsByScope(cards, currentScopeDomain);
|
||||||
var visibleCards = BuildVisibleAnswerCards(primaryCards, overflow);
|
var visibleCards = BuildVisibleAnswerCards(primaryCards, overflow);
|
||||||
var coverage = BuildCoverage(currentScopeDomain, merged, primaryCards, overflow);
|
var coverage = BuildCoverage(currentScopeDomain, merged, primaryCards, overflow);
|
||||||
@@ -402,7 +431,8 @@ internal sealed class UnifiedSearchService : IUnifiedSearchService
|
|||||||
synthesis,
|
synthesis,
|
||||||
suggestions,
|
suggestions,
|
||||||
refinements,
|
refinements,
|
||||||
coverage);
|
coverage,
|
||||||
|
corpusAvailability);
|
||||||
var totalVisibleCardCount = primaryCards.Count + (overflow?.Cards.Count ?? 0);
|
var totalVisibleCardCount = primaryCards.Count + (overflow?.Cards.Count ?? 0);
|
||||||
var response = new UnifiedSearchResponse(
|
var response = new UnifiedSearchResponse(
|
||||||
query,
|
query,
|
||||||
@@ -561,7 +591,10 @@ internal sealed class UnifiedSearchService : IUnifiedSearchService
|
|||||||
.ToArray();
|
.ToArray();
|
||||||
}
|
}
|
||||||
|
|
||||||
private static IReadOnlyList<EntityCard> SelectDominantAnswerCards(IReadOnlyList<EntityCard> visibleCards)
|
private static IReadOnlyList<EntityCard> SelectDominantAnswerCards(
|
||||||
|
IReadOnlyList<EntityCard> visibleCards,
|
||||||
|
QueryPlan plan,
|
||||||
|
AmbientContext? ambient)
|
||||||
{
|
{
|
||||||
if (visibleCards.Count <= 1)
|
if (visibleCards.Count <= 1)
|
||||||
{
|
{
|
||||||
@@ -569,14 +602,44 @@ internal sealed class UnifiedSearchService : IUnifiedSearchService
|
|||||||
}
|
}
|
||||||
|
|
||||||
var topScore = Math.Max(Math.Abs(visibleCards[0].Score), 0.000001d);
|
var topScore = Math.Max(Math.Abs(visibleCards[0].Score), 0.000001d);
|
||||||
|
var scoreBandRatio = ResolveBlendedAnswerScoreBandRatio(plan, ambient, visibleCards);
|
||||||
var blended = visibleCards
|
var blended = visibleCards
|
||||||
.Where(card => (topScore - card.Score) / topScore <= BlendedAnswerScoreBandRatio)
|
.Where(card => (topScore - card.Score) / topScore <= scoreBandRatio)
|
||||||
.Take(MaxContextAnswerCitations)
|
.Take(MaxContextAnswerCitations)
|
||||||
.ToArray();
|
.ToArray();
|
||||||
|
|
||||||
return blended.Length >= 2
|
if (blended.Length >= 2)
|
||||||
? blended
|
{
|
||||||
: visibleCards.Take(1).ToArray();
|
return blended;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (string.Equals(plan.Intent, "compare", StringComparison.OrdinalIgnoreCase))
|
||||||
|
{
|
||||||
|
return visibleCards
|
||||||
|
.Take(Math.Min(2, MaxContextAnswerCitations))
|
||||||
|
.ToArray();
|
||||||
|
}
|
||||||
|
|
||||||
|
return visibleCards.Take(1).ToArray();
|
||||||
|
}
|
||||||
|
|
||||||
|
private static double ResolveBlendedAnswerScoreBandRatio(
|
||||||
|
QueryPlan plan,
|
||||||
|
AmbientContext? ambient,
|
||||||
|
IReadOnlyList<EntityCard> visibleCards)
|
||||||
|
{
|
||||||
|
if (string.Equals(plan.Intent, "compare", StringComparison.OrdinalIgnoreCase))
|
||||||
|
{
|
||||||
|
return CompareBlendedAnswerScoreBandRatio;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (string.Equals(plan.Intent, "troubleshoot", StringComparison.OrdinalIgnoreCase)
|
||||||
|
&& !string.IsNullOrWhiteSpace(ResolveContextDomain(plan, visibleCards, ambient)))
|
||||||
|
{
|
||||||
|
return ScopedTroubleshootBlendedAnswerScoreBandRatio;
|
||||||
|
}
|
||||||
|
|
||||||
|
return BlendedAnswerScoreBandRatio;
|
||||||
}
|
}
|
||||||
|
|
||||||
private static UnifiedSearchCoverage BuildCoverage(
|
private static UnifiedSearchCoverage BuildCoverage(
|
||||||
@@ -974,7 +1037,8 @@ internal sealed class UnifiedSearchService : IUnifiedSearchService
|
|||||||
|
|
||||||
private static string BuildInsufficientEvidence(
|
private static string BuildInsufficientEvidence(
|
||||||
IReadOnlyList<SearchSuggestion>? suggestions,
|
IReadOnlyList<SearchSuggestion>? suggestions,
|
||||||
IReadOnlyList<SearchRefinement>? refinements)
|
IReadOnlyList<SearchRefinement>? refinements,
|
||||||
|
CorpusAvailabilitySnapshot corpusAvailability)
|
||||||
{
|
{
|
||||||
var suggestionCount = suggestions?.Count ?? 0;
|
var suggestionCount = suggestions?.Count ?? 0;
|
||||||
var refinementCount = refinements?.Count ?? 0;
|
var refinementCount = refinements?.Count ?? 0;
|
||||||
@@ -983,9 +1047,70 @@ internal sealed class UnifiedSearchService : IUnifiedSearchService
|
|||||||
return $"No grounded citations were found, but {suggestionCount + refinementCount} recovery hint(s) are available.";
|
return $"No grounded citations were found, but {suggestionCount + refinementCount} recovery hint(s) are available.";
|
||||||
}
|
}
|
||||||
|
|
||||||
|
if (corpusAvailability.Known && corpusAvailability.TotalChunkCount > 0)
|
||||||
|
{
|
||||||
|
return "The ingested corpus is available, but this query did not retrieve grounded citations or result cards.";
|
||||||
|
}
|
||||||
|
|
||||||
return "No grounded citations, result cards, or bounded recovery hints were retrieved.";
|
return "No grounded citations, result cards, or bounded recovery hints were retrieved.";
|
||||||
}
|
}
|
||||||
|
|
||||||
|
private static string BuildInsufficientReason(QueryPlan plan, AmbientContext? ambient)
|
||||||
|
{
|
||||||
|
var scope = ResolveContextDomain(plan, [], ambient) ?? "current scope";
|
||||||
|
return $"No grounded evidence matched the requested terms inside {DescribeDomain(scope)} or nearby weighted domains.";
|
||||||
|
}
|
||||||
|
|
||||||
|
private static string BuildCorpusUnreadySummary(
|
||||||
|
string query,
|
||||||
|
QueryPlan? plan,
|
||||||
|
AmbientContext? ambient,
|
||||||
|
CorpusAvailabilitySnapshot corpusAvailability)
|
||||||
|
{
|
||||||
|
var scope = corpusAvailability.CurrentScopeDomain
|
||||||
|
?? ResolveContextDomain(plan, [], ambient)
|
||||||
|
?? "current scope";
|
||||||
|
|
||||||
|
if (corpusAvailability.Known && corpusAvailability.TotalChunkCount == 0)
|
||||||
|
{
|
||||||
|
return $"\"{query}\" could not be answered because the search corpus is not populated yet.";
|
||||||
|
}
|
||||||
|
|
||||||
|
return $"\"{query}\" could not be answered because {DescribeDomain(scope)} is not populated in search yet.";
|
||||||
|
}
|
||||||
|
|
||||||
|
private static string BuildCorpusUnreadyReason(
|
||||||
|
QueryPlan? plan,
|
||||||
|
AmbientContext? ambient,
|
||||||
|
CorpusAvailabilitySnapshot corpusAvailability)
|
||||||
|
{
|
||||||
|
var scope = corpusAvailability.CurrentScopeDomain
|
||||||
|
?? ResolveContextDomain(plan, [], ambient)
|
||||||
|
?? "current scope";
|
||||||
|
|
||||||
|
if (corpusAvailability.Known && corpusAvailability.TotalChunkCount == 0)
|
||||||
|
{
|
||||||
|
return "No ingested search corpus is available yet, so the query cannot produce grounded results.";
|
||||||
|
}
|
||||||
|
|
||||||
|
return $"The active route maps to {DescribeDomain(scope)}, but that scope has no ingested search corpus yet.";
|
||||||
|
}
|
||||||
|
|
||||||
|
private static string BuildCorpusUnreadyEvidence(CorpusAvailabilitySnapshot corpusAvailability)
|
||||||
|
{
|
||||||
|
if (corpusAvailability.Known && corpusAvailability.TotalChunkCount == 0)
|
||||||
|
{
|
||||||
|
return "No indexed chunks are currently available for the active tenant and filter set.";
|
||||||
|
}
|
||||||
|
|
||||||
|
if (!string.IsNullOrWhiteSpace(corpusAvailability.CurrentScopeDomain))
|
||||||
|
{
|
||||||
|
return $"{DescribeDomain(corpusAvailability.CurrentScopeDomain)} currently has zero indexed chunks for the active tenant and filter set.";
|
||||||
|
}
|
||||||
|
|
||||||
|
return "The active scope is not populated in the indexed corpus.";
|
||||||
|
}
|
||||||
|
|
||||||
private static IReadOnlyList<string> GetGroundedQuestionTemplates(string domain, string title)
|
private static IReadOnlyList<string> GetGroundedQuestionTemplates(string domain, string title)
|
||||||
{
|
{
|
||||||
return domain switch
|
return domain switch
|
||||||
@@ -1186,6 +1311,117 @@ internal sealed class UnifiedSearchService : IUnifiedSearchService
|
|||||||
return null;
|
return null;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
private async Task<CorpusAvailabilitySnapshot> LoadCorpusAvailabilityAsync(
|
||||||
|
KnowledgeSearchFilter? storeFilter,
|
||||||
|
AmbientContext? ambient,
|
||||||
|
CancellationToken cancellationToken)
|
||||||
|
{
|
||||||
|
var currentScopeDomain = ResolveAmbientScopeDomain(ambient);
|
||||||
|
if (_store is not IKnowledgeSearchCorpusAvailabilityStore availabilityStore)
|
||||||
|
{
|
||||||
|
return CorpusAvailabilitySnapshot.Unknown(currentScopeDomain);
|
||||||
|
}
|
||||||
|
|
||||||
|
var domains = await availabilityStore.GetDomainCorpusAvailabilityAsync(
|
||||||
|
storeFilter,
|
||||||
|
cancellationToken).ConfigureAwait(false);
|
||||||
|
|
||||||
|
var domainCounts = domains
|
||||||
|
.Where(static entry => !string.IsNullOrWhiteSpace(entry.Domain))
|
||||||
|
.GroupBy(static entry => entry.Domain.Trim(), StringComparer.OrdinalIgnoreCase)
|
||||||
|
.ToDictionary(
|
||||||
|
static group => group.Key,
|
||||||
|
static group => group.Max(static entry => Math.Max(0, entry.ChunkCount)),
|
||||||
|
StringComparer.OrdinalIgnoreCase);
|
||||||
|
|
||||||
|
var currentScopeChunkCount = !string.IsNullOrWhiteSpace(currentScopeDomain)
|
||||||
|
&& domainCounts.TryGetValue(currentScopeDomain, out var scopedCount)
|
||||||
|
? scopedCount
|
||||||
|
: 0;
|
||||||
|
|
||||||
|
return new CorpusAvailabilitySnapshot(
|
||||||
|
Known: true,
|
||||||
|
CurrentScopeDomain: currentScopeDomain,
|
||||||
|
CurrentScopeChunkCount: currentScopeChunkCount,
|
||||||
|
TotalChunkCount: domainCounts.Values.Sum(),
|
||||||
|
DomainChunkCounts: domainCounts);
|
||||||
|
}
|
||||||
|
|
||||||
|
private static string DetermineSuggestionViabilityState(
|
||||||
|
int cardCount,
|
||||||
|
ContextAnswer? answer,
|
||||||
|
CorpusAvailabilitySnapshot corpusAvailability)
|
||||||
|
{
|
||||||
|
if (cardCount > 0)
|
||||||
|
{
|
||||||
|
return "grounded";
|
||||||
|
}
|
||||||
|
|
||||||
|
if (string.Equals(answer?.Status, "clarify", StringComparison.OrdinalIgnoreCase))
|
||||||
|
{
|
||||||
|
return "needs_clarification";
|
||||||
|
}
|
||||||
|
|
||||||
|
var unreadyCode = ResolveCorpusUnreadyCode(corpusAvailability);
|
||||||
|
if (string.Equals(unreadyCode, "search_corpus_unready", StringComparison.Ordinal))
|
||||||
|
{
|
||||||
|
return "corpus_unready";
|
||||||
|
}
|
||||||
|
|
||||||
|
if (string.Equals(unreadyCode, "current_scope_corpus_unready", StringComparison.Ordinal))
|
||||||
|
{
|
||||||
|
return "scope_unready";
|
||||||
|
}
|
||||||
|
|
||||||
|
return "no_match";
|
||||||
|
}
|
||||||
|
|
||||||
|
private static bool IsSuggestionViable(string viabilityState)
|
||||||
|
{
|
||||||
|
return string.Equals(viabilityState, "grounded", StringComparison.OrdinalIgnoreCase);
|
||||||
|
}
|
||||||
|
|
||||||
|
private static bool IsCurrentScopeReady(CorpusAvailabilitySnapshot corpusAvailability)
|
||||||
|
{
|
||||||
|
return string.IsNullOrWhiteSpace(ResolveCorpusUnreadyCode(corpusAvailability));
|
||||||
|
}
|
||||||
|
|
||||||
|
private static string BuildSuggestionViabilityReason(
|
||||||
|
ContextAnswer? answer,
|
||||||
|
string viabilityState,
|
||||||
|
CorpusAvailabilitySnapshot corpusAvailability)
|
||||||
|
{
|
||||||
|
return viabilityState switch
|
||||||
|
{
|
||||||
|
"grounded" => answer?.Reason ?? "Grounded evidence is available for this suggestion.",
|
||||||
|
"needs_clarification" => answer?.Reason ?? "The query is too broad to surface as a ready-made suggestion.",
|
||||||
|
"scope_unready" => BuildCorpusUnreadyReason(plan: null, ambient: null, corpusAvailability),
|
||||||
|
"corpus_unready" => BuildCorpusUnreadyReason(plan: null, ambient: null, corpusAvailability),
|
||||||
|
_ => answer?.Reason ?? "No grounded evidence matched this suggestion in the current corpus."
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
private static string? ResolveCorpusUnreadyCode(CorpusAvailabilitySnapshot corpusAvailability)
|
||||||
|
{
|
||||||
|
if (!corpusAvailability.Known)
|
||||||
|
{
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (corpusAvailability.TotalChunkCount <= 0)
|
||||||
|
{
|
||||||
|
return "search_corpus_unready";
|
||||||
|
}
|
||||||
|
|
||||||
|
if (!string.IsNullOrWhiteSpace(corpusAvailability.CurrentScopeDomain)
|
||||||
|
&& corpusAvailability.CurrentScopeChunkCount <= 0)
|
||||||
|
{
|
||||||
|
return "current_scope_corpus_unready";
|
||||||
|
}
|
||||||
|
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
|
||||||
private static string DescribeDomain(string domain)
|
private static string DescribeDomain(string domain)
|
||||||
{
|
{
|
||||||
return domain switch
|
return domain switch
|
||||||
@@ -1770,7 +2006,8 @@ internal sealed class UnifiedSearchService : IUnifiedSearchService
|
|||||||
SynthesisResult? synthesis,
|
SynthesisResult? synthesis,
|
||||||
IReadOnlyList<SearchSuggestion>? suggestions,
|
IReadOnlyList<SearchSuggestion>? suggestions,
|
||||||
IReadOnlyList<SearchRefinement>? refinements,
|
IReadOnlyList<SearchRefinement>? refinements,
|
||||||
UnifiedSearchCoverage? coverage)
|
UnifiedSearchCoverage? coverage,
|
||||||
|
CorpusAvailabilitySnapshot corpusAvailability)
|
||||||
{
|
{
|
||||||
if (string.IsNullOrWhiteSpace(query))
|
if (string.IsNullOrWhiteSpace(query))
|
||||||
{
|
{
|
||||||
@@ -1780,7 +2017,7 @@ internal sealed class UnifiedSearchService : IUnifiedSearchService
|
|||||||
var visibleCards = BuildVisibleAnswerCards(cards, overflow);
|
var visibleCards = BuildVisibleAnswerCards(cards, overflow);
|
||||||
if (visibleCards.Count > 0)
|
if (visibleCards.Count > 0)
|
||||||
{
|
{
|
||||||
var answerCards = SelectDominantAnswerCards(visibleCards);
|
var answerCards = SelectDominantAnswerCards(visibleCards, plan, ambient);
|
||||||
var topCard = answerCards[0];
|
var topCard = answerCards[0];
|
||||||
var citations = BuildContextAnswerCitations(answerCards, synthesis);
|
var citations = BuildContextAnswerCitations(answerCards, synthesis);
|
||||||
var questions = BuildGroundedQuestions(query, plan, ambient, topCard);
|
var questions = BuildGroundedQuestions(query, plan, ambient, topCard);
|
||||||
@@ -1798,6 +2035,20 @@ internal sealed class UnifiedSearchService : IUnifiedSearchService
|
|||||||
Questions: questions);
|
Questions: questions);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
var corpusUnreadyCode = ResolveCorpusUnreadyCode(corpusAvailability);
|
||||||
|
if (!string.IsNullOrWhiteSpace(corpusUnreadyCode))
|
||||||
|
{
|
||||||
|
var unreadyQuestions = BuildRecoveryQuestions(query, plan, ambient, suggestions, refinements);
|
||||||
|
return new ContextAnswer(
|
||||||
|
Status: "insufficient",
|
||||||
|
Code: corpusUnreadyCode,
|
||||||
|
Summary: BuildCorpusUnreadySummary(query, plan, ambient, corpusAvailability),
|
||||||
|
Reason: BuildCorpusUnreadyReason(plan, ambient, corpusAvailability),
|
||||||
|
Evidence: BuildCorpusUnreadyEvidence(corpusAvailability),
|
||||||
|
Citations: [],
|
||||||
|
Questions: unreadyQuestions);
|
||||||
|
}
|
||||||
|
|
||||||
if (ShouldClarifyQuery(query, plan, ambient, suggestions, refinements))
|
if (ShouldClarifyQuery(query, plan, ambient, suggestions, refinements))
|
||||||
{
|
{
|
||||||
var clarificationScope = ResolveContextDomain(plan, cards, ambient) ?? "current scope";
|
var clarificationScope = ResolveContextDomain(plan, cards, ambient) ?? "current scope";
|
||||||
@@ -1818,8 +2069,8 @@ internal sealed class UnifiedSearchService : IUnifiedSearchService
|
|||||||
Status: "insufficient",
|
Status: "insufficient",
|
||||||
Code: "no_grounded_evidence",
|
Code: "no_grounded_evidence",
|
||||||
Summary: BuildInsufficientSummary(query, plan, ambient),
|
Summary: BuildInsufficientSummary(query, plan, ambient),
|
||||||
Reason: "No grounded evidence matched the requested terms in the current ingested corpus.",
|
Reason: BuildInsufficientReason(plan, ambient),
|
||||||
Evidence: BuildInsufficientEvidence(suggestions, refinements),
|
Evidence: BuildInsufficientEvidence(suggestions, refinements, corpusAvailability),
|
||||||
Citations: [],
|
Citations: [],
|
||||||
Questions: recoveryQuestions);
|
Questions: recoveryQuestions);
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -102,10 +102,36 @@ public sealed class UnifiedSearchEndpointsIntegrationTests : IDisposable
|
|||||||
payload!.Suggestions.Should().ContainSingle();
|
payload!.Suggestions.Should().ContainSingle();
|
||||||
payload.Suggestions[0].Viable.Should().BeTrue();
|
payload.Suggestions[0].Viable.Should().BeTrue();
|
||||||
payload.Suggestions[0].Status.Should().Be("grounded");
|
payload.Suggestions[0].Status.Should().Be("grounded");
|
||||||
|
payload.Suggestions[0].ViabilityState.Should().Be("grounded");
|
||||||
|
payload.Suggestions[0].ScopeReady.Should().BeTrue();
|
||||||
payload.Coverage.Should().NotBeNull();
|
payload.Coverage.Should().NotBeNull();
|
||||||
payload.Coverage!.Domains.Should().Contain(domain => domain.Domain == "knowledge");
|
payload.Coverage!.Domains.Should().Contain(domain => domain.Domain == "knowledge");
|
||||||
}
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public async Task EvaluateSuggestions_WithUnreadyScope_ReturnsScopeUnreadyState()
|
||||||
|
{
|
||||||
|
using var client = _factory.CreateClient();
|
||||||
|
client.DefaultRequestHeaders.Add("X-StellaOps-Scopes", "advisory-ai:operate");
|
||||||
|
client.DefaultRequestHeaders.Add("X-StellaOps-Tenant", "test-tenant");
|
||||||
|
|
||||||
|
var response = await client.PostAsJsonAsync("/v1/search/suggestions/evaluate", new UnifiedSearchSuggestionViabilityApiRequest
|
||||||
|
{
|
||||||
|
Queries = ["empty knowledge"]
|
||||||
|
});
|
||||||
|
|
||||||
|
response.StatusCode.Should().Be(HttpStatusCode.OK);
|
||||||
|
|
||||||
|
var payload = await response.Content.ReadFromJsonAsync<UnifiedSearchSuggestionViabilityApiResponse>();
|
||||||
|
payload.Should().NotBeNull();
|
||||||
|
payload!.Suggestions.Should().ContainSingle();
|
||||||
|
payload.Suggestions[0].Viable.Should().BeFalse();
|
||||||
|
payload.Suggestions[0].Status.Should().Be("insufficient");
|
||||||
|
payload.Suggestions[0].Code.Should().Be("current_scope_corpus_unready");
|
||||||
|
payload.Suggestions[0].ViabilityState.Should().Be("scope_unready");
|
||||||
|
payload.Suggestions[0].ScopeReady.Should().BeFalse();
|
||||||
|
}
|
||||||
|
|
||||||
[Fact]
|
[Fact]
|
||||||
public async Task Query_WithBroadQuery_ReturnsClarifyContextAnswer()
|
public async Task Query_WithBroadQuery_ReturnsClarifyContextAnswer()
|
||||||
{
|
{
|
||||||
@@ -405,6 +431,37 @@ public sealed class UnifiedSearchEndpointsIntegrationTests : IDisposable
|
|||||||
SearchSuggestionViabilityRequest request,
|
SearchSuggestionViabilityRequest request,
|
||||||
CancellationToken cancellationToken)
|
CancellationToken cancellationToken)
|
||||||
{
|
{
|
||||||
|
if (string.Equals(request.Queries.FirstOrDefault(), "empty knowledge", StringComparison.OrdinalIgnoreCase))
|
||||||
|
{
|
||||||
|
return Task.FromResult(new SearchSuggestionViabilityResponse(
|
||||||
|
Suggestions:
|
||||||
|
[
|
||||||
|
new SearchSuggestionViabilityResult(
|
||||||
|
Query: "empty knowledge",
|
||||||
|
Viable: false,
|
||||||
|
Status: "insufficient",
|
||||||
|
Code: "current_scope_corpus_unready",
|
||||||
|
CardCount: 0,
|
||||||
|
LeadingDomain: "knowledge",
|
||||||
|
Reason: "The active route maps to the knowledge scope, but that scope has no ingested search corpus yet.",
|
||||||
|
ViabilityState: "scope_unready",
|
||||||
|
ScopeReady: false)
|
||||||
|
],
|
||||||
|
Coverage: new UnifiedSearchCoverage(
|
||||||
|
CurrentScopeDomain: "knowledge",
|
||||||
|
CurrentScopeWeighted: true,
|
||||||
|
Domains:
|
||||||
|
[
|
||||||
|
new UnifiedSearchDomainCoverage(
|
||||||
|
Domain: "knowledge",
|
||||||
|
CandidateCount: 0,
|
||||||
|
VisibleCardCount: 0,
|
||||||
|
TopScore: 0d,
|
||||||
|
IsCurrentScope: true,
|
||||||
|
HasVisibleResults: false)
|
||||||
|
])));
|
||||||
|
}
|
||||||
|
|
||||||
return Task.FromResult(new SearchSuggestionViabilityResponse(
|
return Task.FromResult(new SearchSuggestionViabilityResponse(
|
||||||
Suggestions:
|
Suggestions:
|
||||||
[
|
[
|
||||||
@@ -415,7 +472,9 @@ public sealed class UnifiedSearchEndpointsIntegrationTests : IDisposable
|
|||||||
Code: "retrieved_evidence",
|
Code: "retrieved_evidence",
|
||||||
CardCount: 1,
|
CardCount: 1,
|
||||||
LeadingDomain: "knowledge",
|
LeadingDomain: "knowledge",
|
||||||
Reason: "Grounded knowledge evidence is available.")
|
Reason: "Grounded knowledge evidence is available.",
|
||||||
|
ViabilityState: "grounded",
|
||||||
|
ScopeReady: true)
|
||||||
],
|
],
|
||||||
Coverage: new UnifiedSearchCoverage(
|
Coverage: new UnifiedSearchCoverage(
|
||||||
CurrentScopeDomain: "knowledge",
|
CurrentScopeDomain: "knowledge",
|
||||||
|
|||||||
@@ -272,6 +272,93 @@ public sealed class UnifiedSearchServiceTests
|
|||||||
result.ContextAnswer.Citations![0].Title.Should().Be("PostgreSQL connectivity");
|
result.ContextAnswer.Citations![0].Title.Should().Be("PostgreSQL connectivity");
|
||||||
}
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public async Task SearchAsync_blends_compare_queries_when_top_evidence_is_close()
|
||||||
|
{
|
||||||
|
var doctorEmbedding = new float[64];
|
||||||
|
doctorEmbedding[0] = 0.6f;
|
||||||
|
doctorEmbedding[1] = 0.3f;
|
||||||
|
var doctorRow = MakeRow(
|
||||||
|
"chunk-doctor-compare-answer",
|
||||||
|
"doctor_check",
|
||||||
|
"PostgreSQL connectivity",
|
||||||
|
JsonDocument.Parse("{\"domain\":\"knowledge\",\"checkCode\":\"check.core.db.connectivity\"}"),
|
||||||
|
embedding: doctorEmbedding,
|
||||||
|
snippet: "Database connectivity is degraded.");
|
||||||
|
var guideRow = MakeRow(
|
||||||
|
"chunk-guide-compare-answer",
|
||||||
|
"md_section",
|
||||||
|
"Database failover playbook",
|
||||||
|
JsonDocument.Parse("{\"domain\":\"knowledge\",\"path\":\"docs/database-failover.md\"}"),
|
||||||
|
snippet: "Failover guidance covers the same connectivity symptoms.");
|
||||||
|
|
||||||
|
var storeMock = new Mock<IKnowledgeSearchStore>();
|
||||||
|
storeMock.Setup(s => s.SearchFtsAsync(
|
||||||
|
It.IsAny<string>(), It.IsAny<KnowledgeSearchFilter?>(), It.IsAny<int>(),
|
||||||
|
It.IsAny<TimeSpan>(), It.IsAny<CancellationToken>(), It.IsAny<string?>()))
|
||||||
|
.ReturnsAsync(new List<KnowledgeChunkRow> { doctorRow, guideRow });
|
||||||
|
storeMock.Setup(s => s.LoadVectorCandidatesAsync(
|
||||||
|
It.IsAny<float[]>(), It.IsAny<KnowledgeSearchFilter?>(), It.IsAny<int>(),
|
||||||
|
It.IsAny<TimeSpan>(), It.IsAny<CancellationToken>()))
|
||||||
|
.ReturnsAsync(new List<KnowledgeChunkRow> { doctorRow });
|
||||||
|
|
||||||
|
var service = CreateService(storeMock: storeMock);
|
||||||
|
|
||||||
|
var result = await service.SearchAsync(
|
||||||
|
new UnifiedSearchRequest("compare database connectivity and failover guidance"),
|
||||||
|
CancellationToken.None);
|
||||||
|
|
||||||
|
result.ContextAnswer.Should().NotBeNull();
|
||||||
|
result.ContextAnswer!.Code.Should().Be("retrieved_blended_evidence");
|
||||||
|
result.ContextAnswer.Citations.Should().HaveCount(2);
|
||||||
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public async Task SearchAsync_keeps_single_answer_for_scoped_troubleshoot_queries()
|
||||||
|
{
|
||||||
|
var doctorEmbedding = new float[64];
|
||||||
|
doctorEmbedding[0] = 0.6f;
|
||||||
|
doctorEmbedding[1] = 0.3f;
|
||||||
|
var doctorRow = MakeRow(
|
||||||
|
"chunk-doctor-troubleshoot-answer",
|
||||||
|
"doctor_check",
|
||||||
|
"PostgreSQL connectivity",
|
||||||
|
JsonDocument.Parse("{\"domain\":\"knowledge\",\"checkCode\":\"check.core.db.connectivity\"}"),
|
||||||
|
embedding: doctorEmbedding,
|
||||||
|
snippet: "Database connectivity is degraded.");
|
||||||
|
var guideRow = MakeRow(
|
||||||
|
"chunk-guide-troubleshoot-answer",
|
||||||
|
"md_section",
|
||||||
|
"Database failover playbook",
|
||||||
|
JsonDocument.Parse("{\"domain\":\"knowledge\",\"path\":\"docs/database-failover.md\"}"),
|
||||||
|
snippet: "Failover guidance covers the same connectivity symptoms.");
|
||||||
|
|
||||||
|
var storeMock = new Mock<IKnowledgeSearchStore>();
|
||||||
|
storeMock.Setup(s => s.SearchFtsAsync(
|
||||||
|
It.IsAny<string>(), It.IsAny<KnowledgeSearchFilter?>(), It.IsAny<int>(),
|
||||||
|
It.IsAny<TimeSpan>(), It.IsAny<CancellationToken>(), It.IsAny<string?>()))
|
||||||
|
.ReturnsAsync(new List<KnowledgeChunkRow> { doctorRow, guideRow });
|
||||||
|
storeMock.Setup(s => s.LoadVectorCandidatesAsync(
|
||||||
|
It.IsAny<float[]>(), It.IsAny<KnowledgeSearchFilter?>(), It.IsAny<int>(),
|
||||||
|
It.IsAny<TimeSpan>(), It.IsAny<CancellationToken>()))
|
||||||
|
.ReturnsAsync(new List<KnowledgeChunkRow> { doctorRow });
|
||||||
|
|
||||||
|
var service = CreateService(storeMock: storeMock);
|
||||||
|
|
||||||
|
var result = await service.SearchAsync(
|
||||||
|
new UnifiedSearchRequest(
|
||||||
|
"diagnose database connectivity",
|
||||||
|
Ambient: new AmbientContext
|
||||||
|
{
|
||||||
|
CurrentRoute = "/ops/operations/doctor"
|
||||||
|
}),
|
||||||
|
CancellationToken.None);
|
||||||
|
|
||||||
|
result.ContextAnswer.Should().NotBeNull();
|
||||||
|
result.ContextAnswer!.Code.Should().Be("retrieved_evidence");
|
||||||
|
result.ContextAnswer.Citations.Should().ContainSingle();
|
||||||
|
}
|
||||||
|
|
||||||
[Fact]
|
[Fact]
|
||||||
public async Task SearchAsync_returns_clarify_context_answer_for_broad_query_without_matches()
|
public async Task SearchAsync_returns_clarify_context_answer_for_broad_query_without_matches()
|
||||||
{
|
{
|
||||||
@@ -313,6 +400,49 @@ public sealed class UnifiedSearchServiceTests
|
|||||||
result.ContextAnswer.Questions!.Should().OnlyContain(question => question.Kind == "recover");
|
result.ContextAnswer.Questions!.Should().OnlyContain(question => question.Kind == "recover");
|
||||||
}
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public async Task SearchAsync_returns_insufficient_when_current_scope_corpus_is_unready()
|
||||||
|
{
|
||||||
|
var service = CreateService(
|
||||||
|
store: new CorpusAvailabilityTestStore(
|
||||||
|
domainAvailability:
|
||||||
|
[
|
||||||
|
new KnowledgeSearchDomainCorpusAvailability("findings", 4)
|
||||||
|
]));
|
||||||
|
|
||||||
|
var result = await service.SearchAsync(
|
||||||
|
new UnifiedSearchRequest(
|
||||||
|
"status",
|
||||||
|
Ambient: new AmbientContext
|
||||||
|
{
|
||||||
|
CurrentRoute = "/ops/operations/doctor"
|
||||||
|
}),
|
||||||
|
CancellationToken.None);
|
||||||
|
|
||||||
|
result.Cards.Should().BeEmpty();
|
||||||
|
result.ContextAnswer.Should().NotBeNull();
|
||||||
|
result.ContextAnswer!.Status.Should().Be("insufficient");
|
||||||
|
result.ContextAnswer.Code.Should().Be("current_scope_corpus_unready");
|
||||||
|
result.ContextAnswer.Reason.Should().Contain("knowledge scope");
|
||||||
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public async Task SearchAsync_returns_insufficient_when_corpus_is_unready_for_tenant()
|
||||||
|
{
|
||||||
|
var service = CreateService(
|
||||||
|
store: new CorpusAvailabilityTestStore(
|
||||||
|
domainAvailability: []));
|
||||||
|
|
||||||
|
var result = await service.SearchAsync(
|
||||||
|
new UnifiedSearchRequest("status"),
|
||||||
|
CancellationToken.None);
|
||||||
|
|
||||||
|
result.Cards.Should().BeEmpty();
|
||||||
|
result.ContextAnswer.Should().NotBeNull();
|
||||||
|
result.ContextAnswer!.Status.Should().Be("insufficient");
|
||||||
|
result.ContextAnswer.Code.Should().Be("search_corpus_unready");
|
||||||
|
}
|
||||||
|
|
||||||
[Fact]
|
[Fact]
|
||||||
public async Task SearchAsync_records_answer_frame_analytics_for_clarify_state()
|
public async Task SearchAsync_records_answer_frame_analytics_for_clarify_state()
|
||||||
{
|
{
|
||||||
@@ -462,6 +592,59 @@ public sealed class UnifiedSearchServiceTests
|
|||||||
telemetrySink.Events.Should().BeEmpty();
|
telemetrySink.Events.Should().BeEmpty();
|
||||||
}
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public async Task EvaluateSuggestionsAsync_marks_broad_queries_as_not_viable_even_when_they_only_clarify()
|
||||||
|
{
|
||||||
|
var service = CreateService(
|
||||||
|
store: new CorpusAvailabilityTestStore(
|
||||||
|
domainAvailability:
|
||||||
|
[
|
||||||
|
new KnowledgeSearchDomainCorpusAvailability("knowledge", 8)
|
||||||
|
]));
|
||||||
|
|
||||||
|
var result = await service.EvaluateSuggestionsAsync(
|
||||||
|
new SearchSuggestionViabilityRequest(
|
||||||
|
Queries: ["status"],
|
||||||
|
Ambient: new AmbientContext
|
||||||
|
{
|
||||||
|
CurrentRoute = "/ops/operations/doctor"
|
||||||
|
}),
|
||||||
|
CancellationToken.None);
|
||||||
|
|
||||||
|
result.Suggestions.Should().ContainSingle();
|
||||||
|
result.Suggestions[0].Viable.Should().BeFalse();
|
||||||
|
result.Suggestions[0].Status.Should().Be("clarify");
|
||||||
|
result.Suggestions[0].ViabilityState.Should().Be("needs_clarification");
|
||||||
|
result.Suggestions[0].ScopeReady.Should().BeTrue();
|
||||||
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public async Task EvaluateSuggestionsAsync_reports_scope_unready_when_current_route_has_no_corpus()
|
||||||
|
{
|
||||||
|
var service = CreateService(
|
||||||
|
store: new CorpusAvailabilityTestStore(
|
||||||
|
domainAvailability:
|
||||||
|
[
|
||||||
|
new KnowledgeSearchDomainCorpusAvailability("findings", 3)
|
||||||
|
]));
|
||||||
|
|
||||||
|
var result = await service.EvaluateSuggestionsAsync(
|
||||||
|
new SearchSuggestionViabilityRequest(
|
||||||
|
Queries: ["database connectivity"],
|
||||||
|
Ambient: new AmbientContext
|
||||||
|
{
|
||||||
|
CurrentRoute = "/ops/operations/doctor"
|
||||||
|
}),
|
||||||
|
CancellationToken.None);
|
||||||
|
|
||||||
|
result.Suggestions.Should().ContainSingle();
|
||||||
|
result.Suggestions[0].Viable.Should().BeFalse();
|
||||||
|
result.Suggestions[0].Status.Should().Be("insufficient");
|
||||||
|
result.Suggestions[0].Code.Should().Be("current_scope_corpus_unready");
|
||||||
|
result.Suggestions[0].ViabilityState.Should().Be("scope_unready");
|
||||||
|
result.Suggestions[0].ScopeReady.Should().BeFalse();
|
||||||
|
}
|
||||||
|
|
||||||
[Fact]
|
[Fact]
|
||||||
public async Task EvaluateSuggestionsAsync_returns_viability_without_recording_answer_frame_analytics()
|
public async Task EvaluateSuggestionsAsync_returns_viability_without_recording_answer_frame_analytics()
|
||||||
{
|
{
|
||||||
@@ -519,11 +702,13 @@ public sealed class UnifiedSearchServiceTests
|
|||||||
result.Suggestions.Should().Contain(suggestion =>
|
result.Suggestions.Should().Contain(suggestion =>
|
||||||
suggestion.Query == "database connectivity"
|
suggestion.Query == "database connectivity"
|
||||||
&& suggestion.Viable
|
&& suggestion.Viable
|
||||||
&& suggestion.Status == "grounded");
|
&& suggestion.Status == "grounded"
|
||||||
|
&& suggestion.ViabilityState == "grounded");
|
||||||
result.Suggestions.Should().Contain(suggestion =>
|
result.Suggestions.Should().Contain(suggestion =>
|
||||||
suggestion.Query == "manually compare unavailable evidence across environments"
|
suggestion.Query == "manually compare unavailable evidence across environments"
|
||||||
&& !suggestion.Viable
|
&& !suggestion.Viable
|
||||||
&& suggestion.Status == "insufficient");
|
&& suggestion.Status == "insufficient"
|
||||||
|
&& suggestion.ViabilityState == "no_match");
|
||||||
result.Coverage.CurrentScopeDomain.Should().Be("knowledge");
|
result.Coverage.CurrentScopeDomain.Should().Be("knowledge");
|
||||||
|
|
||||||
var events = analyticsService.GetFallbackEventsSnapshot("global", TimeSpan.FromMinutes(1));
|
var events = analyticsService.GetFallbackEventsSnapshot("global", TimeSpan.FromMinutes(1));
|
||||||
@@ -1233,6 +1418,7 @@ public sealed class UnifiedSearchServiceTests
|
|||||||
private static UnifiedSearchService CreateService(
|
private static UnifiedSearchService CreateService(
|
||||||
bool enabled = true,
|
bool enabled = true,
|
||||||
Mock<IKnowledgeSearchStore>? storeMock = null,
|
Mock<IKnowledgeSearchStore>? storeMock = null,
|
||||||
|
IKnowledgeSearchStore? store = null,
|
||||||
UnifiedSearchOptions? unifiedOptions = null,
|
UnifiedSearchOptions? unifiedOptions = null,
|
||||||
SearchAnalyticsService? analyticsService = null,
|
SearchAnalyticsService? analyticsService = null,
|
||||||
SearchQualityMonitor? qualityMonitor = null,
|
SearchQualityMonitor? qualityMonitor = null,
|
||||||
@@ -1255,6 +1441,7 @@ public sealed class UnifiedSearchServiceTests
|
|||||||
var wrappedUnifiedOptions = Options.Create(unifiedOptions ?? new UnifiedSearchOptions());
|
var wrappedUnifiedOptions = Options.Create(unifiedOptions ?? new UnifiedSearchOptions());
|
||||||
|
|
||||||
storeMock ??= new Mock<IKnowledgeSearchStore>();
|
storeMock ??= new Mock<IKnowledgeSearchStore>();
|
||||||
|
var effectiveStore = store ?? storeMock.Object;
|
||||||
|
|
||||||
var vectorEncoder = new Mock<IVectorEncoder>();
|
var vectorEncoder = new Mock<IVectorEncoder>();
|
||||||
var mockEmbedding = new float[64];
|
var mockEmbedding = new float[64];
|
||||||
@@ -1277,7 +1464,7 @@ public sealed class UnifiedSearchServiceTests
|
|||||||
|
|
||||||
return new UnifiedSearchService(
|
return new UnifiedSearchService(
|
||||||
options,
|
options,
|
||||||
storeMock.Object,
|
effectiveStore,
|
||||||
vectorEncoder.Object,
|
vectorEncoder.Object,
|
||||||
planBuilder,
|
planBuilder,
|
||||||
synthesisEngine,
|
synthesisEngine,
|
||||||
@@ -1319,6 +1506,67 @@ public sealed class UnifiedSearchServiceTests
|
|||||||
return storeMock;
|
return storeMock;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
private sealed class CorpusAvailabilityTestStore : IKnowledgeSearchStore, IKnowledgeSearchCorpusAvailabilityStore
|
||||||
|
{
|
||||||
|
private readonly IReadOnlyDictionary<string, IReadOnlyList<KnowledgeChunkRow>> _ftsRows;
|
||||||
|
private readonly IReadOnlyList<KnowledgeSearchDomainCorpusAvailability> _domainAvailability;
|
||||||
|
|
||||||
|
public CorpusAvailabilityTestStore(
|
||||||
|
IReadOnlyDictionary<string, IReadOnlyList<KnowledgeChunkRow>>? ftsRows = null,
|
||||||
|
IReadOnlyList<KnowledgeSearchDomainCorpusAvailability>? domainAvailability = null)
|
||||||
|
{
|
||||||
|
_ftsRows = ftsRows ?? new Dictionary<string, IReadOnlyList<KnowledgeChunkRow>>(StringComparer.OrdinalIgnoreCase);
|
||||||
|
_domainAvailability = domainAvailability ?? [];
|
||||||
|
}
|
||||||
|
|
||||||
|
public Task EnsureSchemaAsync(CancellationToken cancellationToken) => Task.CompletedTask;
|
||||||
|
|
||||||
|
public Task ReplaceIndexAsync(KnowledgeIndexSnapshot snapshot, CancellationToken cancellationToken) => Task.CompletedTask;
|
||||||
|
|
||||||
|
public Task<IReadOnlyList<KnowledgeChunkRow>> SearchFtsAsync(
|
||||||
|
string query,
|
||||||
|
KnowledgeSearchFilter? filters,
|
||||||
|
int take,
|
||||||
|
TimeSpan timeout,
|
||||||
|
CancellationToken cancellationToken,
|
||||||
|
string? locale = null)
|
||||||
|
{
|
||||||
|
var normalized = query.Trim();
|
||||||
|
return Task.FromResult(
|
||||||
|
_ftsRows.TryGetValue(normalized, out var rows)
|
||||||
|
? (IReadOnlyList<KnowledgeChunkRow>)rows.Take(take).ToArray()
|
||||||
|
: []);
|
||||||
|
}
|
||||||
|
|
||||||
|
public Task<IReadOnlyList<KnowledgeChunkRow>> SearchFuzzyAsync(
|
||||||
|
string query,
|
||||||
|
KnowledgeSearchFilter? filters,
|
||||||
|
int take,
|
||||||
|
double similarityThreshold,
|
||||||
|
TimeSpan timeout,
|
||||||
|
CancellationToken cancellationToken)
|
||||||
|
{
|
||||||
|
return Task.FromResult<IReadOnlyList<KnowledgeChunkRow>>([]);
|
||||||
|
}
|
||||||
|
|
||||||
|
public Task<IReadOnlyList<KnowledgeChunkRow>> LoadVectorCandidatesAsync(
|
||||||
|
float[] queryEmbedding,
|
||||||
|
KnowledgeSearchFilter? filters,
|
||||||
|
int take,
|
||||||
|
TimeSpan timeout,
|
||||||
|
CancellationToken cancellationToken)
|
||||||
|
{
|
||||||
|
return Task.FromResult<IReadOnlyList<KnowledgeChunkRow>>([]);
|
||||||
|
}
|
||||||
|
|
||||||
|
public Task<IReadOnlyList<KnowledgeSearchDomainCorpusAvailability>> GetDomainCorpusAvailabilityAsync(
|
||||||
|
KnowledgeSearchFilter? filters,
|
||||||
|
CancellationToken cancellationToken)
|
||||||
|
{
|
||||||
|
return Task.FromResult(_domainAvailability);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
private static KnowledgeChunkRow MakeRow(
|
private static KnowledgeChunkRow MakeRow(
|
||||||
string chunkId,
|
string chunkId,
|
||||||
string kind,
|
string kind,
|
||||||
|
|||||||
Reference in New Issue
Block a user