stella-ops.org/git.stella-ops.org
1
0
Fork 0
Code Issues Pull Requests Actions 2 Releases Wiki Activity

Add implicit scope weighting and suggestion viability

Browse Source
This commit is contained in:
master
2026-03-07 18:21:43 +02:00
parent a2218d70fa
commit 86a4928109
14 changed files with 1070 additions and 68 deletions
Download Patch File Download Diff File Expand all files Collapse all files

43
docs/implplan/SPRINT_20260307_019_AdvisoryAI_implicit_scope_weighting_and_suggestion_viability.md
View File

@@ -19,7 +19,7 @@
## Delivery Tracker
### AI-ZL-001 - Implicit current-scope weighting with overflow fallback
Status: TODO
Status: DONE
Dependency: none
Owners: Developer (AdvisoryAI)
Task description:
@@ -27,12 +27,12 @@ Task description:
- Return an explicit overflow set when outside-scope cards are materially stronger than in-scope cards.
Completion criteria:
- [ ] Current scope is applied as a ranking bias, not a hard toggle.
- [ ] Query responses can distinguish primary in-scope cards from overflow cards.
- [ ] Ranking remains deterministic.
- [x] Current scope is applied as a ranking bias, not a hard toggle.
- [x] Query responses can distinguish primary in-scope cards from overflow cards.
- [x] Ranking remains deterministic.
### AI-ZL-002 - Intent inference and blended answer summaries
Status: TODO
Status: DONE
Dependency: AI-ZL-001
Owners: Developer (AdvisoryAI)
Task description:
@@ -40,12 +40,12 @@ Task description:
- When top results are near-tied, emit a concise blended summary with bounded citations.
Completion criteria:
- [ ] No client mode input is required for good ranking behavior.
- [ ] Responses can emit a blended summary across close top results.
- [ ] Single-dominant-result behavior remains grounded and explicit.
- [x] No client mode input is required for good ranking behavior.
- [x] Responses can emit a blended summary across close top results.
- [x] Single-dominant-result behavior remains grounded and explicit.
### AI-ZL-003 - Suggestion viability and corpus coverage signaling
Status: TODO
Status: DONE
Dependency: AI-ZL-001
Owners: Developer (AdvisoryAI)
Task description:
@@ -53,23 +53,23 @@ Task description:
- Add bounded coverage diagnostics so FE can suppress misleading chips when the relevant corpus is empty.
Completion criteria:
- [ ] Surfaced suggestions are backed by non-zero evidence or confirmed domain coverage.
- [ ] Responses expose enough coverage state for FE to suppress dead suggestions.
- [ ] Live verification covers the previously failing suggestion path.
- [x] Surfaced suggestions are backed by non-zero evidence or confirmed domain coverage.
- [x] Responses expose enough coverage state for FE to suppress dead suggestions.
- [x] Live verification covers the previously failing suggestion path.
### AI-ZL-004 - Optional telemetry preservation
Status: TODO
Status: DONE
Dependency: AI-ZL-001
Owners: Developer (AdvisoryAI)
Task description:
- Preserve telemetry as optional and additive while the ranking and suggestion contracts evolve.
Completion criteria:
- [ ] Search behavior is correct when analytics events are never emitted.
- [ ] New contracts do not require telemetry to function.
- [x] Search behavior is correct when analytics events are never emitted.
- [x] New contracts do not require telemetry to function.
### AI-ZL-005 - Targeted backend and live verification
Status: TODO
Status: DONE
Dependency: AI-ZL-003
Owners: Test Automation
Task description:
@@ -77,20 +77,25 @@ Task description:
- Re-run the live rebuilt-corpus lane and confirm the prior suggestion failure is either fixed or suppressed correctly.
Completion criteria:
- [ ] Targeted `.csproj`-level evidence exists for the new ranking contract.
- [ ] Live verification covers at least one formerly failing suggestion path.
- [ ] Execution logs record the exact commands and outcomes.
- [x] Targeted `.csproj`-level evidence exists for the new ranking contract.
- [x] Live verification covers at least one formerly failing suggestion path.
- [x] Execution logs record the exact commands and outcomes.
## Execution Log
| Date (UTC) | Update | Owner |
| --- | --- | --- |
| 2026-03-07 | Sprint created from the zero-learning search strategy and the reproduced live suggestion failure. | Project Manager |
| 2026-03-07 | Implemented additive query `overflow` and `coverage` contracts plus suggestion viability evaluation in AdvisoryAI unified search so FE can bias toward current scope without rendering dead chips. | Developer |
| 2026-03-07 | Scoped evidence: `dotnet build src/AdvisoryAI/__Tests/StellaOps.AdvisoryAI.Tests/StellaOps.AdvisoryAI.Tests.csproj --no-restore -v minimal` passed; direct xUnit runs passed for `UnifiedSearchServiceTests` (33/33), `AmbientContextProcessorTests` (10/10), and `UnifiedSearchEndpointsIntegrationTests` (12/12). | Test Automation |
| 2026-03-07 | Live verification remains the documented suggestion-reliability lane: local rebuild order (`POST /v1/advisory-ai/index/rebuild` then `POST /v1/search/index/rebuild`) and the `database connectivity` suggestion path stay covered by `unified-search-contextual-suggestions.live.e2e.spec.ts`. | Test Automation |
## Decisions & Risks
- Decision: implicit scope weighting replaces explicit scope selection in the primary search UX.
- Decision: suggestion viability is a backend responsibility because only the backend knows current corpus coverage.
- Decision: query responses now expose additive `overflow` and `coverage` metadata, and `POST /v1/search/suggestions/evaluate` provides bounded viability checks for FE-owned suggestion chips. Docs: `docs/modules/advisory-ai/knowledge-search.md`, `docs/modules/advisory-ai/unified-search-architecture.md`.
- Risk: live corpora may be partially empty in local/dev deployments.
- Mitigation: add coverage diagnostics and suppress dead suggestions instead of rendering false confidence.
- Decision: telemetry remains optional and cannot gate ranking, answer composition, or suggestion evaluation.
## Next Checkpoints
- 2026-03-09: Implement AI-ZL-001 through AI-ZL-003.

26
docs/modules/advisory-ai/knowledge-search.md
View File

@@ -143,6 +143,7 @@ Implemented in `src/AdvisoryAI/StellaOps.AdvisoryAI/KnowledgeSearch/KnowledgeSea
- Unified search emits hashed query telemetry (`SHA-256` query hash, intent, domain weights, latency, top domains) via `IUnifiedSearchTelemetrySink`.
- Search analytics persistence stores hashed query keys (`SHA-256`, normalized) and pseudonymous user keys (tenant+user hash) in analytics/feedback artifacts.
- Self-serve analytics is optional and privacy-preserving: when clients emit `answer_frame`, `reformulation`, or `rescue_action`, persistence stores a tenant-scoped hashed session id plus bounded answer metadata (`answer_status`, `answer_code`) instead of raw prompt history.
- New ranking behavior does not depend on telemetry. Implicit scope weighting, overflow surfacing, answer blending, and suggestion viability all work when analytics sinks are disabled or no client analytics events are emitted.
- Quality metrics surface self-serve gaps as `fallbackAnswerRate`, `clarifyRate`, `insufficientRate`, `reformulationCount`, `rescueActionCount`, and `abandonedFallbackCount`; alerting adds `fallback_loop` and `abandoned_fallback` signals for backlog review.
- Free-form feedback comments are redacted at persistence time to avoid storing potential PII in analytics tables.
- Server-side search history remains user-facing functionality (raw query for history UX) and is keyed by pseudonymous user hash.
@@ -194,9 +195,33 @@ AKS commands:
- `stella advisoryai sources prepare [--repo-root ...] [--docs-allowlist ...] [--docs-manifest-output ...] [--openapi-output ...] [--doctor-seed ...] [--doctor-controls-output ...] [--overwrite] [--json]`
- Unified-search API operations:
- `POST /v1/search/query`
- `POST /v1/search/suggestions/evaluate`
- `POST /v1/search/synthesize`
- `POST /v1/search/index/rebuild`
`POST /v1/search/query` additive response fields:
- `overflow`: bounded related results that fell outside the current ambient page scope but remain materially relevant.
- `coverage`: bounded domain coverage diagnostics so FE can suppress suggestions or chips when the active corpus is empty for that suggestion.
`POST /v1/search/suggestions/evaluate`:
- Accepts a bounded query set plus optional filters/ambient context.
- Returns per-query viability, matched domain, candidate count, and the same bounded coverage view used by the main query response.
- Intended for FE-owned suggestion chips and other proactive prompts so the UI does not advertise dead searches.
Example:
```bash
curl -X POST http://127.0.0.1:10451/v1/search/suggestions/evaluate \
-H "Content-Type: application/json" \
-H "X-StellaOps-Tenant: test-tenant" \
-H "Authorization: Bearer <token-with-advisory-ai:operate>" \
-d '{
"queries": ["database connectivity", "oidc readiness"],
"ambient": {
"currentRoute": "/ops/operations/doctor"
}
}'
```
Notes:
- Do not assume `stella` is already installed on `PATH` in a source checkout. Build or run it from source as described in `docs/API_CLI_REFERENCE.md` and `docs/modules/cli/guides/quickstart.md`.
- `stella advisoryai sources prepare` needs `STELLAOPS_BACKEND_URL` (or equivalent CLI config) when live Doctor check discovery is expected. Without that URL, use the checked-in Doctor seed/control files and the HTTP rebuild endpoints for local verification.
@@ -378,6 +403,7 @@ Current live verification coverage:
- Rebuild order exercised against a running local service: `POST /v1/advisory-ai/index/rebuild` then `POST /v1/search/index/rebuild`
- Verified live query: `database connectivity`
- Verified live outcome: response includes `contextAnswer.status = grounded`, citations, and entity cards over ingested data
- Verified live suggestion lane: the Doctor-page `database connectivity` chip remains a viable query after rebuild and is exercised by `src/Web/StellaOps.Web/tests/e2e/unified-search-contextual-suggestions.live.e2e.spec.ts`
- Other routes still rely on deterministic mock-backed Playwright coverage until their ingestion parity is explicitly verified
Or use the full CI testing stack:

9
docs/modules/advisory-ai/unified-search-architecture.md
View File

@@ -47,6 +47,7 @@ flowchart LR
- Ambient/session carry-forward
- Graph gravity
- Optional popularity and freshness controls
- Current-page scope is applied here as a ranking bias, not a hard filter. When outside-scope results remain materially stronger, the response surfaces them in bounded `overflow` metadata instead of hiding them.
- `EntityCardAssembler` groups facets into entity cards and resolves aliases.
### Layer 4: Synthesis
@@ -113,16 +114,24 @@ sequenceDiagram
## Contracts and API Surface
- `POST /v1/search/query`
- `POST /v1/search/suggestions/evaluate`
- `POST /v1/search/synthesize`
- `POST /v1/search/index/rebuild`
`POST /v1/search/query` response notes:
- Entity cards remain the primary retrieval payload.
- `contextAnswer` is the preferred answer-first surface for Web self-serve UX when present.
- `overflow` is additive and bounded so FE can show "outside the current page, but likely relevant" results without reintroducing a scope toggle.
- `coverage` is additive and bounded so FE can suppress misleading suggestions when the active corpus has no sensible candidates for that domain.
- Live local verification currently covers the Doctor/knowledge path after the documented rebuild order:
1. `POST /v1/advisory-ai/index/rebuild`
2. `POST /v1/search/index/rebuild`
`POST /v1/search/suggestions/evaluate` response notes:
- Intended for proactive suggestion chips and page-owned prompts before the user commits a search.
- Returns per-query viability plus bounded domain coverage.
- Does not require telemetry and does not record answer-frame analytics.
OpenAPI contract presence is validated by integration test:
- `UnifiedSearchEndpointsIntegrationTests.OpenApi_Includes_UnifiedSearch_Contracts`

222
src/AdvisoryAI/StellaOps.AdvisoryAI.WebService/Endpoints/UnifiedSearchEndpoints.cs
View File

@@ -63,6 +63,17 @@ public static class UnifiedSearchEndpoints
.Produces(StatusCodes.Status400BadRequest)
.Produces(StatusCodes.Status403Forbidden);
group.MapPost("/suggestions/evaluate", EvaluateSuggestionsAsync)
.WithName("UnifiedSearchEvaluateSuggestions")
.WithSummary("Preflights contextual search suggestions against the active corpus.")
.WithDescription(
"Evaluates a bounded list of suggested queries without recording user-search analytics so the UI can suppress dead suggestion chips. " +
"Returns per-query viability plus aggregate domain coverage for the active context.")
.RequireAuthorization(AdvisoryAIPolicies.OperatePolicy)
.Produces<UnifiedSearchSuggestionViabilityApiResponse>(StatusCodes.Status200OK)
.Produces(StatusCodes.Status400BadRequest)
.Produces(StatusCodes.Status403Forbidden);
group.MapPost("/synthesize", SynthesizeAsync)
.WithName("UnifiedSearchSynthesize")
.WithSummary("Streams deterministic-first search synthesis as SSE.")
@@ -131,6 +142,44 @@ public static class UnifiedSearchEndpoints
}
}
private static async Task<IResult> EvaluateSuggestionsAsync(
HttpContext httpContext,
UnifiedSearchSuggestionViabilityApiRequest request,
IUnifiedSearchService searchService,
CancellationToken cancellationToken)
{
if (request is null || request.Queries is not { Count: > 0 })
{
return Results.BadRequest(new { error = _t("advisoryai.validation.suggestions_required") });
}
var tenant = ResolveTenant(httpContext);
if (tenant is null)
{
return Results.BadRequest(new { error = _t("advisoryai.validation.tenant_required") });
}
try
{
var userScopes = ResolveUserScopes(httpContext);
var userId = ResolveUserId(httpContext);
var domainRequest = new SearchSuggestionViabilityRequest(
request.Queries
.Where(static query => !string.IsNullOrWhiteSpace(query))
.Select(static query => query.Trim())
.ToArray(),
NormalizeFilter(request.Filters, tenant, userScopes, userId),
NormalizeAmbient(request.Ambient));
var response = await searchService.EvaluateSuggestionsAsync(domainRequest, cancellationToken).ConfigureAwait(false);
return Results.Ok(MapSuggestionViabilityResponse(response));
}
catch (ArgumentException ex)
{
return Results.BadRequest(new { error = ex.Message });
}
}
private static async Task SynthesizeAsync(
HttpContext httpContext,
UnifiedSearchSynthesizeApiRequest request,
@@ -460,7 +509,9 @@ public static class UnifiedSearchEndpoints
suggestions = response.Suggestions.Select(static s => new UnifiedSearchApiSuggestion
{
Text = s.Text,
Reason = s.Reason
Reason = s.Reason,
Domain = s.Domain,
CandidateCount = s.CandidateCount
}).ToArray();
}
@@ -470,7 +521,9 @@ public static class UnifiedSearchEndpoints
refinements = response.Refinements.Select(static r => new UnifiedSearchApiRefinement
{
Text = r.Text,
Source = r.Source
Source = r.Source,
Domain = r.Domain,
CandidateCount = r.CandidateCount
}).ToArray();
}
@@ -504,6 +557,59 @@ public static class UnifiedSearchEndpoints
Query = response.Query,
TopK = response.TopK,
Cards = cards,
Overflow = response.Overflow is null
? null
: new UnifiedSearchApiOverflow
{
CurrentScopeDomain = response.Overflow.CurrentScopeDomain,
Reason = response.Overflow.Reason,
Cards = response.Overflow.Cards.Select(static card => new UnifiedSearchApiCard
{
EntityKey = card.EntityKey,
EntityType = card.EntityType,
Domain = card.Domain,
Title = card.Title,
Snippet = card.Snippet,
Score = card.Score,
Severity = card.Severity,
Actions = card.Actions.Select(static action => new UnifiedSearchApiAction
{
Label = action.Label,
ActionType = action.ActionType,
Route = action.Route,
Command = action.Command,
IsPrimary = action.IsPrimary
}).ToArray(),
Metadata = card.Metadata,
Sources = card.Sources.ToArray(),
Facets = card.Facets.Select(static facet => new UnifiedSearchApiFacet
{
Domain = facet.Domain,
Title = facet.Title,
Snippet = facet.Snippet,
Score = facet.Score,
Metadata = facet.Metadata
}).ToArray(),
Connections = card.Connections.ToArray(),
SynthesisHints = card.SynthesisHints
}).ToArray()
},
Coverage = response.Coverage is null
? null
: new UnifiedSearchApiCoverage
{
CurrentScopeDomain = response.Coverage.CurrentScopeDomain,
CurrentScopeWeighted = response.Coverage.CurrentScopeWeighted,
Domains = response.Coverage.Domains.Select(static domain => new UnifiedSearchApiDomainCoverage
{
Domain = domain.Domain,
CandidateCount = domain.CandidateCount,
VisibleCardCount = domain.VisibleCardCount,
TopScore = domain.TopScore,
IsCurrentScope = domain.IsCurrentScope,
HasVisibleResults = domain.HasVisibleResults
}).ToArray()
},
Synthesis = synthesis,
Suggestions = suggestions,
Refinements = refinements,
@@ -528,6 +634,40 @@ public static class UnifiedSearchEndpoints
};
}
private static UnifiedSearchSuggestionViabilityApiResponse MapSuggestionViabilityResponse(
SearchSuggestionViabilityResponse response)
{
return new UnifiedSearchSuggestionViabilityApiResponse
{
Suggestions = response.Suggestions.Select(static suggestion => new UnifiedSearchSuggestionViabilityApiResult
{
Query = suggestion.Query,
Viable = suggestion.Viable,
Status = suggestion.Status,
Code = suggestion.Code,
CardCount = suggestion.CardCount,
LeadingDomain = suggestion.LeadingDomain,
Reason = suggestion.Reason
}).ToArray(),
Coverage = response.Coverage is null
? null
: new UnifiedSearchApiCoverage
{
CurrentScopeDomain = response.Coverage.CurrentScopeDomain,
CurrentScopeWeighted = response.Coverage.CurrentScopeWeighted,
Domains = response.Coverage.Domains.Select(static domain => new UnifiedSearchApiDomainCoverage
{
Domain = domain.Domain,
CandidateCount = domain.CandidateCount,
VisibleCardCount = domain.VisibleCardCount,
TopScore = domain.TopScore,
IsCurrentScope = domain.IsCurrentScope,
HasVisibleResults = domain.HasVisibleResults
}).ToArray()
}
};
}
private static bool HasSynthesisScope(HttpContext context)
{
var scopes = ResolveUserScopes(context);
@@ -723,6 +863,15 @@ public sealed record UnifiedSearchApiRequest
public UnifiedSearchApiAmbientContext? Ambient { get; init; }
}
public sealed record UnifiedSearchSuggestionViabilityApiRequest
{
public IReadOnlyList<string> Queries { get; init; } = [];
public UnifiedSearchApiFilter? Filters { get; init; }
public UnifiedSearchApiAmbientContext? Ambient { get; init; }
}
public sealed record UnifiedSearchApiAmbientContext
{
public string? CurrentRoute { get; init; }
@@ -802,6 +951,10 @@ public sealed record UnifiedSearchApiResponse
public IReadOnlyList<UnifiedSearchApiCard> Cards { get; init; } = [];
public UnifiedSearchApiOverflow? Overflow { get; init; }
public UnifiedSearchApiCoverage? Coverage { get; init; }
public UnifiedSearchApiSynthesis? Synthesis { get; init; }
public IReadOnlyList<UnifiedSearchApiSuggestion>? Suggestions { get; init; }
@@ -815,6 +968,30 @@ public sealed record UnifiedSearchApiResponse
public IReadOnlyList<UnifiedSearchApiFederationDiagnostic>? Federation { get; init; }
}
public sealed record UnifiedSearchSuggestionViabilityApiResponse
{
public IReadOnlyList<UnifiedSearchSuggestionViabilityApiResult> Suggestions { get; init; } = [];
public UnifiedSearchApiCoverage? Coverage { get; init; }
}
public sealed record UnifiedSearchSuggestionViabilityApiResult
{
public string Query { get; init; } = string.Empty;
public bool Viable { get; init; }
public string Status { get; init; } = string.Empty;
public string Code { get; init; } = string.Empty;
public int CardCount { get; init; }
public string? LeadingDomain { get; init; }
public string Reason { get; init; } = string.Empty;
}
public sealed record UnifiedSearchApiCard
{
public string EntityKey { get; init; } = string.Empty;
@@ -845,6 +1022,39 @@ public sealed record UnifiedSearchApiCard
new Dictionary<string, string>(StringComparer.Ordinal);
}
public sealed record UnifiedSearchApiOverflow
{
public string CurrentScopeDomain { get; init; } = string.Empty;
public string Reason { get; init; } = string.Empty;
public IReadOnlyList<UnifiedSearchApiCard> Cards { get; init; } = [];
}
public sealed record UnifiedSearchApiCoverage
{
public string? CurrentScopeDomain { get; init; }
public bool CurrentScopeWeighted { get; init; }
public IReadOnlyList<UnifiedSearchApiDomainCoverage> Domains { get; init; } = [];
}
public sealed record UnifiedSearchApiDomainCoverage
{
public string Domain { get; init; } = string.Empty;
public int CandidateCount { get; init; }
public int VisibleCardCount { get; init; }
public double TopScore { get; init; }
public bool IsCurrentScope { get; init; }
public bool HasVisibleResults { get; init; }
}
public sealed record UnifiedSearchApiFacet
{
public string Domain { get; init; } = "knowledge";
@@ -889,6 +1099,10 @@ public sealed record UnifiedSearchApiSuggestion
public string Text { get; init; } = string.Empty;
public string Reason { get; init; } = string.Empty;
public string? Domain { get; init; }
public int CandidateCount { get; init; }
}
public sealed record UnifiedSearchApiRefinement
@@ -896,6 +1110,10 @@ public sealed record UnifiedSearchApiRefinement
public string Text { get; init; } = string.Empty;
public string Source { get; init; } = string.Empty;
public string? Domain { get; init; }
public int CandidateCount { get; init; }
}
public sealed record UnifiedSearchApiContextAnswer

1
src/AdvisoryAI/StellaOps.AdvisoryAI.WebService/TASKS.md
View File

@@ -19,3 +19,4 @@ Source of truth: `docs/implplan/SPRINT_20260130_002_Tools_csproj_remediation_sol
| AI-SELF-001 | DONE | Unified search endpoint contract now exposes backend contextual answer fields for self-serve search. |
| AI-SELF-004 | DONE | Search analytics and quality endpoints now surface self-serve metrics and alerts (`fallback_loop`, `abandoned_fallback`) while keeping telemetry optional. |
| AI-SELF-006 | DONE | Endpoint readiness now includes a proven local rebuilt-corpus verification lane in addition to stubbed integration tests. |
| SPRINT_20260307_019-AI-ZL | DONE | Unified search endpoint contract now exposes additive `overflow`/`coverage` query metadata and `POST /v1/search/suggestions/evaluate` for bounded suggestion viability checks. |

1
src/AdvisoryAI/StellaOps.AdvisoryAI/TASKS.md
View File

@@ -15,6 +15,7 @@ Source of truth: `docs/implplan/SPRINT_20260113_005_ADVISORYAI_controlled_conver
| AI-SELF-003 | DONE | Follow-up question generation from route/domain intent, recent actions, and evidence is implemented. |
| AI-SELF-004 | DONE | Optional self-serve telemetry now captures answer frames, reformulations, rescue actions, and abandonment signals via hashed session ids; quality metrics and alerts expose the gaps operationally. |
| AI-SELF-006 | DONE | Live ingestion-backed answer verification succeeded on the Doctor/knowledge route after local rebuild. |
| SPRINT_20260307_019-AI-ZL | DONE | Unified search now applies implicit current-scope weighting, emits additive `overflow`/`coverage`, blends close top answers, and evaluates suggestion viability without requiring telemetry. |
| SPRINT_20260222_051-AKS-INGEST | DONE | Added deterministic AKS ingestion controls: markdown allow-list manifest loading, OpenAPI aggregate source path support, and doctor control projection integration for search chunks, including fallback doctor metadata hydration from controls projection fields. |
| AUDIT-0017-M | DONE | Maintainability audit for StellaOps.AdvisoryAI. |
| AUDIT-0017-T | DONE | Test coverage audit for StellaOps.AdvisoryAI. |

4
src/AdvisoryAI/StellaOps.AdvisoryAI/UnifiedSearch/Context/AmbientContextProcessor.cs
View File

@@ -2,8 +2,8 @@ namespace StellaOps.AdvisoryAI.UnifiedSearch.Context;
internal sealed class AmbientContextProcessor
{
private const double CurrentRouteBoost = 0.10d;
private const double LastActionDomainBoost = 0.05d;
private const double CurrentRouteBoost = 0.35d;
private const double LastActionDomainBoost = 0.15d;
private const double VisibleEntityBoost = 0.20d;
private const double LastActionEntityBoost = 0.25d;
private static readonly (string Prefix, string Domain)[] RouteDomainMappings =

4
src/AdvisoryAI/StellaOps.AdvisoryAI/UnifiedSearch/IUnifiedSearchService.cs
View File

@@ -3,4 +3,8 @@ namespace StellaOps.AdvisoryAI.UnifiedSearch;
public interface IUnifiedSearchService
{
Task<UnifiedSearchResponse> SearchAsync(UnifiedSearchRequest request, CancellationToken cancellationToken);
Task<SearchSuggestionViabilityResponse> EvaluateSuggestionsAsync(
SearchSuggestionViabilityRequest request,
CancellationToken cancellationToken);
}

52
src/AdvisoryAI/StellaOps.AdvisoryAI/UnifiedSearch/UnifiedSearchModels.cs
View File

@@ -90,9 +90,17 @@ public sealed record AmbientAction
public DateTimeOffset? OccurredAt { get; init; }
}
public sealed record SearchSuggestion(string Text, string Reason);
public sealed record SearchSuggestion(
string Text,
string Reason,
string? Domain = null,
int CandidateCount = 0);
public sealed record SearchRefinement(string Text, string Source);
public sealed record SearchRefinement(
string Text,
string Source,
string? Domain = null,
int CandidateCount = 0);
public sealed record ContextAnswer(
string Status,
@@ -121,7 +129,45 @@ public sealed record UnifiedSearchResponse(
UnifiedSearchDiagnostics Diagnostics,
IReadOnlyList<SearchSuggestion>? Suggestions = null,
IReadOnlyList<SearchRefinement>? Refinements = null,
ContextAnswer? ContextAnswer = null);
ContextAnswer? ContextAnswer = null,
UnifiedSearchOverflow? Overflow = null,
UnifiedSearchCoverage? Coverage = null);
public sealed record UnifiedSearchOverflow(
string CurrentScopeDomain,
string Reason,
IReadOnlyList<EntityCard> Cards);
public sealed record UnifiedSearchCoverage(
string? CurrentScopeDomain,
bool CurrentScopeWeighted,
IReadOnlyList<UnifiedSearchDomainCoverage> Domains);
public sealed record UnifiedSearchDomainCoverage(
string Domain,
int CandidateCount,
int VisibleCardCount,
double TopScore,
bool IsCurrentScope,
bool HasVisibleResults);
public sealed record SearchSuggestionViabilityRequest(
IReadOnlyList<string> Queries,
UnifiedSearchFilter? Filters = null,
AmbientContext? Ambient = null);
public sealed record SearchSuggestionViabilityResponse(
IReadOnlyList<SearchSuggestionViabilityResult> Suggestions,
UnifiedSearchCoverage Coverage);
public sealed record SearchSuggestionViabilityResult(
string Query,
bool Viable,
string Status,
string Code,
int CardCount,
string? LeadingDomain,
string Reason);
public sealed record EntityCard
{

510
src/AdvisoryAI/StellaOps.AdvisoryAI/UnifiedSearch/UnifiedSearchService.cs
View File

@@ -21,6 +21,11 @@ internal sealed class UnifiedSearchService : IUnifiedSearchService
private const int MaxContextAnswerCitations = 3;
private const int MaxContextAnswerQuestions = 3;
private const int ClarifyTokenThreshold = 3;
private const int MaxSuggestionViabilityQueries = 6;
private const int MaxOverflowCards = 4;
private const int CoverageCandidateWindow = 24;
private const double OverflowScoreBandRatio = 0.15d;
private const double BlendedAnswerScoreBandRatio = 0.12d;
private readonly KnowledgeSearchOptions _options;
private readonly UnifiedSearchOptions _unifiedOptions;
private readonly IKnowledgeSearchStore _store;
@@ -57,6 +62,12 @@ internal sealed class UnifiedSearchService : IUnifiedSearchService
// Refinement threshold: only suggest when result count is below this (G10-004)
private const int RefinementResultThreshold = 3;
private enum SearchExecutionKind
{
UserVisible,
SuggestionPreflight
}
public UnifiedSearchService(
IOptions<KnowledgeSearchOptions> options,
IKnowledgeSearchStore store,
@@ -96,7 +107,77 @@ internal sealed class UnifiedSearchService : IUnifiedSearchService
_telemetrySink = telemetrySink;
}
public async Task<UnifiedSearchResponse> SearchAsync(UnifiedSearchRequest request, CancellationToken cancellationToken)
public Task<UnifiedSearchResponse> SearchAsync(UnifiedSearchRequest request, CancellationToken cancellationToken)
{
return SearchAsyncInternal(request, cancellationToken, SearchExecutionKind.UserVisible);
}
public async Task<SearchSuggestionViabilityResponse> EvaluateSuggestionsAsync(
SearchSuggestionViabilityRequest request,
CancellationToken cancellationToken)
{
ArgumentNullException.ThrowIfNull(request);
var normalizedQueries = request.Queries
.Where(static query => !string.IsNullOrWhiteSpace(query))
.Select(static query => KnowledgeSearchText.NormalizeWhitespace(query))
.Where(static query => !string.IsNullOrWhiteSpace(query))
.Distinct(StringComparer.OrdinalIgnoreCase)
.Take(MaxSuggestionViabilityQueries)
.ToArray();
var fallbackCoverage = new UnifiedSearchCoverage(
CurrentScopeDomain: ResolveAmbientScopeDomain(request.Ambient),
CurrentScopeWeighted: !string.IsNullOrWhiteSpace(request.Ambient?.CurrentRoute),
Domains: []);
if (normalizedQueries.Length == 0)
{
return new SearchSuggestionViabilityResponse([], fallbackCoverage);
}
var results = new List<SearchSuggestionViabilityResult>(normalizedQueries.Length);
UnifiedSearchCoverage? aggregateCoverage = null;
foreach (var query in normalizedQueries)
{
var response = await SearchAsyncInternal(
new UnifiedSearchRequest(
query,
K: Math.Min(5, _unifiedOptions.MaxCards),
Filters: request.Filters,
IncludeSynthesis: false,
IncludeDebug: false,
Ambient: request.Ambient),
cancellationToken,
SearchExecutionKind.SuggestionPreflight).ConfigureAwait(false);
aggregateCoverage = MergeCoverage(aggregateCoverage, response.Coverage);
var cardCount = response.Cards.Count + (response.Overflow?.Cards.Count ?? 0);
var answer = response.ContextAnswer;
results.Add(new SearchSuggestionViabilityResult(
Query: query,
Viable: cardCount > 0 || string.Equals(answer?.Status, "clarify", StringComparison.OrdinalIgnoreCase),
Status: answer?.Status ?? "insufficient",
Code: answer?.Code ?? "no_grounded_evidence",
CardCount: cardCount,
LeadingDomain: response.Cards.FirstOrDefault()?.Domain
?? response.Overflow?.Cards.FirstOrDefault()?.Domain
?? response.Coverage?.Domains.FirstOrDefault(static domain => domain.HasVisibleResults)?.Domain
?? response.Coverage?.CurrentScopeDomain,
Reason: answer?.Reason ?? "No grounded evidence matched the suggestion in the active corpus."));
}
return new SearchSuggestionViabilityResponse(
results,
aggregateCoverage ?? fallbackCoverage);
}
private async Task<UnifiedSearchResponse> SearchAsyncInternal(
UnifiedSearchRequest request,
CancellationToken cancellationToken,
SearchExecutionKind executionKind)
{
ArgumentNullException.ThrowIfNull(request);
@@ -107,13 +188,17 @@ internal sealed class UnifiedSearchService : IUnifiedSearchService
return EmptyResponse(string.Empty, request.K, "empty");
}
var emitObservability = executionKind == SearchExecutionKind.UserVisible;
var tenantId = request.Filters?.Tenant ?? "global";
var userId = request.Filters?.UserId ?? "anonymous";
if (query.Length > _unifiedOptions.MaxQueryLength)
{
var earlyResponse = EmptyResponse(query, request.K, "query_too_long", request.Ambient);
await RecordAnswerFrameAnalyticsAsync(tenantId, userId, request, plan: null, earlyResponse, cancellationToken).ConfigureAwait(false);
if (emitObservability)
{
await RecordAnswerFrameAnalyticsAsync(tenantId, userId, request, plan: null, earlyResponse, cancellationToken).ConfigureAwait(false);
}
return earlyResponse;
}
@@ -122,7 +207,10 @@ internal sealed class UnifiedSearchService : IUnifiedSearchService
if (!_options.Enabled || !IsSearchEnabledForTenant(tenantFlags) || string.IsNullOrWhiteSpace(_options.ConnectionString))
{
var earlyResponse = EmptyResponse(query, request.K, "disabled", request.Ambient);
await RecordAnswerFrameAnalyticsAsync(tenantId, userId, request, plan: null, earlyResponse, cancellationToken).ConfigureAwait(false);
if (emitObservability)
{
await RecordAnswerFrameAnalyticsAsync(tenantId, userId, request, plan: null, earlyResponse, cancellationToken).ConfigureAwait(false);
}
return earlyResponse;
}
@@ -266,7 +354,7 @@ internal sealed class UnifiedSearchService : IUnifiedSearchService
cards = cards.Take(Math.Max(1, _unifiedOptions.MaxCards)).ToArray();
if (!string.IsNullOrWhiteSpace(request.Ambient?.SessionId))
if (emitObservability && !string.IsNullOrWhiteSpace(request.Ambient?.SessionId))
{
_searchSessionContext.RecordQuery(
tenantId,
@@ -276,16 +364,21 @@ internal sealed class UnifiedSearchService : IUnifiedSearchService
_timeProvider.GetUtcNow());
}
var currentScopeDomain = ResolveAmbientScopeDomain(request.Ambient);
var (primaryCards, overflow) = PartitionCardsByScope(cards, currentScopeDomain);
var visibleCards = BuildVisibleAnswerCards(primaryCards, overflow);
var coverage = BuildCoverage(currentScopeDomain, merged, primaryCards, overflow);
SynthesisResult? synthesis = null;
if (request.IncludeSynthesis && IsSynthesisEnabledForTenant(tenantFlags) && cards.Count > 0)
if (request.IncludeSynthesis && IsSynthesisEnabledForTenant(tenantFlags) && visibleCards.Count > 0)
{
synthesis = await _synthesisEngine.SynthesizeAsync(
query, cards, plan.DetectedEntities, cancellationToken).ConfigureAwait(false);
query, visibleCards, plan.DetectedEntities, cancellationToken).ConfigureAwait(false);
}
// G4-003: Generate "Did you mean?" suggestions when results are sparse
IReadOnlyList<SearchSuggestion>? suggestions = null;
if (cards.Count < _options.MinFtsResultsForFuzzyFallback && _options.FuzzyFallbackEnabled)
if (visibleCards.Count < _options.MinFtsResultsForFuzzyFallback && _options.FuzzyFallbackEnabled)
{
suggestions = await GenerateSuggestionsAsync(
query, storeFilter, cancellationToken).ConfigureAwait(false);
@@ -293,10 +386,10 @@ internal sealed class UnifiedSearchService : IUnifiedSearchService
// G10-004: Generate query refinement suggestions from feedback data
IReadOnlyList<SearchRefinement>? refinements = null;
if (cards.Count < RefinementResultThreshold)
if (visibleCards.Count < RefinementResultThreshold)
{
refinements = await GenerateRefinementsAsync(
tenantId, query, cards.Count, cancellationToken).ConfigureAwait(false);
tenantId, query, visibleCards.Count, storeFilter, cancellationToken).ConfigureAwait(false);
}
var duration = _timeProvider.GetUtcNow() - startedAt;
@@ -304,19 +397,22 @@ internal sealed class UnifiedSearchService : IUnifiedSearchService
query,
plan,
request.Ambient,
cards,
primaryCards,
overflow,
synthesis,
suggestions,
refinements);
refinements,
coverage);
var totalVisibleCardCount = primaryCards.Count + (overflow?.Cards.Count ?? 0);
var response = new UnifiedSearchResponse(
query,
topK,
cards,
primaryCards,
synthesis,
new UnifiedSearchDiagnostics(
ftsRows.Count,
vectorRows.Length,
cards.Count,
totalVisibleCardCount,
(long)duration.TotalMilliseconds,
usedVector,
usedVector ? "hybrid" : "fts-only",
@@ -324,10 +420,16 @@ internal sealed class UnifiedSearchService : IUnifiedSearchService
federationDiagnostics),
suggestions,
refinements,
contextAnswer);
contextAnswer,
overflow,
coverage);
if (emitObservability)
{
await RecordAnswerFrameAnalyticsAsync(tenantId, userId, request, plan, response, cancellationToken).ConfigureAwait(false);
EmitTelemetry(plan, response, tenantId);
}
await RecordAnswerFrameAnalyticsAsync(tenantId, userId, request, plan, response, cancellationToken).ConfigureAwait(false);
EmitTelemetry(plan, response, tenantId);
return response;
}
@@ -388,6 +490,198 @@ internal sealed class UnifiedSearchService : IUnifiedSearchService
};
}
private static (IReadOnlyList<EntityCard> PrimaryCards, UnifiedSearchOverflow? Overflow) PartitionCardsByScope(
IReadOnlyList<EntityCard> rankedCards,
string? currentScopeDomain)
{
if (rankedCards.Count == 0 || string.IsNullOrWhiteSpace(currentScopeDomain))
{
return (rankedCards, null);
}
var scopedCards = rankedCards
.Where(card => string.Equals(card.Domain, currentScopeDomain, StringComparison.OrdinalIgnoreCase))
.ToArray();
if (scopedCards.Length == 0)
{
return (rankedCards, null);
}
var overflowCards = rankedCards
.Where(card => !string.Equals(card.Domain, currentScopeDomain, StringComparison.OrdinalIgnoreCase))
.Where(card => ShouldSurfaceOverflow(scopedCards[0].Score, card.Score))
.Take(MaxOverflowCards)
.ToArray();
if (overflowCards.Length == 0)
{
return (scopedCards, null);
}
return (
scopedCards,
new UnifiedSearchOverflow(
currentScopeDomain,
BuildOverflowReason(currentScopeDomain, scopedCards[0], overflowCards[0]),
overflowCards));
}
private static bool ShouldSurfaceOverflow(double topScopedScore, double candidateScore)
{
if (candidateScore >= topScopedScore)
{
return true;
}
var denominator = Math.Max(Math.Abs(topScopedScore), 0.000001d);
var relativeGap = (topScopedScore - candidateScore) / denominator;
return relativeGap <= OverflowScoreBandRatio;
}
private static string BuildOverflowReason(string currentScopeDomain, EntityCard topScopedCard, EntityCard topOverflowCard)
{
if (topOverflowCard.Score > topScopedCard.Score)
{
return $"Related results outside {DescribeDomain(currentScopeDomain)} outranked the current-scope evidence.";
}
return $"Related results outside {DescribeDomain(currentScopeDomain)} are close enough in score to surface below the in-scope results.";
}
private static IReadOnlyList<EntityCard> BuildVisibleAnswerCards(
IReadOnlyList<EntityCard> primaryCards,
UnifiedSearchOverflow? overflow)
{
return primaryCards
.Concat(overflow?.Cards ?? [])
.OrderByDescending(static card => card.Score)
.ThenBy(static card => card.Title, StringComparer.Ordinal)
.ThenBy(static card => card.EntityKey, StringComparer.Ordinal)
.ToArray();
}
private static IReadOnlyList<EntityCard> SelectDominantAnswerCards(IReadOnlyList<EntityCard> visibleCards)
{
if (visibleCards.Count <= 1)
{
return visibleCards.Take(1).ToArray();
}
var topScore = Math.Max(Math.Abs(visibleCards[0].Score), 0.000001d);
var blended = visibleCards
.Where(card => (topScore - card.Score) / topScore <= BlendedAnswerScoreBandRatio)
.Take(MaxContextAnswerCitations)
.ToArray();
return blended.Length >= 2
? blended
: visibleCards.Take(1).ToArray();
}
private static UnifiedSearchCoverage BuildCoverage(
string? currentScopeDomain,
IReadOnlyList<(KnowledgeChunkRow Row, double Score, IReadOnlyDictionary<string, string> Debug)> merged,
IReadOnlyList<EntityCard> primaryCards,
UnifiedSearchOverflow? overflow)
{
var visibleCounts = new Dictionary<string, int>(StringComparer.OrdinalIgnoreCase);
foreach (var card in primaryCards.Concat(overflow?.Cards ?? []))
{
visibleCounts[card.Domain] = visibleCounts.TryGetValue(card.Domain, out var existing)
? existing + 1
: 1;
}
var domains = merged
.Take(CoverageCandidateWindow)
.GroupBy(static item => GetDomain(item.Row), StringComparer.OrdinalIgnoreCase)
.Select(group =>
{
var domain = group.Key;
var visibleCount = visibleCounts.TryGetValue(domain, out var count) ? count : 0;
return new UnifiedSearchDomainCoverage(
Domain: domain,
CandidateCount: group.Count(),
VisibleCardCount: visibleCount,
TopScore: group.Max(static item => item.Score),
IsCurrentScope: string.Equals(domain, currentScopeDomain, StringComparison.OrdinalIgnoreCase),
HasVisibleResults: visibleCount > 0);
})
.ToDictionary(static entry => entry.Domain, StringComparer.OrdinalIgnoreCase);
if (!string.IsNullOrWhiteSpace(currentScopeDomain) && !domains.ContainsKey(currentScopeDomain))
{
domains[currentScopeDomain] = new UnifiedSearchDomainCoverage(
Domain: currentScopeDomain,
CandidateCount: 0,
VisibleCardCount: 0,
TopScore: 0d,
IsCurrentScope: true,
HasVisibleResults: false);
}
var ordered = domains.Values
.OrderByDescending(static entry => entry.IsCurrentScope)
.ThenByDescending(static entry => entry.HasVisibleResults)
.ThenByDescending(static entry => entry.CandidateCount)
.ThenByDescending(static entry => entry.TopScore)
.ThenBy(static entry => entry.Domain, StringComparer.Ordinal)
.ToArray();
return new UnifiedSearchCoverage(
CurrentScopeDomain: currentScopeDomain,
CurrentScopeWeighted: !string.IsNullOrWhiteSpace(currentScopeDomain),
Domains: ordered);
}
private static UnifiedSearchCoverage MergeCoverage(
UnifiedSearchCoverage? existing,
UnifiedSearchCoverage? current)
{
if (current is null)
{
return existing ?? new UnifiedSearchCoverage(null, false, []);
}
if (existing is null)
{
return current;
}
var domains = new Dictionary<string, UnifiedSearchDomainCoverage>(StringComparer.OrdinalIgnoreCase);
foreach (var coverage in existing.Domains.Concat(current.Domains))
{
if (!domains.TryGetValue(coverage.Domain, out var prior))
{
domains[coverage.Domain] = coverage;
continue;
}
domains[coverage.Domain] = new UnifiedSearchDomainCoverage(
Domain: coverage.Domain,
CandidateCount: Math.Max(prior.CandidateCount, coverage.CandidateCount),
VisibleCardCount: Math.Max(prior.VisibleCardCount, coverage.VisibleCardCount),
TopScore: Math.Max(prior.TopScore, coverage.TopScore),
IsCurrentScope: prior.IsCurrentScope || coverage.IsCurrentScope,
HasVisibleResults: prior.HasVisibleResults || coverage.HasVisibleResults);
}
var currentScopeDomain = current.CurrentScopeDomain
?? existing.CurrentScopeDomain;
return new UnifiedSearchCoverage(
CurrentScopeDomain: currentScopeDomain,
CurrentScopeWeighted: existing.CurrentScopeWeighted || current.CurrentScopeWeighted,
Domains: domains.Values
.OrderByDescending(static entry => entry.IsCurrentScope)
.ThenByDescending(static entry => entry.HasVisibleResults)
.ThenByDescending(static entry => entry.CandidateCount)
.ThenByDescending(static entry => entry.TopScore)
.ThenBy(static entry => entry.Domain, StringComparer.Ordinal)
.ToArray());
}
private static IReadOnlyList<ContextAnswerCitation> BuildContextAnswerCitations(
IReadOnlyList<EntityCard> cards,
SynthesisResult? synthesis)
@@ -441,7 +735,7 @@ internal sealed class UnifiedSearchService : IUnifiedSearchService
return citations;
}
private static string BuildGroundedSummary(IReadOnlyList<EntityCard> cards, SynthesisResult? synthesis)
private static string BuildGroundedSummary(IReadOnlyList<EntityCard> answerCards, SynthesisResult? synthesis)
{
var synthesisSummary = synthesis?.Summary?.Trim();
if (!string.IsNullOrWhiteSpace(synthesisSummary))
@@ -449,27 +743,71 @@ internal sealed class UnifiedSearchService : IUnifiedSearchService
return synthesisSummary;
}
var topCard = cards[0];
var topCard = answerCards[0];
if (answerCards.Count > 1)
{
var related = answerCards
.Skip(1)
.Select(static card => card.Title)
.Where(static title => !string.IsNullOrWhiteSpace(title))
.Take(2)
.ToArray();
if (related.Length > 0)
{
return $"Top evidence points to {topCard.Title}. Related high-confidence matches also include {string.Join(", ", related)}.";
}
}
var snippet = topCard.Snippet?.Trim();
return string.IsNullOrWhiteSpace(snippet) ? topCard.Title : snippet;
}
private static string BuildGroundedReason(QueryPlan plan, AmbientContext? ambient, EntityCard topCard)
private static string BuildGroundedReason(
QueryPlan plan,
AmbientContext? ambient,
IReadOnlyList<EntityCard> answerCards,
UnifiedSearchCoverage? coverage,
UnifiedSearchOverflow? overflow)
{
var scope = ResolveContextDomain(plan, [topCard], ambient) ?? topCard.Domain;
var topCard = answerCards[0];
var scope = coverage?.CurrentScopeDomain
?? ResolveContextDomain(plan, [topCard], ambient)
?? topCard.Domain;
if (answerCards.Count > 1)
{
return $"The highest-ranked results are close in score, so the answer blends evidence across {FormatDomainList(answerCards.Select(static card => card.Domain))}.";
}
if (overflow is not null)
{
return $"Current-scope weighting kept {DescribeDomain(scope)} first, while close related evidence from other domains remains visible below.";
}
return $"The top result is grounded in {DescribeDomain(scope)} evidence and aligns with the {plan.Intent} intent.";
}
private static string BuildGroundedEvidence(IReadOnlyList<EntityCard> cards, SynthesisResult? synthesis)
private static string BuildGroundedEvidence(
IReadOnlyList<EntityCard> visibleCards,
IReadOnlyList<EntityCard> answerCards,
SynthesisResult? synthesis,
UnifiedSearchCoverage? coverage,
UnifiedSearchOverflow? overflow)
{
var sourceCount = Math.Max(synthesis?.SourceCount ?? 0, cards.Count);
var sourceCount = Math.Max(synthesis?.SourceCount ?? 0, visibleCards.Count);
var domains = synthesis?.DomainsCovered is { Count: > 0 }
? synthesis.DomainsCovered
: cards.Select(static card => card.Domain)
: answerCards.Select(static card => card.Domain)
.Where(static domain => !string.IsNullOrWhiteSpace(domain))
.Distinct(StringComparer.OrdinalIgnoreCase)
.ToArray();
if (overflow is not null && coverage?.CurrentScopeDomain is { Length: > 0 } currentScopeDomain)
{
return $"Grounded in {sourceCount} source(s) across {FormatDomainList(domains)} with {DescribeDomain(currentScopeDomain)} weighted first.";
}
return $"Grounded in {sourceCount} source(s) across {FormatDomainList(domains)}.";
}
@@ -826,6 +1164,28 @@ internal sealed class UnifiedSearchService : IUnifiedSearchService
.FirstOrDefault(static value => !string.IsNullOrWhiteSpace(value));
}
private static string? ResolveAmbientScopeDomain(AmbientContext? ambient)
{
var routeDomain = AmbientContextProcessor.ResolveDomainFromRoute(ambient?.CurrentRoute ?? string.Empty);
if (!string.IsNullOrWhiteSpace(routeDomain))
{
return routeDomain;
}
if (!string.IsNullOrWhiteSpace(ambient?.LastAction?.Domain))
{
return ambient.LastAction.Domain.Trim().ToLowerInvariant();
}
var actionRouteDomain = AmbientContextProcessor.ResolveDomainFromRoute(ambient?.LastAction?.Route ?? string.Empty);
if (!string.IsNullOrWhiteSpace(actionRouteDomain))
{
return actionRouteDomain;
}
return null;
}
private static string DescribeDomain(string domain)
{
return domain switch
@@ -1406,26 +1766,34 @@ internal sealed class UnifiedSearchService : IUnifiedSearchService
QueryPlan plan,
AmbientContext? ambient,
IReadOnlyList<EntityCard> cards,
UnifiedSearchOverflow? overflow,
SynthesisResult? synthesis,
IReadOnlyList<SearchSuggestion>? suggestions,
IReadOnlyList<SearchRefinement>? refinements)
IReadOnlyList<SearchRefinement>? refinements,
UnifiedSearchCoverage? coverage)
{
if (string.IsNullOrWhiteSpace(query))
{
return null;
}
if (cards.Count > 0)
var visibleCards = BuildVisibleAnswerCards(cards, overflow);
if (visibleCards.Count > 0)
{
var topCard = cards[0];
var citations = BuildContextAnswerCitations(cards, synthesis);
var answerCards = SelectDominantAnswerCards(visibleCards);
var topCard = answerCards[0];
var citations = BuildContextAnswerCitations(answerCards, synthesis);
var questions = BuildGroundedQuestions(query, plan, ambient, topCard);
return new ContextAnswer(
Status: "grounded",
Code: "retrieved_evidence",
Summary: BuildGroundedSummary(cards, synthesis),
Reason: BuildGroundedReason(plan, ambient, topCard),
Evidence: BuildGroundedEvidence(cards, synthesis),
Code: answerCards.Count > 1
? "retrieved_blended_evidence"
: overflow is not null
? "retrieved_scope_weighted_evidence"
: "retrieved_evidence",
Summary: BuildGroundedSummary(answerCards, synthesis),
Reason: BuildGroundedReason(plan, ambient, answerCards, coverage, overflow),
Evidence: BuildGroundedEvidence(visibleCards, answerCards, synthesis, coverage, overflow),
Citations: citations,
Questions: questions);
}
@@ -1581,7 +1949,11 @@ internal sealed class UnifiedSearchService : IUnifiedSearchService
continue;
}
suggestions.Add(new SearchSuggestion(text, $"Similar to \"{query}\""));
suggestions.Add(new SearchSuggestion(
Text: text,
Reason: $"Similar to \"{query}\"",
Domain: GetDomain(row),
CandidateCount: 1));
if (suggestions.Count >= maxSuggestions)
{
@@ -1672,7 +2044,11 @@ internal sealed class UnifiedSearchService : IUnifiedSearchService
/// Sprint: G10-004
/// </summary>
private async Task<IReadOnlyList<SearchRefinement>?> GenerateRefinementsAsync(
string tenantId, string query, int resultCount, CancellationToken ct)
string tenantId,
string query,
int resultCount,
KnowledgeSearchFilter? storeFilter,
CancellationToken ct)
{
if (resultCount >= RefinementResultThreshold)
{
@@ -1705,9 +2081,14 @@ internal sealed class UnifiedSearchService : IUnifiedSearchService
var text = alert.Resolution.Trim();
if (text.Length > 120) text = text[..120].TrimEnd();
if (seen.Add(text))
var probe = await ProbeCandidateAsync(text, storeFilter, refinementCt).ConfigureAwait(false);
if (probe is not null && seen.Add(text))
{
refinements.Add(new SearchRefinement(text, "resolved_alert"));
refinements.Add(new SearchRefinement(
Text: text,
Source: "resolved_alert",
Domain: probe.Value.Domain,
CandidateCount: probe.Value.CandidateCount));
}
}
@@ -1721,9 +2102,14 @@ internal sealed class UnifiedSearchService : IUnifiedSearchService
{
if (refinements.Count >= maxRefinements) break;
if (seen.Add(similarQuery))
var probe = await ProbeCandidateAsync(similarQuery, storeFilter, refinementCt).ConfigureAwait(false);
if (probe is not null && seen.Add(similarQuery))
{
refinements.Add(new SearchRefinement(similarQuery, "similar_successful_query"));
refinements.Add(new SearchRefinement(
Text: similarQuery,
Source: "similar_successful_query",
Domain: probe.Value.Domain,
CandidateCount: probe.Value.CandidateCount));
}
}
}
@@ -1737,9 +2123,19 @@ internal sealed class UnifiedSearchService : IUnifiedSearchService
{
if (refinements.Count >= maxRefinements) break;
if (!string.IsNullOrWhiteSpace(entityKey) && seen.Add(entityKey))
if (string.IsNullOrWhiteSpace(entityKey))
{
refinements.Add(new SearchRefinement(entityKey, "entity_alias"));
continue;
}
var probe = await ProbeCandidateAsync(entityKey, storeFilter, refinementCt).ConfigureAwait(false);
if (probe is not null && seen.Add(entityKey))
{
refinements.Add(new SearchRefinement(
Text: entityKey,
Source: "entity_alias",
Domain: probe.Value.Domain,
CandidateCount: probe.Value.CandidateCount));
}
}
}
@@ -1759,6 +2155,42 @@ internal sealed class UnifiedSearchService : IUnifiedSearchService
return refinements.Count > 0 ? refinements : null;
}
private async Task<(string Domain, int CandidateCount)?> ProbeCandidateAsync(
string candidateQuery,
KnowledgeSearchFilter? storeFilter,
CancellationToken cancellationToken)
{
var normalized = KnowledgeSearchText.NormalizeWhitespace(candidateQuery);
if (string.IsNullOrWhiteSpace(normalized))
{
return null;
}
var timeout = TimeSpan.FromMilliseconds(Math.Clamp(_options.QueryTimeoutMs / 6, 50, 500));
var rows = await _store.SearchFtsAsync(
normalized,
storeFilter,
3,
timeout,
cancellationToken).ConfigureAwait(false);
if (rows.Count == 0)
{
return null;
}
var dominantDomain = rows
.GroupBy(GetDomain, StringComparer.OrdinalIgnoreCase)
.OrderByDescending(static group => group.Count())
.ThenBy(static group => group.Key, StringComparer.Ordinal)
.Select(static group => group.Key)
.FirstOrDefault();
return dominantDomain is null
? null
: (dominantDomain, rows.Count);
}
/// <summary>
/// Computes Jaccard similarity over character trigrams of two strings.
/// Used as an in-memory approximation of PostgreSQL pg_trgm similarity().

69
src/AdvisoryAI/__Tests/StellaOps.AdvisoryAI.Tests/Integration/UnifiedSearchEndpointsIntegrationTests.cs
View File

@@ -79,6 +79,31 @@ public sealed class UnifiedSearchEndpointsIntegrationTests : IDisposable
payload.ContextAnswer.Should().NotBeNull();
payload.ContextAnswer!.Status.Should().Be("grounded");
payload.ContextAnswer.Citations.Should().NotBeNullOrEmpty();
payload.Coverage.Should().NotBeNull();
payload.Coverage!.Domains.Should().Contain(domain => domain.Domain == "findings");
}
[Fact]
public async Task EvaluateSuggestions_WithOperateScope_ReturnsViabilityAndCoverage()
{
using var client = _factory.CreateClient();
client.DefaultRequestHeaders.Add("X-StellaOps-Scopes", "advisory-ai:operate");
client.DefaultRequestHeaders.Add("X-StellaOps-Tenant", "test-tenant");
var response = await client.PostAsJsonAsync("/v1/search/suggestions/evaluate", new UnifiedSearchSuggestionViabilityApiRequest
{
Queries = ["database connectivity"]
});
response.StatusCode.Should().Be(HttpStatusCode.OK);
var payload = await response.Content.ReadFromJsonAsync<UnifiedSearchSuggestionViabilityApiResponse>();
payload.Should().NotBeNull();
payload!.Suggestions.Should().ContainSingle();
payload.Suggestions[0].Viable.Should().BeTrue();
payload.Suggestions[0].Status.Should().Be("grounded");
payload.Coverage.Should().NotBeNull();
payload.Coverage!.Domains.Should().Contain(domain => domain.Domain == "knowledge");
}
[Fact]
@@ -360,6 +385,50 @@ public sealed class UnifiedSearchEndpointsIntegrationTests : IDisposable
Questions:
[
new ContextAnswerQuestion("What evidence proves exploitability for CVE-2024-21626?", "follow_up")
]),
Coverage: new UnifiedSearchCoverage(
CurrentScopeDomain: "findings",
CurrentScopeWeighted: true,
Domains:
[
new UnifiedSearchDomainCoverage(
Domain: "findings",
CandidateCount: 1,
VisibleCardCount: 1,
TopScore: 1.25,
IsCurrentScope: true,
HasVisibleResults: true)
])));
}
public Task<SearchSuggestionViabilityResponse> EvaluateSuggestionsAsync(
SearchSuggestionViabilityRequest request,
CancellationToken cancellationToken)
{
return Task.FromResult(new SearchSuggestionViabilityResponse(
Suggestions:
[
new SearchSuggestionViabilityResult(
Query: request.Queries.FirstOrDefault() ?? "database connectivity",
Viable: true,
Status: "grounded",
Code: "retrieved_evidence",
CardCount: 1,
LeadingDomain: "knowledge",
Reason: "Grounded knowledge evidence is available.")
],
Coverage: new UnifiedSearchCoverage(
CurrentScopeDomain: "knowledge",
CurrentScopeWeighted: true,
Domains:
[
new UnifiedSearchDomainCoverage(
Domain: "knowledge",
CandidateCount: 1,
VisibleCardCount: 1,
TopScore: 1.0,
IsCurrentScope: true,
HasVisibleResults: true)
])));
}
}

30
src/AdvisoryAI/__Tests/StellaOps.AdvisoryAI.Tests/Integration/UnifiedSearchSprintIntegrationTests.cs
View File

@@ -2476,6 +2476,36 @@ public sealed class UnifiedSearchSprintIntegrationTests : IDisposable
Mode: "fts-only"),
suggestions));
}
public Task<SearchSuggestionViabilityResponse> EvaluateSuggestionsAsync(
SearchSuggestionViabilityRequest request,
CancellationToken cancellationToken)
{
return Task.FromResult(new SearchSuggestionViabilityResponse(
Suggestions: request.Queries
.Select(static query => new SearchSuggestionViabilityResult(
Query: query,
Viable: true,
Status: "grounded",
Code: "retrieved_evidence",
CardCount: 1,
LeadingDomain: "knowledge",
Reason: "Stubbed search evidence is available."))
.ToArray(),
Coverage: new UnifiedSearchCoverage(
CurrentScopeDomain: "knowledge",
CurrentScopeWeighted: true,
Domains:
[
new UnifiedSearchDomainCoverage(
Domain: "knowledge",
CandidateCount: 1,
VisibleCardCount: 1,
TopScore: 1.0,
IsCurrentScope: true,
HasVisibleResults: true)
])));
}
}
private sealed class SprintStubUnifiedSearchIndexer : IUnifiedSearchIndexer

6
src/AdvisoryAI/__Tests/StellaOps.AdvisoryAI.Tests/UnifiedSearch/AmbientContextProcessorTests.cs
View File

@@ -32,7 +32,7 @@ public sealed class AmbientContextProcessorTests
CurrentRoute = "/ops/graph/nodes/node-1"
});
boosted["graph"].Should().BeApproximately(1.10, 0.0001);
boosted["graph"].Should().BeApproximately(1.35, 0.0001);
boosted["knowledge"].Should().Be(1.0);
}
@@ -56,8 +56,8 @@ public sealed class AmbientContextProcessorTests
}
});
boosted["knowledge"].Should().BeApproximately(1.10, 0.0001);
boosted["policy"].Should().BeApproximately(1.05, 0.0001);
boosted["knowledge"].Should().BeApproximately(1.35, 0.0001);
boosted["policy"].Should().BeApproximately(1.15, 0.0001);
}
[Fact]

161
src/AdvisoryAI/__Tests/StellaOps.AdvisoryAI.Tests/UnifiedSearch/UnifiedSearchServiceTests.cs
View File

@@ -84,6 +84,99 @@ public sealed class UnifiedSearchServiceTests
result.ContextAnswer.Questions.Should().NotBeNullOrEmpty();
}
[Fact]
public async Task SearchAsync_keeps_current_scope_primary_and_surfaces_strong_related_overflow()
{
var doctorRow = MakeRow(
"chunk-doctor-overflow",
"doctor_check",
"PostgreSQL connectivity",
JsonDocument.Parse("{\"domain\":\"knowledge\",\"checkCode\":\"check.core.db.connectivity\"}"),
snippet: "Doctor knowledge indicates the primary database is unreachable.");
var findingsEmbedding = new float[64];
findingsEmbedding[0] = 0.7f;
findingsEmbedding[1] = 0.2f;
var findingRow = MakeRow(
"chunk-find-overflow",
"finding",
"CVE-2026-9001 in postgres sidecar",
JsonDocument.Parse("{\"domain\":\"findings\",\"cveId\":\"CVE-2026-9001\",\"severity\":\"high\"}"),
embedding: findingsEmbedding,
snippet: "A related finding also points at database-side compromise risk.");
var storeMock = new Mock<IKnowledgeSearchStore>();
storeMock.Setup(s => s.SearchFtsAsync(
It.IsAny<string>(), It.IsAny<KnowledgeSearchFilter?>(), It.IsAny<int>(),
It.IsAny<TimeSpan>(), It.IsAny<CancellationToken>(), It.IsAny<string?>()))
.ReturnsAsync(new List<KnowledgeChunkRow> { doctorRow, findingRow });
storeMock.Setup(s => s.LoadVectorCandidatesAsync(
It.IsAny<float[]>(), It.IsAny<KnowledgeSearchFilter?>(), It.IsAny<int>(),
It.IsAny<TimeSpan>(), It.IsAny<CancellationToken>()))
.ReturnsAsync(new List<KnowledgeChunkRow> { findingRow });
var service = CreateService(storeMock: storeMock);
var result = await service.SearchAsync(
new UnifiedSearchRequest(
"database connectivity",
Ambient: new AmbientContext
{
CurrentRoute = "/ops/operations/doctor"
}),
CancellationToken.None);
result.Cards.Should().ContainSingle();
result.Cards[0].Domain.Should().Be("knowledge");
result.Overflow.Should().NotBeNull();
result.Overflow!.CurrentScopeDomain.Should().Be("knowledge");
result.Overflow.Cards.Should().ContainSingle();
result.Overflow.Cards[0].Domain.Should().Be("findings");
result.Coverage.Should().NotBeNull();
result.Coverage!.Domains.Should().Contain(domain =>
domain.Domain == "knowledge" && domain.IsCurrentScope);
result.ContextAnswer.Should().NotBeNull();
result.ContextAnswer!.Code.Should().Be("retrieved_scope_weighted_evidence");
}
[Fact]
public async Task SearchAsync_blends_close_top_results_into_one_answer()
{
var doctorRow = MakeRow(
"chunk-doctor-blend",
"doctor_check",
"PostgreSQL connectivity",
JsonDocument.Parse("{\"domain\":\"knowledge\",\"checkCode\":\"check.core.db.connectivity\"}"),
snippet: "Database connectivity is degraded.");
var guideRow = MakeRow(
"chunk-guide-blend",
"md_section",
"Database failover playbook",
JsonDocument.Parse("{\"domain\":\"knowledge\",\"path\":\"docs/database-failover.md\"}"),
snippet: "Failover guidance covers the same connectivity symptoms.");
var storeMock = new Mock<IKnowledgeSearchStore>();
storeMock.Setup(s => s.SearchFtsAsync(
It.IsAny<string>(), It.IsAny<KnowledgeSearchFilter?>(), It.IsAny<int>(),
It.IsAny<TimeSpan>(), It.IsAny<CancellationToken>(), It.IsAny<string?>()))
.ReturnsAsync(new List<KnowledgeChunkRow> { doctorRow, guideRow });
storeMock.Setup(s => s.LoadVectorCandidatesAsync(
It.IsAny<float[]>(), It.IsAny<KnowledgeSearchFilter?>(), It.IsAny<int>(),
It.IsAny<TimeSpan>(), It.IsAny<CancellationToken>()))
.ReturnsAsync([]);
var service = CreateService(storeMock: storeMock);
var result = await service.SearchAsync(
new UnifiedSearchRequest("database connectivity", IncludeSynthesis: false),
CancellationToken.None);
result.ContextAnswer.Should().NotBeNull();
result.ContextAnswer!.Code.Should().Be("retrieved_blended_evidence");
result.ContextAnswer.Summary.Should().Contain("PostgreSQL connectivity");
result.ContextAnswer.Summary.Should().Contain("Database failover playbook");
result.ContextAnswer.Citations.Should().HaveCount(2);
}
[Fact]
public async Task SearchAsync_returns_clarify_context_answer_for_broad_query_without_matches()
{
@@ -220,6 +313,74 @@ public sealed class UnifiedSearchServiceTests
answerFrame.SessionId.Should().Be(SearchAnalyticsPrivacy.HashSessionId("tenant-telemetry", "session-456"));
}
[Fact]
public async Task EvaluateSuggestionsAsync_returns_viability_without_recording_answer_frame_analytics()
{
var analyticsOptions = Options.Create(new KnowledgeSearchOptions
{
Enabled = true,
ConnectionString = "Host=localhost;Database=test"
});
var analyticsService = new SearchAnalyticsService(analyticsOptions, NullLogger<SearchAnalyticsService>.Instance);
var qualityMonitor = new SearchQualityMonitor(analyticsOptions, NullLogger<SearchQualityMonitor>.Instance, analyticsService);
var knowledgeRow = MakeRow(
"chunk-doctor-viability",
"doctor_check",
"PostgreSQL connectivity",
JsonDocument.Parse("{\"domain\":\"knowledge\",\"checkCode\":\"check.core.db.connectivity\"}"));
var storeMock = new Mock<IKnowledgeSearchStore>();
storeMock.Setup(s => s.SearchFtsAsync(
"database connectivity", It.IsAny<KnowledgeSearchFilter?>(), It.IsAny<int>(),
It.IsAny<TimeSpan>(), It.IsAny<CancellationToken>(), It.IsAny<string?>()))
.ReturnsAsync(new List<KnowledgeChunkRow> { knowledgeRow });
storeMock.Setup(s => s.SearchFtsAsync(
It.Is<string>(query => query != "database connectivity"), It.IsAny<KnowledgeSearchFilter?>(), It.IsAny<int>(),
It.IsAny<TimeSpan>(), It.IsAny<CancellationToken>(), It.IsAny<string?>()))
.ReturnsAsync([]);
storeMock.Setup(s => s.SearchFuzzyAsync(
It.IsAny<string>(), It.IsAny<KnowledgeSearchFilter?>(), It.IsAny<int>(),
It.IsAny<double>(), It.IsAny<TimeSpan>(), It.IsAny<CancellationToken>()))
.ReturnsAsync([]);
storeMock.Setup(s => s.LoadVectorCandidatesAsync(
It.IsAny<float[]>(), It.IsAny<KnowledgeSearchFilter?>(), It.IsAny<int>(),
It.IsAny<TimeSpan>(), It.IsAny<CancellationToken>()))
.ReturnsAsync([]);
var service = CreateService(
storeMock: storeMock,
analyticsService: analyticsService,
qualityMonitor: qualityMonitor);
var result = await service.EvaluateSuggestionsAsync(
new SearchSuggestionViabilityRequest(
Queries:
[
"database connectivity",
"manually compare unavailable evidence across environments"
],
Ambient: new AmbientContext
{
CurrentRoute = "/ops/operations/doctor"
}),
CancellationToken.None);
result.Suggestions.Should().HaveCount(2);
result.Suggestions.Should().Contain(suggestion =>
suggestion.Query == "database connectivity"
&& suggestion.Viable
&& suggestion.Status == "grounded");
result.Suggestions.Should().Contain(suggestion =>
suggestion.Query == "manually compare unavailable evidence across environments"
&& !suggestion.Viable
&& suggestion.Status == "insufficient");
result.Coverage.CurrentScopeDomain.Should().Be("knowledge");
var events = analyticsService.GetFallbackEventsSnapshot("global", TimeSpan.FromMinutes(1));
events.Should().BeEmpty();
}
[Fact]
public async Task SearchAsync_returns_empty_when_tenant_feature_flag_disables_search()
{