Add grounded unified search answers and live verification

This commit is contained in:
master
2026-03-07 03:55:51 +02:00
parent 2ff0e1f86b
commit edb947d602
19 changed files with 1180 additions and 32 deletions

View File

@@ -572,6 +572,10 @@ stella advisoryai index rebuild --json
Generate deterministic AdvisoryAI Knowledge Search (AKS) source artifacts used by index rebuild.
Doctor controls output is enriched from configured seed plus locally discovered Doctor checks (when Doctor engine services are available), providing fallback metadata for AdvisoryAI ingestion.
Requirements:
- In a source checkout, do not assume `stella` is already installed on `PATH`; build or run it from source first.
- Set `STELLAOPS_BACKEND_URL` (or the equivalent CLI config file value) when the command needs live Doctor check discovery.
### Synopsis
```bash
@@ -594,6 +598,7 @@ stella advisoryai sources prepare [options]
### Examples
```bash
export STELLAOPS_BACKEND_URL="http://127.0.0.1:10451"
stella advisoryai sources prepare --json
stella advisoryai sources prepare --repo-root . --openapi-output devops/compose/openapi_current.json --overwrite
```

View File

@@ -149,21 +149,25 @@ Completion criteria:
| 2026-03-06 | Added typed chip-context registry contract (`search-context.registry.ts`) and shifted suggestion selection to route-context arrays + bounded last-few-action prioritization + deterministic rotation. | Developer (FE) |
| 2026-03-06 | Synced architecture docs for automatic page-open suggestions and ambient `lastAction` contract: `docs/modules/ui/architecture.md`, `docs/modules/advisory-ai/knowledge-search.md`, `docs/modules/advisory-ai/unified-search-architecture.md`. | Documentation author |
| 2026-03-06 | Added UI governance rule for chip ownership and page-context interface in `docs/modules/ui/search-chip-context-contract.md`. | Documentation author |
| 2026-03-06 | Added exhaustive Playwright query-matrix suite (`tests/e2e/unified-search-exhaustive-matrix.e2e.spec.ts`) generating 1200 deterministic query types with end-to-end UI execution and >=99.5% success gate. | Test Automation |
| 2026-03-06 | Added exhaustive Playwright query-matrix suite (`tests/e2e/unified-search-exhaustive-matrix.e2e.spec.ts`) generating 1200 deterministic query types split into 6 strict 200-query batches; verified 100% success across all batches. | Test Automation |
| 2026-03-06 | Migrated default hostname from 127.1.0.1 to stella-ops.local across envsettings-override, proxy.conf, playwright config, perf fixtures, README, and smoke scripts. | Developer (FE) |
| 2026-03-06 | QA iteration: 23/23 sprint unit tests pass (ambient-context 6, global-search 11, chat-message 6). Live behavioral verification via Playwright confirms contextual placeholders and suggestion chips adapt per page (dashboard/triage/policy/scanning). OIDC login flow works end-to-end at stella-ops.local. | QA |
| 2026-03-07 | Added ingestion-backed contextual suggestion verification for the Doctor route: local rebuild order (`/v1/advisory-ai/index/rebuild` then `/v1/search/index/rebuild`) was exercised and `unified-search-contextual-suggestions.live.e2e.spec.ts` proved page-open chips and chip-triggered search over real ingested search data. | Test Automation |
## Decisions & Risks
- Decision needed: whether route context should remain a hard domain filter in FE (`buildContextFilter`) or become a soft ranking hint only via ambient payload.
- Decision needed: final schema for `lastAction` ambient metadata and retention policy in FE memory/session scope.
- Decision: FE emits `ambient.lastAction` now as a forward-compatible field; current backend deployments may ignore it without regressing behavior.
- Decision: chip definitions are now governed by typed context arrays (`SEARCH_CONTEXT_DEFINITIONS`) and an explicit page-level interface contract (`SearchContextComponent`) instead of ad-hoc route conditionals.
- Decision: exhaustive >1000 query E2E coverage uses a deterministic matrix (1200 queries) with a reliability threshold (`>=99.5%`) to avoid false-red from occasional debounce drop events while still failing on meaningful regressions.
- Decision: exhaustive >1000 query E2E coverage now runs as 6 strict batches of 200 queries each, resetting page state between batches and enforcing 100% per-batch success.
- Decision: contextual suggestion verification now includes one live-ingested route (Doctor/knowledge) in addition to mock-backed regression suites.
- Docs updated: `docs/modules/ui/architecture.md`, `docs/modules/ui/search-chip-context-contract.md`, `docs/modules/advisory-ai/knowledge-search.md`, `docs/modules/advisory-ai/unified-search-architecture.md`.
- Risk: stale action context may bias suggestions toward irrelevant domains.
- Mitigation: TTL + bounded history + explicit reset on session boundaries.
- Risk: route-prefix drift between FE and backend route-domain maps can silently reduce context quality.
- Mitigation: shared route mapping tests and explicit parity checks.
- Risk: mocked suggestion suites can miss ingestion/corpus regressions.
- Mitigation: keep a live-ingested Playwright lane and record rebuild/query evidence in the search sprints.
- Risk: privacy leakage if raw action labels/queries are persisted beyond current controls.
- Mitigation: preserve hashed analytics and limit persisted raw content to existing approved history paths only.

View File

@@ -74,11 +74,12 @@ Owners: Test Automation, QA, Developer (FE)
Task description:
- Add targeted unit coverage for answer-state selection, question generation, and fallback behavior.
- Add Playwright coverage for grounded-answer, clarification, and no-evidence journeys end to end.
- Keep this phase deterministic and frontend-owned; ingestion-backed live verification is handled explicitly in backend and rollout sprints.
Completion criteria:
- [x] Unit coverage validates answer-state selection and contextual question generation.
- [x] Playwright covers grounded-answer, clarify, and fallback flows.
- [x] Tests remain deterministic with route mocks and no live network dependencies.
- [x] Tests remain deterministic with route mocks and no live network dependencies for the FE shell.
### FE-SELF-005 - Docs sync and rollout guidance
Status: DONE
@@ -104,6 +105,7 @@ Completion criteria:
## Decisions & Risks
- Decision: phase 1 is frontend-composed on top of existing unified search payloads so the product can ship a self-serve shell immediately.
- Decision: ingestion-backed verification is mandatory, but it is owned by the backend answer-orchestration and FE rollout sprints rather than this FE shell sprint.
- Decision: page ownership must live in the shared search context registry, not in ad hoc component conditionals.
- Decision: every non-empty search must render a visible answer state even when the answer is only a clarification request or insufficient-evidence message.
- Decision: self-serve questions are governed by `docs/modules/ui/search-self-serve-contract.md`, while contextual chips remain governed by `docs/modules/ui/search-chip-context-contract.md`.

View File

@@ -4,8 +4,9 @@
- Move self-serve search from frontend-composed answers to backend-grounded contextual answers with explicit citations, fallback reasons, and follow-up questions.
- Establish a unified answer contract that search and AdvisoryAI can both consume.
- Add telemetry for unanswered and reformulated journeys so self-serve gaps become measurable backlog items instead of anecdotal feedback.
- Prove the answer contract against a locally ingested corpus, not only stubbed endpoint tests.
- Working directory: `src/AdvisoryAI`.
- Expected evidence: targeted integration tests against the AdvisoryAI test project, updated API/docs, and execution-log entries with command evidence.
- Expected evidence: targeted integration tests against the AdvisoryAI test project, ingestion-backed local rebuild/query evidence, updated API/docs, and execution-log entries with command evidence.
## Dependencies & Concurrency
- Depends on `docs/implplan/SPRINT_20260307_004_FE_self_serve_search_answer_first.md` for the FE shell and contract expectations.
@@ -22,7 +23,7 @@
## Delivery Tracker
### AI-SELF-001 - Unified contextual answer payload
Status: TODO
Status: DONE
Dependency: none
Owners: Developer (AdvisoryAI)
Task description:
@@ -35,7 +36,7 @@ Completion criteria:
- [ ] API docs describe the new answer payload.
### AI-SELF-002 - Grounding and fallback policy
Status: TODO
Status: DONE
Dependency: AI-SELF-001
Owners: Developer (AdvisoryAI), Product Manager
Task description:
@@ -51,7 +52,7 @@ Completion criteria:
- [ ] Synthesis cannot silently masquerade as a grounded answer without sufficient evidence.
### AI-SELF-003 - Follow-up question and clarification generation
Status: TODO
Status: DONE
Dependency: AI-SELF-001
Owners: Developer (AdvisoryAI)
Task description:
@@ -77,7 +78,7 @@ Completion criteria:
- [ ] Tests cover telemetry emission for fallback paths.
### AI-SELF-005 - Targeted behavioral verification
Status: TODO
Status: DONE
Dependency: AI-SELF-003
Owners: Test Automation, QA
Task description:
@@ -88,20 +89,41 @@ Completion criteria:
- [ ] Assertions verify actual answer-state payload content, not only success status codes.
- [ ] Execution log records exact commands and outcomes.
### AI-SELF-006 - Ingestion-backed live corpus verification
Status: DONE
Dependency: AI-SELF-001
Owners: Developer (AdvisoryAI), Test Automation
Task description:
- Rebuild AdvisoryAI and unified-search indexes from the local corpus and verify that contextual answers are returned over real ingested data, not only test doubles.
- Document the exact rebuild order, required local setup, and the query paths currently covered by live verification.
Completion criteria:
- [ ] Local rebuild order is explicit and exercised: `/v1/advisory-ai/index/rebuild` then `/v1/search/index/rebuild`.
- [ ] At least one ingestion-backed query path returns a contextual answer payload from the running local service.
- [ ] Docs and sprint log state which live routes are verified today and which routes still rely on mocks.
## Execution Log
| Date (UTC) | Update | Owner |
| --- | --- | --- |
| 2026-03-07 | Sprint created to formalize backend-grounded contextual answers after the FE answer-first shell. | Project Manager |
| 2026-03-07 | Added explicit ingestion-backed verification scope so backend answer orchestration must be validated against a rebuilt local corpus instead of only stubbed endpoint tests. | Project Manager |
| 2026-03-07 | Implemented `contextAnswer` in unified search/backend API mapping, added deterministic `grounded` / `clarify` / `insufficient` rules plus follow-up question generation, and extended telemetry fields for answer-state visibility. | Developer |
| 2026-03-07 | Verified the AdvisoryAI test project after the contract change with `dotnet test "src/AdvisoryAI/__Tests/StellaOps.AdvisoryAI.Tests/StellaOps.AdvisoryAI.Tests.csproj" --no-restore -v normal` (`877/877` passing). | Test Automation |
| 2026-03-07 | Exercised the live rebuilt-corpus lane against `http://127.0.0.1:10451`: `POST /v1/advisory-ai/index/rebuild`, `POST /v1/search/index/rebuild`, then `POST /v1/search/query` for `database connectivity`, which returned `contextAnswer.status = grounded`, 3 citations, and 10 cards over ingested data. | Test Automation |
## Decisions & Risks
- Decision: the backend contract must return explicit answer states instead of leaving the UI to infer confidence from cards alone.
- Decision: the product requirement is 100% response framing, not 100% hallucinated AI answers.
- Decision: answer orchestration is not considered verified until it passes both targeted `.csproj` integration tests and at least one live query over a rebuilt local corpus.
- Risk: answer payload inflation could make the endpoint harder to evolve if fields are not clearly optional.
- Mitigation: use additive optional fields with strict docs and integration coverage.
- Risk: telemetry may leak raw user intent if not hashed or summarized carefully.
- Mitigation: follow existing hashed-query analytics patterns and avoid raw prompt persistence where not required for history UX.
- Risk: mocked endpoint tests can overstate confidence if ingestion adapters or corpus rebuild order drift.
- Mitigation: keep rebuild order documented, execute it during verification, and record which routes have live-ingested parity.
- Decision: `stella advisoryai sources prepare` is optional for local verification when checked-in Doctor seed/control files are already sufficient, but it requires `STELLAOPS_BACKEND_URL` whenever live Doctor discovery is expected.
## Next Checkpoints
- 2026-03-10: Freeze answer payload shape and fallback taxonomy.
- 2026-03-12: Complete endpoint integration tests for answer states.
- 2026-03-13: Hand off payload contract to FE rollout sprint.
- 2026-03-13: Hand off payload contract and live-ingested verification notes to FE rollout sprint.

View File

@@ -4,8 +4,9 @@
- Roll the self-serve search contract across priority pages so the experience is consistent wherever operators land.
- Close the gap between search, AdvisoryAI, and page workflows by wiring guided actions and telemetry-informed improvements.
- Convert unanswered journeys into visible UX backlog items and stronger end-to-end coverage.
- Pair deterministic mock coverage with explicit live-ingested verification for routes that already have local corpus parity.
- Working directory: `src/Web/StellaOps.Web`.
- Expected evidence: page-contract rollouts, Playwright operator-journey suites, updated docs/task boards, and execution-log entries.
- Expected evidence: page-contract rollouts, Playwright operator-journey suites, live-ingested verification where supported, updated docs/task boards, and execution-log entries.
## Dependencies & Concurrency
- Depends on:
@@ -76,7 +77,20 @@ Task description:
Completion criteria:
- [ ] Playwright covers the priority page journeys end to end.
- [ ] Tests verify grounded, clarify, and recovery paths.
- [ ] Suites remain deterministic with route mocks and local fixtures only.
- [ ] Suites remain deterministic with route mocks/local fixtures for routes that do not yet have live corpus parity.
### FE-ROLL-006 - Live ingested-corpus search verification
Status: DONE
Dependency: FE-ROLL-004
Owners: Test Automation, QA, Developer (FE)
Task description:
- Run Playwright against at least one route backed by a real locally rebuilt AdvisoryAI corpus and verify that suggestions, answer framing, and follow-up handoffs work without route stubs for search data.
- Track route parity explicitly so teams know which pages are mock-only and which already have live corpus validation.
Completion criteria:
- [ ] At least one priority route executes search against a real ingested local corpus in Playwright.
- [ ] Docs call out live-verified routes versus mock-backed routes.
- [ ] Live verification failures feed the rollout gap backlog instead of being hidden behind route mocks.
### FE-ROLL-005 - Docs and rollout readiness
Status: TODO
@@ -94,14 +108,19 @@ Completion criteria:
| Date (UTC) | Update | Owner |
| --- | --- | --- |
| 2026-03-07 | Sprint created to roll out the self-serve search contract after the answer-first shell and backend answer contract are defined. | Project Manager |
| 2026-03-07 | Added explicit live-ingested verification scope so rollout evidence distinguishes mock-backed journeys from real corpus coverage. | Project Manager |
| 2026-03-07 | Re-ran live Playwright verification for the Doctor route against a rebuilt local AdvisoryAI corpus (`unified-search-contextual-suggestions.live.e2e.spec.ts`) and confirmed automatic chips, grounded answer framing, and follow-up chips over real search data. | Test Automation |
## Decisions & Risks
- Decision: rollout should prioritize high-frequency operator pages before broad route coverage.
- Decision: page teams own their self-serve question sets; platform code owns composition and safety rails.
- Decision: mock-backed Playwright remains acceptable for pages without live corpus parity, but rollout is incomplete until at least one priority route is verified against a rebuilt local corpus.
- Risk: inconsistent page adoption would make self-serve feel random across the product.
- Mitigation: maintain a priority rollout list and explicit page-ownership rules.
- Risk: adding guided actions without verification can create shallow or broken handoffs.
- Mitigation: require Playwright coverage for each high-value journey.
- Risk: live-ingested routes can drift from mocked expectations as ingestion adapters evolve.
- Mitigation: document route parity and keep at least one live route in the regular regression pack.
## Next Checkpoints
- 2026-03-12: Priority page rollout started after answer contract freeze.

View File

@@ -124,6 +124,8 @@ Implemented in `src/AdvisoryAI/StellaOps.AdvisoryAI/KnowledgeSearch/KnowledgeSea
- Contract remains backward-compatible: if an API deployment does not yet consume `lastAction`, unknown ambient fields are ignored and base search behavior remains unchanged.
- UI suggestion behavior now combines obvious route defaults with one strategic non-obvious suggestion and action-aware variants (for example, policy/VEX impact and incident timeline pivots).
- Search and AdvisoryAI also share a persisted operator mode (`Find`, `Explain`, `Act`); the UI uses the same mode to rank chips, compose Ask-AI prompts, and label assistant return flows, while backend query contracts remain backward-compatible.
- Unified search now also returns optional `contextAnswer` metadata with `status`, `code`, `summary`, `reason`, `evidence`, bounded `citations`, and bounded follow-up `questions`.
- `contextAnswer.status` is deterministic and must be one of `grounded`, `clarify`, or `insufficient`.
- Unified index lifecycle:
- Manual rebuild endpoint: `POST /v1/search/index/rebuild`.
- Optional background refresh loop is available via `KnowledgeSearchOptions` (`UnifiedAutoIndexEnabled`, `UnifiedAutoIndexOnStartup`, `UnifiedIndexRefreshIntervalSeconds`).
@@ -172,7 +174,9 @@ Global search now consumes AKS and supports:
- Doctor: `Run` (navigate to doctor and copy run command).
- `More` action for "show more like this" local query expansion.
- A shared mode switch (`Find`, `Explain`, `Act`) across search and AdvisoryAI with mode-aware chip ranking and handoff prompts.
- An answer-first FE shell: every non-empty search renders a visible answer state (`grounded`, `clarify`, `insufficient`) before raw cards, using existing synthesis/cards plus page context until a backend `contextAnswer` payload is introduced.
- An answer-first search experience: every non-empty search renders a visible answer state (`grounded`, `clarify`, `insufficient`) before raw cards.
- Preferred source is backend `contextAnswer`.
- FE shell composition remains only as a backward-compatible fallback for older API deployments that do not emit `contextAnswer`.
- Page-owned self-serve questions and clarifiers, defined in `docs/modules/ui/search-self-serve-contract.md`, so search can offer "Common questions" and recovery prompts without per-page conditionals in the component.
- Zero-result rescue actions that keep the current query visible while broadening scope, trying a related pivot, retrying with page context, or opening AdvisoryAI reformulation.
- AdvisoryAI evidence-first next-step cards that can return search pivots (`chat_next_step_search`, `chat_next_step_policy`) back into global search or open cited evidence/context directly.
@@ -191,6 +195,10 @@ AKS commands:
- `POST /v1/search/synthesize`
- `POST /v1/search/index/rebuild`
Notes:
- Do not assume `stella` is already installed on `PATH` in a source checkout. Build or run it from source as described in `docs/API_CLI_REFERENCE.md` and `docs/modules/cli/guides/quickstart.md`.
- `stella advisoryai sources prepare` needs `STELLAOPS_BACKEND_URL` (or equivalent CLI config) when live Doctor check discovery is expected. Without that URL, use the checked-in Doctor seed/control files and the HTTP rebuild endpoints for local verification.
Output:
- Human mode: grouped actionable references.
- JSON mode: stable machine-readable payload.
@@ -318,6 +326,7 @@ export ADVISORYAI__AdvisoryAI__KnowledgeSearch__RepositoryRoot="$(pwd)"
dotnet run --project "src/AdvisoryAI/StellaOps.AdvisoryAI.WebService/StellaOps.AdvisoryAI.WebService.csproj" --no-launch-profile
# In a second shell, rebuild the live corpus in the required order
export STELLAOPS_BACKEND_URL="http://127.0.0.1:10451"
dotnet run --project "src/Cli/StellaOps.Cli/StellaOps.Cli.csproj" -- advisoryai sources prepare --json
dotnet run --project "src/Cli/StellaOps.Cli/StellaOps.Cli.csproj" -- advisoryai index rebuild --json
curl -X POST http://127.0.0.1:10451/v1/search/index/rebuild \
@@ -342,6 +351,7 @@ Local examples:
```bash
# Run directly from source without installing to PATH
export STELLAOPS_BACKEND_URL="http://127.0.0.1:10451"
dotnet run --project "src/Cli/StellaOps.Cli/StellaOps.Cli.csproj" -- advisoryai sources prepare --json
# Publish a reusable local binary
@@ -358,6 +368,12 @@ If the CLI is not built yet, the equivalent HTTP endpoints are:
- `POST /v1/advisory-ai/index/rebuild` for the docs/OpenAPI/Doctor corpus
- `POST /v1/search/index/rebuild` for unified overlay domains
Current live verification coverage:
- Rebuild order exercised against a running local service: `POST /v1/advisory-ai/index/rebuild` then `POST /v1/search/index/rebuild`
- Verified live query: `database connectivity`
- Verified live outcome: response includes `contextAnswer.status = grounded`, citations, and entity cards over ingested data
- Other routes still rely on deterministic mock-backed Playwright coverage until their ingestion parity is explicitly verified
Or use the full CI testing stack:
```bash
docker compose -f devops/compose/docker-compose.testing.yml --profile ci up -d

View File

@@ -56,6 +56,12 @@ flowchart LR
- grounding score
- action suggestions
- If LLM is unavailable or blocked by quota, deterministic output is still returned.
- Query responses may also include a deterministic `contextAnswer` envelope for answer-first search UX:
- `status`: `grounded` | `clarify` | `insufficient`
- `code`, `summary`, `reason`, `evidence`
- bounded `citations`
- bounded follow-up `questions`
- The answer envelope is additive and optional so older clients remain compatible.
## Data Flow
@@ -95,6 +101,13 @@ sequenceDiagram
- `POST /v1/search/synthesize`
- `POST /v1/search/index/rebuild`
`POST /v1/search/query` response notes:
- Entity cards remain the primary retrieval payload.
- `contextAnswer` is the preferred answer-first surface for Web self-serve UX when present.
- Live local verification currently covers the Doctor/knowledge path after the documented rebuild order:
1. `POST /v1/advisory-ai/index/rebuild`
2. `POST /v1/search/index/rebuild`
OpenAPI contract presence is validated by integration test:
- `UnifiedSearchEndpointsIntegrationTests.OpenApi_Includes_UnifiedSearch_Contracts`

View File

@@ -379,6 +379,18 @@ public static class UnifiedSearchEndpoints
CurrentRoute = string.IsNullOrWhiteSpace(ambient.CurrentRoute) ? null : ambient.CurrentRoute.Trim(),
SessionId = string.IsNullOrWhiteSpace(ambient.SessionId) ? null : ambient.SessionId.Trim(),
ResetSession = ambient.ResetSession,
LastAction = ambient.LastAction is null || string.IsNullOrWhiteSpace(ambient.LastAction.Action)
? null
: new AmbientAction
{
Action = ambient.LastAction.Action.Trim(),
Source = string.IsNullOrWhiteSpace(ambient.LastAction.Source) ? null : ambient.LastAction.Source.Trim(),
QueryHint = string.IsNullOrWhiteSpace(ambient.LastAction.QueryHint) ? null : ambient.LastAction.QueryHint.Trim(),
Domain = string.IsNullOrWhiteSpace(ambient.LastAction.Domain) ? null : ambient.LastAction.Domain.Trim().ToLowerInvariant(),
EntityKey = string.IsNullOrWhiteSpace(ambient.LastAction.EntityKey) ? null : ambient.LastAction.EntityKey.Trim(),
Route = string.IsNullOrWhiteSpace(ambient.LastAction.Route) ? null : ambient.LastAction.Route.Trim(),
OccurredAt = ambient.LastAction.OccurredAt
},
VisibleEntityKeys = ambient.VisibleEntityKeys is { Count: > 0 }
? ambient.VisibleEntityKeys
.Where(static value => !string.IsNullOrWhiteSpace(value))
@@ -462,6 +474,31 @@ public static class UnifiedSearchEndpoints
}).ToArray();
}
UnifiedSearchApiContextAnswer? contextAnswer = null;
if (response.ContextAnswer is not null)
{
contextAnswer = new UnifiedSearchApiContextAnswer
{
Status = response.ContextAnswer.Status,
Code = response.ContextAnswer.Code,
Summary = response.ContextAnswer.Summary,
Reason = response.ContextAnswer.Reason,
Evidence = response.ContextAnswer.Evidence,
Citations = response.ContextAnswer.Citations?.Select(static citation => new UnifiedSearchApiContextAnswerCitation
{
EntityKey = citation.EntityKey,
Title = citation.Title,
Domain = citation.Domain,
Route = citation.Route
}).ToArray(),
Questions = response.ContextAnswer.Questions?.Select(static question => new UnifiedSearchApiContextAnswerQuestion
{
Query = question.Query,
Kind = question.Kind
}).ToArray()
};
}
return new UnifiedSearchApiResponse
{
Query = response.Query,
@@ -470,6 +507,7 @@ public static class UnifiedSearchEndpoints
Synthesis = synthesis,
Suggestions = suggestions,
Refinements = refinements,
ContextAnswer = contextAnswer,
Diagnostics = new UnifiedSearchApiDiagnostics
{
FtsMatches = response.Diagnostics.FtsMatches,
@@ -696,6 +734,25 @@ public sealed record UnifiedSearchApiAmbientContext
public string? SessionId { get; init; }
public bool ResetSession { get; init; }
public UnifiedSearchApiAmbientAction? LastAction { get; init; }
}
public sealed record UnifiedSearchApiAmbientAction
{
public string Action { get; init; } = string.Empty;
public string? Source { get; init; }
public string? QueryHint { get; init; }
public string? Domain { get; init; }
public string? EntityKey { get; init; }
public string? Route { get; init; }
public DateTimeOffset? OccurredAt { get; init; }
}
public sealed record UnifiedSearchApiFilter
@@ -751,6 +808,8 @@ public sealed record UnifiedSearchApiResponse
public IReadOnlyList<UnifiedSearchApiRefinement>? Refinements { get; init; }
public UnifiedSearchApiContextAnswer? ContextAnswer { get; init; }
public UnifiedSearchApiDiagnostics Diagnostics { get; init; } = new();
public IReadOnlyList<UnifiedSearchApiFederationDiagnostic>? Federation { get; init; }
@@ -839,6 +898,41 @@ public sealed record UnifiedSearchApiRefinement
public string Source { get; init; } = string.Empty;
}
public sealed record UnifiedSearchApiContextAnswer
{
public string Status { get; init; } = string.Empty;
public string Code { get; init; } = string.Empty;
public string Summary { get; init; } = string.Empty;
public string Reason { get; init; } = string.Empty;
public string Evidence { get; init; } = string.Empty;
public IReadOnlyList<UnifiedSearchApiContextAnswerCitation>? Citations { get; init; }
public IReadOnlyList<UnifiedSearchApiContextAnswerQuestion>? Questions { get; init; }
}
public sealed record UnifiedSearchApiContextAnswerCitation
{
public string EntityKey { get; init; } = string.Empty;
public string Title { get; init; } = string.Empty;
public string Domain { get; init; } = string.Empty;
public string? Route { get; init; }
}
public sealed record UnifiedSearchApiContextAnswerQuestion
{
public string Query { get; init; } = string.Empty;
public string Kind { get; init; } = "follow_up";
}
public sealed record UnifiedSearchApiDiagnostics
{
public int FtsMatches { get; init; }

View File

@@ -16,3 +16,5 @@ Source of truth: `docs/implplan/SPRINT_20260130_002_Tools_csproj_remediation_sol
| SPRINT_20260224_003-LOC-202 | DONE | `SPRINT_20260224_003_AdvisoryAI_translation_rollout_remaining_phases.md`: phase-3.4 AdvisoryAI slice completed (remote bundle wiring, localized validation keys in search/unified-search endpoints, `en-US`+`de-DE` service bundles, and de-DE integration coverage). |
| SPRINT_20260224_G1-G10 | DONE | Search improvement sprints G1G10 implemented. New endpoints: `SearchAnalyticsEndpoints.cs` (history, events, popularity), `SearchFeedbackEndpoints.cs` (feedback, quality alerts, metrics). Extended: `UnifiedSearchEndpoints.cs` (suggestions, refinements, previews, diagnostics.activeEncoder). Extended: `KnowledgeSearchEndpoints.cs` (activeEncoder in diagnostics). See `docs/modules/advisory-ai/knowledge-search.md` for full testing guide. |
| AI-SELF-001 | DONE | Unified search endpoint contract now exposes backend contextual answer fields for self-serve search. |
| AI-SELF-006 | DONE | Endpoint readiness now includes a proven local rebuilt-corpus verification lane in addition to stubbed integration tests. |

View File

@@ -10,6 +10,11 @@ Source of truth: `docs/implplan/SPRINT_20260113_005_ADVISORYAI_controlled_conver
| SPRINT_20260224_102-G1-005 | DONE | ONNX missing-model fallback integration evidence added (`G1_OnnxEncoderSelection_MissingModelPath_FallsBackToDeterministicHashEncoder`). |
| SPRINT_20260224_102-G1-004 | DONE | Semantic recall benchmark corpus and assertions complete (48 queries; no exact-term regression; semantic recall uplift proven). |
| SPRINT_20260224_102-G1-001 | DOING | ONNX runtime package + license docs completed; model asset provisioning at `models/all-MiniLM-L6-v2.onnx` still pending deployment packaging. |
| AI-SELF-001 | DONE | Unified search now emits the contextual answer payload (`contextAnswer`) with answer state, citations, and follow-up questions for self-serve search. |
| AI-SELF-002 | DONE | Deterministic grounded/clarify/insufficient fallback policy is implemented in unified search orchestration. |
| AI-SELF-003 | DONE | Follow-up question generation from route/domain intent, recent actions, and evidence is implemented. |
| AI-SELF-004 | TODO | Telemetry for unanswered and reformulated self-serve journeys is still pending. |
| AI-SELF-006 | DONE | Live ingestion-backed answer verification succeeded on the Doctor/knowledge route after local rebuild. |
| SPRINT_20260222_051-AKS-INGEST | DONE | Added deterministic AKS ingestion controls: markdown allow-list manifest loading, OpenAPI aggregate source path support, and doctor control projection integration for search chunks, including fallback doctor metadata hydration from controls projection fields. |
| AUDIT-0017-M | DONE | Maintainability audit for StellaOps.AdvisoryAI. |
| AUDIT-0017-T | DONE | Test coverage audit for StellaOps.AdvisoryAI. |

View File

@@ -2,17 +2,29 @@ namespace StellaOps.AdvisoryAI.UnifiedSearch.Context;
internal sealed class AmbientContextProcessor
{
private const double CurrentRouteBoost = 0.10d;
private const double LastActionDomainBoost = 0.05d;
private const double VisibleEntityBoost = 0.20d;
private const double LastActionEntityBoost = 0.25d;
private static readonly (string Prefix, string Domain)[] RouteDomainMappings =
[
("/console/findings", "findings"),
("/security/triage", "findings"),
("/security/findings", "findings"),
("/security/advisories-vex", "vex"),
("/ops/policies", "policy"),
("/ops/policy", "policy"),
("/ops/graph", "graph"),
("/security/reach", "graph"),
("/ops/audit", "timeline"),
("/ops/timeline", "timeline"),
("/audit", "timeline"),
("/console/scans", "scanner"),
("/ops/operations/jobs", "opsmemory"),
("/ops/operations/scheduler", "opsmemory"),
("/ops/doctor", "knowledge"),
("/ops/operations/doctor", "knowledge"),
("/ops/operations/system-health", "knowledge"),
("/docs", "knowledge")
];
@@ -23,18 +35,26 @@ internal sealed class AmbientContextProcessor
var output = new Dictionary<string, double>(baseWeights, StringComparer.OrdinalIgnoreCase);
if (ambient is null || string.IsNullOrWhiteSpace(ambient.CurrentRoute))
{
var actionOnlyDomain = ResolveDomainFromAmbientAction(ambient?.LastAction);
if (!string.IsNullOrWhiteSpace(actionOnlyDomain))
{
ApplyBoost(output, actionOnlyDomain, LastActionDomainBoost);
}
return output;
}
var domain = ResolveDomainFromRoute(ambient.CurrentRoute);
if (string.IsNullOrWhiteSpace(domain))
if (!string.IsNullOrWhiteSpace(domain))
{
return output;
ApplyBoost(output, domain, CurrentRouteBoost);
}
output[domain] = output.TryGetValue(domain, out var existing)
? existing + 0.10d
: 1.10d;
var actionDomain = ResolveDomainFromAmbientAction(ambient.LastAction);
if (!string.IsNullOrWhiteSpace(actionDomain))
{
ApplyBoost(output, actionDomain, LastActionDomainBoost);
}
return output;
}
@@ -54,12 +74,21 @@ internal sealed class AmbientContextProcessor
continue;
}
map[entityKey.Trim()] = Math.Max(
map.TryGetValue(entityKey.Trim(), out var existing) ? existing : 0d,
0.20d);
var normalized = entityKey.Trim();
map[normalized] = Math.Max(
map.TryGetValue(normalized, out var existing) ? existing : 0d,
VisibleEntityBoost);
}
}
if (!string.IsNullOrWhiteSpace(ambient?.LastAction?.EntityKey))
{
var normalized = ambient.LastAction.EntityKey.Trim();
map[normalized] = Math.Max(
map.TryGetValue(normalized, out var existing) ? existing : 0d,
LastActionEntityBoost);
}
foreach (var entry in session.EntityBoosts)
{
map[entry.Key] = Math.Max(
@@ -120,5 +149,31 @@ internal sealed class AmbientContextProcessor
return null;
}
}
private static string? ResolveDomainFromAmbientAction(AmbientAction? action)
{
if (action is null)
{
return null;
}
if (!string.IsNullOrWhiteSpace(action.Domain))
{
return action.Domain.Trim().ToLowerInvariant();
}
if (!string.IsNullOrWhiteSpace(action.Route))
{
return ResolveDomainFromRoute(action.Route);
}
return null;
}
private static void ApplyBoost(IDictionary<string, double> output, string domain, double amount)
{
output[domain] = output.TryGetValue(domain, out var existing)
? existing + amount
: 1d + amount;
}
}

View File

@@ -69,12 +69,50 @@ public sealed record AmbientContext
public string? SessionId { get; init; }
public bool ResetSession { get; init; }
public AmbientAction? LastAction { get; init; }
}
public sealed record AmbientAction
{
public string Action { get; init; } = string.Empty;
public string? Source { get; init; }
public string? QueryHint { get; init; }
public string? Domain { get; init; }
public string? EntityKey { get; init; }
public string? Route { get; init; }
public DateTimeOffset? OccurredAt { get; init; }
}
public sealed record SearchSuggestion(string Text, string Reason);
public sealed record SearchRefinement(string Text, string Source);
public sealed record ContextAnswer(
string Status,
string Code,
string Summary,
string Reason,
string Evidence,
IReadOnlyList<ContextAnswerCitation>? Citations = null,
IReadOnlyList<ContextAnswerQuestion>? Questions = null);
public sealed record ContextAnswerCitation(
string EntityKey,
string Title,
string Domain,
string? Route = null);
public sealed record ContextAnswerQuestion(
string Query,
string Kind = "follow_up");
public sealed record UnifiedSearchResponse(
string Query,
int TopK,
@@ -82,7 +120,8 @@ public sealed record UnifiedSearchResponse(
SynthesisResult? Synthesis,
UnifiedSearchDiagnostics Diagnostics,
IReadOnlyList<SearchSuggestion>? Suggestions = null,
IReadOnlyList<SearchRefinement>? Refinements = null);
IReadOnlyList<SearchRefinement>? Refinements = null,
ContextAnswer? ContextAnswer = null);
public sealed record EntityCard
{

View File

@@ -18,6 +18,9 @@ namespace StellaOps.AdvisoryAI.UnifiedSearch;
internal sealed class UnifiedSearchService : IUnifiedSearchService
{
private const int MaxContextAnswerCitations = 3;
private const int MaxContextAnswerQuestions = 3;
private const int ClarifyTokenThreshold = 3;
private readonly KnowledgeSearchOptions _options;
private readonly UnifiedSearchOptions _unifiedOptions;
private readonly IKnowledgeSearchStore _store;
@@ -106,7 +109,7 @@ internal sealed class UnifiedSearchService : IUnifiedSearchService
if (query.Length > _unifiedOptions.MaxQueryLength)
{
return EmptyResponse(query, request.K, "query_too_long");
return EmptyResponse(query, request.K, "query_too_long", request.Ambient);
}
var tenantId = request.Filters?.Tenant ?? "global";
@@ -115,7 +118,7 @@ internal sealed class UnifiedSearchService : IUnifiedSearchService
if (!_options.Enabled || !IsSearchEnabledForTenant(tenantFlags) || string.IsNullOrWhiteSpace(_options.ConnectionString))
{
return EmptyResponse(query, request.K, "disabled");
return EmptyResponse(query, request.K, "disabled", request.Ambient);
}
if (request.Ambient?.ResetSession == true &&
@@ -292,6 +295,14 @@ internal sealed class UnifiedSearchService : IUnifiedSearchService
}
var duration = _timeProvider.GetUtcNow() - startedAt;
var contextAnswer = BuildContextAnswer(
query,
plan,
request.Ambient,
cards,
synthesis,
suggestions,
refinements);
var response = new UnifiedSearchResponse(
query,
topK,
@@ -307,7 +318,8 @@ internal sealed class UnifiedSearchService : IUnifiedSearchService
plan,
federationDiagnostics),
suggestions,
refinements);
refinements,
contextAnswer);
EmitTelemetry(plan, response, tenantId);
return response;
@@ -370,6 +382,489 @@ internal sealed class UnifiedSearchService : IUnifiedSearchService
};
}
private static IReadOnlyList<ContextAnswerCitation> BuildContextAnswerCitations(
IReadOnlyList<EntityCard> cards,
SynthesisResult? synthesis)
{
var citations = new List<ContextAnswerCitation>();
var seen = new HashSet<string>(StringComparer.OrdinalIgnoreCase);
if (synthesis?.Citations is { Count: > 0 })
{
foreach (var citation in synthesis.Citations)
{
var card = cards.FirstOrDefault(card =>
string.Equals(card.EntityKey, citation.EntityKey, StringComparison.OrdinalIgnoreCase));
if (card is null || !seen.Add(card.EntityKey))
{
continue;
}
citations.Add(new ContextAnswerCitation(
card.EntityKey,
string.IsNullOrWhiteSpace(card.Title) ? citation.Title : card.Title,
card.Domain,
GetPrimaryActionRoute(card)));
if (citations.Count >= MaxContextAnswerCitations)
{
return citations;
}
}
}
foreach (var card in cards)
{
if (!seen.Add(card.EntityKey))
{
continue;
}
citations.Add(new ContextAnswerCitation(
card.EntityKey,
card.Title,
card.Domain,
GetPrimaryActionRoute(card)));
if (citations.Count >= MaxContextAnswerCitations)
{
break;
}
}
return citations;
}
private static string BuildGroundedSummary(IReadOnlyList<EntityCard> cards, SynthesisResult? synthesis)
{
var synthesisSummary = synthesis?.Summary?.Trim();
if (!string.IsNullOrWhiteSpace(synthesisSummary))
{
return synthesisSummary;
}
var topCard = cards[0];
var snippet = topCard.Snippet?.Trim();
return string.IsNullOrWhiteSpace(snippet) ? topCard.Title : snippet;
}
private static string BuildGroundedReason(QueryPlan plan, AmbientContext? ambient, EntityCard topCard)
{
var scope = ResolveContextDomain(plan, [topCard], ambient) ?? topCard.Domain;
return $"The top result is grounded in {DescribeDomain(scope)} evidence and aligns with the {plan.Intent} intent.";
}
private static string BuildGroundedEvidence(IReadOnlyList<EntityCard> cards, SynthesisResult? synthesis)
{
var sourceCount = Math.Max(synthesis?.SourceCount ?? 0, cards.Count);
var domains = synthesis?.DomainsCovered is { Count: > 0 }
? synthesis.DomainsCovered
: cards.Select(static card => card.Domain)
.Where(static domain => !string.IsNullOrWhiteSpace(domain))
.Distinct(StringComparer.OrdinalIgnoreCase)
.ToArray();
return $"Grounded in {sourceCount} source(s) across {FormatDomainList(domains)}.";
}
private static bool ShouldClarifyQuery(
string query,
QueryPlan plan,
AmbientContext? ambient,
IReadOnlyList<SearchSuggestion>? suggestions,
IReadOnlyList<SearchRefinement>? refinements)
{
if (plan.DetectedEntities.Count > 0)
{
return false;
}
if (refinements is { Count: > 0 } || suggestions is { Count: > 0 })
{
return true;
}
var tokenCount = query.Split(
' ',
StringSplitOptions.RemoveEmptyEntries | StringSplitOptions.TrimEntries).Length;
if (tokenCount <= ClarifyTokenThreshold)
{
return true;
}
var contextDomain = ResolveContextDomain(plan, [], ambient);
if (!string.IsNullOrWhiteSpace(contextDomain) &&
(string.Equals(plan.Intent, "explore", StringComparison.OrdinalIgnoreCase) ||
string.Equals(plan.Intent, "navigate", StringComparison.OrdinalIgnoreCase)))
{
return true;
}
return false;
}
private static string BuildClarifySummary(string query, string contextDomain)
{
return $"\"{query}\" is too broad for {DescribeDomain(contextDomain)}. Narrow it to a specific target before we answer.";
}
private static string BuildClarifyReason(QueryPlan plan, AmbientContext? ambient)
{
var scope = ResolveContextDomain(plan, [], ambient) ?? "current scope";
return $"No grounded entity was retrieved, and the {plan.Intent} query needs a narrower target in {DescribeDomain(scope)}.";
}
private static string BuildClarifyEvidence(
IReadOnlyList<SearchSuggestion>? suggestions,
IReadOnlyList<SearchRefinement>? refinements,
string contextDomain)
{
var suggestionCount = suggestions?.Count ?? 0;
var refinementCount = refinements?.Count ?? 0;
if (suggestionCount > 0 || refinementCount > 0)
{
return $"{suggestionCount + refinementCount} bounded recovery hint(s) are available for {DescribeDomain(contextDomain)}.";
}
return $"No grounded evidence was retrieved yet from {DescribeDomain(contextDomain)}.";
}
private static IReadOnlyList<ContextAnswerQuestion> BuildGroundedQuestions(
string query,
QueryPlan plan,
AmbientContext? ambient,
EntityCard topCard)
{
var prompts = new List<ContextAnswerQuestion>();
var topTitle = topCard.Title.Trim();
foreach (var question in GetGroundedQuestionTemplates(topCard.Domain, topTitle))
{
prompts.Add(new ContextAnswerQuestion(question, "follow_up"));
}
if (!string.IsNullOrWhiteSpace(ambient?.LastAction?.QueryHint))
{
var lastHint = ambient.LastAction.QueryHint.Trim();
if (!lastHint.Equals(query, StringComparison.OrdinalIgnoreCase))
{
prompts.Add(new ContextAnswerQuestion(
$"How does {topTitle} relate to {lastHint}?",
"follow_up"));
}
}
return DistinctQuestions(prompts);
}
private static IReadOnlyList<ContextAnswerQuestion> BuildClarifyingQuestions(
string query,
QueryPlan? plan,
AmbientContext? ambient)
{
var prompts = new List<ContextAnswerQuestion>();
var domain = ResolveContextDomain(plan, [], ambient);
foreach (var question in GetClarifyingQuestionTemplates(domain))
{
prompts.Add(new ContextAnswerQuestion(question, "clarify"));
}
if (!string.IsNullOrWhiteSpace(ambient?.LastAction?.QueryHint))
{
var lastHint = ambient.LastAction.QueryHint.Trim();
if (!lastHint.Equals(query, StringComparison.OrdinalIgnoreCase))
{
prompts.Add(new ContextAnswerQuestion(
$"Do you want to continue from {lastHint}?",
"clarify"));
}
}
return DistinctQuestions(prompts);
}
private static IReadOnlyList<ContextAnswerQuestion> BuildRecoveryQuestions(
string query,
QueryPlan plan,
AmbientContext? ambient,
IReadOnlyList<SearchSuggestion>? suggestions,
IReadOnlyList<SearchRefinement>? refinements)
{
var prompts = new List<ContextAnswerQuestion>();
if (refinements is { Count: > 0 })
{
prompts.AddRange(refinements.Select(static refinement =>
new ContextAnswerQuestion(refinement.Text, "recover")));
}
if (suggestions is { Count: > 0 })
{
prompts.AddRange(suggestions.Select(static suggestion =>
new ContextAnswerQuestion(suggestion.Text, "recover")));
}
if (prompts.Count == 0)
{
var domain = ResolveContextDomain(plan, [], ambient);
foreach (var question in GetRecoveryQuestionTemplates(domain, query))
{
prompts.Add(new ContextAnswerQuestion(question, "recover"));
}
}
return DistinctQuestions(prompts);
}
private static IReadOnlyList<ContextAnswerQuestion> BuildFallbackQuestionsForDisabledSearch(AmbientContext? ambient)
{
var domain = ResolveContextDomain(plan: null, cards: [], ambient: ambient);
return DistinctQuestions(GetRecoveryQuestionTemplates(domain, "current search")
.Select(static question => new ContextAnswerQuestion(question, "recover"))
.ToList());
}
private static string BuildInsufficientSummary(string query, QueryPlan plan, AmbientContext? ambient)
{
var scope = ResolveContextDomain(plan, [], ambient) ?? "current scope";
return $"No grounded answer was found for \"{query}\" in {DescribeDomain(scope)}.";
}
private static string BuildInsufficientEvidence(
IReadOnlyList<SearchSuggestion>? suggestions,
IReadOnlyList<SearchRefinement>? refinements)
{
var suggestionCount = suggestions?.Count ?? 0;
var refinementCount = refinements?.Count ?? 0;
if (suggestionCount > 0 || refinementCount > 0)
{
return $"No grounded citations were found, but {suggestionCount + refinementCount} recovery hint(s) are available.";
}
return "No grounded citations, result cards, or bounded recovery hints were retrieved.";
}
private static IReadOnlyList<string> GetGroundedQuestionTemplates(string domain, string title)
{
return domain switch
{
"findings" =>
[
$"Why does {title} block release?",
$"What is the safest remediation for {title}?",
$"What evidence proves exploitability for {title}?"
],
"vex" =>
[
$"Why is {title} marked not affected?",
$"What evidence conflicts with {title}?",
$"Which components are covered by {title}?"
],
"policy" =>
[
$"Why is {title} failing?",
$"What findings are impacted by {title}?",
$"What is the safest exception path for {title}?"
],
"graph" =>
[
$"Which path makes {title} reachable?",
$"What is the blast radius of {title}?",
$"What should I inspect next from {title}?"
],
"timeline" =>
[
$"What changed before {title}?",
$"What else happened around {title}?",
$"Which release is most related to {title}?"
],
"opsmemory" =>
[
$"Have we seen {title} before?",
$"What runbook usually resolves {title}?",
$"What repeated failures are related to {title}?"
],
_ =>
[
$"How do I verify the fix for {title}?",
$"What changed before {title} failed?",
$"What should I inspect next for {title}?"
]
};
}
private static IReadOnlyList<string> GetClarifyingQuestionTemplates(string? domain)
{
return domain switch
{
"findings" =>
[
"Which CVE, workload, or package should I narrow this to?",
"Should I focus on reachable, production, or unresolved findings?"
],
"vex" =>
[
"Which statement, component, or product range should I narrow this to?",
"Do you need exploitability meaning, coverage, or conflict evidence?"
],
"policy" =>
[
"Which rule, environment, or control should I narrow this to?",
"Do you need recent failures, exceptions, or promotion impact?"
],
"graph" =>
[
"Which node, package, or edge should I narrow this to?",
"Do you want reachability, impact, or next-step guidance?"
],
"timeline" =>
[
"Which deployment, incident, or time window should I narrow this to?",
"Do you want causes, impacts, or follow-up events?"
],
"opsmemory" =>
[
"Which job, incident, or recurring failure should I narrow this to?",
"Do you want precedent, likely cause, or recommended recovery?"
],
_ =>
[
"Which check, component, or symptom should I narrow this to?",
"Do you want diagnosis, remediation, or verification steps?"
]
};
}
private static IReadOnlyList<string> GetRecoveryQuestionTemplates(string? domain, string query)
{
return domain switch
{
"findings" =>
[
$"reachable {query}",
$"unresolved {query}",
$"critical {query}"
],
"policy" =>
[
$"policy exceptions {query}",
$"failing policy gates {query}",
$"promotion impact {query}"
],
"timeline" =>
[
$"recent events {query}",
$"deployment history {query}",
$"incident timeline {query}"
],
_ =>
[
query,
$"strongest evidence {query}",
$"related {query}"
]
};
}
private static IReadOnlyList<ContextAnswerQuestion> DistinctQuestions(IReadOnlyList<ContextAnswerQuestion> questions)
{
var seen = new HashSet<string>(StringComparer.OrdinalIgnoreCase);
var output = new List<ContextAnswerQuestion>();
foreach (var question in questions)
{
if (string.IsNullOrWhiteSpace(question.Query) || !seen.Add(question.Query.Trim()))
{
continue;
}
output.Add(question with { Query = question.Query.Trim() });
if (output.Count >= MaxContextAnswerQuestions)
{
break;
}
}
return output;
}
private static string? ResolveContextDomain(
QueryPlan? plan,
IReadOnlyList<EntityCard> cards,
AmbientContext? ambient)
{
if (cards.Count > 0 && !string.IsNullOrWhiteSpace(cards[0].Domain))
{
return cards[0].Domain;
}
var routeDomain = AmbientContextProcessor.ResolveDomainFromRoute(ambient?.CurrentRoute ?? string.Empty);
if (!string.IsNullOrWhiteSpace(routeDomain))
{
return routeDomain;
}
var actionDomain = ambient?.LastAction?.Domain;
if (!string.IsNullOrWhiteSpace(actionDomain))
{
return actionDomain.Trim().ToLowerInvariant();
}
var actionRouteDomain = AmbientContextProcessor.ResolveDomainFromRoute(ambient?.LastAction?.Route ?? string.Empty);
if (!string.IsNullOrWhiteSpace(actionRouteDomain))
{
return actionRouteDomain;
}
return plan?.DomainWeights
.OrderByDescending(static pair => pair.Value)
.ThenBy(static pair => pair.Key, StringComparer.Ordinal)
.Select(static pair => pair.Key)
.FirstOrDefault(static value => !string.IsNullOrWhiteSpace(value));
}
private static string DescribeDomain(string domain)
{
return domain switch
{
"findings" => "the findings scope",
"vex" => "the VEX scope",
"policy" => "the policy scope",
"graph" => "the graph scope",
"timeline" => "the timeline scope",
"opsmemory" => "the operations-memory scope",
"knowledge" => "the knowledge scope",
"scanner" => "the scanner scope",
_ => "the current scope"
};
}
private static string FormatDomainList(IEnumerable<string> domains)
{
var labels = domains
.Where(static domain => !string.IsNullOrWhiteSpace(domain))
.Select(static domain => domain switch
{
"findings" => "findings",
"vex" => "VEX",
"policy" => "policy",
"graph" => "graph",
"timeline" => "timeline",
"opsmemory" => "ops memory",
"scanner" => "scanner",
_ => domain
})
.Distinct(StringComparer.OrdinalIgnoreCase)
.Take(3)
.ToArray();
return labels.Length == 0 ? "knowledge" : string.Join(", ", labels);
}
private static string? GetPrimaryActionRoute(EntityCard card)
{
return card.Actions
.FirstOrDefault(static action => !string.IsNullOrWhiteSpace(action.Route))?
.Route;
}
private const int PreviewContentMaxLength = 2000;
private static EntityCardPreview? BuildPreview(KnowledgeChunkRow row, string domain)
@@ -889,14 +1384,99 @@ internal sealed class UnifiedSearchService : IUnifiedSearchService
return Math.Clamp(requested.Value, 1, 100);
}
private UnifiedSearchResponse EmptyResponse(string query, int? topK, string mode)
private UnifiedSearchResponse EmptyResponse(string query, int? topK, string mode, AmbientContext? ambient = null)
{
return new UnifiedSearchResponse(
query,
ResolveTopK(topK),
[],
null,
new UnifiedSearchDiagnostics(0, 0, 0, 0, false, mode));
new UnifiedSearchDiagnostics(0, 0, 0, 0, false, mode),
ContextAnswer: BuildEmptyContextAnswer(query, mode, ambient));
}
private ContextAnswer? BuildContextAnswer(
string query,
QueryPlan plan,
AmbientContext? ambient,
IReadOnlyList<EntityCard> cards,
SynthesisResult? synthesis,
IReadOnlyList<SearchSuggestion>? suggestions,
IReadOnlyList<SearchRefinement>? refinements)
{
if (string.IsNullOrWhiteSpace(query))
{
return null;
}
if (cards.Count > 0)
{
var topCard = cards[0];
var citations = BuildContextAnswerCitations(cards, synthesis);
var questions = BuildGroundedQuestions(query, plan, ambient, topCard);
return new ContextAnswer(
Status: "grounded",
Code: "retrieved_evidence",
Summary: BuildGroundedSummary(cards, synthesis),
Reason: BuildGroundedReason(plan, ambient, topCard),
Evidence: BuildGroundedEvidence(cards, synthesis),
Citations: citations,
Questions: questions);
}
if (ShouldClarifyQuery(query, plan, ambient, suggestions, refinements))
{
var clarificationScope = ResolveContextDomain(plan, cards, ambient) ?? "current scope";
return new ContextAnswer(
Status: "clarify",
Code: refinements is { Count: > 0 } || suggestions is { Count: > 0 }
? "query_needs_scope_with_recovery"
: "query_needs_scope",
Summary: BuildClarifySummary(query, clarificationScope),
Reason: BuildClarifyReason(plan, ambient),
Evidence: BuildClarifyEvidence(suggestions, refinements, clarificationScope),
Citations: [],
Questions: BuildClarifyingQuestions(query, plan, ambient));
}
var recoveryQuestions = BuildRecoveryQuestions(query, plan, ambient, suggestions, refinements);
return new ContextAnswer(
Status: "insufficient",
Code: "no_grounded_evidence",
Summary: BuildInsufficientSummary(query, plan, ambient),
Reason: "No grounded evidence matched the requested terms in the current ingested corpus.",
Evidence: BuildInsufficientEvidence(suggestions, refinements),
Citations: [],
Questions: recoveryQuestions);
}
private ContextAnswer? BuildEmptyContextAnswer(string query, string mode, AmbientContext? ambient)
{
if (string.IsNullOrWhiteSpace(query))
{
return null;
}
return mode switch
{
"query_too_long" => new ContextAnswer(
Status: "clarify",
Code: "query_too_long",
Summary: "Shorten the search to a specific component, check, policy, or CVE.",
Reason: "The query exceeded the bounded unified-search length limit, so it cannot be ranked safely.",
Evidence: "No search was executed. Narrow the query before retrying.",
Citations: [],
Questions: BuildClarifyingQuestions(query, plan: null, ambient: ambient)),
"disabled" => new ContextAnswer(
Status: "insufficient",
Code: "search_disabled",
Summary: "Unified search is not enabled for the current environment or tenant.",
Reason: "The request could not be grounded because unified search is disabled before retrieval.",
Evidence: "No search index was queried.",
Citations: [],
Questions: BuildFallbackQuestionsForDisabledSearch(ambient)),
_ => null
};
}
private static string? GetMetadataString(JsonElement metadata, string propertyName)
@@ -1272,6 +1852,10 @@ internal sealed class UnifiedSearchService : IUnifiedSearchService
DurationMs: response.Diagnostics.DurationMs,
UsedVector: response.Diagnostics.UsedVector,
DomainWeights: new Dictionary<string, double>(plan.DomainWeights, StringComparer.Ordinal),
TopDomains: topDomains));
TopDomains: topDomains,
AnswerStatus: response.ContextAnswer?.Status,
AnswerCode: response.ContextAnswer?.Code,
HasSuggestions: response.Suggestions is { Count: > 0 },
HasRefinements: response.Refinements is { Count: > 0 }));
}
}

View File

@@ -14,7 +14,11 @@ public sealed record UnifiedSearchTelemetryEvent(
long DurationMs,
bool UsedVector,
IReadOnlyDictionary<string, double> DomainWeights,
IReadOnlyList<string> TopDomains);
IReadOnlyList<string> TopDomains,
string? AnswerStatus = null,
string? AnswerCode = null,
bool HasSuggestions = false,
bool HasRefinements = false);
public interface IUnifiedSearchTelemetrySink
{
@@ -45,13 +49,17 @@ internal sealed class LoggingUnifiedSearchTelemetrySink : IUnifiedSearchTelemetr
: string.Join(",", telemetryEvent.TopDomains.OrderBy(static value => value, StringComparer.Ordinal));
_logger.LogInformation(
"unified_search telemetry tenant={Tenant} query_hash={QueryHash} intent={Intent} results={ResultCount} duration_ms={DurationMs} used_vector={UsedVector} top_domains={TopDomains} weights={Weights}",
"unified_search telemetry tenant={Tenant} query_hash={QueryHash} intent={Intent} results={ResultCount} duration_ms={DurationMs} used_vector={UsedVector} answer_status={AnswerStatus} answer_code={AnswerCode} has_suggestions={HasSuggestions} has_refinements={HasRefinements} top_domains={TopDomains} weights={Weights}",
telemetryEvent.Tenant,
telemetryEvent.QueryHash,
telemetryEvent.Intent,
telemetryEvent.ResultCount,
telemetryEvent.DurationMs,
telemetryEvent.UsedVector,
telemetryEvent.AnswerStatus ?? "-",
telemetryEvent.AnswerCode ?? "-",
telemetryEvent.HasSuggestions,
telemetryEvent.HasRefinements,
topDomains,
weights);
}

View File

@@ -141,6 +141,7 @@ Build or publish the CLI from this repository:
```bash
# One-shot invocation without installing to PATH
export STELLAOPS_BACKEND_URL="http://127.0.0.1:10451"
dotnet run --project "src/Cli/StellaOps.Cli/StellaOps.Cli.csproj" -- advisoryai sources prepare --json
# Publish a reusable local binary
@@ -159,6 +160,7 @@ For live search and Playwright suggestion tests, rebuild both indexes in this or
```bash
# 1. Knowledge corpus: docs + OpenAPI + Doctor checks
export STELLAOPS_BACKEND_URL="http://127.0.0.1:10451"
dotnet run --project "src/Cli/StellaOps.Cli/StellaOps.Cli.csproj" -- advisoryai sources prepare --json
dotnet run --project "src/Cli/StellaOps.Cli/StellaOps.Cli.csproj" -- advisoryai index rebuild --json
@@ -182,6 +184,10 @@ curl -X POST http://127.0.0.1:10451/v1/search/index/rebuild \
-H "X-StellaOps-Tenant: test-tenant"
```
Notes:
- `stella advisoryai sources prepare` needs `STELLAOPS_BACKEND_URL` or equivalent CLI config when it performs live Doctor discovery. If you only need local search verification and the checked-in Doctor seed/control files are sufficient, the HTTP-only rebuild path is valid.
- Current live verification coverage includes the Doctor/knowledge query `database connectivity`, which returns `contextAnswer.status = grounded` plus citations after the rebuild sequence above.
Migration files (all idempotent, safe to re-run):
| File | Content |
| --- | --- |

View File

@@ -76,6 +76,53 @@ public sealed class UnifiedSearchEndpointsIntegrationTests : IDisposable
payload!.Query.Should().Be("cve-2024-21626");
payload.Cards.Should().NotBeEmpty();
payload.Cards.Should().Contain(card => card.Domain == "findings");
payload.ContextAnswer.Should().NotBeNull();
payload.ContextAnswer!.Status.Should().Be("grounded");
payload.ContextAnswer.Citations.Should().NotBeNullOrEmpty();
}
[Fact]
public async Task Query_WithBroadQuery_ReturnsClarifyContextAnswer()
{
using var client = _factory.CreateClient();
client.DefaultRequestHeaders.Add("X-StellaOps-Scopes", "advisory-ai:operate");
client.DefaultRequestHeaders.Add("X-StellaOps-Tenant", "test-tenant");
var response = await client.PostAsJsonAsync("/v1/search/query", new UnifiedSearchApiRequest
{
Q = "status"
});
response.StatusCode.Should().Be(HttpStatusCode.OK);
var payload = await response.Content.ReadFromJsonAsync<UnifiedSearchApiResponse>();
payload.Should().NotBeNull();
payload!.Cards.Should().BeEmpty();
payload.ContextAnswer.Should().NotBeNull();
payload.ContextAnswer!.Status.Should().Be("clarify");
payload.ContextAnswer.Questions.Should().NotBeNullOrEmpty();
}
[Fact]
public async Task Query_WithNoMatches_ReturnsInsufficientContextAnswer()
{
using var client = _factory.CreateClient();
client.DefaultRequestHeaders.Add("X-StellaOps-Scopes", "advisory-ai:operate");
client.DefaultRequestHeaders.Add("X-StellaOps-Tenant", "test-tenant");
var response = await client.PostAsJsonAsync("/v1/search/query", new UnifiedSearchApiRequest
{
Q = "manually compare unavailable evidence across environments"
});
response.StatusCode.Should().Be(HttpStatusCode.OK);
var payload = await response.Content.ReadFromJsonAsync<UnifiedSearchApiResponse>();
payload.Should().NotBeNull();
payload!.Cards.Should().BeEmpty();
payload.ContextAnswer.Should().NotBeNull();
payload.ContextAnswer!.Status.Should().Be("insufficient");
payload.ContextAnswer.Code.Should().Be("no_grounded_evidence");
}
[Fact]
@@ -211,6 +258,61 @@ public sealed class UnifiedSearchEndpointsIntegrationTests : IDisposable
{
public Task<UnifiedSearchResponse> SearchAsync(UnifiedSearchRequest request, CancellationToken cancellationToken)
{
var normalizedQuery = request.Q.Trim();
if (normalizedQuery.Equals("status", StringComparison.OrdinalIgnoreCase))
{
return Task.FromResult(new UnifiedSearchResponse(
normalizedQuery,
request.K ?? 10,
[],
null,
new UnifiedSearchDiagnostics(
FtsMatches: 0,
VectorMatches: 0,
EntityCardCount: 0,
DurationMs: 5,
UsedVector: false,
Mode: "fts-only"),
ContextAnswer: new ContextAnswer(
Status: "clarify",
Code: "query_needs_scope",
Summary: "\"status\" is too broad for the knowledge scope.",
Reason: "The query needs a narrower target before grounded evidence can be returned.",
Evidence: "No grounded evidence was retrieved yet from the knowledge scope.",
Citations: [],
Questions:
[
new ContextAnswerQuestion("Which check or symptom should I narrow this to?", "clarify")
])));
}
if (normalizedQuery.Equals("manually compare unavailable evidence across environments", StringComparison.OrdinalIgnoreCase))
{
return Task.FromResult(new UnifiedSearchResponse(
normalizedQuery,
request.K ?? 10,
[],
null,
new UnifiedSearchDiagnostics(
FtsMatches: 0,
VectorMatches: 0,
EntityCardCount: 0,
DurationMs: 5,
UsedVector: false,
Mode: "fts-only"),
ContextAnswer: new ContextAnswer(
Status: "insufficient",
Code: "no_grounded_evidence",
Summary: "No grounded answer was found for the requested query.",
Reason: "No grounded evidence matched the requested terms in the current corpus.",
Evidence: "No grounded citations, result cards, or bounded recovery hints were retrieved.",
Citations: [],
Questions:
[
new ContextAnswerQuestion("strongest evidence manually compare unavailable evidence across environments", "recover")
])));
}
var cards = new[]
{
new EntityCard
@@ -230,7 +332,7 @@ public sealed class UnifiedSearchEndpointsIntegrationTests : IDisposable
};
return Task.FromResult(new UnifiedSearchResponse(
request.Q.Trim(),
normalizedQuery,
request.K ?? 10,
cards,
null,
@@ -240,7 +342,25 @@ public sealed class UnifiedSearchEndpointsIntegrationTests : IDisposable
EntityCardCount: cards.Length,
DurationMs: 5,
UsedVector: false,
Mode: "fts-only")));
Mode: "fts-only"),
ContextAnswer: new ContextAnswer(
Status: "grounded",
Code: "retrieved_evidence",
Summary: "Container breakout via runc",
Reason: "The top result contains direct findings evidence.",
Evidence: "Grounded in 1 source across findings.",
Citations:
[
new ContextAnswerCitation(
"cve:CVE-2024-21626",
"CVE-2024-21626",
"findings",
"/security/triage?q=CVE-2024-21626")
],
Questions:
[
new ContextAnswerQuestion("What evidence proves exploitability for CVE-2024-21626?", "follow_up")
])));
}
}

View File

@@ -21,3 +21,5 @@ Source of truth: `docs/implplan/SPRINT_20260130_002_Tools_csproj_remediation_sol
| SPRINT_20260224_G5-005-BENCH | DONE | FTS recall benchmark: 12 tests in `FtsRecallBenchmarkTests.cs`, 34-query fixture (`fts-recall-benchmark.json`), `FtsRecallBenchmarkStore` (Simple vs English). Simple ~59% vs English ~100% Recall@10 (41pp improvement). |
| SPRINT_20260224_G1-004-BENCH | DONE | Semantic recall benchmark: 13 tests in `SemanticRecallBenchmarkTests.cs`, 48-query fixture (`semantic-recall-benchmark.json`), `SemanticRecallBenchmarkStore` (33 chunks), `SemanticSimulationEncoder` (40+ semantic groups). Semantic strictly outperforms hash on synonym queries. |
| AI-SELF-005 | DONE | Integration coverage now asserts grounded, clarify, and insufficient contextual-answer states through the real endpoint contract. |
| AI-SELF-006 | DONE | Verification includes a real local corpus rebuild and a live query assertion, not only test doubles. |

View File

@@ -7,6 +7,16 @@ namespace StellaOps.AdvisoryAI.Tests.UnifiedSearch;
public sealed class AmbientContextProcessorTests
{
[Theory]
[InlineData("/ops/operations/doctor/check.core.db.connectivity", "knowledge")]
[InlineData("/security/findings/fnd-42", "findings")]
[InlineData("/ops/timeline/releases/rel-42", "timeline")]
[InlineData("/ops/operations/jobs/job-7", "opsmemory")]
public void ResolveDomainFromRoute_matches_current_frontend_route_prefixes(string route, string expectedDomain)
{
AmbientContextProcessor.ResolveDomainFromRoute(route).Should().Be(expectedDomain);
}
[Fact]
public void ApplyRouteBoost_boosts_matching_domain_from_route()
{
@@ -26,6 +36,30 @@ public sealed class AmbientContextProcessorTests
boosted["knowledge"].Should().Be(1.0);
}
[Fact]
public void ApplyRouteBoost_adds_secondary_boost_from_last_action_domain()
{
var processor = new AmbientContextProcessor();
var weights = new Dictionary<string, double>(StringComparer.OrdinalIgnoreCase)
{
["knowledge"] = 1.0,
["policy"] = 1.0
};
var boosted = processor.ApplyRouteBoost(weights, new AmbientContext
{
CurrentRoute = "/ops/operations/doctor",
LastAction = new AmbientAction
{
Action = "search_answer_to_chat",
Domain = "policy"
}
});
boosted["knowledge"].Should().BeApproximately(1.10, 0.0001);
boosted["policy"].Should().BeApproximately(1.05, 0.0001);
}
[Fact]
public void BuildEntityBoostMap_merges_visible_entities_and_session_boosts()
{
@@ -49,6 +83,26 @@ public sealed class AmbientContextProcessorTests
map["image:registry.io/app:v1"].Should().BeApproximately(0.20, 0.0001);
}
[Fact]
public void BuildEntityBoostMap_prioritizes_last_action_entity_key()
{
var processor = new AmbientContextProcessor();
var map = processor.BuildEntityBoostMap(
new AmbientContext
{
VisibleEntityKeys = ["cve:CVE-2025-1234"],
LastAction = new AmbientAction
{
Action = "search_result_open",
EntityKey = "cve:CVE-2025-1234"
}
},
SearchSessionSnapshot.Empty);
map["cve:CVE-2025-1234"].Should().BeApproximately(0.25, 0.0001);
}
[Fact]
public void CarryForwardEntities_adds_session_entities_for_followup_queries_without_new_entities()
{

View File

@@ -45,6 +45,85 @@ public sealed class UnifiedSearchServiceTests
result.Diagnostics.Mode.Should().Be("disabled");
}
[Fact]
public async Task SearchAsync_returns_grounded_context_answer_when_cards_exist()
{
var doctorRow = MakeRow(
"chunk-doctor",
"doctor_check",
"PostgreSQL connectivity",
JsonDocument.Parse("{\"domain\":\"knowledge\",\"checkCode\":\"check.core.db.connectivity\"}"),
snippet: "PostgreSQL connectivity is failing because the gateway cannot reach the primary database.");
var storeMock = new Mock<IKnowledgeSearchStore>();
storeMock.Setup(s => s.SearchFtsAsync(
It.IsAny<string>(), It.IsAny<KnowledgeSearchFilter?>(), It.IsAny<int>(),
It.IsAny<TimeSpan>(), It.IsAny<CancellationToken>(), It.IsAny<string?>()))
.ReturnsAsync(new List<KnowledgeChunkRow> { doctorRow });
storeMock.Setup(s => s.LoadVectorCandidatesAsync(
It.IsAny<float[]>(), It.IsAny<KnowledgeSearchFilter?>(), It.IsAny<int>(),
It.IsAny<TimeSpan>(), It.IsAny<CancellationToken>()))
.ReturnsAsync([]);
var service = CreateService(storeMock: storeMock);
var result = await service.SearchAsync(
new UnifiedSearchRequest(
"database connectivity",
Ambient: new AmbientContext
{
CurrentRoute = "/ops/operations/doctor"
}),
CancellationToken.None);
result.ContextAnswer.Should().NotBeNull();
result.ContextAnswer!.Status.Should().Be("grounded");
result.ContextAnswer.Code.Should().Be("retrieved_evidence");
result.ContextAnswer.Citations.Should().NotBeNullOrEmpty();
result.ContextAnswer.Questions.Should().NotBeNullOrEmpty();
}
[Fact]
public async Task SearchAsync_returns_clarify_context_answer_for_broad_query_without_matches()
{
var storeMock = CreateEmptyStoreMock();
var service = CreateService(storeMock: storeMock);
var result = await service.SearchAsync(
new UnifiedSearchRequest(
"status",
Ambient: new AmbientContext
{
CurrentRoute = "/ops/operations/doctor"
}),
CancellationToken.None);
result.Cards.Should().BeEmpty();
result.ContextAnswer.Should().NotBeNull();
result.ContextAnswer!.Status.Should().Be("clarify");
result.ContextAnswer.Code.Should().StartWith("query_needs_scope");
result.ContextAnswer.Questions.Should().NotBeNullOrEmpty();
result.ContextAnswer.Questions!.Should().OnlyContain(question => question.Kind == "clarify");
}
[Fact]
public async Task SearchAsync_returns_insufficient_context_answer_for_specific_query_without_matches()
{
var storeMock = CreateEmptyStoreMock();
var service = CreateService(storeMock: storeMock);
var result = await service.SearchAsync(
new UnifiedSearchRequest("manually compare unavailable evidence across environments"),
CancellationToken.None);
result.Cards.Should().BeEmpty();
result.ContextAnswer.Should().NotBeNull();
result.ContextAnswer!.Status.Should().Be("insufficient");
result.ContextAnswer.Code.Should().Be("no_grounded_evidence");
result.ContextAnswer.Questions.Should().NotBeNullOrEmpty();
result.ContextAnswer.Questions!.Should().OnlyContain(question => question.Kind == "recover");
}
[Fact]
public async Task SearchAsync_returns_empty_when_tenant_feature_flag_disables_search()
{
@@ -799,6 +878,25 @@ public sealed class UnifiedSearchServiceTests
unifiedOptions: wrappedUnifiedOptions);
}
private static Mock<IKnowledgeSearchStore> CreateEmptyStoreMock()
{
var storeMock = new Mock<IKnowledgeSearchStore>();
storeMock.Setup(s => s.SearchFtsAsync(
It.IsAny<string>(), It.IsAny<KnowledgeSearchFilter?>(), It.IsAny<int>(),
It.IsAny<TimeSpan>(), It.IsAny<CancellationToken>(), It.IsAny<string?>()))
.ReturnsAsync([]);
storeMock.Setup(s => s.SearchFuzzyAsync(
It.IsAny<string>(), It.IsAny<KnowledgeSearchFilter?>(), It.IsAny<int>(),
It.IsAny<double>(), It.IsAny<TimeSpan>(), It.IsAny<CancellationToken>()))
.ReturnsAsync([]);
storeMock.Setup(s => s.LoadVectorCandidatesAsync(
It.IsAny<float[]>(), It.IsAny<KnowledgeSearchFilter?>(), It.IsAny<int>(),
It.IsAny<TimeSpan>(), It.IsAny<CancellationToken>()))
.ReturnsAsync([]);
return storeMock;
}
private static KnowledgeChunkRow MakeRow(
string chunkId,
string kind,