120 lines
4.0 KiB
Markdown
120 lines
4.0 KiB
Markdown
# Unified Search Architecture
|
|
|
|
This document defines the architecture for AdvisoryAI unified search (Sprint 100, Phase 4 hardening).
|
|
|
|
## Goals
|
|
- Help operators and users unfamiliar with Stella Ops terminology find relevant results quickly.
|
|
- Merge platform knowledge, findings, VEX, policy, graph, timeline, scanner, and OpsMemory signals into one deterministic ranking stream.
|
|
- Keep the system offline-capable and tenant-safe.
|
|
|
|
## Four-Layer Architecture
|
|
|
|
```mermaid
|
|
flowchart LR
|
|
Q[Layer 1: Query Understanding]
|
|
R[Layer 2: Federated Retrieval]
|
|
F[Layer 3: Fusion and Entity Cards]
|
|
S[Layer 4: Synthesis]
|
|
|
|
Q --> R --> F --> S
|
|
```
|
|
|
|
### Layer 1: Query Understanding
|
|
- Input: `UnifiedSearchRequest` (`q`, filters, ambient context, session id).
|
|
- Components:
|
|
- `EntityExtractor`
|
|
- `IntentClassifier`
|
|
- `DomainWeightCalculator`
|
|
- `AmbientContextProcessor`
|
|
- `SearchSessionContextService`
|
|
- Output: `QueryPlan` with intent, detected entities, domain weights, and context boosts.
|
|
|
|
### Layer 2: Federated Retrieval
|
|
- Sources queried in parallel:
|
|
- Primary universal index (`IKnowledgeSearchStore` FTS + vector candidates)
|
|
- Optional federated backends via `FederatedSearchDispatcher`
|
|
- Ingestion adapters keep index coverage aligned across domains:
|
|
- Findings, VEX, Policy (live + snapshot fallback)
|
|
- Graph, Timeline, Scanner, OpsMemory snapshots
|
|
- Platform catalog
|
|
- Tenant isolation is enforced in request filters and chunk identities.
|
|
|
|
### Layer 3: Fusion and Entity Cards
|
|
- `WeightedRrfFusion` merges lexical + vector candidates with domain weights.
|
|
- Additional boosts:
|
|
- Entity proximity
|
|
- Ambient/session carry-forward
|
|
- Graph gravity
|
|
- Optional popularity and freshness controls
|
|
- `EntityCardAssembler` groups facets into entity cards and resolves aliases.
|
|
|
|
### Layer 4: Synthesis
|
|
- Deterministic synthesis is always available from top cards.
|
|
- Optional LLM tier (`SearchSynthesisService`) streams over SSE with:
|
|
- quota enforcement
|
|
- grounding score
|
|
- action suggestions
|
|
- If LLM is unavailable or blocked by quota, deterministic output is still returned.
|
|
|
|
## Data Flow
|
|
|
|
```mermaid
|
|
sequenceDiagram
|
|
participant UI as Web UI / API Client
|
|
participant API as UnifiedSearchEndpoints
|
|
participant PLAN as QueryUnderstanding
|
|
participant IDX as KnowledgeSearchStore
|
|
participant FED as FederatedDispatcher
|
|
participant FUS as WeightedRrfFusion
|
|
participant CARDS as EntityCardAssembler
|
|
participant SYN as SearchSynthesisService
|
|
participant ANA as SearchAnalyticsService
|
|
|
|
UI->>API: POST /v1/search/query
|
|
API->>PLAN: Build QueryPlan
|
|
PLAN-->>API: intent + entities + domain weights
|
|
API->>IDX: SearchFtsAsync + LoadVectorCandidatesAsync
|
|
API->>FED: DispatchAsync (optional)
|
|
IDX-->>API: lexical + vector rows
|
|
FED-->>API: federated rows + diagnostics
|
|
API->>FUS: Fuse rankings
|
|
FUS-->>API: ranked rows
|
|
API->>CARDS: Assemble entity cards
|
|
CARDS-->>API: entity cards
|
|
API->>ANA: Record query/click/zero_result
|
|
API-->>UI: UnifiedSearchResponse
|
|
|
|
UI->>API: POST /v1/search/synthesize
|
|
API->>SYN: ExecuteAsync
|
|
SYN-->>UI: SSE deterministic-first + optional LLM chunks
|
|
```
|
|
|
|
## Contracts and API Surface
|
|
- `POST /v1/search/query`
|
|
- `POST /v1/search/synthesize`
|
|
- `POST /v1/search/index/rebuild`
|
|
|
|
OpenAPI contract presence is validated by integration test:
|
|
- `UnifiedSearchEndpointsIntegrationTests.OpenApi_Includes_UnifiedSearch_Contracts`
|
|
|
|
## Determinism Rules
|
|
- Stable ordering tie-breaks by `kind` then `chunkId`.
|
|
- Ranking benchmark includes a deterministic stability hash across top results.
|
|
- Session context is ephemeral and expires by inactivity timeout.
|
|
|
|
## Configuration
|
|
Primary section: `AdvisoryAI:UnifiedSearch`
|
|
- `Enabled`
|
|
- `BaseDomainWeights`
|
|
- `Weighting.*` (domain/intent/entity/role boosts)
|
|
- `Federation.*`
|
|
- `GravityBoost.*`
|
|
- `Synthesis.*`
|
|
- `Ingestion.*`
|
|
- `Session.*`
|
|
- `TenantFeatureFlags.<tenant>.{Enabled,FederationEnabled,SynthesisEnabled}`
|
|
|
|
Detailed operator config and examples:
|
|
- `docs/operations/unified-search-operations.md`
|
|
- `docs/modules/advisory-ai/knowledge-search.md`
|