stella-ops.org/git.stella-ops.org
1
0
Fork 0
Code Issues Pull Requests Actions 2 Releases Wiki Activity

user settings and breadcrumb fixes

Browse Source
This commit is contained in:
master
2026-03-07 17:14:02 +02:00
parent 1fa2e69032
commit 44c2b896e7
12 changed files with 645 additions and 278 deletions
Download Patch File Download Diff File Expand all files Collapse all files

20
docs/operations/unified-search-operations.md
View File

@@ -8,7 +8,8 @@ Runbook for AdvisoryAI unified search setup, operations, troubleshooting, perfor
2. Configure `AdvisoryAI:UnifiedSearch` options.
3. Ensure model artifact path exists when `VectorEncoderType=onnx`:
- default: `models/all-MiniLM-L6-v2.onnx`
4. Rebuild index:
4. Rebuild indexes in order when verifying live search quality:
- `POST /v1/advisory-ai/index/rebuild`
- `POST /v1/search/index/rebuild`
5. Verify query endpoint:
- `POST /v1/search/query` with `X-StellaOps-Tenant` and `advisory-ai:operate` scope.
@@ -24,8 +25,11 @@ Runbook for AdvisoryAI unified search setup, operations, troubleshooting, perfor
## Monitoring
Track per-tenant and global:
- Query throughput (`query`, `click`, `zero_result`, `synthesis` events)
- Self-serve journey signals (`answer_frame`, `reformulation`, `rescue_action`)
- P50/P95/P99 latency for `/v1/search/query`
- Zero-result rate
- Fallback answer rate, clarify rate, insufficient-evidence rate
- Reformulation count, rescue-action count, abandoned fallback count
- Synthesis quota denials
- Index size and rebuild duration
- Active encoder diagnostics (`diagnostics.activeEncoder`)
@@ -104,6 +108,7 @@ Example:
- Verify tenant header is present.
- Verify `UnifiedSearch.Enabled` and tenant flag `Enabled`.
- Run index rebuild and check chunk count.
- If suggestions also fail, verify both rebuild steps were run in order and re-check with a known live query such as `database connectivity`.
### Symptom: poor semantic recall
- Verify `VectorEncoderType` and active encoder diagnostics.
@@ -114,6 +119,12 @@ Example:
- Check `SynthesisEnabled` (global + tenant).
- Check quota counters and provider configuration.
### Symptom: search feels self-serve weak
- Inspect `GET /v1/advisory-ai/search/quality/metrics?period=7d`.
- Watch `fallbackAnswerRate`, `clarifyRate`, `insufficientRate`, `reformulationCount`, `rescueActionCount`, and `abandonedFallbackCount`.
- Inspect `GET /v1/advisory-ai/search/quality/alerts` for `fallback_loop` and `abandoned_fallback`.
- Treat repeated fallback loops as ranking/context gaps; treat abandoned fallback sessions as UX/product gaps.
### Symptom: high latency
- Check federated backend timeout budget.
- Review `EXPLAIN (ANALYZE)` plans.
@@ -140,4 +151,11 @@ dotnet test src/AdvisoryAI/__Tests/StellaOps.AdvisoryAI.Tests/StellaOps.Advisory
# Performance envelope
dotnet test src/AdvisoryAI/__Tests/StellaOps.AdvisoryAI.Tests/StellaOps.AdvisoryAI.Tests.csproj \
-- --filter-class StellaOps.AdvisoryAI.Tests.UnifiedSearch.UnifiedSearchPerformanceEnvelopeTests
# Self-serve telemetry and gap surfacing slice
dotnet build src/AdvisoryAI/__Tests/StellaOps.AdvisoryAI.Tests/StellaOps.AdvisoryAI.Tests.csproj -v minimal
src/AdvisoryAI/__Tests/StellaOps.AdvisoryAI.Tests/bin/Debug/net10.0/StellaOps.AdvisoryAI.Tests.exe \
-method "StellaOps.AdvisoryAI.Tests.Integration.UnifiedSearchSprintIntegrationTests.G10_SelfServeMetrics_IncludeFallbackReformulationAndRescueSignals" \
-method "StellaOps.AdvisoryAI.Tests.Integration.UnifiedSearchSprintIntegrationTests.G10_RecoveredFallbackSessions_DoNotCountAsAbandoned" \
-reporter verbose -noColor
```