stella-ops.org/git.stella-ops.org
1
0
Fork 0
Code Issues Pull Requests Actions 3 Releases Wiki Activity

user settings and breadcrumb fixes

Browse Source
This commit is contained in:
master
2026-03-07 17:14:02 +02:00
parent 1fa2e69032
commit 44c2b896e7
12 changed files with 645 additions and 278 deletions
Download Patch File Download Diff File Expand all files Collapse all files

17
docs/modules/advisory-ai/knowledge-search.md
View File

@@ -142,6 +142,8 @@ Implemented in `src/AdvisoryAI/StellaOps.AdvisoryAI/KnowledgeSearch/KnowledgeSea
- Query telemetry:
- Unified search emits hashed query telemetry (`SHA-256` query hash, intent, domain weights, latency, top domains) via `IUnifiedSearchTelemetrySink`.
- Search analytics persistence stores hashed query keys (`SHA-256`, normalized) and pseudonymous user keys (tenant+user hash) in analytics/feedback artifacts.
- Self-serve analytics is optional and privacy-preserving: when clients emit `answer_frame`, `reformulation`, or `rescue_action`, persistence stores a tenant-scoped hashed session id plus bounded answer metadata (`answer_status`, `answer_code`) instead of raw prompt history.
- Quality metrics surface self-serve gaps as `fallbackAnswerRate`, `clarifyRate`, `insufficientRate`, `reformulationCount`, `rescueActionCount`, and `abandonedFallbackCount`; alerting adds `fallback_loop` and `abandoned_fallback` signals for backlog review.
- Free-form feedback comments are redacted at persistence time to avoid storing potential PII in analytics tables.
- Server-side search history remains user-facing functionality (raw query for history UX) and is keyed by pseudonymous user hash.
- Web fallback behavior: when unified search fails, `UnifiedSearchClient` falls back to legacy AKS (`/v1/advisory-ai/search`) and maps grouped legacy results into unified cards (`diagnostics.mode = legacy-fallback`).
@@ -294,22 +296,25 @@ Run the full suite:
dotnet test "src/AdvisoryAI/__Tests/StellaOps.AdvisoryAI.Tests/StellaOps.AdvisoryAI.Tests.csproj" -v normal
```
Targeted runs in this project use xUnit v3 / Microsoft.Testing.Platform.
Do not use VSTest `dotnet test --filter ...` syntax here; use xUnit pass-through or the built test executable instead.
Run only the search sprint integration tests:
```bash
dotnet test "src/AdvisoryAI/__Tests/StellaOps.AdvisoryAI.Tests/StellaOps.AdvisoryAI.Tests.csproj" \
--filter "FullyQualifiedName~UnifiedSearchSprintIntegrationTests" -v normal
-- --filter-class StellaOps.AdvisoryAI.Tests.Integration.UnifiedSearchSprintIntegrationTests
```
Run only the FTS recall benchmark:
```bash
dotnet test "src/AdvisoryAI/__Tests/StellaOps.AdvisoryAI.Tests/StellaOps.AdvisoryAI.Tests.csproj" \
--filter "FullyQualifiedName~FtsRecallBenchmarkTests" -v normal
-- --filter-class StellaOps.AdvisoryAI.Tests.KnowledgeSearch.FtsRecallBenchmarkTests
```
Run only the semantic recall benchmark:
```bash
dotnet test "src/AdvisoryAI/__Tests/StellaOps.AdvisoryAI.Tests/StellaOps.AdvisoryAI.Tests.csproj" \
--filter "FullyQualifiedName~SemanticRecallBenchmarkTests" -v normal
-- --filter-class StellaOps.AdvisoryAI.Tests.KnowledgeSearch.SemanticRecallBenchmarkTests
```
**For live database tests** (e.g., full AKS rebuild + query against real Postgres with pg_trgm/pgvector):
@@ -334,8 +339,9 @@ curl -X POST http://127.0.0.1:10451/v1/search/index/rebuild \
-H "X-StellaOps-Tenant: test-tenant"
# Run tests with the Live category (requires database)
dotnet test "src/AdvisoryAI/__Tests/StellaOps.AdvisoryAI.Tests/StellaOps.AdvisoryAI.Tests.csproj" \
--filter "Category=Live" -v normal
dotnet build "src/AdvisoryAI/__Tests/StellaOps.AdvisoryAI.Tests/StellaOps.AdvisoryAI.Tests.csproj" -v minimal
src/AdvisoryAI/__Tests/StellaOps.AdvisoryAI.Tests/bin/Debug/net10.0/StellaOps.AdvisoryAI.Tests.exe \
-trait "Category=Live" -reporter verbose -noColor
```
### CLI setup in a source checkout
@@ -401,6 +407,7 @@ The search sprints added several migrations under `src/AdvisoryAI/StellaOps.Advi
| `005_search_feedback.sql` | G10 (110) | `search_feedback` + `search_quality_alerts` tables |
| `005_search_analytics.sql` | G6 (106) | `search_events` + `search_history` tables |
| `007_multilingual_fts.sql` | G9 (109) | `body_tsv_de`, `body_tsv_fr`, `body_tsv_es`, `body_tsv_ru` tsvector columns + GIN indexes |
| `008_search_self_serve_analytics.sql` | AI-SELF-004 | `session_id`, `answer_status`, `answer_code` analytics columns plus self-serve indexes |
All migrations are idempotent (IF NOT EXISTS guards). They run automatically via `EnsureSchemaAsync()` at service startup.

15
docs/modules/advisory-ai/unified-search-architecture.md
View File

@@ -63,6 +63,21 @@ flowchart LR
- bounded follow-up `questions`
- The answer envelope is additive and optional so older clients remain compatible.
### Telemetry and gap surfacing
- Search analytics stays optional at the client layer; queries still work when analytics events are never emitted.
- When enabled, the self-serve lane records `answer_frame`, `reformulation`, and `rescue_action` with hashed query keys, hashed tenant-scoped session ids, and bounded answer metadata.
- Quality review surfaces:
- `GET /v1/advisory-ai/search/quality/metrics`
- `GET /v1/advisory-ai/search/quality/alerts`
- Current self-serve gap signals:
- fallback answer rate
- clarify rate
- insufficient-evidence rate
- reformulation count
- rescue-action count
- abandoned fallback count
- `fallback_loop` and `abandoned_fallback` alerts
## Data Flow
```mermaid

20
docs/operations/unified-search-operations.md
View File

@@ -8,7 +8,8 @@ Runbook for AdvisoryAI unified search setup, operations, troubleshooting, perfor
2. Configure `AdvisoryAI:UnifiedSearch` options.
3. Ensure model artifact path exists when `VectorEncoderType=onnx`:
- default: `models/all-MiniLM-L6-v2.onnx`
4. Rebuild index:
4. Rebuild indexes in order when verifying live search quality:
- `POST /v1/advisory-ai/index/rebuild`
- `POST /v1/search/index/rebuild`
5. Verify query endpoint:
- `POST /v1/search/query` with `X-StellaOps-Tenant` and `advisory-ai:operate` scope.
@@ -24,8 +25,11 @@ Runbook for AdvisoryAI unified search setup, operations, troubleshooting, perfor
## Monitoring
Track per-tenant and global:
- Query throughput (`query`, `click`, `zero_result`, `synthesis` events)
- Self-serve journey signals (`answer_frame`, `reformulation`, `rescue_action`)
- P50/P95/P99 latency for `/v1/search/query`
- Zero-result rate
- Fallback answer rate, clarify rate, insufficient-evidence rate
- Reformulation count, rescue-action count, abandoned fallback count
- Synthesis quota denials
- Index size and rebuild duration
- Active encoder diagnostics (`diagnostics.activeEncoder`)
@@ -104,6 +108,7 @@ Example:
- Verify tenant header is present.
- Verify `UnifiedSearch.Enabled` and tenant flag `Enabled`.
- Run index rebuild and check chunk count.
- If suggestions also fail, verify both rebuild steps were run in order and re-check with a known live query such as `database connectivity`.
### Symptom: poor semantic recall
- Verify `VectorEncoderType` and active encoder diagnostics.
@@ -114,6 +119,12 @@ Example:
- Check `SynthesisEnabled` (global + tenant).
- Check quota counters and provider configuration.
### Symptom: search feels self-serve weak
- Inspect `GET /v1/advisory-ai/search/quality/metrics?period=7d`.
- Watch `fallbackAnswerRate`, `clarifyRate`, `insufficientRate`, `reformulationCount`, `rescueActionCount`, and `abandonedFallbackCount`.
- Inspect `GET /v1/advisory-ai/search/quality/alerts` for `fallback_loop` and `abandoned_fallback`.
- Treat repeated fallback loops as ranking/context gaps; treat abandoned fallback sessions as UX/product gaps.
### Symptom: high latency
- Check federated backend timeout budget.
- Review `EXPLAIN (ANALYZE)` plans.
@@ -140,4 +151,11 @@ dotnet test src/AdvisoryAI/__Tests/StellaOps.AdvisoryAI.Tests/StellaOps.Advisory
# Performance envelope
dotnet test src/AdvisoryAI/__Tests/StellaOps.AdvisoryAI.Tests/StellaOps.AdvisoryAI.Tests.csproj \
-- --filter-class StellaOps.AdvisoryAI.Tests.UnifiedSearch.UnifiedSearchPerformanceEnvelopeTests
# Self-serve telemetry and gap surfacing slice
dotnet build src/AdvisoryAI/__Tests/StellaOps.AdvisoryAI.Tests/StellaOps.AdvisoryAI.Tests.csproj -v minimal
src/AdvisoryAI/__Tests/StellaOps.AdvisoryAI.Tests/bin/Debug/net10.0/StellaOps.AdvisoryAI.Tests.exe \
-method "StellaOps.AdvisoryAI.Tests.Integration.UnifiedSearchSprintIntegrationTests.G10_SelfServeMetrics_IncludeFallbackReformulationAndRescueSignals" \
-method "StellaOps.AdvisoryAI.Tests.Integration.UnifiedSearchSprintIntegrationTests.G10_RecoveredFallbackSessions_DoNotCountAsAbandoned" \
-reporter verbose -noColor
```