stella-ops.org/git.stella-ops.org
1
0
Fork 0
Code Issues Pull Requests Actions 2 Releases Wiki Activity

Search/AdvisoryAI and DAL conversion to EF finishes up. Preparation for microservices consolidation.

Browse Source
This commit is contained in:
master
2026-02-25 18:19:22 +02:00
parent 4db038123b
commit 63c70a6d37
447 changed files with 52257 additions and 2636 deletions
Download Patch File Download Diff File Expand all files Collapse all files

13
docs/modules/advisory-ai/CHANGELOG.md Normal file
View File

@@ -0,0 +1,13 @@
# AdvisoryAI Changelog
## 2026-02-25
- Unified search security hardening:
- Tenant-scoped chunk/doc IDs for findings, VEX, and policy live adapters to prevent cross-tenant identity collisions during incremental indexing.
- Backend and frontend snippet sanitization tightened to strip script/HTML payloads before rendering.
- Threat model for unified search added to `knowledge-search.md` (tenant isolation, injection, XSS, query-amplification controls).
## 2026-02-24
- Unified search migration and deprecation updates:
- Platform catalog entities are ingested into unified search so type-level platform navigation is available from `/v1/search/query`.
- Legacy search endpoint `/v1/advisory-ai/search` emits `Deprecation: true` and `Sunset: Thu, 31 Dec 2026 23:59:59 GMT`.
- Consumer migration target: move all search consumers to `/v1/search/query` before the sunset date.

69
docs/modules/advisory-ai/knowledge-search.md
View File

@@ -26,6 +26,9 @@ LLMs can still be used as optional formatters later, but AKS correctness is grou
- API endpoint: `POST /v1/advisory-ai/search`.
- Index rebuild endpoint: `POST /v1/advisory-ai/index/rebuild`.
Unified-search architecture reference:
- `docs/modules/advisory-ai/unified-search-architecture.md`
## Data model
AKS schema tables:
- `advisoryai.kb_doc`: canonical source docs with product/version/content hash metadata.
@@ -119,8 +122,10 @@ Implemented in `src/AdvisoryAI/StellaOps.AdvisoryAI/KnowledgeSearch/KnowledgeSea
- Unified index lifecycle:
- Manual rebuild endpoint: `POST /v1/search/index/rebuild`.
- Optional background refresh loop is available via `KnowledgeSearchOptions` (`UnifiedAutoIndexEnabled`, `UnifiedAutoIndexOnStartup`, `UnifiedIndexRefreshIntervalSeconds`).
- Unified ingestion adapters now ingest from deterministic snapshot files (findings/vex/policy) plus platform catalog projection, replacing hardcoded sample chunks.
- Default snapshot paths:
- Unified ingestion adapters for findings/vex/policy now use live upstream service payloads as primary source, with deterministic snapshot fallback only when upstream endpoints are unavailable or unconfigured.
- Live adapters: `FindingsSearchAdapter`, `VexSearchAdapter`, `PolicySearchAdapter`.
- Platform catalog remains a deterministic snapshot projection via `PlatformCatalogIngestionAdapter`.
- Default snapshot fallback paths:
- `src/AdvisoryAI/StellaOps.AdvisoryAI/UnifiedSearch/Snapshots/findings.snapshot.json`
- `src/AdvisoryAI/StellaOps.AdvisoryAI/UnifiedSearch/Snapshots/vex.snapshot.json`
- `src/AdvisoryAI/StellaOps.AdvisoryAI/UnifiedSearch/Snapshots/policy.snapshot.json`
@@ -129,9 +134,28 @@ Implemented in `src/AdvisoryAI/StellaOps.AdvisoryAI/KnowledgeSearch/KnowledgeSea
- Ranking no longer depends on ambient wall-clock time unless that option is enabled.
- Query telemetry:
- Unified search emits hashed query telemetry (`SHA-256` query hash, intent, domain weights, latency, top domains) via `IUnifiedSearchTelemetrySink`.
- Search analytics persistence stores hashed query keys (`SHA-256`, normalized) and pseudonymous user keys (tenant+user hash) in analytics/feedback artifacts.
- Free-form feedback comments are redacted at persistence time to avoid storing potential PII in analytics tables.
- Server-side search history remains user-facing functionality (raw query for history UX) and is keyed by pseudonymous user hash.
- Web fallback behavior: when unified search fails, `UnifiedSearchClient` falls back to legacy AKS (`/v1/advisory-ai/search`) and maps grouped legacy results into unified cards (`diagnostics.mode = legacy-fallback`).
- UI now shows an explicit degraded-mode banner for `legacy-fallback` / `fallback-empty` modes and clears it automatically on recovery.
- Degraded-mode enter/exit transitions emit analytics markers (`__degraded_mode_enter__`, `__degraded_mode_exit__`); server-side search history intentionally ignores `__*` synthetic markers.
- Deprecation timeline and migration milestones are tracked in `docs/modules/advisory-ai/CHANGELOG.md`.
## Unified search threat model (USRCH-POL-005)
Primary attack vectors and implemented mitigations:
- Cross-tenant data leakage:
- Risk: chunks from tenant A becoming visible in tenant B through weak filtering or identity collisions.
- Mitigations: mandatory tenant context on AKS/unified endpoints; tenant-aware store filters (`metadata.tenant` + `global` allowance); tenant-scoped chunk/doc identity for findings/vex/policy live adapters to prevent cross-tenant upsert collisions.
- Prompt/content injection from indexed sources:
- Risk: untrusted indexed text influencing synthesis or downstream operators.
- Mitigations: deterministic retrieval-first pipeline; synthesis grounding enforcement; analytics stores hashed query identifiers only; prompt payloads are not persisted in raw form.
- UI/script injection via snippets:
- Risk: malicious `<script>`/HTML in indexed body or highlighted snippets leading to XSS in search result cards.
- Mitigations: backend snippet sanitization strips script and HTML tags before response mapping; web client normalizes and strips tags again as defense-in-depth.
- Query-amplification and expensive-query DoS:
- Risk: oversized/invalid filters and high-rate query floods increasing DB and fusion cost.
- Mitigations: `q` length cap (512), strict allowlist validation for domains/entity types, per-tenant rate limiting, bounded candidate limits/timeouts in retrieval stages.
## Web behavior
Global search now consumes AKS and supports:
@@ -143,13 +167,19 @@ Global search now consumes AKS and supports:
- Doctor: `Run` (navigate to doctor and copy run command).
- `More` action for "show more like this" local query expansion.
- Search-quality metrics taxonomy is standardized on `query`, `click`, and `zero_result` event types (no legacy `search` event dependency in quality SQL).
- Synthesis usage is tracked via dedicated `synthesis` analytics events, while quality aggregates continue to compute totals from `query` + `zero_result`.
- Quality dashboard query dimensions are exposed as query hashes (not raw query text) for privacy-preserving analytics.
## CLI behavior
AKS commands:
- `stella search "<query>" [--type docs|api|doctor] [--product ...] [--version ...] [--service ...] [--tag ...] [--k N] [--json]`
- `stella search "<query>" [--type docs|api|doctor] [--product ...] [--version ...] [--service ...] [--tag ...] [--k N] [--synthesize] [--json]`
- `stella doctor suggest "<symptom>" [--product ...] [--version ...] [--k N] [--json]`
- `stella advisoryai index rebuild [--json]`
- `stella advisoryai sources prepare [--repo-root ...] [--docs-allowlist ...] [--docs-manifest-output ...] [--openapi-output ...] [--doctor-seed ...] [--doctor-controls-output ...] [--overwrite] [--json]`
- Unified-search API operations:
- `POST /v1/search/query`
- `POST /v1/search/synthesize`
- `POST /v1/search/index/rebuild`
Output:
- Human mode: grouped actionable references.
@@ -168,6 +198,16 @@ Tests:
- verifies deterministic dataset generation with >= 1000 queries.
- verifies recall/latency metrics and top-k match behavior.
Unified-search quality benchmarks:
- Corpus: `src/AdvisoryAI/__Tests/StellaOps.AdvisoryAI.Tests/TestData/unified-search-quality-corpus.json` (250 graded queries).
- Runner: `UnifiedSearchQualityBenchmarkRunner`.
- Fast PR gate: `UnifiedSearchQualityBenchmarkFastSubsetTests` (50 queries).
- Full suite: `UnifiedSearchQualityBenchmarkTests` and `UnifiedSearchPerformanceEnvelopeTests`.
- Reports:
- `docs/modules/advisory-ai/unified-search-ranking-benchmark.md`
- `docs/modules/advisory-ai/unified-search-release-readiness.md`
- `docs/operations/unified-search-operations.md`
## Dedicated AKS test DB
Compose profile:
- `devops/compose/docker-compose.advisoryai-knowledge-test.yml`
@@ -387,10 +427,28 @@ All in `KnowledgeSearchOptions` (`src/AdvisoryAI/StellaOps.AdvisoryAI/KnowledgeS
| `LlmAdapterBaseUrl` | `null` | G3 | LLM adapter service URL |
| `LlmProviderId` | `null` | G3 | LLM provider selection |
| `PopularityBoostEnabled` | `false` | G6 | Enable click-weighted ranking |
| `PopularityBoostWeight` | `0.1` | G6 | Popularity boost factor |
| `RoleBasedBiasEnabled` | `false` | G6 | Enable scope-based domain weighting |
| `PopularityBoostWeight` | `0.05` | G6 | Popularity boost factor |
| `RoleBasedBiasEnabled` | `true` | G6 | Enable scope-based domain weighting |
| `SearchQualityMonitorEnabled` | `true` | G10 | Enable periodic quality-alert refresh |
| `SearchQualityMonitorIntervalSeconds` | `300` | G10 | Quality-alert refresh cadence |
| `SearchAnalyticsRetentionEnabled` | `true` | G10 | Enable automatic analytics/feedback/history pruning |
| `SearchAnalyticsRetentionDays` | `90` | G10 | Retention window for search analytics artifacts |
| `SearchAnalyticsRetentionIntervalSeconds` | `3600` | G10 | Retention pruning cadence |
| `FtsLanguageConfigs` | `{}` | G9 | Per-locale FTS config map |
Unified-search options (`UnifiedSearchOptions`, `src/AdvisoryAI/StellaOps.AdvisoryAI/UnifiedSearch/UnifiedSearchOptions.cs`):
- `Enabled`
- `BaseDomainWeights`
- `Weighting.*`
- `Federation.*`
- `GravityBoost.*`
- `Synthesis.*`
- `Ingestion.*`
- `Session.*`
- `TenantFeatureFlags.<tenant>.Enabled`
- `TenantFeatureFlags.<tenant>.FederationEnabled`
- `TenantFeatureFlags.<tenant>.SynthesisEnabled`
## Known limitations and follow-ups
- YAML OpenAPI ingestion is not included in MVP.
- End-to-end benchmark against live Postgres-backed AKS service is planned as a follow-up CI lane.
@@ -398,4 +456,3 @@ All in `KnowledgeSearchOptions` (`src/AdvisoryAI/StellaOps.AdvisoryAI/KnowledgeS
- ONNX model file (`all-MiniLM-L6-v2.onnx`, ~80MB) must be provisioned separately for deployments opting into `VectorEncoderType=onnx`. Air-gap bundles must include the model.
- Doctor seed localization covers de-DE and fr-FR only. Other locales (es-ES, ru-RU, bg-BG, etc.) use English fallback.
- Search quality dashboard deferred items: low-quality results table, top queries table, 30-day trend chart (require additional backend aggregation queries).
- Periodic `SearchQualityMonitor` background job not yet wired (zero-result alerting runs on-demand via metrics endpoint).

119
docs/modules/advisory-ai/unified-search-architecture.md Normal file
View File

@@ -0,0 +1,119 @@
# Unified Search Architecture
This document defines the architecture for AdvisoryAI unified search (Sprint 100, Phase 4 hardening).
## Goals
- Help operators and users unfamiliar with Stella Ops terminology find relevant results quickly.
- Merge platform knowledge, findings, VEX, policy, graph, timeline, scanner, and OpsMemory signals into one deterministic ranking stream.
- Keep the system offline-capable and tenant-safe.
## Four-Layer Architecture
```mermaid
flowchart LR
Q[Layer 1: Query Understanding]
R[Layer 2: Federated Retrieval]
F[Layer 3: Fusion and Entity Cards]
S[Layer 4: Synthesis]
Q --> R --> F --> S
```
### Layer 1: Query Understanding
- Input: `UnifiedSearchRequest` (`q`, filters, ambient context, session id).
- Components:
- `EntityExtractor`
- `IntentClassifier`
- `DomainWeightCalculator`
- `AmbientContextProcessor`
- `SearchSessionContextService`
- Output: `QueryPlan` with intent, detected entities, domain weights, and context boosts.
### Layer 2: Federated Retrieval
- Sources queried in parallel:
- Primary universal index (`IKnowledgeSearchStore` FTS + vector candidates)
- Optional federated backends via `FederatedSearchDispatcher`
- Ingestion adapters keep index coverage aligned across domains:
- Findings, VEX, Policy (live + snapshot fallback)
- Graph, Timeline, Scanner, OpsMemory snapshots
- Platform catalog
- Tenant isolation is enforced in request filters and chunk identities.
### Layer 3: Fusion and Entity Cards
- `WeightedRrfFusion` merges lexical + vector candidates with domain weights.
- Additional boosts:
- Entity proximity
- Ambient/session carry-forward
- Graph gravity
- Optional popularity and freshness controls
- `EntityCardAssembler` groups facets into entity cards and resolves aliases.
### Layer 4: Synthesis
- Deterministic synthesis is always available from top cards.
- Optional LLM tier (`SearchSynthesisService`) streams over SSE with:
- quota enforcement
- grounding score
- action suggestions
- If LLM is unavailable or blocked by quota, deterministic output is still returned.
## Data Flow
```mermaid
sequenceDiagram
participant UI as Web UI / API Client
participant API as UnifiedSearchEndpoints
participant PLAN as QueryUnderstanding
participant IDX as KnowledgeSearchStore
participant FED as FederatedDispatcher
participant FUS as WeightedRrfFusion
participant CARDS as EntityCardAssembler
participant SYN as SearchSynthesisService
participant ANA as SearchAnalyticsService
UI->>API: POST /v1/search/query
API->>PLAN: Build QueryPlan
PLAN-->>API: intent + entities + domain weights
API->>IDX: SearchFtsAsync + LoadVectorCandidatesAsync
API->>FED: DispatchAsync (optional)
IDX-->>API: lexical + vector rows
FED-->>API: federated rows + diagnostics
API->>FUS: Fuse rankings
FUS-->>API: ranked rows
API->>CARDS: Assemble entity cards
CARDS-->>API: entity cards
API->>ANA: Record query/click/zero_result
API-->>UI: UnifiedSearchResponse
UI->>API: POST /v1/search/synthesize
API->>SYN: ExecuteAsync
SYN-->>UI: SSE deterministic-first + optional LLM chunks
```
## Contracts and API Surface
- `POST /v1/search/query`
- `POST /v1/search/synthesize`
- `POST /v1/search/index/rebuild`
OpenAPI contract presence is validated by integration test:
- `UnifiedSearchEndpointsIntegrationTests.OpenApi_Includes_UnifiedSearch_Contracts`
## Determinism Rules
- Stable ordering tie-breaks by `kind` then `chunkId`.
- Ranking benchmark includes a deterministic stability hash across top results.
- Session context is ephemeral and expires by inactivity timeout.
## Configuration
Primary section: `AdvisoryAI:UnifiedSearch`
- `Enabled`
- `BaseDomainWeights`
- `Weighting.*` (domain/intent/entity/role boosts)
- `Federation.*`
- `GravityBoost.*`
- `Synthesis.*`
- `Ingestion.*`
- `Session.*`
- `TenantFeatureFlags.<tenant>.{Enabled,FederationEnabled,SynthesisEnabled}`
Detailed operator config and examples:
- `docs/operations/unified-search-operations.md`
- `docs/modules/advisory-ai/knowledge-search.md`

54
docs/modules/advisory-ai/unified-search-ranking-benchmark.md Normal file
View File

@@ -0,0 +1,54 @@
# Unified Search Ranking Benchmark and Tuning Report
## Corpus
- File: `src/AdvisoryAI/__Tests/StellaOps.AdvisoryAI.Tests/TestData/unified-search-quality-corpus.json`
- Cases: 250 queries
- Archetypes: `cve_lookup`, `package_image`, `documentation`, `doctor_diagnostic`, `policy_search`, `audit_timeline`, `cross_domain`, `conversational_followup`
- Labels: relevance grades `0..3`
## Metrics
- Precision@1, @3, @5, @10
- Recall@10
- NDCG@10
- Entity-card top hit accuracy
- Cross-domain recall
- Ranking stability hash (SHA-256)
## Quality Gates
- P@1 >= 0.80
- NDCG@10 >= 0.70
- Entity-card accuracy >= 0.85
- Cross-domain recall >= 0.60
## Tuning Method
- Deterministic grid search over weighting parameters used by `DomainWeightCalculator`.
- Parameter ranges:
- `CveBoostFindings`: {0.35, 0.45}
- `CveBoostVex`: {0.30, 0.38}
- `PackageBoostGraph`: {0.20, 0.36, 0.48}
- `PackageBoostScanner`: {0.12, 0.28, 0.40}
- `AuditBoostTimeline`: {0.10, 0.24, 0.34}
- `PolicyBoostPolicy`: {0.30, 0.38}
- Tie-breakers: NDCG@10, then P@1, then stability hash.
## Baseline vs Tuned
_Values populated from `UnifiedSearchQualityBenchmarkTests` output._
| Variant | P@1 | NDCG@10 | Entity Accuracy | Cross-domain Recall | Gates Passed |
| --- | --- | --- | --- | --- | --- |
| Baseline (legacy weighting) | 0.9560 | 0.9522 | 0.9560 | 1.0000 | Yes |
| Tuned defaults | 0.9600 | 0.9598 | 0.9600 | 1.0000 | Yes |
Reference hashes from benchmark output:
- Baseline: `FF32EBE1DF1705A524B20B5A114B0CF496F1CA05147FC9FD869312903B8F40E9`
- Tuned defaults: `B5A12ACFE304E6A4620BBB2E9280FEE2E29E952B3E832F92C69FFA10760DA957`
## Tuned Defaults Applied
- `UnifiedSearchOptions.BaseDomainWeights`
- knowledge=1.05, findings=1.20, vex=1.15, policy=1.10, graph=1.15, timeline=1.05, scanner=1.10, opsmemory=1.05
- `UnifiedSearchOptions.Weighting`
- cve/security/policy/troubleshoot/package/audit/role boosts aligned with tuned values in `UnifiedSearchOptions.cs`
## Determinism
- Repeat runs produce identical stability hash for fixed corpus + options.
- Fast subset (50 queries) and full suite (250 queries) both run in CI lanes.

37
docs/modules/advisory-ai/unified-search-release-readiness.md Normal file
View File

@@ -0,0 +1,37 @@
# Unified Search Release Readiness
## Release Checklist
- [x] Schema migration path validated (clean + existing DB paths exercised in integration and migration checks).
- [x] Ingestion adapters validated across domains (findings, vex, policy, graph, timeline, scanner, opsmemory, platform).
- [x] Ranking quality gates satisfied by benchmark suite.
- [x] Performance envelope validated (50 concurrent load profile and regression guard).
- [x] Tenant isolation validated.
- [x] Accessibility checks retained in existing Web UI search suites.
- [x] Legacy endpoint compatibility and deprecation headers validated.
- [x] Analytics collection and retention loop validated.
- [x] Runbooks and architecture docs updated.
## Rollback Plan (Tested)
1. Disable per-tenant unified search:
- `AdvisoryAI:UnifiedSearch:TenantFeatureFlags:<tenant>:Enabled=false`
2. Optionally disable high-cost features first:
- `FederationEnabled=false`
- `SynthesisEnabled=false`
3. Keep index tables intact (no data loss).
4. Keep legacy platform endpoint active during rollback window.
5. Re-enable tenant gradually after quality/perf re-check.
## Known Issues
- ONNX model artifact in repository is a packaging placeholder path. Deployments must supply the licensed production model artifact for true semantic inference.
- Environment-level UI E2E reliability depends on full backend stack availability.
## Feature Flags
Path: `AdvisoryAI:UnifiedSearch:TenantFeatureFlags`
- `Enabled`
- `FederationEnabled`
- `SynthesisEnabled`
These flags are evaluated per tenant at request time in `UnifiedSearchService`.
## Archive Criteria
Search sprint files are eligible for archive only when all delivery tracker tasks are marked `DONE` and acceptance criteria are checked.

2
docs/modules/ui/architecture.md
View File

@@ -192,6 +192,8 @@ Each feature folder builds as a **standalone route** (lazy loaded). All HTTP sha
* **Search -> assistant handoff**: result cards and synthesis panel expose `Ask AI` actions that route to `/security/triage?openChat=true` and seed chat context through `SearchChatContextService`.
* **Assistant host**: `/security/triage` mounts `SecurityTriageChatHostComponent`, which consumes `openChat` intent deterministically and opens the chat drawer in the primary shell.
* **Assistant -> search return**: assistant responses expose `Search for more` and `Search related` actions; these populate global search query/domain context and focus the search surface.
* **Guided discovery empty state**: when global search is focused with an empty query, the panel renders an 8-domain guide (findings, VEX, policy, docs, API, health, operations, timeline), contextual suggestion chips, and quick actions (`Getting Started`, `Run Health Check`, `View Recent Scans`).
* **Shared route-context suggestions**: `AmbientContextService` is the single source for route-aware suggestion sets used by both global search and AdvisoryAI chat onboarding prompts, ensuring consistent context shifts as navigation changes.
* **Fallback transparency**: when unified search drops to legacy fallback, global search displays an explicit degraded banner and emits enter/exit telemetry markers for operator visibility.
---