stella-ops.org/git.stella-ops.org
1
0
Fork 0
Code Issues Pull Requests Actions 2 Releases Wiki Activity

Harden live-backed unified search weighting and indexing

Browse Source
This commit is contained in:
master
2026-03-08 02:23:43 +02:00
parent c7b7ddf436
commit 145e67a544
26 changed files with 1585 additions and 207 deletions
Download Patch File Download Diff File Expand all files Collapse all files

25
docs-archived/implplan/SPRINT_20260307_037_AdvisoryAI_search_answer_proof_and_weighting.md
View File

@@ -21,7 +21,7 @@
## Delivery Tracker ## Delivery Tracker
### AI-SF-001 - Strengthen automatic in-scope weighting and overflow suppression ### AI-SF-001 - Strengthen automatic in-scope weighting and overflow suppression
Status: TODO Status: DONE
Dependency: none Dependency: none
Owners: Developer Owners: Developer
Task description: Task description:
@@ -29,12 +29,12 @@ Task description:
- Keep overflow only when outside-scope evidence still materially improves the answer. - Keep overflow only when outside-scope evidence still materially improves the answer.
Completion criteria: Completion criteria:
- [ ] Service tests cover current-route winners, close-score blends, and suppressed overflow cases. - [x] Service tests cover current-route winners, close-score blends, and suppressed overflow cases.
- [ ] Coverage metadata still explains the winning scope without FE heuristics. - [x] Coverage metadata still explains the winning scope without FE heuristics.
- [ ] Unsupported or weak current-scope corpora do not hide clearly better outside-scope answers. - [x] Unsupported or weak current-scope corpora do not hide clearly better outside-scope answers.
### AI-SF-002 - Make blended answers and suggestion viability stricter ### AI-SF-002 - Make blended answers and suggestion viability stricter
Status: TODO Status: DONE
Dependency: AI-SF-001 Dependency: AI-SF-001
Owners: Developer Owners: Developer
Task description: Task description:
@@ -42,12 +42,12 @@ Task description:
- Keep suggestion viability grounded-only for surfaced chips and detect unsupported/empty corpora explicitly. - Keep suggestion viability grounded-only for surfaced chips and detect unsupported/empty corpora explicitly.
Completion criteria: Completion criteria:
- [ ] Blended and dominant answer paths are tested separately. - [x] Blended and dominant answer paths are tested separately.
- [ ] Clarify-only or unsupported suggestions do not pass visible viability. - [x] Clarify-only or unsupported suggestions do not pass visible viability.
- [ ] Corpus-readiness states remain explicit in the response contract. - [x] Corpus-readiness states remain explicit in the response contract.
### AI-SF-003 - Preserve optional telemetry and deterministic fallbacks ### AI-SF-003 - Preserve optional telemetry and deterministic fallbacks
Status: TODO Status: DONE
Dependency: AI-SF-002 Dependency: AI-SF-002
Owners: Developer Owners: Developer
Task description: Task description:
@@ -55,14 +55,15 @@ Task description:
- Ensure the final correction pass does not reintroduce telemetry coupling. - Ensure the final correction pass does not reintroduce telemetry coupling.
Completion criteria: Completion criteria:
- [ ] Search behavior remains stable with telemetry disabled. - [x] Search behavior remains stable with telemetry disabled.
- [ ] Tests cover telemetry-disabled search and suggestion flows. - [x] Tests cover telemetry-disabled search and suggestion flows.
- [ ] Docs state clearly that telemetry is optional infrastructure. - [x] Docs state clearly that telemetry is optional infrastructure.
## Execution Log ## Execution Log
| Date (UTC) | Update | Owner | | Date (UTC) | Update | Owner |
| --- | --- | --- | | --- | --- | --- |
| 2026-03-07 | Sprint created for the backend weighting and suggestion-proof half of the final search-first correction pass. | Project Manager | | 2026-03-07 | Sprint created for the backend weighting and suggestion-proof half of the final search-first correction pass. | Project Manager |
| 2026-03-08 | Completed the backend weighting and viability hardening pass: stronger route/action boosts, unified-index FTS/vector persistence, enriched findings/policy/VEX ingestion bodies for route-language recall, and DB-backed similar-query lookup repair. Evidence: `dotnet build "src/AdvisoryAI/__Tests/StellaOps.AdvisoryAI.Tests/StellaOps.AdvisoryAI.Tests.csproj" --no-restore -v minimal -p:BuildInParallel=false -p:UseSharedCompilation=false`; `StellaOps.AdvisoryAI.Tests.exe -class "StellaOps.AdvisoryAI.Tests.UnifiedSearch.UnifiedSearchServiceTests"` (`45/45`); `StellaOps.AdvisoryAI.Tests.exe -class "StellaOps.AdvisoryAI.Tests.Integration.UnifiedSearchLiveAdapterIntegrationTests"` (`14/14`); `StellaOps.AdvisoryAI.Tests.exe -class "StellaOps.AdvisoryAI.Tests.UnifiedSearch.SearchAnalyticsServiceTests"` (`1/1`); `StellaOps.AdvisoryAI.Tests.exe -method "StellaOps.AdvisoryAI.Tests.Integration.UnifiedSearchSprintIntegrationTests.G10_SimilarSuccessfulQueries_LoadFromDatabaseWithoutFallingBack"` (`1/1`). | Developer |
## Decisions & Risks ## Decisions & Risks
- Decision: search answer shape is inferred, not selected by the operator. - Decision: search answer shape is inferred, not selected by the operator.

10
docs/API_CLI_REFERENCE.md
View File

@@ -536,16 +536,16 @@ Use one of these local workflows first:
```bash ```bash
# Run the CLI directly from source # Run the CLI directly from source
dotnet run --project "src/Cli/StellaOps.Cli/StellaOps.Cli.csproj" -- advisoryai index rebuild --json dotnet run --project "src/Cli/StellaOps.Cli/StellaOps.Cli.csproj" -- --help
# Publish a reusable local binary # Publish a reusable local binary
dotnet publish "src/Cli/StellaOps.Cli/StellaOps.Cli.csproj" -c Release -o ".artifacts/stella-cli" dotnet publish "src/Cli/StellaOps.Cli/StellaOps.Cli.csproj" -c Release -o ".artifacts/stella-cli"
# Windows # Windows
.artifacts/stella-cli/StellaOps.Cli.exe advisoryai index rebuild --json .artifacts/stella-cli/StellaOps.Cli.exe --help
# Linux/macOS # Linux/macOS
./.artifacts/stella-cli/StellaOps.Cli advisoryai index rebuild --json ./.artifacts/stella-cli/StellaOps.Cli --help
``` ```
Related docs: Related docs:
@@ -554,6 +554,10 @@ Related docs:
Rebuild the AdvisoryAI deterministic knowledge index from local markdown, OpenAPI specs, and Doctor metadata. Rebuild the AdvisoryAI deterministic knowledge index from local markdown, OpenAPI specs, and Doctor metadata.
Local source-checkout note:
- `stella advisoryai index rebuild` calls an authenticated backend endpoint.
- For the unauthenticated local live-search verification lane, use `stella advisoryai sources prepare` plus the direct HTTP rebuild calls documented in `src/AdvisoryAI/__Tests/INFRASTRUCTURE.md`.
### Synopsis ### Synopsis
```bash ```bash

18
docs/modules/advisory-ai/knowledge-search.md
View File

@@ -358,16 +358,19 @@ docker compose -f devops/compose/docker-compose.advisoryai-knowledge-test.yml ps
# Start the local AdvisoryAI service against that database # Start the local AdvisoryAI service against that database
export AdvisoryAI__KnowledgeSearch__ConnectionString="Host=localhost;Port=55432;Database=advisoryai_knowledge_test;Username=stellaops_knowledge;Password=stellaops_knowledge" export AdvisoryAI__KnowledgeSearch__ConnectionString="Host=localhost;Port=55432;Database=advisoryai_knowledge_test;Username=stellaops_knowledge;Password=stellaops_knowledge"
export AdvisoryAI__KnowledgeSearch__RepositoryRoot="$(pwd)"
dotnet run --project "src/AdvisoryAI/StellaOps.AdvisoryAI.WebService/StellaOps.AdvisoryAI.WebService.csproj" --no-launch-profile dotnet run --project "src/AdvisoryAI/StellaOps.AdvisoryAI.WebService/StellaOps.AdvisoryAI.WebService.csproj" --no-launch-profile
# In a second shell, rebuild the live corpus in the required order # In a second shell, rebuild the live corpus in the required order
export STELLAOPS_BACKEND_URL="http://127.0.0.1:10451" export STELLAOPS_BACKEND_URL="http://127.0.0.1:10451"
dotnet run --project "src/Cli/StellaOps.Cli/StellaOps.Cli.csproj" -- advisoryai sources prepare --json dotnet run --project "src/Cli/StellaOps.Cli/StellaOps.Cli.csproj" -- advisoryai sources prepare --json
dotnet run --project "src/Cli/StellaOps.Cli/StellaOps.Cli.csproj" -- advisoryai index rebuild --json curl -X POST http://127.0.0.1:10451/v1/advisory-ai/index/rebuild \
-H "X-StellaOps-Scopes: advisory-ai:admin" \
-H "X-StellaOps-Tenant: test-tenant" \
-H "X-StellaOps-Actor: local-search-test"
curl -X POST http://127.0.0.1:10451/v1/search/index/rebuild \ curl -X POST http://127.0.0.1:10451/v1/search/index/rebuild \
-H "X-StellaOps-Scopes: advisory-ai:admin" \ -H "X-StellaOps-Scopes: advisory-ai:admin" \
-H "X-StellaOps-Tenant: test-tenant" -H "X-StellaOps-Tenant: test-tenant" \
-H "X-StellaOps-Actor: local-search-test"
# Run tests with the Live category (requires database) # Run tests with the Live category (requires database)
dotnet build "src/AdvisoryAI/__Tests/StellaOps.AdvisoryAI.Tests/StellaOps.AdvisoryAI.Tests.csproj" -v minimal dotnet build "src/AdvisoryAI/__Tests/StellaOps.AdvisoryAI.Tests/StellaOps.AdvisoryAI.Tests.csproj" -v minimal
@@ -375,6 +378,11 @@ src/AdvisoryAI/__Tests/StellaOps.AdvisoryAI.Tests/bin/Debug/net10.0/StellaOps.Ad
-trait "Category=Live" -reporter verbose -noColor -trait "Category=Live" -reporter verbose -noColor
``` ```
Notes:
- AdvisoryAI knowledge ingestion now auto-detects the repository root from the current working directory and `AppContext.BaseDirectory` when it is running inside a normal Stella Ops source checkout.
- Set `AdvisoryAI__KnowledgeSearch__RepositoryRoot` only when you are running the service from a non-standard layout or a packaged binary tree that is not inside the repository.
- `stella advisoryai index rebuild` and `stella search index rebuild` invoke authenticated backend endpoints. For a local source-checkout verification lane without a signed-in CLI session, use `sources prepare` via CLI and the direct HTTP rebuild calls above with explicit `X-StellaOps-*` headers.
### CLI setup in a source checkout ### CLI setup in a source checkout
Do not assume `stella` is already installed on the machine running local AdvisoryAI work. Do not assume `stella` is already installed on the machine running local AdvisoryAI work.
@@ -406,11 +414,11 @@ If the CLI is not built yet, the equivalent HTTP endpoints are:
- `POST /v1/search/index/rebuild` for unified overlay domains - `POST /v1/search/index/rebuild` for unified overlay domains
Current live verification coverage: Current live verification coverage:
- Rebuild order exercised against a running local service: `POST /v1/advisory-ai/index/rebuild` then `POST /v1/search/index/rebuild` - Rebuild order exercised against a running local service: `POST /v1/advisory-ai/index/rebuild` then `POST /v1/search/index/rebuild`, both with explicit `X-StellaOps-Scopes`, `X-StellaOps-Tenant`, and `X-StellaOps-Actor` headers
- Verified live query: `database connectivity` - Verified live query: `database connectivity`
- Verified live outcome: response includes `contextAnswer.status = grounded`, citations, and entity cards over ingested data - Verified live outcome: response includes `contextAnswer.status = grounded`, citations, and entity cards over ingested data
- Verified live suggestion lane: `src/Web/StellaOps.Web/tests/e2e/unified-search-contextual-suggestions.live.e2e.spec.ts` now preflights corpus readiness, validates suggestion viability, executes every surfaced Doctor suggestion, asserts grounded answer states for surfaced live suggestions, verifies follow-up chips after result open, and verifies Ask-AdvisoryAI inherits the live query context - Verified live suggestion lane: `src/Web/StellaOps.Web/tests/e2e/unified-search-contextual-suggestions.live.e2e.spec.ts` now preflights corpus readiness, validates suggestion viability, executes every surfaced Doctor suggestion, asserts grounded answer states for surfaced live suggestions, verifies follow-up chips after result open, and verifies Ask-AdvisoryAI inherits the live query context
- Verified combined browser gate on 2026-03-07: `20/20` Playwright tests passed across deterministic UX, telemetry-off search flows, self-serve answer panel, and the live suggestion lane against the ingested local corpus - Verified combined browser gate on 2026-03-08: `24/24` executed tests passed with `3` explicit route-unready skips across deterministic UX, telemetry-off search flows, self-serve answer panel, and the supported-route live suggestion lane against the ingested local corpus
- Verified local corpus baseline on 2026-03-07 after `advisoryai sources prepare`: `documentCount = 470`, `chunkCount = 9050`, `apiOperationCount = 2190`, `doctorProjectionCount = 8` - Verified local corpus baseline on 2026-03-07 after `advisoryai sources prepare`: `documentCount = 470`, `chunkCount = 9050`, `apiOperationCount = 2190`, `doctorProjectionCount = 8`
- Other routes still rely on deterministic mock-backed Playwright coverage until their ingestion parity is explicitly verified - Other routes still rely on deterministic mock-backed Playwright coverage until their ingestion parity is explicitly verified

8
docs/modules/cli/guides/quickstart.md
View File

@@ -37,12 +37,16 @@ dotnet run --project "src/Cli/StellaOps.Cli/StellaOps.Cli.csproj" -- --help
dotnet publish "src/Cli/StellaOps.Cli/StellaOps.Cli.csproj" -c Release -o ".artifacts/stella-cli" dotnet publish "src/Cli/StellaOps.Cli/StellaOps.Cli.csproj" -c Release -o ".artifacts/stella-cli"
# Windows # Windows
.artifacts/stella-cli/StellaOps.Cli.exe advisoryai index rebuild --json .artifacts/stella-cli/StellaOps.Cli.exe --help
# Linux/macOS # Linux/macOS
./.artifacts/stella-cli/StellaOps.Cli advisoryai index rebuild --json ./.artifacts/stella-cli/StellaOps.Cli --help
``` ```
For local AdvisoryAI live-search verification from a source checkout:
- use `stella advisoryai sources prepare` from the local CLI build or `dotnet run`
- then use the authenticated HTTP rebuild steps in `src/AdvisoryAI/__Tests/INFRASTRUCTURE.md`
#### Option 1: .NET Tool (Recommended) #### Option 1: .NET Tool (Recommended)
```bash ```bash

12
src/AdvisoryAI/AGENTS.md
View File

@@ -63,8 +63,16 @@ dotnet test src/AdvisoryAI/__Tests/StellaOps.AdvisoryAI.Tests/StellaOps.Advisory
docker compose -f devops/compose/docker-compose.advisoryai-knowledge-test.yml up -d docker compose -f devops/compose/docker-compose.advisoryai-knowledge-test.yml up -d
# Database at localhost:55432, user: stellaops_knowledge, db: advisoryai_knowledge_test # Database at localhost:55432, user: stellaops_knowledge, db: advisoryai_knowledge_test
# Requires extensions: pgvector, pg_trgm (auto-created by init script) # Requires extensions: pgvector, pg_trgm (auto-created by init script)
stella advisoryai sources prepare --json export STELLAOPS_BACKEND_URL="http://127.0.0.1:10451"
stella advisoryai index rebuild --json dotnet run --project "src/Cli/StellaOps.Cli/StellaOps.Cli.csproj" -- advisoryai sources prepare --json
curl -X POST http://127.0.0.1:10451/v1/advisory-ai/index/rebuild \
-H "X-StellaOps-Scopes: advisory-ai:admin" \
-H "X-StellaOps-Tenant: test-tenant" \
-H "X-StellaOps-Actor: advisoryai-agent"
curl -X POST http://127.0.0.1:10451/v1/search/index/rebuild \
-H "X-StellaOps-Scopes: advisory-ai:admin" \
-H "X-StellaOps-Tenant: test-tenant" \
-H "X-StellaOps-Actor: advisoryai-agent"
dotnet test src/AdvisoryAI/__Tests/StellaOps.AdvisoryAI.Tests/StellaOps.AdvisoryAI.Tests.csproj \ dotnet test src/AdvisoryAI/__Tests/StellaOps.AdvisoryAI.Tests/StellaOps.AdvisoryAI.Tests.csproj \
--filter "Category=Live" -v normal --filter "Category=Live" -v normal
``` ```

22
src/AdvisoryAI/StellaOps.AdvisoryAI/KnowledgeSearch/KnowledgeIndexer.cs
View File

@@ -82,11 +82,23 @@ internal sealed class KnowledgeIndexer : IKnowledgeIndexer
private EffectiveIngestionOptions ResolveEffectiveOptions() private EffectiveIngestionOptions ResolveEffectiveOptions()
{ {
var repositoryRoot = string.IsNullOrWhiteSpace(_options.RepositoryRoot) var repositoryRootResolution = KnowledgeSearchRepositoryRootResolver.Resolve(_options);
? Directory.GetCurrentDirectory() if (!repositoryRootResolution.Validated)
: Path.IsPathRooted(_options.RepositoryRoot) {
? Path.GetFullPath(_options.RepositoryRoot) _logger.LogWarning(
: Path.GetFullPath(Path.Combine(Directory.GetCurrentDirectory(), _options.RepositoryRoot)); "Knowledge ingestion could not validate repository root from source {Source}; using {RepositoryRoot}. Relative corpus paths may resolve to an empty index.",
repositoryRootResolution.Source,
repositoryRootResolution.Path);
}
else if (!string.Equals(repositoryRootResolution.Source, "configured", StringComparison.Ordinal))
{
_logger.LogInformation(
"Auto-detected AdvisoryAI repository root from {Source}: {RepositoryRoot}.",
repositoryRootResolution.Source,
repositoryRootResolution.Path);
}
var repositoryRoot = repositoryRootResolution.Path;
var markdownRoots = (_options.MarkdownRoots ?? []) var markdownRoots = (_options.MarkdownRoots ?? [])
.Where(static root => !string.IsNullOrWhiteSpace(root)) .Where(static root => !string.IsNullOrWhiteSpace(root))

15
src/AdvisoryAI/StellaOps.AdvisoryAI/KnowledgeSearch/KnowledgeSearchBenchmarkDatasetGenerator.cs
View File

@@ -120,17 +120,16 @@ internal sealed class KnowledgeSearchBenchmarkDatasetGenerator : IKnowledgeSearc
private string ResolveRepositoryRoot() private string ResolveRepositoryRoot()
{ {
if (string.IsNullOrWhiteSpace(_options.RepositoryRoot)) var resolution = KnowledgeSearchRepositoryRootResolver.Resolve(_options);
if (!resolution.Validated)
{ {
return Directory.GetCurrentDirectory(); _logger.LogWarning(
"Knowledge benchmark dataset generation could not validate repository root from source {Source}; using {RepositoryRoot}.",
resolution.Source,
resolution.Path);
} }
if (Path.IsPathRooted(_options.RepositoryRoot)) return resolution.Path;
{
return Path.GetFullPath(_options.RepositoryRoot);
}
return Path.GetFullPath(Path.Combine(Directory.GetCurrentDirectory(), _options.RepositoryRoot));
} }
private IReadOnlyList<BenchmarkTarget> LoadMarkdownTargets(string repositoryRoot, CancellationToken cancellationToken) private IReadOnlyList<BenchmarkTarget> LoadMarkdownTargets(string repositoryRoot, CancellationToken cancellationToken)

6
src/AdvisoryAI/StellaOps.AdvisoryAI/KnowledgeSearch/KnowledgeSearchOptions.cs
View File

@@ -55,13 +55,13 @@ public sealed class KnowledgeSearchOptions
public List<string> OpenApiRoots { get; set; } = ["src", "devops/compose"]; public List<string> OpenApiRoots { get; set; } = ["src", "devops/compose"];
public string UnifiedFindingsSnapshotPath { get; set; } = public string UnifiedFindingsSnapshotPath { get; set; } =
"UnifiedSearch/Snapshots/findings.snapshot.json"; "src/AdvisoryAI/StellaOps.AdvisoryAI/UnifiedSearch/Snapshots/findings.snapshot.json";
public string UnifiedVexSnapshotPath { get; set; } = public string UnifiedVexSnapshotPath { get; set; } =
"UnifiedSearch/Snapshots/vex.snapshot.json"; "src/AdvisoryAI/StellaOps.AdvisoryAI/UnifiedSearch/Snapshots/vex.snapshot.json";
public string UnifiedPolicySnapshotPath { get; set; } = public string UnifiedPolicySnapshotPath { get; set; } =
"UnifiedSearch/Snapshots/policy.snapshot.json"; "src/AdvisoryAI/StellaOps.AdvisoryAI/UnifiedSearch/Snapshots/policy.snapshot.json";
public bool UnifiedAutoIndexEnabled { get; set; } public bool UnifiedAutoIndexEnabled { get; set; }

195
src/AdvisoryAI/StellaOps.AdvisoryAI/KnowledgeSearch/KnowledgeSearchRepositoryRootResolver.cs Normal file
View File

@@ -0,0 +1,195 @@
namespace StellaOps.AdvisoryAI.KnowledgeSearch;
internal sealed record KnowledgeSearchRepositoryRootResolution(
string Path,
bool Validated,
string Source);
internal static class KnowledgeSearchRepositoryRootResolver
{
public static string ResolvePath(KnowledgeSearchOptions options, string configuredPath)
{
ArgumentNullException.ThrowIfNull(options);
ArgumentException.ThrowIfNullOrWhiteSpace(configuredPath);
if (Path.IsPathRooted(configuredPath))
{
return Path.GetFullPath(configuredPath);
}
var repositoryRoot = Resolve(options);
return Path.GetFullPath(Path.Combine(repositoryRoot.Path, configuredPath));
}
public static KnowledgeSearchRepositoryRootResolution Resolve(KnowledgeSearchOptions options)
{
return Resolve(options, Directory.GetCurrentDirectory(), AppContext.BaseDirectory);
}
internal static KnowledgeSearchRepositoryRootResolution Resolve(
KnowledgeSearchOptions options,
string? currentDirectory,
string? appBaseDirectory)
{
ArgumentNullException.ThrowIfNull(options);
var configuredRoot = NormalizeConfiguredRoot(options.RepositoryRoot, currentDirectory);
var relativeMarkers = BuildRelativeMarkers(options);
foreach (var (candidate, source, allowAncestorSearch) in EnumerateCandidates(configuredRoot, currentDirectory, appBaseDirectory))
{
var resolved = FindRepositoryRoot(candidate, relativeMarkers, allowAncestorSearch);
if (resolved is null)
{
continue;
}
return new KnowledgeSearchRepositoryRootResolution(resolved, true, source);
}
if (!string.IsNullOrWhiteSpace(configuredRoot))
{
return new KnowledgeSearchRepositoryRootResolution(configuredRoot, false, "configured_fallback");
}
var fallbackPath =
NormalizeAbsolutePath(currentDirectory)
?? NormalizeAbsolutePath(appBaseDirectory)
?? Path.GetFullPath(Directory.GetCurrentDirectory());
return new KnowledgeSearchRepositoryRootResolution(fallbackPath, false, "current_directory_fallback");
}
private static IEnumerable<(string Candidate, string Source, bool AllowAncestorSearch)> EnumerateCandidates(
string? configuredRoot,
string? currentDirectory,
string? appBaseDirectory)
{
var seen = new HashSet<string>(StringComparer.OrdinalIgnoreCase);
if (!string.IsNullOrWhiteSpace(configuredRoot) && seen.Add(configuredRoot))
{
yield return (configuredRoot, "configured", false);
}
var normalizedCurrentDirectory = NormalizeAbsolutePath(currentDirectory);
if (!string.IsNullOrWhiteSpace(normalizedCurrentDirectory) && seen.Add(normalizedCurrentDirectory))
{
yield return (normalizedCurrentDirectory, "current_directory", true);
}
var normalizedAppBaseDirectory = NormalizeAbsolutePath(appBaseDirectory);
if (!string.IsNullOrWhiteSpace(normalizedAppBaseDirectory) && seen.Add(normalizedAppBaseDirectory))
{
yield return (normalizedAppBaseDirectory, "app_base_directory", true);
}
}
private static string? FindRepositoryRoot(string? startPath, IReadOnlyList<string> relativeMarkers, bool allowAncestorSearch)
{
var normalizedStartPath = NormalizeAbsolutePath(startPath);
if (string.IsNullOrWhiteSpace(normalizedStartPath))
{
return null;
}
if (!allowAncestorSearch)
{
return LooksLikeRepositoryRoot(normalizedStartPath, relativeMarkers)
? normalizedStartPath
: null;
}
var current = new DirectoryInfo(normalizedStartPath);
while (current is not null)
{
if (LooksLikeRepositoryRoot(current.FullName, relativeMarkers))
{
return current.FullName;
}
current = current.Parent;
}
return null;
}
private static bool LooksLikeRepositoryRoot(string candidate, IReadOnlyList<string> relativeMarkers)
{
if (!Directory.Exists(candidate))
{
return false;
}
var hasRepoShape =
File.Exists(Path.Combine(candidate, "global.json"))
|| (Directory.Exists(Path.Combine(candidate, "src")) && Directory.Exists(Path.Combine(candidate, "docs")));
if (!hasRepoShape)
{
return false;
}
foreach (var marker in relativeMarkers)
{
var markerPath = Path.Combine(candidate, marker);
if (Directory.Exists(markerPath) || File.Exists(markerPath))
{
return true;
}
}
return false;
}
private static string? NormalizeConfiguredRoot(string? configuredRoot, string? currentDirectory)
{
if (string.IsNullOrWhiteSpace(configuredRoot))
{
return null;
}
if (Path.IsPathRooted(configuredRoot))
{
return Path.GetFullPath(configuredRoot);
}
var baseDirectory = NormalizeAbsolutePath(currentDirectory) ?? Path.GetFullPath(Directory.GetCurrentDirectory());
return Path.GetFullPath(Path.Combine(baseDirectory, configuredRoot));
}
private static List<string> BuildRelativeMarkers(KnowledgeSearchOptions options)
{
var markers = new HashSet<string>(StringComparer.OrdinalIgnoreCase);
AddRelativeMarker(markers, options.MarkdownAllowListPath);
AddRelativeMarker(markers, options.DoctorSeedPath);
AddRelativeMarker(markers, options.DoctorControlsPath);
AddRelativeMarker(markers, options.OpenApiAggregatePath);
AddRelativeMarker(markers, "docs");
AddRelativeMarker(markers, "src");
AddRelativeMarker(markers, "devops");
return markers.ToList();
}
private static void AddRelativeMarker(ISet<string> markers, string? value)
{
if (string.IsNullOrWhiteSpace(value) || Path.IsPathRooted(value))
{
return;
}
markers.Add(value.Trim());
}
private static string? NormalizeAbsolutePath(string? path)
{
if (string.IsNullOrWhiteSpace(path))
{
return null;
}
return Path.GetFullPath(path);
}
}

2
src/AdvisoryAI/StellaOps.AdvisoryAI/TASKS.md
View File

@@ -17,7 +17,7 @@ Source of truth: `docs/implplan/SPRINT_20260113_005_ADVISORYAI_controlled_conver
| AI-SELF-006 | DONE | Live ingestion-backed answer verification succeeded on the Doctor/knowledge route after local rebuild. | | AI-SELF-006 | DONE | Live ingestion-backed answer verification succeeded on the Doctor/knowledge route after local rebuild. |
| SPRINT_20260307_019-AI-ZL | DONE | Unified search now applies implicit current-scope weighting, emits additive `overflow`/`coverage`, blends close top answers, and evaluates suggestion viability without requiring telemetry. | | SPRINT_20260307_019-AI-ZL | DONE | Unified search now applies implicit current-scope weighting, emits additive `overflow`/`coverage`, blends close top answers, and evaluates suggestion viability without requiring telemetry. |
| SPRINT_20260307_033-AI-ZL | DONE | Unified search now derives answer blending from query/context, exposes grounded-only suggestion viability with corpus-readiness states, and keeps analytics/feedback telemetry fully optional. | | SPRINT_20260307_033-AI-ZL | DONE | Unified search now derives answer blending from query/context, exposes grounded-only suggestion viability with corpus-readiness states, and keeps analytics/feedback telemetry fully optional. |
| SPRINT_20260307_037-AI-SF | TODO | Final search-first correction pass: stronger in-scope weighting, stricter blended-answer thresholds, grounded-only supported-route suggestion viability, and telemetry-independent correctness. | | SPRINT_20260307_037-AI-SF | DONE | Final search-first correction pass: stronger in-scope weighting, stricter blended-answer thresholds, grounded-only supported-route suggestion viability, and telemetry-independent correctness. |
| SPRINT_20260222_051-AKS-INGEST | DONE | Added deterministic AKS ingestion controls: markdown allow-list manifest loading, OpenAPI aggregate source path support, and doctor control projection integration for search chunks, including fallback doctor metadata hydration from controls projection fields. | | SPRINT_20260222_051-AKS-INGEST | DONE | Added deterministic AKS ingestion controls: markdown allow-list manifest loading, OpenAPI aggregate source path support, and doctor control projection integration for search chunks, including fallback doctor metadata hydration from controls projection fields. |
| AUDIT-0017-M | DONE | Maintainability audit for StellaOps.AdvisoryAI. | | AUDIT-0017-M | DONE | Maintainability audit for StellaOps.AdvisoryAI. |
| AUDIT-0017-T | DONE | Test coverage audit for StellaOps.AdvisoryAI. | | AUDIT-0017-T | DONE | Test coverage audit for StellaOps.AdvisoryAI. |

81
src/AdvisoryAI/StellaOps.AdvisoryAI/UnifiedSearch/Adapters/FindingIngestionAdapter.cs
View File

@@ -68,9 +68,18 @@ internal sealed class FindingIngestionAdapter : ISearchIngestionAdapter
var tenant = ReadString(entry, "tenant") ?? "global"; var tenant = ReadString(entry, "tenant") ?? "global";
var tags = ReadStringArray(entry, "tags", ["finding", "vulnerability", severity]); var tags = ReadStringArray(entry, "tags", ["finding", "vulnerability", severity]);
var body = string.IsNullOrWhiteSpace(description) var body = BuildBody(
? $"{title}\nSeverity: {severity}" findingId,
: $"{title}\n{description}\nSeverity: {severity}"; cveId,
title,
description,
severity,
service,
reachability: null,
environment: null,
product: null,
policyBadge: null,
tags);
var chunkId = KnowledgeSearchText.StableId("chunk", "finding", findingId, cveId); var chunkId = KnowledgeSearchText.StableId("chunk", "finding", findingId, cveId);
var docId = KnowledgeSearchText.StableId("doc", "finding", findingId); var docId = KnowledgeSearchText.StableId("doc", "finding", findingId);
var embedding = _vectorEncoder.Encode(body); var embedding = _vectorEncoder.Encode(body);
@@ -116,15 +125,71 @@ internal sealed class FindingIngestionAdapter : ISearchIngestionAdapter
})); }));
} }
private string ResolvePath(string configuredPath) private static string BuildBody(
string findingId,
string cveId,
string title,
string description,
string severity,
string service,
string? reachability,
string? environment,
string? product,
string? policyBadge,
IReadOnlyList<string> tags)
{ {
if (Path.IsPathRooted(configuredPath)) var bodyParts = new List<string>
{ {
return configuredPath; $"Finding: {title}",
$"Finding ID: {findingId}",
$"CVE: {cveId}",
"Finding type: vulnerability finding",
$"Severity: {severity}",
$"Reachable: {NormalizeTextOrDefault(reachability, "unknown")}",
"Status: open unresolved"
};
if (!string.IsNullOrWhiteSpace(product))
{
bodyParts.Add($"Product: {product}");
} }
var root = string.IsNullOrWhiteSpace(_options.RepositoryRoot) ? "." : _options.RepositoryRoot; if (!string.IsNullOrWhiteSpace(environment))
return Path.GetFullPath(Path.Combine(root, configuredPath)); {
bodyParts.Add($"Environment: {environment}");
}
if (!string.IsNullOrWhiteSpace(policyBadge))
{
bodyParts.Add($"Policy gate: {policyBadge}");
}
if (!string.IsNullOrWhiteSpace(service))
{
bodyParts.Add($"Service: {service}");
}
if (tags.Count > 0)
{
bodyParts.Add($"Tags: {string.Join(", ", tags)}");
}
if (!string.IsNullOrWhiteSpace(description))
{
bodyParts.Add(description);
}
return string.Join("\n", bodyParts);
}
private static string NormalizeTextOrDefault(string? value, string fallback)
{
return string.IsNullOrWhiteSpace(value) ? fallback : value.Trim();
}
private string ResolvePath(string configuredPath)
{
return KnowledgeSearchRepositoryRootResolver.ResolvePath(_options, configuredPath);
} }
private static string? ReadString(JsonElement obj, string propertyName) private static string? ReadString(JsonElement obj, string propertyName)

126
src/AdvisoryAI/StellaOps.AdvisoryAI/UnifiedSearch/Adapters/FindingsSearchAdapter.cs
View File

@@ -181,23 +181,19 @@ internal sealed class FindingsSearchAdapter : ISearchIngestionAdapter
? $"{cveId} [{severity}]" ? $"{cveId} [{severity}]"
: $"{cveId} - {component} [{severity}]"; : $"{cveId} - {component} [{severity}]";
var bodyParts = new List<string> { title }; var body = BuildBody(
if (!string.IsNullOrWhiteSpace(description)) findingId,
{ cveId,
bodyParts.Add(description); title,
} description,
if (!string.IsNullOrWhiteSpace(reachability)) severity,
{ "scanner",
bodyParts.Add($"Reachability: {reachability}"); reachability,
} environment,
if (!string.IsNullOrWhiteSpace(environment)) product,
{ policyBadge,
bodyParts.Add($"Environment: {environment}"); tags,
} status: ReadString(entry, "status") ?? ReadString(entry, "Status"));
bodyParts.Add($"Severity: {severity}");
var body = string.Join("\n", bodyParts);
// Scope ids by tenant to prevent cross-tenant overwrite collisions // Scope ids by tenant to prevent cross-tenant overwrite collisions
// when different tenants have identical finding ids/cve pairs. // when different tenants have identical finding ids/cve pairs.
var chunkId = KnowledgeSearchText.StableId("chunk", "finding", tenantIdentity, findingId, cveId); var chunkId = KnowledgeSearchText.StableId("chunk", "finding", tenantIdentity, findingId, cveId);
@@ -279,9 +275,19 @@ internal sealed class FindingsSearchAdapter : ISearchIngestionAdapter
var tenantIdentity = NormalizeTenantForIdentity(tenant); var tenantIdentity = NormalizeTenantForIdentity(tenant);
var tags = ReadStringArray(entry, "tags", ["finding", "vulnerability", severity]); var tags = ReadStringArray(entry, "tags", ["finding", "vulnerability", severity]);
var body = string.IsNullOrWhiteSpace(description) var body = BuildBody(
? $"{title}\nSeverity: {severity}" findingId,
: $"{title}\n{description}\nSeverity: {severity}"; cveId,
title,
description,
severity,
service,
reachability: null,
environment: null,
product: service,
policyBadge: null,
tags,
status: null);
// Scope ids by tenant to prevent cross-tenant overwrite collisions // Scope ids by tenant to prevent cross-tenant overwrite collisions
// when different tenants have identical finding ids/cve pairs. // when different tenants have identical finding ids/cve pairs.
var chunkId = KnowledgeSearchText.StableId("chunk", "finding", tenantIdentity, findingId, cveId); var chunkId = KnowledgeSearchText.StableId("chunk", "finding", tenantIdentity, findingId, cveId);
@@ -331,15 +337,85 @@ internal sealed class FindingsSearchAdapter : ISearchIngestionAdapter
})); }));
} }
private string ResolvePath(string configuredPath) private static string BuildBody(
string findingId,
string cveId,
string title,
string description,
string severity,
string service,
string? reachability,
string? environment,
string? product,
string? policyBadge,
IReadOnlyList<string> tags,
string? status)
{ {
if (Path.IsPathRooted(configuredPath)) var bodyParts = new List<string>
{ {
return configuredPath; $"Finding: {title}",
$"Finding ID: {findingId}",
$"CVE: {cveId}",
"Finding type: vulnerability finding",
$"Severity: {severity}",
$"Reachable: {NormalizeTextOrDefault(reachability, "unknown")}",
$"Status: {ResolveFindingStatus(status)}"
};
if (!string.IsNullOrWhiteSpace(product))
{
bodyParts.Add($"Product: {product}");
} }
var root = string.IsNullOrWhiteSpace(_options.RepositoryRoot) ? "." : _options.RepositoryRoot; if (!string.IsNullOrWhiteSpace(environment))
return Path.GetFullPath(Path.Combine(root, configuredPath)); {
bodyParts.Add($"Environment: {environment}");
}
if (!string.IsNullOrWhiteSpace(policyBadge))
{
bodyParts.Add($"Policy gate: {policyBadge}");
}
if (!string.IsNullOrWhiteSpace(service))
{
bodyParts.Add($"Service: {service}");
}
if (tags.Count > 0)
{
bodyParts.Add($"Tags: {string.Join(", ", tags)}");
}
if (!string.IsNullOrWhiteSpace(description))
{
bodyParts.Add(description);
}
return string.Join("\n", bodyParts);
}
private static string ResolveFindingStatus(string? status)
{
if (string.IsNullOrWhiteSpace(status))
{
return "open unresolved";
}
var normalized = status.Trim().Replace('_', ' ');
return normalized.Contains("open", StringComparison.OrdinalIgnoreCase)
? normalized
: $"{normalized} unresolved";
}
private static string NormalizeTextOrDefault(string? value, string fallback)
{
return string.IsNullOrWhiteSpace(value) ? fallback : value.Trim();
}
private string ResolvePath(string configuredPath)
{
return KnowledgeSearchRepositoryRootResolver.ResolvePath(_options, configuredPath);
} }
private static string? ReadString(JsonElement obj, string propertyName) private static string? ReadString(JsonElement obj, string propertyName)

92
src/AdvisoryAI/StellaOps.AdvisoryAI/UnifiedSearch/Adapters/PolicyRuleIngestionAdapter.cs
View File

@@ -67,9 +67,15 @@ internal sealed class PolicyRuleIngestionAdapter : ISearchIngestionAdapter
var tenant = ReadString(entry, "tenant") ?? "global"; var tenant = ReadString(entry, "tenant") ?? "global";
var tags = ReadStringArray(entry, "tags", ["policy", "rule"]); var tags = ReadStringArray(entry, "tags", ["policy", "rule"]);
var body = string.IsNullOrWhiteSpace(decision) var body = BuildBody(
? $"{title}\nRule: {ruleId}\n{description}" ruleId,
: $"{title}\nRule: {ruleId}\nDecision: {decision}\n{description}"; title,
description,
decision,
service,
scope: null,
environment: null,
tags);
var chunkId = KnowledgeSearchText.StableId("chunk", "policy_rule", ruleId); var chunkId = KnowledgeSearchText.StableId("chunk", "policy_rule", ruleId);
var docId = KnowledgeSearchText.StableId("doc", "policy_rule", ruleId); var docId = KnowledgeSearchText.StableId("doc", "policy_rule", ruleId);
var embedding = _vectorEncoder.Encode(body); var embedding = _vectorEncoder.Encode(body);
@@ -113,15 +119,85 @@ internal sealed class PolicyRuleIngestionAdapter : ISearchIngestionAdapter
})); }));
} }
private string ResolvePath(string configuredPath) private static string BuildBody(
string ruleId,
string title,
string description,
string? decision,
string service,
string? scope,
string? environment,
IReadOnlyList<string> tags)
{ {
if (Path.IsPathRooted(configuredPath)) var normalizedDecision = NormalizeTextOrDefault(decision, "unknown");
var bodyParts = new List<string>
{ {
return configuredPath; $"Policy rule: {title}",
$"Rule ID: {ruleId}",
$"Gate decision: {normalizedDecision}",
$"Gate state: {ResolveGateState(normalizedDecision)}",
"Policy domain: policy rule gate"
};
if (!string.IsNullOrWhiteSpace(scope))
{
bodyParts.Add($"Scope: {scope}");
} }
var root = string.IsNullOrWhiteSpace(_options.RepositoryRoot) ? "." : _options.RepositoryRoot; if (!string.IsNullOrWhiteSpace(environment))
return Path.GetFullPath(Path.Combine(root, configuredPath)); {
bodyParts.Add($"Environment: {environment}");
}
if (!string.IsNullOrWhiteSpace(service))
{
bodyParts.Add($"Service: {service}");
}
if (tags.Count > 0)
{
bodyParts.Add($"Tags: {string.Join(", ", tags)}");
}
if (!string.IsNullOrWhiteSpace(description))
{
bodyParts.Add(description);
}
return string.Join("\n", bodyParts);
}
private static string ResolveGateState(string decision)
{
if (decision.Contains("deny", StringComparison.OrdinalIgnoreCase) ||
decision.Contains("block", StringComparison.OrdinalIgnoreCase))
{
return "failing blocking";
}
if (decision.Contains("warn", StringComparison.OrdinalIgnoreCase))
{
return "warning advisory";
}
if (decision.Contains("pass", StringComparison.OrdinalIgnoreCase) ||
decision.Contains("allow", StringComparison.OrdinalIgnoreCase) ||
decision.Contains("require", StringComparison.OrdinalIgnoreCase))
{
return "passing";
}
return "unknown";
}
private static string NormalizeTextOrDefault(string? value, string fallback)
{
return string.IsNullOrWhiteSpace(value) ? fallback : value.Trim();
}
private string ResolvePath(string configuredPath)
{
return KnowledgeSearchRepositoryRootResolver.ResolvePath(_options, configuredPath);
} }
private static string? ReadString(JsonElement obj, string propertyName) private static string? ReadString(JsonElement obj, string propertyName)

135
src/AdvisoryAI/StellaOps.AdvisoryAI/UnifiedSearch/Adapters/PolicySearchAdapter.cs
View File

@@ -191,21 +191,17 @@ internal sealed class PolicySearchAdapter : ISearchIngestionAdapter
? $"{ruleId} [{enforcement}]" ? $"{ruleId} [{enforcement}]"
: $"{ruleId} - {bomRef} [{enforcement}]"; : $"{ruleId} - {bomRef} [{enforcement}]";
var bodyParts = new List<string> { title, $"Rule: {ruleId}", $"Enforcement: {enforcement}" }; var body = BuildBody(
if (!string.IsNullOrWhiteSpace(description)) ruleId,
{ title,
bodyParts.Add(description); description,
} enforcement,
if (!string.IsNullOrWhiteSpace(bomRef)) scope,
{ environment,
bodyParts.Add($"Scope: {bomRef}"); verdictHash,
} ciContext,
if (!string.IsNullOrWhiteSpace(verdictHash)) actor,
{ tags);
bodyParts.Add($"Verdict: {verdictHash}");
}
var body = string.Join("\n", bodyParts);
// Scope ids by tenant to prevent cross-tenant overwrite collisions // Scope ids by tenant to prevent cross-tenant overwrite collisions
// when rule ids are reused in different tenants. // when rule ids are reused in different tenants.
var chunkId = KnowledgeSearchText.StableId("chunk", "policy_rule", tenantIdentity, ruleId); var chunkId = KnowledgeSearchText.StableId("chunk", "policy_rule", tenantIdentity, ruleId);
@@ -289,9 +285,17 @@ internal sealed class PolicySearchAdapter : ISearchIngestionAdapter
var tenantIdentity = NormalizeTenantForIdentity(tenant); var tenantIdentity = NormalizeTenantForIdentity(tenant);
var tags = ReadStringArray(entry, "tags", ["policy", "rule"]); var tags = ReadStringArray(entry, "tags", ["policy", "rule"]);
var body = string.IsNullOrWhiteSpace(decision) var body = BuildBody(
? $"{title}\nRule: {ruleId}\n{description}" ruleId,
: $"{title}\nRule: {ruleId}\nDecision: {decision}\n{description}"; title,
description,
decision,
scope: string.Empty,
environment: string.Empty,
verdictHash: string.Empty,
ciContext: string.Empty,
actor: string.Empty,
tags);
// Scope ids by tenant to prevent cross-tenant overwrite collisions // Scope ids by tenant to prevent cross-tenant overwrite collisions
// when rule ids are reused in different tenants. // when rule ids are reused in different tenants.
var chunkId = KnowledgeSearchText.StableId("chunk", "policy_rule", tenantIdentity, ruleId); var chunkId = KnowledgeSearchText.StableId("chunk", "policy_rule", tenantIdentity, ruleId);
@@ -339,15 +343,100 @@ internal sealed class PolicySearchAdapter : ISearchIngestionAdapter
})); }));
} }
private string ResolvePath(string configuredPath) private static string BuildBody(
string ruleId,
string title,
string description,
string? decision,
string? scope,
string? environment,
string? verdictHash,
string? ciContext,
string? actor,
IReadOnlyList<string> tags)
{ {
if (Path.IsPathRooted(configuredPath)) var normalizedDecision = NormalizeTextOrDefault(decision, "unknown");
var bodyParts = new List<string>
{ {
return configuredPath; $"Policy rule: {title}",
$"Rule ID: {ruleId}",
$"Gate decision: {normalizedDecision}",
$"Gate state: {ResolveGateState(normalizedDecision)}",
"Policy domain: policy rule gate"
};
if (!string.IsNullOrWhiteSpace(scope))
{
bodyParts.Add($"Scope: {scope}");
} }
var root = string.IsNullOrWhiteSpace(_options.RepositoryRoot) ? "." : _options.RepositoryRoot; if (!string.IsNullOrWhiteSpace(environment))
return Path.GetFullPath(Path.Combine(root, configuredPath)); {
bodyParts.Add($"Environment: {environment}");
}
if (!string.IsNullOrWhiteSpace(verdictHash))
{
bodyParts.Add($"Verdict: {verdictHash}");
}
if (!string.IsNullOrWhiteSpace(ciContext))
{
bodyParts.Add($"CI context: {ciContext}");
}
if (!string.IsNullOrWhiteSpace(actor))
{
bodyParts.Add($"Actor: {actor}");
}
if (tags.Count > 0)
{
bodyParts.Add($"Tags: {string.Join(", ", tags)}");
}
if (!string.IsNullOrWhiteSpace(description))
{
bodyParts.Add(description);
}
return string.Join("\n", bodyParts);
}
private static string ResolveGateState(string decision)
{
if (decision.Contains("deny", StringComparison.OrdinalIgnoreCase) ||
decision.Contains("block", StringComparison.OrdinalIgnoreCase) ||
decision.Contains("mandatory", StringComparison.OrdinalIgnoreCase))
{
return "failing blocking";
}
if (decision.Contains("warn", StringComparison.OrdinalIgnoreCase) ||
decision.Contains("advisory", StringComparison.OrdinalIgnoreCase))
{
return "warning advisory";
}
if (decision.Contains("pass", StringComparison.OrdinalIgnoreCase) ||
decision.Contains("allow", StringComparison.OrdinalIgnoreCase) ||
decision.Contains("informational", StringComparison.OrdinalIgnoreCase) ||
decision.Contains("require", StringComparison.OrdinalIgnoreCase))
{
return "passing";
}
return "unknown";
}
private static string NormalizeTextOrDefault(string? value, string fallback)
{
return string.IsNullOrWhiteSpace(value) ? fallback : value.Trim();
}
private string ResolvePath(string configuredPath)
{
return KnowledgeSearchRepositoryRootResolver.ResolvePath(_options, configuredPath);
} }
private static string? ReadString(JsonElement obj, string propertyName) private static string? ReadString(JsonElement obj, string propertyName)

92
src/AdvisoryAI/StellaOps.AdvisoryAI/UnifiedSearch/Adapters/VexSearchAdapter.cs
View File

@@ -187,21 +187,16 @@ internal sealed class VexSearchAdapter : ISearchIngestionAdapter
? $"VEX: {cveId} ({status})" ? $"VEX: {cveId} ({status})"
: $"VEX: {cveId} - {product} ({status})"; : $"VEX: {cveId} - {product} ({status})";
var bodyParts = new List<string> { title, $"Status: {status}" }; var body = BuildBody(
if (!string.IsNullOrWhiteSpace(justification)) statementId,
{ cveId,
bodyParts.Add($"Justification: {justification}"); title,
} status,
if (!string.IsNullOrWhiteSpace(advisoryTitle)) justification,
{ product,
bodyParts.Add($"Advisory: {advisoryTitle}"); advisoryTitle,
} severity,
if (!string.IsNullOrWhiteSpace(severity)) tags);
{
bodyParts.Add($"Severity: {severity}");
}
var body = string.Join("\n", bodyParts);
// Scope ids by tenant to prevent cross-tenant overwrite collisions // Scope ids by tenant to prevent cross-tenant overwrite collisions
// when statement ids are reused in different tenants. // when statement ids are reused in different tenants.
var chunkId = KnowledgeSearchText.StableId("chunk", "vex_statement", tenantIdentity, statementId); var chunkId = KnowledgeSearchText.StableId("chunk", "vex_statement", tenantIdentity, statementId);
@@ -283,9 +278,16 @@ internal sealed class VexSearchAdapter : ISearchIngestionAdapter
var tags = ReadStringArray(entry, "tags", ["vex", "statement", status]); var tags = ReadStringArray(entry, "tags", ["vex", "statement", status]);
var title = $"VEX: {cveId} ({status})"; var title = $"VEX: {cveId} ({status})";
var body = string.IsNullOrWhiteSpace(justification) var body = BuildBody(
? $"{title}\nStatus: {status}" statementId,
: $"{title}\nStatus: {status}\nJustification: {justification}"; cveId,
title,
status,
justification,
product: null,
advisoryTitle: null,
severity: null,
tags);
// Scope ids by tenant to prevent cross-tenant overwrite collisions // Scope ids by tenant to prevent cross-tenant overwrite collisions
// when statement ids are reused in different tenants. // when statement ids are reused in different tenants.
var chunkId = KnowledgeSearchText.StableId("chunk", "vex_statement", tenantIdentity, statementId); var chunkId = KnowledgeSearchText.StableId("chunk", "vex_statement", tenantIdentity, statementId);
@@ -333,15 +335,59 @@ internal sealed class VexSearchAdapter : ISearchIngestionAdapter
})); }));
} }
private string ResolvePath(string configuredPath) private static string BuildBody(
string statementId,
string cveId,
string title,
string status,
string justification,
string? product,
string? advisoryTitle,
string? severity,
IReadOnlyList<string> tags)
{ {
if (Path.IsPathRooted(configuredPath)) var statusLabel = status.Replace('_', ' ');
var bodyParts = new List<string>
{ {
return configuredPath; $"VEX statement: {title}",
$"Statement ID: {statementId}",
$"CVE: {cveId}",
$"Disposition: {statusLabel}",
$"Marked as: {statusLabel}",
"Conflict evidence: none recorded"
};
if (!string.IsNullOrWhiteSpace(product))
{
bodyParts.Add($"Covered component: {product}");
} }
var root = string.IsNullOrWhiteSpace(_options.RepositoryRoot) ? "." : _options.RepositoryRoot; if (!string.IsNullOrWhiteSpace(advisoryTitle))
return Path.GetFullPath(Path.Combine(root, configuredPath)); {
bodyParts.Add($"Advisory: {advisoryTitle}");
}
if (!string.IsNullOrWhiteSpace(severity))
{
bodyParts.Add($"Severity: {severity}");
}
if (tags.Count > 0)
{
bodyParts.Add($"Tags: {string.Join(", ", tags.Select(static tag => tag.Replace('_', ' ')))}");
}
if (!string.IsNullOrWhiteSpace(justification))
{
bodyParts.Add($"Justification: {justification}");
}
return string.Join("\n", bodyParts);
}
private string ResolvePath(string configuredPath)
{
return KnowledgeSearchRepositoryRootResolver.ResolvePath(_options, configuredPath);
} }
private static string? ReadString(JsonElement obj, string propertyName) private static string? ReadString(JsonElement obj, string propertyName)

53
src/AdvisoryAI/StellaOps.AdvisoryAI/UnifiedSearch/Adapters/VexStatementIngestionAdapter.cs
View File

@@ -68,9 +68,14 @@ internal sealed class VexStatementIngestionAdapter : ISearchIngestionAdapter
var tags = ReadStringArray(entry, "tags", ["vex", "statement", status]); var tags = ReadStringArray(entry, "tags", ["vex", "statement", status]);
var title = $"VEX: {cveId} ({status})"; var title = $"VEX: {cveId} ({status})";
var body = string.IsNullOrWhiteSpace(justification) var body = BuildBody(
? $"{title}\nStatus: {status}" statementId,
: $"{title}\nStatus: {status}\nJustification: {justification}"; cveId,
title,
status,
justification,
product: null,
tags);
var chunkId = KnowledgeSearchText.StableId("chunk", "vex_statement", statementId); var chunkId = KnowledgeSearchText.StableId("chunk", "vex_statement", statementId);
var docId = KnowledgeSearchText.StableId("doc", "vex_statement", cveId); var docId = KnowledgeSearchText.StableId("doc", "vex_statement", cveId);
var embedding = _vectorEncoder.Encode(body); var embedding = _vectorEncoder.Encode(body);
@@ -116,15 +121,47 @@ internal sealed class VexStatementIngestionAdapter : ISearchIngestionAdapter
})); }));
} }
private string ResolvePath(string configuredPath) private static string BuildBody(
string statementId,
string cveId,
string title,
string status,
string justification,
string? product,
IReadOnlyList<string> tags)
{ {
if (Path.IsPathRooted(configuredPath)) var statusLabel = status.Replace('_', ' ');
var bodyParts = new List<string>
{ {
return configuredPath; $"VEX statement: {title}",
$"Statement ID: {statementId}",
$"CVE: {cveId}",
$"Disposition: {statusLabel}",
$"Marked as: {statusLabel}",
"Conflict evidence: none recorded"
};
if (!string.IsNullOrWhiteSpace(product))
{
bodyParts.Add($"Covered component: {product}");
} }
var root = string.IsNullOrWhiteSpace(_options.RepositoryRoot) ? "." : _options.RepositoryRoot; if (tags.Count > 0)
return Path.GetFullPath(Path.Combine(root, configuredPath)); {
bodyParts.Add($"Tags: {string.Join(", ", tags.Select(static tag => tag.Replace('_', ' ')))}");
}
if (!string.IsNullOrWhiteSpace(justification))
{
bodyParts.Add($"Justification: {justification}");
}
return string.Join("\n", bodyParts);
}
private string ResolvePath(string configuredPath)
{
return KnowledgeSearchRepositoryRootResolver.ResolvePath(_options, configuredPath);
} }
private static string? ReadString(JsonElement obj, string propertyName) private static string? ReadString(JsonElement obj, string propertyName)

21
src/AdvisoryAI/StellaOps.AdvisoryAI/UnifiedSearch/Analytics/SearchAnalyticsService.cs
View File

@@ -349,13 +349,20 @@ internal sealed class SearchAnalyticsService
await conn.OpenAsync(ct).ConfigureAwait(false); await conn.OpenAsync(ct).ConfigureAwait(false);
await using var cmd = new NpgsqlCommand(@" await using var cmd = new NpgsqlCommand(@"
SELECT DISTINCT query SELECT candidate_query
FROM advisoryai.search_history FROM (
WHERE tenant_id = @tenant_id SELECT
AND result_count > 0 lower(query) AS normalized_query,
AND lower(query) <> lower(@query) MIN(query) AS candidate_query,
AND similarity(query, @query) > 0.2 MAX(similarity(query, @query)) AS similarity_score
ORDER BY similarity(query, @query) DESC FROM advisoryai.search_history
WHERE tenant_id = @tenant_id
AND result_count > 0
AND lower(query) <> lower(@query)
AND similarity(query, @query) > 0.2
GROUP BY lower(query)
) AS ranked
ORDER BY similarity_score DESC, candidate_query ASC
LIMIT @limit", conn); LIMIT @limit", conn);
cmd.CommandTimeout = 5; cmd.CommandTimeout = 5;

4
src/AdvisoryAI/StellaOps.AdvisoryAI/UnifiedSearch/Context/AmbientContextProcessor.cs
View File

@@ -2,8 +2,8 @@ namespace StellaOps.AdvisoryAI.UnifiedSearch.Context;
internal sealed class AmbientContextProcessor internal sealed class AmbientContextProcessor
{ {
private const double CurrentRouteBoost = 0.35d; private const double CurrentRouteBoost = 0.85d;
private const double LastActionDomainBoost = 0.15d; private const double LastActionDomainBoost = 0.20d;
private const double VisibleEntityBoost = 0.20d; private const double VisibleEntityBoost = 0.20d;
private const double LastActionEntityBoost = 0.25d; private const double LastActionEntityBoost = 0.25d;
private static readonly (string Prefix, string Domain)[] RouteDomainMappings = private static readonly (string Prefix, string Domain)[] RouteDomainMappings =

241
src/AdvisoryAI/StellaOps.AdvisoryAI/UnifiedSearch/UnifiedSearchIndexer.cs
View File

@@ -5,6 +5,7 @@ using NpgsqlTypes;
using StellaOps.AdvisoryAI.KnowledgeSearch; using StellaOps.AdvisoryAI.KnowledgeSearch;
using System.Text.Json; using System.Text.Json;
using System.Diagnostics; using System.Diagnostics;
using System.Globalization;
using System.Linq; using System.Linq;
namespace StellaOps.AdvisoryAI.UnifiedSearch; namespace StellaOps.AdvisoryAI.UnifiedSearch;
@@ -254,6 +255,7 @@ internal sealed class UnifiedSearchIndexer : IUnifiedSearchIndexer
{ {
await using var dataSource = new NpgsqlDataSourceBuilder(_options.ConnectionString).Build(); await using var dataSource = new NpgsqlDataSourceBuilder(_options.ConnectionString).Build();
await using var connection = await dataSource.OpenConnectionAsync(cancellationToken).ConfigureAwait(false); await using var connection = await dataSource.OpenConnectionAsync(cancellationToken).ConfigureAwait(false);
var hasEmbeddingVectorColumn = await HasEmbeddingVectorColumnAsync(connection, cancellationToken).ConfigureAwait(false);
// Ensure parent documents exist for each unique DocId // Ensure parent documents exist for each unique DocId
var uniqueDocIds = chunks.Select(static c => c.DocId).Distinct(StringComparer.Ordinal).ToArray(); var uniqueDocIds = chunks.Select(static c => c.DocId).Distinct(StringComparer.Ordinal).ToArray();
@@ -263,57 +265,163 @@ internal sealed class UnifiedSearchIndexer : IUnifiedSearchIndexer
await EnsureDocumentExistsAsync(connection, docId, chunk, cancellationToken).ConfigureAwait(false); await EnsureDocumentExistsAsync(connection, docId, chunk, cancellationToken).ConfigureAwait(false);
} }
const string sql = """ var sql = hasEmbeddingVectorColumn
INSERT INTO advisoryai.kb_chunk ? """
( INSERT INTO advisoryai.kb_chunk
chunk_id, doc_id, kind, anchor, section_path, (
span_start, span_end, title, body, body_tsv, chunk_id, doc_id, kind, anchor, section_path,
embedding, metadata, domain, entity_key, entity_type, freshness, span_start, span_end, title, body, body_tsv,
indexed_at body_tsv_en, body_tsv_de, body_tsv_fr, body_tsv_es, body_tsv_ru,
) embedding, embedding_vec, metadata, domain, entity_key, entity_type, freshness,
VALUES indexed_at
( )
@chunk_id, @doc_id, @kind, @anchor, @section_path, VALUES
@span_start, @span_end, @title, @body, (
setweight(to_tsvector('simple', coalesce(@title, '')), 'A') || @chunk_id, @doc_id, @kind, @anchor, @section_path,
setweight(to_tsvector('simple', coalesce(@section_path, '')), 'B') || @span_start, @span_end, @title, @body,
setweight(to_tsvector('simple', coalesce(@body, '')), 'D'), setweight(to_tsvector('simple', coalesce(@title, '')), 'A') ||
@embedding, @metadata::jsonb, @domain, @entity_key, @entity_type, @freshness, setweight(to_tsvector('simple', coalesce(@section_path, '')), 'B') ||
NOW() setweight(to_tsvector('simple', coalesce(@body, '')), 'D'),
) setweight(to_tsvector('english', coalesce(@title, '')), 'A') ||
ON CONFLICT (chunk_id) DO UPDATE SET setweight(to_tsvector('english', coalesce(@section_path, '')), 'B') ||
doc_id = EXCLUDED.doc_id, setweight(to_tsvector('english', coalesce(@body, '')), 'D'),
kind = EXCLUDED.kind, setweight(to_tsvector('german', coalesce(@title, '')), 'A') ||
anchor = EXCLUDED.anchor, setweight(to_tsvector('german', coalesce(@section_path, '')), 'B') ||
section_path = EXCLUDED.section_path, setweight(to_tsvector('german', coalesce(@body, '')), 'D'),
span_start = EXCLUDED.span_start, setweight(to_tsvector('french', coalesce(@title, '')), 'A') ||
span_end = EXCLUDED.span_end, setweight(to_tsvector('french', coalesce(@section_path, '')), 'B') ||
title = EXCLUDED.title, setweight(to_tsvector('french', coalesce(@body, '')), 'D'),
body = EXCLUDED.body, setweight(to_tsvector('spanish', coalesce(@title, '')), 'A') ||
body_tsv = EXCLUDED.body_tsv, setweight(to_tsvector('spanish', coalesce(@section_path, '')), 'B') ||
embedding = EXCLUDED.embedding, setweight(to_tsvector('spanish', coalesce(@body, '')), 'D'),
metadata = EXCLUDED.metadata, setweight(to_tsvector('russian', coalesce(@title, '')), 'A') ||
domain = EXCLUDED.domain, setweight(to_tsvector('russian', coalesce(@section_path, '')), 'B') ||
entity_key = EXCLUDED.entity_key, setweight(to_tsvector('russian', coalesce(@body, '')), 'D'),
entity_type = EXCLUDED.entity_type, @embedding, CAST(@embedding_vector AS vector), @metadata::jsonb, @domain, @entity_key, @entity_type, @freshness,
freshness = EXCLUDED.freshness, NOW()
indexed_at = NOW() )
WHERE advisoryai.kb_chunk.doc_id IS DISTINCT FROM EXCLUDED.doc_id ON CONFLICT (chunk_id) DO UPDATE SET
OR advisoryai.kb_chunk.kind IS DISTINCT FROM EXCLUDED.kind doc_id = EXCLUDED.doc_id,
OR advisoryai.kb_chunk.anchor IS DISTINCT FROM EXCLUDED.anchor kind = EXCLUDED.kind,
OR advisoryai.kb_chunk.section_path IS DISTINCT FROM EXCLUDED.section_path anchor = EXCLUDED.anchor,
OR advisoryai.kb_chunk.span_start IS DISTINCT FROM EXCLUDED.span_start section_path = EXCLUDED.section_path,
OR advisoryai.kb_chunk.span_end IS DISTINCT FROM EXCLUDED.span_end span_start = EXCLUDED.span_start,
OR advisoryai.kb_chunk.title IS DISTINCT FROM EXCLUDED.title span_end = EXCLUDED.span_end,
OR advisoryai.kb_chunk.body IS DISTINCT FROM EXCLUDED.body title = EXCLUDED.title,
OR advisoryai.kb_chunk.body_tsv IS DISTINCT FROM EXCLUDED.body_tsv body = EXCLUDED.body,
OR advisoryai.kb_chunk.embedding IS DISTINCT FROM EXCLUDED.embedding body_tsv = EXCLUDED.body_tsv,
OR advisoryai.kb_chunk.metadata IS DISTINCT FROM EXCLUDED.metadata body_tsv_en = EXCLUDED.body_tsv_en,
OR advisoryai.kb_chunk.domain IS DISTINCT FROM EXCLUDED.domain body_tsv_de = EXCLUDED.body_tsv_de,
OR advisoryai.kb_chunk.entity_key IS DISTINCT FROM EXCLUDED.entity_key body_tsv_fr = EXCLUDED.body_tsv_fr,
OR advisoryai.kb_chunk.entity_type IS DISTINCT FROM EXCLUDED.entity_type body_tsv_es = EXCLUDED.body_tsv_es,
OR advisoryai.kb_chunk.freshness IS DISTINCT FROM EXCLUDED.freshness; body_tsv_ru = EXCLUDED.body_tsv_ru,
"""; embedding = EXCLUDED.embedding,
embedding_vec = EXCLUDED.embedding_vec,
metadata = EXCLUDED.metadata,
domain = EXCLUDED.domain,
entity_key = EXCLUDED.entity_key,
entity_type = EXCLUDED.entity_type,
freshness = EXCLUDED.freshness,
indexed_at = NOW()
WHERE advisoryai.kb_chunk.doc_id IS DISTINCT FROM EXCLUDED.doc_id
OR advisoryai.kb_chunk.kind IS DISTINCT FROM EXCLUDED.kind
OR advisoryai.kb_chunk.anchor IS DISTINCT FROM EXCLUDED.anchor
OR advisoryai.kb_chunk.section_path IS DISTINCT FROM EXCLUDED.section_path
OR advisoryai.kb_chunk.span_start IS DISTINCT FROM EXCLUDED.span_start
OR advisoryai.kb_chunk.span_end IS DISTINCT FROM EXCLUDED.span_end
OR advisoryai.kb_chunk.title IS DISTINCT FROM EXCLUDED.title
OR advisoryai.kb_chunk.body IS DISTINCT FROM EXCLUDED.body
OR advisoryai.kb_chunk.body_tsv IS DISTINCT FROM EXCLUDED.body_tsv
OR advisoryai.kb_chunk.body_tsv_en IS DISTINCT FROM EXCLUDED.body_tsv_en
OR advisoryai.kb_chunk.body_tsv_de IS DISTINCT FROM EXCLUDED.body_tsv_de
OR advisoryai.kb_chunk.body_tsv_fr IS DISTINCT FROM EXCLUDED.body_tsv_fr
OR advisoryai.kb_chunk.body_tsv_es IS DISTINCT FROM EXCLUDED.body_tsv_es
OR advisoryai.kb_chunk.body_tsv_ru IS DISTINCT FROM EXCLUDED.body_tsv_ru
OR advisoryai.kb_chunk.embedding IS DISTINCT FROM EXCLUDED.embedding
OR advisoryai.kb_chunk.embedding_vec IS DISTINCT FROM EXCLUDED.embedding_vec
OR advisoryai.kb_chunk.metadata IS DISTINCT FROM EXCLUDED.metadata
OR advisoryai.kb_chunk.domain IS DISTINCT FROM EXCLUDED.domain
OR advisoryai.kb_chunk.entity_key IS DISTINCT FROM EXCLUDED.entity_key
OR advisoryai.kb_chunk.entity_type IS DISTINCT FROM EXCLUDED.entity_type
OR advisoryai.kb_chunk.freshness IS DISTINCT FROM EXCLUDED.freshness;
"""
: """
INSERT INTO advisoryai.kb_chunk
(
chunk_id, doc_id, kind, anchor, section_path,
span_start, span_end, title, body, body_tsv,
body_tsv_en, body_tsv_de, body_tsv_fr, body_tsv_es, body_tsv_ru,
embedding, metadata, domain, entity_key, entity_type, freshness,
indexed_at
)
VALUES
(
@chunk_id, @doc_id, @kind, @anchor, @section_path,
@span_start, @span_end, @title, @body,
setweight(to_tsvector('simple', coalesce(@title, '')), 'A') ||
setweight(to_tsvector('simple', coalesce(@section_path, '')), 'B') ||
setweight(to_tsvector('simple', coalesce(@body, '')), 'D'),
setweight(to_tsvector('english', coalesce(@title, '')), 'A') ||
setweight(to_tsvector('english', coalesce(@section_path, '')), 'B') ||
setweight(to_tsvector('english', coalesce(@body, '')), 'D'),
setweight(to_tsvector('german', coalesce(@title, '')), 'A') ||
setweight(to_tsvector('german', coalesce(@section_path, '')), 'B') ||
setweight(to_tsvector('german', coalesce(@body, '')), 'D'),
setweight(to_tsvector('french', coalesce(@title, '')), 'A') ||
setweight(to_tsvector('french', coalesce(@section_path, '')), 'B') ||
setweight(to_tsvector('french', coalesce(@body, '')), 'D'),
setweight(to_tsvector('spanish', coalesce(@title, '')), 'A') ||
setweight(to_tsvector('spanish', coalesce(@section_path, '')), 'B') ||
setweight(to_tsvector('spanish', coalesce(@body, '')), 'D'),
setweight(to_tsvector('russian', coalesce(@title, '')), 'A') ||
setweight(to_tsvector('russian', coalesce(@section_path, '')), 'B') ||
setweight(to_tsvector('russian', coalesce(@body, '')), 'D'),
@embedding, @metadata::jsonb, @domain, @entity_key, @entity_type, @freshness,
NOW()
)
ON CONFLICT (chunk_id) DO UPDATE SET
doc_id = EXCLUDED.doc_id,
kind = EXCLUDED.kind,
anchor = EXCLUDED.anchor,
section_path = EXCLUDED.section_path,
span_start = EXCLUDED.span_start,
span_end = EXCLUDED.span_end,
title = EXCLUDED.title,
body = EXCLUDED.body,
body_tsv = EXCLUDED.body_tsv,
body_tsv_en = EXCLUDED.body_tsv_en,
body_tsv_de = EXCLUDED.body_tsv_de,
body_tsv_fr = EXCLUDED.body_tsv_fr,
body_tsv_es = EXCLUDED.body_tsv_es,
body_tsv_ru = EXCLUDED.body_tsv_ru,
embedding = EXCLUDED.embedding,
metadata = EXCLUDED.metadata,
domain = EXCLUDED.domain,
entity_key = EXCLUDED.entity_key,
entity_type = EXCLUDED.entity_type,
freshness = EXCLUDED.freshness,
indexed_at = NOW()
WHERE advisoryai.kb_chunk.doc_id IS DISTINCT FROM EXCLUDED.doc_id
OR advisoryai.kb_chunk.kind IS DISTINCT FROM EXCLUDED.kind
OR advisoryai.kb_chunk.anchor IS DISTINCT FROM EXCLUDED.anchor
OR advisoryai.kb_chunk.section_path IS DISTINCT FROM EXCLUDED.section_path
OR advisoryai.kb_chunk.span_start IS DISTINCT FROM EXCLUDED.span_start
OR advisoryai.kb_chunk.span_end IS DISTINCT FROM EXCLUDED.span_end
OR advisoryai.kb_chunk.title IS DISTINCT FROM EXCLUDED.title
OR advisoryai.kb_chunk.body IS DISTINCT FROM EXCLUDED.body
OR advisoryai.kb_chunk.body_tsv IS DISTINCT FROM EXCLUDED.body_tsv
OR advisoryai.kb_chunk.body_tsv_en IS DISTINCT FROM EXCLUDED.body_tsv_en
OR advisoryai.kb_chunk.body_tsv_de IS DISTINCT FROM EXCLUDED.body_tsv_de
OR advisoryai.kb_chunk.body_tsv_fr IS DISTINCT FROM EXCLUDED.body_tsv_fr
OR advisoryai.kb_chunk.body_tsv_es IS DISTINCT FROM EXCLUDED.body_tsv_es
OR advisoryai.kb_chunk.body_tsv_ru IS DISTINCT FROM EXCLUDED.body_tsv_ru
OR advisoryai.kb_chunk.embedding IS DISTINCT FROM EXCLUDED.embedding
OR advisoryai.kb_chunk.metadata IS DISTINCT FROM EXCLUDED.metadata
OR advisoryai.kb_chunk.domain IS DISTINCT FROM EXCLUDED.domain
OR advisoryai.kb_chunk.entity_key IS DISTINCT FROM EXCLUDED.entity_key
OR advisoryai.kb_chunk.entity_type IS DISTINCT FROM EXCLUDED.entity_type
OR advisoryai.kb_chunk.freshness IS DISTINCT FROM EXCLUDED.freshness;
""";
await using var command = connection.CreateCommand(); await using var command = connection.CreateCommand();
command.CommandText = sql; command.CommandText = sql;
@@ -336,6 +444,11 @@ internal sealed class UnifiedSearchIndexer : IUnifiedSearchIndexer
"embedding", "embedding",
NpgsqlDbType.Array | NpgsqlDbType.Real, NpgsqlDbType.Array | NpgsqlDbType.Real,
chunk.Embedding is null ? Array.Empty<float>() : chunk.Embedding); chunk.Embedding is null ? Array.Empty<float>() : chunk.Embedding);
if (hasEmbeddingVectorColumn)
{
var vectorLiteral = chunk.Embedding is null ? (object)DBNull.Value : BuildVectorLiteral(chunk.Embedding);
command.Parameters.AddWithValue("embedding_vector", vectorLiteral);
}
command.Parameters.AddWithValue("metadata", NpgsqlDbType.Jsonb, chunk.Metadata.RootElement.GetRawText()); command.Parameters.AddWithValue("metadata", NpgsqlDbType.Jsonb, chunk.Metadata.RootElement.GetRawText());
command.Parameters.AddWithValue("domain", chunk.Domain); command.Parameters.AddWithValue("domain", chunk.Domain);
command.Parameters.AddWithValue("entity_key", (object?)chunk.EntityKey ?? DBNull.Value); command.Parameters.AddWithValue("entity_key", (object?)chunk.EntityKey ?? DBNull.Value);
@@ -349,6 +462,32 @@ internal sealed class UnifiedSearchIndexer : IUnifiedSearchIndexer
return affectedRows; return affectedRows;
} }
private static async Task<bool> HasEmbeddingVectorColumnAsync(
NpgsqlConnection connection,
CancellationToken cancellationToken)
{
const string sql = """
SELECT EXISTS (
SELECT 1
FROM information_schema.columns
WHERE table_schema = 'advisoryai'
AND table_name = 'kb_chunk'
AND column_name = 'embedding_vec'
);
""";
await using var command = connection.CreateCommand();
command.CommandText = sql;
command.CommandTimeout = 30;
var result = await command.ExecuteScalarAsync(cancellationToken).ConfigureAwait(false);
return result is bool value && value;
}
private static string BuildVectorLiteral(float[] values)
{
return "[" + string.Join(",", values.Select(static value => value.ToString("G9", CultureInfo.InvariantCulture))) + "]";
}
private static async Task EnsureDocumentExistsAsync( private static async Task EnsureDocumentExistsAsync(
NpgsqlConnection connection, NpgsqlConnection connection,
string docId, string docId,

119
src/AdvisoryAI/StellaOps.AdvisoryAI/UnifiedSearch/UnifiedSearchService.cs
View File

@@ -179,7 +179,11 @@ internal sealed class UnifiedSearchService : IUnifiedSearchService
aggregateCoverage = MergeCoverage(aggregateCoverage, response.Coverage); aggregateCoverage = MergeCoverage(aggregateCoverage, response.Coverage);
var cardCount = response.Cards.Count + (response.Overflow?.Cards.Count ?? 0); var cardCount = response.Cards.Count + (response.Overflow?.Cards.Count ?? 0);
var answer = response.ContextAnswer; var answer = response.ContextAnswer;
var viabilityState = DetermineSuggestionViabilityState(cardCount, answer, corpusAvailability); var viabilityState = DetermineSuggestionViabilityState(
cardCount,
answer,
response.Coverage,
corpusAvailability);
results.Add(new SearchSuggestionViabilityResult( results.Add(new SearchSuggestionViabilityResult(
Query: query, Query: query,
@@ -191,7 +195,11 @@ internal sealed class UnifiedSearchService : IUnifiedSearchService
?? response.Overflow?.Cards.FirstOrDefault()?.Domain ?? response.Overflow?.Cards.FirstOrDefault()?.Domain
?? response.Coverage?.Domains.FirstOrDefault(static domain => domain.HasVisibleResults)?.Domain ?? response.Coverage?.Domains.FirstOrDefault(static domain => domain.HasVisibleResults)?.Domain
?? response.Coverage?.CurrentScopeDomain, ?? response.Coverage?.CurrentScopeDomain,
Reason: BuildSuggestionViabilityReason(answer, viabilityState, corpusAvailability), Reason: BuildSuggestionViabilityReason(
answer,
viabilityState,
response.Coverage,
corpusAvailability),
ViabilityState: viabilityState, ViabilityState: viabilityState,
ScopeReady: IsCurrentScopeReady(corpusAvailability))); ScopeReady: IsCurrentScopeReady(corpusAvailability)));
} }
@@ -834,7 +842,8 @@ internal sealed class UnifiedSearchService : IUnifiedSearchService
UnifiedSearchOverflow? overflow) UnifiedSearchOverflow? overflow)
{ {
var topCard = answerCards[0]; var topCard = answerCards[0];
var scope = coverage?.CurrentScopeDomain var currentScopeDomain = coverage?.CurrentScopeDomain;
var groundedScope = ResolveGroundedScopeDomain(coverage, topCard)
?? ResolveContextDomain(plan, [topCard], ambient) ?? ResolveContextDomain(plan, [topCard], ambient)
?? topCard.Domain; ?? topCard.Domain;
@@ -843,12 +852,19 @@ internal sealed class UnifiedSearchService : IUnifiedSearchService
return $"The highest-ranked results are close in score, so the answer blends evidence across {FormatDomainList(answerCards.Select(static card => card.Domain))}."; return $"The highest-ranked results are close in score, so the answer blends evidence across {FormatDomainList(answerCards.Select(static card => card.Domain))}.";
} }
if (overflow is not null) if (overflow is not null && !string.IsNullOrWhiteSpace(currentScopeDomain))
{ {
return $"Current-scope weighting kept {DescribeDomain(scope)} first, while close related evidence from other domains remains visible below."; return $"Current-scope weighting kept {DescribeDomain(currentScopeDomain)} first, while close related evidence from other domains remains visible below.";
} }
return $"The top result is grounded in {DescribeDomain(scope)} evidence and aligns with the {plan.Intent} intent."; if (!string.IsNullOrWhiteSpace(currentScopeDomain)
&& !string.Equals(groundedScope, currentScopeDomain, StringComparison.OrdinalIgnoreCase)
&& !HasVisibleCurrentScopeResults(coverage))
{
return $"The best grounded evidence came from {DescribeDomain(groundedScope)} after current-page weighting found no stronger match in {DescribeDomain(currentScopeDomain)}.";
}
return $"The top result is grounded in {DescribeDomain(groundedScope)} evidence and aligns with the {plan.Intent} intent.";
} }
private static string BuildGroundedEvidence( private static string BuildGroundedEvidence(
@@ -871,6 +887,13 @@ internal sealed class UnifiedSearchService : IUnifiedSearchService
return $"Grounded in {sourceCount} source(s) across {FormatDomainList(domains)} with {DescribeDomain(currentScopeDomain)} weighted first."; return $"Grounded in {sourceCount} source(s) across {FormatDomainList(domains)} with {DescribeDomain(currentScopeDomain)} weighted first.";
} }
if (coverage?.CurrentScopeDomain is { Length: > 0 } visibleScopeDomain
&& !HasVisibleCurrentScopeResults(coverage)
&& !string.Equals(answerCards[0].Domain, visibleScopeDomain, StringComparison.OrdinalIgnoreCase))
{
return $"Grounded in {sourceCount} source(s) across {FormatDomainList(domains)}. No visible evidence ranked in {DescribeDomain(visibleScopeDomain)}, so the answer falls back to {DescribeDomain(answerCards[0].Domain)}.";
}
return $"Grounded in {sourceCount} source(s) across {FormatDomainList(domains)}."; return $"Grounded in {sourceCount} source(s) across {FormatDomainList(domains)}.";
} }
@@ -1350,10 +1373,26 @@ internal sealed class UnifiedSearchService : IUnifiedSearchService
private static string DetermineSuggestionViabilityState( private static string DetermineSuggestionViabilityState(
int cardCount, int cardCount,
ContextAnswer? answer, ContextAnswer? answer,
UnifiedSearchCoverage? coverage,
CorpusAvailabilitySnapshot corpusAvailability) CorpusAvailabilitySnapshot corpusAvailability)
{ {
if (string.Equals(ResolveCorpusUnreadyCode(corpusAvailability), "current_scope_corpus_unready", StringComparison.Ordinal)
&& cardCount > 0
&& !HasVisibleCurrentScopeResults(coverage))
{
return "scope_unready";
}
if (cardCount > 0) if (cardCount > 0)
{ {
if (!string.IsNullOrWhiteSpace(coverage?.CurrentScopeDomain)
&& !string.Equals(ResolveCorpusUnreadyCode(corpusAvailability), "current_scope_corpus_unready", StringComparison.Ordinal)
&& !HasVisibleCurrentScopeResults(coverage)
&& CardCountExistsOutsideCurrentScope(answer, coverage))
{
return "outside_scope_only";
}
return "grounded"; return "grounded";
} }
@@ -1389,18 +1428,86 @@ internal sealed class UnifiedSearchService : IUnifiedSearchService
private static string BuildSuggestionViabilityReason( private static string BuildSuggestionViabilityReason(
ContextAnswer? answer, ContextAnswer? answer,
string viabilityState, string viabilityState,
UnifiedSearchCoverage? coverage,
CorpusAvailabilitySnapshot corpusAvailability) CorpusAvailabilitySnapshot corpusAvailability)
{ {
return viabilityState switch return viabilityState switch
{ {
"grounded" => answer?.Reason ?? "Grounded evidence is available for this suggestion.", "grounded" => answer?.Reason ?? "Grounded evidence is available for this suggestion.",
"outside_scope_only" => BuildOutsideScopeOnlyReason(coverage),
"needs_clarification" => answer?.Reason ?? "The query is too broad to surface as a ready-made suggestion.", "needs_clarification" => answer?.Reason ?? "The query is too broad to surface as a ready-made suggestion.",
"scope_unready" => BuildCorpusUnreadyReason(plan: null, ambient: null, corpusAvailability), "scope_unready" => BuildCorpusUnreadyReason(plan: null, ambient: null, corpusAvailability),
"corpus_unready" => BuildCorpusUnreadyReason(plan: null, ambient: null, corpusAvailability), "corpus_unready" => BuildCorpusUnreadyReason(plan: null, ambient: null, corpusAvailability),
"no_match" when CardCountExistsOutsideCurrentScope(answer, coverage) =>
BuildOutsideScopeOnlyReason(coverage),
_ => answer?.Reason ?? "No grounded evidence matched this suggestion in the current corpus." _ => answer?.Reason ?? "No grounded evidence matched this suggestion in the current corpus."
}; };
} }
private static bool HasVisibleCurrentScopeResults(UnifiedSearchCoverage? coverage)
{
if (coverage is null || string.IsNullOrWhiteSpace(coverage.CurrentScopeDomain))
{
return false;
}
return coverage.Domains.Any(domain =>
domain.IsCurrentScope
&& domain.HasVisibleResults
&& domain.VisibleCardCount > 0);
}
private static string? ResolveGroundedScopeDomain(
UnifiedSearchCoverage? coverage,
EntityCard topCard)
{
if (coverage is null)
{
return null;
}
if (HasVisibleCurrentScopeResults(coverage)
&& !string.IsNullOrWhiteSpace(coverage.CurrentScopeDomain))
{
return coverage.CurrentScopeDomain;
}
if (!string.IsNullOrWhiteSpace(topCard.Domain))
{
return topCard.Domain;
}
return coverage.CurrentScopeDomain;
}
private static bool CardCountExistsOutsideCurrentScope(
ContextAnswer? answer,
UnifiedSearchCoverage? coverage)
{
if (!string.Equals(answer?.Status, "grounded", StringComparison.OrdinalIgnoreCase)
|| coverage is null
|| string.IsNullOrWhiteSpace(coverage.CurrentScopeDomain))
{
return false;
}
return !HasVisibleCurrentScopeResults(coverage)
&& coverage.Domains.Any(domain => domain.HasVisibleResults && !domain.IsCurrentScope);
}
private static string BuildOutsideScopeOnlyReason(UnifiedSearchCoverage? coverage)
{
var currentScopeDomain = coverage?.CurrentScopeDomain;
var visibleDomain = coverage?.Domains.FirstOrDefault(static domain => domain.HasVisibleResults)?.Domain;
if (!string.IsNullOrWhiteSpace(currentScopeDomain) && !string.IsNullOrWhiteSpace(visibleDomain))
{
return $"The current page did not return grounded evidence in {DescribeDomain(currentScopeDomain)}, but related evidence is available in {DescribeDomain(visibleDomain)}.";
}
return "The current page did not return grounded evidence, but related evidence is available outside the current scope.";
}
private static string? ResolveCorpusUnreadyCode(CorpusAvailabilitySnapshot corpusAvailability) private static string? ResolveCorpusUnreadyCode(CorpusAvailabilitySnapshot corpusAvailability)
{ {
if (!corpusAvailability.Known) if (!corpusAvailability.Known)

20
src/AdvisoryAI/__Tests/INFRASTRUCTURE.md
View File

@@ -149,9 +149,12 @@ Migrations run automatically when the service starts (`EnsureSchemaAsync()`). Or
```bash ```bash
# Configure connection string for the local AdvisoryAI WebService # Configure connection string for the local AdvisoryAI WebService
export AdvisoryAI__KnowledgeSearch__ConnectionString="Host=localhost;Port=55432;Database=advisoryai_knowledge_test;Username=stellaops_knowledge;Password=stellaops_knowledge" export AdvisoryAI__KnowledgeSearch__ConnectionString="Host=localhost;Port=55432;Database=advisoryai_knowledge_test;Username=stellaops_knowledge;Password=stellaops_knowledge"
export AdvisoryAI__KnowledgeSearch__RepositoryRoot="$(pwd)"
``` ```
Notes:
- AdvisoryAI knowledge ingestion now auto-detects the repository root by walking up from the current working directory and `AppContext.BaseDirectory` until it finds a Stella Ops checkout shape (`global.json`, `src/`, `docs/`, `devops/`).
- `AdvisoryAI__KnowledgeSearch__RepositoryRoot` is still supported, but only for non-standard layouts where the service binary is not running from inside a normal repository checkout.
#### CLI availability in a source checkout #### CLI availability in a source checkout
Do not assume `stella` already exists on `PATH` in a local repo checkout. Do not assume `stella` already exists on `PATH` in a local repo checkout.
@@ -186,12 +189,16 @@ For live search and Playwright suggestion tests, rebuild both indexes in this or
# 1. Knowledge corpus: docs + OpenAPI + Doctor checks # 1. Knowledge corpus: docs + OpenAPI + Doctor checks
export STELLAOPS_BACKEND_URL="http://127.0.0.1:10451" export STELLAOPS_BACKEND_URL="http://127.0.0.1:10451"
dotnet run --project "src/Cli/StellaOps.Cli/StellaOps.Cli.csproj" -- advisoryai sources prepare --json dotnet run --project "src/Cli/StellaOps.Cli/StellaOps.Cli.csproj" -- advisoryai sources prepare --json
dotnet run --project "src/Cli/StellaOps.Cli/StellaOps.Cli.csproj" -- advisoryai index rebuild --json curl -X POST http://127.0.0.1:10451/v1/advisory-ai/index/rebuild \
-H "X-StellaOps-Scopes: advisory-ai:admin" \
-H "X-StellaOps-Tenant: test-tenant" \
-H "X-StellaOps-Actor: local-search-test"
# 2. Unified domain overlays: platform, graph, scanner, timeline, opsmemory # 2. Unified domain overlays: platform, graph, scanner, timeline, opsmemory
curl -X POST http://127.0.0.1:10451/v1/search/index/rebuild \ curl -X POST http://127.0.0.1:10451/v1/search/index/rebuild \
-H "X-StellaOps-Scopes: advisory-ai:admin" \ -H "X-StellaOps-Scopes: advisory-ai:admin" \
-H "X-StellaOps-Tenant: test-tenant" -H "X-StellaOps-Tenant: test-tenant" \
-H "X-StellaOps-Actor: local-search-test"
``` ```
Or use HTTP only when the CLI is not built yet: Or use HTTP only when the CLI is not built yet:
@@ -200,16 +207,19 @@ Or use HTTP only when the CLI is not built yet:
# 1. Knowledge corpus rebuild # 1. Knowledge corpus rebuild
curl -X POST http://127.0.0.1:10451/v1/advisory-ai/index/rebuild \ curl -X POST http://127.0.0.1:10451/v1/advisory-ai/index/rebuild \
-H "X-StellaOps-Scopes: advisory-ai:admin" \ -H "X-StellaOps-Scopes: advisory-ai:admin" \
-H "X-StellaOps-Tenant: test-tenant" -H "X-StellaOps-Tenant: test-tenant" \
-H "X-StellaOps-Actor: local-search-test"
# 2. Unified overlay rebuild # 2. Unified overlay rebuild
curl -X POST http://127.0.0.1:10451/v1/search/index/rebuild \ curl -X POST http://127.0.0.1:10451/v1/search/index/rebuild \
-H "X-StellaOps-Scopes: advisory-ai:admin" \ -H "X-StellaOps-Scopes: advisory-ai:admin" \
-H "X-StellaOps-Tenant: test-tenant" -H "X-StellaOps-Tenant: test-tenant" \
-H "X-StellaOps-Actor: local-search-test"
``` ```
Notes: Notes:
- `stella advisoryai sources prepare` needs `STELLAOPS_BACKEND_URL` or equivalent CLI config when it performs live Doctor discovery. If you only need local search verification and the checked-in Doctor seed/control files are sufficient, the HTTP-only rebuild path is valid. - `stella advisoryai sources prepare` needs `STELLAOPS_BACKEND_URL` or equivalent CLI config when it performs live Doctor discovery. If you only need local search verification and the checked-in Doctor seed/control files are sufficient, the HTTP-only rebuild path is valid.
- `stella advisoryai index rebuild` and `stella search index rebuild` call authenticated backend endpoints. In the local live-search test lane above, use direct HTTP requests with explicit `X-StellaOps-*` headers unless you already have an authenticated CLI session.
- Current live verification coverage includes the Doctor/knowledge query `database connectivity`, which returns `contextAnswer.status = grounded` plus citations after the rebuild sequence above. - Current live verification coverage includes the Doctor/knowledge query `database connectivity`, which returns `contextAnswer.status = grounded` plus citations after the rebuild sequence above.
Migration files (all idempotent, safe to re-run): Migration files (all idempotent, safe to re-run):

107
src/AdvisoryAI/__Tests/StellaOps.AdvisoryAI.Tests/Integration/UnifiedSearchLiveAdapterIntegrationTests.cs
View File

@@ -59,6 +59,9 @@ public sealed class UnifiedSearchLiveAdapterIntegrationTests
chunks[0].EntityType.Should().Be("finding"); chunks[0].EntityType.Should().Be("finding");
chunks[0].EntityKey.Should().Be("cve:CVE-2026-0001"); chunks[0].EntityKey.Should().Be("cve:CVE-2026-0001");
chunks[0].Title.Should().Contain("CVE-2026-0001"); chunks[0].Title.Should().Contain("CVE-2026-0001");
chunks[0].Body.Should().Contain("Finding type: vulnerability finding");
chunks[0].Body.Should().Contain("Reachable: reachable");
chunks[0].Body.Should().Contain("Status: open unresolved");
handler.Requests.Should().ContainSingle(); handler.Requests.Should().ContainSingle();
handler.Requests[0].Tenant.Should().Be("global"); handler.Requests[0].Tenant.Should().Be("global");
@@ -101,6 +104,8 @@ public sealed class UnifiedSearchLiveAdapterIntegrationTests
chunks[0].EntityKey.Should().Be("cve:CVE-2026-4242"); chunks[0].EntityKey.Should().Be("cve:CVE-2026-4242");
chunks[0].Domain.Should().Be("findings"); chunks[0].Domain.Should().Be("findings");
chunks[0].Title.Should().Contain("Snapshot finding"); chunks[0].Title.Should().Contain("Snapshot finding");
chunks[0].Body.Should().Contain("Finding type: vulnerability finding");
chunks[0].Body.Should().Contain("Status: open unresolved");
} }
finally finally
{ {
@@ -144,6 +149,9 @@ public sealed class UnifiedSearchLiveAdapterIntegrationTests
chunks[0].EntityType.Should().Be("vex_statement"); chunks[0].EntityType.Should().Be("vex_statement");
chunks[0].EntityKey.Should().Be("cve:CVE-2026-1111"); chunks[0].EntityKey.Should().Be("cve:CVE-2026-1111");
chunks[0].Title.Should().Contain("CVE-2026-1111"); chunks[0].Title.Should().Contain("CVE-2026-1111");
chunks[0].Body.Should().Contain("Marked as: not affected");
chunks[0].Body.Should().Contain("Covered component: pkg:nuget/Contoso.Widget");
chunks[0].Body.Should().Contain("Conflict evidence: none recorded");
handler.Requests.Should().ContainSingle(); handler.Requests.Should().ContainSingle();
handler.Requests[0].Tenant.Should().Be("global"); handler.Requests[0].Tenant.Should().Be("global");
@@ -186,6 +194,8 @@ public sealed class UnifiedSearchLiveAdapterIntegrationTests
chunks[0].EntityType.Should().Be("vex_statement"); chunks[0].EntityType.Should().Be("vex_statement");
chunks[0].EntityKey.Should().Be("cve:CVE-2026-5151"); chunks[0].EntityKey.Should().Be("cve:CVE-2026-5151");
chunks[0].Title.Should().Contain("CVE-2026-5151"); chunks[0].Title.Should().Contain("CVE-2026-5151");
chunks[0].Body.Should().Contain("Marked as: not affected");
chunks[0].Body.Should().Contain("Conflict evidence: none recorded");
} }
finally finally
{ {
@@ -227,6 +237,8 @@ public sealed class UnifiedSearchLiveAdapterIntegrationTests
chunks[0].EntityType.Should().Be("policy_rule"); chunks[0].EntityType.Should().Be("policy_rule");
chunks[0].EntityKey.Should().Be("rule:DENY-CRITICAL-PROD"); chunks[0].EntityKey.Should().Be("rule:DENY-CRITICAL-PROD");
chunks[0].Title.Should().Contain("DENY-CRITICAL-PROD"); chunks[0].Title.Should().Contain("DENY-CRITICAL-PROD");
chunks[0].Body.Should().Contain("Policy domain: policy rule gate");
chunks[0].Body.Should().Contain("Gate state: failing blocking");
handler.Requests.Should().ContainSingle(); handler.Requests.Should().ContainSingle();
handler.Requests[0].Tenant.Should().Be("global"); handler.Requests[0].Tenant.Should().Be("global");
@@ -269,6 +281,8 @@ public sealed class UnifiedSearchLiveAdapterIntegrationTests
chunks[0].EntityType.Should().Be("policy_rule"); chunks[0].EntityType.Should().Be("policy_rule");
chunks[0].EntityKey.Should().Be("rule:ALLOW-STAGING-SMOKE"); chunks[0].EntityKey.Should().Be("rule:ALLOW-STAGING-SMOKE");
chunks[0].Title.Should().Contain("Allow staging smoke tests"); chunks[0].Title.Should().Contain("Allow staging smoke tests");
chunks[0].Body.Should().Contain("Policy domain: policy rule gate");
chunks[0].Body.Should().Contain("Gate decision: warn");
} }
finally finally
{ {
@@ -358,6 +372,85 @@ public sealed class UnifiedSearchLiveAdapterIntegrationTests
(await CountDomainChunksAsync(connection, "policy")).Should().Be(4); (await CountDomainChunksAsync(connection, "policy")).Should().Be(4);
} }
[Fact]
public async Task UnifiedSearchIndexer_RebuildAllAsync_PopulatesEnglishFtsColumns_AndRecallsUnifiedDomains()
{
await using var fixture = await StartPostgresOrSkipAsync();
var options = Options.Create(new KnowledgeSearchOptions
{
Enabled = true,
ConnectionString = fixture.ConnectionString,
FtsLanguageConfig = "english",
FindingsAdapterEnabled = true,
FindingsAdapterBaseUrl = "http://scanner.local",
VexAdapterEnabled = true,
VexAdapterBaseUrl = "http://concelier.local",
PolicyAdapterEnabled = true,
PolicyAdapterBaseUrl = "http://policy.local"
});
await using var store = new PostgresKnowledgeSearchStore(options, NullLogger<PostgresKnowledgeSearchStore>.Instance);
await EnsureKnowledgeSchemaAsync(fixture.ConnectionString);
var findingsHandler = new RecordingHttpMessageHandler(_ => JsonResponse(BuildFindingsPayload(3)));
var vexHandler = new RecordingHttpMessageHandler(_ => JsonResponse(BuildVexPayload(3)));
var policyHandler = new RecordingHttpMessageHandler(_ => JsonResponse(BuildPolicyPayload(3)));
var indexer = new UnifiedSearchIndexer(
options,
store,
[
new FindingsSearchAdapter(
new SingleClientFactory(findingsHandler, "http://scanner.local"),
CreateVectorEncoder(),
options,
NullLogger<FindingsSearchAdapter>.Instance),
new VexSearchAdapter(
new SingleClientFactory(vexHandler, "http://concelier.local"),
CreateVectorEncoder(),
options,
NullLogger<VexSearchAdapter>.Instance),
new PolicySearchAdapter(
new SingleClientFactory(policyHandler, "http://policy.local"),
CreateVectorEncoder(),
options,
NullLogger<PolicySearchAdapter>.Instance)
],
NullLogger<UnifiedSearchIndexer>.Instance);
await indexer.RebuildAllAsync(CancellationToken.None);
await using var connection = new NpgsqlConnection(fixture.ConnectionString);
await connection.OpenAsync();
(await CountEnglishTsvRowsAsync(connection, "findings")).Should().Be(3);
(await CountEnglishTsvRowsAsync(connection, "policy")).Should().Be(3);
(await CountEnglishTsvRowsAsync(connection, "vex")).Should().Be(3);
var findingsRows = await store.SearchFtsAsync(
"critical findings",
new KnowledgeSearchFilter { Tenant = "global" },
10,
TimeSpan.FromSeconds(2),
CancellationToken.None);
findingsRows.Should().Contain(row => row.Kind == "finding" && row.Body.Contains("Finding type: vulnerability finding"));
var policyRows = await store.SearchFtsAsync(
"failing policy gates",
new KnowledgeSearchFilter { Tenant = "global" },
10,
TimeSpan.FromSeconds(2),
CancellationToken.None);
policyRows.Should().Contain(row => row.Kind == "policy_rule" && row.Body.Contains("Policy domain: policy rule gate"));
var vexRows = await store.SearchFtsAsync(
"marked not affected",
new KnowledgeSearchFilter { Tenant = "global" },
10,
TimeSpan.FromSeconds(2),
CancellationToken.None);
vexRows.Should().Contain(row => row.Kind == "vex_statement" && row.Body.Contains("Marked as: not affected"));
}
[Fact] [Fact]
public async Task UnifiedSearchIndexer_RebuildAllAsync_EnsuresSchema_WhenTablesAreMissing() public async Task UnifiedSearchIndexer_RebuildAllAsync_EnsuresSchema_WhenTablesAreMissing()
{ {
@@ -888,6 +981,20 @@ public sealed class UnifiedSearchLiveAdapterIntegrationTests
return Convert.ToInt32(scalar, System.Globalization.CultureInfo.InvariantCulture); return Convert.ToInt32(scalar, System.Globalization.CultureInfo.InvariantCulture);
} }
private static async Task<int> CountEnglishTsvRowsAsync(NpgsqlConnection connection, string domain)
{
await using var command = connection.CreateCommand();
command.CommandText = """
SELECT COUNT(*)
FROM advisoryai.kb_chunk
WHERE domain = @domain
AND body_tsv_en IS NOT NULL;
""";
command.Parameters.AddWithValue("domain", domain);
var scalar = await command.ExecuteScalarAsync();
return Convert.ToInt32(scalar, CultureInfo.InvariantCulture);
}
private static async Task<DateTimeOffset> ReadIndexedAtAsync(NpgsqlConnection connection, string chunkId) private static async Task<DateTimeOffset> ReadIndexedAtAsync(NpgsqlConnection connection, string chunkId)
{ {
await using var command = connection.CreateCommand(); await using var command = connection.CreateCommand();

27
src/AdvisoryAI/__Tests/StellaOps.AdvisoryAI.Tests/Integration/UnifiedSearchSprintIntegrationTests.cs
View File

@@ -1744,6 +1744,33 @@ public sealed class UnifiedSearchSprintIntegrationTests : IDisposable
stored.Comment.Should().BeNull("free-form comments are redacted to avoid storing potential PII"); stored.Comment.Should().BeNull("free-form comments are redacted to avoid storing potential PII");
} }
[Fact]
public async Task G10_SimilarSuccessfulQueries_LoadFromDatabaseWithoutFallingBack()
{
var tenant = $"similar-queries-{Guid.NewGuid():N}";
const string userId = "similar-user";
using (var seedScope = _factory.Services.CreateScope())
{
var analytics = seedScope.ServiceProvider.GetRequiredService<SearchAnalyticsService>();
await analytics.RecordHistoryAsync(tenant, userId, "database connectivity", 2);
await analytics.RecordHistoryAsync(tenant, userId, "database health", 1);
await analytics.RecordHistoryAsync(tenant, userId, "database readiness", 1);
}
using var queryScope = _factory.Services.CreateScope();
var queryAnalytics = queryScope.ServiceProvider.GetRequiredService<SearchAnalyticsService>();
var similar = await queryAnalytics.FindSimilarSuccessfulQueriesAsync(
tenant,
"database connectivty",
3,
CancellationToken.None);
similar.Should().Contain("database connectivity");
similar.Should().Contain(query => query.StartsWith("database ", StringComparison.OrdinalIgnoreCase));
}
[Fact] [Fact]
public async Task G10_AnalyticsCollection_Overhead_IsBelowFiveMillisecondsPerEvent() public async Task G10_AnalyticsCollection_Overhead_IsBelowFiveMillisecondsPerEvent()
{ {

102
src/AdvisoryAI/__Tests/StellaOps.AdvisoryAI.Tests/KnowledgeSearch/KnowledgeSearchRepositoryRootResolverTests.cs Normal file
View File

@@ -0,0 +1,102 @@
using StellaOps.AdvisoryAI.KnowledgeSearch;
using Xunit;
namespace StellaOps.AdvisoryAI.Tests.KnowledgeSearch;
[Trait("Category", "Unit")]
public sealed class KnowledgeSearchRepositoryRootResolverTests
{
[Fact]
public void Resolve_detects_repository_root_from_service_directory_without_explicit_override()
{
using var fixture = RepositoryRootFixture.Create();
var options = new KnowledgeSearchOptions();
var resolution = KnowledgeSearchRepositoryRootResolver.Resolve(
options,
fixture.ServiceDirectory,
fixture.AppBaseDirectory);
Assert.True(resolution.Validated);
Assert.Equal(fixture.RepositoryRoot, resolution.Path);
}
[Fact]
public void Resolve_falls_back_from_invalid_configured_root_to_detected_repository_root()
{
using var fixture = RepositoryRootFixture.Create();
var options = new KnowledgeSearchOptions
{
RepositoryRoot = Path.Combine(fixture.WorkspaceRoot, "invalid-root"),
};
var resolution = KnowledgeSearchRepositoryRootResolver.Resolve(
options,
fixture.ServiceDirectory,
fixture.AppBaseDirectory);
Assert.True(resolution.Validated);
Assert.Equal(fixture.RepositoryRoot, resolution.Path);
Assert.NotEqual("configured", resolution.Source);
}
private sealed class RepositoryRootFixture : IDisposable
{
private RepositoryRootFixture(
string workspaceRoot,
string repositoryRoot,
string serviceDirectory,
string appBaseDirectory)
{
WorkspaceRoot = workspaceRoot;
RepositoryRoot = repositoryRoot;
ServiceDirectory = serviceDirectory;
AppBaseDirectory = appBaseDirectory;
}
public string WorkspaceRoot { get; }
public string RepositoryRoot { get; }
public string ServiceDirectory { get; }
public string AppBaseDirectory { get; }
public static RepositoryRootFixture Create()
{
var workspaceRoot = Path.Combine(Path.GetTempPath(), "stellaops-knowledge-root-" + Guid.NewGuid().ToString("N"));
var repositoryRoot = Path.Combine(workspaceRoot, "repo");
var serviceDirectory = Path.Combine(repositoryRoot, "src", "AdvisoryAI", "StellaOps.AdvisoryAI.WebService");
var appBaseDirectory = Path.Combine(serviceDirectory, "bin", "Debug", "net10.0");
Directory.CreateDirectory(repositoryRoot);
Directory.CreateDirectory(Path.Combine(repositoryRoot, "docs"));
Directory.CreateDirectory(Path.Combine(repositoryRoot, "devops", "compose"));
Directory.CreateDirectory(Path.Combine(repositoryRoot, "src", "AdvisoryAI", "StellaOps.AdvisoryAI", "KnowledgeSearch"));
Directory.CreateDirectory(appBaseDirectory);
File.WriteAllText(Path.Combine(repositoryRoot, "global.json"), "{}");
File.WriteAllText(Path.Combine(repositoryRoot, "docs", "README.md"), "# docs");
File.WriteAllText(Path.Combine(repositoryRoot, "devops", "compose", "openapi_current.json"), "{}");
File.WriteAllText(
Path.Combine(repositoryRoot, "src", "AdvisoryAI", "StellaOps.AdvisoryAI", "KnowledgeSearch", "knowledge-docs-allowlist.json"),
"[]");
File.WriteAllText(
Path.Combine(repositoryRoot, "src", "AdvisoryAI", "StellaOps.AdvisoryAI", "KnowledgeSearch", "doctor-search-seed.json"),
"[]");
File.WriteAllText(
Path.Combine(repositoryRoot, "src", "AdvisoryAI", "StellaOps.AdvisoryAI", "KnowledgeSearch", "doctor-search-controls.json"),
"[]");
return new RepositoryRootFixture(workspaceRoot, repositoryRoot, serviceDirectory, appBaseDirectory);
}
public void Dispose()
{
if (Directory.Exists(WorkspaceRoot))
{
Directory.Delete(WorkspaceRoot, recursive: true);
}
}
}
}

134
src/AdvisoryAI/__Tests/StellaOps.AdvisoryAI.Tests/UnifiedSearch/UnifiedSearchIngestionAdaptersTests.cs
View File

@@ -5,6 +5,7 @@ using StellaOps.AdvisoryAI.KnowledgeSearch;
using StellaOps.AdvisoryAI.UnifiedSearch; using StellaOps.AdvisoryAI.UnifiedSearch;
using StellaOps.AdvisoryAI.UnifiedSearch.Adapters; using StellaOps.AdvisoryAI.UnifiedSearch.Adapters;
using StellaOps.AdvisoryAI.Vectorization; using StellaOps.AdvisoryAI.Vectorization;
using System.Net.Http;
using System.Text.Json; using System.Text.Json;
using Xunit; using Xunit;
@@ -289,6 +290,50 @@ public sealed class UnifiedSearchIngestionAdaptersTests
} }
} }
[Fact]
public async Task Default_snapshot_paths_resolve_from_repository_root_when_running_from_service_directory()
{
using var fixture = RepositoryRootFixture.Create();
var originalDirectory = Directory.GetCurrentDirectory();
try
{
Directory.SetCurrentDirectory(fixture.ServiceDirectory);
var findingsAdapter = new FindingIngestionAdapter(
new StubVectorEncoder(),
Options.Create(new KnowledgeSearchOptions()),
NullLogger<FindingIngestionAdapter>.Instance);
var policyAdapter = new PolicySearchAdapter(
new StubHttpClientFactory(),
new StubVectorEncoder(),
Options.Create(new KnowledgeSearchOptions()),
NullLogger<PolicySearchAdapter>.Instance);
var vexAdapter = new VexSearchAdapter(
new StubHttpClientFactory(),
new StubVectorEncoder(),
Options.Create(new KnowledgeSearchOptions()),
NullLogger<VexSearchAdapter>.Instance);
var findings = await findingsAdapter.ProduceChunksAsync(CancellationToken.None);
var policies = await policyAdapter.ProduceChunksAsync(CancellationToken.None);
var vexStatements = await vexAdapter.ProduceChunksAsync(CancellationToken.None);
findings.Should().ContainSingle();
findings[0].EntityKey.Should().Be("cve:CVE-2026-1111");
policies.Should().ContainSingle();
policies[0].EntityKey.Should().Be("rule:release.production.gate");
vexStatements.Should().ContainSingle();
vexStatements[0].EntityKey.Should().Be("cve:CVE-2026-1111");
}
finally
{
Directory.SetCurrentDirectory(originalDirectory);
}
}
private static string CreateTempDirectory() private static string CreateTempDirectory()
{ {
var path = Path.Combine(Path.GetTempPath(), "stellaops-adapter-tests-" + Guid.NewGuid().ToString("N")); var path = Path.Combine(Path.GetTempPath(), "stellaops-adapter-tests-" + Guid.NewGuid().ToString("N"));
@@ -308,4 +353,93 @@ public sealed class UnifiedSearchIngestionAdaptersTests
{ {
public float[] Encode(string text) => [0.12f, 0.34f, 0.56f, 0.78f]; public float[] Encode(string text) => [0.12f, 0.34f, 0.56f, 0.78f];
} }
private sealed class StubHttpClientFactory : IHttpClientFactory
{
public HttpClient CreateClient(string name) => new();
}
private sealed class RepositoryRootFixture : IDisposable
{
private RepositoryRootFixture(string workspaceRoot, string repositoryRoot, string serviceDirectory)
{
WorkspaceRoot = workspaceRoot;
RepositoryRoot = repositoryRoot;
ServiceDirectory = serviceDirectory;
}
public string WorkspaceRoot { get; }
public string RepositoryRoot { get; }
public string ServiceDirectory { get; }
public static RepositoryRootFixture Create()
{
var workspaceRoot = Path.Combine(Path.GetTempPath(), "stellaops-unified-snapshots-" + Guid.NewGuid().ToString("N"));
var repositoryRoot = Path.Combine(workspaceRoot, "repo");
var snapshotDirectory = Path.Combine(repositoryRoot, "src", "AdvisoryAI", "StellaOps.AdvisoryAI", "UnifiedSearch", "Snapshots");
var serviceDirectory = Path.Combine(repositoryRoot, "src", "AdvisoryAI", "StellaOps.AdvisoryAI.WebService");
Directory.CreateDirectory(Path.Combine(repositoryRoot, "docs"));
Directory.CreateDirectory(Path.Combine(repositoryRoot, "devops"));
Directory.CreateDirectory(snapshotDirectory);
Directory.CreateDirectory(serviceDirectory);
File.WriteAllText(Path.Combine(repositoryRoot, "global.json"), "{}");
File.WriteAllText(Path.Combine(repositoryRoot, "docs", "README.md"), "# docs");
File.WriteAllText(Path.Combine(snapshotDirectory, "findings.snapshot.json"), JsonSerializer.Serialize(new[]
{
new
{
cveId = "CVE-2026-1111",
findingId = "finding-1",
severity = "high",
title = "Critical finding",
description = "reachable production issue",
service = "scanner",
tenant = "tenant-a",
tags = new[] { "finding", "reachable" },
}
}));
File.WriteAllText(Path.Combine(snapshotDirectory, "policy.snapshot.json"), JsonSerializer.Serialize(new[]
{
new
{
ruleId = "release.production.gate",
title = "Production release gate",
description = "Blocks production on unresolved reachability.",
decision = "deny",
service = "policy",
tenant = "tenant-a",
tags = new[] { "policy", "release" },
freshness = "2026-03-07T00:00:00Z",
}
}));
File.WriteAllText(Path.Combine(snapshotDirectory, "vex.snapshot.json"), JsonSerializer.Serialize(new[]
{
new
{
cveId = "CVE-2026-1111",
status = "affected",
statementId = "stmt-1",
justification = "reachable in production",
service = "vex-hub",
tenant = "tenant-a",
tags = new[] { "vex", "affected" },
freshness = "2026-03-07T00:00:00Z",
}
}));
return new RepositoryRootFixture(workspaceRoot, repositoryRoot, serviceDirectory);
}
public void Dispose()
{
if (Directory.Exists(WorkspaceRoot))
{
Directory.Delete(WorkspaceRoot, recursive: true);
}
}
}
} }

125
src/AdvisoryAI/__Tests/StellaOps.AdvisoryAI.Tests/UnifiedSearch/UnifiedSearchServiceTests.cs
View File

@@ -190,6 +190,51 @@ public sealed class UnifiedSearchServiceTests
result.ContextAnswer.Citations.Should().ContainSingle(); result.ContextAnswer.Citations.Should().ContainSingle();
} }
[Fact]
public async Task SearchAsync_describes_the_actual_winning_domain_when_current_scope_has_no_visible_results()
{
var knowledgeRow = MakeRow(
"chunk-doctor-fallback-answer",
"doctor_check",
"PostgreSQL connectivity",
JsonDocument.Parse("{\"domain\":\"knowledge\",\"checkCode\":\"check.core.db.connectivity\"}"),
snippet: "Doctor knowledge confirms the database endpoint is down.");
var service = CreateService(
store: new CorpusAvailabilityTestStore(
ftsRows: new Dictionary<string, IReadOnlyList<KnowledgeChunkRow>>(StringComparer.OrdinalIgnoreCase)
{
["database connectivity"] = [knowledgeRow]
},
domainAvailability:
[
new KnowledgeSearchDomainCorpusAvailability("knowledge", 8),
new KnowledgeSearchDomainCorpusAvailability("findings", 0)
]));
var result = await service.SearchAsync(
new UnifiedSearchRequest(
"database connectivity",
Ambient: new AmbientContext
{
CurrentRoute = "/security/triage"
}),
CancellationToken.None);
result.Cards.Should().ContainSingle();
result.Cards[0].Domain.Should().Be("knowledge");
result.Coverage.Should().NotBeNull();
result.Coverage!.CurrentScopeDomain.Should().Be("findings");
result.Coverage.Domains.Should().Contain(domain =>
domain.Domain == "findings" && domain.IsCurrentScope && !domain.HasVisibleResults);
result.ContextAnswer.Should().NotBeNull();
result.ContextAnswer!.Reason.Should().Contain("knowledge scope");
result.ContextAnswer.Reason.Should().Contain("findings scope");
result.ContextAnswer.Reason.Should().NotContain("grounded in the findings scope");
result.ContextAnswer.Evidence.Should().Contain("No visible evidence ranked in the findings scope");
result.ContextAnswer.Evidence.Should().Contain("falls back to the knowledge scope");
}
[Fact] [Fact]
public async Task SearchAsync_blends_close_top_results_into_one_answer() public async Task SearchAsync_blends_close_top_results_into_one_answer()
{ {
@@ -645,6 +690,86 @@ public sealed class UnifiedSearchServiceTests
result.Suggestions[0].ScopeReady.Should().BeFalse(); result.Suggestions[0].ScopeReady.Should().BeFalse();
} }
[Fact]
public async Task EvaluateSuggestionsAsync_rejects_outside_scope_only_hits_when_the_current_route_corpus_is_unready()
{
var knowledgeRow = MakeRow(
"chunk-doctor-viability-fallback",
"doctor_check",
"PostgreSQL connectivity",
JsonDocument.Parse("{\"domain\":\"knowledge\",\"checkCode\":\"check.core.db.connectivity\"}"),
snippet: "Doctor knowledge confirms the database endpoint is down.");
var service = CreateService(
store: new CorpusAvailabilityTestStore(
ftsRows: new Dictionary<string, IReadOnlyList<KnowledgeChunkRow>>(StringComparer.OrdinalIgnoreCase)
{
["database connectivity"] = [knowledgeRow]
},
domainAvailability:
[
new KnowledgeSearchDomainCorpusAvailability("knowledge", 8),
new KnowledgeSearchDomainCorpusAvailability("findings", 0)
]));
var result = await service.EvaluateSuggestionsAsync(
new SearchSuggestionViabilityRequest(
Queries: ["database connectivity"],
Ambient: new AmbientContext
{
CurrentRoute = "/security/triage"
}),
CancellationToken.None);
result.Suggestions.Should().ContainSingle();
result.Suggestions[0].Viable.Should().BeFalse();
result.Suggestions[0].Status.Should().Be("grounded");
result.Suggestions[0].LeadingDomain.Should().Be("knowledge");
result.Suggestions[0].ViabilityState.Should().Be("scope_unready");
result.Suggestions[0].Reason.Should().Contain("findings scope");
result.Suggestions[0].ScopeReady.Should().BeFalse();
}
[Fact]
public async Task EvaluateSuggestionsAsync_rejects_outside_scope_only_hits_when_current_scope_is_populated()
{
var knowledgeRow = MakeRow(
"chunk-doctor-viability-outside-scope",
"doctor_check",
"PostgreSQL connectivity",
JsonDocument.Parse("{\"domain\":\"knowledge\",\"checkCode\":\"check.core.db.connectivity\"}"),
snippet: "Doctor knowledge confirms the database endpoint is down.");
var service = CreateService(
store: new CorpusAvailabilityTestStore(
ftsRows: new Dictionary<string, IReadOnlyList<KnowledgeChunkRow>>(StringComparer.OrdinalIgnoreCase)
{
["critical findings"] = [knowledgeRow]
},
domainAvailability:
[
new KnowledgeSearchDomainCorpusAvailability("knowledge", 8),
new KnowledgeSearchDomainCorpusAvailability("findings", 3)
]));
var result = await service.EvaluateSuggestionsAsync(
new SearchSuggestionViabilityRequest(
Queries: ["critical findings"],
Ambient: new AmbientContext
{
CurrentRoute = "/security/triage"
}),
CancellationToken.None);
result.Suggestions.Should().ContainSingle();
result.Suggestions[0].Viable.Should().BeFalse();
result.Suggestions[0].Status.Should().Be("grounded");
result.Suggestions[0].LeadingDomain.Should().Be("knowledge");
result.Suggestions[0].ViabilityState.Should().Be("outside_scope_only");
result.Suggestions[0].Reason.Should().Contain("findings scope");
result.Suggestions[0].ScopeReady.Should().BeTrue();
}
[Fact] [Fact]
public async Task EvaluateSuggestionsAsync_returns_viability_without_recording_answer_frame_analytics() public async Task EvaluateSuggestionsAsync_returns_viability_without_recording_answer_frame_analytics()
{ {