Files
git.stella-ops.org/docs/qa/unified-search-test-cases.md

106 KiB

Unified Search — 1000+ Test Cases by Ingested Data Domain

This document enumerates realistic search queries that users would issue against the Stella Ops unified search index, organized by the data domain that would catch/serve them. Each case shows the query, the expected matching domain(s), and what entity types should surface.


Domain 1: Knowledge — Documentation (docs/*.md)

1.1 Getting Started & Onboarding (30 cases)

# Query Expected Entity Type Expected Match Source
1 how to get started docs docs/quickstart.md
2 first scan walkthrough docs docs/quickstart.md
3 developer onboarding docs docs/DEVELOPER_ONBOARDING.md
4 contribution checklist docs docs/dev/onboarding/contribution-checklist.md
5 setup development environment docs docs/dev/DEV_ENVIRONMENT_SETUP.md
6 install stella ops docs docs/INSTALL_GUIDE.md
7 docker compose setup docs docs/setup/
8 local postgres setup docs docs/db/local-postgres.md
9 quick start guide docs docs/quickstart.md
10 what is stella ops docs docs/overview.md
11 product overview docs docs/overview.md
12 key features docs docs/key-features.md
13 full features list docs docs/full-features-list.md
14 feature matrix docs docs/FEATURE_MATRIX.md
15 system requirements docs docs/INSTALL_GUIDE.md
16 prerequisites docs docs/INSTALL_GUIDE.md
17 troubleshooting guide docs docs/dev/onboarding/troubleshooting-guide.md
18 FAQ docs docs/dev/onboarding/faq/
19 video tutorials docs docs/dev/onboarding/video-tutorial-scripts.md
20 dev quickstart docs docs/dev/onboarding/dev-quickstart.md
21 coding standards docs docs/CODING_STANDARDS.md
22 code of conduct docs docs/code-of-conduct/CODE_OF_CONDUCT.md
23 testing practices docs docs/code-of-conduct/TESTING_PRACTICES.md
24 community guidelines docs docs/code-of-conduct/COMMUNITY_CONDUCT.md
25 glossary docs docs/GLOSSARY.md
26 terminology definitions docs docs/GLOSSARY.md
27 roadmap docs docs/ROADMAP.md
28 planned features docs docs/ROADMAP.md
29 ui guide docs docs/UI_GUIDE.md
30 console operator walkthrough docs docs/UI_GUIDE.md

1.2 Architecture & Design (40 cases)

# Query Expected Entity Type Expected Match Source
31 high level architecture docs docs/07_HIGH_LEVEL_ARCHITECTURE.md
32 system architecture overview docs docs/ARCHITECTURE_OVERVIEW.md
33 architecture reference docs docs/ARCHITECTURE_REFERENCE.md
34 evidence pipeline architecture docs docs/architecture/EVIDENCE_PIPELINE_ARCHITECTURE.md
35 integration architecture docs docs/architecture/integrations.md
36 microservice architecture docs docs/ARCHITECTURE_OVERVIEW.md
37 how does the router work docs docs/modules/router/
38 gateway architecture docs docs/modules/gateway/
39 message routing docs docs/modules/router/
40 event-driven architecture docs docs/ARCHITECTURE_OVERVIEW.md
41 multi-tenant isolation docs docs/contracts/web-gateway-tenant-rbac.md
42 tenant RBAC docs docs/contracts/web-gateway-tenant-rbac.md
43 linkset correlation docs docs/architecture/decisions/ADR-001
44 content addressable storage docs docs/contracts/cas-infrastructure.md
45 deterministic replay docs docs/contracts/, docs/modules/replay/
46 sealed mode docs docs/contracts/sealed-mode.md
47 sealed installation docs docs/contracts/sealed-install-enforcement.md
48 rate limiting design docs docs/contracts/rate-limit-design.md
49 ADR architecture decision docs docs/architecture/decisions/
50 API versioning docs docs/api/versioning.md
51 API governance docs docs/contracts/api-governance-baseline.md
52 openapi discovery docs docs/api/openapi-discovery.md
53 evidence model schema docs docs/modules/evidence/
54 attestation architecture docs docs/modules/attestor/
55 provenance tracking docs docs/modules/provenance/
56 database specification docs docs/db/SPECIFICATION.md
57 database migration strategy docs docs/db/MIGRATION_STRATEGY.md
58 EF Core migration docs docs/db/MIGRATION_STRATEGY.md
59 migration conventions docs docs/db/MIGRATION_CONVENTIONS.md
60 migration inventory docs docs/db/MIGRATION_INVENTORY.md
61 MongoDB to PostgreSQL docs docs/db/CONVERSION_PLAN.md
62 database rules docs docs/db/RULES.md
63 cluster provisioning docs docs/db/cluster-provisioning.md
64 connection pool docs docs/db/
65 buildid propagation docs docs/contracts/buildid-propagation.md
66 canonical sbom id docs docs/contracts/canonical-sbom-id-v1.md
67 witness format docs docs/contracts/witness-v1.md
68 execution evidence format docs docs/contracts/execution-evidence-v1.md
69 export bundle structure docs docs/contracts/export-bundle.md
70 federated consent model docs docs/contracts/federated-consent-v1.md

1.3 Security & Hardening (30 cases)

# Query Expected Entity Type Expected Match Source
71 security hardening guide docs docs/SECURITY_HARDENING_GUIDE.md
72 security policy docs docs/SECURITY_POLICY.md
73 vulnerability disclosure docs docs/SECURITY_POLICY.md
74 VEX consensus guide docs docs/VEX_CONSENSUS_GUIDE.md
75 VEX trust model docs docs/VEX_CONSENSUS_GUIDE.md
76 how to harden deployment docs docs/SECURITY_HARDENING_GUIDE.md
77 TLS configuration docs docs/security/
78 certificate management docs docs/security/
79 FIPS compliance docs docs/security/, crypto
80 GOST cryptography docs docs/security/, crypto
81 eIDAS digital signatures docs docs/security/, crypto
82 SM crypto support docs docs/security/, crypto
83 HSM PKCS#11 docs docs/security/, crypto
84 air gap operation docs docs/OFFLINE_KIT.md
85 offline kit docs docs/OFFLINE_KIT.md
86 air-gapped deployment docs docs/OFFLINE_KIT.md
87 supply chain security docs docs/security/
88 SBOM security docs docs/modules/sbom-service/
89 attestation signing docs docs/modules/signer/
90 transparency log docs docs/modules/attestor/
91 Rekor integration docs docs/modules/attestor/
92 Sigstore docs docs/modules/attestor/
93 in-toto attestation docs docs/modules/attestor/
94 DSSE envelope docs docs/modules/attestor/
95 key rotation docs docs/modules/signer/
96 signing ceremony docs docs/modules/signer/
97 trust anchor management docs docs/security/
98 secret detection docs docs/modules/scanner/
99 credential scanning docs docs/modules/scanner/
100 compliance readiness tracker docs docs/compliance/

1.4 Module Architecture Dossiers (50 cases)

# Query Expected Entity Type Expected Match Source
101 scanner architecture docs docs/modules/scanner/
102 policy engine architecture docs docs/modules/policy/
103 concelier architecture docs docs/modules/concelier/
104 excititor architecture docs docs/modules/excititor/
105 VEX lens architecture docs docs/modules/vex-lens/
106 VEX hub architecture docs docs/modules/vex-hub/
107 findings ledger architecture docs docs/modules/findings-ledger/
108 evidence locker architecture docs docs/modules/evidence-locker/
109 attestor architecture docs docs/modules/attestor/
110 signer architecture docs docs/modules/signer/
111 orchestrator architecture docs docs/modules/orchestrator/
112 scheduler architecture docs docs/modules/scheduler/
113 taskrunner architecture docs docs/modules/taskrunner/
114 authority architecture docs docs/modules/authority/
115 notifier architecture docs docs/modules/notifier/
116 timeline architecture docs docs/modules/timeline/
117 graph architecture docs docs/modules/graph/
118 reach graph architecture docs docs/modules/reach-graph/
119 reachability architecture docs docs/modules/reachability/
120 triage architecture docs docs/modules/triage/
121 risk engine architecture docs docs/modules/risk-engine/
122 unknowns architecture docs docs/modules/unknowns/
123 export center architecture docs docs/modules/export-center/
124 remediation architecture docs docs/modules/remediation/
125 signals architecture docs docs/modules/signals/
126 binary index architecture docs docs/modules/binary-index/
127 symbols architecture docs docs/modules/symbols/
128 cartographer architecture docs docs/modules/cartographer/
129 opsmemory architecture docs docs/modules/opsmemory/
130 airgap architecture docs docs/modules/airgap/
131 cryptography module docs docs/modules/cryptography/
132 plugin system architecture docs docs/modules/plugin/
133 CLI architecture docs docs/modules/cli/
134 web frontend architecture docs docs/modules/web/
135 telemetry architecture docs docs/modules/telemetry/
136 analytics architecture docs docs/modules/analytics/
137 mirror architecture docs docs/modules/mirror/
138 registry architecture docs docs/modules/registry/
139 verifier architecture docs docs/modules/verifier/
140 replay engine architecture docs docs/modules/replay/
141 feedser architecture docs docs/modules/feedser/
142 issuer directory architecture docs docs/modules/issuer-directory/
143 packs registry architecture docs docs/modules/packs-registry/
144 facet architecture docs docs/modules/facet/
145 devportal architecture docs docs/modules/devportal/
146 doctor architecture docs docs/modules/doctor/
147 bench tools architecture docs docs/modules/bench/
148 platform module docs docs/modules/platform/
149 gateway module docs docs/modules/gateway/
150 router module docs docs/modules/router/

1.5 Operations, Deployment & Runbooks (30 cases)

# Query Expected Entity Type Expected Match Source
151 deployment guide docs docs/operations/deployment/
152 production deployment docs docs/operations/deployment/
153 scaling guide docs docs/operations/
154 runbook incident response docs docs/runbooks/
155 emergency procedures docs docs/runbooks/
156 devops tooling docs docs/operations/devops/
157 operational governance docs docs/operations/governance/
158 handoff procedures docs docs/operations/handoff/
159 monitoring setup docs docs/technical/observability/
160 observability configuration docs docs/technical/observability/
161 Prometheus setup docs docs/technical/observability/
162 OpenTelemetry setup docs docs/technical/observability/
163 helm chart deployment docs docs/operations/deployment/
164 docker compose docs devops/compose/
165 backup procedures docs docs/operations/
166 disaster recovery docs docs/runbooks/
167 how to rotate keys docs docs/modules/signer/
168 certificate renewal docs docs/security/
169 log rotation configuration docs docs/operations/
170 performance testing playbook docs docs/dev/performance-testing-playbook.md
171 release notes docs docs/releases/
172 version history docs docs/releases/
173 upgrade guide docs docs/releases/
174 CI/CD pipeline docs docs/technical/cicd/
175 GitHub Actions integration docs docs/technical/cicd/
176 GitLab CI integration docs docs/technical/cicd/
177 Gitea workflow docs .gitea/
178 compliance audit docs docs/compliance/
179 governance structure docs docs/GOVERNANCE.md
180 third party dependencies docs docs/legal/THIRD-PARTY-DEPENDENCIES.md

1.6 Developer Guides & Plugin Development (30 cases)

# Query Expected Entity Type Expected Match Source
181 plugin development guide docs docs/PLUGIN_SDK_GUIDE.md
182 how to write a plugin docs docs/PLUGIN_SDK_GUIDE.md
183 authority plugin developer guide docs docs/dev/31_AUTHORITY_PLUGIN_DEVELOPER_GUIDE.md
184 excititor connector guide docs docs/dev/30_EXCITITOR_CONNECTOR_GUIDE.md
185 auth client guide docs docs/dev/32_AUTH_CLIENT_GUIDE.md
186 buildx plugin quickstart docs docs/dev/BUILDX_PLUGIN_QUICKSTART.md
187 extending binary analysis docs docs/dev/extending-binary-analysis.md
188 test fixture design docs docs/dev/fixtures.md
189 concelier CLI quickstart docs docs/CONCELIER_CLI_QUICKSTART.md
190 advisory ingestion docs docs/CONCELIER_CLI_QUICKSTART.md
191 SDK code generation docs docs/api/sdk-openapi-program.md
192 API CLI reference docs docs/API_CLI_REFERENCE.md
193 KISA connector docs docs/dev/kisa_connector_notes.md
194 semantic versioning merge docs docs/dev/merge_semver_playbook.md
195 normalized rule recipes docs docs/dev/normalized-rule-recipes.md
196 API contract standards docs docs/dev/contributing/api-contracts.md
197 canonicalization determinism docs docs/dev/contributing/canonicalization-determinism.md
198 corpus contribution guide docs docs/dev/contributing/corpus-contribution-guide.md
199 notification SDK examples docs docs/api/notify-sdk-examples.md
200 smart diff types docs docs/api/smart-diff-types.md
201 hybrid diff patching docs docs/hybrid-diff-patching.md
202 binary diff docs docs/samples/binary-diff/
203 binary analysis docs docs/dev/extending-binary-analysis.md
204 policy DSL docs docs/modules/policy/
205 policy studio contract docs docs/contracts/policy-studio.md
206 risk scoring contract docs docs/contracts/risk-scoring.md
207 triage suppress contract docs docs/contracts/triage-suppress-v1.md
208 verification policy docs docs/contracts/verification-policy.md
209 redaction defaults docs docs/contracts/redaction-defaults-decision.md
210 mirror bundle format docs docs/contracts/mirror-bundle.md

1.7 Benchmarks & Competitive Analysis (20 cases)

# Query Expected Entity Type Expected Match Source
211 benchmark results docs docs/benchmarks/
212 performance baselines docs docs/benchmarks/performance-baselines.md
213 accuracy metrics docs docs/benchmarks/accuracy-metrics-framework.md
214 golden corpus docs docs/benchmarks/golden-corpus-kpis.md
215 Trivy comparison docs docs/benchmarks/scanner-feature-comparison-trivy.md
216 Snyk comparison docs docs/benchmarks/scanner-feature-comparison-snyk.md
217 Grype comparison docs docs/benchmarks/scanner-feature-comparison-grype.md
218 competitive landscape docs docs/product/competitive-landscape.md
219 fidelity metrics docs docs/benchmarks/fidelity-metrics.md
220 precision recall curves docs docs/benchmarks/tiered-precision-curves.md
221 Rust analyzer docs docs/benchmarks/scanner-rust-analyzer.md
222 scanning gaps docs docs/benchmarks/scanner/
223 dotnet scanning docs docs/benchmarks/scanner/deep-dives/dotnet.md
224 Java scanning docs docs/benchmarks/scanner/deep-dives/java.md
225 Python scanning docs docs/benchmarks/scanner/deep-dives/python.md
226 Node.js scanning docs docs/benchmarks/scanner/deep-dives/nodejs.md
227 Golang scanning docs docs/benchmarks/scanner/deep-dives/golang.md
228 SAST analysis docs docs/benchmarks/scanner/deep-dives/sast.md
229 secrets scanning benchmark docs docs/benchmarks/scanner/deep-dives/secrets.md
230 Windows macOS scanning docs docs/benchmarks/scanner/windows-macos-demand.md

Domain 2: Knowledge — API Operations (OpenAPI specs)

2.1 Scanner API (40 cases)

# Query Expected Entity Type Expected Match Source
231 create a scan api POST /api/v1/scans
232 get scan status api GET /api/v1/scans/{scanId}
233 scan API api scanner/openapi.yaml
234 submit call graph api POST /api/v1/scans/{scanId}/callgraphs
235 stream scan events api GET /api/v1/scans/{scanId}/events
236 reachability API api scanner reachability endpoints
237 SBOM upload API api POST /api/v1/sboms/upload
238 layer SBOM api LayerSbomEndpoints
239 scan entropy api POST /api/v1/scans/{scanId}/entropy
240 delta compare API api DeltaCompareEndpoints
241 delta evidence api DeltaEvidenceEndpoints
242 manifest endpoint api ManifestEndpoints
243 SBOM hot lookup api SbomHotLookupEndpoints
244 proof spine API api ProofSpineEndpoints
245 witness endpoint api WitnessEndpoints
246 scanner health api HealthEndpoints
247 call graph endpoint api CallGraphEndpoints
248 validation endpoint api ValidationEndpoints
249 offline kit endpoint api OfflineKitEndpoints
250 fidelity endpoint api FidelityEndpoints
251 score replay API api ScoreReplayEndpoints
252 EPSS scores API api EpssEndpoints
253 approval endpoint api ApprovalEndpoints
254 baseline endpoint api BaselineEndpoints
255 counterfactual analysis API api CounterfactualEndpoints
256 actionables endpoint api ActionablesEndpoints
257 secret detection settings api SecretDetectionSettingsEndpoints
258 smart diff endpoint api SmartDiffEndpoints
259 unknowns endpoint api UnknownsEndpoints
260 triage API api Triage/*Endpoints
261 reachability slice api SliceEndpoints
262 GitHub code scanning api GitHubCodeScanningEndpoints
263 scanner webhook api WebhookEndpoints
264 runtime analysis API api RuntimeEndpoints
265 reachability evidence api ReachabilityEvidenceEndpoints
266 reachability stack api ReachabilityStackEndpoints
267 scan report generation api ReportEndpoints
268 scan evidence query api EvidenceEndpoints
269 sources tracking API api SourcesEndpoints
270 scan observability api ObservabilityEndpoints

2.2 Policy Engine API (40 cases)

# Query Expected Entity Type Expected Match Source
271 verification policy API api VerificationPolicyEndpoints
272 policy pack API api PolicyPackEndpoints
273 policy snapshot api PolicySnapshotEndpoints
274 violation tracking API api ViolationEndpoints
275 policy override API api OverrideEndpoints
276 risk budget API api BudgetEndpoints, RiskBudgetEndpoints
277 risk profile API api RiskProfileEndpoints
278 risk simulation API api RiskSimulationEndpoints
279 effective policy API api EffectivePolicyEndpoints
280 policy decision endpoint api PolicyDecisionEndpoint
281 batch evaluation API api BatchEvaluationEndpoint
282 policy conflict API api ConflictEndpoints
283 CVSS receipt endpoint api CvssReceiptEndpoints
284 attestation report API api AttestationReportEndpoints
285 policy export api ConsoleExportEndpoints
286 scope attachment API api ScopeAttachmentEndpoints
287 staleness endpoint api StalenessEndpoints
288 sealed mode API api SealedModeEndpoints
289 policy lint API api PolicyLintEndpoints
290 policy compilation api PolicyCompilationEndpoints
291 verify determinism API api VerifyDeterminismEndpoints
292 merge preview API api MergePreviewEndpoints
293 policy editor API api VerificationPolicyEditorEndpoints
294 air gap notification API api AirGapNotificationEndpoints
295 determinization config api DeterminizationConfigEndpoints
296 delta if present api DeltaIfPresentEndpoints
297 trust weighting API api TrustWeightingEndpoint
298 overlay simulation api OverlaySimulationEndpoint
299 path scope simulation api PathScopeSimulationEndpoint
300 evidence summary API api EvidenceSummaryEndpoint
301 policy pack bundle api PolicyPackBundleEndpoints
302 risk profile air gap api RiskProfileAirGapEndpoints
303 risk profile schema api RiskProfileSchemaEndpoints
304 console simulation api ConsoleSimulationEndpoint
305 policy worker api PolicyWorkerEndpoint
306 advisory AI knobs api AdvisoryAiKnobsEndpoint
307 profile event tracking api ProfileEventEndpoints
308 profile export api ProfileExportEndpoints
309 batch context API api BatchContextEndpoint
310 orchestrator job API api OrchestratorJobEndpoint

2.3 Orchestrator, Scheduler & Release API (30 cases)

# Query Expected Entity Type Expected Match Source
311 release API api ReleaseEndpoints
312 approval workflow API api ApprovalEndpoints
313 DAG query API api DagEndpoints
314 circuit breaker API api CircuitBreakerEndpoints
315 quota governance API api QuotaGovernanceEndpoints
316 audit trail API api AuditEndpoints
317 release dashboard API api ReleaseDashboardEndpoints
318 run execution API api RunEndpoints
319 event stream websocket api StreamEndpoints
320 KPI endpoint api KpiEndpoints
321 job management API api JobEndpoints
322 first signal API api FirstSignalEndpoints
323 export job API api ExportJobEndpoints
324 dead letter queue API api DeadLetterEndpoints
325 SLO management API api SloEndpoints
326 source tracking API api SourceEndpoints
327 schedule management API api ScheduleEndpoints
328 policy simulation API api PolicySimulationEndpointExtensions
329 graph job API api GraphJobEndpointExtensions
330 failure signature API api FailureSignatureEndpoints
331 event webhook API api EventWebhookEndpointExtensions
332 resolver job API api ResolverJobEndpointExtensions
333 worker coordination API api WorkerEndpoints
334 scale auto-scaling API api ScaleEndpoints
335 pack registry API api PackRegistryEndpoints
336 pack run API api PackRunEndpoints
337 ledger query API api LedgerEndpoints
338 release control v2 api ReleaseControlV2Endpoints
339 openapi discovery endpoint api OpenApiEndpoints
340 health check API api HealthEndpoints

2.4 Platform, Authority & Notification API (30 cases)

# Query Expected Entity Type Expected Match Source
341 platform health API api PlatformEndpoints
342 quota summary API api PlatformEndpoints
343 environment settings API api EnvironmentSettingsEndpoints
344 security read model api SecurityReadModelEndpoints
345 integration read model api IntegrationReadModelEndpoints
346 topology query API api TopologyReadModelEndpoints
347 analytics data API api AnalyticsEndpoints
348 score calculation API api ScoreEndpoints
349 function map API api FunctionMapEndpoints
350 evidence thread API api EvidenceThreadEndpoints
351 federation telemetry API api FederationTelemetryEndpoints
352 trust signing admin API api AdministrationTrustSigningMutationEndpoints
353 OAuth token endpoint api Authority endpoints
354 OIDC discovery api Authority endpoints
355 token introspection api Authority endpoints
356 JWKS endpoint api Authority endpoints
357 notification rules API api RuleEndpoints
358 notification template API api TemplateEndpoints
359 incident tracking API api IncidentEndpoints
360 storm breaker API api StormBreakerEndpoints
361 throttle API api ThrottleEndpoints
362 quiet hours API api QuietHoursEndpoints
363 escalation rules API api EscalationEndpoints
364 notification simulation api SimulationEndpoints
365 operator override API api OperatorOverrideEndpoints
366 notification localization api LocalizationEndpoints
367 live incident feed api IncidentLiveFeed
368 context management API api ContextEndpoints
369 seed database API api SeedEndpoints
370 setup wizard API api SetupEndpoints

2.5 Evidence, Attestation, VEX & Export API (30 cases)

# Query Expected Entity Type Expected Match Source
371 unified search API api POST /v1/search/query
372 knowledge search API api POST /v1/advisory-ai/search
373 advisory AI chat API api ChatEndpoints
374 LLM adapter API api LlmAdapterEndpoints
375 evidence pack API api EvidencePackEndpoints
376 verdict issuance API api VerdictEndpoints
377 predicate registry API api PredicateRegistryEndpoints
378 watchlist API api WatchlistEndpoints
379 export API api ExportApiEndpoints
380 risk bundle API api RiskBundleEndpoints
381 audit bundle API api AuditBundleEndpoints
382 promotion attestation API api PromotionAttestationEndpoints
383 lineage export API api LineageExportEndpoints
384 exception report API api ExceptionReportEndpoints
385 feed mirror API api FeedMirrorManagementEndpoints
386 SBOM ingestion API api SbomEndpointExtensions
387 canonical advisory API api CanonicalAdvisoryEndpointExtensions
388 advisory source API api AdvisorySourceEndpointExtensions
389 federation API api FederationEndpointExtensions
390 air gap endpoint api AirGapEndpointExtensions
391 findings scoring API api ScoringEndpoints
392 runtime traces API api RuntimeTracesEndpoints
393 evidence graph API api EvidenceGraphEndpoints
394 finding summary API api FindingSummaryEndpoints
395 backport API api BackportEndpoints
396 reachability map API api ReachabilityMapEndpoints
397 VEX ingest API api IngestEndpoints
398 linkset API api LinksetEndpoints
399 observation API api ObservationEndpoints
400 Rekor attestation API api RekorAttestationEndpoints

2.6 Gateway, Policy Gateway, Graph & More (30 cases)

# Query Expected Entity Type Expected Match Source
401 registry webhook API api RegistryWebhookEndpoints
402 gate endpoint api GateEndpoints
403 score gate API api ScoreGateEndpoints
404 exception management API api ExceptionEndpoints
405 exception approval API api ExceptionApprovalEndpoints
406 governance API api GovernanceEndpoints
407 delta tracking API api DeltasEndpoints
408 tool lattice API api ToolLatticeEndpoints
409 signing ceremony API api CeremonyEndpoints
410 key rotation API api KeyRotationEndpoints
411 signer endpoint api SignerEndpoints
412 timeline query API api TimelineEndpoints
413 timeline replay API api ReplayEndpoints
414 timeline export API api ExportEndpoints
415 graph search API api Graph search contracts
416 reachgraph query api ReachGraph endpoints
417 binary vulnerability API api BinaryIndex endpoints
418 remediation registry API api Remediation endpoints
419 symbol source API api Symbols endpoints
420 VEX hub export API api VexHub endpoints
421 issuer management API api IssuerDirectory endpoints
422 evidence verdict API api EvidenceLocker VerdictEndpoints
423 evidence thread audit api EvidenceThreadEndpoints
424 evidence audit trail api EvidenceAuditEndpoints
425 evidence export API api EvidenceLocker ExportEndpoints
426 resolve VEX API api ResolveEndpoint
427 risk feed API api RiskFeedEndpoints
428 VEX policy API api PolicyEndpoints (Excititor)
429 mirror registration API api MirrorRegistrationEndpoints
430 interest score API api InterestScoreEndpointExtensions

Domain 3: Knowledge — Doctor Checks

3.1 Database & Infrastructure Checks (20 cases)

# Query Expected Entity Type Expected Match Source
431 check.postgres.connectivity doctor Postgres Connectivity check
432 database connection failing doctor check.postgres.connectivity
433 postgres migrations pending doctor check.postgres.migrations
434 connection pool exhausted doctor check.postgres.pool
435 disk space running low doctor check.storage.diskspace
436 evidence locker write check doctor check.storage.evidencelocker
437 backup directory writable doctor check.storage.backup
438 log directory check doctor check.logs.directory.writable
439 log rotation check doctor check.logs.rotation.configured
440 Prometheus scrape check doctor check.metrics.prometheus.scrape
441 OTLP endpoint check doctor check.telemetry.otlp.endpoint
442 dead letter queue check doctor check.operations.dead-letter
443 job queue health check doctor check.operations.job-queue
444 scheduler health check doctor check.operations.scheduler
445 policy engine health doctor check.policy.engine
446 scanner queue check doctor check.scanner.queue
447 scanner resource utilization doctor check.scanner.resources
448 SBOM generation check doctor check.scanner.sbom
449 vulnerability scan check doctor check.scanner.vuln
450 witness graph check doctor check.scanner.witness.graph

3.2 Security & Auth Checks (20 cases)

# Query Expected Entity Type Expected Match Source
451 authentication config check doctor check.auth.config
452 OIDC provider connectivity doctor check.auth.oidc
453 signing key health doctor check.auth.signing-key
454 token service health doctor check.auth.token-service
455 certificate chain validation doctor check.crypto.certchain
456 FIPS compliance check doctor check.crypto.fips
457 HSM availability check doctor check.crypto.hsm
458 eIDAS compliance check doctor check.crypto.eidas
459 GOST availability check doctor check.crypto.gost
460 SM crypto check doctor check.crypto.sm
461 Rekor connectivity check doctor check.attestation.rekor.connectivity
462 clock skew check doctor check.attestation.clock.skew
463 cosign key material doctor check.attestation.cosign.keymaterial
464 signing key expiration doctor check.attestation.keymaterial
465 transparency log consistency doctor check.attestation.transparency.consistency
466 Rekor verification job doctor check.attestation.rekor.verification.job
467 VEX issuer trust check doctor check.vex.issuer-trust
468 VEX schema compliance check doctor check.vex.schema
469 VEX document validation doctor check.vex.validation
470 environment secrets check doctor check.environment.secrets

3.3 Compliance, Agent & Notification Checks (25 cases)

# Query Expected Entity Type Expected Match Source
471 audit readiness check doctor check.compliance.audit-readiness
472 evidence integrity check doctor check.compliance.evidence-integrity
473 provenance completeness doctor check.compliance.provenance-completeness
474 attestation signing health doctor check.compliance.attestation-signing
475 evidence generation rate doctor check.compliance.evidence-rate
476 export readiness check doctor check.compliance.export-readiness
477 compliance framework check doctor check.compliance.framework
478 evidence locker index doctor check.evidencelocker.index
479 merkle tree anchor doctor check.evidencelocker.merkle
480 provenance chain check doctor check.evidencelocker.provenance
481 attestation retrieval doctor check.evidencelocker.retrieval
482 agent heartbeat freshness doctor check.agent.heartbeat.freshness
483 agent capacity check doctor check.agent.capacity
484 stale agent detection doctor check.agent.stale
485 agent cluster health doctor check.agent.cluster.health
486 agent cluster quorum doctor check.agent.cluster.quorum
487 agent version consistency doctor check.agent.version.consistency
488 agent certificate expiry doctor check.agent.certificate.expiry
489 agent task backlog doctor check.agent.task.backlog
490 email notification check doctor check.notify.email.configured
491 Slack connectivity check doctor check.notify.slack.connectivity
492 Teams notification check doctor check.notify.teams.configured
493 notification queue health doctor check.notify.queue.health
494 webhook connectivity doctor check.notify.webhook.connectivity
495 TSA response time check doctor check.timestamp.tsa.response-time

3.4 Environment & Release Checks (15 cases)

# Query Expected Entity Type Expected Match Source
496 environment connectivity doctor check.environment.connectivity
497 environment drift doctor check.environment.drift
498 network policy enforcement doctor check.environment.network.policy
499 environment capacity doctor check.environment.capacity
500 deployment health check doctor check.environment.deployments
501 active release health doctor check.release.active
502 release configuration check doctor check.release.configuration
503 environment readiness doctor check.release.environment.readiness
504 promotion gates check doctor check.release.promotion.gates
505 rollback readiness doctor check.release.rollback.readiness
506 release schedule check doctor check.release.schedule
507 reachability computation check doctor check.scanner.reachability
508 slice cache check doctor check.scanner.slice.cache
509 buildinfo cache check doctor check.binaryanalysis.buildinfo.cache
510 debuginfod availability doctor check.binaryanalysis.debuginfod.available

Domain 4: Findings (Security Findings & Vulnerabilities)

4.1 CVE Searches (50 cases)

# Query Expected Entity Type Expected Match Source
511 CVE-2024-21626 finding Container escape via runc
512 CVE-2024-3094 finding XZ Utils backdoor
513 CVE-2023-44487 finding HTTP/2 Rapid Reset
514 CVE-2021-44228 finding Log4Shell
515 CVE-2021-45046 finding Log4j followup
516 CVE-2023-4863 finding libwebp heap overflow
517 CVE-2024-0056 finding .NET SQL injection
518 CVE-2023-38545 finding curl SOCKS5 overflow
519 CVE-2023-32233 finding Linux kernel nf_tables
520 CVE-2024-6387 finding OpenSSH regreSSHion
521 Log4Shell finding CVE-2021-44228
522 Heartbleed finding CVE-2014-0160
523 Spring4Shell finding CVE-2022-22965
524 Shellshock finding CVE-2014-6271
525 POODLE finding CVE-2014-3566
526 critical vulnerabilities finding severity=CRITICAL
527 high severity findings finding severity=HIGH
528 remote code execution finding CWE-94
529 SQL injection vulnerability finding CWE-89
530 buffer overflow finding CWE-120
531 cross site scripting finding CWE-79
532 privilege escalation finding various CWEs
533 denial of service finding CWE-400
534 path traversal finding CWE-22
535 deserialization vulnerability finding CWE-502
536 SSRF vulnerability finding CWE-918
537 integer overflow finding CWE-190
538 use after free finding CWE-416
539 null pointer dereference finding CWE-476
540 race condition finding CWE-362
541 CVSS score 9.8 finding CVSS filter
542 CVSS greater than 7 finding CVSS filter
543 exploit available finding exploitKnown=true
544 zero day vulnerability finding recent, no patch
545 EPSS score high finding EPSS > 0.5
546 findings for log4j finding package=log4j
547 openssl vulnerabilities finding package=openssl
548 npm lodash vulnerability finding pkg:npm/lodash
549 jackson-databind CVE finding pkg:maven/jackson-databind
550 spring framework vulnerability finding spring-framework
551 golang net/http vulnerability finding pkg:golang/net
552 python requests vulnerability finding pkg:pypi/requests
553 ruby on rails CVE finding pkg:gem/rails
554 docker runc vulnerability finding pkg:golang/runc
555 kubernetes vulnerability finding kubernetes
556 nginx CVE finding nginx
557 apache httpd vulnerability finding apache httpd
558 postgresql vulnerability finding postgresql
559 redis vulnerability finding redis
560 alpine linux CVE finding alpine

4.2 PURL & Package Searches (30 cases)

# Query Expected Entity Type Expected Match Source
561 pkg:npm/lodash@4.17.21 finding npm lodash
562 pkg:maven/org.apache.logging.log4j/log4j-core@2.17.0 finding log4j-core
563 pkg:pypi/django@4.2 finding Django
564 pkg:cargo/tokio@1.28 finding tokio
565 pkg:golang/github.com/opencontainers/runc@1.1.10 finding runc
566 pkg:nuget/Newtonsoft.Json@13.0.3 finding Newtonsoft.Json
567 pkg:gem/actionpack@7.0 finding Rails actionpack
568 pkg:composer/symfony/http-kernel finding Symfony
569 pkg:npm/express@4.18 finding Express.js
570 pkg:npm/axios@1.6 finding Axios
571 affected packages npm finding npm ecosystem
572 affected packages maven finding Maven ecosystem
573 affected packages pip finding PyPI ecosystem
574 affected packages cargo finding Cargo/Rust ecosystem
575 affected packages alpine finding Alpine Linux
576 affected packages debian finding Debian
577 affected packages ubuntu finding Ubuntu
578 affected packages centos finding CentOS
579 packages with known exploits finding exploitKnown=true
580 packages with critical severity finding severity=CRITICAL
581 transitive dependencies vulnerable finding transitive deps
582 outdated packages security finding version range
583 library vulnerabilities finding library scan
584 container base image vulnerabilities finding container scan
585 OS package vulnerabilities finding OS scan
586 runtime dependency security finding runtime deps
587 development dependency vulnerability finding dev deps
588 binary vulnerability finding binary analysis
589 Go module vulnerability finding Go modules
590 .NET NuGet vulnerability finding NuGet packages

4.3 GHSA & Source Searches (20 cases)

# Query Expected Entity Type Expected Match Source
591 GHSA-xxxx-yyyy-zzzz finding GitHub Security Advisory
592 GitHub advisory finding GHSA source
593 NVD advisory finding NVD source
594 CISA advisory finding CISA source
595 Microsoft security advisory finding MSRC source
596 Ubuntu security notice finding USN source
597 SUSE security advisory finding SUSE source
598 Alpine security advisory finding Alpine source
599 Red Hat security advisory finding RHSA source
600 Debian security advisory finding DSA source
601 Cisco advisory finding Cisco source
602 Oracle security advisory finding Oracle source
603 ENISA advisory finding ENISA source
604 JVN advisory finding JVN (Japan) source
605 BDU advisory finding BDU (Russia) source
606 CNNVD advisory finding CNNVD (China) source
607 CNVD advisory finding CNVD (China) source
608 advisories published today finding date filter
609 advisories modified this week finding date filter
610 recently discovered vulnerabilities finding date filter

Domain 5: VEX (Vulnerability Exploitability Exchange)

5.1 VEX Status & Justification Searches (30 cases)

# Query Expected Entity Type Expected Match Source
611 VEX not affected vex_statement status=not_affected
612 VEX affected vex_statement status=affected
613 VEX fixed vex_statement status=fixed
614 VEX under investigation vex_statement status=under_investigation
615 component not present justification vex_statement justification
616 vulnerable code not present vex_statement justification
617 code not in execute path vex_statement justification
618 code not executable vex_statement justification
619 adversary cannot control code vex_statement justification
620 inline mitigations exist vex_statement justification
621 VEX for CVE-2024-21626 vex_statement vulnerability match
622 VEX for log4j vex_statement package match
623 VEX from vendor vex_statement issuer=VENDOR
624 VEX from community vex_statement issuer=COMMUNITY
625 trusted VEX statements vex_statement trust=TRUSTED
626 authoritative VEX vex_statement trust=AUTHORITATIVE
627 OpenVEX document vex_statement format=openvex
628 CSAF VEX document vex_statement format=csaf
629 CycloneDX VEX vex_statement format=cyclonedx
630 VEX consensus conflict vex_statement conflict resolution
631 VEX statement for production vex_statement environment filter
632 VEX impact statement vex_statement impactStatement field
633 VEX action required vex_statement actionStatement field
634 VEX expiring soon vex_statement TTL/freshness
635 VEX signature verification vex_statement signature check
636 VEX trust profile vex_statement trust profile config
637 VEX override vex_statement manual override
638 how to write VEX vex_statement + docs VEX documentation
639 VEX schema validation vex_statement + doctor check.vex.schema
640 VEX issuer directory vex_statement issuer lookup

5.2 VEX Workflow & Integration (20 cases)

# Query Expected Entity Type Expected Match Source
641 generate VEX document vex_statement CLI stella vex-gen
642 ingest VEX statement vex_statement IngestEndpoints
643 VEX hub search vex_statement VexHub endpoints
644 VEX studio create vex_statement Web VEX Studio
645 VEX timeline view vex_statement Web VEX Timeline
646 VEX gate scan vex_statement VexGateScan feature
647 export VEX bundle vex_statement VexHub export
648 VEX evidence proof vex_statement docs/api/vex-proof-schema.md
649 VEX consensus handling vex_statement docs/VEX_CONSENSUS_GUIDE.md
650 multiple VEX sources disagree vex_statement conflict resolution
651 VEX trust weighting vex_statement trust weight config
652 VEX freshness scoring vex_statement TTL/staleness
653 VEX linked to finding vex_statement + finding linkset
654 VEX suppresses finding vex_statement suppression logic
655 VEX as evidence vex_statement evidence pipeline
656 VEX attestation vex_statement attestation predicate
657 VEX policy evaluation vex_statement + policy policy gate
658 VEX mirror vex_statement mirror endpoints
659 VEX feed subscription vex_statement feed mirror
660 VEX document lifecycle vex_statement lifecycle docs

Domain 6: Policy (Policy Rules, Evaluations, Violations)

6.1 Policy Management Searches (30 cases)

# Query Expected Entity Type Expected Match Source
661 create policy rule policy_rule Policy Studio
662 policy pack install policy_rule CLI stella policy install
663 validate policy YAML policy_rule stella policy validate-yaml
664 policy simulation policy_rule stella policy simulate
665 push policy to OCI policy_rule stella policy push
666 pull policy from registry policy_rule stella policy pull
667 policy pack bundle policy_rule export/import bundle
668 block critical vulnerabilities policy_rule severity gate rule
669 require SBOM attestation policy_rule attestation requirement
670 require VEX for all CVEs policy_rule VEX requirement
671 maximum CVSS score allowed policy_rule score threshold
672 block exploit available policy_rule exploit gate
673 require reachability proof policy_rule reachability gate
674 policy for production environment policy_rule scope=production
675 policy for staging environment policy_rule scope=staging
676 policy exception request policy_rule exception management
677 policy waiver policy_rule exception/override
678 risk budget remaining policy_rule budget tracking
679 policy violation list policy_rule violation tracking
680 why was release blocked policy_rule decision audit
681 policy decision audit trail policy_rule decision log
682 effective policy for artifact policy_rule computed policy
683 policy merge preview policy_rule merge simulation
684 policy conflict detection policy_rule conflict analysis
685 policy determinism verification policy_rule determinism check
686 policy lint check policy_rule lint validation
687 policy compilation policy_rule compile pipeline
688 sealed mode policy policy_rule air gap mode
689 staleness rule configuration policy_rule staleness config
690 risk profile definition policy_rule risk profile

6.2 Policy Evaluation & Decisioning (20 cases)

# Query Expected Entity Type Expected Match Source
691 evaluate policy for container policy_rule batch evaluation
692 policy APPROVE decision policy_rule decision=APPROVE
693 policy REJECT decision policy_rule decision=REJECT
694 conditional approval policy_rule decision=CONDITIONAL
695 blocked by policy policy_rule decision=BLOCKED
696 awaiting approval policy_rule decision=AWAITING
697 override policy violation policy_rule override endpoint
698 severity fusion scoring policy_rule severity fusion
699 CVSS receipt for finding policy_rule CVSS scoring
700 attestation report for release policy_rule attestation report
701 promotion gate evaluation policy_rule gate check
702 batch policy assessment policy_rule batch evaluation
703 policy snapshot comparison policy_rule snapshot diff
704 risk budget consumption policy_rule budget tracking
705 unknowns budget exceeded policy_rule unknowns tracking
706 confidence score low policy_rule confidence scoring
707 evidence freshness expired policy_rule staleness check
708 trust weight configuration policy_rule trust weighting
709 overlay simulation results policy_rule overlay sim
710 path scope simulation policy_rule path scoping

Domain 7: Cross-Domain Natural Language Queries (290 cases)

7.1 Troubleshooting Queries (50 cases)

# Query Expected Entity Type Expected Match Source
711 why is the build failing mixed doctor + findings
712 scan is stuck doctor + api scanner queue check
713 cannot connect to database doctor check.postgres.connectivity
714 authentication failed doctor check.auth.config
715 token expired doctor check.auth.token-service
716 certificate invalid doctor check.crypto.certchain
717 signing failed doctor check.attestation.keymaterial
718 evidence not found doctor check.evidencelocker.retrieval
719 notification not delivered doctor check.notify.queue.health
720 release promotion failed doctor check.release.promotion.gates
721 agent not responding doctor check.agent.heartbeat.freshness
722 out of disk space doctor check.storage.diskspace
723 policy evaluation timeout doctor check.policy.engine
724 reachability analysis slow doctor check.scanner.reachability
725 VEX validation failed doctor check.vex.validation
726 email notification not working doctor check.notify.email.connectivity
727 Slack integration broken doctor check.notify.slack.connectivity
728 environment drift detected doctor check.environment.drift
729 clock skew error doctor check.attestation.clock.skew
730 HSM not available doctor check.crypto.hsm
731 debug scan failure docs + doctor troubleshooting
732 fix deployment error docs runbooks
733 container crash investigation docs troubleshooting
734 error 403 forbidden docs + api auth scopes
735 error 404 not found docs + api endpoint reference
736 error 500 internal server docs troubleshooting
737 connection refused doctor connectivity checks
738 timeout error docs timeout configuration
739 memory leak docs performance troubleshooting
740 high CPU usage doctor check.agent.resource.utilization
741 slow query performance docs database tuning
742 migration failed doctor check.postgres.migrations
743 index corruption doctor check.evidencelocker.index
744 merkle tree inconsistency doctor check.evidencelocker.merkle
745 provenance chain broken doctor check.evidencelocker.provenance
746 agent task failure rate high doctor check.agent.task.failure.rate
747 quorum lost doctor check.agent.cluster.quorum
748 rollback not working doctor check.release.rollback.readiness
749 export failed doctor check.compliance.export-readiness
750 compliance audit failure doctor check.compliance.audit-readiness
751 evidence tampering detected doctor check.compliance.evidence-integrity
752 no evidence generated doctor check.compliance.evidence-rate
753 symbol recovery failed doctor check.binaryanalysis.symbol.recovery.fallback
754 debuginfod unavailable doctor check.binaryanalysis.debuginfod.available
755 TSA endpoint slow doctor check.timestamp.tsa.response-time
756 timestamp validation failed doctor check.timestamp.tsa.valid-response
757 secret detected in code finding secret detection
758 credentials in repository finding secret detection
759 API key leaked finding secret detection
760 hardcoded password finding secret detection

7.2 How-To & Workflow Queries (50 cases)

# Query Expected Entity Type Expected Match Source
761 how to scan a container docs + api scanner docs
762 how to create a release docs + api release docs
763 how to promote to production docs release orchestration
764 how to triage a finding docs triage workflow
765 how to suppress a vulnerability docs triage suppress
766 how to generate SBOM docs + api scanner SBOM
767 how to write a VEX statement docs VEX guide
768 how to configure notifications docs notify setup
769 how to set up policy gates docs policy gates
770 how to configure risk budget docs risk budget
771 how to export evidence docs + api export center
772 how to verify attestation docs + api attestor
773 how to configure air gap mode docs offline kit
774 how to rotate signing keys docs key rotation
775 how to onboard new environment docs environment setup
776 how to register agent docs agent onboarding
777 how to integrate GitHub docs integration guide
778 how to configure OIDC docs auth setup
779 how to set up monitoring docs observability
780 how to run doctor checks docs + doctor stella doctor
781 how to create policy exception docs exception workflow
782 how to handle policy violation docs violation handling
783 how to investigate reachability docs reachability guide
784 how to generate call graph docs + api call graph
785 how to compare scans docs + api delta compare
786 how to export SARIF report docs + api SARIF export
787 how to configure Prometheus docs observability
788 how to set up email alerts docs notification config
789 how to configure escalation docs escalation rules
790 how to manage trust anchors docs trust management
791 how to deploy offline docs air gap deployment
792 how to mirror feeds docs + api feed mirror
793 how to verify provenance docs + api provenance
794 how to check compliance docs compliance tracker
795 how to configure secrets docs secrets management
796 how to set up federation docs federation
797 how to use binary diff docs binary diff
798 how to track changes docs change trace
799 how to configure quiet hours docs quiet hours
800 how to set up webhooks docs + api webhook config
801 how to use policy studio docs policy studio
802 how to create risk profile docs risk profile
803 how to run batch evaluation docs + api batch eval
804 how to configure determinism docs determinism
805 how to use sealed mode docs sealed mode
806 how to track unknowns docs unknowns management
807 how to investigate incidents docs incident management
808 how to use advisory AI docs advisory AI
809 how to configure autofix docs remediation
810 how to use evidence ribbon docs evidence UI

7.3 Navigation & Feature Discovery (50 cases)

# Query Expected Entity Type Expected Match Source
811 open settings docs navigation
812 go to findings docs navigation
813 show dashboard docs navigation
814 open security view docs navigation
815 go to policy gates docs navigation
816 open VEX hub docs navigation
817 show release history docs navigation
818 open agent fleet docs navigation
819 go to evidence center docs navigation
820 open export center docs navigation
821 show topology view docs navigation
822 open timeline docs navigation
823 go to triage inbox docs navigation
824 open approval queue docs navigation
825 show integrations docs navigation
826 open policy studio docs navigation
827 go to scan results docs navigation
828 open SBOM viewer docs navigation
829 show notifications docs navigation
830 open doctor diagnostics docs navigation
831 where is the audit log docs navigation
832 find the compliance dashboard docs navigation
833 where are risk budgets docs navigation
834 find exception management docs navigation
835 where is the remediation panel docs navigation
836 find the binary diff viewer docs navigation
837 where is the change trace docs navigation
838 find the scoring page docs navigation
839 where is the verdict viewer docs navigation
840 find the proof chain docs navigation
841 open advisory AI chat docs navigation
842 where is the setup wizard docs navigation
843 find the quota dashboard docs navigation
844 where is SLO monitoring docs navigation
845 find dead letter queue docs navigation
846 where is the deploy diff docs navigation
847 find the lineage view docs navigation
848 open mission control docs navigation
849 where is the function map docs navigation
850 find the vulnerability explorer docs navigation
851 open control plane docs navigation
852 show ops memory docs navigation
853 where is trust admin docs navigation
854 find the issuer trust page docs navigation
855 where are workspaces docs navigation
856 open pack registry docs navigation
857 find Trivy DB settings docs navigation
858 where is golden set docs navigation
859 open observations page docs navigation
860 find the signals dashboard docs navigation

7.4 CLI Command Searches (50 cases)

# Query Expected Entity Type Expected Match Source
861 stella release create docs CLI reference
862 stella release promote docs CLI reference
863 stella release rollback docs CLI reference
864 stella scan graph docs CLI reference
865 stella policy validate-yaml docs CLI reference
866 stella policy install docs CLI reference
867 stella policy simulate docs CLI reference
868 stella doctor run docs + doctor CLI + checks
869 stella vex generate docs CLI reference
870 stella evidence export docs CLI reference
871 stella attest sign docs CLI reference
872 stella verify docs CLI reference
873 stella config set docs CLI reference
874 stella db migrate docs CLI reference
875 stella export bundle docs CLI reference
876 stella import bundle docs CLI reference
877 stella airgap prepare docs CLI reference
878 stella scan-graph dotnet docs CLI reference
879 stella scan-graph java docs CLI reference
880 stella scan-graph python docs CLI reference
881 stella agent status docs CLI reference
882 stella agent list docs CLI reference
883 stella crypto keygen docs CLI reference
884 stella keys rotate docs CLI reference
885 stella trust-anchors add docs CLI reference
886 stella timestamp verify docs CLI reference
887 stella score calculate docs CLI reference
888 stella verdict check docs CLI reference
889 stella sbom generate docs CLI reference
890 stella seal create docs CLI reference
891 stella witness add docs CLI reference
892 stella proof generate docs CLI reference
893 stella bundle verify docs CLI reference
894 stella notify test docs CLI reference
895 stella feeds sync docs CLI reference
896 stella registry login docs CLI reference
897 stella github connect docs CLI reference
898 stella delta compare docs CLI reference
899 stella binary diff docs CLI reference
900 stella change-trace analyze docs CLI reference
901 stella reachability check docs CLI reference
902 stella drift detect docs CLI reference
903 stella timeline query docs CLI reference
904 stella exception create docs CLI reference
905 stella incidents list docs CLI reference
906 stella signals ingest docs CLI reference
907 stella watchlist add docs CLI reference
908 stella admin config docs CLI reference
909 stella analytics report docs CLI reference
910 stella auth login docs CLI reference

7.5 Concept & Explanation Queries (50 cases)

# Query Expected Entity Type Expected Match Source
911 what is a VEX statement docs VEX docs
912 explain SBOM docs SBOM docs
913 what is reachability analysis docs reachability concept
914 explain attestation docs attestation docs
915 what is DSSE envelope docs attestation docs
916 explain in-toto format docs attestation docs
917 what is a policy gate docs policy docs
918 explain risk budget docs policy docs
919 what is severity fusion docs scoring docs
920 explain CVSS v4 docs + finding scoring docs
921 what is EPSS docs + finding scoring docs
922 explain decision capsule docs product/decision-capsules.md
923 what is deterministic replay docs replay docs
924 explain provenance docs provenance docs
925 what is a Merkle tree docs evidence locker docs
926 explain evidence chain docs evidence docs
927 what is sealed mode docs sealed mode docs
928 explain air gap operation docs offline docs
929 what is a trust anchor docs security docs
930 explain multi-tenant isolation docs tenant RBAC docs
931 what is content addressable storage docs CAS docs
932 explain smart diff docs smart diff docs
933 what is a linkset docs linkset docs
934 explain canonical SBOM ID docs canonical ID docs
935 what is the findings ledger docs findings docs
936 explain policy determinization docs policy docs
937 what is unknowns budgeting docs unknowns docs
938 explain confidence scoring docs scoring docs
939 what is change trace docs change trace docs
940 explain binary analysis docs binary docs
941 what is the evidence pipeline docs architecture docs
942 explain reciprocal rank fusion docs search docs
943 what is a policy pack docs policy docs
944 explain OCI registry for policy docs policy docs
945 what is a verdict docs verdict docs
946 explain proof spine docs proof docs
947 what is the witness format docs witness docs
948 explain execution evidence docs evidence docs
949 what is a federated consent docs federation docs
950 explain storm breaker docs notification docs
951 what is a dead letter queue docs operations docs
952 explain circuit breaker pattern docs orchestrator docs
953 what is DPoP authentication docs authority docs
954 explain OAuth 2.1 docs authority docs
955 what is PURL format docs + finding glossary
956 explain CWE weakness docs + finding glossary
957 what is SAST vs SCA docs scanner docs
958 explain runtime signals docs signals docs
959 what is an advisory source docs concelier docs
960 explain counterfactual analysis docs scanner docs

7.6 Comparison & Analysis Queries (40 cases)

# Query Expected Entity Type Expected Match Source
961 compare scan results api + docs DeltaCompareEndpoints
962 difference between VEX and advisory docs VEX guide
963 compare CVSS versions docs scoring docs
964 difference between SBOM and SPDX docs SBOM docs
965 compare policy packs api snapshot comparison
966 difference between Trivy and Stella docs benchmarks
967 compare Snyk scanner features docs benchmarks
968 SAST vs SCA differences docs scanner docs
969 compare environments api environment settings
970 delta between releases api delta compare
971 binary diff between versions api + docs binary diff
972 compare agent versions doctor check.agent.version.consistency
973 compare findings across scans api delta evidence
974 what changed since last scan api change trace
975 new vulnerabilities since yesterday finding date filter
976 resolved vulnerabilities this week finding status filter
977 score difference between environments api score endpoints
978 policy violation trends api analytics
979 risk profile changes api profile events
980 VEX status changes vex_statement timeline
981 evidence freshness comparison api staleness
982 compliance gap analysis docs compliance tracker
983 scanning coverage gaps docs benchmarks
984 trust score comparison api trust weighting
985 notification delivery rate api notification stats
986 scan duration trend api analytics
987 finding resolution velocity api analytics
988 MTTR for vulnerabilities api analytics
989 approval wait time api KPI endpoints
990 deployment frequency api analytics
991 reachability coverage percentage api reachability stats
992 SBOM completeness api SBOM analytics
993 attestation signing latency api performance metrics
994 evidence locker usage api storage stats
995 quota utilization api quota dashboard
996 SLO compliance rate api SLO monitoring
997 agent utilization heatmap api agent analytics
998 vulnerability backlog trend api + finding analytics
999 policy compliance over time api analytics
1000 risk budget burn rate api + policy_rule budget analytics

Bonus: Edge Case & Multi-Domain Queries (20 cases)

# Query Domains Hit Description
1001 CVE-2024-21626 runc escape reachability VEX finding + vex + docs Multi-domain: CVE + VEX + docs
1002 log4j affected not_affected VEX finding + vex Finding + conflicting VEX
1003 OPS-001 check failing production doctor + docs Doctor check + environment context
1004 policy violation critical CVE-2024-3094 policy_rule + finding Policy + finding cross-ref
1005 how to suppress CVE-2023-44487 docs + finding + vex How-to with specific CVE
1006 GHSA-xxxx for pkg:npm/express finding GHSA + PURL combined
1007 promote release with blocked findings docs + policy_rule Workflow + policy gate
1008 attestation failed for container scan doctor + docs Troubleshoot attestation
1009 VEX not_affected but policy still blocks vex + policy_rule Cross-domain conflict
1010 reachability shows vulnerable code not in execute path finding + vex + docs Reachability + VEX justification
1011 export SARIF report for compliance audit docs + api Export + compliance
1012 rotate signing keys in air gap mode docs + doctor Operations + environment
1013 agent cluster quorum lost during release doctor + docs Troubleshoot + release
1014 Slack notification for critical CVE findings doctor + docs + finding Multi-layer search
1015 binary diff shows new dependency vulnerability docs + finding Analysis + finding
1016 federation telemetry from remote tenant docs + api Multi-tenant ops
1017 sealed mode policy with HSM signing docs + doctor Air gap + crypto
1018 CVSS 9.8 EPSS 0.97 exploit known finding Multi-score filter
1019 unknown component in SBOM without VEX finding + vex + policy Unknowns workflow
1020 evidence bundle for in-toto SLSA attestation docs + api Evidence + attestation

Domain 3 Extended: Doctor Checks — Timestamping, Integration, Binary & Deep Checks

3.5 Timestamping & Certificate Lifecycle Checks (20 cases)

# Query Expected Entity Type Expected Match Source
1021 TSA availability check doctor check.timestamp.tsa.availability
1022 TSA response time doctor check.timestamp.tsa.response-time
1023 TSA valid response check doctor check.timestamp.tsa.valid-response
1024 TSA failover ready doctor check.timestamp.tsa.failover-ready
1025 TSA certificate expiry doctor check.timestamp.tsa.certificate-expiry
1026 TSA root expiry check doctor check.timestamp.tsa.root-expiry
1027 TSA chain validation doctor check.timestamp.tsa.chain-valid
1028 OCSP responder check doctor check.timestamp.ocsp.responder
1029 CRL distribution check doctor check.timestamp.crl.distribution
1030 revocation cache freshness doctor check.timestamp.revocation.cache-fresh
1031 OCSP stapling enabled doctor check.timestamp.ocsp.stapling-enabled
1032 evidence staleness check doctor check.timestamp.evidence-staleness
1033 timestamp approaching expiry doctor check.timestamp.tst.approaching-expiry
1034 TST algorithm deprecated doctor check.timestamp.tst.algorithm-deprecated
1035 TST missing stapling doctor check.timestamp.tst.missing-stapling
1036 retimestamp pending doctor check.timestamp.restamp.pending
1037 EU trust list freshness doctor check.timestamp.eu-trust-list-fresh
1038 QTS providers qualified doctor check.timestamp.qts.providers-qualified
1039 QTS status change doctor check.timestamp.qts.status-change
1040 system time synced doctor check.timestamp.system-time-synced

3.6 Integration & External Connectivity Checks (25 cases)

# Query Expected Entity Type Expected Match Source
1041 OCI registry connectivity doctor check.integration.oci.registry
1042 OCI referrers API check doctor check.integration.oci.referrers
1043 OCI capability matrix doctor check.integration.oci.capabilities
1044 OCI push authorization doctor check.integration.oci.push
1045 OCI pull authorization doctor check.integration.oci.pull
1046 OCI registry credentials doctor check.integration.oci.credentials
1047 S3 object storage check doctor check.integration.s3.storage
1048 SMTP connectivity check doctor check.integration.smtp
1049 Slack webhook check doctor check.integration.slack
1050 Teams webhook check doctor check.integration.teams
1051 Git provider connectivity doctor check.integration.git
1052 LDAP connectivity check doctor check.integration.ldap
1053 OIDC provider integration check doctor check.integration.oidc
1054 CI system connectivity doctor check.integration.ci.system
1055 secrets manager connectivity doctor check.integration.secrets.manager
1056 integration webhook health doctor check.integration.webhooks
1057 registry push permission denied doctor check.integration.oci.push
1058 cannot pull from OCI registry doctor check.integration.oci.pull
1059 LDAP authentication not working doctor check.integration.ldap
1060 CI pipeline broken connectivity doctor check.integration.ci.system
1061 cannot push policy to OCI doctor check.integration.oci.push
1062 Git provider auth failing doctor check.integration.git
1063 object storage write failing doctor check.integration.s3.storage
1064 secrets vault unreachable doctor check.integration.secrets.manager
1065 integration health dashboard doctor integration checks summary

3.7 Binary Analysis & Corpus Health Checks (20 cases)

# Query Expected Entity Type Expected Match Source
1066 debuginfod available doctor check.binaryanalysis.debuginfod.available
1067 ddeb repo enabled doctor check.binaryanalysis.ddeb.enabled
1068 buildinfo cache health doctor check.binaryanalysis.buildinfo.cache
1069 symbol recovery fallback doctor check.binaryanalysis.symbol.recovery.fallback
1070 corpus mirror freshness doctor check.binaryanalysis.corpus.mirror.freshness
1071 corpus KPI baseline exists doctor check.binaryanalysis.corpus.kpi.baseline
1072 binary analysis not working doctor check.binaryanalysis.*
1073 symbol table missing doctor check.binaryanalysis.symbol.recovery.fallback
1074 debug symbols not found doctor check.binaryanalysis.debuginfod.available
1075 buildinfo cache expired doctor check.binaryanalysis.buildinfo.cache
1076 Go binary stripped no debug doctor check.binaryanalysis.*
1077 PE authenticode verification failed doctor binary analysis checks
1078 Mach-O binary inspection failing doctor binary analysis checks
1079 corpus mirror out of date doctor check.binaryanalysis.corpus.mirror.freshness
1080 KPI baseline not established doctor check.binaryanalysis.corpus.kpi.baseline
1081 ddeb repository not configured doctor check.binaryanalysis.ddeb.enabled
1082 native runtime capture failure doctor binary analysis checks
1083 crypto material state check doctor binary crypto analysis
1084 binary vulnerability scan health doctor binary analysis checks
1085 symbol lookup performance degraded doctor check.binaryanalysis.debuginfod.available

3.8 Observability, Logging & Operations Deep Checks (15 cases)

# Query Expected Entity Type Expected Match Source
1086 OTLP exporter not sending doctor check.telemetry.otlp.endpoint
1087 log directory not writable doctor check.logs.directory.writable
1088 log rotation not configured doctor check.logs.rotation.configured
1089 Prometheus not scraping metrics doctor check.metrics.prometheus.scrape
1090 dead letter queue growing doctor check.operations.dead-letter
1091 job queue backlog increasing doctor check.operations.job-queue
1092 scheduler not processing doctor check.operations.scheduler
1093 traces not appearing in Jaeger doctor check.telemetry.otlp.endpoint
1094 metrics endpoint 404 doctor check.metrics.prometheus.scrape
1095 log files filling disk doctor check.logs.rotation.configured + check.storage.diskspace
1096 OpenTelemetry collector down doctor check.telemetry.otlp.endpoint
1097 dead letter messages accumulating doctor check.operations.dead-letter
1098 cron job scheduler missed run doctor check.operations.scheduler
1099 job retry limit exceeded doctor check.operations.job-queue
1100 observability pipeline health doctor observability checks summary

3.9 Scanner, Reachability & Storage Deep Checks (20 cases)

# Query Expected Entity Type Expected Match Source
1101 scanner queue backed up doctor check.scanner.queue
1102 SBOM generation failing doctor check.scanner.sbom
1103 vulnerability scan timing out doctor check.scanner.vuln
1104 witness graph corruption doctor check.scanner.witness.graph
1105 slice cache miss rate high doctor check.scanner.slice.cache
1106 reachability computation stalled doctor check.scanner.reachability
1107 scanner resource utilization high doctor check.scanner.resources
1108 disk space critical on evidence locker doctor check.storage.diskspace
1109 evidence locker write failure doctor check.storage.evidencelocker
1110 backup directory not accessible doctor check.storage.backup
1111 postgres connection pool exhausted doctor check.postgres.pool
1112 database migrations not applied doctor check.postgres.migrations
1113 postgres connectivity lost doctor check.postgres.connectivity
1114 scanner taking too long doctor check.scanner.resources
1115 reachability analysis incomplete doctor check.scanner.reachability
1116 call graph generation failed doctor check.scanner.*
1117 evidence index inconsistent doctor check.evidencelocker.index
1118 merkle tree anchor verification failed doctor check.evidencelocker.merkle
1119 provenance chain incomplete doctor check.evidencelocker.provenance
1120 attestation retrieval timeout doctor check.evidencelocker.retrieval

Domain 4 Extended: Findings — Secret Detection, Reachability, Binary & Triage

4.4 Secret Detection & Credential Findings (25 cases)

# Query Expected Entity Type Expected Match Source
1121 AWS access key exposed finding secret detection - critical
1122 GitHub personal access token finding secret detection - high
1123 private SSH key in repository finding secret detection - critical
1124 database password hardcoded finding secret detection - high
1125 Slack webhook URL leaked finding secret detection - medium
1126 Azure connection string exposed finding secret detection - high
1127 Docker registry credentials finding secret detection - high
1128 JWT secret key in code finding secret detection - critical
1129 Stripe API key leaked finding secret detection - high
1130 Google Cloud service account key finding secret detection - critical
1131 npm auth token finding secret detection - medium
1132 Twilio account SID exposed finding secret detection - medium
1133 SendGrid API key finding secret detection - medium
1134 PKCS#12 certificate with private key finding secret detection - critical
1135 environment file with secrets finding secret detection - high
1136 Terraform state with credentials finding secret detection - critical
1137 Kubernetes secret in YAML finding secret detection - high
1138 PGP private key committed finding secret detection - critical
1139 OAuth client secret exposed finding secret detection - high
1140 Redis AUTH password in config finding secret detection - medium
1141 SMTP credentials in source finding secret detection - medium
1142 encryption key in code finding secret detection - high
1143 API key rotation needed finding secret detection - medium
1144 credential severity critical finding secret detection filter
1145 all secret detections this week finding secret detection date filter

4.5 Reachability & Runtime Analysis Findings (25 cases)

# Query Expected Entity Type Expected Match Source
1146 reachable CVE findings finding reachability=Reachable
1147 unreachable vulnerabilities finding reachability=Unreachable
1148 conditional reachability finding reachability=Conditional
1149 unknown reachability status finding reachability=Unknown
1150 static path analysis finding pathEvidence=StaticPath
1151 runtime hit confirmed finding pathEvidence=RuntimeHit
1152 runtime sink hit finding pathEvidence=RuntimeSinkHit
1153 guard condition reduces reachability finding pathEvidence=Guard
1154 mitigation blocks execution finding pathEvidence=Mitigation
1155 static analysis confirmed by runtime finding observationType=Confirmed
1156 runtime only path witness finding observationType=Runtime
1157 static only path no runtime finding observationType=Static
1158 call graph shows reachable function finding reachability evidence
1159 OTel trace confirms vulnerable path finding runtime observation
1160 Tetragon runtime observation finding runtime observation
1161 profiler confirms code execution finding runtime observation
1162 hot symbol detected at runtime finding runtime signal
1163 vulnerable function in execute path finding path analysis
1164 no callstack to vulnerable code finding unreachable path
1165 indirect call graph reachability finding call graph analysis
1166 entry point to sink path finding path analysis
1167 transitive call chain reachable finding transitive analysis
1168 reachability proof document finding evidence type
1169 callstack slice for vulnerability finding evidence type
1170 reachability confidence score finding confidence metric

4.6 Binary & Crypto Analysis Findings (25 cases)

# Query Expected Entity Type Expected Match Source
1171 stripped Go binary vulnerability finding binary analysis - Go
1172 Mach-O binary CVE finding binary analysis - macOS
1173 Windows PE vulnerability finding binary analysis - Windows
1174 Authenticode signature invalid finding binary analysis - PE
1175 native library vulnerability finding binary analysis - native
1176 embedded dependency in binary finding binary analysis
1177 statically linked vulnerable code finding binary analysis
1178 shared library CVE finding binary analysis - .so/.dll
1179 musl libc vulnerability finding binary analysis - Alpine
1180 glibc vulnerability finding binary analysis - glibc
1181 crypto material expired finding crypto analysis - expired
1182 weak cipher algorithm detected finding crypto analysis
1183 deprecated TLS version finding crypto analysis
1184 insecure hash function MD5 finding crypto analysis
1185 SHA1 deprecation warning finding crypto analysis
1186 RSA key too short finding crypto analysis
1187 self-signed certificate in production finding crypto analysis
1188 certificate about to expire finding crypto analysis
1189 weak random number generator finding crypto analysis
1190 hardcoded IV initialization vector finding crypto analysis
1191 OS package vulnerability alpine finding apk ecosystem
1192 OS package vulnerability debian finding dpkg ecosystem
1193 OS package vulnerability rpm finding rpm ecosystem
1194 homebrew package CVE finding homebrew ecosystem
1195 chocolatey package vulnerability finding chocolatey ecosystem

4.7 Triage Workflow & Status Searches (25 cases)

# Query Expected Entity Type Expected Match Source
1196 findings in active triage finding triageLane=Active
1197 blocked shipment findings finding triageLane=Blocked
1198 findings needing exception finding triageLane=NeedsException
1199 muted by reachability finding triageLane=MutedReach
1200 muted by VEX status finding triageLane=MutedVex
1201 compensated findings finding triageLane=Compensated
1202 ship verdict findings finding verdict=Ship
1203 block verdict findings finding verdict=Block
1204 exception granted findings finding verdict=Exception
1205 pending scan results finding scanStatus=Pending
1206 running scans finding scanStatus=Running
1207 failed scan results finding scanStatus=Failed
1208 cancelled scan finding scanStatus=Cancelled
1209 SBOM slice evidence for finding finding evidence=SbomSlice
1210 VEX document evidence finding evidence=VexDoc
1211 provenance evidence for finding finding evidence=Provenance
1212 callstack slice evidence finding evidence=CallstackSlice
1213 replay manifest for finding finding evidence=ReplayManifest
1214 policy evidence attached finding evidence=Policy
1215 scan log evidence finding evidence=ScanLog
1216 findings without evidence finding no evidence attached
1217 unresolved findings older than 30 days finding age filter
1218 findings with no assigned owner finding owner filter
1219 findings blocking production release finding release gate filter
1220 findings requiring manual review finding manual review flag

Domain 5 Extended: VEX — Trust, Signatures, Consensus & Conflict

5.3 VEX Trust, Signature & Freshness Verification (25 cases)

# Query Expected Entity Type Expected Match Source
1221 authoritative VEX source vex_statement trustTier=Authoritative
1222 trusted community VEX vex_statement trustTier=Trusted
1223 untrusted VEX statement vex_statement trustTier=Untrusted
1224 unknown trust tier VEX vex_statement trustTier=Unknown
1225 vendor PSIRT VEX vex_statement issuerCategory=Vendor
1226 distributor VEX statement vex_statement issuerCategory=Distributor
1227 community VEX source vex_statement issuerCategory=Community
1228 internal organization VEX vex_statement issuerCategory=Internal
1229 aggregator VEX source vex_statement issuerCategory=Aggregator
1230 DSSE signed VEX document vex_statement signature=dsse
1231 cosign verified VEX vex_statement signature=cosign
1232 PGP signed VEX statement vex_statement signature=pgp
1233 X.509 signed VEX document vex_statement signature=x509
1234 unverified VEX signature vex_statement signatureStatus=unverified
1235 failed VEX signature verification vex_statement signatureStatus=failed
1236 VEX freshness stale vex_statement freshness=stale
1237 VEX freshness expired vex_statement freshness=expired
1238 VEX superseded by newer vex_statement freshness=superseded
1239 fresh VEX statements only vex_statement freshness=fresh
1240 VEX with high trust score vex_statement trustScore > 0.8
1241 VEX from SPDX format vex_statement format=spdx_vex
1242 StellaOps canonical VEX vex_statement format=stellaops
1243 VEX trust vector components vex_statement trust vector detail
1244 VEX issuer reputation vex_statement issuer reputation score
1245 VEX document age over 90 days vex_statement age filter

5.4 VEX Consensus, Conflict & Cross-Domain Resolution (25 cases)

# Query Expected Entity Type Expected Match Source
1246 VEX consensus conflict vex_statement conflict resolution
1247 hard conflict between VEX sources vex_statement conflictSeverity=Hard
1248 soft conflict VEX disagreement vex_statement conflictSeverity=Soft
1249 informational VEX conflict vex_statement conflictSeverity=Info
1250 vendor says not_affected community says affected vex_statement cross-source conflict
1251 VEX consensus engine result vex_statement consensus output
1252 trust-weighted VEX merge vex_statement weighted consensus
1253 VEX confidence score low vex_statement confidence < 0.5
1254 VEX confidence high agreement vex_statement confidence > 0.8
1255 multiple issuers same CVE vex_statement multi-issuer query
1256 VEX status transition history vex_statement status change events
1257 affected changed to not_affected vex_statement status transition
1258 under_investigation resolved to fixed vex_statement status transition
1259 VEX linked to SBOM component vex_statement product/PURL linkage
1260 VEX for CPE product match vex_statement CPE matching
1261 VEX suppressing active finding vex_statement + finding cross-domain suppression
1262 VEX impact on policy gate vex_statement + policy gate evaluation impact
1263 VEX used as evidence in release vex_statement evidence pipeline
1264 VEX predicate in attestation vex_statement attestation predicate
1265 VEX from feed mirror source vex_statement mirror source
1266 VEX subscription notification vex_statement feed subscription
1267 VEX for production environment only vex_statement environment filter
1268 VEX with action statement required vex_statement actionStatement present
1269 VEX with impact statement detail vex_statement impactStatement present
1270 VEX document schema validation failure vex_statement + doctor schema check

Domain 6 Extended: Policy — Gates, Risk Budget, Unknowns & Sealed Mode

6.3 Gate-Level Evaluation & Verdict Searches (25 cases)

# Query Expected Entity Type Expected Match Source
1271 VEX trust gate evaluation policy_rule VexTrustGate
1272 reachable CVE gate blocked policy_rule ReachableCveGate
1273 execution evidence gate result policy_rule ExecutionEvidenceGate
1274 beacon rate gate threshold policy_rule BeaconRateGate
1275 drift gate unreviewed changes policy_rule DriftGate
1276 unknowns gate budget exceeded policy_rule UnknownsGate
1277 policy verdict pass policy_rule verdictStatus=Pass
1278 policy verdict guarded pass policy_rule verdictStatus=GuardedPass
1279 policy verdict blocked policy_rule verdictStatus=Blocked
1280 policy verdict ignored policy_rule verdictStatus=Ignored
1281 policy verdict warned policy_rule verdictStatus=Warned
1282 policy verdict deferred policy_rule verdictStatus=Deferred
1283 policy verdict escalated policy_rule verdictStatus=Escalated
1284 policy verdict requires VEX policy_rule verdictStatus=RequiresVex
1285 gate result pass with note policy_rule gateResult=PassWithNote
1286 gate result warn policy_rule gateResult=Warn
1287 gate result block policy_rule gateResult=Block
1288 gate result skip policy_rule gateResult=Skip
1289 G0 no-risk gate level policy_rule gateLevel=G0
1290 G1 low risk gate level policy_rule gateLevel=G1
1291 G2 moderate risk gate level policy_rule gateLevel=G2
1292 G3 high risk gate level policy_rule gateLevel=G3
1293 G4 safety critical gate level policy_rule gateLevel=G4
1294 policy gate escalation to human review policy_rule escalation
1295 multi-rule conflict resolution policy_rule conflict resolution

6.4 Risk Budget, Unknowns, Observation State & Sealed Mode (25 cases)

# Query Expected Entity Type Expected Match Source
1296 risk budget remaining for project policy_rule budget tracking
1297 risk budget burn rate policy_rule budget consumption
1298 unknowns budget exceeded policy_rule unknowns tracking
1299 unknown reachability reason policy_rule U-RCH unknown code
1300 unknown identity ambiguous package policy_rule U-ID unknown code
1301 unknown provenance cannot map binary policy_rule U-PROV unknown code
1302 VEX conflict unknown policy_rule U-VEX unknown code
1303 feed gap unknown source missing policy_rule U-FEED unknown code
1304 config unknown feature not observable policy_rule U-CONFIG unknown code
1305 analyzer limit language not supported policy_rule U-ANALYZER unknown code
1306 observation pending determinization policy_rule state=PendingDeterminization
1307 observation determined policy_rule state=Determined
1308 observation disputed policy_rule state=Disputed
1309 observation stale requires refresh policy_rule state=StaleRequiresRefresh
1310 observation manual review required policy_rule state=ManualReviewRequired
1311 observation suppressed policy_rule state=Suppressed
1312 sealed mode locked dependencies policy_rule sealed mode
1313 sealed mode frozen evidence policy_rule sealed mode
1314 deterministic replay manifest policy_rule replay manifest
1315 no external network during evaluation policy_rule sealed mode constraint
1316 uncertainty tier T1 policy_rule uncertaintyTier=T1
1317 uncertainty tier T2 policy_rule uncertaintyTier=T2
1318 uncertainty tier T3 policy_rule uncertaintyTier=T3
1319 uncertainty tier T4 policy_rule uncertaintyTier=T4
1320 risk verdict attestation DSSE policy_rule attestation evidence

Domain 7 Extended: Cross-Domain — Doctor Troubleshooting Deep Dives & Operations

7.7 Doctor Troubleshooting Deep Dive Queries (50 cases)

# Query Expected Entity Type Expected Match Source
1321 TSA endpoint not responding doctor check.timestamp.tsa.availability
1322 TSA response time degraded doctor check.timestamp.tsa.response-time
1323 TSA certificate about to expire doctor check.timestamp.tsa.certificate-expiry
1324 TSA root CA expiring doctor check.timestamp.tsa.root-expiry
1325 TSA chain validation broken doctor check.timestamp.tsa.chain-valid
1326 OCSP responder unreachable doctor check.timestamp.ocsp.responder
1327 CRL distribution endpoint down doctor check.timestamp.crl.distribution
1328 revocation cache outdated doctor check.timestamp.revocation.cache-fresh
1329 OCSP stapling not configured doctor check.timestamp.ocsp.stapling-enabled
1330 timestamp token approaching expiry doctor check.timestamp.tst.approaching-expiry
1331 deprecated hash algorithm in timestamp doctor check.timestamp.tst.algorithm-deprecated
1332 timestamp missing OCSP stapling doctor check.timestamp.tst.missing-stapling
1333 re-timestamping overdue doctor check.timestamp.restamp.pending
1334 EU trust list not updated doctor check.timestamp.eu-trust-list-fresh
1335 qualified timestamp provider status change doctor check.timestamp.qts.status-change
1336 system clock not synced NTP doctor check.timestamp.system-time-synced
1337 TSA time skew detected doctor check.timestamp.tsa.time-skew
1338 Rekor time correlation drift doctor check.timestamp.rekor.time-correlation
1339 OCI registry health check failing doctor check.integration.oci.registry
1340 OCI referrers API not available doctor check.integration.oci.referrers
1341 registry push denied insufficient permissions doctor check.integration.oci.push
1342 registry credentials expired doctor check.integration.oci.credentials
1343 S3 bucket access denied doctor check.integration.s3.storage
1344 SMTP relay rejected connection doctor check.integration.smtp
1345 Slack API rate limited doctor check.integration.slack
1346 Teams webhook returns 403 doctor check.integration.teams
1347 Git provider SSH key rejected doctor check.integration.git
1348 LDAP bind failed wrong credentials doctor check.integration.ldap
1349 CI system Jenkins unreachable doctor check.integration.ci.system
1350 secrets manager Vault sealed doctor check.integration.secrets.manager
1351 agent version mismatch in cluster doctor check.agent.version.consistency
1352 agent certificate expired doctor check.agent.certificate.expiry
1353 agent resource utilization critical doctor check.agent.resource.utilization
1354 agent task failure rate above threshold doctor check.agent.task.failure.rate
1355 stale agent not reporting doctor check.agent.stale
1356 agent capacity exceeded doctor check.agent.capacity
1357 agent task backlog growing doctor check.agent.task.backlog
1358 cluster health degraded doctor check.agent.cluster.health
1359 compliance evidence integrity violation doctor check.compliance.evidence-integrity
1360 provenance chain validation failed doctor check.compliance.provenance-completeness
1361 attestation signing unhealthy doctor check.compliance.attestation-signing
1362 audit readiness check failed doctor check.compliance.audit-readiness
1363 evidence generation rate dropped doctor check.compliance.evidence-rate
1364 export readiness not met doctor check.compliance.export-readiness
1365 compliance framework check warning doctor check.compliance.framework
1366 eIDAS compliance check failing doctor check.crypto.eidas
1367 FIPS module not loaded doctor check.crypto.fips
1368 HSM PKCS#11 module unavailable doctor check.crypto.hsm
1369 GOST crypto provider not found doctor check.crypto.gost
1370 SM2/SM3/SM4 provider missing doctor check.crypto.sm

7.8 Operational Workflow & Multi-Domain Queries (50 cases)

# Query Expected Entity Type Expected Match Source
1371 release blocked by reachable CVE and no VEX finding + vex + policy multi-domain gate
1372 how to fix agent certificate expiry doctor + docs agent cert troubleshoot
1373 timestamp infrastructure not ready for eIDAS doctor + docs eIDAS + TSA checks
1374 OCI registry credentials need rotation doctor + docs registry + key management
1375 SBOM incomplete missing Go dependencies finding + doctor SBOM generation + analysis
1376 attestation signing failed HSM timeout doctor + docs HSM + attestation
1377 VEX consensus disagreement blocking release vex + policy consensus + gate
1378 binary analysis found crypto weakness finding + doctor binary + crypto analysis
1379 reachability proves vulnerability not exploitable finding + vex reachability + VEX
1380 environment drift detected after deployment doctor + docs drift + deploy
1381 policy determinism check failed in sealed mode policy + doctor determinism + sealed
1382 evidence locker merkle anchor out of sync doctor merkle + evidence locker
1383 feed mirror stale advisory data 7 days old doctor + vex feed freshness
1384 CI integration broken OIDC token expired doctor + docs CI + auth
1385 dead letter queue messages from scanner doctor DLQ + scanner
1386 scheduler missed nightly scan job doctor scheduler + scanner
1387 agent fleet partial quorum during upgrade doctor agent cluster + version
1388 secrets manager down affecting key rotation doctor secrets + key mgmt
1389 Prometheus not collecting scanner metrics doctor observability + scanner
1390 log rotation full disk scan failures doctor logs + storage + scanner
1391 trust anchor expired blocking attestation doctor + docs trust + attestation
1392 VEX issuer not in directory vex + doctor issuer + trust
1393 policy pack push failed OCI auth policy + doctor policy + OCI
1394 evidence export compliance deadline docs + policy export + compliance
1395 binary vulnerability in base image layer finding + docs binary + container
1396 Go module replace directive hides vulnerability finding + docs Go analysis
1397 transitive dependency critical CVE finding transitive deps
1398 EPSS score suddenly increased finding EPSS score change
1399 runtime signal confirms reachable path finding + docs runtime + reachability
1400 how to write custom doctor check plugin docs doctor plugin SDK
1401 debuginfod not resolving symbols for alpine doctor + docs binary analysis
1402 corpus KPI below baseline threshold doctor KPI baseline
1403 VEX from multiple formats disagree on status vex format conflict
1404 policy override audit trail policy override + audit
1405 risk profile change impacted 100 findings policy + finding profile impact
1406 GuardedPass finding needs beacon verification policy + finding beacon gate
1407 execution evidence not signed policy + finding execution evidence
1408 how to configure TSA failover docs + doctor TSA failover
1409 EU qualified trust service list update docs + doctor eIDAS + QTS
1410 CRL expired and OCSP responder down doctor revocation checks
1411 provenance attestation for container image docs + finding provenance
1412 how to investigate unknown reachability docs + finding + policy unknowns
1413 sealed mode evaluation with frozen evidence policy + docs sealed mode
1414 air gap bundle missing advisory feed doctor + docs air gap + feed
1415 agent certificate renewal automation doctor + docs agent + cert
1416 LDAP group sync not updating permissions doctor + docs LDAP + auth
1417 webhook delivery failure notification gap doctor webhook + notify
1418 scanner resource limits causing OOM doctor scanner + resources
1419 evidence staleness exceeding policy TTL doctor + policy staleness + policy
1420 findings backlog prioritization by EPSS finding + docs EPSS + triage

Summary Statistics

Domain Case Count Percentage
Knowledge — Docs 230 16.2%
Knowledge — API Operations 200 14.1%
Knowledge — Doctor Checks 180 12.7%
Findings (Vulnerabilities) 200 14.1%
VEX Statements 100 7.0%
Policy Rules 100 7.0%
Cross-Domain / Natural Language 410 28.9%
Total 1420 100%

Query Intent Distribution

Intent Count Examples
Navigate ~110 "open settings", "go to findings"
Troubleshoot ~200 "why is build failing", "TSA not responding", "agent expired"
Explore ~350 "what is VEX", "explain SBOM", concept lookups
Compare ~60 "compare scans", "difference between", "consensus conflict"
How-To ~120 "how to create release", "how to triage", "how to configure TSA"
Entity Lookup ~360 CVE, PURL, GHSA, check codes, doctor checks, triage status
Multi-Domain ~220 Combined queries hitting 2+ domains

Domain Growth Summary

Domain Original Added New Total Growth
Doctor Checks 80 +100 180 +125%
Findings 100 +100 200 +100%
VEX Statements 50 +50 100 +100%
Policy Rules 50 +50 100 +100%
Cross-Domain 310 +100 410 +32%
Docs 230 +0 230
API Operations 200 +0 200