# Unified Search — 1000+ Test Cases by Ingested Data Domain This document enumerates realistic search queries that users would issue against the Stella Ops unified search index, organized by the data domain that would catch/serve them. Each case shows the query, the expected matching domain(s), and what entity types should surface. --- ## Domain 1: Knowledge — Documentation (docs/*.md) ### 1.1 Getting Started & Onboarding (30 cases) | # | Query | Expected Entity Type | Expected Match Source | |---|-------|---------------------|----------------------| | 1 | `how to get started` | docs | docs/quickstart.md | | 2 | `first scan walkthrough` | docs | docs/quickstart.md | | 3 | `developer onboarding` | docs | docs/DEVELOPER_ONBOARDING.md | | 4 | `contribution checklist` | docs | docs/dev/onboarding/contribution-checklist.md | | 5 | `setup development environment` | docs | docs/dev/DEV_ENVIRONMENT_SETUP.md | | 6 | `install stella ops` | docs | docs/INSTALL_GUIDE.md | | 7 | `docker compose setup` | docs | docs/setup/ | | 8 | `local postgres setup` | docs | docs/db/local-postgres.md | | 9 | `quick start guide` | docs | docs/quickstart.md | | 10 | `what is stella ops` | docs | docs/overview.md | | 11 | `product overview` | docs | docs/overview.md | | 12 | `key features` | docs | docs/key-features.md | | 13 | `full features list` | docs | docs/full-features-list.md | | 14 | `feature matrix` | docs | docs/FEATURE_MATRIX.md | | 15 | `system requirements` | docs | docs/INSTALL_GUIDE.md | | 16 | `prerequisites` | docs | docs/INSTALL_GUIDE.md | | 17 | `troubleshooting guide` | docs | docs/dev/onboarding/troubleshooting-guide.md | | 18 | `FAQ` | docs | docs/dev/onboarding/faq/ | | 19 | `video tutorials` | docs | docs/dev/onboarding/video-tutorial-scripts.md | | 20 | `dev quickstart` | docs | docs/dev/onboarding/dev-quickstart.md | | 21 | `coding standards` | docs | docs/CODING_STANDARDS.md | | 22 | `code of conduct` | docs | docs/code-of-conduct/CODE_OF_CONDUCT.md | | 23 | `testing practices` | docs | docs/code-of-conduct/TESTING_PRACTICES.md | | 24 | `community guidelines` | docs | docs/code-of-conduct/COMMUNITY_CONDUCT.md | | 25 | `glossary` | docs | docs/GLOSSARY.md | | 26 | `terminology definitions` | docs | docs/GLOSSARY.md | | 27 | `roadmap` | docs | docs/ROADMAP.md | | 28 | `planned features` | docs | docs/ROADMAP.md | | 29 | `ui guide` | docs | docs/UI_GUIDE.md | | 30 | `console operator walkthrough` | docs | docs/UI_GUIDE.md | ### 1.2 Architecture & Design (40 cases) | # | Query | Expected Entity Type | Expected Match Source | |---|-------|---------------------|----------------------| | 31 | `high level architecture` | docs | docs/07_HIGH_LEVEL_ARCHITECTURE.md | | 32 | `system architecture overview` | docs | docs/ARCHITECTURE_OVERVIEW.md | | 33 | `architecture reference` | docs | docs/ARCHITECTURE_REFERENCE.md | | 34 | `evidence pipeline architecture` | docs | docs/architecture/EVIDENCE_PIPELINE_ARCHITECTURE.md | | 35 | `integration architecture` | docs | docs/architecture/integrations.md | | 36 | `microservice architecture` | docs | docs/ARCHITECTURE_OVERVIEW.md | | 37 | `how does the router work` | docs | docs/modules/router/ | | 38 | `gateway architecture` | docs | docs/modules/gateway/ | | 39 | `message routing` | docs | docs/modules/router/ | | 40 | `event-driven architecture` | docs | docs/ARCHITECTURE_OVERVIEW.md | | 41 | `multi-tenant isolation` | docs | docs/contracts/web-gateway-tenant-rbac.md | | 42 | `tenant RBAC` | docs | docs/contracts/web-gateway-tenant-rbac.md | | 43 | `linkset correlation` | docs | docs/architecture/decisions/ADR-001 | | 44 | `content addressable storage` | docs | docs/contracts/cas-infrastructure.md | | 45 | `deterministic replay` | docs | docs/contracts/, docs/modules/replay/ | | 46 | `sealed mode` | docs | docs/contracts/sealed-mode.md | | 47 | `sealed installation` | docs | docs/contracts/sealed-install-enforcement.md | | 48 | `rate limiting design` | docs | docs/contracts/rate-limit-design.md | | 49 | `ADR architecture decision` | docs | docs/architecture/decisions/ | | 50 | `API versioning` | docs | docs/api/versioning.md | | 51 | `API governance` | docs | docs/contracts/api-governance-baseline.md | | 52 | `openapi discovery` | docs | docs/api/openapi-discovery.md | | 53 | `evidence model schema` | docs | docs-archived/modules/evidence/ | | 54 | `attestation architecture` | docs | docs/modules/attestor/ | | 55 | `provenance tracking` | docs | docs/modules/provenance/ | | 56 | `database specification` | docs | docs/db/SPECIFICATION.md | | 57 | `database migration strategy` | docs | docs/db/MIGRATION_STRATEGY.md | | 58 | `EF Core migration` | docs | docs/db/MIGRATION_STRATEGY.md | | 59 | `migration conventions` | docs | docs/db/MIGRATION_CONVENTIONS.md | | 60 | `migration inventory` | docs | docs/db/MIGRATION_INVENTORY.md | | 61 | `MongoDB to PostgreSQL` | docs | docs/db/CONVERSION_PLAN.md | | 62 | `database rules` | docs | docs/db/RULES.md | | 63 | `cluster provisioning` | docs | docs/db/cluster-provisioning.md | | 64 | `connection pool` | docs | docs/db/ | | 65 | `buildid propagation` | docs | docs/contracts/buildid-propagation.md | | 66 | `canonical sbom id` | docs | docs/contracts/canonical-sbom-id-v1.md | | 67 | `witness format` | docs | docs/contracts/witness-v1.md | | 68 | `execution evidence format` | docs | docs/contracts/execution-evidence-v1.md | | 69 | `export bundle structure` | docs | docs/contracts/export-bundle.md | | 70 | `federated consent model` | docs | docs/contracts/federated-consent-v1.md | ### 1.3 Security & Hardening (30 cases) | # | Query | Expected Entity Type | Expected Match Source | |---|-------|---------------------|----------------------| | 71 | `security hardening guide` | docs | docs/SECURITY_HARDENING_GUIDE.md | | 72 | `security policy` | docs | docs/SECURITY_POLICY.md | | 73 | `vulnerability disclosure` | docs | docs/SECURITY_POLICY.md | | 74 | `VEX consensus guide` | docs | docs/VEX_CONSENSUS_GUIDE.md | | 75 | `VEX trust model` | docs | docs/VEX_CONSENSUS_GUIDE.md | | 76 | `how to harden deployment` | docs | docs/SECURITY_HARDENING_GUIDE.md | | 77 | `TLS configuration` | docs | docs/security/ | | 78 | `certificate management` | docs | docs/security/ | | 79 | `FIPS compliance` | docs | docs/security/, crypto | | 80 | `GOST cryptography` | docs | docs/security/, crypto | | 81 | `eIDAS digital signatures` | docs | docs/security/, crypto | | 82 | `SM crypto support` | docs | docs/security/, crypto | | 83 | `HSM PKCS#11` | docs | docs/security/, crypto | | 84 | `air gap operation` | docs | docs/OFFLINE_KIT.md | | 85 | `offline kit` | docs | docs/OFFLINE_KIT.md | | 86 | `air-gapped deployment` | docs | docs/OFFLINE_KIT.md | | 87 | `supply chain security` | docs | docs/security/ | | 88 | `SBOM security` | docs | docs/modules/sbom-service/ | | 89 | `attestation signing` | docs | docs/modules/signer/ | | 90 | `transparency log` | docs | docs/modules/attestor/ | | 91 | `Rekor integration` | docs | docs/modules/attestor/ | | 92 | `Sigstore` | docs | docs/modules/attestor/ | | 93 | `in-toto attestation` | docs | docs/modules/attestor/ | | 94 | `DSSE envelope` | docs | docs/modules/attestor/ | | 95 | `key rotation` | docs | docs/modules/signer/ | | 96 | `signing ceremony` | docs | docs/modules/signer/ | | 97 | `trust anchor management` | docs | docs/security/ | | 98 | `secret detection` | docs | docs/modules/scanner/ | | 99 | `credential scanning` | docs | docs/modules/scanner/ | | 100 | `compliance readiness tracker` | docs | docs/compliance/ | ### 1.4 Module Architecture Dossiers (50 cases) | # | Query | Expected Entity Type | Expected Match Source | |---|-------|---------------------|----------------------| | 101 | `scanner architecture` | docs | docs/modules/scanner/ | | 102 | `policy engine architecture` | docs | docs/modules/policy/ | | 103 | `concelier architecture` | docs | docs/modules/concelier/ | | 104 | `excititor architecture` | docs | docs/modules/excititor/ | | 105 | `VEX lens architecture` | docs | docs/modules/vex-lens/ | | 106 | `VEX hub architecture` | docs | docs/modules/vex-hub/ | | 107 | `findings ledger architecture` | docs | docs/modules/findings-ledger/ | | 108 | `evidence locker architecture` | docs | docs/modules/evidence-locker/ | | 109 | `attestor architecture` | docs | docs/modules/attestor/ | | 110 | `signer architecture` | docs | docs/modules/signer/ | | 111 | `orchestrator architecture` | docs | docs/modules/orchestrator/ | | 112 | `scheduler architecture` | docs | docs/modules/scheduler/ | | 113 | `taskrunner architecture` | docs | docs/modules/taskrunner/ | | 114 | `authority architecture` | docs | docs/modules/authority/ | | 115 | `notifier architecture` | docs | docs/modules/notifier/ | | 116 | `timeline architecture` | docs | docs/modules/timeline/ | | 117 | `graph architecture` | docs | docs/modules/graph/ | | 118 | `reach graph architecture` | docs | docs/modules/reach-graph/ | | 119 | `reachability architecture` | docs | docs-archived/modules/reachability/ | | 120 | `triage architecture` | docs | docs-archived/modules/triage/ | | 121 | `risk engine architecture` | docs | docs/modules/risk-engine/ | | 122 | `unknowns architecture` | docs | docs/modules/unknowns/ | | 123 | `export center architecture` | docs | docs/modules/export-center/ | | 124 | `remediation architecture` | docs | docs/modules/remediation/ | | 125 | `signals architecture` | docs | docs/modules/signals/ | | 126 | `binary index architecture` | docs | docs/modules/binary-index/ | | 127 | `symbols architecture` | docs | docs/modules/symbols/ | | 128 | `cartographer architecture` | docs | docs/modules/cartographer/ | | 129 | `opsmemory architecture` | docs | docs/modules/opsmemory/ | | 130 | `airgap architecture` | docs | docs/modules/airgap/ | | 131 | `cryptography module` | docs | docs/modules/cryptography/ | | 132 | `plugin system architecture` | docs | docs/modules/plugin/ | | 133 | `CLI architecture` | docs | docs/modules/cli/ | | 134 | `web frontend architecture` | docs | docs/modules/web/ | | 135 | `telemetry architecture` | docs | docs/modules/telemetry/ | | 136 | `analytics architecture` | docs | docs-archived/modules/analytics/ | | 137 | `mirror architecture` | docs | docs/modules/mirror/ | | 138 | `registry architecture` | docs | docs/modules/registry/ | | 139 | `verifier architecture` | docs | docs/modules/verifier/ | | 140 | `replay engine architecture` | docs | docs/modules/replay/ | | 141 | `feedser architecture` | docs | docs/modules/feedser/ | | 142 | `issuer directory architecture` | docs | docs/modules/issuer-directory/ | | 143 | `packs registry architecture` | docs | docs/modules/packs-registry/ | | 144 | `facet architecture` | docs | docs-archived/modules/facet/ | | 145 | `devportal architecture` | docs | docs/modules/devportal/ | | 146 | `doctor architecture` | docs | docs/modules/doctor/ | | 147 | `bench tools architecture` | docs | docs/modules/bench/ | | 148 | `platform module` | docs | docs/modules/platform/ | | 149 | `gateway module` | docs | docs/modules/gateway/ | | 150 | `router module` | docs | docs/modules/router/ | ### 1.5 Operations, Deployment & Runbooks (30 cases) | # | Query | Expected Entity Type | Expected Match Source | |---|-------|---------------------|----------------------| | 151 | `deployment guide` | docs | docs/operations/deployment/ | | 152 | `production deployment` | docs | docs/operations/deployment/ | | 153 | `scaling guide` | docs | docs/operations/ | | 154 | `runbook incident response` | docs | docs/runbooks/ | | 155 | `emergency procedures` | docs | docs/runbooks/ | | 156 | `devops tooling` | docs | docs/operations/devops/ | | 157 | `operational governance` | docs | docs/operations/governance/ | | 158 | `handoff procedures` | docs | docs/operations/handoff/ | | 159 | `monitoring setup` | docs | docs/technical/observability/ | | 160 | `observability configuration` | docs | docs/technical/observability/ | | 161 | `Prometheus setup` | docs | docs/technical/observability/ | | 162 | `OpenTelemetry setup` | docs | docs/technical/observability/ | | 163 | `helm chart deployment` | docs | docs/operations/deployment/ | | 164 | `docker compose` | docs | devops/compose/ | | 165 | `backup procedures` | docs | docs/operations/ | | 166 | `disaster recovery` | docs | docs/runbooks/ | | 167 | `how to rotate keys` | docs | docs/modules/signer/ | | 168 | `certificate renewal` | docs | docs/security/ | | 169 | `log rotation configuration` | docs | docs/operations/ | | 170 | `performance testing playbook` | docs | docs/dev/performance-testing-playbook.md | | 171 | `release notes` | docs | docs/releases/ | | 172 | `version history` | docs | docs/releases/ | | 173 | `upgrade guide` | docs | docs/releases/ | | 174 | `CI/CD pipeline` | docs | docs/technical/cicd/ | | 175 | `GitHub Actions integration` | docs | docs/technical/cicd/ | | 176 | `GitLab CI integration` | docs | docs/technical/cicd/ | | 177 | `Gitea workflow` | docs | .gitea/ | | 178 | `compliance audit` | docs | docs/compliance/ | | 179 | `governance structure` | docs | docs/GOVERNANCE.md | | 180 | `third party dependencies` | docs | docs/legal/THIRD-PARTY-DEPENDENCIES.md | ### 1.6 Developer Guides & Plugin Development (30 cases) | # | Query | Expected Entity Type | Expected Match Source | |---|-------|---------------------|----------------------| | 181 | `plugin development guide` | docs | docs/PLUGIN_SDK_GUIDE.md | | 182 | `how to write a plugin` | docs | docs/PLUGIN_SDK_GUIDE.md | | 183 | `authority plugin developer guide` | docs | docs/dev/31_AUTHORITY_PLUGIN_DEVELOPER_GUIDE.md | | 184 | `excititor connector guide` | docs | docs/dev/30_EXCITITOR_CONNECTOR_GUIDE.md | | 185 | `auth client guide` | docs | docs/dev/32_AUTH_CLIENT_GUIDE.md | | 186 | `buildx plugin quickstart` | docs | docs/dev/BUILDX_PLUGIN_QUICKSTART.md | | 187 | `extending binary analysis` | docs | docs/dev/extending-binary-analysis.md | | 188 | `test fixture design` | docs | docs/dev/fixtures.md | | 189 | `concelier CLI quickstart` | docs | docs/CONCELIER_CLI_QUICKSTART.md | | 190 | `advisory ingestion` | docs | docs/CONCELIER_CLI_QUICKSTART.md | | 191 | `SDK code generation` | docs | docs/api/sdk-openapi-program.md | | 192 | `API CLI reference` | docs | docs/API_CLI_REFERENCE.md | | 193 | `KISA connector` | docs | docs/dev/kisa_connector_notes.md | | 194 | `semantic versioning merge` | docs | docs/dev/merge_semver_playbook.md | | 195 | `normalized rule recipes` | docs | docs/dev/normalized-rule-recipes.md | | 196 | `API contract standards` | docs | docs/dev/contributing/api-contracts.md | | 197 | `canonicalization determinism` | docs | docs/dev/contributing/canonicalization-determinism.md | | 198 | `corpus contribution guide` | docs | docs/dev/contributing/corpus-contribution-guide.md | | 199 | `notification SDK examples` | docs | docs/api/notify-sdk-examples.md | | 200 | `smart diff types` | docs | docs/api/smart-diff-types.md | | 201 | `hybrid diff patching` | docs | docs/hybrid-diff-patching.md | | 202 | `binary diff` | docs | docs/samples/binary-diff/ | | 203 | `binary analysis` | docs | docs/dev/extending-binary-analysis.md | | 204 | `policy DSL` | docs | docs/modules/policy/ | | 205 | `policy studio contract` | docs | docs/contracts/policy-studio.md | | 206 | `risk scoring contract` | docs | docs/contracts/risk-scoring.md | | 207 | `triage suppress contract` | docs | docs/contracts/triage-suppress-v1.md | | 208 | `verification policy` | docs | docs/contracts/verification-policy.md | | 209 | `redaction defaults` | docs | docs/contracts/redaction-defaults-decision.md | | 210 | `mirror bundle format` | docs | docs/contracts/mirror-bundle.md | ### 1.7 Benchmarks & Competitive Analysis (20 cases) | # | Query | Expected Entity Type | Expected Match Source | |---|-------|---------------------|----------------------| | 211 | `benchmark results` | docs | docs/benchmarks/ | | 212 | `performance baselines` | docs | docs/benchmarks/performance-baselines.md | | 213 | `accuracy metrics` | docs | docs/benchmarks/accuracy-metrics-framework.md | | 214 | `golden corpus` | docs | docs/benchmarks/golden-corpus-kpis.md | | 215 | `Trivy comparison` | docs | docs/benchmarks/scanner-feature-comparison-trivy.md | | 216 | `Snyk comparison` | docs | docs/benchmarks/scanner-feature-comparison-snyk.md | | 217 | `Grype comparison` | docs | docs/benchmarks/scanner-feature-comparison-grype.md | | 218 | `competitive landscape` | docs | docs/product/competitive-landscape.md | | 219 | `fidelity metrics` | docs | docs/benchmarks/fidelity-metrics.md | | 220 | `precision recall curves` | docs | docs/benchmarks/tiered-precision-curves.md | | 221 | `Rust analyzer` | docs | docs/benchmarks/scanner-rust-analyzer.md | | 222 | `scanning gaps` | docs | docs/benchmarks/scanner/ | | 223 | `dotnet scanning` | docs | docs/benchmarks/scanner/deep-dives/dotnet.md | | 224 | `Java scanning` | docs | docs/benchmarks/scanner/deep-dives/java.md | | 225 | `Python scanning` | docs | docs/benchmarks/scanner/deep-dives/python.md | | 226 | `Node.js scanning` | docs | docs/benchmarks/scanner/deep-dives/nodejs.md | | 227 | `Golang scanning` | docs | docs/benchmarks/scanner/deep-dives/golang.md | | 228 | `SAST analysis` | docs | docs/benchmarks/scanner/deep-dives/sast.md | | 229 | `secrets scanning benchmark` | docs | docs/benchmarks/scanner/deep-dives/secrets.md | | 230 | `Windows macOS scanning` | docs | docs/benchmarks/scanner/windows-macos-demand.md | --- ## Domain 2: Knowledge — API Operations (OpenAPI specs) ### 2.1 Scanner API (40 cases) | # | Query | Expected Entity Type | Expected Match Source | |---|-------|---------------------|----------------------| | 231 | `create a scan` | api | POST /api/v1/scans | | 232 | `get scan status` | api | GET /api/v1/scans/{scanId} | | 233 | `scan API` | api | scanner/openapi.yaml | | 234 | `submit call graph` | api | POST /api/v1/scans/{scanId}/callgraphs | | 235 | `stream scan events` | api | GET /api/v1/scans/{scanId}/events | | 236 | `reachability API` | api | scanner reachability endpoints | | 237 | `SBOM upload API` | api | POST /api/v1/sboms/upload | | 238 | `layer SBOM` | api | LayerSbomEndpoints | | 239 | `scan entropy` | api | POST /api/v1/scans/{scanId}/entropy | | 240 | `delta compare API` | api | DeltaCompareEndpoints | | 241 | `delta evidence` | api | DeltaEvidenceEndpoints | | 242 | `manifest endpoint` | api | ManifestEndpoints | | 243 | `SBOM hot lookup` | api | SbomHotLookupEndpoints | | 244 | `proof spine API` | api | ProofSpineEndpoints | | 245 | `witness endpoint` | api | WitnessEndpoints | | 246 | `scanner health` | api | HealthEndpoints | | 247 | `call graph endpoint` | api | CallGraphEndpoints | | 248 | `validation endpoint` | api | ValidationEndpoints | | 249 | `offline kit endpoint` | api | OfflineKitEndpoints | | 250 | `fidelity endpoint` | api | FidelityEndpoints | | 251 | `score replay API` | api | ScoreReplayEndpoints | | 252 | `EPSS scores API` | api | EpssEndpoints | | 253 | `approval endpoint` | api | ApprovalEndpoints | | 254 | `baseline endpoint` | api | BaselineEndpoints | | 255 | `counterfactual analysis API` | api | CounterfactualEndpoints | | 256 | `actionables endpoint` | api | ActionablesEndpoints | | 257 | `secret detection settings` | api | SecretDetectionSettingsEndpoints | | 258 | `smart diff endpoint` | api | SmartDiffEndpoints | | 259 | `unknowns endpoint` | api | UnknownsEndpoints | | 260 | `triage API` | api | Triage/*Endpoints | | 261 | `reachability slice` | api | SliceEndpoints | | 262 | `GitHub code scanning` | api | GitHubCodeScanningEndpoints | | 263 | `scanner webhook` | api | WebhookEndpoints | | 264 | `runtime analysis API` | api | RuntimeEndpoints | | 265 | `reachability evidence` | api | ReachabilityEvidenceEndpoints | | 266 | `reachability stack` | api | ReachabilityStackEndpoints | | 267 | `scan report generation` | api | ReportEndpoints | | 268 | `scan evidence query` | api | EvidenceEndpoints | | 269 | `sources tracking API` | api | SourcesEndpoints | | 270 | `scan observability` | api | ObservabilityEndpoints | ### 2.2 Policy Engine API (40 cases) | # | Query | Expected Entity Type | Expected Match Source | |---|-------|---------------------|----------------------| | 271 | `verification policy API` | api | VerificationPolicyEndpoints | | 272 | `policy pack API` | api | PolicyPackEndpoints | | 273 | `policy snapshot` | api | PolicySnapshotEndpoints | | 274 | `violation tracking API` | api | ViolationEndpoints | | 275 | `policy override API` | api | OverrideEndpoints | | 276 | `risk budget API` | api | BudgetEndpoints, RiskBudgetEndpoints | | 277 | `risk profile API` | api | RiskProfileEndpoints | | 278 | `risk simulation API` | api | RiskSimulationEndpoints | | 279 | `effective policy API` | api | EffectivePolicyEndpoints | | 280 | `policy decision endpoint` | api | PolicyDecisionEndpoint | | 281 | `batch evaluation API` | api | BatchEvaluationEndpoint | | 282 | `policy conflict API` | api | ConflictEndpoints | | 283 | `CVSS receipt endpoint` | api | CvssReceiptEndpoints | | 284 | `attestation report API` | api | AttestationReportEndpoints | | 285 | `policy export` | api | ConsoleExportEndpoints | | 286 | `scope attachment API` | api | ScopeAttachmentEndpoints | | 287 | `staleness endpoint` | api | StalenessEndpoints | | 288 | `sealed mode API` | api | SealedModeEndpoints | | 289 | `policy lint API` | api | PolicyLintEndpoints | | 290 | `policy compilation` | api | PolicyCompilationEndpoints | | 291 | `verify determinism API` | api | VerifyDeterminismEndpoints | | 292 | `merge preview API` | api | MergePreviewEndpoints | | 293 | `policy editor API` | api | VerificationPolicyEditorEndpoints | | 294 | `air gap notification API` | api | AirGapNotificationEndpoints | | 295 | `determinization config` | api | DeterminizationConfigEndpoints | | 296 | `delta if present` | api | DeltaIfPresentEndpoints | | 297 | `trust weighting API` | api | TrustWeightingEndpoint | | 298 | `overlay simulation` | api | OverlaySimulationEndpoint | | 299 | `path scope simulation` | api | PathScopeSimulationEndpoint | | 300 | `evidence summary API` | api | EvidenceSummaryEndpoint | | 301 | `policy pack bundle` | api | PolicyPackBundleEndpoints | | 302 | `risk profile air gap` | api | RiskProfileAirGapEndpoints | | 303 | `risk profile schema` | api | RiskProfileSchemaEndpoints | | 304 | `console simulation` | api | ConsoleSimulationEndpoint | | 305 | `policy worker` | api | PolicyWorkerEndpoint | | 306 | `advisory AI knobs` | api | AdvisoryAiKnobsEndpoint | | 307 | `profile event tracking` | api | ProfileEventEndpoints | | 308 | `profile export` | api | ProfileExportEndpoints | | 309 | `batch context API` | api | BatchContextEndpoint | | 310 | `orchestrator job API` | api | OrchestratorJobEndpoint | ### 2.3 Orchestrator, Scheduler & Release API (30 cases) | # | Query | Expected Entity Type | Expected Match Source | |---|-------|---------------------|----------------------| | 311 | `release API` | api | ReleaseEndpoints | | 312 | `approval workflow API` | api | ApprovalEndpoints | | 313 | `DAG query API` | api | DagEndpoints | | 314 | `circuit breaker API` | api | CircuitBreakerEndpoints | | 315 | `quota governance API` | api | QuotaGovernanceEndpoints | | 316 | `audit trail API` | api | AuditEndpoints | | 317 | `release dashboard API` | api | ReleaseDashboardEndpoints | | 318 | `run execution API` | api | RunEndpoints | | 319 | `event stream websocket` | api | StreamEndpoints | | 320 | `KPI endpoint` | api | KpiEndpoints | | 321 | `job management API` | api | JobEndpoints | | 322 | `first signal API` | api | FirstSignalEndpoints | | 323 | `export job API` | api | ExportJobEndpoints | | 324 | `dead letter queue API` | api | DeadLetterEndpoints | | 325 | `SLO management API` | api | SloEndpoints | | 326 | `source tracking API` | api | SourceEndpoints | | 327 | `schedule management API` | api | ScheduleEndpoints | | 328 | `policy simulation API` | api | PolicySimulationEndpointExtensions | | 329 | `graph job API` | api | GraphJobEndpointExtensions | | 330 | `failure signature API` | api | FailureSignatureEndpoints | | 331 | `event webhook API` | api | EventWebhookEndpointExtensions | | 332 | `resolver job API` | api | ResolverJobEndpointExtensions | | 333 | `worker coordination API` | api | WorkerEndpoints | | 334 | `scale auto-scaling API` | api | ScaleEndpoints | | 335 | `pack registry API` | api | PackRegistryEndpoints | | 336 | `pack run API` | api | PackRunEndpoints | | 337 | `ledger query API` | api | LedgerEndpoints | | 338 | `release control v2` | api | ReleaseControlV2Endpoints | | 339 | `openapi discovery endpoint` | api | OpenApiEndpoints | | 340 | `health check API` | api | HealthEndpoints | ### 2.4 Platform, Authority & Notification API (30 cases) | # | Query | Expected Entity Type | Expected Match Source | |---|-------|---------------------|----------------------| | 341 | `platform health API` | api | PlatformEndpoints | | 342 | `quota summary API` | api | PlatformEndpoints | | 343 | `environment settings API` | api | EnvironmentSettingsEndpoints | | 344 | `security read model` | api | SecurityReadModelEndpoints | | 345 | `integration read model` | api | IntegrationReadModelEndpoints | | 346 | `topology query API` | api | TopologyReadModelEndpoints | | 347 | `analytics data API` | api | AnalyticsEndpoints | | 348 | `score calculation API` | api | ScoreEndpoints | | 349 | `function map API` | api | FunctionMapEndpoints | | 350 | `evidence thread API` | api | EvidenceThreadEndpoints | | 351 | `federation telemetry API` | api | FederationTelemetryEndpoints | | 352 | `trust signing admin API` | api | AdministrationTrustSigningMutationEndpoints | | 353 | `OAuth token endpoint` | api | Authority endpoints | | 354 | `OIDC discovery` | api | Authority endpoints | | 355 | `token introspection` | api | Authority endpoints | | 356 | `JWKS endpoint` | api | Authority endpoints | | 357 | `notification rules API` | api | RuleEndpoints | | 358 | `notification template API` | api | TemplateEndpoints | | 359 | `incident tracking API` | api | IncidentEndpoints | | 360 | `storm breaker API` | api | StormBreakerEndpoints | | 361 | `throttle API` | api | ThrottleEndpoints | | 362 | `quiet hours API` | api | QuietHoursEndpoints | | 363 | `escalation rules API` | api | EscalationEndpoints | | 364 | `notification simulation` | api | SimulationEndpoints | | 365 | `operator override API` | api | OperatorOverrideEndpoints | | 366 | `notification localization` | api | LocalizationEndpoints | | 367 | `live incident feed` | api | IncidentLiveFeed | | 368 | `context management API` | api | ContextEndpoints | | 369 | `seed database API` | api | SeedEndpoints | | 370 | `setup wizard API` | api | SetupEndpoints | ### 2.5 Evidence, Attestation, VEX & Export API (30 cases) | # | Query | Expected Entity Type | Expected Match Source | |---|-------|---------------------|----------------------| | 371 | `unified search API` | api | POST /v1/search/query | | 372 | `knowledge search API` | api | POST /v1/advisory-ai/search | | 373 | `advisory AI chat API` | api | ChatEndpoints | | 374 | `LLM adapter API` | api | LlmAdapterEndpoints | | 375 | `evidence pack API` | api | EvidencePackEndpoints | | 376 | `verdict issuance API` | api | VerdictEndpoints | | 377 | `predicate registry API` | api | PredicateRegistryEndpoints | | 378 | `watchlist API` | api | WatchlistEndpoints | | 379 | `export API` | api | ExportApiEndpoints | | 380 | `risk bundle API` | api | RiskBundleEndpoints | | 381 | `audit bundle API` | api | AuditBundleEndpoints | | 382 | `promotion attestation API` | api | PromotionAttestationEndpoints | | 383 | `lineage export API` | api | LineageExportEndpoints | | 384 | `exception report API` | api | ExceptionReportEndpoints | | 385 | `feed mirror API` | api | FeedMirrorManagementEndpoints | | 386 | `SBOM ingestion API` | api | SbomEndpointExtensions | | 387 | `canonical advisory API` | api | CanonicalAdvisoryEndpointExtensions | | 388 | `advisory source API` | api | AdvisorySourceEndpointExtensions | | 389 | `federation API` | api | FederationEndpointExtensions | | 390 | `air gap endpoint` | api | AirGapEndpointExtensions | | 391 | `findings scoring API` | api | ScoringEndpoints | | 392 | `runtime traces API` | api | RuntimeTracesEndpoints | | 393 | `evidence graph API` | api | EvidenceGraphEndpoints | | 394 | `finding summary API` | api | FindingSummaryEndpoints | | 395 | `backport API` | api | BackportEndpoints | | 396 | `reachability map API` | api | ReachabilityMapEndpoints | | 397 | `VEX ingest API` | api | IngestEndpoints | | 398 | `linkset API` | api | LinksetEndpoints | | 399 | `observation API` | api | ObservationEndpoints | | 400 | `Rekor attestation API` | api | RekorAttestationEndpoints | ### 2.6 Gateway, Policy Gateway, Graph & More (30 cases) | # | Query | Expected Entity Type | Expected Match Source | |---|-------|---------------------|----------------------| | 401 | `registry webhook API` | api | RegistryWebhookEndpoints | | 402 | `gate endpoint` | api | GateEndpoints | | 403 | `score gate API` | api | ScoreGateEndpoints | | 404 | `exception management API` | api | ExceptionEndpoints | | 405 | `exception approval API` | api | ExceptionApprovalEndpoints | | 406 | `governance API` | api | GovernanceEndpoints | | 407 | `delta tracking API` | api | DeltasEndpoints | | 408 | `tool lattice API` | api | ToolLatticeEndpoints | | 409 | `signing ceremony API` | api | CeremonyEndpoints | | 410 | `key rotation API` | api | KeyRotationEndpoints | | 411 | `signer endpoint` | api | SignerEndpoints | | 412 | `timeline query API` | api | TimelineEndpoints | | 413 | `timeline replay API` | api | ReplayEndpoints | | 414 | `timeline export API` | api | ExportEndpoints | | 415 | `graph search API` | api | Graph search contracts | | 416 | `reachgraph query` | api | ReachGraph endpoints | | 417 | `binary vulnerability API` | api | BinaryIndex endpoints | | 418 | `remediation registry API` | api | Remediation endpoints | | 419 | `symbol source API` | api | Symbols endpoints | | 420 | `VEX hub export API` | api | VexHub endpoints | | 421 | `issuer management API` | api | IssuerDirectory endpoints | | 422 | `evidence verdict API` | api | EvidenceLocker VerdictEndpoints | | 423 | `evidence thread audit` | api | EvidenceThreadEndpoints | | 424 | `evidence audit trail` | api | EvidenceAuditEndpoints | | 425 | `evidence export API` | api | EvidenceLocker ExportEndpoints | | 426 | `resolve VEX API` | api | ResolveEndpoint | | 427 | `risk feed API` | api | RiskFeedEndpoints | | 428 | `VEX policy API` | api | PolicyEndpoints (Excititor) | | 429 | `mirror registration API` | api | MirrorRegistrationEndpoints | | 430 | `interest score API` | api | InterestScoreEndpointExtensions | --- ## Domain 3: Knowledge — Doctor Checks ### 3.1 Database & Infrastructure Checks (20 cases) | # | Query | Expected Entity Type | Expected Match Source | |---|-------|---------------------|----------------------| | 431 | `check.postgres.connectivity` | doctor | Postgres Connectivity check | | 432 | `database connection failing` | doctor | check.postgres.connectivity | | 433 | `postgres migrations pending` | doctor | check.postgres.migrations | | 434 | `connection pool exhausted` | doctor | check.postgres.pool | | 435 | `disk space running low` | doctor | check.storage.diskspace | | 436 | `evidence locker write check` | doctor | check.storage.evidencelocker | | 437 | `backup directory writable` | doctor | check.storage.backup | | 438 | `log directory check` | doctor | check.logs.directory.writable | | 439 | `log rotation check` | doctor | check.logs.rotation.configured | | 440 | `Prometheus scrape check` | doctor | check.metrics.prometheus.scrape | | 441 | `OTLP endpoint check` | doctor | check.telemetry.otlp.endpoint | | 442 | `dead letter queue check` | doctor | check.operations.dead-letter | | 443 | `job queue health check` | doctor | check.operations.job-queue | | 444 | `scheduler health check` | doctor | check.operations.scheduler | | 445 | `policy engine health` | doctor | check.policy.engine | | 446 | `scanner queue check` | doctor | check.scanner.queue | | 447 | `scanner resource utilization` | doctor | check.scanner.resources | | 448 | `SBOM generation check` | doctor | check.scanner.sbom | | 449 | `vulnerability scan check` | doctor | check.scanner.vuln | | 450 | `witness graph check` | doctor | check.scanner.witness.graph | ### 3.2 Security & Auth Checks (20 cases) | # | Query | Expected Entity Type | Expected Match Source | |---|-------|---------------------|----------------------| | 451 | `authentication config check` | doctor | check.auth.config | | 452 | `OIDC provider connectivity` | doctor | check.auth.oidc | | 453 | `signing key health` | doctor | check.auth.signing-key | | 454 | `token service health` | doctor | check.auth.token-service | | 455 | `certificate chain validation` | doctor | check.crypto.certchain | | 456 | `FIPS compliance check` | doctor | check.crypto.fips | | 457 | `HSM availability check` | doctor | check.crypto.hsm | | 458 | `eIDAS compliance check` | doctor | check.crypto.eidas | | 459 | `GOST availability check` | doctor | check.crypto.gost | | 460 | `SM crypto check` | doctor | check.crypto.sm | | 461 | `Rekor connectivity check` | doctor | check.attestation.rekor.connectivity | | 462 | `clock skew check` | doctor | check.attestation.clock.skew | | 463 | `cosign key material` | doctor | check.attestation.cosign.keymaterial | | 464 | `signing key expiration` | doctor | check.attestation.keymaterial | | 465 | `transparency log consistency` | doctor | check.attestation.transparency.consistency | | 466 | `Rekor verification job` | doctor | check.attestation.rekor.verification.job | | 467 | `VEX issuer trust check` | doctor | check.vex.issuer-trust | | 468 | `VEX schema compliance check` | doctor | check.vex.schema | | 469 | `VEX document validation` | doctor | check.vex.validation | | 470 | `environment secrets check` | doctor | check.environment.secrets | ### 3.3 Compliance, Agent & Notification Checks (25 cases) | # | Query | Expected Entity Type | Expected Match Source | |---|-------|---------------------|----------------------| | 471 | `audit readiness check` | doctor | check.compliance.audit-readiness | | 472 | `evidence integrity check` | doctor | check.compliance.evidence-integrity | | 473 | `provenance completeness` | doctor | check.compliance.provenance-completeness | | 474 | `attestation signing health` | doctor | check.compliance.attestation-signing | | 475 | `evidence generation rate` | doctor | check.compliance.evidence-rate | | 476 | `export readiness check` | doctor | check.compliance.export-readiness | | 477 | `compliance framework check` | doctor | check.compliance.framework | | 478 | `evidence locker index` | doctor | check.evidencelocker.index | | 479 | `merkle tree anchor` | doctor | check.evidencelocker.merkle | | 480 | `provenance chain check` | doctor | check.evidencelocker.provenance | | 481 | `attestation retrieval` | doctor | check.evidencelocker.retrieval | | 482 | `agent heartbeat freshness` | doctor | check.agent.heartbeat.freshness | | 483 | `agent capacity check` | doctor | check.agent.capacity | | 484 | `stale agent detection` | doctor | check.agent.stale | | 485 | `agent cluster health` | doctor | check.agent.cluster.health | | 486 | `agent cluster quorum` | doctor | check.agent.cluster.quorum | | 487 | `agent version consistency` | doctor | check.agent.version.consistency | | 488 | `agent certificate expiry` | doctor | check.agent.certificate.expiry | | 489 | `agent task backlog` | doctor | check.agent.task.backlog | | 490 | `email notification check` | doctor | check.notify.email.configured | | 491 | `Slack connectivity check` | doctor | check.notify.slack.connectivity | | 492 | `Teams notification check` | doctor | check.notify.teams.configured | | 493 | `notification queue health` | doctor | check.notify.queue.health | | 494 | `webhook connectivity` | doctor | check.notify.webhook.connectivity | | 495 | `TSA response time check` | doctor | check.timestamp.tsa.response-time | ### 3.4 Environment & Release Checks (15 cases) | # | Query | Expected Entity Type | Expected Match Source | |---|-------|---------------------|----------------------| | 496 | `environment connectivity` | doctor | check.environment.connectivity | | 497 | `environment drift` | doctor | check.environment.drift | | 498 | `network policy enforcement` | doctor | check.environment.network.policy | | 499 | `environment capacity` | doctor | check.environment.capacity | | 500 | `deployment health check` | doctor | check.environment.deployments | | 501 | `active release health` | doctor | check.release.active | | 502 | `release configuration check` | doctor | check.release.configuration | | 503 | `environment readiness` | doctor | check.release.environment.readiness | | 504 | `promotion gates check` | doctor | check.release.promotion.gates | | 505 | `rollback readiness` | doctor | check.release.rollback.readiness | | 506 | `release schedule check` | doctor | check.release.schedule | | 507 | `reachability computation check` | doctor | check.scanner.reachability | | 508 | `slice cache check` | doctor | check.scanner.slice.cache | | 509 | `buildinfo cache check` | doctor | check.binaryanalysis.buildinfo.cache | | 510 | `debuginfod availability` | doctor | check.binaryanalysis.debuginfod.available | --- ## Domain 4: Findings (Security Findings & Vulnerabilities) ### 4.1 CVE Searches (50 cases) | # | Query | Expected Entity Type | Expected Match Source | |---|-------|---------------------|----------------------| | 511 | `CVE-2024-21626` | finding | Container escape via runc | | 512 | `CVE-2024-3094` | finding | XZ Utils backdoor | | 513 | `CVE-2023-44487` | finding | HTTP/2 Rapid Reset | | 514 | `CVE-2021-44228` | finding | Log4Shell | | 515 | `CVE-2021-45046` | finding | Log4j followup | | 516 | `CVE-2023-4863` | finding | libwebp heap overflow | | 517 | `CVE-2024-0056` | finding | .NET SQL injection | | 518 | `CVE-2023-38545` | finding | curl SOCKS5 overflow | | 519 | `CVE-2023-32233` | finding | Linux kernel nf_tables | | 520 | `CVE-2024-6387` | finding | OpenSSH regreSSHion | | 521 | `Log4Shell` | finding | CVE-2021-44228 | | 522 | `Heartbleed` | finding | CVE-2014-0160 | | 523 | `Spring4Shell` | finding | CVE-2022-22965 | | 524 | `Shellshock` | finding | CVE-2014-6271 | | 525 | `POODLE` | finding | CVE-2014-3566 | | 526 | `critical vulnerabilities` | finding | severity=CRITICAL | | 527 | `high severity findings` | finding | severity=HIGH | | 528 | `remote code execution` | finding | CWE-94 | | 529 | `SQL injection vulnerability` | finding | CWE-89 | | 530 | `buffer overflow` | finding | CWE-120 | | 531 | `cross site scripting` | finding | CWE-79 | | 532 | `privilege escalation` | finding | various CWEs | | 533 | `denial of service` | finding | CWE-400 | | 534 | `path traversal` | finding | CWE-22 | | 535 | `deserialization vulnerability` | finding | CWE-502 | | 536 | `SSRF vulnerability` | finding | CWE-918 | | 537 | `integer overflow` | finding | CWE-190 | | 538 | `use after free` | finding | CWE-416 | | 539 | `null pointer dereference` | finding | CWE-476 | | 540 | `race condition` | finding | CWE-362 | | 541 | `CVSS score 9.8` | finding | CVSS filter | | 542 | `CVSS greater than 7` | finding | CVSS filter | | 543 | `exploit available` | finding | exploitKnown=true | | 544 | `zero day vulnerability` | finding | recent, no patch | | 545 | `EPSS score high` | finding | EPSS > 0.5 | | 546 | `findings for log4j` | finding | package=log4j | | 547 | `openssl vulnerabilities` | finding | package=openssl | | 548 | `npm lodash vulnerability` | finding | pkg:npm/lodash | | 549 | `jackson-databind CVE` | finding | pkg:maven/jackson-databind | | 550 | `spring framework vulnerability` | finding | spring-framework | | 551 | `golang net/http vulnerability` | finding | pkg:golang/net | | 552 | `python requests vulnerability` | finding | pkg:pypi/requests | | 553 | `ruby on rails CVE` | finding | pkg:gem/rails | | 554 | `docker runc vulnerability` | finding | pkg:golang/runc | | 555 | `kubernetes vulnerability` | finding | kubernetes | | 556 | `nginx CVE` | finding | nginx | | 557 | `apache httpd vulnerability` | finding | apache httpd | | 558 | `postgresql vulnerability` | finding | postgresql | | 559 | `redis vulnerability` | finding | redis | | 560 | `alpine linux CVE` | finding | alpine | ### 4.2 PURL & Package Searches (30 cases) | # | Query | Expected Entity Type | Expected Match Source | |---|-------|---------------------|----------------------| | 561 | `pkg:npm/lodash@4.17.21` | finding | npm lodash | | 562 | `pkg:maven/org.apache.logging.log4j/log4j-core@2.17.0` | finding | log4j-core | | 563 | `pkg:pypi/django@4.2` | finding | Django | | 564 | `pkg:cargo/tokio@1.28` | finding | tokio | | 565 | `pkg:golang/github.com/opencontainers/runc@1.1.10` | finding | runc | | 566 | `pkg:nuget/Newtonsoft.Json@13.0.3` | finding | Newtonsoft.Json | | 567 | `pkg:gem/actionpack@7.0` | finding | Rails actionpack | | 568 | `pkg:composer/symfony/http-kernel` | finding | Symfony | | 569 | `pkg:npm/express@4.18` | finding | Express.js | | 570 | `pkg:npm/axios@1.6` | finding | Axios | | 571 | `affected packages npm` | finding | npm ecosystem | | 572 | `affected packages maven` | finding | Maven ecosystem | | 573 | `affected packages pip` | finding | PyPI ecosystem | | 574 | `affected packages cargo` | finding | Cargo/Rust ecosystem | | 575 | `affected packages alpine` | finding | Alpine Linux | | 576 | `affected packages debian` | finding | Debian | | 577 | `affected packages ubuntu` | finding | Ubuntu | | 578 | `affected packages centos` | finding | CentOS | | 579 | `packages with known exploits` | finding | exploitKnown=true | | 580 | `packages with critical severity` | finding | severity=CRITICAL | | 581 | `transitive dependencies vulnerable` | finding | transitive deps | | 582 | `outdated packages security` | finding | version range | | 583 | `library vulnerabilities` | finding | library scan | | 584 | `container base image vulnerabilities` | finding | container scan | | 585 | `OS package vulnerabilities` | finding | OS scan | | 586 | `runtime dependency security` | finding | runtime deps | | 587 | `development dependency vulnerability` | finding | dev deps | | 588 | `binary vulnerability` | finding | binary analysis | | 589 | `Go module vulnerability` | finding | Go modules | | 590 | `.NET NuGet vulnerability` | finding | NuGet packages | ### 4.3 GHSA & Source Searches (20 cases) | # | Query | Expected Entity Type | Expected Match Source | |---|-------|---------------------|----------------------| | 591 | `GHSA-xxxx-yyyy-zzzz` | finding | GitHub Security Advisory | | 592 | `GitHub advisory` | finding | GHSA source | | 593 | `NVD advisory` | finding | NVD source | | 594 | `CISA advisory` | finding | CISA source | | 595 | `Microsoft security advisory` | finding | MSRC source | | 596 | `Ubuntu security notice` | finding | USN source | | 597 | `SUSE security advisory` | finding | SUSE source | | 598 | `Alpine security advisory` | finding | Alpine source | | 599 | `Red Hat security advisory` | finding | RHSA source | | 600 | `Debian security advisory` | finding | DSA source | | 601 | `Cisco advisory` | finding | Cisco source | | 602 | `Oracle security advisory` | finding | Oracle source | | 603 | `ENISA advisory` | finding | ENISA source | | 604 | `JVN advisory` | finding | JVN (Japan) source | | 605 | `BDU advisory` | finding | BDU (Russia) source | | 606 | `CNNVD advisory` | finding | CNNVD (China) source | | 607 | `CNVD advisory` | finding | CNVD (China) source | | 608 | `advisories published today` | finding | date filter | | 609 | `advisories modified this week` | finding | date filter | | 610 | `recently discovered vulnerabilities` | finding | date filter | --- ## Domain 5: VEX (Vulnerability Exploitability Exchange) ### 5.1 VEX Status & Justification Searches (30 cases) | # | Query | Expected Entity Type | Expected Match Source | |---|-------|---------------------|----------------------| | 611 | `VEX not affected` | vex_statement | status=not_affected | | 612 | `VEX affected` | vex_statement | status=affected | | 613 | `VEX fixed` | vex_statement | status=fixed | | 614 | `VEX under investigation` | vex_statement | status=under_investigation | | 615 | `component not present justification` | vex_statement | justification | | 616 | `vulnerable code not present` | vex_statement | justification | | 617 | `code not in execute path` | vex_statement | justification | | 618 | `code not executable` | vex_statement | justification | | 619 | `adversary cannot control code` | vex_statement | justification | | 620 | `inline mitigations exist` | vex_statement | justification | | 621 | `VEX for CVE-2024-21626` | vex_statement | vulnerability match | | 622 | `VEX for log4j` | vex_statement | package match | | 623 | `VEX from vendor` | vex_statement | issuer=VENDOR | | 624 | `VEX from community` | vex_statement | issuer=COMMUNITY | | 625 | `trusted VEX statements` | vex_statement | trust=TRUSTED | | 626 | `authoritative VEX` | vex_statement | trust=AUTHORITATIVE | | 627 | `OpenVEX document` | vex_statement | format=openvex | | 628 | `CSAF VEX document` | vex_statement | format=csaf | | 629 | `CycloneDX VEX` | vex_statement | format=cyclonedx | | 630 | `VEX consensus conflict` | vex_statement | conflict resolution | | 631 | `VEX statement for production` | vex_statement | environment filter | | 632 | `VEX impact statement` | vex_statement | impactStatement field | | 633 | `VEX action required` | vex_statement | actionStatement field | | 634 | `VEX expiring soon` | vex_statement | TTL/freshness | | 635 | `VEX signature verification` | vex_statement | signature check | | 636 | `VEX trust profile` | vex_statement | trust profile config | | 637 | `VEX override` | vex_statement | manual override | | 638 | `how to write VEX` | vex_statement + docs | VEX documentation | | 639 | `VEX schema validation` | vex_statement + doctor | check.vex.schema | | 640 | `VEX issuer directory` | vex_statement | issuer lookup | ### 5.2 VEX Workflow & Integration (20 cases) | # | Query | Expected Entity Type | Expected Match Source | |---|-------|---------------------|----------------------| | 641 | `generate VEX document` | vex_statement | CLI stella vex-gen | | 642 | `ingest VEX statement` | vex_statement | IngestEndpoints | | 643 | `VEX hub search` | vex_statement | VexHub endpoints | | 644 | `VEX studio create` | vex_statement | Web VEX Studio | | 645 | `VEX timeline view` | vex_statement | Web VEX Timeline | | 646 | `VEX gate scan` | vex_statement | VexGateScan feature | | 647 | `export VEX bundle` | vex_statement | VexHub export | | 648 | `VEX evidence proof` | vex_statement | docs/api/vex-proof-schema.md | | 649 | `VEX consensus handling` | vex_statement | docs/VEX_CONSENSUS_GUIDE.md | | 650 | `multiple VEX sources disagree` | vex_statement | conflict resolution | | 651 | `VEX trust weighting` | vex_statement | trust weight config | | 652 | `VEX freshness scoring` | vex_statement | TTL/staleness | | 653 | `VEX linked to finding` | vex_statement + finding | linkset | | 654 | `VEX suppresses finding` | vex_statement | suppression logic | | 655 | `VEX as evidence` | vex_statement | evidence pipeline | | 656 | `VEX attestation` | vex_statement | attestation predicate | | 657 | `VEX policy evaluation` | vex_statement + policy | policy gate | | 658 | `VEX mirror` | vex_statement | mirror endpoints | | 659 | `VEX feed subscription` | vex_statement | feed mirror | | 660 | `VEX document lifecycle` | vex_statement | lifecycle docs | --- ## Domain 6: Policy (Policy Rules, Evaluations, Violations) ### 6.1 Policy Management Searches (30 cases) | # | Query | Expected Entity Type | Expected Match Source | |---|-------|---------------------|----------------------| | 661 | `create policy rule` | policy_rule | Policy Studio | | 662 | `policy pack install` | policy_rule | CLI stella policy install | | 663 | `validate policy YAML` | policy_rule | stella policy validate-yaml | | 664 | `policy simulation` | policy_rule | stella policy simulate | | 665 | `push policy to OCI` | policy_rule | stella policy push | | 666 | `pull policy from registry` | policy_rule | stella policy pull | | 667 | `policy pack bundle` | policy_rule | export/import bundle | | 668 | `block critical vulnerabilities` | policy_rule | severity gate rule | | 669 | `require SBOM attestation` | policy_rule | attestation requirement | | 670 | `require VEX for all CVEs` | policy_rule | VEX requirement | | 671 | `maximum CVSS score allowed` | policy_rule | score threshold | | 672 | `block exploit available` | policy_rule | exploit gate | | 673 | `require reachability proof` | policy_rule | reachability gate | | 674 | `policy for production environment` | policy_rule | scope=production | | 675 | `policy for staging environment` | policy_rule | scope=staging | | 676 | `policy exception request` | policy_rule | exception management | | 677 | `policy waiver` | policy_rule | exception/override | | 678 | `risk budget remaining` | policy_rule | budget tracking | | 679 | `policy violation list` | policy_rule | violation tracking | | 680 | `why was release blocked` | policy_rule | decision audit | | 681 | `policy decision audit trail` | policy_rule | decision log | | 682 | `effective policy for artifact` | policy_rule | computed policy | | 683 | `policy merge preview` | policy_rule | merge simulation | | 684 | `policy conflict detection` | policy_rule | conflict analysis | | 685 | `policy determinism verification` | policy_rule | determinism check | | 686 | `policy lint check` | policy_rule | lint validation | | 687 | `policy compilation` | policy_rule | compile pipeline | | 688 | `sealed mode policy` | policy_rule | air gap mode | | 689 | `staleness rule configuration` | policy_rule | staleness config | | 690 | `risk profile definition` | policy_rule | risk profile | ### 6.2 Policy Evaluation & Decisioning (20 cases) | # | Query | Expected Entity Type | Expected Match Source | |---|-------|---------------------|----------------------| | 691 | `evaluate policy for container` | policy_rule | batch evaluation | | 692 | `policy APPROVE decision` | policy_rule | decision=APPROVE | | 693 | `policy REJECT decision` | policy_rule | decision=REJECT | | 694 | `conditional approval` | policy_rule | decision=CONDITIONAL | | 695 | `blocked by policy` | policy_rule | decision=BLOCKED | | 696 | `awaiting approval` | policy_rule | decision=AWAITING | | 697 | `override policy violation` | policy_rule | override endpoint | | 698 | `severity fusion scoring` | policy_rule | severity fusion | | 699 | `CVSS receipt for finding` | policy_rule | CVSS scoring | | 700 | `attestation report for release` | policy_rule | attestation report | | 701 | `promotion gate evaluation` | policy_rule | gate check | | 702 | `batch policy assessment` | policy_rule | batch evaluation | | 703 | `policy snapshot comparison` | policy_rule | snapshot diff | | 704 | `risk budget consumption` | policy_rule | budget tracking | | 705 | `unknowns budget exceeded` | policy_rule | unknowns tracking | | 706 | `confidence score low` | policy_rule | confidence scoring | | 707 | `evidence freshness expired` | policy_rule | staleness check | | 708 | `trust weight configuration` | policy_rule | trust weighting | | 709 | `overlay simulation results` | policy_rule | overlay sim | | 710 | `path scope simulation` | policy_rule | path scoping | --- ## Domain 7: Cross-Domain Natural Language Queries (290 cases) ### 7.1 Troubleshooting Queries (50 cases) | # | Query | Expected Entity Type | Expected Match Source | |---|-------|---------------------|----------------------| | 711 | `why is the build failing` | mixed | doctor + findings | | 712 | `scan is stuck` | doctor + api | scanner queue check | | 713 | `cannot connect to database` | doctor | check.postgres.connectivity | | 714 | `authentication failed` | doctor | check.auth.config | | 715 | `token expired` | doctor | check.auth.token-service | | 716 | `certificate invalid` | doctor | check.crypto.certchain | | 717 | `signing failed` | doctor | check.attestation.keymaterial | | 718 | `evidence not found` | doctor | check.evidencelocker.retrieval | | 719 | `notification not delivered` | doctor | check.notify.queue.health | | 720 | `release promotion failed` | doctor | check.release.promotion.gates | | 721 | `agent not responding` | doctor | check.agent.heartbeat.freshness | | 722 | `out of disk space` | doctor | check.storage.diskspace | | 723 | `policy evaluation timeout` | doctor | check.policy.engine | | 724 | `reachability analysis slow` | doctor | check.scanner.reachability | | 725 | `VEX validation failed` | doctor | check.vex.validation | | 726 | `email notification not working` | doctor | check.notify.email.connectivity | | 727 | `Slack integration broken` | doctor | check.notify.slack.connectivity | | 728 | `environment drift detected` | doctor | check.environment.drift | | 729 | `clock skew error` | doctor | check.attestation.clock.skew | | 730 | `HSM not available` | doctor | check.crypto.hsm | | 731 | `debug scan failure` | docs + doctor | troubleshooting | | 732 | `fix deployment error` | docs | runbooks | | 733 | `container crash investigation` | docs | troubleshooting | | 734 | `error 403 forbidden` | docs + api | auth scopes | | 735 | `error 404 not found` | docs + api | endpoint reference | | 736 | `error 500 internal server` | docs | troubleshooting | | 737 | `connection refused` | doctor | connectivity checks | | 738 | `timeout error` | docs | timeout configuration | | 739 | `memory leak` | docs | performance troubleshooting | | 740 | `high CPU usage` | doctor | check.agent.resource.utilization | | 741 | `slow query performance` | docs | database tuning | | 742 | `migration failed` | doctor | check.postgres.migrations | | 743 | `index corruption` | doctor | check.evidencelocker.index | | 744 | `merkle tree inconsistency` | doctor | check.evidencelocker.merkle | | 745 | `provenance chain broken` | doctor | check.evidencelocker.provenance | | 746 | `agent task failure rate high` | doctor | check.agent.task.failure.rate | | 747 | `quorum lost` | doctor | check.agent.cluster.quorum | | 748 | `rollback not working` | doctor | check.release.rollback.readiness | | 749 | `export failed` | doctor | check.compliance.export-readiness | | 750 | `compliance audit failure` | doctor | check.compliance.audit-readiness | | 751 | `evidence tampering detected` | doctor | check.compliance.evidence-integrity | | 752 | `no evidence generated` | doctor | check.compliance.evidence-rate | | 753 | `symbol recovery failed` | doctor | check.binaryanalysis.symbol.recovery.fallback | | 754 | `debuginfod unavailable` | doctor | check.binaryanalysis.debuginfod.available | | 755 | `TSA endpoint slow` | doctor | check.timestamp.tsa.response-time | | 756 | `timestamp validation failed` | doctor | check.timestamp.tsa.valid-response | | 757 | `secret detected in code` | finding | secret detection | | 758 | `credentials in repository` | finding | secret detection | | 759 | `API key leaked` | finding | secret detection | | 760 | `hardcoded password` | finding | secret detection | ### 7.2 How-To & Workflow Queries (50 cases) | # | Query | Expected Entity Type | Expected Match Source | |---|-------|---------------------|----------------------| | 761 | `how to scan a container` | docs + api | scanner docs | | 762 | `how to create a release` | docs + api | release docs | | 763 | `how to promote to production` | docs | release orchestration | | 764 | `how to triage a finding` | docs | triage workflow | | 765 | `how to suppress a vulnerability` | docs | triage suppress | | 766 | `how to generate SBOM` | docs + api | scanner SBOM | | 767 | `how to write a VEX statement` | docs | VEX guide | | 768 | `how to configure notifications` | docs | notify setup | | 769 | `how to set up policy gates` | docs | policy gates | | 770 | `how to configure risk budget` | docs | risk budget | | 771 | `how to export evidence` | docs + api | export center | | 772 | `how to verify attestation` | docs + api | attestor | | 773 | `how to configure air gap mode` | docs | offline kit | | 774 | `how to rotate signing keys` | docs | key rotation | | 775 | `how to onboard new environment` | docs | environment setup | | 776 | `how to register agent` | docs | agent onboarding | | 777 | `how to integrate GitHub` | docs | integration guide | | 778 | `how to configure OIDC` | docs | auth setup | | 779 | `how to set up monitoring` | docs | observability | | 780 | `how to run doctor checks` | docs + doctor | stella doctor | | 781 | `how to create policy exception` | docs | exception workflow | | 782 | `how to handle policy violation` | docs | violation handling | | 783 | `how to investigate reachability` | docs | reachability guide | | 784 | `how to generate call graph` | docs + api | call graph | | 785 | `how to compare scans` | docs + api | delta compare | | 786 | `how to export SARIF report` | docs + api | SARIF export | | 787 | `how to configure Prometheus` | docs | observability | | 788 | `how to set up email alerts` | docs | notification config | | 789 | `how to configure escalation` | docs | escalation rules | | 790 | `how to manage trust anchors` | docs | trust management | | 791 | `how to deploy offline` | docs | air gap deployment | | 792 | `how to mirror feeds` | docs + api | feed mirror | | 793 | `how to verify provenance` | docs + api | provenance | | 794 | `how to check compliance` | docs | compliance tracker | | 795 | `how to configure secrets` | docs | secrets management | | 796 | `how to set up federation` | docs | federation | | 797 | `how to use binary diff` | docs | binary diff | | 798 | `how to track changes` | docs | change trace | | 799 | `how to configure quiet hours` | docs | quiet hours | | 800 | `how to set up webhooks` | docs + api | webhook config | | 801 | `how to use policy studio` | docs | policy studio | | 802 | `how to create risk profile` | docs | risk profile | | 803 | `how to run batch evaluation` | docs + api | batch eval | | 804 | `how to configure determinism` | docs | determinism | | 805 | `how to use sealed mode` | docs | sealed mode | | 806 | `how to track unknowns` | docs | unknowns management | | 807 | `how to investigate incidents` | docs | incident management | | 808 | `how to use advisory AI` | docs | advisory AI | | 809 | `how to configure autofix` | docs | remediation | | 810 | `how to use evidence ribbon` | docs | evidence UI | ### 7.3 Navigation & Feature Discovery (50 cases) | # | Query | Expected Entity Type | Expected Match Source | |---|-------|---------------------|----------------------| | 811 | `open settings` | docs | navigation | | 812 | `go to findings` | docs | navigation | | 813 | `show dashboard` | docs | navigation | | 814 | `open security view` | docs | navigation | | 815 | `go to policy gates` | docs | navigation | | 816 | `open VEX hub` | docs | navigation | | 817 | `show release history` | docs | navigation | | 818 | `open agent fleet` | docs | navigation | | 819 | `go to evidence center` | docs | navigation | | 820 | `open export center` | docs | navigation | | 821 | `show topology view` | docs | navigation | | 822 | `open timeline` | docs | navigation | | 823 | `go to triage inbox` | docs | navigation | | 824 | `open approval queue` | docs | navigation | | 825 | `show integrations` | docs | navigation | | 826 | `open policy studio` | docs | navigation | | 827 | `go to scan results` | docs | navigation | | 828 | `open SBOM viewer` | docs | navigation | | 829 | `show notifications` | docs | navigation | | 830 | `open doctor diagnostics` | docs | navigation | | 831 | `where is the audit log` | docs | navigation | | 832 | `find the compliance dashboard` | docs | navigation | | 833 | `where are risk budgets` | docs | navigation | | 834 | `find exception management` | docs | navigation | | 835 | `where is the remediation panel` | docs | navigation | | 836 | `find the binary diff viewer` | docs | navigation | | 837 | `where is the change trace` | docs | navigation | | 838 | `find the scoring page` | docs | navigation | | 839 | `where is the verdict viewer` | docs | navigation | | 840 | `find the proof chain` | docs | navigation | | 841 | `open advisory AI chat` | docs | navigation | | 842 | `where is the setup wizard` | docs | navigation | | 843 | `find the quota dashboard` | docs | navigation | | 844 | `where is SLO monitoring` | docs | navigation | | 845 | `find dead letter queue` | docs | navigation | | 846 | `where is the deploy diff` | docs | navigation | | 847 | `find the lineage view` | docs | navigation | | 848 | `open mission control` | docs | navigation | | 849 | `where is the function map` | docs | navigation | | 850 | `find the vulnerability explorer` | docs | navigation | | 851 | `open control plane` | docs | navigation | | 852 | `show ops memory` | docs | navigation | | 853 | `where is trust admin` | docs | navigation | | 854 | `find the issuer trust page` | docs | navigation | | 855 | `where are workspaces` | docs | navigation | | 856 | `open pack registry` | docs | navigation | | 857 | `find Trivy DB settings` | docs | navigation | | 858 | `where is golden set` | docs | navigation | | 859 | `open observations page` | docs | navigation | | 860 | `find the signals dashboard` | docs | navigation | ### 7.4 CLI Command Searches (50 cases) | # | Query | Expected Entity Type | Expected Match Source | |---|-------|---------------------|----------------------| | 861 | `stella release create` | docs | CLI reference | | 862 | `stella release promote` | docs | CLI reference | | 863 | `stella release rollback` | docs | CLI reference | | 864 | `stella scan graph` | docs | CLI reference | | 865 | `stella policy validate-yaml` | docs | CLI reference | | 866 | `stella policy install` | docs | CLI reference | | 867 | `stella policy simulate` | docs | CLI reference | | 868 | `stella doctor run` | docs + doctor | CLI + checks | | 869 | `stella vex generate` | docs | CLI reference | | 870 | `stella evidence export` | docs | CLI reference | | 871 | `stella attest sign` | docs | CLI reference | | 872 | `stella verify` | docs | CLI reference | | 873 | `stella config set` | docs | CLI reference | | 874 | `stella db migrate` | docs | CLI reference | | 875 | `stella export bundle` | docs | CLI reference | | 876 | `stella import bundle` | docs | CLI reference | | 877 | `stella airgap prepare` | docs | CLI reference | | 878 | `stella scan-graph dotnet` | docs | CLI reference | | 879 | `stella scan-graph java` | docs | CLI reference | | 880 | `stella scan-graph python` | docs | CLI reference | | 881 | `stella agent status` | docs | CLI reference | | 882 | `stella agent list` | docs | CLI reference | | 883 | `stella crypto keygen` | docs | CLI reference | | 884 | `stella keys rotate` | docs | CLI reference | | 885 | `stella trust-anchors add` | docs | CLI reference | | 886 | `stella timestamp verify` | docs | CLI reference | | 887 | `stella score calculate` | docs | CLI reference | | 888 | `stella verdict check` | docs | CLI reference | | 889 | `stella sbom generate` | docs | CLI reference | | 890 | `stella seal create` | docs | CLI reference | | 891 | `stella witness add` | docs | CLI reference | | 892 | `stella proof generate` | docs | CLI reference | | 893 | `stella bundle verify` | docs | CLI reference | | 894 | `stella notify test` | docs | CLI reference | | 895 | `stella feeds sync` | docs | CLI reference | | 896 | `stella registry login` | docs | CLI reference | | 897 | `stella github connect` | docs | CLI reference | | 898 | `stella delta compare` | docs | CLI reference | | 899 | `stella binary diff` | docs | CLI reference | | 900 | `stella change-trace analyze` | docs | CLI reference | | 901 | `stella reachability check` | docs | CLI reference | | 902 | `stella drift detect` | docs | CLI reference | | 903 | `stella timeline query` | docs | CLI reference | | 904 | `stella exception create` | docs | CLI reference | | 905 | `stella incidents list` | docs | CLI reference | | 906 | `stella signals ingest` | docs | CLI reference | | 907 | `stella watchlist add` | docs | CLI reference | | 908 | `stella admin config` | docs | CLI reference | | 909 | `stella analytics report` | docs | CLI reference | | 910 | `stella auth login` | docs | CLI reference | ### 7.5 Concept & Explanation Queries (50 cases) | # | Query | Expected Entity Type | Expected Match Source | |---|-------|---------------------|----------------------| | 911 | `what is a VEX statement` | docs | VEX docs | | 912 | `explain SBOM` | docs | SBOM docs | | 913 | `what is reachability analysis` | docs | reachability concept | | 914 | `explain attestation` | docs | attestation docs | | 915 | `what is DSSE envelope` | docs | attestation docs | | 916 | `explain in-toto format` | docs | attestation docs | | 917 | `what is a policy gate` | docs | policy docs | | 918 | `explain risk budget` | docs | policy docs | | 919 | `what is severity fusion` | docs | scoring docs | | 920 | `explain CVSS v4` | docs + finding | scoring docs | | 921 | `what is EPSS` | docs + finding | scoring docs | | 922 | `explain decision capsule` | docs | product/decision-capsules.md | | 923 | `what is deterministic replay` | docs | replay docs | | 924 | `explain provenance` | docs | provenance docs | | 925 | `what is a Merkle tree` | docs | evidence locker docs | | 926 | `explain evidence chain` | docs | evidence docs | | 927 | `what is sealed mode` | docs | sealed mode docs | | 928 | `explain air gap operation` | docs | offline docs | | 929 | `what is a trust anchor` | docs | security docs | | 930 | `explain multi-tenant isolation` | docs | tenant RBAC docs | | 931 | `what is content addressable storage` | docs | CAS docs | | 932 | `explain smart diff` | docs | smart diff docs | | 933 | `what is a linkset` | docs | linkset docs | | 934 | `explain canonical SBOM ID` | docs | canonical ID docs | | 935 | `what is the findings ledger` | docs | findings docs | | 936 | `explain policy determinization` | docs | policy docs | | 937 | `what is unknowns budgeting` | docs | unknowns docs | | 938 | `explain confidence scoring` | docs | scoring docs | | 939 | `what is change trace` | docs | change trace docs | | 940 | `explain binary analysis` | docs | binary docs | | 941 | `what is the evidence pipeline` | docs | architecture docs | | 942 | `explain reciprocal rank fusion` | docs | search docs | | 943 | `what is a policy pack` | docs | policy docs | | 944 | `explain OCI registry for policy` | docs | policy docs | | 945 | `what is a verdict` | docs | verdict docs | | 946 | `explain proof spine` | docs | proof docs | | 947 | `what is the witness format` | docs | witness docs | | 948 | `explain execution evidence` | docs | evidence docs | | 949 | `what is a federated consent` | docs | federation docs | | 950 | `explain storm breaker` | docs | notification docs | | 951 | `what is a dead letter queue` | docs | operations docs | | 952 | `explain circuit breaker pattern` | docs | orchestrator docs | | 953 | `what is DPoP authentication` | docs | authority docs | | 954 | `explain OAuth 2.1` | docs | authority docs | | 955 | `what is PURL format` | docs + finding | glossary | | 956 | `explain CWE weakness` | docs + finding | glossary | | 957 | `what is SAST vs SCA` | docs | scanner docs | | 958 | `explain runtime signals` | docs | signals docs | | 959 | `what is an advisory source` | docs | concelier docs | | 960 | `explain counterfactual analysis` | docs | scanner docs | ### 7.6 Comparison & Analysis Queries (40 cases) | # | Query | Expected Entity Type | Expected Match Source | |---|-------|---------------------|----------------------| | 961 | `compare scan results` | api + docs | DeltaCompareEndpoints | | 962 | `difference between VEX and advisory` | docs | VEX guide | | 963 | `compare CVSS versions` | docs | scoring docs | | 964 | `difference between SBOM and SPDX` | docs | SBOM docs | | 965 | `compare policy packs` | api | snapshot comparison | | 966 | `difference between Trivy and Stella` | docs | benchmarks | | 967 | `compare Snyk scanner features` | docs | benchmarks | | 968 | `SAST vs SCA differences` | docs | scanner docs | | 969 | `compare environments` | api | environment settings | | 970 | `delta between releases` | api | delta compare | | 971 | `binary diff between versions` | api + docs | binary diff | | 972 | `compare agent versions` | doctor | check.agent.version.consistency | | 973 | `compare findings across scans` | api | delta evidence | | 974 | `what changed since last scan` | api | change trace | | 975 | `new vulnerabilities since yesterday` | finding | date filter | | 976 | `resolved vulnerabilities this week` | finding | status filter | | 977 | `score difference between environments` | api | score endpoints | | 978 | `policy violation trends` | api | analytics | | 979 | `risk profile changes` | api | profile events | | 980 | `VEX status changes` | vex_statement | timeline | | 981 | `evidence freshness comparison` | api | staleness | | 982 | `compliance gap analysis` | docs | compliance tracker | | 983 | `scanning coverage gaps` | docs | benchmarks | | 984 | `trust score comparison` | api | trust weighting | | 985 | `notification delivery rate` | api | notification stats | | 986 | `scan duration trend` | api | analytics | | 987 | `finding resolution velocity` | api | analytics | | 988 | `MTTR for vulnerabilities` | api | analytics | | 989 | `approval wait time` | api | KPI endpoints | | 990 | `deployment frequency` | api | analytics | | 991 | `reachability coverage percentage` | api | reachability stats | | 992 | `SBOM completeness` | api | SBOM analytics | | 993 | `attestation signing latency` | api | performance metrics | | 994 | `evidence locker usage` | api | storage stats | | 995 | `quota utilization` | api | quota dashboard | | 996 | `SLO compliance rate` | api | SLO monitoring | | 997 | `agent utilization heatmap` | api | agent analytics | | 998 | `vulnerability backlog trend` | api + finding | analytics | | 999 | `policy compliance over time` | api | analytics | | 1000 | `risk budget burn rate` | api + policy_rule | budget analytics | --- ## Bonus: Edge Case & Multi-Domain Queries (20 cases) | # | Query | Domains Hit | Description | |---|-------|------------|-------------| | 1001 | `CVE-2024-21626 runc escape reachability VEX` | finding + vex + docs | Multi-domain: CVE + VEX + docs | | 1002 | `log4j affected not_affected VEX` | finding + vex | Finding + conflicting VEX | | 1003 | `OPS-001 check failing production` | doctor + docs | Doctor check + environment context | | 1004 | `policy violation critical CVE-2024-3094` | policy_rule + finding | Policy + finding cross-ref | | 1005 | `how to suppress CVE-2023-44487` | docs + finding + vex | How-to with specific CVE | | 1006 | `GHSA-xxxx for pkg:npm/express` | finding | GHSA + PURL combined | | 1007 | `promote release with blocked findings` | docs + policy_rule | Workflow + policy gate | | 1008 | `attestation failed for container scan` | doctor + docs | Troubleshoot attestation | | 1009 | `VEX not_affected but policy still blocks` | vex + policy_rule | Cross-domain conflict | | 1010 | `reachability shows vulnerable code not in execute path` | finding + vex + docs | Reachability + VEX justification | | 1011 | `export SARIF report for compliance audit` | docs + api | Export + compliance | | 1012 | `rotate signing keys in air gap mode` | docs + doctor | Operations + environment | | 1013 | `agent cluster quorum lost during release` | doctor + docs | Troubleshoot + release | | 1014 | `Slack notification for critical CVE findings` | doctor + docs + finding | Multi-layer search | | 1015 | `binary diff shows new dependency vulnerability` | docs + finding | Analysis + finding | | 1016 | `federation telemetry from remote tenant` | docs + api | Multi-tenant ops | | 1017 | `sealed mode policy with HSM signing` | docs + doctor | Air gap + crypto | | 1018 | `CVSS 9.8 EPSS 0.97 exploit known` | finding | Multi-score filter | | 1019 | `unknown component in SBOM without VEX` | finding + vex + policy | Unknowns workflow | | 1020 | `evidence bundle for in-toto SLSA attestation` | docs + api | Evidence + attestation | --- ## Domain 3 Extended: Doctor Checks — Timestamping, Integration, Binary & Deep Checks ### 3.5 Timestamping & Certificate Lifecycle Checks (20 cases) | # | Query | Expected Entity Type | Expected Match Source | |---|-------|---------------------|----------------------| | 1021 | `TSA availability check` | doctor | check.timestamp.tsa.availability | | 1022 | `TSA response time` | doctor | check.timestamp.tsa.response-time | | 1023 | `TSA valid response check` | doctor | check.timestamp.tsa.valid-response | | 1024 | `TSA failover ready` | doctor | check.timestamp.tsa.failover-ready | | 1025 | `TSA certificate expiry` | doctor | check.timestamp.tsa.certificate-expiry | | 1026 | `TSA root expiry check` | doctor | check.timestamp.tsa.root-expiry | | 1027 | `TSA chain validation` | doctor | check.timestamp.tsa.chain-valid | | 1028 | `OCSP responder check` | doctor | check.timestamp.ocsp.responder | | 1029 | `CRL distribution check` | doctor | check.timestamp.crl.distribution | | 1030 | `revocation cache freshness` | doctor | check.timestamp.revocation.cache-fresh | | 1031 | `OCSP stapling enabled` | doctor | check.timestamp.ocsp.stapling-enabled | | 1032 | `evidence staleness check` | doctor | check.timestamp.evidence-staleness | | 1033 | `timestamp approaching expiry` | doctor | check.timestamp.tst.approaching-expiry | | 1034 | `TST algorithm deprecated` | doctor | check.timestamp.tst.algorithm-deprecated | | 1035 | `TST missing stapling` | doctor | check.timestamp.tst.missing-stapling | | 1036 | `retimestamp pending` | doctor | check.timestamp.restamp.pending | | 1037 | `EU trust list freshness` | doctor | check.timestamp.eu-trust-list-fresh | | 1038 | `QTS providers qualified` | doctor | check.timestamp.qts.providers-qualified | | 1039 | `QTS status change` | doctor | check.timestamp.qts.status-change | | 1040 | `system time synced` | doctor | check.timestamp.system-time-synced | ### 3.6 Integration & External Connectivity Checks (25 cases) | # | Query | Expected Entity Type | Expected Match Source | |---|-------|---------------------|----------------------| | 1041 | `OCI registry connectivity` | doctor | check.integration.oci.registry | | 1042 | `OCI referrers API check` | doctor | check.integration.oci.referrers | | 1043 | `OCI capability matrix` | doctor | check.integration.oci.capabilities | | 1044 | `OCI push authorization` | doctor | check.integration.oci.push | | 1045 | `OCI pull authorization` | doctor | check.integration.oci.pull | | 1046 | `OCI registry credentials` | doctor | check.integration.oci.credentials | | 1047 | `S3 object storage check` | doctor | check.integration.s3.storage | | 1048 | `SMTP connectivity check` | doctor | check.integration.smtp | | 1049 | `Slack webhook check` | doctor | check.integration.slack | | 1050 | `Teams webhook check` | doctor | check.integration.teams | | 1051 | `Git provider connectivity` | doctor | check.integration.git | | 1052 | `LDAP connectivity check` | doctor | check.integration.ldap | | 1053 | `OIDC provider integration check` | doctor | check.integration.oidc | | 1054 | `CI system connectivity` | doctor | check.integration.ci.system | | 1055 | `secrets manager connectivity` | doctor | check.integration.secrets.manager | | 1056 | `integration webhook health` | doctor | check.integration.webhooks | | 1057 | `registry push permission denied` | doctor | check.integration.oci.push | | 1058 | `cannot pull from OCI registry` | doctor | check.integration.oci.pull | | 1059 | `LDAP authentication not working` | doctor | check.integration.ldap | | 1060 | `CI pipeline broken connectivity` | doctor | check.integration.ci.system | | 1061 | `cannot push policy to OCI` | doctor | check.integration.oci.push | | 1062 | `Git provider auth failing` | doctor | check.integration.git | | 1063 | `object storage write failing` | doctor | check.integration.s3.storage | | 1064 | `secrets vault unreachable` | doctor | check.integration.secrets.manager | | 1065 | `integration health dashboard` | doctor | integration checks summary | ### 3.7 Binary Analysis & Corpus Health Checks (20 cases) | # | Query | Expected Entity Type | Expected Match Source | |---|-------|---------------------|----------------------| | 1066 | `debuginfod available` | doctor | check.binaryanalysis.debuginfod.available | | 1067 | `ddeb repo enabled` | doctor | check.binaryanalysis.ddeb.enabled | | 1068 | `buildinfo cache health` | doctor | check.binaryanalysis.buildinfo.cache | | 1069 | `symbol recovery fallback` | doctor | check.binaryanalysis.symbol.recovery.fallback | | 1070 | `corpus mirror freshness` | doctor | check.binaryanalysis.corpus.mirror.freshness | | 1071 | `corpus KPI baseline exists` | doctor | check.binaryanalysis.corpus.kpi.baseline | | 1072 | `binary analysis not working` | doctor | check.binaryanalysis.* | | 1073 | `symbol table missing` | doctor | check.binaryanalysis.symbol.recovery.fallback | | 1074 | `debug symbols not found` | doctor | check.binaryanalysis.debuginfod.available | | 1075 | `buildinfo cache expired` | doctor | check.binaryanalysis.buildinfo.cache | | 1076 | `Go binary stripped no debug` | doctor | check.binaryanalysis.* | | 1077 | `PE authenticode verification failed` | doctor | binary analysis checks | | 1078 | `Mach-O binary inspection failing` | doctor | binary analysis checks | | 1079 | `corpus mirror out of date` | doctor | check.binaryanalysis.corpus.mirror.freshness | | 1080 | `KPI baseline not established` | doctor | check.binaryanalysis.corpus.kpi.baseline | | 1081 | `ddeb repository not configured` | doctor | check.binaryanalysis.ddeb.enabled | | 1082 | `native runtime capture failure` | doctor | binary analysis checks | | 1083 | `crypto material state check` | doctor | binary crypto analysis | | 1084 | `binary vulnerability scan health` | doctor | binary analysis checks | | 1085 | `symbol lookup performance degraded` | doctor | check.binaryanalysis.debuginfod.available | ### 3.8 Observability, Logging & Operations Deep Checks (15 cases) | # | Query | Expected Entity Type | Expected Match Source | |---|-------|---------------------|----------------------| | 1086 | `OTLP exporter not sending` | doctor | check.telemetry.otlp.endpoint | | 1087 | `log directory not writable` | doctor | check.logs.directory.writable | | 1088 | `log rotation not configured` | doctor | check.logs.rotation.configured | | 1089 | `Prometheus not scraping metrics` | doctor | check.metrics.prometheus.scrape | | 1090 | `dead letter queue growing` | doctor | check.operations.dead-letter | | 1091 | `job queue backlog increasing` | doctor | check.operations.job-queue | | 1092 | `scheduler not processing` | doctor | check.operations.scheduler | | 1093 | `traces not appearing in Jaeger` | doctor | check.telemetry.otlp.endpoint | | 1094 | `metrics endpoint 404` | doctor | check.metrics.prometheus.scrape | | 1095 | `log files filling disk` | doctor | check.logs.rotation.configured + check.storage.diskspace | | 1096 | `OpenTelemetry collector down` | doctor | check.telemetry.otlp.endpoint | | 1097 | `dead letter messages accumulating` | doctor | check.operations.dead-letter | | 1098 | `cron job scheduler missed run` | doctor | check.operations.scheduler | | 1099 | `job retry limit exceeded` | doctor | check.operations.job-queue | | 1100 | `observability pipeline health` | doctor | observability checks summary | ### 3.9 Scanner, Reachability & Storage Deep Checks (20 cases) | # | Query | Expected Entity Type | Expected Match Source | |---|-------|---------------------|----------------------| | 1101 | `scanner queue backed up` | doctor | check.scanner.queue | | 1102 | `SBOM generation failing` | doctor | check.scanner.sbom | | 1103 | `vulnerability scan timing out` | doctor | check.scanner.vuln | | 1104 | `witness graph corruption` | doctor | check.scanner.witness.graph | | 1105 | `slice cache miss rate high` | doctor | check.scanner.slice.cache | | 1106 | `reachability computation stalled` | doctor | check.scanner.reachability | | 1107 | `scanner resource utilization high` | doctor | check.scanner.resources | | 1108 | `disk space critical on evidence locker` | doctor | check.storage.diskspace | | 1109 | `evidence locker write failure` | doctor | check.storage.evidencelocker | | 1110 | `backup directory not accessible` | doctor | check.storage.backup | | 1111 | `postgres connection pool exhausted` | doctor | check.postgres.pool | | 1112 | `database migrations not applied` | doctor | check.postgres.migrations | | 1113 | `postgres connectivity lost` | doctor | check.postgres.connectivity | | 1114 | `scanner taking too long` | doctor | check.scanner.resources | | 1115 | `reachability analysis incomplete` | doctor | check.scanner.reachability | | 1116 | `call graph generation failed` | doctor | check.scanner.* | | 1117 | `evidence index inconsistent` | doctor | check.evidencelocker.index | | 1118 | `merkle tree anchor verification failed` | doctor | check.evidencelocker.merkle | | 1119 | `provenance chain incomplete` | doctor | check.evidencelocker.provenance | | 1120 | `attestation retrieval timeout` | doctor | check.evidencelocker.retrieval | --- ## Domain 4 Extended: Findings — Secret Detection, Reachability, Binary & Triage ### 4.4 Secret Detection & Credential Findings (25 cases) | # | Query | Expected Entity Type | Expected Match Source | |---|-------|---------------------|----------------------| | 1121 | `AWS access key exposed` | finding | secret detection - critical | | 1122 | `GitHub personal access token` | finding | secret detection - high | | 1123 | `private SSH key in repository` | finding | secret detection - critical | | 1124 | `database password hardcoded` | finding | secret detection - high | | 1125 | `Slack webhook URL leaked` | finding | secret detection - medium | | 1126 | `Azure connection string exposed` | finding | secret detection - high | | 1127 | `Docker registry credentials` | finding | secret detection - high | | 1128 | `JWT secret key in code` | finding | secret detection - critical | | 1129 | `Stripe API key leaked` | finding | secret detection - high | | 1130 | `Google Cloud service account key` | finding | secret detection - critical | | 1131 | `npm auth token` | finding | secret detection - medium | | 1132 | `Twilio account SID exposed` | finding | secret detection - medium | | 1133 | `SendGrid API key` | finding | secret detection - medium | | 1134 | `PKCS#12 certificate with private key` | finding | secret detection - critical | | 1135 | `environment file with secrets` | finding | secret detection - high | | 1136 | `Terraform state with credentials` | finding | secret detection - critical | | 1137 | `Kubernetes secret in YAML` | finding | secret detection - high | | 1138 | `PGP private key committed` | finding | secret detection - critical | | 1139 | `OAuth client secret exposed` | finding | secret detection - high | | 1140 | `Redis AUTH password in config` | finding | secret detection - medium | | 1141 | `SMTP credentials in source` | finding | secret detection - medium | | 1142 | `encryption key in code` | finding | secret detection - high | | 1143 | `API key rotation needed` | finding | secret detection - medium | | 1144 | `credential severity critical` | finding | secret detection filter | | 1145 | `all secret detections this week` | finding | secret detection date filter | ### 4.5 Reachability & Runtime Analysis Findings (25 cases) | # | Query | Expected Entity Type | Expected Match Source | |---|-------|---------------------|----------------------| | 1146 | `reachable CVE findings` | finding | reachability=Reachable | | 1147 | `unreachable vulnerabilities` | finding | reachability=Unreachable | | 1148 | `conditional reachability` | finding | reachability=Conditional | | 1149 | `unknown reachability status` | finding | reachability=Unknown | | 1150 | `static path analysis` | finding | pathEvidence=StaticPath | | 1151 | `runtime hit confirmed` | finding | pathEvidence=RuntimeHit | | 1152 | `runtime sink hit` | finding | pathEvidence=RuntimeSinkHit | | 1153 | `guard condition reduces reachability` | finding | pathEvidence=Guard | | 1154 | `mitigation blocks execution` | finding | pathEvidence=Mitigation | | 1155 | `static analysis confirmed by runtime` | finding | observationType=Confirmed | | 1156 | `runtime only path witness` | finding | observationType=Runtime | | 1157 | `static only path no runtime` | finding | observationType=Static | | 1158 | `call graph shows reachable function` | finding | reachability evidence | | 1159 | `OTel trace confirms vulnerable path` | finding | runtime observation | | 1160 | `Tetragon runtime observation` | finding | runtime observation | | 1161 | `profiler confirms code execution` | finding | runtime observation | | 1162 | `hot symbol detected at runtime` | finding | runtime signal | | 1163 | `vulnerable function in execute path` | finding | path analysis | | 1164 | `no callstack to vulnerable code` | finding | unreachable path | | 1165 | `indirect call graph reachability` | finding | call graph analysis | | 1166 | `entry point to sink path` | finding | path analysis | | 1167 | `transitive call chain reachable` | finding | transitive analysis | | 1168 | `reachability proof document` | finding | evidence type | | 1169 | `callstack slice for vulnerability` | finding | evidence type | | 1170 | `reachability confidence score` | finding | confidence metric | ### 4.6 Binary & Crypto Analysis Findings (25 cases) | # | Query | Expected Entity Type | Expected Match Source | |---|-------|---------------------|----------------------| | 1171 | `stripped Go binary vulnerability` | finding | binary analysis - Go | | 1172 | `Mach-O binary CVE` | finding | binary analysis - macOS | | 1173 | `Windows PE vulnerability` | finding | binary analysis - Windows | | 1174 | `Authenticode signature invalid` | finding | binary analysis - PE | | 1175 | `native library vulnerability` | finding | binary analysis - native | | 1176 | `embedded dependency in binary` | finding | binary analysis | | 1177 | `statically linked vulnerable code` | finding | binary analysis | | 1178 | `shared library CVE` | finding | binary analysis - .so/.dll | | 1179 | `musl libc vulnerability` | finding | binary analysis - Alpine | | 1180 | `glibc vulnerability` | finding | binary analysis - glibc | | 1181 | `crypto material expired` | finding | crypto analysis - expired | | 1182 | `weak cipher algorithm detected` | finding | crypto analysis | | 1183 | `deprecated TLS version` | finding | crypto analysis | | 1184 | `insecure hash function MD5` | finding | crypto analysis | | 1185 | `SHA1 deprecation warning` | finding | crypto analysis | | 1186 | `RSA key too short` | finding | crypto analysis | | 1187 | `self-signed certificate in production` | finding | crypto analysis | | 1188 | `certificate about to expire` | finding | crypto analysis | | 1189 | `weak random number generator` | finding | crypto analysis | | 1190 | `hardcoded IV initialization vector` | finding | crypto analysis | | 1191 | `OS package vulnerability alpine` | finding | apk ecosystem | | 1192 | `OS package vulnerability debian` | finding | dpkg ecosystem | | 1193 | `OS package vulnerability rpm` | finding | rpm ecosystem | | 1194 | `homebrew package CVE` | finding | homebrew ecosystem | | 1195 | `chocolatey package vulnerability` | finding | chocolatey ecosystem | ### 4.7 Triage Workflow & Status Searches (25 cases) | # | Query | Expected Entity Type | Expected Match Source | |---|-------|---------------------|----------------------| | 1196 | `findings in active triage` | finding | triageLane=Active | | 1197 | `blocked shipment findings` | finding | triageLane=Blocked | | 1198 | `findings needing exception` | finding | triageLane=NeedsException | | 1199 | `muted by reachability` | finding | triageLane=MutedReach | | 1200 | `muted by VEX status` | finding | triageLane=MutedVex | | 1201 | `compensated findings` | finding | triageLane=Compensated | | 1202 | `ship verdict findings` | finding | verdict=Ship | | 1203 | `block verdict findings` | finding | verdict=Block | | 1204 | `exception granted findings` | finding | verdict=Exception | | 1205 | `pending scan results` | finding | scanStatus=Pending | | 1206 | `running scans` | finding | scanStatus=Running | | 1207 | `failed scan results` | finding | scanStatus=Failed | | 1208 | `cancelled scan` | finding | scanStatus=Cancelled | | 1209 | `SBOM slice evidence for finding` | finding | evidence=SbomSlice | | 1210 | `VEX document evidence` | finding | evidence=VexDoc | | 1211 | `provenance evidence for finding` | finding | evidence=Provenance | | 1212 | `callstack slice evidence` | finding | evidence=CallstackSlice | | 1213 | `replay manifest for finding` | finding | evidence=ReplayManifest | | 1214 | `policy evidence attached` | finding | evidence=Policy | | 1215 | `scan log evidence` | finding | evidence=ScanLog | | 1216 | `findings without evidence` | finding | no evidence attached | | 1217 | `unresolved findings older than 30 days` | finding | age filter | | 1218 | `findings with no assigned owner` | finding | owner filter | | 1219 | `findings blocking production release` | finding | release gate filter | | 1220 | `findings requiring manual review` | finding | manual review flag | --- ## Domain 5 Extended: VEX — Trust, Signatures, Consensus & Conflict ### 5.3 VEX Trust, Signature & Freshness Verification (25 cases) | # | Query | Expected Entity Type | Expected Match Source | |---|-------|---------------------|----------------------| | 1221 | `authoritative VEX source` | vex_statement | trustTier=Authoritative | | 1222 | `trusted community VEX` | vex_statement | trustTier=Trusted | | 1223 | `untrusted VEX statement` | vex_statement | trustTier=Untrusted | | 1224 | `unknown trust tier VEX` | vex_statement | trustTier=Unknown | | 1225 | `vendor PSIRT VEX` | vex_statement | issuerCategory=Vendor | | 1226 | `distributor VEX statement` | vex_statement | issuerCategory=Distributor | | 1227 | `community VEX source` | vex_statement | issuerCategory=Community | | 1228 | `internal organization VEX` | vex_statement | issuerCategory=Internal | | 1229 | `aggregator VEX source` | vex_statement | issuerCategory=Aggregator | | 1230 | `DSSE signed VEX document` | vex_statement | signature=dsse | | 1231 | `cosign verified VEX` | vex_statement | signature=cosign | | 1232 | `PGP signed VEX statement` | vex_statement | signature=pgp | | 1233 | `X.509 signed VEX document` | vex_statement | signature=x509 | | 1234 | `unverified VEX signature` | vex_statement | signatureStatus=unverified | | 1235 | `failed VEX signature verification` | vex_statement | signatureStatus=failed | | 1236 | `VEX freshness stale` | vex_statement | freshness=stale | | 1237 | `VEX freshness expired` | vex_statement | freshness=expired | | 1238 | `VEX superseded by newer` | vex_statement | freshness=superseded | | 1239 | `fresh VEX statements only` | vex_statement | freshness=fresh | | 1240 | `VEX with high trust score` | vex_statement | trustScore > 0.8 | | 1241 | `VEX from SPDX format` | vex_statement | format=spdx_vex | | 1242 | `StellaOps canonical VEX` | vex_statement | format=stellaops | | 1243 | `VEX trust vector components` | vex_statement | trust vector detail | | 1244 | `VEX issuer reputation` | vex_statement | issuer reputation score | | 1245 | `VEX document age over 90 days` | vex_statement | age filter | ### 5.4 VEX Consensus, Conflict & Cross-Domain Resolution (25 cases) | # | Query | Expected Entity Type | Expected Match Source | |---|-------|---------------------|----------------------| | 1246 | `VEX consensus conflict` | vex_statement | conflict resolution | | 1247 | `hard conflict between VEX sources` | vex_statement | conflictSeverity=Hard | | 1248 | `soft conflict VEX disagreement` | vex_statement | conflictSeverity=Soft | | 1249 | `informational VEX conflict` | vex_statement | conflictSeverity=Info | | 1250 | `vendor says not_affected community says affected` | vex_statement | cross-source conflict | | 1251 | `VEX consensus engine result` | vex_statement | consensus output | | 1252 | `trust-weighted VEX merge` | vex_statement | weighted consensus | | 1253 | `VEX confidence score low` | vex_statement | confidence < 0.5 | | 1254 | `VEX confidence high agreement` | vex_statement | confidence > 0.8 | | 1255 | `multiple issuers same CVE` | vex_statement | multi-issuer query | | 1256 | `VEX status transition history` | vex_statement | status change events | | 1257 | `affected changed to not_affected` | vex_statement | status transition | | 1258 | `under_investigation resolved to fixed` | vex_statement | status transition | | 1259 | `VEX linked to SBOM component` | vex_statement | product/PURL linkage | | 1260 | `VEX for CPE product match` | vex_statement | CPE matching | | 1261 | `VEX suppressing active finding` | vex_statement + finding | cross-domain suppression | | 1262 | `VEX impact on policy gate` | vex_statement + policy | gate evaluation impact | | 1263 | `VEX used as evidence in release` | vex_statement | evidence pipeline | | 1264 | `VEX predicate in attestation` | vex_statement | attestation predicate | | 1265 | `VEX from feed mirror source` | vex_statement | mirror source | | 1266 | `VEX subscription notification` | vex_statement | feed subscription | | 1267 | `VEX for production environment only` | vex_statement | environment filter | | 1268 | `VEX with action statement required` | vex_statement | actionStatement present | | 1269 | `VEX with impact statement detail` | vex_statement | impactStatement present | | 1270 | `VEX document schema validation failure` | vex_statement + doctor | schema check | --- ## Domain 6 Extended: Policy — Gates, Risk Budget, Unknowns & Sealed Mode ### 6.3 Gate-Level Evaluation & Verdict Searches (25 cases) | # | Query | Expected Entity Type | Expected Match Source | |---|-------|---------------------|----------------------| | 1271 | `VEX trust gate evaluation` | policy_rule | VexTrustGate | | 1272 | `reachable CVE gate blocked` | policy_rule | ReachableCveGate | | 1273 | `execution evidence gate result` | policy_rule | ExecutionEvidenceGate | | 1274 | `beacon rate gate threshold` | policy_rule | BeaconRateGate | | 1275 | `drift gate unreviewed changes` | policy_rule | DriftGate | | 1276 | `unknowns gate budget exceeded` | policy_rule | UnknownsGate | | 1277 | `policy verdict pass` | policy_rule | verdictStatus=Pass | | 1278 | `policy verdict guarded pass` | policy_rule | verdictStatus=GuardedPass | | 1279 | `policy verdict blocked` | policy_rule | verdictStatus=Blocked | | 1280 | `policy verdict ignored` | policy_rule | verdictStatus=Ignored | | 1281 | `policy verdict warned` | policy_rule | verdictStatus=Warned | | 1282 | `policy verdict deferred` | policy_rule | verdictStatus=Deferred | | 1283 | `policy verdict escalated` | policy_rule | verdictStatus=Escalated | | 1284 | `policy verdict requires VEX` | policy_rule | verdictStatus=RequiresVex | | 1285 | `gate result pass with note` | policy_rule | gateResult=PassWithNote | | 1286 | `gate result warn` | policy_rule | gateResult=Warn | | 1287 | `gate result block` | policy_rule | gateResult=Block | | 1288 | `gate result skip` | policy_rule | gateResult=Skip | | 1289 | `G0 no-risk gate level` | policy_rule | gateLevel=G0 | | 1290 | `G1 low risk gate level` | policy_rule | gateLevel=G1 | | 1291 | `G2 moderate risk gate level` | policy_rule | gateLevel=G2 | | 1292 | `G3 high risk gate level` | policy_rule | gateLevel=G3 | | 1293 | `G4 safety critical gate level` | policy_rule | gateLevel=G4 | | 1294 | `policy gate escalation to human review` | policy_rule | escalation | | 1295 | `multi-rule conflict resolution` | policy_rule | conflict resolution | ### 6.4 Risk Budget, Unknowns, Observation State & Sealed Mode (25 cases) | # | Query | Expected Entity Type | Expected Match Source | |---|-------|---------------------|----------------------| | 1296 | `risk budget remaining for project` | policy_rule | budget tracking | | 1297 | `risk budget burn rate` | policy_rule | budget consumption | | 1298 | `unknowns budget exceeded` | policy_rule | unknowns tracking | | 1299 | `unknown reachability reason` | policy_rule | U-RCH unknown code | | 1300 | `unknown identity ambiguous package` | policy_rule | U-ID unknown code | | 1301 | `unknown provenance cannot map binary` | policy_rule | U-PROV unknown code | | 1302 | `VEX conflict unknown` | policy_rule | U-VEX unknown code | | 1303 | `feed gap unknown source missing` | policy_rule | U-FEED unknown code | | 1304 | `config unknown feature not observable` | policy_rule | U-CONFIG unknown code | | 1305 | `analyzer limit language not supported` | policy_rule | U-ANALYZER unknown code | | 1306 | `observation pending determinization` | policy_rule | state=PendingDeterminization | | 1307 | `observation determined` | policy_rule | state=Determined | | 1308 | `observation disputed` | policy_rule | state=Disputed | | 1309 | `observation stale requires refresh` | policy_rule | state=StaleRequiresRefresh | | 1310 | `observation manual review required` | policy_rule | state=ManualReviewRequired | | 1311 | `observation suppressed` | policy_rule | state=Suppressed | | 1312 | `sealed mode locked dependencies` | policy_rule | sealed mode | | 1313 | `sealed mode frozen evidence` | policy_rule | sealed mode | | 1314 | `deterministic replay manifest` | policy_rule | replay manifest | | 1315 | `no external network during evaluation` | policy_rule | sealed mode constraint | | 1316 | `uncertainty tier T1` | policy_rule | uncertaintyTier=T1 | | 1317 | `uncertainty tier T2` | policy_rule | uncertaintyTier=T2 | | 1318 | `uncertainty tier T3` | policy_rule | uncertaintyTier=T3 | | 1319 | `uncertainty tier T4` | policy_rule | uncertaintyTier=T4 | | 1320 | `risk verdict attestation DSSE` | policy_rule | attestation evidence | --- ## Domain 7 Extended: Cross-Domain — Doctor Troubleshooting Deep Dives & Operations ### 7.7 Doctor Troubleshooting Deep Dive Queries (50 cases) | # | Query | Expected Entity Type | Expected Match Source | |---|-------|---------------------|----------------------| | 1321 | `TSA endpoint not responding` | doctor | check.timestamp.tsa.availability | | 1322 | `TSA response time degraded` | doctor | check.timestamp.tsa.response-time | | 1323 | `TSA certificate about to expire` | doctor | check.timestamp.tsa.certificate-expiry | | 1324 | `TSA root CA expiring` | doctor | check.timestamp.tsa.root-expiry | | 1325 | `TSA chain validation broken` | doctor | check.timestamp.tsa.chain-valid | | 1326 | `OCSP responder unreachable` | doctor | check.timestamp.ocsp.responder | | 1327 | `CRL distribution endpoint down` | doctor | check.timestamp.crl.distribution | | 1328 | `revocation cache outdated` | doctor | check.timestamp.revocation.cache-fresh | | 1329 | `OCSP stapling not configured` | doctor | check.timestamp.ocsp.stapling-enabled | | 1330 | `timestamp token approaching expiry` | doctor | check.timestamp.tst.approaching-expiry | | 1331 | `deprecated hash algorithm in timestamp` | doctor | check.timestamp.tst.algorithm-deprecated | | 1332 | `timestamp missing OCSP stapling` | doctor | check.timestamp.tst.missing-stapling | | 1333 | `re-timestamping overdue` | doctor | check.timestamp.restamp.pending | | 1334 | `EU trust list not updated` | doctor | check.timestamp.eu-trust-list-fresh | | 1335 | `qualified timestamp provider status change` | doctor | check.timestamp.qts.status-change | | 1336 | `system clock not synced NTP` | doctor | check.timestamp.system-time-synced | | 1337 | `TSA time skew detected` | doctor | check.timestamp.tsa.time-skew | | 1338 | `Rekor time correlation drift` | doctor | check.timestamp.rekor.time-correlation | | 1339 | `OCI registry health check failing` | doctor | check.integration.oci.registry | | 1340 | `OCI referrers API not available` | doctor | check.integration.oci.referrers | | 1341 | `registry push denied insufficient permissions` | doctor | check.integration.oci.push | | 1342 | `registry credentials expired` | doctor | check.integration.oci.credentials | | 1343 | `S3 bucket access denied` | doctor | check.integration.s3.storage | | 1344 | `SMTP relay rejected connection` | doctor | check.integration.smtp | | 1345 | `Slack API rate limited` | doctor | check.integration.slack | | 1346 | `Teams webhook returns 403` | doctor | check.integration.teams | | 1347 | `Git provider SSH key rejected` | doctor | check.integration.git | | 1348 | `LDAP bind failed wrong credentials` | doctor | check.integration.ldap | | 1349 | `CI system Jenkins unreachable` | doctor | check.integration.ci.system | | 1350 | `secrets manager Vault sealed` | doctor | check.integration.secrets.manager | | 1351 | `agent version mismatch in cluster` | doctor | check.agent.version.consistency | | 1352 | `agent certificate expired` | doctor | check.agent.certificate.expiry | | 1353 | `agent resource utilization critical` | doctor | check.agent.resource.utilization | | 1354 | `agent task failure rate above threshold` | doctor | check.agent.task.failure.rate | | 1355 | `stale agent not reporting` | doctor | check.agent.stale | | 1356 | `agent capacity exceeded` | doctor | check.agent.capacity | | 1357 | `agent task backlog growing` | doctor | check.agent.task.backlog | | 1358 | `cluster health degraded` | doctor | check.agent.cluster.health | | 1359 | `compliance evidence integrity violation` | doctor | check.compliance.evidence-integrity | | 1360 | `provenance chain validation failed` | doctor | check.compliance.provenance-completeness | | 1361 | `attestation signing unhealthy` | doctor | check.compliance.attestation-signing | | 1362 | `audit readiness check failed` | doctor | check.compliance.audit-readiness | | 1363 | `evidence generation rate dropped` | doctor | check.compliance.evidence-rate | | 1364 | `export readiness not met` | doctor | check.compliance.export-readiness | | 1365 | `compliance framework check warning` | doctor | check.compliance.framework | | 1366 | `eIDAS compliance check failing` | doctor | check.crypto.eidas | | 1367 | `FIPS module not loaded` | doctor | check.crypto.fips | | 1368 | `HSM PKCS#11 module unavailable` | doctor | check.crypto.hsm | | 1369 | `GOST crypto provider not found` | doctor | check.crypto.gost | | 1370 | `SM2/SM3/SM4 provider missing` | doctor | check.crypto.sm | ### 7.8 Operational Workflow & Multi-Domain Queries (50 cases) | # | Query | Expected Entity Type | Expected Match Source | |---|-------|---------------------|----------------------| | 1371 | `release blocked by reachable CVE and no VEX` | finding + vex + policy | multi-domain gate | | 1372 | `how to fix agent certificate expiry` | doctor + docs | agent cert troubleshoot | | 1373 | `timestamp infrastructure not ready for eIDAS` | doctor + docs | eIDAS + TSA checks | | 1374 | `OCI registry credentials need rotation` | doctor + docs | registry + key management | | 1375 | `SBOM incomplete missing Go dependencies` | finding + doctor | SBOM generation + analysis | | 1376 | `attestation signing failed HSM timeout` | doctor + docs | HSM + attestation | | 1377 | `VEX consensus disagreement blocking release` | vex + policy | consensus + gate | | 1378 | `binary analysis found crypto weakness` | finding + doctor | binary + crypto analysis | | 1379 | `reachability proves vulnerability not exploitable` | finding + vex | reachability + VEX | | 1380 | `environment drift detected after deployment` | doctor + docs | drift + deploy | | 1381 | `policy determinism check failed in sealed mode` | policy + doctor | determinism + sealed | | 1382 | `evidence locker merkle anchor out of sync` | doctor | merkle + evidence locker | | 1383 | `feed mirror stale advisory data 7 days old` | doctor + vex | feed freshness | | 1384 | `CI integration broken OIDC token expired` | doctor + docs | CI + auth | | 1385 | `dead letter queue messages from scanner` | doctor | DLQ + scanner | | 1386 | `scheduler missed nightly scan job` | doctor | scheduler + scanner | | 1387 | `agent fleet partial quorum during upgrade` | doctor | agent cluster + version | | 1388 | `secrets manager down affecting key rotation` | doctor | secrets + key mgmt | | 1389 | `Prometheus not collecting scanner metrics` | doctor | observability + scanner | | 1390 | `log rotation full disk scan failures` | doctor | logs + storage + scanner | | 1391 | `trust anchor expired blocking attestation` | doctor + docs | trust + attestation | | 1392 | `VEX issuer not in directory` | vex + doctor | issuer + trust | | 1393 | `policy pack push failed OCI auth` | policy + doctor | policy + OCI | | 1394 | `evidence export compliance deadline` | docs + policy | export + compliance | | 1395 | `binary vulnerability in base image layer` | finding + docs | binary + container | | 1396 | `Go module replace directive hides vulnerability` | finding + docs | Go analysis | | 1397 | `transitive dependency critical CVE` | finding | transitive deps | | 1398 | `EPSS score suddenly increased` | finding | EPSS score change | | 1399 | `runtime signal confirms reachable path` | finding + docs | runtime + reachability | | 1400 | `how to write custom doctor check plugin` | docs | doctor plugin SDK | | 1401 | `debuginfod not resolving symbols for alpine` | doctor + docs | binary analysis | | 1402 | `corpus KPI below baseline threshold` | doctor | KPI baseline | | 1403 | `VEX from multiple formats disagree on status` | vex | format conflict | | 1404 | `policy override audit trail` | policy | override + audit | | 1405 | `risk profile change impacted 100 findings` | policy + finding | profile impact | | 1406 | `GuardedPass finding needs beacon verification` | policy + finding | beacon gate | | 1407 | `execution evidence not signed` | policy + finding | execution evidence | | 1408 | `how to configure TSA failover` | docs + doctor | TSA failover | | 1409 | `EU qualified trust service list update` | docs + doctor | eIDAS + QTS | | 1410 | `CRL expired and OCSP responder down` | doctor | revocation checks | | 1411 | `provenance attestation for container image` | docs + finding | provenance | | 1412 | `how to investigate unknown reachability` | docs + finding + policy | unknowns | | 1413 | `sealed mode evaluation with frozen evidence` | policy + docs | sealed mode | | 1414 | `air gap bundle missing advisory feed` | doctor + docs | air gap + feed | | 1415 | `agent certificate renewal automation` | doctor + docs | agent + cert | | 1416 | `LDAP group sync not updating permissions` | doctor + docs | LDAP + auth | | 1417 | `webhook delivery failure notification gap` | doctor | webhook + notify | | 1418 | `scanner resource limits causing OOM` | doctor | scanner + resources | | 1419 | `evidence staleness exceeding policy TTL` | doctor + policy | staleness + policy | | 1420 | `findings backlog prioritization by EPSS` | finding + docs | EPSS + triage | --- ## Summary Statistics | Domain | Case Count | Percentage | |--------|-----------|------------| | Knowledge — Docs | 230 | 16.2% | | Knowledge — API Operations | 200 | 14.1% | | Knowledge — Doctor Checks | 180 | 12.7% | | Findings (Vulnerabilities) | 200 | 14.1% | | VEX Statements | 100 | 7.0% | | Policy Rules | 100 | 7.0% | | Cross-Domain / Natural Language | 410 | 28.9% | | **Total** | **1420** | **100%** | ### Query Intent Distribution | Intent | Count | Examples | |--------|-------|---------| | Navigate | ~110 | "open settings", "go to findings" | | Troubleshoot | ~200 | "why is build failing", "TSA not responding", "agent expired" | | Explore | ~350 | "what is VEX", "explain SBOM", concept lookups | | Compare | ~60 | "compare scans", "difference between", "consensus conflict" | | How-To | ~120 | "how to create release", "how to triage", "how to configure TSA" | | Entity Lookup | ~360 | CVE, PURL, GHSA, check codes, doctor checks, triage status | | Multi-Domain | ~220 | Combined queries hitting 2+ domains | ### Domain Growth Summary | Domain | Original | Added | New Total | Growth | |--------|----------|-------|-----------|--------| | Doctor Checks | 80 | +100 | 180 | +125% | | Findings | 100 | +100 | 200 | +100% | | VEX Statements | 50 | +50 | 100 | +100% | | Policy Rules | 50 | +50 | 100 | +100% | | Cross-Domain | 310 | +100 | 410 | +32% | | Docs | 230 | +0 | 230 | — | | API Operations | 200 | +0 | 200 | — |