1743 lines
106 KiB
Markdown
1743 lines
106 KiB
Markdown
# Unified Search — 1000+ Test Cases by Ingested Data Domain
|
|
|
|
This document enumerates realistic search queries that users would issue against the Stella Ops unified search index, organized by the data domain that would catch/serve them. Each case shows the query, the expected matching domain(s), and what entity types should surface.
|
|
|
|
---
|
|
|
|
## Domain 1: Knowledge — Documentation (docs/*.md)
|
|
|
|
### 1.1 Getting Started & Onboarding (30 cases)
|
|
|
|
| # | Query | Expected Entity Type | Expected Match Source |
|
|
|---|-------|---------------------|----------------------|
|
|
| 1 | `how to get started` | docs | docs/quickstart.md |
|
|
| 2 | `first scan walkthrough` | docs | docs/quickstart.md |
|
|
| 3 | `developer onboarding` | docs | docs/DEVELOPER_ONBOARDING.md |
|
|
| 4 | `contribution checklist` | docs | docs/dev/onboarding/contribution-checklist.md |
|
|
| 5 | `setup development environment` | docs | docs/dev/DEV_ENVIRONMENT_SETUP.md |
|
|
| 6 | `install stella ops` | docs | docs/INSTALL_GUIDE.md |
|
|
| 7 | `docker compose setup` | docs | docs/setup/ |
|
|
| 8 | `local postgres setup` | docs | docs/db/local-postgres.md |
|
|
| 9 | `quick start guide` | docs | docs/quickstart.md |
|
|
| 10 | `what is stella ops` | docs | docs/overview.md |
|
|
| 11 | `product overview` | docs | docs/overview.md |
|
|
| 12 | `key features` | docs | docs/key-features.md |
|
|
| 13 | `full features list` | docs | docs/full-features-list.md |
|
|
| 14 | `feature matrix` | docs | docs/FEATURE_MATRIX.md |
|
|
| 15 | `system requirements` | docs | docs/INSTALL_GUIDE.md |
|
|
| 16 | `prerequisites` | docs | docs/INSTALL_GUIDE.md |
|
|
| 17 | `troubleshooting guide` | docs | docs/dev/onboarding/troubleshooting-guide.md |
|
|
| 18 | `FAQ` | docs | docs/dev/onboarding/faq/ |
|
|
| 19 | `video tutorials` | docs | docs/dev/onboarding/video-tutorial-scripts.md |
|
|
| 20 | `dev quickstart` | docs | docs/dev/onboarding/dev-quickstart.md |
|
|
| 21 | `coding standards` | docs | docs/CODING_STANDARDS.md |
|
|
| 22 | `code of conduct` | docs | docs/code-of-conduct/CODE_OF_CONDUCT.md |
|
|
| 23 | `testing practices` | docs | docs/code-of-conduct/TESTING_PRACTICES.md |
|
|
| 24 | `community guidelines` | docs | docs/code-of-conduct/COMMUNITY_CONDUCT.md |
|
|
| 25 | `glossary` | docs | docs/GLOSSARY.md |
|
|
| 26 | `terminology definitions` | docs | docs/GLOSSARY.md |
|
|
| 27 | `roadmap` | docs | docs/ROADMAP.md |
|
|
| 28 | `planned features` | docs | docs/ROADMAP.md |
|
|
| 29 | `ui guide` | docs | docs/UI_GUIDE.md |
|
|
| 30 | `console operator walkthrough` | docs | docs/UI_GUIDE.md |
|
|
|
|
### 1.2 Architecture & Design (40 cases)
|
|
|
|
| # | Query | Expected Entity Type | Expected Match Source |
|
|
|---|-------|---------------------|----------------------|
|
|
| 31 | `high level architecture` | docs | docs/07_HIGH_LEVEL_ARCHITECTURE.md |
|
|
| 32 | `system architecture overview` | docs | docs/ARCHITECTURE_OVERVIEW.md |
|
|
| 33 | `architecture reference` | docs | docs/ARCHITECTURE_REFERENCE.md |
|
|
| 34 | `evidence pipeline architecture` | docs | docs/architecture/EVIDENCE_PIPELINE_ARCHITECTURE.md |
|
|
| 35 | `integration architecture` | docs | docs/architecture/integrations.md |
|
|
| 36 | `microservice architecture` | docs | docs/ARCHITECTURE_OVERVIEW.md |
|
|
| 37 | `how does the router work` | docs | docs/modules/router/ |
|
|
| 38 | `gateway architecture` | docs | docs/modules/gateway/ |
|
|
| 39 | `message routing` | docs | docs/modules/router/ |
|
|
| 40 | `event-driven architecture` | docs | docs/ARCHITECTURE_OVERVIEW.md |
|
|
| 41 | `multi-tenant isolation` | docs | docs/contracts/web-gateway-tenant-rbac.md |
|
|
| 42 | `tenant RBAC` | docs | docs/contracts/web-gateway-tenant-rbac.md |
|
|
| 43 | `linkset correlation` | docs | docs/architecture/decisions/ADR-001 |
|
|
| 44 | `content addressable storage` | docs | docs/contracts/cas-infrastructure.md |
|
|
| 45 | `deterministic replay` | docs | docs/contracts/, docs/modules/replay/ |
|
|
| 46 | `sealed mode` | docs | docs/contracts/sealed-mode.md |
|
|
| 47 | `sealed installation` | docs | docs/contracts/sealed-install-enforcement.md |
|
|
| 48 | `rate limiting design` | docs | docs/contracts/rate-limit-design.md |
|
|
| 49 | `ADR architecture decision` | docs | docs/architecture/decisions/ |
|
|
| 50 | `API versioning` | docs | docs/api/versioning.md |
|
|
| 51 | `API governance` | docs | docs/contracts/api-governance-baseline.md |
|
|
| 52 | `openapi discovery` | docs | docs/api/openapi-discovery.md |
|
|
| 53 | `evidence model schema` | docs | docs-archived/modules/evidence/ |
|
|
| 54 | `attestation architecture` | docs | docs/modules/attestor/ |
|
|
| 55 | `provenance tracking` | docs | docs/modules/provenance/ |
|
|
| 56 | `database specification` | docs | docs/db/SPECIFICATION.md |
|
|
| 57 | `database migration strategy` | docs | docs/db/MIGRATION_STRATEGY.md |
|
|
| 58 | `EF Core migration` | docs | docs/db/MIGRATION_STRATEGY.md |
|
|
| 59 | `migration conventions` | docs | docs/db/MIGRATION_CONVENTIONS.md |
|
|
| 60 | `migration inventory` | docs | docs/db/MIGRATION_INVENTORY.md |
|
|
| 61 | `MongoDB to PostgreSQL` | docs | docs/db/CONVERSION_PLAN.md |
|
|
| 62 | `database rules` | docs | docs/db/RULES.md |
|
|
| 63 | `cluster provisioning` | docs | docs/db/cluster-provisioning.md |
|
|
| 64 | `connection pool` | docs | docs/db/ |
|
|
| 65 | `buildid propagation` | docs | docs/contracts/buildid-propagation.md |
|
|
| 66 | `canonical sbom id` | docs | docs/contracts/canonical-sbom-id-v1.md |
|
|
| 67 | `witness format` | docs | docs/contracts/witness-v1.md |
|
|
| 68 | `execution evidence format` | docs | docs/contracts/execution-evidence-v1.md |
|
|
| 69 | `export bundle structure` | docs | docs/contracts/export-bundle.md |
|
|
| 70 | `federated consent model` | docs | docs/contracts/federated-consent-v1.md |
|
|
|
|
### 1.3 Security & Hardening (30 cases)
|
|
|
|
| # | Query | Expected Entity Type | Expected Match Source |
|
|
|---|-------|---------------------|----------------------|
|
|
| 71 | `security hardening guide` | docs | docs/SECURITY_HARDENING_GUIDE.md |
|
|
| 72 | `security policy` | docs | docs/SECURITY_POLICY.md |
|
|
| 73 | `vulnerability disclosure` | docs | docs/SECURITY_POLICY.md |
|
|
| 74 | `VEX consensus guide` | docs | docs/VEX_CONSENSUS_GUIDE.md |
|
|
| 75 | `VEX trust model` | docs | docs/VEX_CONSENSUS_GUIDE.md |
|
|
| 76 | `how to harden deployment` | docs | docs/SECURITY_HARDENING_GUIDE.md |
|
|
| 77 | `TLS configuration` | docs | docs/security/ |
|
|
| 78 | `certificate management` | docs | docs/security/ |
|
|
| 79 | `FIPS compliance` | docs | docs/security/, crypto |
|
|
| 80 | `GOST cryptography` | docs | docs/security/, crypto |
|
|
| 81 | `eIDAS digital signatures` | docs | docs/security/, crypto |
|
|
| 82 | `SM crypto support` | docs | docs/security/, crypto |
|
|
| 83 | `HSM PKCS#11` | docs | docs/security/, crypto |
|
|
| 84 | `air gap operation` | docs | docs/OFFLINE_KIT.md |
|
|
| 85 | `offline kit` | docs | docs/OFFLINE_KIT.md |
|
|
| 86 | `air-gapped deployment` | docs | docs/OFFLINE_KIT.md |
|
|
| 87 | `supply chain security` | docs | docs/security/ |
|
|
| 88 | `SBOM security` | docs | docs/modules/sbom-service/ |
|
|
| 89 | `attestation signing` | docs | docs/modules/signer/ |
|
|
| 90 | `transparency log` | docs | docs/modules/attestor/ |
|
|
| 91 | `Rekor integration` | docs | docs/modules/attestor/ |
|
|
| 92 | `Sigstore` | docs | docs/modules/attestor/ |
|
|
| 93 | `in-toto attestation` | docs | docs/modules/attestor/ |
|
|
| 94 | `DSSE envelope` | docs | docs/modules/attestor/ |
|
|
| 95 | `key rotation` | docs | docs/modules/signer/ |
|
|
| 96 | `signing ceremony` | docs | docs/modules/signer/ |
|
|
| 97 | `trust anchor management` | docs | docs/security/ |
|
|
| 98 | `secret detection` | docs | docs/modules/scanner/ |
|
|
| 99 | `credential scanning` | docs | docs/modules/scanner/ |
|
|
| 100 | `compliance readiness tracker` | docs | docs/compliance/ |
|
|
|
|
### 1.4 Module Architecture Dossiers (50 cases)
|
|
|
|
| # | Query | Expected Entity Type | Expected Match Source |
|
|
|---|-------|---------------------|----------------------|
|
|
| 101 | `scanner architecture` | docs | docs/modules/scanner/ |
|
|
| 102 | `policy engine architecture` | docs | docs/modules/policy/ |
|
|
| 103 | `concelier architecture` | docs | docs/modules/concelier/ |
|
|
| 104 | `excititor architecture` | docs | docs/modules/excititor/ |
|
|
| 105 | `VEX lens architecture` | docs | docs/modules/vex-lens/ |
|
|
| 106 | `VEX hub architecture` | docs | docs/modules/vex-hub/ |
|
|
| 107 | `findings ledger architecture` | docs | docs/modules/findings-ledger/ |
|
|
| 108 | `evidence locker architecture` | docs | docs/modules/evidence-locker/ |
|
|
| 109 | `attestor architecture` | docs | docs/modules/attestor/ |
|
|
| 110 | `signer architecture` | docs | docs/modules/signer/ |
|
|
| 111 | `orchestrator architecture` | docs | docs/modules/orchestrator/ |
|
|
| 112 | `scheduler architecture` | docs | docs/modules/scheduler/ |
|
|
| 113 | `taskrunner architecture` | docs | docs/modules/taskrunner/ |
|
|
| 114 | `authority architecture` | docs | docs/modules/authority/ |
|
|
| 115 | `notifier architecture` | docs | docs/modules/notifier/ |
|
|
| 116 | `timeline architecture` | docs | docs/modules/timeline/ |
|
|
| 117 | `graph architecture` | docs | docs/modules/graph/ |
|
|
| 118 | `reach graph architecture` | docs | docs/modules/reach-graph/ |
|
|
| 119 | `reachability architecture` | docs | docs-archived/modules/reachability/ |
|
|
| 120 | `triage architecture` | docs | docs-archived/modules/triage/ |
|
|
| 121 | `risk engine architecture` | docs | docs/modules/risk-engine/ |
|
|
| 122 | `unknowns architecture` | docs | docs/modules/unknowns/ |
|
|
| 123 | `export center architecture` | docs | docs/modules/export-center/ |
|
|
| 124 | `remediation architecture` | docs | docs/modules/remediation/ |
|
|
| 125 | `signals architecture` | docs | docs/modules/signals/ |
|
|
| 126 | `binary index architecture` | docs | docs/modules/binary-index/ |
|
|
| 127 | `symbols architecture` | docs | docs/modules/symbols/ |
|
|
| 128 | `cartographer architecture` | docs | docs/modules/cartographer/ |
|
|
| 129 | `opsmemory architecture` | docs | docs/modules/opsmemory/ |
|
|
| 130 | `airgap architecture` | docs | docs/modules/airgap/ |
|
|
| 131 | `cryptography module` | docs | docs/modules/cryptography/ |
|
|
| 132 | `plugin system architecture` | docs | docs/modules/plugin/ |
|
|
| 133 | `CLI architecture` | docs | docs/modules/cli/ |
|
|
| 134 | `web frontend architecture` | docs | docs/modules/web/ |
|
|
| 135 | `telemetry architecture` | docs | docs/modules/telemetry/ |
|
|
| 136 | `analytics architecture` | docs | docs-archived/modules/analytics/ |
|
|
| 137 | `mirror architecture` | docs | docs/modules/mirror/ |
|
|
| 138 | `registry architecture` | docs | docs/modules/registry/ |
|
|
| 139 | `verifier architecture` | docs | docs/modules/verifier/ |
|
|
| 140 | `replay engine architecture` | docs | docs/modules/replay/ |
|
|
| 141 | `feedser architecture` | docs | docs/modules/feedser/ |
|
|
| 142 | `issuer directory architecture` | docs | docs/modules/issuer-directory/ |
|
|
| 143 | `packs registry architecture` | docs | docs/modules/packs-registry/ |
|
|
| 144 | `facet architecture` | docs | docs-archived/modules/facet/ |
|
|
| 145 | `devportal architecture` | docs | docs/modules/devportal/ |
|
|
| 146 | `doctor architecture` | docs | docs/modules/doctor/ |
|
|
| 147 | `bench tools architecture` | docs | docs/modules/bench/ |
|
|
| 148 | `platform module` | docs | docs/modules/platform/ |
|
|
| 149 | `gateway module` | docs | docs/modules/gateway/ |
|
|
| 150 | `router module` | docs | docs/modules/router/ |
|
|
|
|
### 1.5 Operations, Deployment & Runbooks (30 cases)
|
|
|
|
| # | Query | Expected Entity Type | Expected Match Source |
|
|
|---|-------|---------------------|----------------------|
|
|
| 151 | `deployment guide` | docs | docs/operations/deployment/ |
|
|
| 152 | `production deployment` | docs | docs/operations/deployment/ |
|
|
| 153 | `scaling guide` | docs | docs/operations/ |
|
|
| 154 | `runbook incident response` | docs | docs/runbooks/ |
|
|
| 155 | `emergency procedures` | docs | docs/runbooks/ |
|
|
| 156 | `devops tooling` | docs | docs/operations/devops/ |
|
|
| 157 | `operational governance` | docs | docs/operations/governance/ |
|
|
| 158 | `handoff procedures` | docs | docs/operations/handoff/ |
|
|
| 159 | `monitoring setup` | docs | docs/technical/observability/ |
|
|
| 160 | `observability configuration` | docs | docs/technical/observability/ |
|
|
| 161 | `Prometheus setup` | docs | docs/technical/observability/ |
|
|
| 162 | `OpenTelemetry setup` | docs | docs/technical/observability/ |
|
|
| 163 | `helm chart deployment` | docs | docs/operations/deployment/ |
|
|
| 164 | `docker compose` | docs | devops/compose/ |
|
|
| 165 | `backup procedures` | docs | docs/operations/ |
|
|
| 166 | `disaster recovery` | docs | docs/runbooks/ |
|
|
| 167 | `how to rotate keys` | docs | docs/modules/signer/ |
|
|
| 168 | `certificate renewal` | docs | docs/security/ |
|
|
| 169 | `log rotation configuration` | docs | docs/operations/ |
|
|
| 170 | `performance testing playbook` | docs | docs/dev/performance-testing-playbook.md |
|
|
| 171 | `release notes` | docs | docs/releases/ |
|
|
| 172 | `version history` | docs | docs/releases/ |
|
|
| 173 | `upgrade guide` | docs | docs/releases/ |
|
|
| 174 | `CI/CD pipeline` | docs | docs/technical/cicd/ |
|
|
| 175 | `GitHub Actions integration` | docs | docs/technical/cicd/ |
|
|
| 176 | `GitLab CI integration` | docs | docs/technical/cicd/ |
|
|
| 177 | `Gitea workflow` | docs | .gitea/ |
|
|
| 178 | `compliance audit` | docs | docs/compliance/ |
|
|
| 179 | `governance structure` | docs | docs/GOVERNANCE.md |
|
|
| 180 | `third party dependencies` | docs | docs/legal/THIRD-PARTY-DEPENDENCIES.md |
|
|
|
|
### 1.6 Developer Guides & Plugin Development (30 cases)
|
|
|
|
| # | Query | Expected Entity Type | Expected Match Source |
|
|
|---|-------|---------------------|----------------------|
|
|
| 181 | `plugin development guide` | docs | docs/PLUGIN_SDK_GUIDE.md |
|
|
| 182 | `how to write a plugin` | docs | docs/PLUGIN_SDK_GUIDE.md |
|
|
| 183 | `authority plugin developer guide` | docs | docs/dev/31_AUTHORITY_PLUGIN_DEVELOPER_GUIDE.md |
|
|
| 184 | `excititor connector guide` | docs | docs/dev/30_EXCITITOR_CONNECTOR_GUIDE.md |
|
|
| 185 | `auth client guide` | docs | docs/dev/32_AUTH_CLIENT_GUIDE.md |
|
|
| 186 | `buildx plugin quickstart` | docs | docs/dev/BUILDX_PLUGIN_QUICKSTART.md |
|
|
| 187 | `extending binary analysis` | docs | docs/dev/extending-binary-analysis.md |
|
|
| 188 | `test fixture design` | docs | docs/dev/fixtures.md |
|
|
| 189 | `concelier CLI quickstart` | docs | docs/CONCELIER_CLI_QUICKSTART.md |
|
|
| 190 | `advisory ingestion` | docs | docs/CONCELIER_CLI_QUICKSTART.md |
|
|
| 191 | `SDK code generation` | docs | docs/api/sdk-openapi-program.md |
|
|
| 192 | `API CLI reference` | docs | docs/API_CLI_REFERENCE.md |
|
|
| 193 | `KISA connector` | docs | docs/dev/kisa_connector_notes.md |
|
|
| 194 | `semantic versioning merge` | docs | docs/dev/merge_semver_playbook.md |
|
|
| 195 | `normalized rule recipes` | docs | docs/dev/normalized-rule-recipes.md |
|
|
| 196 | `API contract standards` | docs | docs/dev/contributing/api-contracts.md |
|
|
| 197 | `canonicalization determinism` | docs | docs/dev/contributing/canonicalization-determinism.md |
|
|
| 198 | `corpus contribution guide` | docs | docs/dev/contributing/corpus-contribution-guide.md |
|
|
| 199 | `notification SDK examples` | docs | docs/api/notify-sdk-examples.md |
|
|
| 200 | `smart diff types` | docs | docs/api/smart-diff-types.md |
|
|
| 201 | `hybrid diff patching` | docs | docs/hybrid-diff-patching.md |
|
|
| 202 | `binary diff` | docs | docs/samples/binary-diff/ |
|
|
| 203 | `binary analysis` | docs | docs/dev/extending-binary-analysis.md |
|
|
| 204 | `policy DSL` | docs | docs/modules/policy/ |
|
|
| 205 | `policy studio contract` | docs | docs/contracts/policy-studio.md |
|
|
| 206 | `risk scoring contract` | docs | docs/contracts/risk-scoring.md |
|
|
| 207 | `triage suppress contract` | docs | docs/contracts/triage-suppress-v1.md |
|
|
| 208 | `verification policy` | docs | docs/contracts/verification-policy.md |
|
|
| 209 | `redaction defaults` | docs | docs/contracts/redaction-defaults-decision.md |
|
|
| 210 | `mirror bundle format` | docs | docs/contracts/mirror-bundle.md |
|
|
|
|
### 1.7 Benchmarks & Competitive Analysis (20 cases)
|
|
|
|
| # | Query | Expected Entity Type | Expected Match Source |
|
|
|---|-------|---------------------|----------------------|
|
|
| 211 | `benchmark results` | docs | docs/benchmarks/ |
|
|
| 212 | `performance baselines` | docs | docs/benchmarks/performance-baselines.md |
|
|
| 213 | `accuracy metrics` | docs | docs/benchmarks/accuracy-metrics-framework.md |
|
|
| 214 | `golden corpus` | docs | docs/benchmarks/golden-corpus-kpis.md |
|
|
| 215 | `Trivy comparison` | docs | docs/benchmarks/scanner-feature-comparison-trivy.md |
|
|
| 216 | `Snyk comparison` | docs | docs/benchmarks/scanner-feature-comparison-snyk.md |
|
|
| 217 | `Grype comparison` | docs | docs/benchmarks/scanner-feature-comparison-grype.md |
|
|
| 218 | `competitive landscape` | docs | docs/product/competitive-landscape.md |
|
|
| 219 | `fidelity metrics` | docs | docs/benchmarks/fidelity-metrics.md |
|
|
| 220 | `precision recall curves` | docs | docs/benchmarks/tiered-precision-curves.md |
|
|
| 221 | `Rust analyzer` | docs | docs/benchmarks/scanner-rust-analyzer.md |
|
|
| 222 | `scanning gaps` | docs | docs/benchmarks/scanner/ |
|
|
| 223 | `dotnet scanning` | docs | docs/benchmarks/scanner/deep-dives/dotnet.md |
|
|
| 224 | `Java scanning` | docs | docs/benchmarks/scanner/deep-dives/java.md |
|
|
| 225 | `Python scanning` | docs | docs/benchmarks/scanner/deep-dives/python.md |
|
|
| 226 | `Node.js scanning` | docs | docs/benchmarks/scanner/deep-dives/nodejs.md |
|
|
| 227 | `Golang scanning` | docs | docs/benchmarks/scanner/deep-dives/golang.md |
|
|
| 228 | `SAST analysis` | docs | docs/benchmarks/scanner/deep-dives/sast.md |
|
|
| 229 | `secrets scanning benchmark` | docs | docs/benchmarks/scanner/deep-dives/secrets.md |
|
|
| 230 | `Windows macOS scanning` | docs | docs/benchmarks/scanner/windows-macos-demand.md |
|
|
|
|
---
|
|
|
|
## Domain 2: Knowledge — API Operations (OpenAPI specs)
|
|
|
|
### 2.1 Scanner API (40 cases)
|
|
|
|
| # | Query | Expected Entity Type | Expected Match Source |
|
|
|---|-------|---------------------|----------------------|
|
|
| 231 | `create a scan` | api | POST /api/v1/scans |
|
|
| 232 | `get scan status` | api | GET /api/v1/scans/{scanId} |
|
|
| 233 | `scan API` | api | scanner/openapi.yaml |
|
|
| 234 | `submit call graph` | api | POST /api/v1/scans/{scanId}/callgraphs |
|
|
| 235 | `stream scan events` | api | GET /api/v1/scans/{scanId}/events |
|
|
| 236 | `reachability API` | api | scanner reachability endpoints |
|
|
| 237 | `SBOM upload API` | api | POST /api/v1/sboms/upload |
|
|
| 238 | `layer SBOM` | api | LayerSbomEndpoints |
|
|
| 239 | `scan entropy` | api | POST /api/v1/scans/{scanId}/entropy |
|
|
| 240 | `delta compare API` | api | DeltaCompareEndpoints |
|
|
| 241 | `delta evidence` | api | DeltaEvidenceEndpoints |
|
|
| 242 | `manifest endpoint` | api | ManifestEndpoints |
|
|
| 243 | `SBOM hot lookup` | api | SbomHotLookupEndpoints |
|
|
| 244 | `proof spine API` | api | ProofSpineEndpoints |
|
|
| 245 | `witness endpoint` | api | WitnessEndpoints |
|
|
| 246 | `scanner health` | api | HealthEndpoints |
|
|
| 247 | `call graph endpoint` | api | CallGraphEndpoints |
|
|
| 248 | `validation endpoint` | api | ValidationEndpoints |
|
|
| 249 | `offline kit endpoint` | api | OfflineKitEndpoints |
|
|
| 250 | `fidelity endpoint` | api | FidelityEndpoints |
|
|
| 251 | `score replay API` | api | ScoreReplayEndpoints |
|
|
| 252 | `EPSS scores API` | api | EpssEndpoints |
|
|
| 253 | `approval endpoint` | api | ApprovalEndpoints |
|
|
| 254 | `baseline endpoint` | api | BaselineEndpoints |
|
|
| 255 | `counterfactual analysis API` | api | CounterfactualEndpoints |
|
|
| 256 | `actionables endpoint` | api | ActionablesEndpoints |
|
|
| 257 | `secret detection settings` | api | SecretDetectionSettingsEndpoints |
|
|
| 258 | `smart diff endpoint` | api | SmartDiffEndpoints |
|
|
| 259 | `unknowns endpoint` | api | UnknownsEndpoints |
|
|
| 260 | `triage API` | api | Triage/*Endpoints |
|
|
| 261 | `reachability slice` | api | SliceEndpoints |
|
|
| 262 | `GitHub code scanning` | api | GitHubCodeScanningEndpoints |
|
|
| 263 | `scanner webhook` | api | WebhookEndpoints |
|
|
| 264 | `runtime analysis API` | api | RuntimeEndpoints |
|
|
| 265 | `reachability evidence` | api | ReachabilityEvidenceEndpoints |
|
|
| 266 | `reachability stack` | api | ReachabilityStackEndpoints |
|
|
| 267 | `scan report generation` | api | ReportEndpoints |
|
|
| 268 | `scan evidence query` | api | EvidenceEndpoints |
|
|
| 269 | `sources tracking API` | api | SourcesEndpoints |
|
|
| 270 | `scan observability` | api | ObservabilityEndpoints |
|
|
|
|
### 2.2 Policy Engine API (40 cases)
|
|
|
|
| # | Query | Expected Entity Type | Expected Match Source |
|
|
|---|-------|---------------------|----------------------|
|
|
| 271 | `verification policy API` | api | VerificationPolicyEndpoints |
|
|
| 272 | `policy pack API` | api | PolicyPackEndpoints |
|
|
| 273 | `policy snapshot` | api | PolicySnapshotEndpoints |
|
|
| 274 | `violation tracking API` | api | ViolationEndpoints |
|
|
| 275 | `policy override API` | api | OverrideEndpoints |
|
|
| 276 | `risk budget API` | api | BudgetEndpoints, RiskBudgetEndpoints |
|
|
| 277 | `risk profile API` | api | RiskProfileEndpoints |
|
|
| 278 | `risk simulation API` | api | RiskSimulationEndpoints |
|
|
| 279 | `effective policy API` | api | EffectivePolicyEndpoints |
|
|
| 280 | `policy decision endpoint` | api | PolicyDecisionEndpoint |
|
|
| 281 | `batch evaluation API` | api | BatchEvaluationEndpoint |
|
|
| 282 | `policy conflict API` | api | ConflictEndpoints |
|
|
| 283 | `CVSS receipt endpoint` | api | CvssReceiptEndpoints |
|
|
| 284 | `attestation report API` | api | AttestationReportEndpoints |
|
|
| 285 | `policy export` | api | ConsoleExportEndpoints |
|
|
| 286 | `scope attachment API` | api | ScopeAttachmentEndpoints |
|
|
| 287 | `staleness endpoint` | api | StalenessEndpoints |
|
|
| 288 | `sealed mode API` | api | SealedModeEndpoints |
|
|
| 289 | `policy lint API` | api | PolicyLintEndpoints |
|
|
| 290 | `policy compilation` | api | PolicyCompilationEndpoints |
|
|
| 291 | `verify determinism API` | api | VerifyDeterminismEndpoints |
|
|
| 292 | `merge preview API` | api | MergePreviewEndpoints |
|
|
| 293 | `policy editor API` | api | VerificationPolicyEditorEndpoints |
|
|
| 294 | `air gap notification API` | api | AirGapNotificationEndpoints |
|
|
| 295 | `determinization config` | api | DeterminizationConfigEndpoints |
|
|
| 296 | `delta if present` | api | DeltaIfPresentEndpoints |
|
|
| 297 | `trust weighting API` | api | TrustWeightingEndpoint |
|
|
| 298 | `overlay simulation` | api | OverlaySimulationEndpoint |
|
|
| 299 | `path scope simulation` | api | PathScopeSimulationEndpoint |
|
|
| 300 | `evidence summary API` | api | EvidenceSummaryEndpoint |
|
|
| 301 | `policy pack bundle` | api | PolicyPackBundleEndpoints |
|
|
| 302 | `risk profile air gap` | api | RiskProfileAirGapEndpoints |
|
|
| 303 | `risk profile schema` | api | RiskProfileSchemaEndpoints |
|
|
| 304 | `console simulation` | api | ConsoleSimulationEndpoint |
|
|
| 305 | `policy worker` | api | PolicyWorkerEndpoint |
|
|
| 306 | `advisory AI knobs` | api | AdvisoryAiKnobsEndpoint |
|
|
| 307 | `profile event tracking` | api | ProfileEventEndpoints |
|
|
| 308 | `profile export` | api | ProfileExportEndpoints |
|
|
| 309 | `batch context API` | api | BatchContextEndpoint |
|
|
| 310 | `orchestrator job API` | api | OrchestratorJobEndpoint |
|
|
|
|
### 2.3 Orchestrator, Scheduler & Release API (30 cases)
|
|
|
|
| # | Query | Expected Entity Type | Expected Match Source |
|
|
|---|-------|---------------------|----------------------|
|
|
| 311 | `release API` | api | ReleaseEndpoints |
|
|
| 312 | `approval workflow API` | api | ApprovalEndpoints |
|
|
| 313 | `DAG query API` | api | DagEndpoints |
|
|
| 314 | `circuit breaker API` | api | CircuitBreakerEndpoints |
|
|
| 315 | `quota governance API` | api | QuotaGovernanceEndpoints |
|
|
| 316 | `audit trail API` | api | AuditEndpoints |
|
|
| 317 | `release dashboard API` | api | ReleaseDashboardEndpoints |
|
|
| 318 | `run execution API` | api | RunEndpoints |
|
|
| 319 | `event stream websocket` | api | StreamEndpoints |
|
|
| 320 | `KPI endpoint` | api | KpiEndpoints |
|
|
| 321 | `job management API` | api | JobEndpoints |
|
|
| 322 | `first signal API` | api | FirstSignalEndpoints |
|
|
| 323 | `export job API` | api | ExportJobEndpoints |
|
|
| 324 | `dead letter queue API` | api | DeadLetterEndpoints |
|
|
| 325 | `SLO management API` | api | SloEndpoints |
|
|
| 326 | `source tracking API` | api | SourceEndpoints |
|
|
| 327 | `schedule management API` | api | ScheduleEndpoints |
|
|
| 328 | `policy simulation API` | api | PolicySimulationEndpointExtensions |
|
|
| 329 | `graph job API` | api | GraphJobEndpointExtensions |
|
|
| 330 | `failure signature API` | api | FailureSignatureEndpoints |
|
|
| 331 | `event webhook API` | api | EventWebhookEndpointExtensions |
|
|
| 332 | `resolver job API` | api | ResolverJobEndpointExtensions |
|
|
| 333 | `worker coordination API` | api | WorkerEndpoints |
|
|
| 334 | `scale auto-scaling API` | api | ScaleEndpoints |
|
|
| 335 | `pack registry API` | api | PackRegistryEndpoints |
|
|
| 336 | `pack run API` | api | PackRunEndpoints |
|
|
| 337 | `ledger query API` | api | LedgerEndpoints |
|
|
| 338 | `release control v2` | api | ReleaseControlV2Endpoints |
|
|
| 339 | `openapi discovery endpoint` | api | OpenApiEndpoints |
|
|
| 340 | `health check API` | api | HealthEndpoints |
|
|
|
|
### 2.4 Platform, Authority & Notification API (30 cases)
|
|
|
|
| # | Query | Expected Entity Type | Expected Match Source |
|
|
|---|-------|---------------------|----------------------|
|
|
| 341 | `platform health API` | api | PlatformEndpoints |
|
|
| 342 | `quota summary API` | api | PlatformEndpoints |
|
|
| 343 | `environment settings API` | api | EnvironmentSettingsEndpoints |
|
|
| 344 | `security read model` | api | SecurityReadModelEndpoints |
|
|
| 345 | `integration read model` | api | IntegrationReadModelEndpoints |
|
|
| 346 | `topology query API` | api | TopologyReadModelEndpoints |
|
|
| 347 | `analytics data API` | api | AnalyticsEndpoints |
|
|
| 348 | `score calculation API` | api | ScoreEndpoints |
|
|
| 349 | `function map API` | api | FunctionMapEndpoints |
|
|
| 350 | `evidence thread API` | api | EvidenceThreadEndpoints |
|
|
| 351 | `federation telemetry API` | api | FederationTelemetryEndpoints |
|
|
| 352 | `trust signing admin API` | api | AdministrationTrustSigningMutationEndpoints |
|
|
| 353 | `OAuth token endpoint` | api | Authority endpoints |
|
|
| 354 | `OIDC discovery` | api | Authority endpoints |
|
|
| 355 | `token introspection` | api | Authority endpoints |
|
|
| 356 | `JWKS endpoint` | api | Authority endpoints |
|
|
| 357 | `notification rules API` | api | RuleEndpoints |
|
|
| 358 | `notification template API` | api | TemplateEndpoints |
|
|
| 359 | `incident tracking API` | api | IncidentEndpoints |
|
|
| 360 | `storm breaker API` | api | StormBreakerEndpoints |
|
|
| 361 | `throttle API` | api | ThrottleEndpoints |
|
|
| 362 | `quiet hours API` | api | QuietHoursEndpoints |
|
|
| 363 | `escalation rules API` | api | EscalationEndpoints |
|
|
| 364 | `notification simulation` | api | SimulationEndpoints |
|
|
| 365 | `operator override API` | api | OperatorOverrideEndpoints |
|
|
| 366 | `notification localization` | api | LocalizationEndpoints |
|
|
| 367 | `live incident feed` | api | IncidentLiveFeed |
|
|
| 368 | `context management API` | api | ContextEndpoints |
|
|
| 369 | `seed database API` | api | SeedEndpoints |
|
|
| 370 | `setup wizard API` | api | SetupEndpoints |
|
|
|
|
### 2.5 Evidence, Attestation, VEX & Export API (30 cases)
|
|
|
|
| # | Query | Expected Entity Type | Expected Match Source |
|
|
|---|-------|---------------------|----------------------|
|
|
| 371 | `unified search API` | api | POST /v1/search/query |
|
|
| 372 | `knowledge search API` | api | POST /v1/advisory-ai/search |
|
|
| 373 | `advisory AI chat API` | api | ChatEndpoints |
|
|
| 374 | `LLM adapter API` | api | LlmAdapterEndpoints |
|
|
| 375 | `evidence pack API` | api | EvidencePackEndpoints |
|
|
| 376 | `verdict issuance API` | api | VerdictEndpoints |
|
|
| 377 | `predicate registry API` | api | PredicateRegistryEndpoints |
|
|
| 378 | `watchlist API` | api | WatchlistEndpoints |
|
|
| 379 | `export API` | api | ExportApiEndpoints |
|
|
| 380 | `risk bundle API` | api | RiskBundleEndpoints |
|
|
| 381 | `audit bundle API` | api | AuditBundleEndpoints |
|
|
| 382 | `promotion attestation API` | api | PromotionAttestationEndpoints |
|
|
| 383 | `lineage export API` | api | LineageExportEndpoints |
|
|
| 384 | `exception report API` | api | ExceptionReportEndpoints |
|
|
| 385 | `feed mirror API` | api | FeedMirrorManagementEndpoints |
|
|
| 386 | `SBOM ingestion API` | api | SbomEndpointExtensions |
|
|
| 387 | `canonical advisory API` | api | CanonicalAdvisoryEndpointExtensions |
|
|
| 388 | `advisory source API` | api | AdvisorySourceEndpointExtensions |
|
|
| 389 | `federation API` | api | FederationEndpointExtensions |
|
|
| 390 | `air gap endpoint` | api | AirGapEndpointExtensions |
|
|
| 391 | `findings scoring API` | api | ScoringEndpoints |
|
|
| 392 | `runtime traces API` | api | RuntimeTracesEndpoints |
|
|
| 393 | `evidence graph API` | api | EvidenceGraphEndpoints |
|
|
| 394 | `finding summary API` | api | FindingSummaryEndpoints |
|
|
| 395 | `backport API` | api | BackportEndpoints |
|
|
| 396 | `reachability map API` | api | ReachabilityMapEndpoints |
|
|
| 397 | `VEX ingest API` | api | IngestEndpoints |
|
|
| 398 | `linkset API` | api | LinksetEndpoints |
|
|
| 399 | `observation API` | api | ObservationEndpoints |
|
|
| 400 | `Rekor attestation API` | api | RekorAttestationEndpoints |
|
|
|
|
### 2.6 Gateway, Policy Gateway, Graph & More (30 cases)
|
|
|
|
| # | Query | Expected Entity Type | Expected Match Source |
|
|
|---|-------|---------------------|----------------------|
|
|
| 401 | `registry webhook API` | api | RegistryWebhookEndpoints |
|
|
| 402 | `gate endpoint` | api | GateEndpoints |
|
|
| 403 | `score gate API` | api | ScoreGateEndpoints |
|
|
| 404 | `exception management API` | api | ExceptionEndpoints |
|
|
| 405 | `exception approval API` | api | ExceptionApprovalEndpoints |
|
|
| 406 | `governance API` | api | GovernanceEndpoints |
|
|
| 407 | `delta tracking API` | api | DeltasEndpoints |
|
|
| 408 | `tool lattice API` | api | ToolLatticeEndpoints |
|
|
| 409 | `signing ceremony API` | api | CeremonyEndpoints |
|
|
| 410 | `key rotation API` | api | KeyRotationEndpoints |
|
|
| 411 | `signer endpoint` | api | SignerEndpoints |
|
|
| 412 | `timeline query API` | api | TimelineEndpoints |
|
|
| 413 | `timeline replay API` | api | ReplayEndpoints |
|
|
| 414 | `timeline export API` | api | ExportEndpoints |
|
|
| 415 | `graph search API` | api | Graph search contracts |
|
|
| 416 | `reachgraph query` | api | ReachGraph endpoints |
|
|
| 417 | `binary vulnerability API` | api | BinaryIndex endpoints |
|
|
| 418 | `remediation registry API` | api | Remediation endpoints |
|
|
| 419 | `symbol source API` | api | Symbols endpoints |
|
|
| 420 | `VEX hub export API` | api | VexHub endpoints |
|
|
| 421 | `issuer management API` | api | IssuerDirectory endpoints |
|
|
| 422 | `evidence verdict API` | api | EvidenceLocker VerdictEndpoints |
|
|
| 423 | `evidence thread audit` | api | EvidenceThreadEndpoints |
|
|
| 424 | `evidence audit trail` | api | EvidenceAuditEndpoints |
|
|
| 425 | `evidence export API` | api | EvidenceLocker ExportEndpoints |
|
|
| 426 | `resolve VEX API` | api | ResolveEndpoint |
|
|
| 427 | `risk feed API` | api | RiskFeedEndpoints |
|
|
| 428 | `VEX policy API` | api | PolicyEndpoints (Excititor) |
|
|
| 429 | `mirror registration API` | api | MirrorRegistrationEndpoints |
|
|
| 430 | `interest score API` | api | InterestScoreEndpointExtensions |
|
|
|
|
---
|
|
|
|
## Domain 3: Knowledge — Doctor Checks
|
|
|
|
### 3.1 Database & Infrastructure Checks (20 cases)
|
|
|
|
| # | Query | Expected Entity Type | Expected Match Source |
|
|
|---|-------|---------------------|----------------------|
|
|
| 431 | `check.postgres.connectivity` | doctor | Postgres Connectivity check |
|
|
| 432 | `database connection failing` | doctor | check.postgres.connectivity |
|
|
| 433 | `postgres migrations pending` | doctor | check.postgres.migrations |
|
|
| 434 | `connection pool exhausted` | doctor | check.postgres.pool |
|
|
| 435 | `disk space running low` | doctor | check.storage.diskspace |
|
|
| 436 | `evidence locker write check` | doctor | check.storage.evidencelocker |
|
|
| 437 | `backup directory writable` | doctor | check.storage.backup |
|
|
| 438 | `log directory check` | doctor | check.logs.directory.writable |
|
|
| 439 | `log rotation check` | doctor | check.logs.rotation.configured |
|
|
| 440 | `Prometheus scrape check` | doctor | check.metrics.prometheus.scrape |
|
|
| 441 | `OTLP endpoint check` | doctor | check.telemetry.otlp.endpoint |
|
|
| 442 | `dead letter queue check` | doctor | check.operations.dead-letter |
|
|
| 443 | `job queue health check` | doctor | check.operations.job-queue |
|
|
| 444 | `scheduler health check` | doctor | check.operations.scheduler |
|
|
| 445 | `policy engine health` | doctor | check.policy.engine |
|
|
| 446 | `scanner queue check` | doctor | check.scanner.queue |
|
|
| 447 | `scanner resource utilization` | doctor | check.scanner.resources |
|
|
| 448 | `SBOM generation check` | doctor | check.scanner.sbom |
|
|
| 449 | `vulnerability scan check` | doctor | check.scanner.vuln |
|
|
| 450 | `witness graph check` | doctor | check.scanner.witness.graph |
|
|
|
|
### 3.2 Security & Auth Checks (20 cases)
|
|
|
|
| # | Query | Expected Entity Type | Expected Match Source |
|
|
|---|-------|---------------------|----------------------|
|
|
| 451 | `authentication config check` | doctor | check.auth.config |
|
|
| 452 | `OIDC provider connectivity` | doctor | check.auth.oidc |
|
|
| 453 | `signing key health` | doctor | check.auth.signing-key |
|
|
| 454 | `token service health` | doctor | check.auth.token-service |
|
|
| 455 | `certificate chain validation` | doctor | check.crypto.certchain |
|
|
| 456 | `FIPS compliance check` | doctor | check.crypto.fips |
|
|
| 457 | `HSM availability check` | doctor | check.crypto.hsm |
|
|
| 458 | `eIDAS compliance check` | doctor | check.crypto.eidas |
|
|
| 459 | `GOST availability check` | doctor | check.crypto.gost |
|
|
| 460 | `SM crypto check` | doctor | check.crypto.sm |
|
|
| 461 | `Rekor connectivity check` | doctor | check.attestation.rekor.connectivity |
|
|
| 462 | `clock skew check` | doctor | check.attestation.clock.skew |
|
|
| 463 | `cosign key material` | doctor | check.attestation.cosign.keymaterial |
|
|
| 464 | `signing key expiration` | doctor | check.attestation.keymaterial |
|
|
| 465 | `transparency log consistency` | doctor | check.attestation.transparency.consistency |
|
|
| 466 | `Rekor verification job` | doctor | check.attestation.rekor.verification.job |
|
|
| 467 | `VEX issuer trust check` | doctor | check.vex.issuer-trust |
|
|
| 468 | `VEX schema compliance check` | doctor | check.vex.schema |
|
|
| 469 | `VEX document validation` | doctor | check.vex.validation |
|
|
| 470 | `environment secrets check` | doctor | check.environment.secrets |
|
|
|
|
### 3.3 Compliance, Agent & Notification Checks (25 cases)
|
|
|
|
| # | Query | Expected Entity Type | Expected Match Source |
|
|
|---|-------|---------------------|----------------------|
|
|
| 471 | `audit readiness check` | doctor | check.compliance.audit-readiness |
|
|
| 472 | `evidence integrity check` | doctor | check.compliance.evidence-integrity |
|
|
| 473 | `provenance completeness` | doctor | check.compliance.provenance-completeness |
|
|
| 474 | `attestation signing health` | doctor | check.compliance.attestation-signing |
|
|
| 475 | `evidence generation rate` | doctor | check.compliance.evidence-rate |
|
|
| 476 | `export readiness check` | doctor | check.compliance.export-readiness |
|
|
| 477 | `compliance framework check` | doctor | check.compliance.framework |
|
|
| 478 | `evidence locker index` | doctor | check.evidencelocker.index |
|
|
| 479 | `merkle tree anchor` | doctor | check.evidencelocker.merkle |
|
|
| 480 | `provenance chain check` | doctor | check.evidencelocker.provenance |
|
|
| 481 | `attestation retrieval` | doctor | check.evidencelocker.retrieval |
|
|
| 482 | `agent heartbeat freshness` | doctor | check.agent.heartbeat.freshness |
|
|
| 483 | `agent capacity check` | doctor | check.agent.capacity |
|
|
| 484 | `stale agent detection` | doctor | check.agent.stale |
|
|
| 485 | `agent cluster health` | doctor | check.agent.cluster.health |
|
|
| 486 | `agent cluster quorum` | doctor | check.agent.cluster.quorum |
|
|
| 487 | `agent version consistency` | doctor | check.agent.version.consistency |
|
|
| 488 | `agent certificate expiry` | doctor | check.agent.certificate.expiry |
|
|
| 489 | `agent task backlog` | doctor | check.agent.task.backlog |
|
|
| 490 | `email notification check` | doctor | check.notify.email.configured |
|
|
| 491 | `Slack connectivity check` | doctor | check.notify.slack.connectivity |
|
|
| 492 | `Teams notification check` | doctor | check.notify.teams.configured |
|
|
| 493 | `notification queue health` | doctor | check.notify.queue.health |
|
|
| 494 | `webhook connectivity` | doctor | check.notify.webhook.connectivity |
|
|
| 495 | `TSA response time check` | doctor | check.timestamp.tsa.response-time |
|
|
|
|
### 3.4 Environment & Release Checks (15 cases)
|
|
|
|
| # | Query | Expected Entity Type | Expected Match Source |
|
|
|---|-------|---------------------|----------------------|
|
|
| 496 | `environment connectivity` | doctor | check.environment.connectivity |
|
|
| 497 | `environment drift` | doctor | check.environment.drift |
|
|
| 498 | `network policy enforcement` | doctor | check.environment.network.policy |
|
|
| 499 | `environment capacity` | doctor | check.environment.capacity |
|
|
| 500 | `deployment health check` | doctor | check.environment.deployments |
|
|
| 501 | `active release health` | doctor | check.release.active |
|
|
| 502 | `release configuration check` | doctor | check.release.configuration |
|
|
| 503 | `environment readiness` | doctor | check.release.environment.readiness |
|
|
| 504 | `promotion gates check` | doctor | check.release.promotion.gates |
|
|
| 505 | `rollback readiness` | doctor | check.release.rollback.readiness |
|
|
| 506 | `release schedule check` | doctor | check.release.schedule |
|
|
| 507 | `reachability computation check` | doctor | check.scanner.reachability |
|
|
| 508 | `slice cache check` | doctor | check.scanner.slice.cache |
|
|
| 509 | `buildinfo cache check` | doctor | check.binaryanalysis.buildinfo.cache |
|
|
| 510 | `debuginfod availability` | doctor | check.binaryanalysis.debuginfod.available |
|
|
|
|
---
|
|
|
|
## Domain 4: Findings (Security Findings & Vulnerabilities)
|
|
|
|
### 4.1 CVE Searches (50 cases)
|
|
|
|
| # | Query | Expected Entity Type | Expected Match Source |
|
|
|---|-------|---------------------|----------------------|
|
|
| 511 | `CVE-2024-21626` | finding | Container escape via runc |
|
|
| 512 | `CVE-2024-3094` | finding | XZ Utils backdoor |
|
|
| 513 | `CVE-2023-44487` | finding | HTTP/2 Rapid Reset |
|
|
| 514 | `CVE-2021-44228` | finding | Log4Shell |
|
|
| 515 | `CVE-2021-45046` | finding | Log4j followup |
|
|
| 516 | `CVE-2023-4863` | finding | libwebp heap overflow |
|
|
| 517 | `CVE-2024-0056` | finding | .NET SQL injection |
|
|
| 518 | `CVE-2023-38545` | finding | curl SOCKS5 overflow |
|
|
| 519 | `CVE-2023-32233` | finding | Linux kernel nf_tables |
|
|
| 520 | `CVE-2024-6387` | finding | OpenSSH regreSSHion |
|
|
| 521 | `Log4Shell` | finding | CVE-2021-44228 |
|
|
| 522 | `Heartbleed` | finding | CVE-2014-0160 |
|
|
| 523 | `Spring4Shell` | finding | CVE-2022-22965 |
|
|
| 524 | `Shellshock` | finding | CVE-2014-6271 |
|
|
| 525 | `POODLE` | finding | CVE-2014-3566 |
|
|
| 526 | `critical vulnerabilities` | finding | severity=CRITICAL |
|
|
| 527 | `high severity findings` | finding | severity=HIGH |
|
|
| 528 | `remote code execution` | finding | CWE-94 |
|
|
| 529 | `SQL injection vulnerability` | finding | CWE-89 |
|
|
| 530 | `buffer overflow` | finding | CWE-120 |
|
|
| 531 | `cross site scripting` | finding | CWE-79 |
|
|
| 532 | `privilege escalation` | finding | various CWEs |
|
|
| 533 | `denial of service` | finding | CWE-400 |
|
|
| 534 | `path traversal` | finding | CWE-22 |
|
|
| 535 | `deserialization vulnerability` | finding | CWE-502 |
|
|
| 536 | `SSRF vulnerability` | finding | CWE-918 |
|
|
| 537 | `integer overflow` | finding | CWE-190 |
|
|
| 538 | `use after free` | finding | CWE-416 |
|
|
| 539 | `null pointer dereference` | finding | CWE-476 |
|
|
| 540 | `race condition` | finding | CWE-362 |
|
|
| 541 | `CVSS score 9.8` | finding | CVSS filter |
|
|
| 542 | `CVSS greater than 7` | finding | CVSS filter |
|
|
| 543 | `exploit available` | finding | exploitKnown=true |
|
|
| 544 | `zero day vulnerability` | finding | recent, no patch |
|
|
| 545 | `EPSS score high` | finding | EPSS > 0.5 |
|
|
| 546 | `findings for log4j` | finding | package=log4j |
|
|
| 547 | `openssl vulnerabilities` | finding | package=openssl |
|
|
| 548 | `npm lodash vulnerability` | finding | pkg:npm/lodash |
|
|
| 549 | `jackson-databind CVE` | finding | pkg:maven/jackson-databind |
|
|
| 550 | `spring framework vulnerability` | finding | spring-framework |
|
|
| 551 | `golang net/http vulnerability` | finding | pkg:golang/net |
|
|
| 552 | `python requests vulnerability` | finding | pkg:pypi/requests |
|
|
| 553 | `ruby on rails CVE` | finding | pkg:gem/rails |
|
|
| 554 | `docker runc vulnerability` | finding | pkg:golang/runc |
|
|
| 555 | `kubernetes vulnerability` | finding | kubernetes |
|
|
| 556 | `nginx CVE` | finding | nginx |
|
|
| 557 | `apache httpd vulnerability` | finding | apache httpd |
|
|
| 558 | `postgresql vulnerability` | finding | postgresql |
|
|
| 559 | `redis vulnerability` | finding | redis |
|
|
| 560 | `alpine linux CVE` | finding | alpine |
|
|
|
|
### 4.2 PURL & Package Searches (30 cases)
|
|
|
|
| # | Query | Expected Entity Type | Expected Match Source |
|
|
|---|-------|---------------------|----------------------|
|
|
| 561 | `pkg:npm/lodash@4.17.21` | finding | npm lodash |
|
|
| 562 | `pkg:maven/org.apache.logging.log4j/log4j-core@2.17.0` | finding | log4j-core |
|
|
| 563 | `pkg:pypi/django@4.2` | finding | Django |
|
|
| 564 | `pkg:cargo/tokio@1.28` | finding | tokio |
|
|
| 565 | `pkg:golang/github.com/opencontainers/runc@1.1.10` | finding | runc |
|
|
| 566 | `pkg:nuget/Newtonsoft.Json@13.0.3` | finding | Newtonsoft.Json |
|
|
| 567 | `pkg:gem/actionpack@7.0` | finding | Rails actionpack |
|
|
| 568 | `pkg:composer/symfony/http-kernel` | finding | Symfony |
|
|
| 569 | `pkg:npm/express@4.18` | finding | Express.js |
|
|
| 570 | `pkg:npm/axios@1.6` | finding | Axios |
|
|
| 571 | `affected packages npm` | finding | npm ecosystem |
|
|
| 572 | `affected packages maven` | finding | Maven ecosystem |
|
|
| 573 | `affected packages pip` | finding | PyPI ecosystem |
|
|
| 574 | `affected packages cargo` | finding | Cargo/Rust ecosystem |
|
|
| 575 | `affected packages alpine` | finding | Alpine Linux |
|
|
| 576 | `affected packages debian` | finding | Debian |
|
|
| 577 | `affected packages ubuntu` | finding | Ubuntu |
|
|
| 578 | `affected packages centos` | finding | CentOS |
|
|
| 579 | `packages with known exploits` | finding | exploitKnown=true |
|
|
| 580 | `packages with critical severity` | finding | severity=CRITICAL |
|
|
| 581 | `transitive dependencies vulnerable` | finding | transitive deps |
|
|
| 582 | `outdated packages security` | finding | version range |
|
|
| 583 | `library vulnerabilities` | finding | library scan |
|
|
| 584 | `container base image vulnerabilities` | finding | container scan |
|
|
| 585 | `OS package vulnerabilities` | finding | OS scan |
|
|
| 586 | `runtime dependency security` | finding | runtime deps |
|
|
| 587 | `development dependency vulnerability` | finding | dev deps |
|
|
| 588 | `binary vulnerability` | finding | binary analysis |
|
|
| 589 | `Go module vulnerability` | finding | Go modules |
|
|
| 590 | `.NET NuGet vulnerability` | finding | NuGet packages |
|
|
|
|
### 4.3 GHSA & Source Searches (20 cases)
|
|
|
|
| # | Query | Expected Entity Type | Expected Match Source |
|
|
|---|-------|---------------------|----------------------|
|
|
| 591 | `GHSA-xxxx-yyyy-zzzz` | finding | GitHub Security Advisory |
|
|
| 592 | `GitHub advisory` | finding | GHSA source |
|
|
| 593 | `NVD advisory` | finding | NVD source |
|
|
| 594 | `CISA advisory` | finding | CISA source |
|
|
| 595 | `Microsoft security advisory` | finding | MSRC source |
|
|
| 596 | `Ubuntu security notice` | finding | USN source |
|
|
| 597 | `SUSE security advisory` | finding | SUSE source |
|
|
| 598 | `Alpine security advisory` | finding | Alpine source |
|
|
| 599 | `Red Hat security advisory` | finding | RHSA source |
|
|
| 600 | `Debian security advisory` | finding | DSA source |
|
|
| 601 | `Cisco advisory` | finding | Cisco source |
|
|
| 602 | `Oracle security advisory` | finding | Oracle source |
|
|
| 603 | `ENISA advisory` | finding | ENISA source |
|
|
| 604 | `JVN advisory` | finding | JVN (Japan) source |
|
|
| 605 | `BDU advisory` | finding | BDU (Russia) source |
|
|
| 606 | `CNNVD advisory` | finding | CNNVD (China) source |
|
|
| 607 | `CNVD advisory` | finding | CNVD (China) source |
|
|
| 608 | `advisories published today` | finding | date filter |
|
|
| 609 | `advisories modified this week` | finding | date filter |
|
|
| 610 | `recently discovered vulnerabilities` | finding | date filter |
|
|
|
|
---
|
|
|
|
## Domain 5: VEX (Vulnerability Exploitability Exchange)
|
|
|
|
### 5.1 VEX Status & Justification Searches (30 cases)
|
|
|
|
| # | Query | Expected Entity Type | Expected Match Source |
|
|
|---|-------|---------------------|----------------------|
|
|
| 611 | `VEX not affected` | vex_statement | status=not_affected |
|
|
| 612 | `VEX affected` | vex_statement | status=affected |
|
|
| 613 | `VEX fixed` | vex_statement | status=fixed |
|
|
| 614 | `VEX under investigation` | vex_statement | status=under_investigation |
|
|
| 615 | `component not present justification` | vex_statement | justification |
|
|
| 616 | `vulnerable code not present` | vex_statement | justification |
|
|
| 617 | `code not in execute path` | vex_statement | justification |
|
|
| 618 | `code not executable` | vex_statement | justification |
|
|
| 619 | `adversary cannot control code` | vex_statement | justification |
|
|
| 620 | `inline mitigations exist` | vex_statement | justification |
|
|
| 621 | `VEX for CVE-2024-21626` | vex_statement | vulnerability match |
|
|
| 622 | `VEX for log4j` | vex_statement | package match |
|
|
| 623 | `VEX from vendor` | vex_statement | issuer=VENDOR |
|
|
| 624 | `VEX from community` | vex_statement | issuer=COMMUNITY |
|
|
| 625 | `trusted VEX statements` | vex_statement | trust=TRUSTED |
|
|
| 626 | `authoritative VEX` | vex_statement | trust=AUTHORITATIVE |
|
|
| 627 | `OpenVEX document` | vex_statement | format=openvex |
|
|
| 628 | `CSAF VEX document` | vex_statement | format=csaf |
|
|
| 629 | `CycloneDX VEX` | vex_statement | format=cyclonedx |
|
|
| 630 | `VEX consensus conflict` | vex_statement | conflict resolution |
|
|
| 631 | `VEX statement for production` | vex_statement | environment filter |
|
|
| 632 | `VEX impact statement` | vex_statement | impactStatement field |
|
|
| 633 | `VEX action required` | vex_statement | actionStatement field |
|
|
| 634 | `VEX expiring soon` | vex_statement | TTL/freshness |
|
|
| 635 | `VEX signature verification` | vex_statement | signature check |
|
|
| 636 | `VEX trust profile` | vex_statement | trust profile config |
|
|
| 637 | `VEX override` | vex_statement | manual override |
|
|
| 638 | `how to write VEX` | vex_statement + docs | VEX documentation |
|
|
| 639 | `VEX schema validation` | vex_statement + doctor | check.vex.schema |
|
|
| 640 | `VEX issuer directory` | vex_statement | issuer lookup |
|
|
|
|
### 5.2 VEX Workflow & Integration (20 cases)
|
|
|
|
| # | Query | Expected Entity Type | Expected Match Source |
|
|
|---|-------|---------------------|----------------------|
|
|
| 641 | `generate VEX document` | vex_statement | CLI stella vex-gen |
|
|
| 642 | `ingest VEX statement` | vex_statement | IngestEndpoints |
|
|
| 643 | `VEX hub search` | vex_statement | VexHub endpoints |
|
|
| 644 | `VEX studio create` | vex_statement | Web VEX Studio |
|
|
| 645 | `VEX timeline view` | vex_statement | Web VEX Timeline |
|
|
| 646 | `VEX gate scan` | vex_statement | VexGateScan feature |
|
|
| 647 | `export VEX bundle` | vex_statement | VexHub export |
|
|
| 648 | `VEX evidence proof` | vex_statement | docs/api/vex-proof-schema.md |
|
|
| 649 | `VEX consensus handling` | vex_statement | docs/VEX_CONSENSUS_GUIDE.md |
|
|
| 650 | `multiple VEX sources disagree` | vex_statement | conflict resolution |
|
|
| 651 | `VEX trust weighting` | vex_statement | trust weight config |
|
|
| 652 | `VEX freshness scoring` | vex_statement | TTL/staleness |
|
|
| 653 | `VEX linked to finding` | vex_statement + finding | linkset |
|
|
| 654 | `VEX suppresses finding` | vex_statement | suppression logic |
|
|
| 655 | `VEX as evidence` | vex_statement | evidence pipeline |
|
|
| 656 | `VEX attestation` | vex_statement | attestation predicate |
|
|
| 657 | `VEX policy evaluation` | vex_statement + policy | policy gate |
|
|
| 658 | `VEX mirror` | vex_statement | mirror endpoints |
|
|
| 659 | `VEX feed subscription` | vex_statement | feed mirror |
|
|
| 660 | `VEX document lifecycle` | vex_statement | lifecycle docs |
|
|
|
|
---
|
|
|
|
## Domain 6: Policy (Policy Rules, Evaluations, Violations)
|
|
|
|
### 6.1 Policy Management Searches (30 cases)
|
|
|
|
| # | Query | Expected Entity Type | Expected Match Source |
|
|
|---|-------|---------------------|----------------------|
|
|
| 661 | `create policy rule` | policy_rule | Policy Studio |
|
|
| 662 | `policy pack install` | policy_rule | CLI stella policy install |
|
|
| 663 | `validate policy YAML` | policy_rule | stella policy validate-yaml |
|
|
| 664 | `policy simulation` | policy_rule | stella policy simulate |
|
|
| 665 | `push policy to OCI` | policy_rule | stella policy push |
|
|
| 666 | `pull policy from registry` | policy_rule | stella policy pull |
|
|
| 667 | `policy pack bundle` | policy_rule | export/import bundle |
|
|
| 668 | `block critical vulnerabilities` | policy_rule | severity gate rule |
|
|
| 669 | `require SBOM attestation` | policy_rule | attestation requirement |
|
|
| 670 | `require VEX for all CVEs` | policy_rule | VEX requirement |
|
|
| 671 | `maximum CVSS score allowed` | policy_rule | score threshold |
|
|
| 672 | `block exploit available` | policy_rule | exploit gate |
|
|
| 673 | `require reachability proof` | policy_rule | reachability gate |
|
|
| 674 | `policy for production environment` | policy_rule | scope=production |
|
|
| 675 | `policy for staging environment` | policy_rule | scope=staging |
|
|
| 676 | `policy exception request` | policy_rule | exception management |
|
|
| 677 | `policy waiver` | policy_rule | exception/override |
|
|
| 678 | `risk budget remaining` | policy_rule | budget tracking |
|
|
| 679 | `policy violation list` | policy_rule | violation tracking |
|
|
| 680 | `why was release blocked` | policy_rule | decision audit |
|
|
| 681 | `policy decision audit trail` | policy_rule | decision log |
|
|
| 682 | `effective policy for artifact` | policy_rule | computed policy |
|
|
| 683 | `policy merge preview` | policy_rule | merge simulation |
|
|
| 684 | `policy conflict detection` | policy_rule | conflict analysis |
|
|
| 685 | `policy determinism verification` | policy_rule | determinism check |
|
|
| 686 | `policy lint check` | policy_rule | lint validation |
|
|
| 687 | `policy compilation` | policy_rule | compile pipeline |
|
|
| 688 | `sealed mode policy` | policy_rule | air gap mode |
|
|
| 689 | `staleness rule configuration` | policy_rule | staleness config |
|
|
| 690 | `risk profile definition` | policy_rule | risk profile |
|
|
|
|
### 6.2 Policy Evaluation & Decisioning (20 cases)
|
|
|
|
| # | Query | Expected Entity Type | Expected Match Source |
|
|
|---|-------|---------------------|----------------------|
|
|
| 691 | `evaluate policy for container` | policy_rule | batch evaluation |
|
|
| 692 | `policy APPROVE decision` | policy_rule | decision=APPROVE |
|
|
| 693 | `policy REJECT decision` | policy_rule | decision=REJECT |
|
|
| 694 | `conditional approval` | policy_rule | decision=CONDITIONAL |
|
|
| 695 | `blocked by policy` | policy_rule | decision=BLOCKED |
|
|
| 696 | `awaiting approval` | policy_rule | decision=AWAITING |
|
|
| 697 | `override policy violation` | policy_rule | override endpoint |
|
|
| 698 | `severity fusion scoring` | policy_rule | severity fusion |
|
|
| 699 | `CVSS receipt for finding` | policy_rule | CVSS scoring |
|
|
| 700 | `attestation report for release` | policy_rule | attestation report |
|
|
| 701 | `promotion gate evaluation` | policy_rule | gate check |
|
|
| 702 | `batch policy assessment` | policy_rule | batch evaluation |
|
|
| 703 | `policy snapshot comparison` | policy_rule | snapshot diff |
|
|
| 704 | `risk budget consumption` | policy_rule | budget tracking |
|
|
| 705 | `unknowns budget exceeded` | policy_rule | unknowns tracking |
|
|
| 706 | `confidence score low` | policy_rule | confidence scoring |
|
|
| 707 | `evidence freshness expired` | policy_rule | staleness check |
|
|
| 708 | `trust weight configuration` | policy_rule | trust weighting |
|
|
| 709 | `overlay simulation results` | policy_rule | overlay sim |
|
|
| 710 | `path scope simulation` | policy_rule | path scoping |
|
|
|
|
---
|
|
|
|
## Domain 7: Cross-Domain Natural Language Queries (290 cases)
|
|
|
|
### 7.1 Troubleshooting Queries (50 cases)
|
|
|
|
| # | Query | Expected Entity Type | Expected Match Source |
|
|
|---|-------|---------------------|----------------------|
|
|
| 711 | `why is the build failing` | mixed | doctor + findings |
|
|
| 712 | `scan is stuck` | doctor + api | scanner queue check |
|
|
| 713 | `cannot connect to database` | doctor | check.postgres.connectivity |
|
|
| 714 | `authentication failed` | doctor | check.auth.config |
|
|
| 715 | `token expired` | doctor | check.auth.token-service |
|
|
| 716 | `certificate invalid` | doctor | check.crypto.certchain |
|
|
| 717 | `signing failed` | doctor | check.attestation.keymaterial |
|
|
| 718 | `evidence not found` | doctor | check.evidencelocker.retrieval |
|
|
| 719 | `notification not delivered` | doctor | check.notify.queue.health |
|
|
| 720 | `release promotion failed` | doctor | check.release.promotion.gates |
|
|
| 721 | `agent not responding` | doctor | check.agent.heartbeat.freshness |
|
|
| 722 | `out of disk space` | doctor | check.storage.diskspace |
|
|
| 723 | `policy evaluation timeout` | doctor | check.policy.engine |
|
|
| 724 | `reachability analysis slow` | doctor | check.scanner.reachability |
|
|
| 725 | `VEX validation failed` | doctor | check.vex.validation |
|
|
| 726 | `email notification not working` | doctor | check.notify.email.connectivity |
|
|
| 727 | `Slack integration broken` | doctor | check.notify.slack.connectivity |
|
|
| 728 | `environment drift detected` | doctor | check.environment.drift |
|
|
| 729 | `clock skew error` | doctor | check.attestation.clock.skew |
|
|
| 730 | `HSM not available` | doctor | check.crypto.hsm |
|
|
| 731 | `debug scan failure` | docs + doctor | troubleshooting |
|
|
| 732 | `fix deployment error` | docs | runbooks |
|
|
| 733 | `container crash investigation` | docs | troubleshooting |
|
|
| 734 | `error 403 forbidden` | docs + api | auth scopes |
|
|
| 735 | `error 404 not found` | docs + api | endpoint reference |
|
|
| 736 | `error 500 internal server` | docs | troubleshooting |
|
|
| 737 | `connection refused` | doctor | connectivity checks |
|
|
| 738 | `timeout error` | docs | timeout configuration |
|
|
| 739 | `memory leak` | docs | performance troubleshooting |
|
|
| 740 | `high CPU usage` | doctor | check.agent.resource.utilization |
|
|
| 741 | `slow query performance` | docs | database tuning |
|
|
| 742 | `migration failed` | doctor | check.postgres.migrations |
|
|
| 743 | `index corruption` | doctor | check.evidencelocker.index |
|
|
| 744 | `merkle tree inconsistency` | doctor | check.evidencelocker.merkle |
|
|
| 745 | `provenance chain broken` | doctor | check.evidencelocker.provenance |
|
|
| 746 | `agent task failure rate high` | doctor | check.agent.task.failure.rate |
|
|
| 747 | `quorum lost` | doctor | check.agent.cluster.quorum |
|
|
| 748 | `rollback not working` | doctor | check.release.rollback.readiness |
|
|
| 749 | `export failed` | doctor | check.compliance.export-readiness |
|
|
| 750 | `compliance audit failure` | doctor | check.compliance.audit-readiness |
|
|
| 751 | `evidence tampering detected` | doctor | check.compliance.evidence-integrity |
|
|
| 752 | `no evidence generated` | doctor | check.compliance.evidence-rate |
|
|
| 753 | `symbol recovery failed` | doctor | check.binaryanalysis.symbol.recovery.fallback |
|
|
| 754 | `debuginfod unavailable` | doctor | check.binaryanalysis.debuginfod.available |
|
|
| 755 | `TSA endpoint slow` | doctor | check.timestamp.tsa.response-time |
|
|
| 756 | `timestamp validation failed` | doctor | check.timestamp.tsa.valid-response |
|
|
| 757 | `secret detected in code` | finding | secret detection |
|
|
| 758 | `credentials in repository` | finding | secret detection |
|
|
| 759 | `API key leaked` | finding | secret detection |
|
|
| 760 | `hardcoded password` | finding | secret detection |
|
|
|
|
### 7.2 How-To & Workflow Queries (50 cases)
|
|
|
|
| # | Query | Expected Entity Type | Expected Match Source |
|
|
|---|-------|---------------------|----------------------|
|
|
| 761 | `how to scan a container` | docs + api | scanner docs |
|
|
| 762 | `how to create a release` | docs + api | release docs |
|
|
| 763 | `how to promote to production` | docs | release orchestration |
|
|
| 764 | `how to triage a finding` | docs | triage workflow |
|
|
| 765 | `how to suppress a vulnerability` | docs | triage suppress |
|
|
| 766 | `how to generate SBOM` | docs + api | scanner SBOM |
|
|
| 767 | `how to write a VEX statement` | docs | VEX guide |
|
|
| 768 | `how to configure notifications` | docs | notify setup |
|
|
| 769 | `how to set up policy gates` | docs | policy gates |
|
|
| 770 | `how to configure risk budget` | docs | risk budget |
|
|
| 771 | `how to export evidence` | docs + api | export center |
|
|
| 772 | `how to verify attestation` | docs + api | attestor |
|
|
| 773 | `how to configure air gap mode` | docs | offline kit |
|
|
| 774 | `how to rotate signing keys` | docs | key rotation |
|
|
| 775 | `how to onboard new environment` | docs | environment setup |
|
|
| 776 | `how to register agent` | docs | agent onboarding |
|
|
| 777 | `how to integrate GitHub` | docs | integration guide |
|
|
| 778 | `how to configure OIDC` | docs | auth setup |
|
|
| 779 | `how to set up monitoring` | docs | observability |
|
|
| 780 | `how to run doctor checks` | docs + doctor | stella doctor |
|
|
| 781 | `how to create policy exception` | docs | exception workflow |
|
|
| 782 | `how to handle policy violation` | docs | violation handling |
|
|
| 783 | `how to investigate reachability` | docs | reachability guide |
|
|
| 784 | `how to generate call graph` | docs + api | call graph |
|
|
| 785 | `how to compare scans` | docs + api | delta compare |
|
|
| 786 | `how to export SARIF report` | docs + api | SARIF export |
|
|
| 787 | `how to configure Prometheus` | docs | observability |
|
|
| 788 | `how to set up email alerts` | docs | notification config |
|
|
| 789 | `how to configure escalation` | docs | escalation rules |
|
|
| 790 | `how to manage trust anchors` | docs | trust management |
|
|
| 791 | `how to deploy offline` | docs | air gap deployment |
|
|
| 792 | `how to mirror feeds` | docs + api | feed mirror |
|
|
| 793 | `how to verify provenance` | docs + api | provenance |
|
|
| 794 | `how to check compliance` | docs | compliance tracker |
|
|
| 795 | `how to configure secrets` | docs | secrets management |
|
|
| 796 | `how to set up federation` | docs | federation |
|
|
| 797 | `how to use binary diff` | docs | binary diff |
|
|
| 798 | `how to track changes` | docs | change trace |
|
|
| 799 | `how to configure quiet hours` | docs | quiet hours |
|
|
| 800 | `how to set up webhooks` | docs + api | webhook config |
|
|
| 801 | `how to use policy studio` | docs | policy studio |
|
|
| 802 | `how to create risk profile` | docs | risk profile |
|
|
| 803 | `how to run batch evaluation` | docs + api | batch eval |
|
|
| 804 | `how to configure determinism` | docs | determinism |
|
|
| 805 | `how to use sealed mode` | docs | sealed mode |
|
|
| 806 | `how to track unknowns` | docs | unknowns management |
|
|
| 807 | `how to investigate incidents` | docs | incident management |
|
|
| 808 | `how to use advisory AI` | docs | advisory AI |
|
|
| 809 | `how to configure autofix` | docs | remediation |
|
|
| 810 | `how to use evidence ribbon` | docs | evidence UI |
|
|
|
|
### 7.3 Navigation & Feature Discovery (50 cases)
|
|
|
|
| # | Query | Expected Entity Type | Expected Match Source |
|
|
|---|-------|---------------------|----------------------|
|
|
| 811 | `open settings` | docs | navigation |
|
|
| 812 | `go to findings` | docs | navigation |
|
|
| 813 | `show dashboard` | docs | navigation |
|
|
| 814 | `open security view` | docs | navigation |
|
|
| 815 | `go to policy gates` | docs | navigation |
|
|
| 816 | `open VEX hub` | docs | navigation |
|
|
| 817 | `show release history` | docs | navigation |
|
|
| 818 | `open agent fleet` | docs | navigation |
|
|
| 819 | `go to evidence center` | docs | navigation |
|
|
| 820 | `open export center` | docs | navigation |
|
|
| 821 | `show topology view` | docs | navigation |
|
|
| 822 | `open timeline` | docs | navigation |
|
|
| 823 | `go to triage inbox` | docs | navigation |
|
|
| 824 | `open approval queue` | docs | navigation |
|
|
| 825 | `show integrations` | docs | navigation |
|
|
| 826 | `open policy studio` | docs | navigation |
|
|
| 827 | `go to scan results` | docs | navigation |
|
|
| 828 | `open SBOM viewer` | docs | navigation |
|
|
| 829 | `show notifications` | docs | navigation |
|
|
| 830 | `open doctor diagnostics` | docs | navigation |
|
|
| 831 | `where is the audit log` | docs | navigation |
|
|
| 832 | `find the compliance dashboard` | docs | navigation |
|
|
| 833 | `where are risk budgets` | docs | navigation |
|
|
| 834 | `find exception management` | docs | navigation |
|
|
| 835 | `where is the remediation panel` | docs | navigation |
|
|
| 836 | `find the binary diff viewer` | docs | navigation |
|
|
| 837 | `where is the change trace` | docs | navigation |
|
|
| 838 | `find the scoring page` | docs | navigation |
|
|
| 839 | `where is the verdict viewer` | docs | navigation |
|
|
| 840 | `find the proof chain` | docs | navigation |
|
|
| 841 | `open advisory AI chat` | docs | navigation |
|
|
| 842 | `where is the setup wizard` | docs | navigation |
|
|
| 843 | `find the quota dashboard` | docs | navigation |
|
|
| 844 | `where is SLO monitoring` | docs | navigation |
|
|
| 845 | `find dead letter queue` | docs | navigation |
|
|
| 846 | `where is the deploy diff` | docs | navigation |
|
|
| 847 | `find the lineage view` | docs | navigation |
|
|
| 848 | `open mission control` | docs | navigation |
|
|
| 849 | `where is the function map` | docs | navigation |
|
|
| 850 | `find the vulnerability explorer` | docs | navigation |
|
|
| 851 | `open control plane` | docs | navigation |
|
|
| 852 | `show ops memory` | docs | navigation |
|
|
| 853 | `where is trust admin` | docs | navigation |
|
|
| 854 | `find the issuer trust page` | docs | navigation |
|
|
| 855 | `where are workspaces` | docs | navigation |
|
|
| 856 | `open pack registry` | docs | navigation |
|
|
| 857 | `find Trivy DB settings` | docs | navigation |
|
|
| 858 | `where is golden set` | docs | navigation |
|
|
| 859 | `open observations page` | docs | navigation |
|
|
| 860 | `find the signals dashboard` | docs | navigation |
|
|
|
|
### 7.4 CLI Command Searches (50 cases)
|
|
|
|
| # | Query | Expected Entity Type | Expected Match Source |
|
|
|---|-------|---------------------|----------------------|
|
|
| 861 | `stella release create` | docs | CLI reference |
|
|
| 862 | `stella release promote` | docs | CLI reference |
|
|
| 863 | `stella release rollback` | docs | CLI reference |
|
|
| 864 | `stella scan graph` | docs | CLI reference |
|
|
| 865 | `stella policy validate-yaml` | docs | CLI reference |
|
|
| 866 | `stella policy install` | docs | CLI reference |
|
|
| 867 | `stella policy simulate` | docs | CLI reference |
|
|
| 868 | `stella doctor run` | docs + doctor | CLI + checks |
|
|
| 869 | `stella vex generate` | docs | CLI reference |
|
|
| 870 | `stella evidence export` | docs | CLI reference |
|
|
| 871 | `stella attest sign` | docs | CLI reference |
|
|
| 872 | `stella verify` | docs | CLI reference |
|
|
| 873 | `stella config set` | docs | CLI reference |
|
|
| 874 | `stella db migrate` | docs | CLI reference |
|
|
| 875 | `stella export bundle` | docs | CLI reference |
|
|
| 876 | `stella import bundle` | docs | CLI reference |
|
|
| 877 | `stella airgap prepare` | docs | CLI reference |
|
|
| 878 | `stella scan-graph dotnet` | docs | CLI reference |
|
|
| 879 | `stella scan-graph java` | docs | CLI reference |
|
|
| 880 | `stella scan-graph python` | docs | CLI reference |
|
|
| 881 | `stella agent status` | docs | CLI reference |
|
|
| 882 | `stella agent list` | docs | CLI reference |
|
|
| 883 | `stella crypto keygen` | docs | CLI reference |
|
|
| 884 | `stella keys rotate` | docs | CLI reference |
|
|
| 885 | `stella trust-anchors add` | docs | CLI reference |
|
|
| 886 | `stella timestamp verify` | docs | CLI reference |
|
|
| 887 | `stella score calculate` | docs | CLI reference |
|
|
| 888 | `stella verdict check` | docs | CLI reference |
|
|
| 889 | `stella sbom generate` | docs | CLI reference |
|
|
| 890 | `stella seal create` | docs | CLI reference |
|
|
| 891 | `stella witness add` | docs | CLI reference |
|
|
| 892 | `stella proof generate` | docs | CLI reference |
|
|
| 893 | `stella bundle verify` | docs | CLI reference |
|
|
| 894 | `stella notify test` | docs | CLI reference |
|
|
| 895 | `stella feeds sync` | docs | CLI reference |
|
|
| 896 | `stella registry login` | docs | CLI reference |
|
|
| 897 | `stella github connect` | docs | CLI reference |
|
|
| 898 | `stella delta compare` | docs | CLI reference |
|
|
| 899 | `stella binary diff` | docs | CLI reference |
|
|
| 900 | `stella change-trace analyze` | docs | CLI reference |
|
|
| 901 | `stella reachability check` | docs | CLI reference |
|
|
| 902 | `stella drift detect` | docs | CLI reference |
|
|
| 903 | `stella timeline query` | docs | CLI reference |
|
|
| 904 | `stella exception create` | docs | CLI reference |
|
|
| 905 | `stella incidents list` | docs | CLI reference |
|
|
| 906 | `stella signals ingest` | docs | CLI reference |
|
|
| 907 | `stella watchlist add` | docs | CLI reference |
|
|
| 908 | `stella admin config` | docs | CLI reference |
|
|
| 909 | `stella analytics report` | docs | CLI reference |
|
|
| 910 | `stella auth login` | docs | CLI reference |
|
|
|
|
### 7.5 Concept & Explanation Queries (50 cases)
|
|
|
|
| # | Query | Expected Entity Type | Expected Match Source |
|
|
|---|-------|---------------------|----------------------|
|
|
| 911 | `what is a VEX statement` | docs | VEX docs |
|
|
| 912 | `explain SBOM` | docs | SBOM docs |
|
|
| 913 | `what is reachability analysis` | docs | reachability concept |
|
|
| 914 | `explain attestation` | docs | attestation docs |
|
|
| 915 | `what is DSSE envelope` | docs | attestation docs |
|
|
| 916 | `explain in-toto format` | docs | attestation docs |
|
|
| 917 | `what is a policy gate` | docs | policy docs |
|
|
| 918 | `explain risk budget` | docs | policy docs |
|
|
| 919 | `what is severity fusion` | docs | scoring docs |
|
|
| 920 | `explain CVSS v4` | docs + finding | scoring docs |
|
|
| 921 | `what is EPSS` | docs + finding | scoring docs |
|
|
| 922 | `explain decision capsule` | docs | product/decision-capsules.md |
|
|
| 923 | `what is deterministic replay` | docs | replay docs |
|
|
| 924 | `explain provenance` | docs | provenance docs |
|
|
| 925 | `what is a Merkle tree` | docs | evidence locker docs |
|
|
| 926 | `explain evidence chain` | docs | evidence docs |
|
|
| 927 | `what is sealed mode` | docs | sealed mode docs |
|
|
| 928 | `explain air gap operation` | docs | offline docs |
|
|
| 929 | `what is a trust anchor` | docs | security docs |
|
|
| 930 | `explain multi-tenant isolation` | docs | tenant RBAC docs |
|
|
| 931 | `what is content addressable storage` | docs | CAS docs |
|
|
| 932 | `explain smart diff` | docs | smart diff docs |
|
|
| 933 | `what is a linkset` | docs | linkset docs |
|
|
| 934 | `explain canonical SBOM ID` | docs | canonical ID docs |
|
|
| 935 | `what is the findings ledger` | docs | findings docs |
|
|
| 936 | `explain policy determinization` | docs | policy docs |
|
|
| 937 | `what is unknowns budgeting` | docs | unknowns docs |
|
|
| 938 | `explain confidence scoring` | docs | scoring docs |
|
|
| 939 | `what is change trace` | docs | change trace docs |
|
|
| 940 | `explain binary analysis` | docs | binary docs |
|
|
| 941 | `what is the evidence pipeline` | docs | architecture docs |
|
|
| 942 | `explain reciprocal rank fusion` | docs | search docs |
|
|
| 943 | `what is a policy pack` | docs | policy docs |
|
|
| 944 | `explain OCI registry for policy` | docs | policy docs |
|
|
| 945 | `what is a verdict` | docs | verdict docs |
|
|
| 946 | `explain proof spine` | docs | proof docs |
|
|
| 947 | `what is the witness format` | docs | witness docs |
|
|
| 948 | `explain execution evidence` | docs | evidence docs |
|
|
| 949 | `what is a federated consent` | docs | federation docs |
|
|
| 950 | `explain storm breaker` | docs | notification docs |
|
|
| 951 | `what is a dead letter queue` | docs | operations docs |
|
|
| 952 | `explain circuit breaker pattern` | docs | orchestrator docs |
|
|
| 953 | `what is DPoP authentication` | docs | authority docs |
|
|
| 954 | `explain OAuth 2.1` | docs | authority docs |
|
|
| 955 | `what is PURL format` | docs + finding | glossary |
|
|
| 956 | `explain CWE weakness` | docs + finding | glossary |
|
|
| 957 | `what is SAST vs SCA` | docs | scanner docs |
|
|
| 958 | `explain runtime signals` | docs | signals docs |
|
|
| 959 | `what is an advisory source` | docs | concelier docs |
|
|
| 960 | `explain counterfactual analysis` | docs | scanner docs |
|
|
|
|
### 7.6 Comparison & Analysis Queries (40 cases)
|
|
|
|
| # | Query | Expected Entity Type | Expected Match Source |
|
|
|---|-------|---------------------|----------------------|
|
|
| 961 | `compare scan results` | api + docs | DeltaCompareEndpoints |
|
|
| 962 | `difference between VEX and advisory` | docs | VEX guide |
|
|
| 963 | `compare CVSS versions` | docs | scoring docs |
|
|
| 964 | `difference between SBOM and SPDX` | docs | SBOM docs |
|
|
| 965 | `compare policy packs` | api | snapshot comparison |
|
|
| 966 | `difference between Trivy and Stella` | docs | benchmarks |
|
|
| 967 | `compare Snyk scanner features` | docs | benchmarks |
|
|
| 968 | `SAST vs SCA differences` | docs | scanner docs |
|
|
| 969 | `compare environments` | api | environment settings |
|
|
| 970 | `delta between releases` | api | delta compare |
|
|
| 971 | `binary diff between versions` | api + docs | binary diff |
|
|
| 972 | `compare agent versions` | doctor | check.agent.version.consistency |
|
|
| 973 | `compare findings across scans` | api | delta evidence |
|
|
| 974 | `what changed since last scan` | api | change trace |
|
|
| 975 | `new vulnerabilities since yesterday` | finding | date filter |
|
|
| 976 | `resolved vulnerabilities this week` | finding | status filter |
|
|
| 977 | `score difference between environments` | api | score endpoints |
|
|
| 978 | `policy violation trends` | api | analytics |
|
|
| 979 | `risk profile changes` | api | profile events |
|
|
| 980 | `VEX status changes` | vex_statement | timeline |
|
|
| 981 | `evidence freshness comparison` | api | staleness |
|
|
| 982 | `compliance gap analysis` | docs | compliance tracker |
|
|
| 983 | `scanning coverage gaps` | docs | benchmarks |
|
|
| 984 | `trust score comparison` | api | trust weighting |
|
|
| 985 | `notification delivery rate` | api | notification stats |
|
|
| 986 | `scan duration trend` | api | analytics |
|
|
| 987 | `finding resolution velocity` | api | analytics |
|
|
| 988 | `MTTR for vulnerabilities` | api | analytics |
|
|
| 989 | `approval wait time` | api | KPI endpoints |
|
|
| 990 | `deployment frequency` | api | analytics |
|
|
| 991 | `reachability coverage percentage` | api | reachability stats |
|
|
| 992 | `SBOM completeness` | api | SBOM analytics |
|
|
| 993 | `attestation signing latency` | api | performance metrics |
|
|
| 994 | `evidence locker usage` | api | storage stats |
|
|
| 995 | `quota utilization` | api | quota dashboard |
|
|
| 996 | `SLO compliance rate` | api | SLO monitoring |
|
|
| 997 | `agent utilization heatmap` | api | agent analytics |
|
|
| 998 | `vulnerability backlog trend` | api + finding | analytics |
|
|
| 999 | `policy compliance over time` | api | analytics |
|
|
| 1000 | `risk budget burn rate` | api + policy_rule | budget analytics |
|
|
|
|
---
|
|
|
|
## Bonus: Edge Case & Multi-Domain Queries (20 cases)
|
|
|
|
| # | Query | Domains Hit | Description |
|
|
|---|-------|------------|-------------|
|
|
| 1001 | `CVE-2024-21626 runc escape reachability VEX` | finding + vex + docs | Multi-domain: CVE + VEX + docs |
|
|
| 1002 | `log4j affected not_affected VEX` | finding + vex | Finding + conflicting VEX |
|
|
| 1003 | `OPS-001 check failing production` | doctor + docs | Doctor check + environment context |
|
|
| 1004 | `policy violation critical CVE-2024-3094` | policy_rule + finding | Policy + finding cross-ref |
|
|
| 1005 | `how to suppress CVE-2023-44487` | docs + finding + vex | How-to with specific CVE |
|
|
| 1006 | `GHSA-xxxx for pkg:npm/express` | finding | GHSA + PURL combined |
|
|
| 1007 | `promote release with blocked findings` | docs + policy_rule | Workflow + policy gate |
|
|
| 1008 | `attestation failed for container scan` | doctor + docs | Troubleshoot attestation |
|
|
| 1009 | `VEX not_affected but policy still blocks` | vex + policy_rule | Cross-domain conflict |
|
|
| 1010 | `reachability shows vulnerable code not in execute path` | finding + vex + docs | Reachability + VEX justification |
|
|
| 1011 | `export SARIF report for compliance audit` | docs + api | Export + compliance |
|
|
| 1012 | `rotate signing keys in air gap mode` | docs + doctor | Operations + environment |
|
|
| 1013 | `agent cluster quorum lost during release` | doctor + docs | Troubleshoot + release |
|
|
| 1014 | `Slack notification for critical CVE findings` | doctor + docs + finding | Multi-layer search |
|
|
| 1015 | `binary diff shows new dependency vulnerability` | docs + finding | Analysis + finding |
|
|
| 1016 | `federation telemetry from remote tenant` | docs + api | Multi-tenant ops |
|
|
| 1017 | `sealed mode policy with HSM signing` | docs + doctor | Air gap + crypto |
|
|
| 1018 | `CVSS 9.8 EPSS 0.97 exploit known` | finding | Multi-score filter |
|
|
| 1019 | `unknown component in SBOM without VEX` | finding + vex + policy | Unknowns workflow |
|
|
| 1020 | `evidence bundle for in-toto SLSA attestation` | docs + api | Evidence + attestation |
|
|
|
|
---
|
|
|
|
## Domain 3 Extended: Doctor Checks — Timestamping, Integration, Binary & Deep Checks
|
|
|
|
### 3.5 Timestamping & Certificate Lifecycle Checks (20 cases)
|
|
|
|
| # | Query | Expected Entity Type | Expected Match Source |
|
|
|---|-------|---------------------|----------------------|
|
|
| 1021 | `TSA availability check` | doctor | check.timestamp.tsa.availability |
|
|
| 1022 | `TSA response time` | doctor | check.timestamp.tsa.response-time |
|
|
| 1023 | `TSA valid response check` | doctor | check.timestamp.tsa.valid-response |
|
|
| 1024 | `TSA failover ready` | doctor | check.timestamp.tsa.failover-ready |
|
|
| 1025 | `TSA certificate expiry` | doctor | check.timestamp.tsa.certificate-expiry |
|
|
| 1026 | `TSA root expiry check` | doctor | check.timestamp.tsa.root-expiry |
|
|
| 1027 | `TSA chain validation` | doctor | check.timestamp.tsa.chain-valid |
|
|
| 1028 | `OCSP responder check` | doctor | check.timestamp.ocsp.responder |
|
|
| 1029 | `CRL distribution check` | doctor | check.timestamp.crl.distribution |
|
|
| 1030 | `revocation cache freshness` | doctor | check.timestamp.revocation.cache-fresh |
|
|
| 1031 | `OCSP stapling enabled` | doctor | check.timestamp.ocsp.stapling-enabled |
|
|
| 1032 | `evidence staleness check` | doctor | check.timestamp.evidence-staleness |
|
|
| 1033 | `timestamp approaching expiry` | doctor | check.timestamp.tst.approaching-expiry |
|
|
| 1034 | `TST algorithm deprecated` | doctor | check.timestamp.tst.algorithm-deprecated |
|
|
| 1035 | `TST missing stapling` | doctor | check.timestamp.tst.missing-stapling |
|
|
| 1036 | `retimestamp pending` | doctor | check.timestamp.restamp.pending |
|
|
| 1037 | `EU trust list freshness` | doctor | check.timestamp.eu-trust-list-fresh |
|
|
| 1038 | `QTS providers qualified` | doctor | check.timestamp.qts.providers-qualified |
|
|
| 1039 | `QTS status change` | doctor | check.timestamp.qts.status-change |
|
|
| 1040 | `system time synced` | doctor | check.timestamp.system-time-synced |
|
|
|
|
### 3.6 Integration & External Connectivity Checks (25 cases)
|
|
|
|
| # | Query | Expected Entity Type | Expected Match Source |
|
|
|---|-------|---------------------|----------------------|
|
|
| 1041 | `OCI registry connectivity` | doctor | check.integration.oci.registry |
|
|
| 1042 | `OCI referrers API check` | doctor | check.integration.oci.referrers |
|
|
| 1043 | `OCI capability matrix` | doctor | check.integration.oci.capabilities |
|
|
| 1044 | `OCI push authorization` | doctor | check.integration.oci.push |
|
|
| 1045 | `OCI pull authorization` | doctor | check.integration.oci.pull |
|
|
| 1046 | `OCI registry credentials` | doctor | check.integration.oci.credentials |
|
|
| 1047 | `S3 object storage check` | doctor | check.integration.s3.storage |
|
|
| 1048 | `SMTP connectivity check` | doctor | check.integration.smtp |
|
|
| 1049 | `Slack webhook check` | doctor | check.integration.slack |
|
|
| 1050 | `Teams webhook check` | doctor | check.integration.teams |
|
|
| 1051 | `Git provider connectivity` | doctor | check.integration.git |
|
|
| 1052 | `LDAP connectivity check` | doctor | check.integration.ldap |
|
|
| 1053 | `OIDC provider integration check` | doctor | check.integration.oidc |
|
|
| 1054 | `CI system connectivity` | doctor | check.integration.ci.system |
|
|
| 1055 | `secrets manager connectivity` | doctor | check.integration.secrets.manager |
|
|
| 1056 | `integration webhook health` | doctor | check.integration.webhooks |
|
|
| 1057 | `registry push permission denied` | doctor | check.integration.oci.push |
|
|
| 1058 | `cannot pull from OCI registry` | doctor | check.integration.oci.pull |
|
|
| 1059 | `LDAP authentication not working` | doctor | check.integration.ldap |
|
|
| 1060 | `CI pipeline broken connectivity` | doctor | check.integration.ci.system |
|
|
| 1061 | `cannot push policy to OCI` | doctor | check.integration.oci.push |
|
|
| 1062 | `Git provider auth failing` | doctor | check.integration.git |
|
|
| 1063 | `object storage write failing` | doctor | check.integration.s3.storage |
|
|
| 1064 | `secrets vault unreachable` | doctor | check.integration.secrets.manager |
|
|
| 1065 | `integration health dashboard` | doctor | integration checks summary |
|
|
|
|
### 3.7 Binary Analysis & Corpus Health Checks (20 cases)
|
|
|
|
| # | Query | Expected Entity Type | Expected Match Source |
|
|
|---|-------|---------------------|----------------------|
|
|
| 1066 | `debuginfod available` | doctor | check.binaryanalysis.debuginfod.available |
|
|
| 1067 | `ddeb repo enabled` | doctor | check.binaryanalysis.ddeb.enabled |
|
|
| 1068 | `buildinfo cache health` | doctor | check.binaryanalysis.buildinfo.cache |
|
|
| 1069 | `symbol recovery fallback` | doctor | check.binaryanalysis.symbol.recovery.fallback |
|
|
| 1070 | `corpus mirror freshness` | doctor | check.binaryanalysis.corpus.mirror.freshness |
|
|
| 1071 | `corpus KPI baseline exists` | doctor | check.binaryanalysis.corpus.kpi.baseline |
|
|
| 1072 | `binary analysis not working` | doctor | check.binaryanalysis.* |
|
|
| 1073 | `symbol table missing` | doctor | check.binaryanalysis.symbol.recovery.fallback |
|
|
| 1074 | `debug symbols not found` | doctor | check.binaryanalysis.debuginfod.available |
|
|
| 1075 | `buildinfo cache expired` | doctor | check.binaryanalysis.buildinfo.cache |
|
|
| 1076 | `Go binary stripped no debug` | doctor | check.binaryanalysis.* |
|
|
| 1077 | `PE authenticode verification failed` | doctor | binary analysis checks |
|
|
| 1078 | `Mach-O binary inspection failing` | doctor | binary analysis checks |
|
|
| 1079 | `corpus mirror out of date` | doctor | check.binaryanalysis.corpus.mirror.freshness |
|
|
| 1080 | `KPI baseline not established` | doctor | check.binaryanalysis.corpus.kpi.baseline |
|
|
| 1081 | `ddeb repository not configured` | doctor | check.binaryanalysis.ddeb.enabled |
|
|
| 1082 | `native runtime capture failure` | doctor | binary analysis checks |
|
|
| 1083 | `crypto material state check` | doctor | binary crypto analysis |
|
|
| 1084 | `binary vulnerability scan health` | doctor | binary analysis checks |
|
|
| 1085 | `symbol lookup performance degraded` | doctor | check.binaryanalysis.debuginfod.available |
|
|
|
|
### 3.8 Observability, Logging & Operations Deep Checks (15 cases)
|
|
|
|
| # | Query | Expected Entity Type | Expected Match Source |
|
|
|---|-------|---------------------|----------------------|
|
|
| 1086 | `OTLP exporter not sending` | doctor | check.telemetry.otlp.endpoint |
|
|
| 1087 | `log directory not writable` | doctor | check.logs.directory.writable |
|
|
| 1088 | `log rotation not configured` | doctor | check.logs.rotation.configured |
|
|
| 1089 | `Prometheus not scraping metrics` | doctor | check.metrics.prometheus.scrape |
|
|
| 1090 | `dead letter queue growing` | doctor | check.operations.dead-letter |
|
|
| 1091 | `job queue backlog increasing` | doctor | check.operations.job-queue |
|
|
| 1092 | `scheduler not processing` | doctor | check.operations.scheduler |
|
|
| 1093 | `traces not appearing in Jaeger` | doctor | check.telemetry.otlp.endpoint |
|
|
| 1094 | `metrics endpoint 404` | doctor | check.metrics.prometheus.scrape |
|
|
| 1095 | `log files filling disk` | doctor | check.logs.rotation.configured + check.storage.diskspace |
|
|
| 1096 | `OpenTelemetry collector down` | doctor | check.telemetry.otlp.endpoint |
|
|
| 1097 | `dead letter messages accumulating` | doctor | check.operations.dead-letter |
|
|
| 1098 | `cron job scheduler missed run` | doctor | check.operations.scheduler |
|
|
| 1099 | `job retry limit exceeded` | doctor | check.operations.job-queue |
|
|
| 1100 | `observability pipeline health` | doctor | observability checks summary |
|
|
|
|
### 3.9 Scanner, Reachability & Storage Deep Checks (20 cases)
|
|
|
|
| # | Query | Expected Entity Type | Expected Match Source |
|
|
|---|-------|---------------------|----------------------|
|
|
| 1101 | `scanner queue backed up` | doctor | check.scanner.queue |
|
|
| 1102 | `SBOM generation failing` | doctor | check.scanner.sbom |
|
|
| 1103 | `vulnerability scan timing out` | doctor | check.scanner.vuln |
|
|
| 1104 | `witness graph corruption` | doctor | check.scanner.witness.graph |
|
|
| 1105 | `slice cache miss rate high` | doctor | check.scanner.slice.cache |
|
|
| 1106 | `reachability computation stalled` | doctor | check.scanner.reachability |
|
|
| 1107 | `scanner resource utilization high` | doctor | check.scanner.resources |
|
|
| 1108 | `disk space critical on evidence locker` | doctor | check.storage.diskspace |
|
|
| 1109 | `evidence locker write failure` | doctor | check.storage.evidencelocker |
|
|
| 1110 | `backup directory not accessible` | doctor | check.storage.backup |
|
|
| 1111 | `postgres connection pool exhausted` | doctor | check.postgres.pool |
|
|
| 1112 | `database migrations not applied` | doctor | check.postgres.migrations |
|
|
| 1113 | `postgres connectivity lost` | doctor | check.postgres.connectivity |
|
|
| 1114 | `scanner taking too long` | doctor | check.scanner.resources |
|
|
| 1115 | `reachability analysis incomplete` | doctor | check.scanner.reachability |
|
|
| 1116 | `call graph generation failed` | doctor | check.scanner.* |
|
|
| 1117 | `evidence index inconsistent` | doctor | check.evidencelocker.index |
|
|
| 1118 | `merkle tree anchor verification failed` | doctor | check.evidencelocker.merkle |
|
|
| 1119 | `provenance chain incomplete` | doctor | check.evidencelocker.provenance |
|
|
| 1120 | `attestation retrieval timeout` | doctor | check.evidencelocker.retrieval |
|
|
|
|
---
|
|
|
|
## Domain 4 Extended: Findings — Secret Detection, Reachability, Binary & Triage
|
|
|
|
### 4.4 Secret Detection & Credential Findings (25 cases)
|
|
|
|
| # | Query | Expected Entity Type | Expected Match Source |
|
|
|---|-------|---------------------|----------------------|
|
|
| 1121 | `AWS access key exposed` | finding | secret detection - critical |
|
|
| 1122 | `GitHub personal access token` | finding | secret detection - high |
|
|
| 1123 | `private SSH key in repository` | finding | secret detection - critical |
|
|
| 1124 | `database password hardcoded` | finding | secret detection - high |
|
|
| 1125 | `Slack webhook URL leaked` | finding | secret detection - medium |
|
|
| 1126 | `Azure connection string exposed` | finding | secret detection - high |
|
|
| 1127 | `Docker registry credentials` | finding | secret detection - high |
|
|
| 1128 | `JWT secret key in code` | finding | secret detection - critical |
|
|
| 1129 | `Stripe API key leaked` | finding | secret detection - high |
|
|
| 1130 | `Google Cloud service account key` | finding | secret detection - critical |
|
|
| 1131 | `npm auth token` | finding | secret detection - medium |
|
|
| 1132 | `Twilio account SID exposed` | finding | secret detection - medium |
|
|
| 1133 | `SendGrid API key` | finding | secret detection - medium |
|
|
| 1134 | `PKCS#12 certificate with private key` | finding | secret detection - critical |
|
|
| 1135 | `environment file with secrets` | finding | secret detection - high |
|
|
| 1136 | `Terraform state with credentials` | finding | secret detection - critical |
|
|
| 1137 | `Kubernetes secret in YAML` | finding | secret detection - high |
|
|
| 1138 | `PGP private key committed` | finding | secret detection - critical |
|
|
| 1139 | `OAuth client secret exposed` | finding | secret detection - high |
|
|
| 1140 | `Redis AUTH password in config` | finding | secret detection - medium |
|
|
| 1141 | `SMTP credentials in source` | finding | secret detection - medium |
|
|
| 1142 | `encryption key in code` | finding | secret detection - high |
|
|
| 1143 | `API key rotation needed` | finding | secret detection - medium |
|
|
| 1144 | `credential severity critical` | finding | secret detection filter |
|
|
| 1145 | `all secret detections this week` | finding | secret detection date filter |
|
|
|
|
### 4.5 Reachability & Runtime Analysis Findings (25 cases)
|
|
|
|
| # | Query | Expected Entity Type | Expected Match Source |
|
|
|---|-------|---------------------|----------------------|
|
|
| 1146 | `reachable CVE findings` | finding | reachability=Reachable |
|
|
| 1147 | `unreachable vulnerabilities` | finding | reachability=Unreachable |
|
|
| 1148 | `conditional reachability` | finding | reachability=Conditional |
|
|
| 1149 | `unknown reachability status` | finding | reachability=Unknown |
|
|
| 1150 | `static path analysis` | finding | pathEvidence=StaticPath |
|
|
| 1151 | `runtime hit confirmed` | finding | pathEvidence=RuntimeHit |
|
|
| 1152 | `runtime sink hit` | finding | pathEvidence=RuntimeSinkHit |
|
|
| 1153 | `guard condition reduces reachability` | finding | pathEvidence=Guard |
|
|
| 1154 | `mitigation blocks execution` | finding | pathEvidence=Mitigation |
|
|
| 1155 | `static analysis confirmed by runtime` | finding | observationType=Confirmed |
|
|
| 1156 | `runtime only path witness` | finding | observationType=Runtime |
|
|
| 1157 | `static only path no runtime` | finding | observationType=Static |
|
|
| 1158 | `call graph shows reachable function` | finding | reachability evidence |
|
|
| 1159 | `OTel trace confirms vulnerable path` | finding | runtime observation |
|
|
| 1160 | `Tetragon runtime observation` | finding | runtime observation |
|
|
| 1161 | `profiler confirms code execution` | finding | runtime observation |
|
|
| 1162 | `hot symbol detected at runtime` | finding | runtime signal |
|
|
| 1163 | `vulnerable function in execute path` | finding | path analysis |
|
|
| 1164 | `no callstack to vulnerable code` | finding | unreachable path |
|
|
| 1165 | `indirect call graph reachability` | finding | call graph analysis |
|
|
| 1166 | `entry point to sink path` | finding | path analysis |
|
|
| 1167 | `transitive call chain reachable` | finding | transitive analysis |
|
|
| 1168 | `reachability proof document` | finding | evidence type |
|
|
| 1169 | `callstack slice for vulnerability` | finding | evidence type |
|
|
| 1170 | `reachability confidence score` | finding | confidence metric |
|
|
|
|
### 4.6 Binary & Crypto Analysis Findings (25 cases)
|
|
|
|
| # | Query | Expected Entity Type | Expected Match Source |
|
|
|---|-------|---------------------|----------------------|
|
|
| 1171 | `stripped Go binary vulnerability` | finding | binary analysis - Go |
|
|
| 1172 | `Mach-O binary CVE` | finding | binary analysis - macOS |
|
|
| 1173 | `Windows PE vulnerability` | finding | binary analysis - Windows |
|
|
| 1174 | `Authenticode signature invalid` | finding | binary analysis - PE |
|
|
| 1175 | `native library vulnerability` | finding | binary analysis - native |
|
|
| 1176 | `embedded dependency in binary` | finding | binary analysis |
|
|
| 1177 | `statically linked vulnerable code` | finding | binary analysis |
|
|
| 1178 | `shared library CVE` | finding | binary analysis - .so/.dll |
|
|
| 1179 | `musl libc vulnerability` | finding | binary analysis - Alpine |
|
|
| 1180 | `glibc vulnerability` | finding | binary analysis - glibc |
|
|
| 1181 | `crypto material expired` | finding | crypto analysis - expired |
|
|
| 1182 | `weak cipher algorithm detected` | finding | crypto analysis |
|
|
| 1183 | `deprecated TLS version` | finding | crypto analysis |
|
|
| 1184 | `insecure hash function MD5` | finding | crypto analysis |
|
|
| 1185 | `SHA1 deprecation warning` | finding | crypto analysis |
|
|
| 1186 | `RSA key too short` | finding | crypto analysis |
|
|
| 1187 | `self-signed certificate in production` | finding | crypto analysis |
|
|
| 1188 | `certificate about to expire` | finding | crypto analysis |
|
|
| 1189 | `weak random number generator` | finding | crypto analysis |
|
|
| 1190 | `hardcoded IV initialization vector` | finding | crypto analysis |
|
|
| 1191 | `OS package vulnerability alpine` | finding | apk ecosystem |
|
|
| 1192 | `OS package vulnerability debian` | finding | dpkg ecosystem |
|
|
| 1193 | `OS package vulnerability rpm` | finding | rpm ecosystem |
|
|
| 1194 | `homebrew package CVE` | finding | homebrew ecosystem |
|
|
| 1195 | `chocolatey package vulnerability` | finding | chocolatey ecosystem |
|
|
|
|
### 4.7 Triage Workflow & Status Searches (25 cases)
|
|
|
|
| # | Query | Expected Entity Type | Expected Match Source |
|
|
|---|-------|---------------------|----------------------|
|
|
| 1196 | `findings in active triage` | finding | triageLane=Active |
|
|
| 1197 | `blocked shipment findings` | finding | triageLane=Blocked |
|
|
| 1198 | `findings needing exception` | finding | triageLane=NeedsException |
|
|
| 1199 | `muted by reachability` | finding | triageLane=MutedReach |
|
|
| 1200 | `muted by VEX status` | finding | triageLane=MutedVex |
|
|
| 1201 | `compensated findings` | finding | triageLane=Compensated |
|
|
| 1202 | `ship verdict findings` | finding | verdict=Ship |
|
|
| 1203 | `block verdict findings` | finding | verdict=Block |
|
|
| 1204 | `exception granted findings` | finding | verdict=Exception |
|
|
| 1205 | `pending scan results` | finding | scanStatus=Pending |
|
|
| 1206 | `running scans` | finding | scanStatus=Running |
|
|
| 1207 | `failed scan results` | finding | scanStatus=Failed |
|
|
| 1208 | `cancelled scan` | finding | scanStatus=Cancelled |
|
|
| 1209 | `SBOM slice evidence for finding` | finding | evidence=SbomSlice |
|
|
| 1210 | `VEX document evidence` | finding | evidence=VexDoc |
|
|
| 1211 | `provenance evidence for finding` | finding | evidence=Provenance |
|
|
| 1212 | `callstack slice evidence` | finding | evidence=CallstackSlice |
|
|
| 1213 | `replay manifest for finding` | finding | evidence=ReplayManifest |
|
|
| 1214 | `policy evidence attached` | finding | evidence=Policy |
|
|
| 1215 | `scan log evidence` | finding | evidence=ScanLog |
|
|
| 1216 | `findings without evidence` | finding | no evidence attached |
|
|
| 1217 | `unresolved findings older than 30 days` | finding | age filter |
|
|
| 1218 | `findings with no assigned owner` | finding | owner filter |
|
|
| 1219 | `findings blocking production release` | finding | release gate filter |
|
|
| 1220 | `findings requiring manual review` | finding | manual review flag |
|
|
|
|
---
|
|
|
|
## Domain 5 Extended: VEX — Trust, Signatures, Consensus & Conflict
|
|
|
|
### 5.3 VEX Trust, Signature & Freshness Verification (25 cases)
|
|
|
|
| # | Query | Expected Entity Type | Expected Match Source |
|
|
|---|-------|---------------------|----------------------|
|
|
| 1221 | `authoritative VEX source` | vex_statement | trustTier=Authoritative |
|
|
| 1222 | `trusted community VEX` | vex_statement | trustTier=Trusted |
|
|
| 1223 | `untrusted VEX statement` | vex_statement | trustTier=Untrusted |
|
|
| 1224 | `unknown trust tier VEX` | vex_statement | trustTier=Unknown |
|
|
| 1225 | `vendor PSIRT VEX` | vex_statement | issuerCategory=Vendor |
|
|
| 1226 | `distributor VEX statement` | vex_statement | issuerCategory=Distributor |
|
|
| 1227 | `community VEX source` | vex_statement | issuerCategory=Community |
|
|
| 1228 | `internal organization VEX` | vex_statement | issuerCategory=Internal |
|
|
| 1229 | `aggregator VEX source` | vex_statement | issuerCategory=Aggregator |
|
|
| 1230 | `DSSE signed VEX document` | vex_statement | signature=dsse |
|
|
| 1231 | `cosign verified VEX` | vex_statement | signature=cosign |
|
|
| 1232 | `PGP signed VEX statement` | vex_statement | signature=pgp |
|
|
| 1233 | `X.509 signed VEX document` | vex_statement | signature=x509 |
|
|
| 1234 | `unverified VEX signature` | vex_statement | signatureStatus=unverified |
|
|
| 1235 | `failed VEX signature verification` | vex_statement | signatureStatus=failed |
|
|
| 1236 | `VEX freshness stale` | vex_statement | freshness=stale |
|
|
| 1237 | `VEX freshness expired` | vex_statement | freshness=expired |
|
|
| 1238 | `VEX superseded by newer` | vex_statement | freshness=superseded |
|
|
| 1239 | `fresh VEX statements only` | vex_statement | freshness=fresh |
|
|
| 1240 | `VEX with high trust score` | vex_statement | trustScore > 0.8 |
|
|
| 1241 | `VEX from SPDX format` | vex_statement | format=spdx_vex |
|
|
| 1242 | `StellaOps canonical VEX` | vex_statement | format=stellaops |
|
|
| 1243 | `VEX trust vector components` | vex_statement | trust vector detail |
|
|
| 1244 | `VEX issuer reputation` | vex_statement | issuer reputation score |
|
|
| 1245 | `VEX document age over 90 days` | vex_statement | age filter |
|
|
|
|
### 5.4 VEX Consensus, Conflict & Cross-Domain Resolution (25 cases)
|
|
|
|
| # | Query | Expected Entity Type | Expected Match Source |
|
|
|---|-------|---------------------|----------------------|
|
|
| 1246 | `VEX consensus conflict` | vex_statement | conflict resolution |
|
|
| 1247 | `hard conflict between VEX sources` | vex_statement | conflictSeverity=Hard |
|
|
| 1248 | `soft conflict VEX disagreement` | vex_statement | conflictSeverity=Soft |
|
|
| 1249 | `informational VEX conflict` | vex_statement | conflictSeverity=Info |
|
|
| 1250 | `vendor says not_affected community says affected` | vex_statement | cross-source conflict |
|
|
| 1251 | `VEX consensus engine result` | vex_statement | consensus output |
|
|
| 1252 | `trust-weighted VEX merge` | vex_statement | weighted consensus |
|
|
| 1253 | `VEX confidence score low` | vex_statement | confidence < 0.5 |
|
|
| 1254 | `VEX confidence high agreement` | vex_statement | confidence > 0.8 |
|
|
| 1255 | `multiple issuers same CVE` | vex_statement | multi-issuer query |
|
|
| 1256 | `VEX status transition history` | vex_statement | status change events |
|
|
| 1257 | `affected changed to not_affected` | vex_statement | status transition |
|
|
| 1258 | `under_investigation resolved to fixed` | vex_statement | status transition |
|
|
| 1259 | `VEX linked to SBOM component` | vex_statement | product/PURL linkage |
|
|
| 1260 | `VEX for CPE product match` | vex_statement | CPE matching |
|
|
| 1261 | `VEX suppressing active finding` | vex_statement + finding | cross-domain suppression |
|
|
| 1262 | `VEX impact on policy gate` | vex_statement + policy | gate evaluation impact |
|
|
| 1263 | `VEX used as evidence in release` | vex_statement | evidence pipeline |
|
|
| 1264 | `VEX predicate in attestation` | vex_statement | attestation predicate |
|
|
| 1265 | `VEX from feed mirror source` | vex_statement | mirror source |
|
|
| 1266 | `VEX subscription notification` | vex_statement | feed subscription |
|
|
| 1267 | `VEX for production environment only` | vex_statement | environment filter |
|
|
| 1268 | `VEX with action statement required` | vex_statement | actionStatement present |
|
|
| 1269 | `VEX with impact statement detail` | vex_statement | impactStatement present |
|
|
| 1270 | `VEX document schema validation failure` | vex_statement + doctor | schema check |
|
|
|
|
---
|
|
|
|
## Domain 6 Extended: Policy — Gates, Risk Budget, Unknowns & Sealed Mode
|
|
|
|
### 6.3 Gate-Level Evaluation & Verdict Searches (25 cases)
|
|
|
|
| # | Query | Expected Entity Type | Expected Match Source |
|
|
|---|-------|---------------------|----------------------|
|
|
| 1271 | `VEX trust gate evaluation` | policy_rule | VexTrustGate |
|
|
| 1272 | `reachable CVE gate blocked` | policy_rule | ReachableCveGate |
|
|
| 1273 | `execution evidence gate result` | policy_rule | ExecutionEvidenceGate |
|
|
| 1274 | `beacon rate gate threshold` | policy_rule | BeaconRateGate |
|
|
| 1275 | `drift gate unreviewed changes` | policy_rule | DriftGate |
|
|
| 1276 | `unknowns gate budget exceeded` | policy_rule | UnknownsGate |
|
|
| 1277 | `policy verdict pass` | policy_rule | verdictStatus=Pass |
|
|
| 1278 | `policy verdict guarded pass` | policy_rule | verdictStatus=GuardedPass |
|
|
| 1279 | `policy verdict blocked` | policy_rule | verdictStatus=Blocked |
|
|
| 1280 | `policy verdict ignored` | policy_rule | verdictStatus=Ignored |
|
|
| 1281 | `policy verdict warned` | policy_rule | verdictStatus=Warned |
|
|
| 1282 | `policy verdict deferred` | policy_rule | verdictStatus=Deferred |
|
|
| 1283 | `policy verdict escalated` | policy_rule | verdictStatus=Escalated |
|
|
| 1284 | `policy verdict requires VEX` | policy_rule | verdictStatus=RequiresVex |
|
|
| 1285 | `gate result pass with note` | policy_rule | gateResult=PassWithNote |
|
|
| 1286 | `gate result warn` | policy_rule | gateResult=Warn |
|
|
| 1287 | `gate result block` | policy_rule | gateResult=Block |
|
|
| 1288 | `gate result skip` | policy_rule | gateResult=Skip |
|
|
| 1289 | `G0 no-risk gate level` | policy_rule | gateLevel=G0 |
|
|
| 1290 | `G1 low risk gate level` | policy_rule | gateLevel=G1 |
|
|
| 1291 | `G2 moderate risk gate level` | policy_rule | gateLevel=G2 |
|
|
| 1292 | `G3 high risk gate level` | policy_rule | gateLevel=G3 |
|
|
| 1293 | `G4 safety critical gate level` | policy_rule | gateLevel=G4 |
|
|
| 1294 | `policy gate escalation to human review` | policy_rule | escalation |
|
|
| 1295 | `multi-rule conflict resolution` | policy_rule | conflict resolution |
|
|
|
|
### 6.4 Risk Budget, Unknowns, Observation State & Sealed Mode (25 cases)
|
|
|
|
| # | Query | Expected Entity Type | Expected Match Source |
|
|
|---|-------|---------------------|----------------------|
|
|
| 1296 | `risk budget remaining for project` | policy_rule | budget tracking |
|
|
| 1297 | `risk budget burn rate` | policy_rule | budget consumption |
|
|
| 1298 | `unknowns budget exceeded` | policy_rule | unknowns tracking |
|
|
| 1299 | `unknown reachability reason` | policy_rule | U-RCH unknown code |
|
|
| 1300 | `unknown identity ambiguous package` | policy_rule | U-ID unknown code |
|
|
| 1301 | `unknown provenance cannot map binary` | policy_rule | U-PROV unknown code |
|
|
| 1302 | `VEX conflict unknown` | policy_rule | U-VEX unknown code |
|
|
| 1303 | `feed gap unknown source missing` | policy_rule | U-FEED unknown code |
|
|
| 1304 | `config unknown feature not observable` | policy_rule | U-CONFIG unknown code |
|
|
| 1305 | `analyzer limit language not supported` | policy_rule | U-ANALYZER unknown code |
|
|
| 1306 | `observation pending determinization` | policy_rule | state=PendingDeterminization |
|
|
| 1307 | `observation determined` | policy_rule | state=Determined |
|
|
| 1308 | `observation disputed` | policy_rule | state=Disputed |
|
|
| 1309 | `observation stale requires refresh` | policy_rule | state=StaleRequiresRefresh |
|
|
| 1310 | `observation manual review required` | policy_rule | state=ManualReviewRequired |
|
|
| 1311 | `observation suppressed` | policy_rule | state=Suppressed |
|
|
| 1312 | `sealed mode locked dependencies` | policy_rule | sealed mode |
|
|
| 1313 | `sealed mode frozen evidence` | policy_rule | sealed mode |
|
|
| 1314 | `deterministic replay manifest` | policy_rule | replay manifest |
|
|
| 1315 | `no external network during evaluation` | policy_rule | sealed mode constraint |
|
|
| 1316 | `uncertainty tier T1` | policy_rule | uncertaintyTier=T1 |
|
|
| 1317 | `uncertainty tier T2` | policy_rule | uncertaintyTier=T2 |
|
|
| 1318 | `uncertainty tier T3` | policy_rule | uncertaintyTier=T3 |
|
|
| 1319 | `uncertainty tier T4` | policy_rule | uncertaintyTier=T4 |
|
|
| 1320 | `risk verdict attestation DSSE` | policy_rule | attestation evidence |
|
|
|
|
---
|
|
|
|
## Domain 7 Extended: Cross-Domain — Doctor Troubleshooting Deep Dives & Operations
|
|
|
|
### 7.7 Doctor Troubleshooting Deep Dive Queries (50 cases)
|
|
|
|
| # | Query | Expected Entity Type | Expected Match Source |
|
|
|---|-------|---------------------|----------------------|
|
|
| 1321 | `TSA endpoint not responding` | doctor | check.timestamp.tsa.availability |
|
|
| 1322 | `TSA response time degraded` | doctor | check.timestamp.tsa.response-time |
|
|
| 1323 | `TSA certificate about to expire` | doctor | check.timestamp.tsa.certificate-expiry |
|
|
| 1324 | `TSA root CA expiring` | doctor | check.timestamp.tsa.root-expiry |
|
|
| 1325 | `TSA chain validation broken` | doctor | check.timestamp.tsa.chain-valid |
|
|
| 1326 | `OCSP responder unreachable` | doctor | check.timestamp.ocsp.responder |
|
|
| 1327 | `CRL distribution endpoint down` | doctor | check.timestamp.crl.distribution |
|
|
| 1328 | `revocation cache outdated` | doctor | check.timestamp.revocation.cache-fresh |
|
|
| 1329 | `OCSP stapling not configured` | doctor | check.timestamp.ocsp.stapling-enabled |
|
|
| 1330 | `timestamp token approaching expiry` | doctor | check.timestamp.tst.approaching-expiry |
|
|
| 1331 | `deprecated hash algorithm in timestamp` | doctor | check.timestamp.tst.algorithm-deprecated |
|
|
| 1332 | `timestamp missing OCSP stapling` | doctor | check.timestamp.tst.missing-stapling |
|
|
| 1333 | `re-timestamping overdue` | doctor | check.timestamp.restamp.pending |
|
|
| 1334 | `EU trust list not updated` | doctor | check.timestamp.eu-trust-list-fresh |
|
|
| 1335 | `qualified timestamp provider status change` | doctor | check.timestamp.qts.status-change |
|
|
| 1336 | `system clock not synced NTP` | doctor | check.timestamp.system-time-synced |
|
|
| 1337 | `TSA time skew detected` | doctor | check.timestamp.tsa.time-skew |
|
|
| 1338 | `Rekor time correlation drift` | doctor | check.timestamp.rekor.time-correlation |
|
|
| 1339 | `OCI registry health check failing` | doctor | check.integration.oci.registry |
|
|
| 1340 | `OCI referrers API not available` | doctor | check.integration.oci.referrers |
|
|
| 1341 | `registry push denied insufficient permissions` | doctor | check.integration.oci.push |
|
|
| 1342 | `registry credentials expired` | doctor | check.integration.oci.credentials |
|
|
| 1343 | `S3 bucket access denied` | doctor | check.integration.s3.storage |
|
|
| 1344 | `SMTP relay rejected connection` | doctor | check.integration.smtp |
|
|
| 1345 | `Slack API rate limited` | doctor | check.integration.slack |
|
|
| 1346 | `Teams webhook returns 403` | doctor | check.integration.teams |
|
|
| 1347 | `Git provider SSH key rejected` | doctor | check.integration.git |
|
|
| 1348 | `LDAP bind failed wrong credentials` | doctor | check.integration.ldap |
|
|
| 1349 | `CI system Jenkins unreachable` | doctor | check.integration.ci.system |
|
|
| 1350 | `secrets manager Vault sealed` | doctor | check.integration.secrets.manager |
|
|
| 1351 | `agent version mismatch in cluster` | doctor | check.agent.version.consistency |
|
|
| 1352 | `agent certificate expired` | doctor | check.agent.certificate.expiry |
|
|
| 1353 | `agent resource utilization critical` | doctor | check.agent.resource.utilization |
|
|
| 1354 | `agent task failure rate above threshold` | doctor | check.agent.task.failure.rate |
|
|
| 1355 | `stale agent not reporting` | doctor | check.agent.stale |
|
|
| 1356 | `agent capacity exceeded` | doctor | check.agent.capacity |
|
|
| 1357 | `agent task backlog growing` | doctor | check.agent.task.backlog |
|
|
| 1358 | `cluster health degraded` | doctor | check.agent.cluster.health |
|
|
| 1359 | `compliance evidence integrity violation` | doctor | check.compliance.evidence-integrity |
|
|
| 1360 | `provenance chain validation failed` | doctor | check.compliance.provenance-completeness |
|
|
| 1361 | `attestation signing unhealthy` | doctor | check.compliance.attestation-signing |
|
|
| 1362 | `audit readiness check failed` | doctor | check.compliance.audit-readiness |
|
|
| 1363 | `evidence generation rate dropped` | doctor | check.compliance.evidence-rate |
|
|
| 1364 | `export readiness not met` | doctor | check.compliance.export-readiness |
|
|
| 1365 | `compliance framework check warning` | doctor | check.compliance.framework |
|
|
| 1366 | `eIDAS compliance check failing` | doctor | check.crypto.eidas |
|
|
| 1367 | `FIPS module not loaded` | doctor | check.crypto.fips |
|
|
| 1368 | `HSM PKCS#11 module unavailable` | doctor | check.crypto.hsm |
|
|
| 1369 | `GOST crypto provider not found` | doctor | check.crypto.gost |
|
|
| 1370 | `SM2/SM3/SM4 provider missing` | doctor | check.crypto.sm |
|
|
|
|
### 7.8 Operational Workflow & Multi-Domain Queries (50 cases)
|
|
|
|
| # | Query | Expected Entity Type | Expected Match Source |
|
|
|---|-------|---------------------|----------------------|
|
|
| 1371 | `release blocked by reachable CVE and no VEX` | finding + vex + policy | multi-domain gate |
|
|
| 1372 | `how to fix agent certificate expiry` | doctor + docs | agent cert troubleshoot |
|
|
| 1373 | `timestamp infrastructure not ready for eIDAS` | doctor + docs | eIDAS + TSA checks |
|
|
| 1374 | `OCI registry credentials need rotation` | doctor + docs | registry + key management |
|
|
| 1375 | `SBOM incomplete missing Go dependencies` | finding + doctor | SBOM generation + analysis |
|
|
| 1376 | `attestation signing failed HSM timeout` | doctor + docs | HSM + attestation |
|
|
| 1377 | `VEX consensus disagreement blocking release` | vex + policy | consensus + gate |
|
|
| 1378 | `binary analysis found crypto weakness` | finding + doctor | binary + crypto analysis |
|
|
| 1379 | `reachability proves vulnerability not exploitable` | finding + vex | reachability + VEX |
|
|
| 1380 | `environment drift detected after deployment` | doctor + docs | drift + deploy |
|
|
| 1381 | `policy determinism check failed in sealed mode` | policy + doctor | determinism + sealed |
|
|
| 1382 | `evidence locker merkle anchor out of sync` | doctor | merkle + evidence locker |
|
|
| 1383 | `feed mirror stale advisory data 7 days old` | doctor + vex | feed freshness |
|
|
| 1384 | `CI integration broken OIDC token expired` | doctor + docs | CI + auth |
|
|
| 1385 | `dead letter queue messages from scanner` | doctor | DLQ + scanner |
|
|
| 1386 | `scheduler missed nightly scan job` | doctor | scheduler + scanner |
|
|
| 1387 | `agent fleet partial quorum during upgrade` | doctor | agent cluster + version |
|
|
| 1388 | `secrets manager down affecting key rotation` | doctor | secrets + key mgmt |
|
|
| 1389 | `Prometheus not collecting scanner metrics` | doctor | observability + scanner |
|
|
| 1390 | `log rotation full disk scan failures` | doctor | logs + storage + scanner |
|
|
| 1391 | `trust anchor expired blocking attestation` | doctor + docs | trust + attestation |
|
|
| 1392 | `VEX issuer not in directory` | vex + doctor | issuer + trust |
|
|
| 1393 | `policy pack push failed OCI auth` | policy + doctor | policy + OCI |
|
|
| 1394 | `evidence export compliance deadline` | docs + policy | export + compliance |
|
|
| 1395 | `binary vulnerability in base image layer` | finding + docs | binary + container |
|
|
| 1396 | `Go module replace directive hides vulnerability` | finding + docs | Go analysis |
|
|
| 1397 | `transitive dependency critical CVE` | finding | transitive deps |
|
|
| 1398 | `EPSS score suddenly increased` | finding | EPSS score change |
|
|
| 1399 | `runtime signal confirms reachable path` | finding + docs | runtime + reachability |
|
|
| 1400 | `how to write custom doctor check plugin` | docs | doctor plugin SDK |
|
|
| 1401 | `debuginfod not resolving symbols for alpine` | doctor + docs | binary analysis |
|
|
| 1402 | `corpus KPI below baseline threshold` | doctor | KPI baseline |
|
|
| 1403 | `VEX from multiple formats disagree on status` | vex | format conflict |
|
|
| 1404 | `policy override audit trail` | policy | override + audit |
|
|
| 1405 | `risk profile change impacted 100 findings` | policy + finding | profile impact |
|
|
| 1406 | `GuardedPass finding needs beacon verification` | policy + finding | beacon gate |
|
|
| 1407 | `execution evidence not signed` | policy + finding | execution evidence |
|
|
| 1408 | `how to configure TSA failover` | docs + doctor | TSA failover |
|
|
| 1409 | `EU qualified trust service list update` | docs + doctor | eIDAS + QTS |
|
|
| 1410 | `CRL expired and OCSP responder down` | doctor | revocation checks |
|
|
| 1411 | `provenance attestation for container image` | docs + finding | provenance |
|
|
| 1412 | `how to investigate unknown reachability` | docs + finding + policy | unknowns |
|
|
| 1413 | `sealed mode evaluation with frozen evidence` | policy + docs | sealed mode |
|
|
| 1414 | `air gap bundle missing advisory feed` | doctor + docs | air gap + feed |
|
|
| 1415 | `agent certificate renewal automation` | doctor + docs | agent + cert |
|
|
| 1416 | `LDAP group sync not updating permissions` | doctor + docs | LDAP + auth |
|
|
| 1417 | `webhook delivery failure notification gap` | doctor | webhook + notify |
|
|
| 1418 | `scanner resource limits causing OOM` | doctor | scanner + resources |
|
|
| 1419 | `evidence staleness exceeding policy TTL` | doctor + policy | staleness + policy |
|
|
| 1420 | `findings backlog prioritization by EPSS` | finding + docs | EPSS + triage |
|
|
|
|
---
|
|
|
|
## Summary Statistics
|
|
|
|
| Domain | Case Count | Percentage |
|
|
|--------|-----------|------------|
|
|
| Knowledge — Docs | 230 | 16.2% |
|
|
| Knowledge — API Operations | 200 | 14.1% |
|
|
| Knowledge — Doctor Checks | 180 | 12.7% |
|
|
| Findings (Vulnerabilities) | 200 | 14.1% |
|
|
| VEX Statements | 100 | 7.0% |
|
|
| Policy Rules | 100 | 7.0% |
|
|
| Cross-Domain / Natural Language | 410 | 28.9% |
|
|
| **Total** | **1420** | **100%** |
|
|
|
|
### Query Intent Distribution
|
|
|
|
| Intent | Count | Examples |
|
|
|--------|-------|---------|
|
|
| Navigate | ~110 | "open settings", "go to findings" |
|
|
| Troubleshoot | ~200 | "why is build failing", "TSA not responding", "agent expired" |
|
|
| Explore | ~350 | "what is VEX", "explain SBOM", concept lookups |
|
|
| Compare | ~60 | "compare scans", "difference between", "consensus conflict" |
|
|
| How-To | ~120 | "how to create release", "how to triage", "how to configure TSA" |
|
|
| Entity Lookup | ~360 | CVE, PURL, GHSA, check codes, doctor checks, triage status |
|
|
| Multi-Domain | ~220 | Combined queries hitting 2+ domains |
|
|
|
|
### Domain Growth Summary
|
|
|
|
| Domain | Original | Added | New Total | Growth |
|
|
|--------|----------|-------|-----------|--------|
|
|
| Doctor Checks | 80 | +100 | 180 | +125% |
|
|
| Findings | 100 | +100 | 200 | +100% |
|
|
| VEX Statements | 50 | +50 | 100 | +100% |
|
|
| Policy Rules | 50 | +50 | 100 | +100% |
|
|
| Cross-Domain | 310 | +100 | 410 | +32% |
|
|
| Docs | 230 | +0 | 230 | — |
|
|
| API Operations | 200 | +0 | 200 | — |
|