Doctor plugin checks: implement health check classes and documentation

Implement remediation-aware health checks across all Doctor plugin modules
(Agent, Attestor, Auth, BinaryAnalysis, Compliance, Crypto, Environment,
EvidenceLocker, Notify, Observability, Operations, Policy, Postgres, Release,
Scanner, Storage, Vex) and their backing library counterparts (AI, Attestation,
Authority, Core, Cryptography, Database, Docker, Integration, Notify,
Observability, Security, ServiceGraph, Sources, Verification).

Each check now emits structured remediation metadata (severity, category,
runbook links, and fix suggestions) consumed by the Doctor dashboard
remediation panel.

Also adds:
- docs/doctor/articles/ knowledge base for check explanations
- Advisory AI search seed and allowlist updates for doctor content
- Sprint plan for doctor checks documentation

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
master
2026-03-27 12:28:00 +02:00
parent fbd24e71de
commit c58a236d70
326 changed files with 18500 additions and 463 deletions

View File

@@ -0,0 +1,75 @@
---
checkId: check.binaryanalysis.corpus.kpi.baseline
plugin: stellaops.doctor.binaryanalysis
severity: warn
tags: [binaryanalysis, corpus, kpi, baseline, regression, ci, groundtruth, security]
---
# KPI Baseline Configuration
## What It Checks
Verifies that a KPI baseline file exists for regression detection in CI gates. The check:
- Looks for a baseline file at the configured directory (default `/var/lib/stella/baselines`) and filename (default `current.json`), configurable via `BinaryAnalysis:Corpus:BaselineDirectory` and `BinaryAnalysis:Corpus:BaselineFilename`.
- If the directory does not exist, warns.
- If the default baseline file is missing but other `.json` files exist in the directory, warns and identifies the latest one.
- Validates the baseline file as JSON and checks for expected KPI fields: `precision`, `recall`, `falseNegativeRate`, `deterministicReplayRate`, `ttfrpP95Ms`.
- Fails if the file exists but is invalid JSON or has no recognized KPI fields.
- Warns if some recommended fields are missing.
## Why It Matters
Without a KPI baseline, CI gates cannot detect regressions in binary matching accuracy. A regression in precision or recall means vulnerability detection quality has degraded without anyone being alerted. The baseline enables automated quality gates that block releases when binary analysis accuracy drops.
## Common Causes
- KPI baseline has never been established (first run of corpus validation not yet completed)
- Baseline directory path misconfigured
- Baseline file was deleted or corrupted
- Baseline created with an older tool version missing newer KPI fields
## How to Fix
### Docker Compose
```bash
# Create baseline directory
docker exec <binaryindex-container> mkdir -p /var/lib/stella/baselines
# Run corpus validation to establish baseline
docker exec <binaryindex-container> stella groundtruth validate run \
--corpus datasets/golden-corpus/seed/ --output-baseline
```
### Bare Metal / systemd
```bash
sudo mkdir -p /var/lib/stella/baselines
# Run validation and save baseline
stella groundtruth validate run \
--corpus datasets/golden-corpus/seed/ \
--output /var/lib/stella/baselines/current.json
# Or promote latest results
stella groundtruth baseline update --from-latest \
--output /var/lib/stella/baselines/current.json
```
### Kubernetes / Helm
```yaml
binaryAnalysis:
corpus:
baselineDirectory: "/var/lib/stella/baselines"
persistence:
enabled: true
```
Run a one-time job to establish the baseline:
```bash
kubectl exec -it <binaryindex-pod> -- stella groundtruth validate run --output-baseline
```
## Verification
```
stella doctor run --check check.binaryanalysis.corpus.kpi.baseline
```
## Related Checks
- `check.binaryanalysis.corpus.mirror.freshness` — verifies corpus mirror data is not stale
- `check.binaryanalysis.symbol.recovery.fallback` — meta-check for symbol recovery availability