stella-ops.org/git.stella-ops.org
1
0
Fork 0
Code Issues Pull Requests Actions 3 Releases Wiki Activity

Doctor plugin checks: implement health check classes and documentation

Browse Source 
Implement remediation-aware health checks across all Doctor plugin modules
(Agent, Attestor, Auth, BinaryAnalysis, Compliance, Crypto, Environment,
EvidenceLocker, Notify, Observability, Operations, Policy, Postgres, Release,
Scanner, Storage, Vex) and their backing library counterparts (AI, Attestation,
Authority, Core, Cryptography, Database, Docker, Integration, Notify,
Observability, Security, ServiceGraph, Sources, Verification).

Each check now emits structured remediation metadata (severity, category,
runbook links, and fix suggestions) consumed by the Doctor dashboard
remediation panel.

Also adds:
- docs/doctor/articles/ knowledge base for check explanations
- Advisory AI search seed and allowlist updates for doctor content
- Sprint plan for doctor checks documentation

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
master
2026-03-27 12:28:00 +02:00
parent fbd24e71de
commit c58a236d70
326 changed files with 18500 additions and 463 deletions
Download Patch File Download Diff File Expand all files Collapse all files

74
docs/doctor/articles/binary-analysis/buildinfo-cache.md Normal file
View File

@@ -0,0 +1,74 @@
---
checkId: check.binaryanalysis.buildinfo.cache
plugin: stellaops.doctor.binaryanalysis
severity: warn
tags: [binaryanalysis, buildinfo, debian, cache, security]
---
# Debian Buildinfo Cache
## What It Checks
Verifies Debian buildinfo service accessibility and local cache directory configuration. The check:
- Tests HTTPS connectivity to `buildinfos.debian.net` and `reproduce.debian.net` via HEAD requests.
- Checks the local cache directory (default `/var/cache/stella/buildinfo`, configurable via `BinaryAnalysis:BuildinfoCache:Directory`) for existence and writability by creating and deleting a temp file.
- Fails if both services are unreachable AND the cache directory does not exist.
- Warns if services are unreachable but the cache exists (offline mode possible), or if services are reachable but the cache directory is missing or not writable.
## Why It Matters
Buildinfo files from Debian are used for reproducible-build verification. Without access to buildinfo services or a local cache, binary analysis cannot verify whether packages were built reproducibly, degrading supply-chain assurance for Debian-based container images.
## Common Causes
- Firewall blocking HTTPS access to Debian buildinfo services
- Network connectivity issues or DNS resolution failure
- Proxy configuration required but not set
- Cache directory not created
- Insufficient permissions on cache directory
## How to Fix
### Docker Compose
```yaml
environment:
BinaryAnalysis__BuildinfoCache__Directory: "/var/cache/stella/buildinfo"
volumes:
- buildinfo-cache:/var/cache/stella/buildinfo
```
Test connectivity:
```bash
docker exec <binaryindex-container> curl -I https://buildinfos.debian.net
```
### Bare Metal / systemd
```bash
# Create cache directory
sudo mkdir -p /var/cache/stella/buildinfo
sudo chmod 755 /var/cache/stella/buildinfo
# Test connectivity
curl -I https://buildinfos.debian.net
# If behind a proxy
export HTTPS_PROXY=http://proxy.example.com:8080
```
### Kubernetes / Helm
```yaml
binaryAnalysis:
buildinfo:
cacheDirectory: "/var/cache/stella/buildinfo"
persistence:
enabled: true
size: 5Gi
```
For air-gapped environments, pre-populate the buildinfo cache with required files or disable this check.
## Verification
```
stella doctor run --check check.binaryanalysis.buildinfo.cache
```
## Related Checks
- `check.binaryanalysis.symbol.recovery.fallback` — meta-check ensuring at least one symbol recovery path is available
- `check.binaryanalysis.debuginfod.available` — verifies debuginfod service connectivity

67
docs/doctor/articles/binary-analysis/corpus-mirror-freshness.md Normal file
View File

@@ -0,0 +1,67 @@
---
checkId: check.binaryanalysis.corpus.mirror.freshness
plugin: stellaops.doctor.binaryanalysis
severity: warn
tags: [binaryanalysis, corpus, mirrors, freshness, security, groundtruth]
---
# Corpus Mirror Freshness
## What It Checks
Verifies that local corpus mirrors are not stale. The check:
- Reads the mirrors root directory (default `/var/lib/stella/mirrors`, configurable via `BinaryAnalysis:Corpus:MirrorsDirectory`).
- Inspects five known mirror subdirectories: `debian/archive`, `debian/snapshot`, `ubuntu/usn-index`, `alpine/secdb`, and `osv`.
- For each existing mirror, finds the most recent file modification time (sampling up to 1000 files) and compares it against a staleness threshold (default 7 days, configurable via `BinaryAnalysis:Corpus:StalenessThresholdDays`).
- Fails if no mirrors exist or all mirrors are stale. Warns if some mirrors are stale. Reports info if all present mirrors are fresh but optional mirrors are missing.
## Why It Matters
Corpus mirrors provide ground-truth vulnerability and package data for binary analysis. Stale mirrors mean symbol recovery operates on outdated data, leading to missed vulnerabilities and inaccurate matching in security scans.
## Common Causes
- Corpus mirrors have not been initialized
- Mirror sync job has not run recently or is disabled
- Network connectivity issues preventing sync
- Air-gapped setup incomplete (mirrors not pre-populated)
## How to Fix
### Docker Compose
```bash
# Initialize all mirrors
docker exec <binaryindex-container> stella groundtruth mirror sync --all
```
### Bare Metal / systemd
```bash
# Create mirrors directory
sudo mkdir -p /var/lib/stella/mirrors
# Sync all mirrors
stella groundtruth mirror sync --all
# Set up a timer for automatic sync
sudo systemctl enable stella-mirror-sync.timer
sudo systemctl start stella-mirror-sync.timer
```
### Kubernetes / Helm
```yaml
binaryAnalysis:
corpus:
mirrorsDirectory: "/var/lib/stella/mirrors"
syncSchedule: "0 2 * * *" # daily at 2am
persistence:
enabled: true
size: 50Gi
```
For air-gapped environments, transfer pre-populated mirrors from an online system.
## Verification
```
stella doctor run --check check.binaryanalysis.corpus.mirror.freshness
```
## Related Checks
- `check.binaryanalysis.corpus.kpi.baseline` — verifies KPI baseline exists for regression detection
- `check.binaryanalysis.symbol.recovery.fallback` — meta-check for symbol recovery path availability

61
docs/doctor/articles/binary-analysis/ddeb-repo-enabled.md Normal file
View File

@@ -0,0 +1,61 @@
---
checkId: check.binaryanalysis.ddeb.enabled
plugin: stellaops.doctor.binaryanalysis
severity: warn
tags: [binaryanalysis, ddeb, ubuntu, symbols, security]
---
# Ubuntu Ddeb Repository
## What It Checks
Verifies Ubuntu debug symbol repository (ddebs.ubuntu.com) is configured and accessible. The check (Linux only):
- Parses `/etc/apt/sources.list` and `/etc/apt/sources.list.d/*.list` (and `.sources` DEB822 files) for entries containing `ddebs.ubuntu.com`.
- Tests HTTP connectivity to `http://ddebs.ubuntu.com` via a HEAD request.
- Detects the distribution codename from `/etc/lsb-release` or `/etc/os-release`.
- Reports different warnings based on whether the repo is configured, reachable, or both.
- Skips on non-Linux platforms.
## Why It Matters
The Ubuntu ddeb repository provides debug symbol packages (`-dbgsym`) needed for binary analysis of Ubuntu-based container images. Without debug symbols, binary matching accuracy is significantly reduced, weakening vulnerability detection for Ubuntu workloads.
## Common Causes
- Ddeb repository not added to apt sources
- Network connectivity issues preventing access to ddebs.ubuntu.com
- Firewall blocking HTTP access
- Running on a non-Ubuntu Linux distribution
## How to Fix
### Docker Compose
Add ddeb repository inside the binary analysis container:
```bash
docker exec <binaryindex-container> bash -c \
'echo "deb http://ddebs.ubuntu.com $(lsb_release -cs) main restricted universe multiverse" > /etc/apt/sources.list.d/ddebs.list'
docker exec <binaryindex-container> apt-key adv --keyserver keyserver.ubuntu.com \
--recv-keys F2EDC64DC5AEE1F6B9C621F0C8CAB6595FDFF622
docker exec <binaryindex-container> apt update
```
### Bare Metal / systemd
```bash
echo "deb http://ddebs.ubuntu.com $(lsb_release -cs) main restricted universe multiverse" \
| sudo tee /etc/apt/sources.list.d/ddebs.list
sudo apt-key adv --keyserver keyserver.ubuntu.com \
--recv-keys F2EDC64DC5AEE1F6B9C621F0C8CAB6595FDFF622
sudo apt update
```
### Kubernetes / Helm
Include the ddeb repository in your container image's Dockerfile or use an init container to configure it at startup.
For air-gapped environments, set up a local ddeb mirror or use offline symbol packages.
## Verification
```
stella doctor run --check check.binaryanalysis.ddeb.enabled
```
## Related Checks
- `check.binaryanalysis.debuginfod.available` — verifies debuginfod service availability
- `check.binaryanalysis.symbol.recovery.fallback` — meta-check for symbol recovery path availability

71
docs/doctor/articles/binary-analysis/debuginfod-availability.md Normal file
View File

@@ -0,0 +1,71 @@
---
checkId: check.binaryanalysis.debuginfod.available
plugin: stellaops.doctor.binaryanalysis
severity: warn
tags: [binaryanalysis, debuginfod, symbols, security]
---
# Debuginfod Availability
## What It Checks
Verifies DEBUGINFOD_URLS environment variable and debuginfod service connectivity. The check:
- Reads the `DEBUGINFOD_URLS` environment variable (space-separated list of URLs).
- If not set, falls back to the default Fedora debuginfod at `https://debuginfod.fedoraproject.org`.
- Tests HTTP connectivity to each URL via HEAD requests.
- Reports info if DEBUGINFOD_URLS is not set but the default is reachable.
- Warns if some configured URLs are unreachable. Fails if none are reachable.
## Why It Matters
Debuginfod provides on-demand debug information (DWARF, source) for ELF binaries. It is the primary mechanism for symbol recovery in binary analysis. Without a reachable debuginfod endpoint, binary matching accuracy drops significantly, reducing the effectiveness of vulnerability correlation and reachability analysis.
## Common Causes
- `DEBUGINFOD_URLS` environment variable is not set
- Configured debuginfod servers may be down
- Firewall blocking HTTPS access to debuginfod servers
- Proxy configuration required but not set
- DNS resolution failure for debuginfod hostnames
## How to Fix
### Docker Compose
```yaml
environment:
DEBUGINFOD_URLS: "https://debuginfod.fedoraproject.org"
```
Test connectivity:
```bash
docker exec <binaryindex-container> curl -I https://debuginfod.fedoraproject.org
```
### Bare Metal / systemd
```bash
# Set the environment variable
export DEBUGINFOD_URLS="https://debuginfod.fedoraproject.org"
# Or add to service file
sudo systemctl edit stellaops-binaryindex
# Add: Environment=DEBUGINFOD_URLS=https://debuginfod.fedoraproject.org
# Verify connectivity
curl -I https://debuginfod.fedoraproject.org
```
### Kubernetes / Helm
```yaml
binaryAnalysis:
debuginfod:
urls: "https://debuginfod.fedoraproject.org"
```
For air-gapped environments, deploy a local debuginfod instance or use offline symbol bundles. See `docs/modules/binary-index/ground-truth-corpus.md` for offline setup.
## Verification
```
stella doctor run --check check.binaryanalysis.debuginfod.available
```
## Related Checks
- `check.binaryanalysis.ddeb.enabled` — verifies Ubuntu ddeb repository availability
- `check.binaryanalysis.buildinfo.cache` — verifies Debian buildinfo service and cache
- `check.binaryanalysis.symbol.recovery.fallback` — meta-check aggregating all symbol sources

75
docs/doctor/articles/binary-analysis/kpi-baseline-exists.md Normal file
View File

@@ -0,0 +1,75 @@
---
checkId: check.binaryanalysis.corpus.kpi.baseline
plugin: stellaops.doctor.binaryanalysis
severity: warn
tags: [binaryanalysis, corpus, kpi, baseline, regression, ci, groundtruth, security]
---
# KPI Baseline Configuration
## What It Checks
Verifies that a KPI baseline file exists for regression detection in CI gates. The check:
- Looks for a baseline file at the configured directory (default `/var/lib/stella/baselines`) and filename (default `current.json`), configurable via `BinaryAnalysis:Corpus:BaselineDirectory` and `BinaryAnalysis:Corpus:BaselineFilename`.
- If the directory does not exist, warns.
- If the default baseline file is missing but other `.json` files exist in the directory, warns and identifies the latest one.
- Validates the baseline file as JSON and checks for expected KPI fields: `precision`, `recall`, `falseNegativeRate`, `deterministicReplayRate`, `ttfrpP95Ms`.
- Fails if the file exists but is invalid JSON or has no recognized KPI fields.
- Warns if some recommended fields are missing.
## Why It Matters
Without a KPI baseline, CI gates cannot detect regressions in binary matching accuracy. A regression in precision or recall means vulnerability detection quality has degraded without anyone being alerted. The baseline enables automated quality gates that block releases when binary analysis accuracy drops.
## Common Causes
- KPI baseline has never been established (first run of corpus validation not yet completed)
- Baseline directory path misconfigured
- Baseline file was deleted or corrupted
- Baseline created with an older tool version missing newer KPI fields
## How to Fix
### Docker Compose
```bash
# Create baseline directory
docker exec <binaryindex-container> mkdir -p /var/lib/stella/baselines
# Run corpus validation to establish baseline
docker exec <binaryindex-container> stella groundtruth validate run \
--corpus datasets/golden-corpus/seed/ --output-baseline
```
### Bare Metal / systemd
```bash
sudo mkdir -p /var/lib/stella/baselines
# Run validation and save baseline
stella groundtruth validate run \
--corpus datasets/golden-corpus/seed/ \
--output /var/lib/stella/baselines/current.json
# Or promote latest results
stella groundtruth baseline update --from-latest \
--output /var/lib/stella/baselines/current.json
```
### Kubernetes / Helm
```yaml
binaryAnalysis:
corpus:
baselineDirectory: "/var/lib/stella/baselines"
persistence:
enabled: true
```
Run a one-time job to establish the baseline:
```bash
kubectl exec -it <binaryindex-pod> -- stella groundtruth validate run --output-baseline
```
## Verification
```
stella doctor run --check check.binaryanalysis.corpus.kpi.baseline
```
## Related Checks
- `check.binaryanalysis.corpus.mirror.freshness` — verifies corpus mirror data is not stale
- `check.binaryanalysis.symbol.recovery.fallback` — meta-check for symbol recovery availability

69
docs/doctor/articles/binary-analysis/symbol-recovery-fallback.md Normal file
View File

@@ -0,0 +1,69 @@
---
checkId: check.binaryanalysis.symbol.recovery.fallback
plugin: stellaops.doctor.binaryanalysis
severity: warn
tags: [binaryanalysis, symbols, fallback, security, meta]
---
# Symbol Recovery Fallback
## What It Checks
Meta-check that ensures at least one symbol recovery path is available. The check aggregates results from three child checks:
- **Debuginfod Availability** (`check.binaryanalysis.debuginfod.available`)
- **Ubuntu Ddeb Repository** (`check.binaryanalysis.ddeb.enabled`) -- skipped on non-Linux
- **Debian Buildinfo Cache** (`check.binaryanalysis.buildinfo.cache`)
Fails if zero sources are available. Reports info if some but not all sources are available. Passes if all sources are operational.
## Why It Matters
Symbol recovery is critical for binary analysis accuracy. If all symbol sources are unavailable, binary matching operates without debug information, severely degrading vulnerability detection quality. Having at least one source ensures a minimum level of binary analysis capability; having multiple sources provides redundancy.
## Common Causes
- All symbol recovery endpoints unreachable
- Network connectivity issues affecting all sources
- Firewall blocking access to symbol servers
- Air-gapped environment without offline symbol cache configured
## How to Fix
### Docker Compose
Configure at least one symbol source:
```yaml
environment:
DEBUGINFOD_URLS: "https://debuginfod.fedoraproject.org"
BinaryAnalysis__BuildinfoCache__Directory: "/var/cache/stella/buildinfo"
```
### Bare Metal / systemd
```bash
# Option 1: Configure debuginfod
export DEBUGINFOD_URLS="https://debuginfod.fedoraproject.org"
# Option 2: Set up buildinfo cache
sudo mkdir -p /var/cache/stella/buildinfo
# Option 3: Configure ddeb repository (Ubuntu)
echo "deb http://ddebs.ubuntu.com $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/ddebs.list
```
### Kubernetes / Helm
```yaml
binaryAnalysis:
debuginfod:
urls: "https://debuginfod.fedoraproject.org"
buildinfo:
cacheDirectory: "/var/cache/stella/buildinfo"
```
For air-gapped environments, set up an offline symbol bundle. See `docs/modules/binary-index/ground-truth-corpus.md` for instructions on creating and importing offline symbol packs.
## Verification
```
stella doctor run --check check.binaryanalysis.symbol.recovery.fallback
```
## Related Checks
- `check.binaryanalysis.debuginfod.available` — individual debuginfod connectivity check
- `check.binaryanalysis.ddeb.enabled` — individual Ubuntu ddeb repository check
- `check.binaryanalysis.buildinfo.cache` — individual Debian buildinfo cache check