196 lines
4.5 KiB
Markdown
196 lines
4.5 KiB
Markdown
# Runbook: Feed Connector - NVD Connector Failures
|
|
|
|
> **Sprint:** SPRINT_20260117_029_DOCS_runbook_coverage
|
|
> **Task:** RUN-006 - Feed Connector Runbooks
|
|
|
|
## Metadata
|
|
|
|
| Field | Value |
|
|
|-------|-------|
|
|
| **Component** | Concelier / NVD Connector |
|
|
| **Severity** | High |
|
|
| **On-call scope** | Platform team |
|
|
| **Last updated** | 2026-01-17 |
|
|
| **Doctor check** | `check.connector.nvd-health` |
|
|
|
|
---
|
|
|
|
## Symptoms
|
|
|
|
- [ ] NVD feed sync failing or stale (> 24h since last successful sync)
|
|
- [ ] Alert `ConnectorNvdSyncFailed` firing
|
|
- [ ] Error: "NVD API request failed" or "rate limit exceeded"
|
|
- [ ] Vulnerability data missing or outdated
|
|
- [ ] Metric `connector_sync_failures_total{source="nvd"}` increasing
|
|
|
|
---
|
|
|
|
## Impact
|
|
|
|
| Impact Type | Description |
|
|
|-------------|-------------|
|
|
| **User-facing** | Vulnerability scans may miss recent CVEs |
|
|
| **Data integrity** | Data becomes stale; no data loss |
|
|
| **SLA impact** | Vulnerability currency SLO violated (target: < 24h) |
|
|
|
|
---
|
|
|
|
## Diagnosis
|
|
|
|
### Quick checks
|
|
|
|
1. **Check Doctor diagnostics:**
|
|
```bash
|
|
stella doctor --check check.connector.nvd-health
|
|
```
|
|
|
|
2. **Check NVD sync status:**
|
|
```bash
|
|
stella admin feeds status --source nvd
|
|
```
|
|
Look for: Last sync time, error message, sync state
|
|
|
|
3. **Check NVD API connectivity:**
|
|
```bash
|
|
stella connector test nvd
|
|
```
|
|
|
|
### Deep diagnosis
|
|
|
|
1. **Check NVD API key status:**
|
|
```bash
|
|
stella connector credentials show nvd
|
|
```
|
|
Problem if: API key expired or rate limit exhausted
|
|
|
|
2. **Check NVD API rate limit:**
|
|
```bash
|
|
stella connector nvd rate-limit-status
|
|
```
|
|
Problem if: Remaining requests = 0, reset time in future
|
|
|
|
3. **Check for NVD API outage:**
|
|
```bash
|
|
stella connector nvd api-status
|
|
```
|
|
Also check: https://nvd.nist.gov/general/news
|
|
|
|
4. **Check sync logs:**
|
|
```bash
|
|
stella connector logs nvd --last 1h --level error
|
|
```
|
|
Look for: HTTP status codes, timeout errors, parsing failures
|
|
|
|
---
|
|
|
|
## Resolution
|
|
|
|
### Immediate mitigation
|
|
|
|
1. **If rate limited, wait for reset:**
|
|
```bash
|
|
stella connector nvd rate-limit-status
|
|
# Wait for reset time, then:
|
|
stella admin feeds refresh --source nvd
|
|
```
|
|
|
|
2. **If API key expired, use anonymous mode (slower):**
|
|
```bash
|
|
stella connector config set nvd.api_key_mode anonymous
|
|
stella admin feeds refresh --source nvd
|
|
```
|
|
|
|
3. **Load from offline bundle if urgent:**
|
|
```bash
|
|
# If you have a recent offline bundle:
|
|
stella offline load --source nvd --package nvd-bundle-latest.tar.gz
|
|
```
|
|
|
|
### Root cause fix
|
|
|
|
**If API key expired or invalid:**
|
|
|
|
1. Generate new NVD API key at https://nvd.nist.gov/developers/request-an-api-key
|
|
|
|
2. Update API key:
|
|
```bash
|
|
stella connector credentials update nvd --api-key <new-key>
|
|
```
|
|
|
|
3. Verify connectivity:
|
|
```bash
|
|
stella connector test nvd
|
|
```
|
|
|
|
**If rate limit consistently exceeded:**
|
|
|
|
1. Increase sync interval to reduce API calls:
|
|
```bash
|
|
stella connector config set nvd.sync_interval 6h
|
|
```
|
|
|
|
2. Enable delta sync to reduce data volume:
|
|
```bash
|
|
stella connector config set nvd.delta_sync true
|
|
```
|
|
|
|
3. Request higher rate limit from NVD (if available)
|
|
|
|
**If network/firewall issue:**
|
|
|
|
1. Verify outbound connectivity to NVD API:
|
|
```bash
|
|
stella connector test nvd --verbose
|
|
```
|
|
|
|
2. Check proxy configuration if required:
|
|
```bash
|
|
stella connector config set nvd.proxy https://proxy:8080
|
|
```
|
|
|
|
**If data parsing failures:**
|
|
|
|
1. Check for NVD schema changes:
|
|
```bash
|
|
stella connector nvd schema-check
|
|
```
|
|
|
|
2. Update connector if schema changed:
|
|
```bash
|
|
stella upgrade --component connector-nvd
|
|
```
|
|
|
|
### Verification
|
|
|
|
```bash
|
|
# Force sync
|
|
stella admin feeds refresh --source nvd --force
|
|
|
|
# Monitor sync progress
|
|
stella admin feeds status --source nvd --watch
|
|
|
|
# Verify recent CVEs are present
|
|
stella vuln query CVE-2026-XXXX # Use a recent CVE ID
|
|
|
|
# Check no errors in recent logs
|
|
stella connector logs nvd --level error --last 1h
|
|
```
|
|
|
|
---
|
|
|
|
## Prevention
|
|
|
|
- [ ] **API Key:** Always use API key (not anonymous) for 10x rate limit
|
|
- [ ] **Monitoring:** Alert on last sync > 24h or sync failure
|
|
- [ ] **Redundancy:** Configure backup connector (OSV, GitHub Advisory) for overlap
|
|
- [ ] **Offline:** Maintain weekly offline bundle for disaster recovery
|
|
|
|
---
|
|
|
|
## Related Resources
|
|
|
|
- **Architecture:** `docs/modules/concelier/connectors.md`
|
|
- **Connector config:** `docs/modules/concelier/operations/connectors/nvd.md`
|
|
- **Related runbooks:** `connector-ghsa.md`, `connector-osv.md`
|
|
- **Dashboard:** Grafana > Stella Ops > Feed Connectors
|