synergy moats product advisory implementations
This commit is contained in:
195
docs/operations/runbooks/connector-nvd.md
Normal file
195
docs/operations/runbooks/connector-nvd.md
Normal file
@@ -0,0 +1,195 @@
|
||||
# Runbook: Feed Connector - NVD Connector Failures
|
||||
|
||||
> **Sprint:** SPRINT_20260117_029_DOCS_runbook_coverage
|
||||
> **Task:** RUN-006 - Feed Connector Runbooks
|
||||
|
||||
## Metadata
|
||||
|
||||
| Field | Value |
|
||||
|-------|-------|
|
||||
| **Component** | Concelier / NVD Connector |
|
||||
| **Severity** | High |
|
||||
| **On-call scope** | Platform team |
|
||||
| **Last updated** | 2026-01-17 |
|
||||
| **Doctor check** | `check.connector.nvd-health` |
|
||||
|
||||
---
|
||||
|
||||
## Symptoms
|
||||
|
||||
- [ ] NVD feed sync failing or stale (> 24h since last successful sync)
|
||||
- [ ] Alert `ConnectorNvdSyncFailed` firing
|
||||
- [ ] Error: "NVD API request failed" or "rate limit exceeded"
|
||||
- [ ] Vulnerability data missing or outdated
|
||||
- [ ] Metric `connector_sync_failures_total{source="nvd"}` increasing
|
||||
|
||||
---
|
||||
|
||||
## Impact
|
||||
|
||||
| Impact Type | Description |
|
||||
|-------------|-------------|
|
||||
| **User-facing** | Vulnerability scans may miss recent CVEs |
|
||||
| **Data integrity** | Data becomes stale; no data loss |
|
||||
| **SLA impact** | Vulnerability currency SLO violated (target: < 24h) |
|
||||
|
||||
---
|
||||
|
||||
## Diagnosis
|
||||
|
||||
### Quick checks
|
||||
|
||||
1. **Check Doctor diagnostics:**
|
||||
```bash
|
||||
stella doctor --check check.connector.nvd-health
|
||||
```
|
||||
|
||||
2. **Check NVD sync status:**
|
||||
```bash
|
||||
stella admin feeds status --source nvd
|
||||
```
|
||||
Look for: Last sync time, error message, sync state
|
||||
|
||||
3. **Check NVD API connectivity:**
|
||||
```bash
|
||||
stella connector test nvd
|
||||
```
|
||||
|
||||
### Deep diagnosis
|
||||
|
||||
1. **Check NVD API key status:**
|
||||
```bash
|
||||
stella connector credentials show nvd
|
||||
```
|
||||
Problem if: API key expired or rate limit exhausted
|
||||
|
||||
2. **Check NVD API rate limit:**
|
||||
```bash
|
||||
stella connector nvd rate-limit-status
|
||||
```
|
||||
Problem if: Remaining requests = 0, reset time in future
|
||||
|
||||
3. **Check for NVD API outage:**
|
||||
```bash
|
||||
stella connector nvd api-status
|
||||
```
|
||||
Also check: https://nvd.nist.gov/general/news
|
||||
|
||||
4. **Check sync logs:**
|
||||
```bash
|
||||
stella connector logs nvd --last 1h --level error
|
||||
```
|
||||
Look for: HTTP status codes, timeout errors, parsing failures
|
||||
|
||||
---
|
||||
|
||||
## Resolution
|
||||
|
||||
### Immediate mitigation
|
||||
|
||||
1. **If rate limited, wait for reset:**
|
||||
```bash
|
||||
stella connector nvd rate-limit-status
|
||||
# Wait for reset time, then:
|
||||
stella admin feeds refresh --source nvd
|
||||
```
|
||||
|
||||
2. **If API key expired, use anonymous mode (slower):**
|
||||
```bash
|
||||
stella connector config set nvd.api_key_mode anonymous
|
||||
stella admin feeds refresh --source nvd
|
||||
```
|
||||
|
||||
3. **Load from offline bundle if urgent:**
|
||||
```bash
|
||||
# If you have a recent offline bundle:
|
||||
stella offline load --source nvd --package nvd-bundle-latest.tar.gz
|
||||
```
|
||||
|
||||
### Root cause fix
|
||||
|
||||
**If API key expired or invalid:**
|
||||
|
||||
1. Generate new NVD API key at https://nvd.nist.gov/developers/request-an-api-key
|
||||
|
||||
2. Update API key:
|
||||
```bash
|
||||
stella connector credentials update nvd --api-key <new-key>
|
||||
```
|
||||
|
||||
3. Verify connectivity:
|
||||
```bash
|
||||
stella connector test nvd
|
||||
```
|
||||
|
||||
**If rate limit consistently exceeded:**
|
||||
|
||||
1. Increase sync interval to reduce API calls:
|
||||
```bash
|
||||
stella connector config set nvd.sync_interval 6h
|
||||
```
|
||||
|
||||
2. Enable delta sync to reduce data volume:
|
||||
```bash
|
||||
stella connector config set nvd.delta_sync true
|
||||
```
|
||||
|
||||
3. Request higher rate limit from NVD (if available)
|
||||
|
||||
**If network/firewall issue:**
|
||||
|
||||
1. Verify outbound connectivity to NVD API:
|
||||
```bash
|
||||
stella connector test nvd --verbose
|
||||
```
|
||||
|
||||
2. Check proxy configuration if required:
|
||||
```bash
|
||||
stella connector config set nvd.proxy https://proxy:8080
|
||||
```
|
||||
|
||||
**If data parsing failures:**
|
||||
|
||||
1. Check for NVD schema changes:
|
||||
```bash
|
||||
stella connector nvd schema-check
|
||||
```
|
||||
|
||||
2. Update connector if schema changed:
|
||||
```bash
|
||||
stella upgrade --component connector-nvd
|
||||
```
|
||||
|
||||
### Verification
|
||||
|
||||
```bash
|
||||
# Force sync
|
||||
stella admin feeds refresh --source nvd --force
|
||||
|
||||
# Monitor sync progress
|
||||
stella admin feeds status --source nvd --watch
|
||||
|
||||
# Verify recent CVEs are present
|
||||
stella vuln query CVE-2026-XXXX # Use a recent CVE ID
|
||||
|
||||
# Check no errors in recent logs
|
||||
stella connector logs nvd --level error --last 1h
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Prevention
|
||||
|
||||
- [ ] **API Key:** Always use API key (not anonymous) for 10x rate limit
|
||||
- [ ] **Monitoring:** Alert on last sync > 24h or sync failure
|
||||
- [ ] **Redundancy:** Configure backup connector (OSV, GitHub Advisory) for overlap
|
||||
- [ ] **Offline:** Maintain weekly offline bundle for disaster recovery
|
||||
|
||||
---
|
||||
|
||||
## Related Resources
|
||||
|
||||
- **Architecture:** `docs/modules/concelier/connectors.md`
|
||||
- **Connector config:** `docs/modules/concelier/operations/connectors/nvd.md`
|
||||
- **Related runbooks:** `connector-ghsa.md`, `connector-osv.md`
|
||||
- **Dashboard:** Grafana > Stella Ops > Feed Connectors
|
||||
Reference in New Issue
Block a user