197 lines
4.4 KiB
Markdown
197 lines
4.4 KiB
Markdown
# Runbook: Feed Connector - GitHub Security Advisories (GHSA) Failures
|
|
|
|
> **Sprint:** SPRINT_20260117_029_DOCS_runbook_coverage
|
|
> **Task:** RUN-006 - Feed Connector Runbooks
|
|
|
|
## Metadata
|
|
|
|
| Field | Value |
|
|
|-------|-------|
|
|
| **Component** | Concelier / GHSA Connector |
|
|
| **Severity** | High |
|
|
| **On-call scope** | Platform team |
|
|
| **Last updated** | 2026-01-17 |
|
|
| **Doctor check** | `check.connector.ghsa-health` |
|
|
|
|
---
|
|
|
|
## Symptoms
|
|
|
|
- [ ] GHSA feed sync failing or stale
|
|
- [ ] Alert `ConnectorGhsaSyncFailed` firing
|
|
- [ ] Error: "GitHub API rate limit exceeded" or "GraphQL query failed"
|
|
- [ ] GitHub Advisory Database vulnerabilities missing
|
|
- [ ] Metric `connector_sync_failures_total{source="ghsa"}` increasing
|
|
|
|
---
|
|
|
|
## Impact
|
|
|
|
| Impact Type | Description |
|
|
|-------------|-------------|
|
|
| **User-facing** | GitHub ecosystem vulnerabilities may be missed |
|
|
| **Data integrity** | Data becomes stale; no data loss |
|
|
| **SLA impact** | Vulnerability currency SLO violated for GitHub packages |
|
|
|
|
---
|
|
|
|
## Diagnosis
|
|
|
|
### Quick checks
|
|
|
|
1. **Check Doctor diagnostics:**
|
|
```bash
|
|
stella doctor --check check.connector.ghsa-health
|
|
```
|
|
|
|
2. **Check GHSA sync status:**
|
|
```bash
|
|
stella admin feeds status --source ghsa
|
|
```
|
|
|
|
3. **Test GitHub API connectivity:**
|
|
```bash
|
|
stella connector test ghsa
|
|
```
|
|
|
|
### Deep diagnosis
|
|
|
|
1. **Check GitHub API rate limit:**
|
|
```bash
|
|
stella connector ghsa rate-limit-status
|
|
```
|
|
Problem if: Remaining = 0, rate limit exceeded
|
|
|
|
2. **Check GitHub token permissions:**
|
|
```bash
|
|
stella connector credentials show ghsa --check-scopes
|
|
```
|
|
Required scopes: `public_repo`, `read:packages` (for private advisory access)
|
|
|
|
3. **Check sync logs:**
|
|
```bash
|
|
stella connector logs ghsa --last 1h --level error
|
|
```
|
|
Look for: GraphQL errors, pagination issues, timeout
|
|
|
|
4. **Check for GitHub API outage:**
|
|
```bash
|
|
stella connector ghsa api-status
|
|
```
|
|
Also check: https://www.githubstatus.com/
|
|
|
|
---
|
|
|
|
## Resolution
|
|
|
|
### Immediate mitigation
|
|
|
|
1. **If rate limited, wait for reset:**
|
|
```bash
|
|
stella connector ghsa rate-limit-status
|
|
# Note the reset time, then:
|
|
stella admin feeds refresh --source ghsa
|
|
```
|
|
|
|
2. **Use secondary token if available:**
|
|
```bash
|
|
stella connector credentials rotate ghsa --to secondary
|
|
stella admin feeds refresh --source ghsa
|
|
```
|
|
|
|
3. **Load from offline bundle:**
|
|
```bash
|
|
stella offline load --source ghsa --package ghsa-bundle-latest.tar.gz
|
|
```
|
|
|
|
### Root cause fix
|
|
|
|
**If rate limit consistently exceeded:**
|
|
|
|
1. Increase sync interval:
|
|
```bash
|
|
stella connector config set ghsa.sync_interval 4h
|
|
```
|
|
|
|
2. Enable incremental sync:
|
|
```bash
|
|
stella connector config set ghsa.incremental_sync true
|
|
```
|
|
|
|
3. Use authenticated requests (10x rate limit):
|
|
```bash
|
|
stella connector credentials update ghsa --token <github-pat>
|
|
```
|
|
|
|
**If token expired or invalid:**
|
|
|
|
1. Generate new GitHub PAT at https://github.com/settings/tokens
|
|
|
|
2. Update token:
|
|
```bash
|
|
stella connector credentials update ghsa --token <new-token>
|
|
```
|
|
|
|
3. Verify scopes:
|
|
```bash
|
|
stella connector credentials show ghsa --check-scopes
|
|
```
|
|
|
|
**If GraphQL query failing:**
|
|
|
|
1. Check for API schema changes:
|
|
```bash
|
|
stella connector ghsa schema-check
|
|
```
|
|
|
|
2. Update connector if schema changed:
|
|
```bash
|
|
stella upgrade --component connector-ghsa
|
|
```
|
|
|
|
**If pagination broken:**
|
|
|
|
1. Reset sync cursor:
|
|
```bash
|
|
stella connector ghsa reset-cursor
|
|
```
|
|
|
|
2. Force full resync:
|
|
```bash
|
|
stella admin feeds refresh --source ghsa --full
|
|
```
|
|
|
|
### Verification
|
|
|
|
```bash
|
|
# Force sync
|
|
stella admin feeds refresh --source ghsa
|
|
|
|
# Monitor sync progress
|
|
stella admin feeds status --source ghsa --watch
|
|
|
|
# Verify recent advisories present
|
|
stella vuln query GHSA-xxxx-xxxx-xxxx # Use a recent GHSA ID
|
|
|
|
# Check no errors
|
|
stella connector logs ghsa --level error --last 1h
|
|
```
|
|
|
|
---
|
|
|
|
## Prevention
|
|
|
|
- [ ] **Authentication:** Always use authenticated requests for 5000/hr rate limit
|
|
- [ ] **Monitoring:** Alert on last sync > 12h or sync failures
|
|
- [ ] **Redundancy:** Use NVD/OSV as backup for GitHub ecosystem coverage
|
|
- [ ] **Token rotation:** Rotate tokens before expiration
|
|
|
|
---
|
|
|
|
## Related Resources
|
|
|
|
- **Architecture:** `docs/modules/concelier/connectors.md`
|
|
- **Connector config:** `docs/modules/concelier/operations/connectors/ghsa.md`
|
|
- **Related runbooks:** `connector-nvd.md`, `connector-osv.md`
|
|
- **GitHub API docs:** https://docs.github.com/en/graphql
|