189 lines
4.3 KiB
Markdown
189 lines
4.3 KiB
Markdown
# Runbook: Scanner - SBOM Generation Failures
|
|
|
|
> **Sprint:** SPRINT_20260117_029_DOCS_runbook_coverage
|
|
> **Task:** RUN-002 - Scanner Runbooks
|
|
|
|
## Metadata
|
|
|
|
| Field | Value |
|
|
|-------|-------|
|
|
| **Component** | Scanner |
|
|
| **Severity** | High |
|
|
| **On-call scope** | Platform team |
|
|
| **Last updated** | 2026-01-17 |
|
|
| **Doctor check** | `check.scanner.sbom-generation` |
|
|
|
|
---
|
|
|
|
## Symptoms
|
|
|
|
- [ ] Scans completing but SBOM generation failing
|
|
- [ ] Alert `ScannerSbomGenerationFailed` firing
|
|
- [ ] Error: "SBOM generation failed" or "unsupported package format"
|
|
- [ ] Partial SBOM with missing components
|
|
- [ ] Metric `scanner_sbom_generation_failures_total` increasing
|
|
|
|
---
|
|
|
|
## Impact
|
|
|
|
| Impact Type | Description |
|
|
|-------------|-------------|
|
|
| **User-facing** | Incomplete vulnerability coverage; missing dependencies not scanned |
|
|
| **Data integrity** | Partial SBOM may miss vulnerabilities; attestations incomplete |
|
|
| **SLA impact** | SBOM completeness SLO violated (target: > 95%) |
|
|
|
|
---
|
|
|
|
## Diagnosis
|
|
|
|
### Quick checks
|
|
|
|
1. **Check Doctor diagnostics:**
|
|
```bash
|
|
stella doctor --check check.scanner.sbom-generation
|
|
```
|
|
|
|
2. **Check failed SBOM jobs:**
|
|
```bash
|
|
stella scanner jobs list --status sbom_failed --last 1h
|
|
```
|
|
|
|
3. **Check SBOM completeness rate:**
|
|
```bash
|
|
stella scanner stats --sbom-metrics
|
|
```
|
|
|
|
### Deep diagnosis
|
|
|
|
1. **Analyze specific failure:**
|
|
```bash
|
|
stella scanner job details <job-id> --sbom-errors
|
|
```
|
|
Look for: Specific package manager or file type causing failure
|
|
|
|
2. **Check for unsupported ecosystems:**
|
|
```bash
|
|
stella sbom analyze --image <image-ref> --verbose
|
|
```
|
|
Look for: "unsupported", "unknown package format", "parsing failed"
|
|
|
|
3. **Check scanner plugin status:**
|
|
```bash
|
|
stella scanner plugins list --status
|
|
```
|
|
Problem if: Package manager plugin disabled or erroring
|
|
|
|
4. **Check for corrupted package files:**
|
|
```bash
|
|
stella image inspect <image-ref> --check-integrity
|
|
```
|
|
|
|
---
|
|
|
|
## Resolution
|
|
|
|
### Immediate mitigation
|
|
|
|
1. **Enable fallback SBOM generation:**
|
|
```bash
|
|
stella scanner config set sbom.fallback_mode true
|
|
stella scan image --image <image-ref> --sbom-fallback
|
|
```
|
|
|
|
2. **Use alternative SBOM generator:**
|
|
```bash
|
|
stella sbom generate --image <image-ref> --generator syft --output sbom.json
|
|
```
|
|
|
|
3. **Generate partial SBOM and continue:**
|
|
```bash
|
|
stella scan image --image <image-ref> --sbom-partial-ok
|
|
```
|
|
|
|
### Root cause fix
|
|
|
|
**If package manager not supported:**
|
|
|
|
1. Check supported package managers:
|
|
```bash
|
|
stella scanner plugins list --type package-manager
|
|
```
|
|
|
|
2. Enable additional plugins:
|
|
```bash
|
|
stella scanner plugins enable <plugin-name>
|
|
```
|
|
|
|
3. For custom package formats, add mapping:
|
|
```bash
|
|
stella scanner config set sbom.custom_mappings.<format> <handler>
|
|
```
|
|
|
|
**If package file corrupted:**
|
|
|
|
1. Identify corrupted files:
|
|
```bash
|
|
stella image layers <image-ref> --verify-packages
|
|
```
|
|
|
|
2. Report to image owner for fix
|
|
|
|
**If memory/resource issue during generation:**
|
|
|
|
1. Increase SBOM generator resources:
|
|
```bash
|
|
stella scanner config set sbom.memory_limit 4Gi
|
|
stella scanner config set sbom.timeout 10m
|
|
```
|
|
|
|
2. Enable streaming mode:
|
|
```bash
|
|
stella scanner config set sbom.streaming_mode true
|
|
```
|
|
|
|
**If plugin crashed:**
|
|
|
|
1. Check plugin logs:
|
|
```bash
|
|
stella scanner plugins logs <plugin-name> --last 30m
|
|
```
|
|
|
|
2. Restart plugin:
|
|
```bash
|
|
stella scanner plugins restart <plugin-name>
|
|
```
|
|
|
|
### Verification
|
|
|
|
```bash
|
|
# Retry SBOM generation
|
|
stella sbom generate --image <image-ref> --output sbom.json
|
|
|
|
# Validate SBOM completeness
|
|
stella sbom validate --file sbom.json --check-completeness
|
|
|
|
# Check component count
|
|
stella sbom stats --file sbom.json
|
|
|
|
# Full scan with SBOM
|
|
stella scan image --image <image-ref>
|
|
```
|
|
|
|
---
|
|
|
|
## Prevention
|
|
|
|
- [ ] **Plugins:** Keep all package manager plugins enabled and updated
|
|
- [ ] **Monitoring:** Alert on SBOM completeness < 90%
|
|
- [ ] **Fallback:** Configure fallback SBOM generator for resilience
|
|
- [ ] **Testing:** Test SBOM generation for new image types before production
|
|
|
|
---
|
|
|
|
## Related Resources
|
|
|
|
- **Architecture:** `docs/modules/scanner/sbom-generation.md`
|
|
- **Related runbooks:** `scanner-oom.md`, `scanner-timeout.md`
|
|
- **SBOM formats:** `docs/formats/sbom-spdx.md`, `docs/formats/sbom-cyclonedx.md`
|