synergy moats product advisory implementations
This commit is contained in:
195
docs/operations/runbooks/attestor-verification-failed.md
Normal file
195
docs/operations/runbooks/attestor-verification-failed.md
Normal file
@@ -0,0 +1,195 @@
|
||||
# Runbook: Attestor - Attestation Verification Failures
|
||||
|
||||
> **Sprint:** SPRINT_20260117_029_DOCS_runbook_coverage
|
||||
> **Task:** RUN-005 - Attestor Runbooks
|
||||
|
||||
## Metadata
|
||||
|
||||
| Field | Value |
|
||||
|-------|-------|
|
||||
| **Component** | Attestor |
|
||||
| **Severity** | High |
|
||||
| **On-call scope** | Platform team, Security team |
|
||||
| **Last updated** | 2026-01-17 |
|
||||
| **Doctor check** | `check.attestor.verification-health` |
|
||||
|
||||
---
|
||||
|
||||
## Symptoms
|
||||
|
||||
- [ ] Attestation verification failing
|
||||
- [ ] Alert `AttestorVerificationFailed` firing
|
||||
- [ ] Error: "signature verification failed" or "invalid attestation"
|
||||
- [ ] Promotions blocked due to failed verification
|
||||
- [ ] Error: "trust anchor not found" or "certificate chain invalid"
|
||||
|
||||
---
|
||||
|
||||
## Impact
|
||||
|
||||
| Impact Type | Description |
|
||||
|-------------|-------------|
|
||||
| **User-facing** | Artifacts cannot be promoted; release blocked |
|
||||
| **Data integrity** | May indicate tampered attestation or configuration issue |
|
||||
| **SLA impact** | Release pipeline blocked until resolved |
|
||||
|
||||
---
|
||||
|
||||
## Diagnosis
|
||||
|
||||
### Quick checks
|
||||
|
||||
1. **Check Doctor diagnostics:**
|
||||
```bash
|
||||
stella doctor --check check.attestor.verification-health
|
||||
```
|
||||
|
||||
2. **Verify specific attestation:**
|
||||
```bash
|
||||
stella verify attestation --attestation <attestation-id> --verbose
|
||||
```
|
||||
|
||||
3. **Check trust anchors:**
|
||||
```bash
|
||||
stella trust-anchors list
|
||||
```
|
||||
|
||||
### Deep diagnosis
|
||||
|
||||
1. **Check attestation details:**
|
||||
```bash
|
||||
stella attest show <attestation-id> --details
|
||||
```
|
||||
Look for: Signer identity, timestamp, subject
|
||||
|
||||
2. **Verify certificate chain:**
|
||||
```bash
|
||||
stella verify cert-chain --attestation <attestation-id>
|
||||
```
|
||||
Problem if: Intermediate cert missing, root not trusted
|
||||
|
||||
3. **Check public key availability:**
|
||||
```bash
|
||||
stella keys show <key-id> --public
|
||||
```
|
||||
|
||||
4. **Check if issuer is trusted:**
|
||||
```bash
|
||||
stella issuer trust-status <issuer-id>
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Resolution
|
||||
|
||||
### Immediate mitigation
|
||||
|
||||
1. **If trust anchor missing, add it:**
|
||||
```bash
|
||||
stella trust-anchors add --cert <issuer-cert.pem>
|
||||
```
|
||||
|
||||
2. **If intermediate cert missing:**
|
||||
```bash
|
||||
stella trust-anchors add-intermediate --cert <intermediate.pem>
|
||||
```
|
||||
|
||||
3. **Re-verify with verbose output:**
|
||||
```bash
|
||||
stella verify attestation --attestation <attestation-id> --verbose
|
||||
```
|
||||
|
||||
### Root cause fix
|
||||
|
||||
**If signature mismatch:**
|
||||
|
||||
1. Check attestation wasn't modified:
|
||||
```bash
|
||||
stella attest integrity-check <attestation-id>
|
||||
```
|
||||
|
||||
2. If modified, regenerate attestation:
|
||||
```bash
|
||||
stella attest create --subject <digest> --type <type> --force
|
||||
```
|
||||
|
||||
**If key rotated and old key not trusted:**
|
||||
|
||||
1. Add old public key to trust anchors:
|
||||
```bash
|
||||
stella trust-anchors add-key --key <old-key.pem> --expires <date>
|
||||
```
|
||||
|
||||
2. Or fetch from issuer directory:
|
||||
```bash
|
||||
stella issuer keys fetch <issuer-id>
|
||||
```
|
||||
|
||||
**If certificate expired:**
|
||||
|
||||
1. Check certificate validity:
|
||||
```bash
|
||||
stella verify cert --attestation <attestation-id> --show-expiry
|
||||
```
|
||||
|
||||
2. Re-sign with valid certificate:
|
||||
```bash
|
||||
stella attest resign <attestation-id>
|
||||
```
|
||||
|
||||
**If issuer not trusted:**
|
||||
|
||||
1. Verify issuer identity:
|
||||
```bash
|
||||
stella issuer show <issuer-id>
|
||||
```
|
||||
|
||||
2. Add to trusted issuers (requires approval):
|
||||
```bash
|
||||
stella issuer trust <issuer-id> --reason "Approved by security team"
|
||||
```
|
||||
|
||||
**If algorithm not supported:**
|
||||
|
||||
1. Check algorithm:
|
||||
```bash
|
||||
stella attest show <attestation-id> | grep algorithm
|
||||
```
|
||||
|
||||
2. Verify crypto provider supports algorithm:
|
||||
```bash
|
||||
stella crypto providers list --algorithms
|
||||
```
|
||||
|
||||
### Verification
|
||||
|
||||
```bash
|
||||
# Verify attestation
|
||||
stella verify attestation --attestation <attestation-id>
|
||||
|
||||
# Verify trust chain
|
||||
stella verify cert-chain --attestation <attestation-id>
|
||||
|
||||
# Test end-to-end verification
|
||||
stella verify artifact --digest <digest>
|
||||
|
||||
# Check no verification errors
|
||||
stella attest logs --filter "verification" --level error --last 30m
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Prevention
|
||||
|
||||
- [ ] **Trust anchors:** Keep trust anchor list current with all valid issuer certs
|
||||
- [ ] **Key rotation:** Plan key rotation with overlap period for verification continuity
|
||||
- [ ] **Monitoring:** Alert on verification failure rate > 0
|
||||
- [ ] **Testing:** Include verification tests in release pipeline
|
||||
|
||||
---
|
||||
|
||||
## Related Resources
|
||||
|
||||
- **Architecture:** `docs/modules/attestor/verification.md`
|
||||
- **Related runbooks:** `attestor-signing-failed.md`, `attestor-key-expired.md`
|
||||
- **Trust management:** `docs/operations/trust-anchors.md`
|
||||
Reference in New Issue
Block a user