stella-ops.org/git.stella-ops.org
1
0
Fork 0
Code Issues Pull Requests Actions 2 Releases Wiki Activity
Files
702a27ac839f1de50f9c9f460000b8f7b2242efd
git.stella-ops.org/docs/operations/runbooks/attestor-rekor-unavailable.md

4.1 KiB
Raw Blame History

Runbook: Attestor - Rekor Transparency Log Unreachable

Sprint: SPRINT_20260117_029_DOCS_runbook_coverage Task: RUN-005 - Attestor Runbooks

Metadata

Field Value
Component Attestor
Severity High
On-call scope Platform team
Last updated 2026-01-17
Doctor check check.attestor.rekor-connectivity

Symptoms

  • Attestation transparency logging failing
  • Alert AttestorRekorUnavailable firing
  • Error: "Rekor server unavailable" or "transparency log submission failed"
  • Attestations created but not anchored to transparency log
  • Verification failing due to missing log entry

Impact

Impact Type Description
User-facing Attestations not publicly verifiable via transparency log
Data integrity Attestations still valid locally; transparency reduced
SLA impact Compliance may require transparency log anchoring

Diagnosis

Quick checks

  1. Check Doctor diagnostics:

    stella doctor --check check.attestor.rekor-connectivity

  2. Check Rekor connectivity:

    stella attest rekor status

  3. Test Rekor endpoint:

    stella attest rekor ping

Deep diagnosis

  1. Check Rekor server URL:

    stella attest config get rekor.url

    Default: https://rekor.sigstore.dev

  2. Check for public Rekor outage:

    stella attest rekor api-status

    Also check: https://status.sigstore.dev/

  3. Check network/proxy issues:

    stella attest rekor test --verbose

    Look for: TLS errors, proxy blocks, timeout

  4. Check pending log entries:

    stella attest rekor pending-entries

Resolution

Immediate mitigation

  1. Queue attestations for later submission:

    stella attest config set rekor.queue_on_failure true
stella attest reload

  2. Disable Rekor requirement temporarily:

    stella attest config set rekor.required false
stella attest reload

    Warning: Reduces transparency guarantees

  3. Use private Rekor instance if available:

    stella attest config set rekor.url https://rekor.internal.example.com
stella attest reload

Root cause fix

If public Rekor outage:

  1. Wait for Sigstore to resolve the issue
  2. Check status at https://status.sigstore.dev/
  3. Process queued entries when service recovers: 
    stella attest rekor process-queue

If network/firewall issue:

  1. Verify outbound HTTPS to rekor.sigstore.dev:

    stella attest rekor connectivity --verbose

  2. Configure proxy if required:

    stella attest config set rekor.proxy https://proxy:8080

  3. Add Rekor endpoints to firewall allowlist:

    • rekor.sigstore.dev:443
    • fulcio.sigstore.dev:443 (for certificate issuance)

If TLS certificate issue:

  1. Check certificate validity:

    stella attest rekor cert-check

  2. Update CA certificates:

    stella crypto ca update

If private Rekor instance issue:

  1. Check private Rekor server status
  2. Verify Rekor database health
  3. Check Rekor signer availability

Verification

# Test Rekor connectivity
stella attest rekor ping

# Submit test entry
stella attest rekor test-submit

# Process any queued entries
stella attest rekor process-queue

# Verify recent attestation in log
stella attest rekor lookup --attestation <attestation-id>

Prevention

  • Redundancy: Configure private Rekor instance as fallback
  • Queuing: Enable queue-on-failure for resilience
  • Monitoring: Alert on Rekor submission failures
  • Offline: Document attestation validity without Rekor for air-gap scenarios

Related Resources

  • Architecture: docs/modules/attestor/transparency-log.md
  • Related runbooks: attestor-signing-failed.md, attestor-verification-failed.md
  • Sigstore docs: https://docs.sigstore.dev/
  • Rekor setup: docs/operations/rekor-configuration.md