26 KiB
26 KiB
Rekor Verification Technical Design
Document ID: DOCS-ATTEST-REKOR-001 Version: 1.0 Last Updated: 2025-12-14 Status: Draft
1. OVERVIEW
This document provides the comprehensive technical design for Rekor transparency log verification in StellaOps. It covers three key capabilities:
- Merkle Proof Verification - Cryptographic verification of inclusion proofs
- Durable Retry Queue - Reliable submission with failure recovery
- Time Skew Validation - Replay protection via timestamp validation
Related Sprints
| Sprint | Priority | Description |
|---|---|---|
| SPRINT_3000_0001_0001 | P0 | Merkle Proof Verification |
| SPRINT_3000_0001_0002 | P1 | Rekor Retry Queue & Metrics |
| SPRINT_3000_0001_0003 | P2 | Time Skew Validation |
2. ARCHITECTURE CONTEXT
Current State
┌─────────────────────────────────────────────────────────────────────┐
│ Attestor Module │
├─────────────────────────────────────────────────────────────────────┤
│ │
│ ┌─────────────────────┐ ┌─────────────────────┐ │
│ │ AttestorSubmission │───►│ IRekorClient │ │
│ │ Service │ │ (HttpRekorClient) │ │
│ └─────────────────────┘ └──────────┬──────────┘ │
│ │ │ │
│ ▼ ▼ │
│ ┌─────────────────────┐ ┌─────────────────────┐ │
│ │ IAttestorEntry │ │ Rekor API │ │
│ │ Repository │ │ (External) │ │
│ └─────────────────────┘ └─────────────────────┘ │
│ │
│ Current Limitations: │
│ ✗ Stores proofs but doesn't verify them cryptographically │
│ ✗ Failed submissions are lost (no retry) │
│ ✗ No integrated_time validation │
│ │
└─────────────────────────────────────────────────────────────────────┘
Target State
┌─────────────────────────────────────────────────────────────────────┐
│ Attestor Module (Enhanced) │
├─────────────────────────────────────────────────────────────────────┤
│ │
│ ┌─────────────────────┐ ┌─────────────────────┐ │
│ │ AttestorSubmission │───►│ IRekorClient │ │
│ │ Service │ │ + VerifyInclusion │◄──┐ │
│ └─────────┬───────────┘ └──────────┬──────────┘ │ │
│ │ │ │ │
│ │ (on failure) │ │ │
│ ▼ ▼ │ │
│ ┌─────────────────────┐ ┌─────────────────────┐ │ │
│ │ IRekorSubmission │ │ MerkleProofVerifier │ │ │
│ │ Queue (PostgreSQL) │ │ CheckpointVerifier │ │ │
│ └─────────┬───────────┘ └─────────────────────┘ │ │
│ │ │ │
│ ▼ │ │
│ ┌─────────────────────┐ ┌─────────────────────┐ │ │
│ │ RekorRetryWorker │───►│ Rekor API │───┘ │
│ │ (Background) │ │ (External) │ │
│ └─────────────────────┘ └─────────────────────┘ │
│ │
│ ┌─────────────────────┐ ┌─────────────────────┐ │
│ │ AttestorVerification│───►│ ITimeSkewValidator │ │
│ │ Service │ │ (integrated_time) │ │
│ └─────────────────────┘ └─────────────────────┘ │
│ │
│ Enhancements: │
│ ✓ Cryptographic Merkle proof verification │
│ ✓ Durable retry queue with exponential backoff │
│ ✓ Time skew detection and alerting │
│ │
└─────────────────────────────────────────────────────────────────────┘
3. COMPONENT DESIGN
3.1 Merkle Proof Verification
3.1.1 Algorithm
Rekor uses RFC 6962 (Certificate Transparency) Merkle tree structure:
Root Hash
/ \
/ \
Hash(0,1) Hash(2,3)
/ \ / \
H(0) H(1) H(2) H(3)
│ │ │ │
Leaf0 Leaf1 Leaf2 Leaf3
Leaf Hash Computation:
leafHash = SHA256(0x00 || RFC6962_Entry)
Interior Node Computation:
interiorHash = SHA256(0x01 || leftChild || rightChild)
Inclusion Proof Verification:
Given:
leafIndex: Position of leaf in treetreeSize: Total number of leavesproofPath[]: Sibling hashes along path to rootexpectedRoot: Root hash from checkpoint
def verify_inclusion(leaf_hash, leaf_index, tree_size, proof_path, expected_root):
current_hash = leaf_hash
current_index = leaf_index
remaining_size = tree_size
for sibling_hash in proof_path:
if current_index % 2 == 1:
# Current node is right child
current_hash = sha256(0x01 || sibling_hash || current_hash)
else:
# Current node is left child
current_hash = sha256(0x01 || current_hash || sibling_hash)
current_index = current_index // 2
remaining_size = (remaining_size + 1) // 2
return current_hash == expected_root
3.1.2 Checkpoint Verification
Rekor checkpoints are signed using the log's private key:
Checkpoint Format:
─────────────────
rekor.sigstore.dev - 1234567
<tree_size>
<root_hash_base64>
— rekor.sigstore.dev <signature>
Verification steps:
- Parse checkpoint text format
- Extract signature and public key hint
- Verify Ed25519/ECDSA signature over checkpoint body
- Extract root hash and tree size
3.1.3 Implementation Classes
/// <summary>
/// RFC 6962 Merkle proof verification.
/// </summary>
public static class MerkleProofVerifier
{
private static readonly byte LeafPrefix = 0x00;
private static readonly byte NodePrefix = 0x01;
public static bool VerifyInclusion(
byte[] leafHash,
long leafIndex,
long treeSize,
IReadOnlyList<byte[]> proofPath,
byte[] expectedRoot)
{
ArgumentNullException.ThrowIfNull(leafHash);
ArgumentNullException.ThrowIfNull(proofPath);
ArgumentNullException.ThrowIfNull(expectedRoot);
if (leafHash.Length != 32 || expectedRoot.Length != 32)
throw new ArgumentException("Hash must be 32 bytes (SHA-256)");
if (leafIndex < 0 || leafIndex >= treeSize)
throw new ArgumentOutOfRangeException(nameof(leafIndex));
var currentHash = leafHash;
var currentIndex = leafIndex;
var currentSize = treeSize;
foreach (var siblingHash in proofPath)
{
if (siblingHash.Length != 32)
throw new ArgumentException("Sibling hash must be 32 bytes");
if (currentIndex % 2 == 1)
{
// Current is right child
currentHash = HashInterior(siblingHash, currentHash);
}
else
{
// Current is left child
currentHash = HashInterior(currentHash, siblingHash);
}
currentIndex /= 2;
currentSize = (currentSize + 1) / 2;
}
return currentHash.AsSpan().SequenceEqual(expectedRoot);
}
private static byte[] HashInterior(byte[] left, byte[] right)
{
Span<byte> buffer = stackalloc byte[1 + 32 + 32];
buffer[0] = NodePrefix;
left.CopyTo(buffer.Slice(1, 32));
right.CopyTo(buffer.Slice(33, 32));
return SHA256.HashData(buffer);
}
public static byte[] ComputeLeafHash(byte[] entryData)
{
Span<byte> buffer = stackalloc byte[1 + entryData.Length];
buffer[0] = LeafPrefix;
entryData.CopyTo(buffer.Slice(1));
return SHA256.HashData(buffer);
}
}
3.2 Durable Retry Queue
3.2.1 State Machine
PENDING
│
│ Worker picks up
▼
SUBMITTING
/ \
/ \
(success) (failure)
/ \
▼ ▼
SUBMITTED RETRYING
│
│ (after backoff delay)
▼
SUBMITTING
│
│ (max attempts exceeded)
▼
DEAD_LETTER
3.2.2 Exponential Backoff
public static TimeSpan CalculateBackoff(int attemptCount, RekorQueueOptions options)
{
var delayMs = options.InitialDelayMs * Math.Pow(options.BackoffMultiplier, attemptCount - 1);
var cappedDelayMs = Math.Min(delayMs, options.MaxDelayMs);
// Add jitter (±10%)
var jitter = Random.Shared.NextDouble() * 0.2 - 0.1;
var finalDelayMs = cappedDelayMs * (1 + jitter);
return TimeSpan.FromMilliseconds(finalDelayMs);
}
Default backoff sequence:
| Attempt | Base Delay | With Jitter |
|---|---|---|
| 1 | 1s | 0.9s - 1.1s |
| 2 | 2s | 1.8s - 2.2s |
| 3 | 4s | 3.6s - 4.4s |
| 4 | 8s | 7.2s - 8.8s |
| 5 | 16s | 14.4s - 17.6s |
3.2.3 Queue Table Design
CREATE TABLE attestor_rekor_queue (
id UUID PRIMARY KEY,
tenant_id TEXT NOT NULL,
bundle_sha256 TEXT NOT NULL UNIQUE, -- Idempotency key
dsse_payload BYTEA NOT NULL,
backend TEXT NOT NULL,
status TEXT NOT NULL,
attempt_count INT NOT NULL DEFAULT 0,
max_attempts INT NOT NULL,
last_attempt_at TIMESTAMPTZ,
last_error TEXT,
next_retry_at TIMESTAMPTZ,
rekor_uuid TEXT, -- Set on success
rekor_log_index BIGINT, -- Set on success
created_at TIMESTAMPTZ NOT NULL,
updated_at TIMESTAMPTZ NOT NULL
);
-- Efficient dequeue query
CREATE INDEX idx_rekor_queue_dequeue
ON attestor_rekor_queue (next_retry_at, status)
WHERE status IN ('pending', 'retrying');
3.2.4 Dequeue Query
-- Atomic dequeue with row locking
WITH eligible AS (
SELECT id
FROM attestor_rekor_queue
WHERE status IN ('pending', 'retrying')
AND (next_retry_at IS NULL OR next_retry_at <= NOW())
ORDER BY next_retry_at NULLS FIRST, created_at
LIMIT :batch_size
FOR UPDATE SKIP LOCKED
)
UPDATE attestor_rekor_queue q
SET status = 'submitting',
updated_at = NOW()
FROM eligible e
WHERE q.id = e.id
RETURNING q.*;
3.3 Time Skew Validation
3.3.1 Threat Model
| Attack | Description | Detection |
|---|---|---|
| Backdated Entry | Attacker inserts entry with old timestamp | Large positive skew |
| Future Timestamp | Attacker pre-dates entry | Negative skew (future) |
| Log Manipulation | Attacker modifies existing entries | Timestamp inconsistency |
3.3.2 Threshold Design
Time Skew Detection Zones
─────────────────────────────────────────────────────────────────
│ REJECT │ WARN │ OK │ WARN │ REJECT │
│ FUTURE │ FUTURE │ │ PAST │ PAST │
─────────────────────────────────────────────────────────────────
◄───────────────────────┼───────────────────────►
-60s -5m NOW +5m +1h
(local time)
Default Thresholds:
• Future tolerance: 60 seconds (beyond = reject)
• Warn threshold: 5 minutes
• Reject threshold: 1 hour
3.3.3 Validation Flow
public TimeSkewResult Validate(DateTimeOffset integratedTime, DateTimeOffset localTime)
{
var skew = integratedTime - localTime;
// Future timestamps are highly suspicious
if (skew > TimeSpan.Zero)
{
if (skew > _options.FutureTolerance)
{
return Rejected($"Future timestamp by {skew}");
}
return Ok(skew); // Within future tolerance
}
// Past timestamps (normal case - Rekor time is in the past)
var absSkew = skew.Duration();
if (absSkew >= _options.RejectThreshold)
{
return Rejected($"Time skew {absSkew} exceeds reject threshold");
}
if (absSkew >= _options.WarnThreshold)
{
return Warning($"Time skew {absSkew} exceeds warn threshold");
}
return Ok(skew);
}
4. DATA FLOW
4.1 Submission Flow (with Queue)
┌─────────┐ ┌──────────────────┐ ┌───────────┐
│ Client │─────►│ SubmissionService │─────►│ RekorAPI │
└─────────┘ └────────┬─────────┘ └─────┬─────┘
│ │
│ (success) │
▼ │
┌─────────────────┐ │
│ Store Entry │◄─────────────┘
│ (status=ok) │
└─────────────────┘
│ (failure)
▼
┌─────────────────┐
│ Enqueue │
│ (status=pending)│
└────────┬────────┘
│
▼
┌─────────────────┐ ┌───────────┐
│ RetryWorker │─────►│ RekorAPI │
│ (background) │ └─────┬─────┘
└─────────────────┘ │
▲ │
│ │
└─────────────────────┘
(retry loop)
4.2 Verification Flow (with Proof Verification)
┌─────────┐ ┌───────────────────┐ ┌─────────────────┐
│ Client │─────►│ VerificationSvc │─────►│ EntryRepository │
└─────────┘ └────────┬──────────┘ └────────┬────────┘
│ │
│◄─────────────────────────┘
│ (AttestorEntry)
▼
┌─────────────────────┐
│ 1. TimeSkewValidator│
│ (integrated_time)│
└────────┬────────────┘
│
▼
┌─────────────────────┐
│ 2. MerkleProof │
│ Verifier │
└────────┬────────────┘
│
▼
┌─────────────────────┐
│ 3. Checkpoint │
│ Verifier │
└────────┬────────────┘
│
▼
┌─────────────────────┐
│ 4. Aggregate Result │
│ (VerificationRpt)│
└─────────────────────┘
5. CONFIGURATION REFERENCE
# attestor.yaml
attestor:
rekor:
primary:
url: https://rekor.sigstore.dev
proof_timeout_ms: 15000
poll_interval_ms: 250
max_attempts: 60
mirror:
enabled: false
url: https://rekor-mirror.internal
verification:
public_key_path: /etc/stellaops/rekor-pub.pem
# Or inline:
# public_key_base64: LS0tLS1CRUdJTi...
allow_offline_without_signature: false
max_checkpoint_age_minutes: 60
queue:
enabled: true
max_attempts: 5
initial_delay_ms: 1000
max_delay_ms: 60000
backoff_multiplier: 2.0
batch_size: 10
poll_interval_ms: 5000
dead_letter_retention_days: 30
time_skew:
enabled: true
warn_threshold_seconds: 300 # 5 minutes
reject_threshold_seconds: 3600 # 1 hour
future_tolerance_seconds: 60 # 1 minute
reject_future_timestamps: true
skip_in_offline_mode: true
6. METRICS REFERENCE
| Metric | Type | Labels | Description |
|---|---|---|---|
attestor.inclusion_verify_total |
Counter | result |
Inclusion proof verifications |
attestor.inclusion_verify_latency_seconds |
Histogram | Verification latency | |
attestor.checkpoint_verify_total |
Counter | result |
Checkpoint signature verifications |
attestor.rekor_queue_depth |
Gauge | Current pending + retrying items | |
attestor.rekor_retry_attempts_total |
Counter | backend, attempt |
Retry attempts |
attestor.rekor_submission_status_total |
Counter | status, backend |
Submission outcomes |
attestor.rekor_queue_wait_seconds |
Histogram | Time in queue before submission | |
attestor.time_skew_detected_total |
Counter | severity, action |
Time skew detections |
attestor.time_skew_seconds |
Histogram | Observed skew distribution |
7. ERROR HANDLING
7.1 Error Taxonomy
| Error Code | Description | Retry? | Action |
|---|---|---|---|
rekor_unavailable |
Rekor API not reachable | Yes | Queue for retry |
rekor_conflict |
Duplicate entry (409) | No | Retrieve existing entry |
rekor_rate_limited |
Rate limit exceeded (429) | Yes | Backoff and retry |
rekor_internal_error |
Server error (5xx) | Yes | Queue for retry |
proof_invalid |
Merkle proof verification failed | No | Reject, log alert |
checkpoint_signature_invalid |
Checkpoint signature failed | No | Reject, log alert |
time_skew_rejected |
Time skew exceeds threshold | No | Reject, log warning |
7.2 Structured Logging
{
"timestamp": "2025-12-14T10:30:00Z",
"level": "Warning",
"message": "Rekor submission failed, queuing for retry",
"error_code": "rekor_unavailable",
"bundle_sha256": "abc123...",
"backend": "primary",
"attempt_count": 1,
"next_retry_at": "2025-12-14T10:30:02Z",
"error_message": "Connection refused"
}
8. SECURITY CONSIDERATIONS
8.1 Key Management
- Rekor public key must be distributed out-of-band
- Support key rotation via versioned key configuration
- Store keys in secure location (not in code/config)
8.2 Trust Model
Trust Hierarchy
───────────────
┌─────────────┐
│ Rekor Root │
│ Public Key │
└──────┬──────┘
│ signs
▼
┌─────────────┐
│ Checkpoint │
│ (root hash) │
└──────┬──────┘
│ commits to
▼
┌─────────────┐
│ Merkle Tree │
│ (entries) │
└──────┬──────┘
│ includes
▼
┌─────────────┐
│ Attestation │
│ (DSSE) │
└─────────────┘
8.3 Offline Security
In air-gapped environments:
- Checkpoint must be pre-distributed with offline bundle
- Proof verification still works (no network needed)
- Time skew validation should be skipped or use bundled reference time
9. TESTING STRATEGY
9.1 Test Categories
| Category | Coverage Target | Tools |
|---|---|---|
| Unit | >90% | xUnit, Moq |
| Integration | >80% | Testcontainers (PostgreSQL) |
| Contract | All public APIs | Snapshot testing |
| Performance | Latency P99 | BenchmarkDotNet |
9.2 Golden Fixtures
Obtain from public Sigstore Rekor instance:
# Get a real Rekor entry for testing
rekor-cli get --uuid 24296fb24b8ad77a... --format json > fixtures/rekor-entry-valid.json
# Get checkpoint
curl https://rekor.sigstore.dev/api/v1/log > fixtures/rekor-checkpoint.json
# Get public key
curl https://rekor.sigstore.dev/api/v1/log/publicKey > fixtures/rekor-pubkey.pem
10. MIGRATION GUIDE
10.1 Database Migrations
Run in order:
00X_rekor_submission_queue.sql- Queue table- Update
AttestorEntryschema if stored in PostgreSQL
10.2 Configuration Migration
# Before (existing)
attestor:
rekor:
primary:
url: https://rekor.sigstore.dev
# After (add new sections)
attestor:
rekor:
primary:
url: https://rekor.sigstore.dev
verification:
public_key_path: /etc/stellaops/rekor-pub.pem
queue:
enabled: true
time_skew:
enabled: true
10.3 Rollback Plan
- Queue table can be dropped if not needed
- All new features are configurable (can disable)
- No breaking changes to existing API contracts