--- checkId: check.evidencelocker.retrieval plugin: stellaops.doctor.evidencelocker severity: fail tags: [evidence, attestation, retrieval, core] --- # Attestation Retrieval ## What It Checks Verifies that attestation artifacts can be retrieved from the evidence locker. The check supports two modes depending on the deployment: **HTTP mode** (when `IHttpClientFactory` is available): Sends a GET request to `{endpoint}/v1/attestations/sample` with a 5-second timeout and measures response latency. **Local file mode** (fallback): Checks the local evidence locker path at `EvidenceLocker:Path`, verifies the `attestations/` subdirectory exists, and attempts to read a sample attestation JSON file. | Condition | Result | |---|---| | Endpoint not configured | Skip | | HTTP request times out (> 5000ms) | Fail | | HTTP error status code | Fail | | Connection error | Fail | | HTTP success but latency > 500ms | Warn | | Local attestations directory missing | Warn | | HTTP success with latency <= 500ms | Pass | | Local file read successful | Pass | Evidence collected: `Endpoint`, `StatusCode`, `LatencyMs`, `Threshold`, `Path`, `SampleAttestation`, `ContentLength`. The check only runs when `EvidenceLocker:Endpoint` or `Services:EvidenceLocker` is configured. ## Why It Matters Attestation retrieval is a core operation used throughout the release pipeline. Release approvals, audit queries, compliance reports, and evidence exports all depend on being able to retrieve attestation artifacts from the evidence locker. If retrieval is slow or failing, release approvals may time out, audit queries will fail, and compliance reports cannot be generated. Latency above 500ms indicates performance degradation that will compound when retrieving multiple attestations during a release or audit. ## Common Causes - Evidence locker service unavailable or not running - Authentication failure when accessing the evidence locker API - Artifact not found (empty or uninitialized evidence locker) - Evidence locker under heavy load causing elevated latency - Network latency between services - Storage backend slow (disk I/O bottleneck) - Local evidence locker path not configured or directory missing - File permission issues on local attestation files ## How to Fix ### Docker Compose ```bash # Check evidence locker service status docker compose ps evidence-locker # Test evidence retrieval docker compose exec evidence-locker stella evidence status # Test authentication docker compose exec evidence-locker stella evidence auth-test # Check service logs for errors docker compose logs evidence-locker --since 5m # If local mode, verify the evidence path and permissions docker compose exec evidence-locker ls -la /data/evidence/attestations/ # Initialize evidence locker if needed docker compose exec evidence-locker stella evidence init # Set endpoint configuration # EvidenceLocker__Endpoint=http://evidence-locker:5080 ``` ### Bare Metal / systemd ```bash # Check service status sudo systemctl status stellaops-evidence-locker # Test evidence retrieval stella evidence status # Test connectivity stella evidence ping # Check attestations directory ls -la /var/lib/stellaops/evidence/attestations/ # Initialize if empty stella evidence init # Check disk I/O iostat -x 1 5 # In appsettings.json: # "EvidenceLocker": { "Endpoint": "http://localhost:5080" } sudo systemctl restart stellaops-evidence-locker ``` ### Kubernetes / Helm ```bash # Check evidence locker pod status kubectl get pods -l app=stellaops-evidence-locker # Check pod logs kubectl logs deploy/stellaops-evidence-locker --since=5m # Test retrieval from within cluster kubectl exec deploy/stellaops-evidence-locker -- stella evidence status # Check persistent volume kubectl describe pvc stellaops-evidence-data # Check for resource constraints kubectl top pod -l app=stellaops-evidence-locker ``` ```yaml # values.yaml evidenceLocker: endpoint: http://stellaops-evidence-locker:5080 resources: requests: memory: 256Mi cpu: 100m limits: memory: 512Mi cpu: 500m ``` ## Verification ``` stella doctor run --check check.evidencelocker.retrieval ``` ## Related Checks - `check.evidencelocker.index` — evidence index consistency affects retrieval accuracy - `check.evidencelocker.provenance` — provenance chain integrity depends on reliable retrieval - `check.evidencelocker.merkle` — Merkle anchor verification requires attestation access - `check.compliance.evidence-rate` — evidence generation feeds the retrieval pipeline - `check.compliance.evidence-integrity` — integrity verification requires successful retrieval