Doctor plugin checks: implement health check classes and documentation

Implement remediation-aware health checks across all Doctor plugin modules
(Agent, Attestor, Auth, BinaryAnalysis, Compliance, Crypto, Environment,
EvidenceLocker, Notify, Observability, Operations, Policy, Postgres, Release,
Scanner, Storage, Vex) and their backing library counterparts (AI, Attestation,
Authority, Core, Cryptography, Database, Docker, Integration, Notify,
Observability, Security, ServiceGraph, Sources, Verification).

Each check now emits structured remediation metadata (severity, category,
runbook links, and fix suggestions) consumed by the Doctor dashboard
remediation panel.

Also adds:
- docs/doctor/articles/ knowledge base for check explanations
- Advisory AI search seed and allowlist updates for doctor content
- Sprint plan for doctor checks documentation

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
master
2026-03-27 12:28:00 +02:00
parent fbd24e71de
commit c58a236d70
326 changed files with 18500 additions and 463 deletions

View File

@@ -0,0 +1,53 @@
---
checkId: check.timestamp.crl.distribution
plugin: stellaops.doctor.timestamping
severity: warn
tags: [timestamping, crl, distribution, revocation]
---
# CRL Distribution Point Availability
## What It Checks
Checks that configured CRL distribution points are accessible. The check:
- Gets configured CDPs from the registry.
- Sends a HEAD request to each CDP URL with a 30-second timeout.
- Reports response status, latency, and CRL size (from Content-Length).
- Fails if all CDPs are unavailable. Warns if some are unavailable.
- Passes (healthy) if no CDPs are configured (optional feature).
## Why It Matters
CRL distribution points provide certificate revocation lists needed to verify that TSA certificates have not been revoked. If CDPs are unavailable, the system cannot download updated CRLs, potentially accepting timestamps from revoked certificates.
## Common Causes
- CRL distribution point server is down
- Network connectivity issues
- Firewall blocking HTTP/HTTPS to CDP URLs
- CDP URL changed by the CA
## How to Fix
### Docker Compose
```bash
docker exec <platform-container> curl -I http://crl.example.com/crl.pem
```
### Bare Metal / systemd
```bash
# Test CDP connectivity
curl -I http://crl.example.com/crl.pem
# Check network and DNS
nslookup crl.example.com
```
### Kubernetes / Helm
Ensure egress NetworkPolicies allow traffic to CRL distribution point URLs.
## Verification
```
stella doctor run --check check.timestamp.crl.distribution
```
## Related Checks
- `check.timestamp.ocsp.responder` — checks OCSP responder availability
- `check.timestamp.revocation.cache-fresh` — checks revocation cache freshness

View File

@@ -0,0 +1,57 @@
---
checkId: check.timestamp.eidas.trustlist.fresh
plugin: stellaops.doctor.timestamping
severity: warn
tags: [timestamping, eidas, trustlist, lotl, compliance]
---
# EU Trust List Freshness
## What It Checks
Checks that the EU Trust List (LOTL -- List of Trusted Lists) is up-to-date. The check:
- Queries the trust list cache for the last refresh timestamp.
- Fails if the cache does not exist (trust list never fetched).
- Fails if the cache is older than the critical threshold (default 7 days).
- Warns if older than the warning threshold (default 3 days).
- Reports the number of TSPs (Trust Service Providers) and QTS (Qualified TSAs) in the cache.
## Why It Matters
The EU Trust List is the authoritative source for eIDAS-qualified trust service providers. A stale trust list may not reflect recent provider additions, withdrawals, or status changes, leading to incorrect qualification decisions for TSA providers operating under eIDAS regulation.
## Common Causes
- Trust list refresh job not running
- Network issues preventing download from the EU publication endpoint
- Air-gapped environment without scheduled trust list updates
## How to Fix
### Docker Compose
```bash
docker exec <platform-container> stella trust-list refresh
```
### Bare Metal / systemd
```bash
stella trust-list refresh
# Schedule automatic refresh
stella trust-list schedule --interval 24h
```
### Kubernetes / Helm
```yaml
timestamping:
eidas:
trustListRefreshSchedule: "0 6 * * *"
warningAgeDays: 3
criticalAgeDays: 7
```
## Verification
```
stella doctor run --check check.timestamp.eidas.trustlist.fresh
```
## Related Checks
- `check.timestamp.eidas.qts.qualified` — checks QTS provider qualification status
- `check.timestamp.eidas.qts.status-change` — alerts on qualification status changes

View File

@@ -0,0 +1,75 @@
---
checkId: check.timestamp.evidence.staleness
plugin: stellaops.doctor.timestamping
severity: warn
tags: [timestamping, evidence, staleness, retimestamp]
---
# Evidence Staleness
## What It Checks
Aggregated check for timestamp evidence staleness across six dimensions:
- **TST Expiry**: timestamps approaching signing certificate expiry (warn at 180 days, critical at 90 days).
- **Deprecated Algorithms**: timestamps using deprecated hash algorithms (e.g., SHA1).
- **Missing Stapling**: timestamps without stapled OCSP/CRL revocation data.
- **Retimestamp Queue**: artifacts pending re-timestamping.
- **OCSP Staleness**: OCSP responses approaching expiry (warn at 3 days).
- **CRL Staleness**: CRLs approaching expiry (warn at 7 days).
Fails if any dimension is unhealthy (count exceeds `CriticalStaleCount`, default 10). Warns if any dimension is degraded.
## Why It Matters
Stale evidence loses its verifiability over time. Expired timestamps, deprecated algorithms, and missing revocation data all weaken the chain of trust. Proactive detection enables scheduled re-timestamping before evidence becomes unverifiable.
## Common Causes
- Re-timestamp jobs not running or failing
- TSA signing certificates approaching expiry
- OCSP/CRL cache not refreshed
- Legacy artifacts signed with SHA1
## How to Fix
### Docker Compose
```bash
# Run evidence refresh
docker exec <platform-container> stella evidence refresh --all
# Run retimestamp queue
docker exec <platform-container> stella retimestamp run
```
### Bare Metal / systemd
```bash
# Check evidence status
stella evidence audit --staleness
# Refresh stale evidence
stella evidence refresh --all
# Process retimestamp queue
stella retimestamp run
# Schedule automatic refresh
stella retimestamp schedule create --interval daily
```
### Kubernetes / Helm
```yaml
timestamping:
evidenceStaleness:
tstWarnDays: 180
tstCriticalDays: 90
criticalStaleCount: 10
retimestampSchedule: "0 1 * * *"
```
## Verification
```
stella doctor run --check check.timestamp.evidence.staleness
```
## Related Checks
- `check.timestamp.evidence.tst.expiry` — focused check for expiring TSTs
- `check.timestamp.evidence.tst.deprecated-algo` — focused check for deprecated algorithms
- `check.timestamp.evidence.tst.missing-stapling` — focused check for missing stapling
- `check.timestamp.evidence.retimestamp.pending` — focused check for pending retimestamps

View File

@@ -0,0 +1,53 @@
---
checkId: check.timestamp.ocsp.responder
plugin: stellaops.doctor.timestamping
severity: warn
tags: [timestamping, ocsp, responder, revocation]
---
# OCSP Responder Availability
## What It Checks
Checks that configured OCSP responders are accessible. The check:
- Gets configured OCSP responders from the registry.
- Sends an OPTIONS request to each responder with a 10-second timeout.
- Considers 2xx and 405 (Method Not Allowed) responses as healthy.
- Fails if all responders are unavailable. Warns if some are unavailable.
- Reports degraded if no responders are configured.
## Why It Matters
OCSP responders provide real-time certificate revocation status. If OCSP responders are unavailable, the system cannot verify whether TSA certificates have been revoked, potentially accepting timestamps from compromised certificates.
## Common Causes
- OCSP responder server is down
- Network connectivity issues
- Firewall blocking HTTP/HTTPS to OCSP URLs
- OCSP responder URL changed by the CA
## How to Fix
### Docker Compose
```bash
# Test OCSP responder connectivity
docker exec <platform-container> curl -v http://ocsp.digicert.com
```
### Bare Metal / systemd
```bash
# Test OCSP responder
openssl ocsp -issuer /path/to/issuer.pem -cert /path/to/cert.pem \
-url http://ocsp.digicert.com -resp_text
```
### Kubernetes / Helm
Ensure egress NetworkPolicies allow traffic to OCSP responder URLs.
## Verification
```
stella doctor run --check check.timestamp.ocsp.responder
```
## Related Checks
- `check.timestamp.ocsp.stapling` — checks OCSP stapling configuration
- `check.timestamp.revocation.cache-fresh` — checks revocation cache freshness
- `check.timestamp.crl.distribution` — checks CRL distribution point availability

View File

@@ -0,0 +1,53 @@
---
checkId: check.timestamp.ocsp.stapling
plugin: stellaops.doctor.timestamping
severity: warn
tags: [timestamping, ocsp, stapling, revocation]
---
# OCSP Stapling Enabled
## What It Checks
Checks whether TSA OCSP stapling is configured and fresh. The check:
- Queries the stapling status provider for all TSA providers.
- Reports which providers have OCSP stapling enabled or disabled.
- Fails if all providers have stapling disabled. Warns if some are disabled.
- Reports degraded if no stapling status data is available.
## Why It Matters
OCSP stapling embeds the OCSP response directly in the TLS handshake or timestamp token, eliminating the need for clients to perform live OCSP lookups. This is critical for air-gapped deployments where live OCSP lookups are impossible, and improves performance for all deployments.
## Common Causes
- OCSP stapling not configured for TSA providers
- Stapling status monitoring not set up
- TSA provider does not support stapling
## How to Fix
### Docker Compose
Enable OCSP stapling in TSA provider configuration:
```yaml
environment:
Timestamping__OcspStapling__Enabled: "true"
```
### Bare Metal / systemd
Configure OCSP stapling in `appsettings.json` and ensure TSA providers support it.
### Kubernetes / Helm
```yaml
timestamping:
ocspStapling:
enabled: true
```
## Verification
```
stella doctor run --check check.timestamp.ocsp.stapling
```
## Related Checks
- `check.timestamp.ocsp.responder` — checks OCSP responder availability
- `check.timestamp.evidence.tst.missing-stapling` — detects timestamps without stapled data
- `check.timestamp.revocation.cache-fresh` — checks revocation cache freshness

View File

@@ -0,0 +1,57 @@
---
checkId: check.timestamp.eidas.qts.qualified
plugin: stellaops.doctor.timestamping
severity: fail
tags: [timestamping, eidas, qts, qualification, compliance]
---
# QTS Providers Qualification
## What It Checks
Checks that configured qualified TSA providers are still listed on the EU Trust List. The check:
- Gets qualified TSA providers from the registry.
- For each provider, queries the trust list cache for current qualification status.
- Fails if any provider is no longer qualified (withdrawn, suspended, or not found).
- Passes if no qualified providers are configured (optional feature) or all are still qualified.
## Why It Matters
Under eIDAS regulation, only qualified TSA providers can produce timestamps with legal effect equivalent to handwritten signatures. If a provider loses qualification, timestamps from that provider no longer meet eIDAS compliance requirements, potentially invalidating evidence used for regulated releases.
## Common Causes
- TSA provider's qualified status withdrawn by a supervisory body
- Provider suspended due to compliance issues
- Provider not yet (re-)listed on the current trust list version
- Trust list cache is stale (check `check.timestamp.eidas.trustlist.fresh`)
## How to Fix
### Docker Compose
```bash
# Refresh trust list first
docker exec <platform-container> stella trust-list refresh
# Check provider status
docker exec <platform-container> stella tsa qualification status
```
### Bare Metal / systemd
```bash
stella trust-list refresh
stella tsa qualification status
# Replace non-qualified provider
stella tsa remove --name "Withdrawn Provider"
stella tsa add --name "New QTS" --url "https://new-qualified-tsa.eu/tsr" --qualified
```
### Kubernetes / Helm
Update TSA provider configuration to use only qualified providers.
## Verification
```
stella doctor run --check check.timestamp.eidas.qts.qualified
```
## Related Checks
- `check.timestamp.eidas.trustlist.fresh` — checks EU Trust List freshness
- `check.timestamp.eidas.qts.status-change` — alerts on qualification status changes

View File

@@ -0,0 +1,52 @@
---
checkId: check.timestamp.eidas.qts.status-change
plugin: stellaops.doctor.timestamping
severity: warn
tags: [timestamping, eidas, qts, status, monitoring]
---
# QTS Status Changes
## What It Checks
Alerts on TSA qualification status changes in the past 7 days. The check:
- Queries the status change tracker for recent changes within a 7-day window.
- Reports each change (provider name, previous status, new status, change date).
- Warns if any withdrawals occurred (provider moved from Qualified to Withdrawn/Suspended/Deprecated).
- Passes if no changes occurred or all changes are positive (e.g., new qualification grants).
## Why It Matters
Qualification status changes require operational response. A withdrawal means the provider's timestamps no longer satisfy eIDAS requirements, and traffic should be migrated to an alternative provider. Early detection of changes enables proactive migration before compliance deadlines.
## Common Causes
- Supervisory body action against a TSA provider
- Provider voluntary withdrawal from qualification
- New provider achieving qualification (positive change)
## How to Fix
### Docker Compose
```bash
# Review recent changes
docker exec <platform-container> stella tsa qualification changes --days 7
# If a provider was withdrawn, add a replacement
docker exec <platform-container> stella tsa add --name "Replacement QTS" --url "https://new-tsa.eu/tsr" --qualified
```
### Bare Metal / systemd
```bash
stella tsa qualification changes --days 7
stella tsa qualification status
```
### Kubernetes / Helm
Review changes and update provider configuration as needed.
## Verification
```
stella doctor run --check check.timestamp.eidas.qts.status-change
```
## Related Checks
- `check.timestamp.eidas.qts.qualified` — checks provider qualification status
- `check.timestamp.eidas.trustlist.fresh` — checks EU Trust List freshness

View File

@@ -0,0 +1,63 @@
---
checkId: check.timestamp.timesync.rekor-correlation
plugin: stellaops.doctor.timestamping
severity: warn
tags: [timestamping, timesync, rekor, correlation, transparency]
---
# TST-Rekor Time Correlation
## What It Checks
Checks that TST genTime and Rekor integratedTime are properly correlated. The check:
- Retrieves recent attestations from the lookback window (default 24 hours).
- For each attestation with both TST and Rekor timestamps, computes the gap.
- **Fails** (unhealthy) if any attestation has TST *after* Rekor (potential backdating -- TST should always precede Rekor integration).
- **Warns** (degraded) if the gap between TST and Rekor exceeds the maximum threshold (default 5 minutes).
## Why It Matters
Proper time correlation between TST (Timestamp Token) and Rekor (transparency log) entries is a key integrity signal. If a TST is dated after its Rekor entry, it may indicate clock manipulation or backdating. Excessive gaps suggest pipeline delays that could indicate operational issues.
## Common Causes
- System clock drift causing ordering violations
- Pipeline delays between timestamping and Rekor submission
- Rekor transparency log ingestion delays
- Network issues causing timestamp reordering
## How to Fix
### Docker Compose
```bash
# Check system time sync
timedatectl status
# Verify pipeline timing
docker exec <platform-container> stella evidence audit --recent --show-timing
```
### Bare Metal / systemd
```bash
# Investigate time synchronization
chronyc tracking
# Review attestation timing
stella evidence audit --recent --show-timing
```
### Kubernetes / Helm
```yaml
timestamping:
rekorCorrelation:
lookbackWindow: "24h"
maximumGap: "5m"
```
Investigate ordering violations immediately as they may indicate tampering.
## Verification
```
stella doctor run --check check.timestamp.timesync.rekor-correlation
```
## Related Checks
- `check.timestamp.timesync.system` — checks system clock synchronization
- `check.timestamp.timesync.tsa-skew` — checks TSA time skew

View File

@@ -0,0 +1,40 @@
---
checkId: check.timestamp.evidence.retimestamp.pending
plugin: stellaops.doctor.timestamping
severity: warn
tags: [timestamping, evidence, retimestamp, queue]
---
# Retimestamp Pending
## What It Checks
Detects artifacts pending re-timestamping. Fails if the pending count exceeds the critical threshold (default 10), otherwise warns.
## Why It Matters
Pending retimestamp work means artifacts are at risk of losing their temporal proof. If retimestamping does not complete before the original timestamp expires, the evidence becomes unverifiable and may need to be regenerated from scratch.
## Common Causes
- Retimestamp queue processor not running
- TSA endpoints unavailable during retimestamp attempts
- Queue backlog from a large batch of expiring timestamps
- Retimestamp job scheduling not configured
## How to Fix
Process the retimestamp queue:
```bash
stella retimestamp run
# Schedule automatic processing
stella retimestamp schedule create --interval daily
```
If TSA endpoints are down, resolve connectivity first (see `check.timestamp.tsa.reachable`).
## Verification
```
stella doctor run --check check.timestamp.evidence.retimestamp.pending
```
## Related Checks
- `check.timestamp.evidence.staleness` — aggregated evidence staleness check
- `check.timestamp.tsa.reachable` — verifies TSA endpoints are reachable

View File

@@ -0,0 +1,57 @@
---
checkId: check.timestamp.revocation.cache-fresh
plugin: stellaops.doctor.timestamping
severity: warn
tags: [timestamping, revocation, cache, ocsp, crl]
---
# Revocation Cache Freshness
## What It Checks
Checks that cached OCSP responses and CRLs are not stale. The check:
- Queries the revocation cache for OCSP response and CRL snapshot timestamps.
- Compares each cached item's age against thresholds: OCSP max age (default 12 hours), CRL max age (default 7 days).
- Fails if all cached items are stale. Warns if some are stale.
- Passes if no cached data exists (optional) or all items are fresh.
## Why It Matters
Stale revocation data means the system may accept certificates that have been revoked since the cache was last updated. For air-gapped environments that cannot perform live lookups, a fresh cache is the only source of revocation information.
## Common Causes
- Revocation cache refresh job not running
- Network issues preventing OCSP/CRL fetches
- Cache storage issues
## How to Fix
### Docker Compose
```bash
# Refresh revocation cache
docker exec <platform-container> stella revocation cache refresh
```
### Bare Metal / systemd
```bash
stella revocation cache refresh
# Schedule automatic refresh
stella revocation cache schedule --ocsp-interval 6h --crl-interval 24h
```
### Kubernetes / Helm
```yaml
timestamping:
revocationCache:
ocspMaxAgeHours: 12
crlMaxAgeDays: 7
refreshSchedule: "0 */6 * * *"
```
## Verification
```
stella doctor run --check check.timestamp.revocation.cache-fresh
```
## Related Checks
- `check.timestamp.ocsp.responder` — checks OCSP responder availability
- `check.timestamp.crl.distribution` — checks CRL distribution point availability

View File

@@ -0,0 +1,75 @@
---
checkId: check.timestamp.timesync.system
plugin: stellaops.doctor.timestamping
severity: fail
tags: [timestamping, timesync, ntp, system]
---
# System Time Synchronization
## What It Checks
Checks that the system clock is synchronized with NTP servers. The check:
- Queries configured NTP servers (defaults to `time.nist.gov` and `pool.ntp.org`) using the NTP protocol (UDP port 123).
- Computes the time skew between the local system clock and each NTP server.
- **Fails** (unhealthy) if skew exceeds the critical threshold (default 5 seconds).
- **Warns** (degraded) if skew exceeds the warning threshold (default 1 second).
- Reports degraded if no NTP servers can be reached.
## Why It Matters
Accurate system time is fundamental to timestamping. Clock skew causes timestamp tokens to have incorrect genTime values, which can invalidate evidence during verification. Large skew can also cause TLS certificate validation failures, authentication token rejection, and incorrect audit log ordering.
## Common Causes
- NTP service not running (chrony, ntpd, systemd-timesyncd)
- NTP servers unreachable (firewall blocking UDP 123)
- Virtual machine time drift (especially paused/resumed VMs)
- Hardware clock issues
## How to Fix
### Docker Compose
Docker containers inherit the host's clock. Fix time sync on the Docker host:
```bash
# Check host time sync
timedatectl status
# Enable NTP sync
sudo timedatectl set-ntp true
# Or configure chrony
sudo systemctl restart chronyd
```
### Bare Metal / systemd
```bash
# Check time sync status
timedatectl status
chronyc tracking
# Force sync
sudo chronyc makestep
# Enable NTP
sudo timedatectl set-ntp true
sudo systemctl enable chronyd
```
### Kubernetes / Helm
Kubernetes nodes must have NTP configured. Verify on each node:
```bash
# On each node
timedatectl status
chronyc tracking
```
Ensure NTP is part of your node provisioning configuration.
## Verification
```
stella doctor run --check check.timestamp.timesync.system
```
## Related Checks
- `check.timestamp.timesync.tsa-skew` — checks skew between system clock and TSA genTime
- `check.timestamp.timesync.rekor-correlation` — checks TST-Rekor time correlation

View File

@@ -0,0 +1,72 @@
---
checkId: check.timestamp.tsa.reachable
plugin: stellaops.doctor.timestamping
severity: fail
tags: [timestamping, tsa, availability, connectivity]
---
# TSA Availability
## What It Checks
Verifies that configured TSA (Time Stamp Authority) endpoints are reachable and responding. The check:
- Probes each endpoint from the `TsaEndpoints` configuration via HTTP HEAD requests.
- Considers HTTP 2xx and 405 (Method Not Allowed) as healthy responses (405 means the TSA is alive but only accepts POST).
- Reports the count of healthy vs. unhealthy endpoints.
- Degrades if no endpoints are configured. Fails if no endpoints are reachable. Warns if some are down.
## Why It Matters
TSA endpoints provide RFC-3161 timestamps that anchor release evidence in time. If no TSA is reachable, new evidence cannot be timestamped, blocking policy-gated releases that require verifiable timestamps. This is a critical-severity check.
## Common Causes
- TSA endpoint server is down or unreachable
- Network connectivity issues or firewall blocking HTTPS
- DNS resolution failure
- TSA provider maintenance or outage
## How to Fix
### Docker Compose
```yaml
environment:
Timestamping__TsaEndpoints__0__Name: "FreeTSA"
Timestamping__TsaEndpoints__0__Url: "https://freetsa.org/tsr"
Timestamping__TsaEndpoints__1__Name: "DigiCert"
Timestamping__TsaEndpoints__1__Url: "http://timestamp.digicert.com"
```
### Bare Metal / systemd
```json
{
"Timestamping": {
"TsaEndpoints": [
{ "Name": "FreeTSA", "Url": "https://freetsa.org/tsr" },
{ "Name": "DigiCert", "Url": "http://timestamp.digicert.com" }
]
}
}
```
Test connectivity:
```bash
curl -I https://freetsa.org/tsr
```
### Kubernetes / Helm
```yaml
timestamping:
tsaEndpoints:
- name: "FreeTSA"
url: "https://freetsa.org/tsr"
- name: "DigiCert"
url: "http://timestamp.digicert.com"
```
## Verification
```
stella doctor run --check check.timestamp.tsa.reachable
```
## Related Checks
- `check.timestamp.tsa.response-time` — measures TSA response latency
- `check.timestamp.tsa.valid-response` — verifies TSA returns valid RFC-3161 responses
- `check.timestamp.tsa.failover-ready` — confirms backup TSA endpoints for failover

View File

@@ -0,0 +1,66 @@
---
checkId: check.timestamp.tsa.cert-expiry
plugin: stellaops.doctor.timestamping
severity: warn
tags: [timestamping, tsa, certificate, expiry]
---
# TSA Certificate Expiry
## What It Checks
Checks if TSA signing certificates are approaching expiry. The check:
- Evaluates each certificate in the `TsaCertificates` configuration list.
- Calculates days remaining until expiry.
- **Fails** (unhealthy) if any certificate is expired or within the critical threshold (default 90 days).
- **Warns** (degraded) if within the warning threshold (default 180 days).
- Passes if all certificates have sufficient validity remaining.
## Why It Matters
An expired TSA signing certificate means new timestamps cannot be validated by relying parties. Evidence signed with an expired certificate may be rejected during compliance audits. Early warning gives operators time to coordinate certificate renewal with the TSA provider before any disruption.
## Common Causes
- TSA provider certificate approaching natural end-of-life
- Certificate renewal not tracked or scheduled
- Using a short-lived certificate without automated renewal
## How to Fix
### Docker Compose
Update the certificate configuration when renewed certificates are obtained from the TSA provider:
```yaml
environment:
Timestamping__TsaCertificates__0__Name: "DigiCert TSA"
Timestamping__TsaCertificates__0__Subject: "CN=DigiCert TSA"
Timestamping__TsaCertificates__0__ExpiresAt: "2027-01-15T00:00:00Z"
```
### Bare Metal / systemd
Contact the TSA provider to obtain renewed certificates and update the trust configuration:
```bash
stella tsa cert update --name "DigiCert TSA" --cert /path/to/new-cert.pem
```
### Kubernetes / Helm
```yaml
timestamping:
certificates:
warnDays: 180
criticalDays: 90
```
Update Kubernetes secrets when new certificates are obtained:
```bash
kubectl create secret generic tsa-certs --from-file=cert.pem=/path/to/new-cert.pem --dry-run=client -o yaml | kubectl apply -f -
```
## Verification
```
stella doctor run --check check.timestamp.tsa.cert-expiry
```
## Related Checks
- `check.timestamp.tsa.root-expiry` — checks TSA root/trust anchor certificate expiry
- `check.timestamp.tsa.chain-valid` — validates TSA certificate chain integrity
- `check.timestamp.tsa.valid-response` — verifies TSA returns valid timestamp tokens

View File

@@ -0,0 +1,65 @@
---
checkId: check.timestamp.tsa.chain-valid
plugin: stellaops.doctor.timestamping
severity: fail
tags: [timestamping, tsa, certificate, chain, validation]
---
# TSA Certificate Chain Validity
## What It Checks
Ensures TSA certificate chains are valid and complete. The check:
- Queries the certificate chain status provider for all configured TSA chains.
- Validates that each chain is complete (leaf to root) and has no errors.
- Fails if all chains are invalid. Warns if some chains are invalid.
- Reports degraded if no chain data is available (provider not configured).
## Why It Matters
An incomplete or broken certificate chain means TSA timestamps cannot be verified end-to-end. Relying parties will reject evidence with unverifiable chains, causing compliance audit failures and blocking release promotions. This is a critical-severity check.
## Common Causes
- Missing intermediate certificates
- Intermediate certificate expired
- Trust store not updated after CA changes
- Misconfigured certificate chain ordering
## How to Fix
### Docker Compose
```bash
# Verify chain manually
openssl verify -CAfile /certs/root.pem -untrusted /certs/intermediate.pem /certs/tsa-leaf.pem
# Update chain configuration
docker exec <platform-container> stella tsa chain update --name "Provider" \
--cert /certs/tsa-leaf.pem --intermediate /certs/intermediate.pem
```
### Bare Metal / systemd
```bash
stella tsa chain validate --all
stella tsa chain update --name "Provider" \
--cert /path/to/leaf.pem --intermediate /path/to/intermediate.pem
```
### Kubernetes / Helm
```yaml
timestamping:
chainValidation:
enabled: true
```
Update certificate chain secrets:
```bash
kubectl create secret generic tsa-chain \
--from-file=leaf.pem --from-file=intermediate.pem --from-file=root.pem
```
## Verification
```
stella doctor run --check check.timestamp.tsa.chain-valid
```
## Related Checks
- `check.timestamp.tsa.cert-expiry` — checks TSA signing certificate expiry
- `check.timestamp.tsa.root-expiry` — checks TSA root certificate expiry

View File

@@ -0,0 +1,71 @@
---
checkId: check.timestamp.tsa.failover-ready
plugin: stellaops.doctor.timestamping
severity: warn
tags: [timestamping, tsa, failover, redundancy]
---
# TSA Failover Readiness
## What It Checks
Confirms that backup TSA endpoints are reachable for failover. The check:
- Fails if no TSA endpoints are configured at all.
- Warns (degraded) if only one endpoint is configured -- failover is not possible with a single endpoint.
- Probes all configured endpoints and counts reachable ones.
- Compares reachable count against `MinHealthyTsas` (default 2).
- Fails or degrades if fewer than the minimum are reachable.
## Why It Matters
TSA providers can experience outages. Without backup endpoints, a single TSA failure blocks all timestamping operations, halting the evidence pipeline and release process. Failover readiness ensures the platform can automatically switch to an alternative TSA without manual intervention.
## Common Causes
- Only one TSA endpoint configured (no backup)
- Backup TSA endpoint down or unreachable
- Network issues to secondary TSA providers
## How to Fix
### Docker Compose
Configure at least two TSA endpoints:
```yaml
environment:
Timestamping__TsaEndpoints__0__Name: "Primary"
Timestamping__TsaEndpoints__0__Url: "https://freetsa.org/tsr"
Timestamping__TsaEndpoints__1__Name: "Backup"
Timestamping__TsaEndpoints__1__Url: "http://timestamp.digicert.com"
Timestamping__MinHealthyTsas: "2"
```
### Bare Metal / systemd
```json
{
"Timestamping": {
"TsaEndpoints": [
{ "Name": "Primary", "Url": "https://freetsa.org/tsr" },
{ "Name": "Backup", "Url": "http://timestamp.digicert.com" }
],
"MinHealthyTsas": 2
}
}
```
### Kubernetes / Helm
```yaml
timestamping:
minHealthyTsas: 2
tsaEndpoints:
- name: "Primary"
url: "https://freetsa.org/tsr"
- name: "Backup"
url: "http://timestamp.digicert.com"
```
## Verification
```
stella doctor run --check check.timestamp.tsa.failover-ready
```
## Related Checks
- `check.timestamp.tsa.reachable` — verifies TSA endpoint reachability
- `check.timestamp.tsa.response-time` — measures TSA response latency

View File

@@ -0,0 +1,65 @@
---
checkId: check.timestamp.tsa.response-time
plugin: stellaops.doctor.timestamping
severity: warn
tags: [timestamping, tsa, latency, performance]
---
# TSA Response Time
## What It Checks
Measures TSA endpoint response times against configurable thresholds. The check:
- Probes each configured TSA endpoint and measures round-trip latency.
- Compares latency against warning threshold (default 5000ms) and critical threshold (default 30000ms).
- Fails if any endpoint exceeds the critical latency threshold.
- Warns if any endpoint exceeds the warning threshold.
- Passes if all endpoints respond within acceptable latency.
- Reports degraded if no endpoints are configured.
## Why It Matters
High TSA latency slows down the evidence generation pipeline. Every release artifact that needs a timestamp will be delayed by slow TSA responses. In high-throughput environments, TSA latency can become a bottleneck that blocks the entire release pipeline.
## Common Causes
- TSA server under heavy load
- Network latency to remote TSA endpoints
- Firewall or proxy adding latency
- TSA provider experiencing service degradation
## How to Fix
### Docker Compose
Consider adding a geographically closer TSA endpoint or a local TSA:
```yaml
environment:
Timestamping__WarnLatencyMs: "5000"
Timestamping__CriticalLatencyMs: "30000"
```
### Bare Metal / systemd
```bash
# Test TSA latency manually
time curl -s -o /dev/null https://freetsa.org/tsr
# Add a faster TSA endpoint
stella tsa add --name "LocalTSA" --url "https://tsa.internal.example.com/tsr"
```
### Kubernetes / Helm
```yaml
timestamping:
warnLatencyMs: 5000
criticalLatencyMs: 30000
```
Consider deploying a local TSA proxy or cache to reduce latency.
## Verification
```
stella doctor run --check check.timestamp.tsa.response-time
```
## Related Checks
- `check.timestamp.tsa.reachable` — verifies TSA endpoints are reachable
- `check.timestamp.tsa.valid-response` — verifies valid RFC-3161 responses
- `check.timestamp.tsa.failover-ready` — confirms failover readiness

View File

@@ -0,0 +1,61 @@
---
checkId: check.timestamp.tsa.root-expiry
plugin: stellaops.doctor.timestamping
severity: warn
tags: [timestamping, tsa, root, certificate, expiry]
---
# TSA Root Certificate Expiry
## What It Checks
Checks if TSA trust anchor (root) certificates are approaching expiry. The check:
- Evaluates each root certificate in the `RootCertificates` configuration list.
- Calculates days remaining until expiry.
- **Fails** (unhealthy) if any root certificate is expired or within the critical threshold (default 180 days).
- **Warns** (degraded) if within the warning threshold (default 365 days).
- Uses longer thresholds than leaf certificates because root renewal requires more coordination.
## Why It Matters
Root certificates anchor the entire TSA trust chain. When a root expires, all timestamps signed by TSAs chained to that root become unverifiable. Root certificate renewal requires updating trust stores across the entire deployment, which takes significant lead time.
## Common Causes
- Root certificate approaching end-of-life (typically 10-20 year lifetime)
- Using a custom root CA with a shorter validity period
- Trust store not updated after provider rotated roots
## How to Fix
### Docker Compose
Update root certificate trust store:
```bash
# Update trust anchors
docker exec <platform-container> stella trust-anchor update --cert /certs/new-root.pem
```
### Bare Metal / systemd
```bash
# Update the trust anchor
stella trust-anchor update --cert /path/to/new-root.pem
# Or update the system trust store
sudo cp /path/to/new-root.pem /usr/local/share/ca-certificates/
sudo update-ca-certificates
```
### Kubernetes / Helm
```yaml
timestamping:
rootCertificates:
warnDays: 365
criticalDays: 180
```
## Verification
```
stella doctor run --check check.timestamp.tsa.root-expiry
```
## Related Checks
- `check.timestamp.tsa.cert-expiry` — checks TSA signing certificate expiry
- `check.timestamp.tsa.chain-valid` — validates TSA certificate chain integrity

View File

@@ -0,0 +1,50 @@
---
checkId: check.timestamp.timesync.tsa-skew
plugin: stellaops.doctor.timestamping
severity: warn
tags: [timestamping, timesync, tsa, skew]
---
# TSA Time Skew
## What It Checks
Checks time skew between the system clock and TSA genTime. The check:
- For each active TSA provider, requests a timestamp token with a random hash.
- Compares the TSA genTime against the local system clock, accounting for network round-trip time.
- **Fails** if skew exceeds the critical threshold (default 30 seconds).
- **Warns** if skew is elevated but below critical.
## Why It Matters
Time skew between the system and TSA indicates that either the local clock or the TSA clock is drifting. This can produce timestamps that appear out of order relative to other events, undermining the temporal integrity of release evidence and audit trails.
## Common Causes
- Local system clock not synchronized with NTP
- TSA provider clock drifting
- High network latency distorting round-trip time estimation
- Proxy or load balancer adding variable latency
## How to Fix
### Docker Compose
Ensure the host clock is synchronized (see `check.timestamp.timesync.system`). If the TSA shows consistent skew, consider using a different provider.
### Bare Metal / systemd
```bash
# Verify system time sync
chronyc tracking
# Test TSA response time
curl -w "@curl-format.txt" -s -o /dev/null https://freetsa.org/tsr
```
### Kubernetes / Helm
Verify node-level NTP synchronization. If a specific TSA consistently shows skew, switch to an alternative provider.
## Verification
```
stella doctor run --check check.timestamp.timesync.tsa-skew
```
## Related Checks
- `check.timestamp.timesync.system` — checks system clock synchronization with NTP
- `check.timestamp.timesync.rekor-correlation` — checks TST-Rekor time correlation

View File

@@ -0,0 +1,65 @@
---
checkId: check.timestamp.tsa.valid-response
plugin: stellaops.doctor.timestamping
severity: fail
tags: [timestamping, tsa, validation, rfc3161]
---
# TSA Valid Response
## What It Checks
Verifies that TSA endpoints return valid RFC-3161 timestamp responses. The check:
- Gets active TSA providers from the registry.
- Sends a dummy SHA-256 hash to each provider and requests a timestamp token.
- Validates that each response is a valid RFC-3161 timestamp token.
- Fails if no providers return valid responses. Warns if some providers fail validation.
- Reports degraded if no providers are configured.
## Why It Matters
A reachable TSA that returns invalid timestamps is worse than no TSA at all -- it produces evidence that appears valid but cannot be verified. Invalid timestamps break the chain of trust for release evidence and can cause compliance audit failures. This is a critical-severity check.
## Common Causes
- TSA provider configuration changed (algorithm, certificate)
- TSA provider returned an error response instead of a valid token
- Network issues causing corrupted responses
- TSA provider using an unsupported algorithm or format
## How to Fix
### Docker Compose
Verify TSA configuration and switch to a known-good provider:
```yaml
environment:
Timestamping__TsaEndpoints__0__Name: "DigiCert"
Timestamping__TsaEndpoints__0__Url: "http://timestamp.digicert.com"
```
### Bare Metal / systemd
```bash
# Test TSA response manually with openssl
openssl ts -query -data /dev/null -sha256 -cert -no_nonce -out /tmp/ts.req
curl -H "Content-Type: application/timestamp-query" --data-binary @/tmp/ts.req \
http://timestamp.digicert.com -o /tmp/ts.resp
openssl ts -reply -in /tmp/ts.resp -text
```
### Kubernetes / Helm
```yaml
timestamping:
tsaEndpoints:
- name: "DigiCert"
url: "http://timestamp.digicert.com"
```
If a TSA consistently returns invalid responses, remove it and add an alternative qualified provider.
## Verification
```
stella doctor run --check check.timestamp.tsa.valid-response
```
## Related Checks
- `check.timestamp.tsa.reachable` — verifies TSA endpoint reachability
- `check.timestamp.tsa.cert-expiry` — checks TSA certificate expiry
- `check.timestamp.tsa.chain-valid` — validates TSA certificate chain

View File

@@ -0,0 +1,35 @@
---
checkId: check.timestamp.evidence.tst.expiry
plugin: stellaops.doctor.timestamping
severity: warn
tags: [timestamping, evidence, tst, expiry]
---
# TST Approaching Expiry
## What It Checks
Detects timestamp tokens approaching signing certificate expiry. Fails if timestamps are within the critical window (default 90 days), warns if within the warning window (default 180 days).
## Why It Matters
Expired timestamp tokens cannot be validated by relying parties. Artifacts with expired timestamps lose their temporal proof, which may invalidate compliance evidence.
## Common Causes
- TSA signing certificates approaching end-of-life
- Re-timestamp jobs not scheduled or failing
## How to Fix
Run the retimestamp workflow to refresh expiring artifacts:
```bash
stella retimestamp run --expiring-within 180d
```
Schedule automatic re-timestamping before expiry.
## Verification
```
stella doctor run --check check.timestamp.evidence.tst.expiry
```
## Related Checks
- `check.timestamp.evidence.staleness` — aggregated evidence staleness check
- `check.timestamp.tsa.cert-expiry` — checks TSA certificate expiry

View File

@@ -0,0 +1,36 @@
---
checkId: check.timestamp.evidence.tst.deprecated-algo
plugin: stellaops.doctor.timestamping
severity: warn
tags: [timestamping, evidence, tst, algorithm, deprecated]
---
# TST Deprecated Algorithms
## What It Checks
Detects timestamps using deprecated hash algorithms (default: SHA1). Fails if the count exceeds the critical threshold (default 10), otherwise warns.
## Why It Matters
Timestamps using deprecated algorithms like SHA1 are vulnerable to collision attacks. Compliance frameworks (eIDAS, FIPS) may reject evidence signed with deprecated algorithms, blocking release attestation verification.
## Common Causes
- Legacy artifacts timestamped with older TSA configurations
- TSA provider still using SHA1 by default
- Migration to SHA-256 not yet completed
## How to Fix
Re-timestamp affected artifacts using approved algorithms:
```bash
stella retimestamp run --algorithm SHA256 --filter deprecated-algo
```
Ensure TSA providers are configured to use SHA-256 or stronger.
## Verification
```
stella doctor run --check check.timestamp.evidence.tst.deprecated-algo
```
## Related Checks
- `check.timestamp.evidence.staleness` — aggregated evidence staleness check
- `check.timestamp.tsa.valid-response` — verifies TSA returns valid responses

View File

@@ -0,0 +1,36 @@
---
checkId: check.timestamp.evidence.tst.missing-stapling
plugin: stellaops.doctor.timestamping
severity: warn
tags: [timestamping, evidence, tst, stapling, ocsp]
---
# TST Missing Stapling
## What It Checks
Detects timestamps without stapled OCSP/CRL revocation data. Fails if the count exceeds the critical threshold (default 10), otherwise warns.
## Why It Matters
Without stapled revocation data, verifiers must perform live OCSP/CRL lookups to confirm certificate validity. In air-gapped environments, these lookups are impossible, making the timestamp unverifiable. Stapling embeds proof-of-non-revocation directly in the timestamp token for offline verification.
## Common Causes
- TSA provider not configured to include stapled responses
- OCSP stapling disabled in TSA configuration
- Legacy timestamps created before stapling was enabled
## How to Fix
Enable OCSP stapling and re-timestamp affected artifacts:
```bash
stella retimestamp run --with-stapling --filter missing-stapling
```
Ensure TSA providers are configured with stapling enabled.
## Verification
```
stella doctor run --check check.timestamp.evidence.tst.missing-stapling
```
## Related Checks
- `check.timestamp.evidence.staleness` — aggregated evidence staleness check
- `check.timestamp.ocsp.stapling` — checks OCSP stapling configuration