Files
git.stella-ops.org/docs/features/dropped/metrics-for-attestation-coverage-and-time-to-evidence.md

3.4 KiB

Metrics for attestation coverage and time-to-evidence

Module

Unknowns

Status

PARTIALLY_IMPLEMENTED

Description

Some metrics services exist but the advisory noted metrics coverage at only 30%.

What's Implemented

  • Unknowns Metrics Service: src/Unknowns/StellaOps.Unknowns.Services/UnknownsMetricsService.cs -- exposes basic Prometheus/OpenTelemetry metrics for the unknowns queue including queue depth, resolution counts, and SLA breach counts.
  • Unknowns SLA Monitor Service: src/Unknowns/StellaOps.Unknowns.Services/UnknownsSlaMonitorService.cs -- monitors SLA compliance for unknown resolution timelines, providing partial time-to-evidence tracking for the unknowns domain.
  • Unknowns SLA Health Check: src/Unknowns/StellaOps.Unknowns.Services/UnknownsSlaHealthCheck.cs -- health check endpoint reporting unknowns SLA status, contributing to observability but not comprehensive attestation coverage metrics.
  • Unknown Ranking Model: src/Unknowns/__Libraries/StellaOps.Unknowns.Core/Models/UnknownRanking.cs -- ranking model that includes priority scoring which could inform metrics prioritization.

What's Missing

  • Attestation Coverage Metrics: No metrics track what percentage of release artifacts have complete attestation chains. The unknowns metrics cover queue health but not attestation completeness across the entire pipeline (e.g., "X% of images have SBOM + VEX + provenance attestations").
  • Time-to-Evidence Metrics: No end-to-end time-to-evidence metric exists tracking the duration from vulnerability discovery to complete evidence availability (scanner result -> reachability analysis -> VEX decision -> attestation). The SLA monitor tracks unknowns resolution time but not the broader evidence pipeline.
  • Per-Provider Coverage Breakdown: No metrics break down attestation coverage by provider/scanner (e.g., "Trivy provides SBOM for 95% of images, reachability analysis covers 60%").
  • Evidence Freshness Metrics: No metrics track evidence staleness (e.g., "SBOM is 30 days old, VEX decision is 7 days old") across the artifact estate.
  • Dashboard Integration: No pre-built Grafana dashboards or Web UI panels exist for visualizing attestation coverage and time-to-evidence trends.
  • Cross-Module Metrics Aggregation: Metrics are siloed per module (Unknowns, Attestor, EvidenceLocker); no aggregation layer combines them into a unified attestation coverage view.

Implementation Plan

  • Define attestation coverage metrics: per-artifact attestation completeness (SBOM present, VEX present, provenance present, reachability analysis present)
  • Implement a metrics aggregation service that queries Attestor, EvidenceLocker, and Unknowns to compute estate-wide attestation coverage percentages
  • Add time-to-evidence histogram metrics tracking the duration from vulnerability publication to complete evidence chain availability
  • Add per-provider coverage breakdown metrics
  • Add evidence freshness metrics (age of latest SBOM, VEX, provenance per artifact)
  • Build Grafana dashboard templates for attestation coverage and time-to-evidence visualization
  • Target: increase metrics coverage from 30% to 90%+ of the advisory specification
  • Unknowns metrics: src/Unknowns/StellaOps.Unknowns.Services/UnknownsMetricsService.cs
  • Attestor proof chain: src/Attestor/__Libraries/StellaOps.Attestor.ProofChain/
  • Evidence locker: src/EvidenceLocker/
  • VexLens (VEX processing): src/VexLens/