Triage Quality KPI Collector Infrastructure

Module

__Libraries

Status

IMPLEMENTED

Description

KpiCollector service for collecting triage quality metrics (false-positive rate, reachability coverage, explainability score, etc.) with multi-domain KPI collection. Measures quality outcomes across reachability, runtime, explainability, replay, unknown budgets, and operational dimensions. Distinct from existing TTE/TTFS metrics which measure timing; this measures triage quality outcomes.

Implementation Details

KpiCollector: src/__Libraries/StellaOps.Metrics/Kpi/KpiCollector.cs -- implements IKpiCollector; CollectAsync() orchestrates collection across all KPI domains: reachability KPIs (coverage %, false-positive rate, graph completeness), runtime KPIs (observation count, runtime coverage, staleness), explainability KPIs (score distribution, citation coverage, explanation completeness), replay KPIs (eligibility rate, success rate, determinism score), unknown budget KPIs (budget utilization, exception count, drift rate), operational KPIs (scan latency, throughput, error rate)
IKpiCollector: src/__Libraries/StellaOps.Metrics/Kpi/KpiCollector.cs -- interface: CollectAsync(), RecordReachabilityResultAsync(result), RecordRuntimeObservationAsync(observation), RecordVerdictAsync(verdict), RecordReplayAttemptAsync(attempt)
RecordReachabilityResultAsync: records reachability analysis outcomes for coverage tracking
RecordRuntimeObservationAsync: records runtime observation data for runtime KPI computation
RecordVerdictAsync: records verdict outcomes for quality KPI computation
RecordReplayAttemptAsync: records replay attempt results for replay success/eligibility metrics
Source: SPRINT_7000_0005_0001_quality_kpis_tracking.md

E2E Test Plan

Verify KpiCollector.CollectAsync aggregates KPIs across all domains
Test reachability KPIs: coverage percentage, false-positive rate, graph completeness
Verify runtime KPIs: observation count, runtime coverage, staleness detection
Test explainability KPIs: score distribution, citation coverage, explanation completeness
Verify replay KPIs: eligibility rate, success rate, determinism score
Test unknown budget KPIs: budget utilization, exception count, drift rate
Verify RecordReachabilityResultAsync/RecordVerdictAsync accumulate data for collection
Test operational KPIs: scan latency, throughput, error rate

2.4 KiB Raw Blame History