semi implemented and features implemented save checkpoint

This commit is contained in:
master
2026-02-08 18:00:49 +02:00
parent 04360dff63
commit 1bf6bbf395
20895 changed files with 716795 additions and 64 deletions

View File

@@ -0,0 +1,28 @@
# Structured Provenance Hints for Unknowns
## Module
Unknowns
## Status
IMPLEMENTED
## Description
Structured provenance hint system for unknown binaries/components with typed hints (BuildIdMatch, DebugLink, ImportTableFingerprint, ExportTableFingerprint, SectionLayout, CompilerSignature, DistroPattern, VersionString, SymbolPattern), confidence scoring, and hypothesis generation for resolution (e.g., "Binary matches distro build-ID, likely backport").
## Implementation Details
- **Provenance Hint Builder**: `src/Unknowns/__Libraries/StellaOps.Unknowns.Core/Hints/ProvenanceHintBuilder.cs` -- fluent builder for constructing typed provenance hints with confidence scores; supports chaining multiple hint sources (build ID, debug link, import table, section layout) into a ranked hypothesis list.
- **IProvenanceHintBuilder Interface**: `src/Unknowns/__Libraries/StellaOps.Unknowns.Core/Hints/IProvenanceHintBuilder.cs` -- interface for the provenance hint builder, enabling dependency injection and testability.
- **Provenance Hint Model**: `src/Unknowns/__Libraries/StellaOps.Unknowns.Core/Models/ProvenanceHint.cs` -- data model for a single provenance hint containing hint type, source evidence, confidence score, and generated hypothesis text.
- **Provenance Hint Type**: `src/Unknowns/__Libraries/StellaOps.Unknowns.Core/Models/ProvenanceHintType.cs` -- enum defining all supported hint types: BuildIdMatch, DebugLink, ImportTableFingerprint, ExportTableFingerprint, SectionLayout, CompilerSignature, DistroPattern, VersionString, SymbolPattern.
- **Provenance Evidence**: `src/Unknowns/__Libraries/StellaOps.Unknowns.Core/Models/ProvenanceEvidence.cs` -- evidence payload associated with a provenance hint (e.g., the matched build ID string, the fingerprint hash, the compiler version string).
- **Native Unknown Context**: `src/Unknowns/__Libraries/StellaOps.Unknowns.Core/Models/NativeUnknownContext.cs` -- context model for native (C/C++/Rust/Go) unknown binaries, providing the binary analysis data that hint builders consume.
- **Native Unknown Classifier**: `src/Unknowns/__Libraries/StellaOps.Unknowns.Core/Services/NativeUnknownClassifier.cs` -- classifies unknown binaries by running all available hint builders and ranking hypotheses by confidence score.
- **Unknown Proof Emitter**: `src/Unknowns/__Libraries/StellaOps.Unknowns.Core/Services/UnknownProofEmitter.cs` -- emits attestation proofs for resolved unknowns, linking the provenance hints that led to identification.
## E2E Test Plan
- [ ] Build a provenance hint for a binary with a known GNU build-ID, invoke `ProvenanceHintBuilder` with BuildIdMatch type, and verify the hint contains the correct build ID, a confidence score > 0.8, and a hypothesis string mentioning the matched package
- [ ] Build hints for a binary with multiple evidence sources (build ID + section layout + compiler signature), and verify `NativeUnknownClassifier` ranks them by descending confidence score
- [ ] Submit a `NativeUnknownContext` for a binary with no matching evidence and verify the classifier returns an empty hint list with no false-positive hypotheses
- [ ] Build a DistroPattern hint for a binary matching a known distro build pattern (e.g., Debian hardening flags) and verify the hypothesis correctly identifies the distribution
- [ ] Resolve an unknown binary using provenance hints and verify `UnknownProofEmitter` produces an attestation proof linking the hints to the resolution decision
- [ ] Verify all `ProvenanceHintType` enum values have corresponding builder paths in `ProvenanceHintBuilder` by constructing one hint of each type and confirming no errors

View File

@@ -0,0 +1,26 @@
# Unknowns SLA Monitoring
## Module
Unknowns
## Status
IMPLEMENTED
## Description
SLA monitoring for unknowns tracking resolution timelines and health checks for unknown queue items.
## Implementation Details
- **Unknowns SLA Monitor Service**: `src/Unknowns/StellaOps.Unknowns.Services/UnknownsSlaMonitorService.cs` -- background service that periodically checks unknown queue items against configured SLA thresholds (time-to-triage, time-to-resolution); raises alerts for SLA breaches.
- **Unknowns SLA Health Check**: `src/Unknowns/StellaOps.Unknowns.Services/UnknownsSlaHealthCheck.cs` -- ASP.NET health check that reports SLA compliance status; returns degraded/unhealthy when unknowns exceed SLA thresholds, enabling integration with orchestrator health monitoring.
- **Unknowns Metrics Service**: `src/Unknowns/StellaOps.Unknowns.Services/UnknownsMetricsService.cs` -- exposes Prometheus/OpenTelemetry metrics for unknown queue depth, average resolution time, SLA breach count, and hint coverage percentage.
- **Grey Queue Watchdog Service**: `src/Unknowns/StellaOps.Unknowns.Services/GreyQueueWatchdogService.cs` -- monitors the grey queue (unknowns awaiting classification) and escalates items that have been pending beyond the configured watchdog timeout.
- **Unknowns Lifecycle Service**: `src/Unknowns/StellaOps.Unknowns.Services/UnknownsLifecycleService.cs` -- manages the full lifecycle of unknown items from ingestion through classification, hint gathering, resolution, and archival; tracks state transitions for SLA timing.
- **Grey Queue Entry Model**: `src/Unknowns/__Libraries/StellaOps.Unknowns.Core/Models/GreyQueueEntry.cs` -- data model for grey queue entries including creation timestamp, last activity timestamp, and SLA deadline fields used by the monitor.
## E2E Test Plan
- [ ] Enqueue an unknown item, let the `UnknownsSlaMonitorService` run its check cycle, and verify the item is reported as within SLA when the elapsed time is below the threshold
- [ ] Enqueue an unknown item with an artificially past creation timestamp (exceeding the SLA threshold), run the monitor, and verify an SLA breach alert is raised
- [ ] Query the `UnknownsSlaHealthCheck` endpoint when all unknowns are within SLA and verify it returns `Healthy`; then introduce an SLA breach and verify it returns `Degraded`
- [ ] Verify `UnknownsMetricsService` exposes correct Prometheus metrics: enqueue an item, resolve it, and verify `unknown_resolution_time_seconds` histogram records the elapsed time
- [ ] Enqueue a grey queue item, let `GreyQueueWatchdogService` run, and verify the item is escalated when it exceeds the watchdog timeout
- [ ] Track an unknown item through its full lifecycle via `UnknownsLifecycleService` (ingestion -> classification -> resolution) and verify SLA timestamps are recorded at each state transition