stella-ops.org/git.stella-ops.org
1
0
Fork 0
Code Issues Pull Requests Actions 12 Releases Wiki Activity

feat(metrics): Implement scan metrics repository and PostgreSQL integration
Some checks failed
Docs CI / lint-and-preview (push) Has been cancelled
Details

Browse Source 
- Added IScanMetricsRepository interface for scan metrics persistence and retrieval.
- Implemented PostgresScanMetricsRepository for PostgreSQL database interactions, including methods for saving and retrieving scan metrics and execution phases.
- Introduced methods for obtaining TTE statistics and recent scans for tenants.
- Implemented deletion of old metrics for retention purposes.

test(tests): Add SCA Failure Catalogue tests for FC6-FC10

- Created ScaCatalogueDeterminismTests to validate determinism properties of SCA Failure Catalogue fixtures.
- Developed ScaFailureCatalogueTests to ensure correct handling of specific failure modes in the scanner.
- Included tests for manifest validation, file existence, and expected findings across multiple failure cases.

feat(telemetry): Integrate scan completion metrics into the pipeline

- Introduced IScanCompletionMetricsIntegration interface and ScanCompletionMetricsIntegration class to record metrics upon scan completion.
- Implemented proof coverage and TTE metrics recording with logging for scan completion summaries.
This commit is contained in:
master
2025-12-16 14:00:35 +02:00
parent b55d9fa68d
commit 415eff1207
27 changed files with 3620 additions and 35 deletions
Download Patch File Download Diff File Expand all files Collapse all files

10
docs/implplan/SPRINT_0350_0001_0001_ci_quality_gates_foundation.md
View File

@@ -42,14 +42,14 @@ Read before implementation:
|---|---------|--------|---------------------------|--------|-----------------|
| 1 | QGATE-0350-001 | DONE | None | Platform | Create `scripts/ci/compute-reachability-metrics.sh` to compute recall/precision from corpus |
| 2 | QGATE-0350-002 | DONE | After #1 | Platform | Create `scripts/ci/reachability-thresholds.yaml` with enforcement thresholds |
| 3 | QGATE-0350-003 | TODO | After #2 | Platform | Add reachability gate job to `build-test-deploy.yml` |
| 3 | QGATE-0350-003 | DONE | After #2 | Platform | Add reachability gate job to `build-test-deploy.yml` |
| 4 | QGATE-0350-004 | DONE | None | Platform | Create `scripts/ci/compute-ttfs-metrics.sh` to extract TTFS from test runs |
| 5 | QGATE-0350-005 | DONE | After #4 | Platform | Create `bench/baselines/ttfs-baseline.json` with p50/p95 targets |
| 6 | QGATE-0350-006 | TODO | After #5 | Platform | Add TTFS regression gate to `build-test-deploy.yml` |
| 6 | QGATE-0350-006 | DONE | After #5 | Platform | Add TTFS regression gate to `build-test-deploy.yml` |
| 7 | QGATE-0350-007 | DONE | None | Platform | Create `scripts/ci/enforce-performance-slos.sh` for scan/compute SLOs |
| 8 | QGATE-0350-008 | TODO | After #7 | Platform | Add performance SLO gate to `build-test-deploy.yml` |
| 9 | QGATE-0350-009 | TODO | After #3, #6, #8 | Platform | Create `docs/testing/ci-quality-gates.md` documentation |
| 10 | QGATE-0350-010 | TODO | After #9 | Platform | Add quality gate status badges to repository README |
| 8 | QGATE-0350-008 | DONE | After #7 | Platform | Add performance SLO gate to `build-test-deploy.yml` |
| 9 | QGATE-0350-009 | DONE | After #3, #6, #8 | Platform | Create `docs/testing/ci-quality-gates.md` documentation |
| 10 | QGATE-0350-010 | DONE | After #9 | Platform | Add quality gate status badges to repository README |
## Wave Coordination

4
docs/implplan/SPRINT_0351_0001_0001_sca_failure_catalogue_completion.md
View File

@@ -68,9 +68,9 @@ The SCA Failure Catalogue covers real-world scanner failure modes that have occu
| 5 | SCA-0351-005 | DONE | None | Scanner | Create FC10 fixture: CVE Split/Merge failure case |
| 6 | SCA-0351-006 | DONE | After #1-5 | Scanner | Create DSSE manifests for all new fixtures |
| 7 | SCA-0351-007 | DONE | After #6 | Scanner | Update `tests/fixtures/sca/catalogue/inputs.lock` |
| 8 | SCA-0351-008 | TODO | After #7 | Scanner | Add xUnit tests for FC6-FC10 in Scanner test project |
| 8 | SCA-0351-008 | DONE | After #7 | Scanner | Add xUnit tests for FC6-FC10 in Scanner test project |
| 9 | SCA-0351-009 | DONE | After #8 | Scanner | Update `tests/fixtures/sca/catalogue/README.md` documentation |
| 10 | SCA-0351-010 | TODO | After #9 | Scanner | Validate all fixtures pass determinism checks |
| 10 | SCA-0351-010 | DONE | After #9 | Scanner | Validate all fixtures pass determinism checks |
## Wave Coordination

28
docs/implplan/SPRINT_0501_0002_0001_proof_chain_content_addressed_ids.md
View File

@@ -379,20 +379,20 @@ public interface ISubjectExtractor
| # | Task ID | Status | Key Dependency / Next Step | Owners | Task Definition |
|---|---------|--------|---------------------------|--------|-----------------|
| 1 | PROOF-ID-0001 | DOING | None | Attestor Guild | Create `StellaOps.Attestor.ProofChain` library project structure |
| 2 | PROOF-ID-0002 | DOING | Task 1 | Attestor Guild | Implement `ContentAddressedId` base record and derived types |
| 3 | PROOF-ID-0003 | DOING | Task 1 | Attestor Guild | Implement `IJsonCanonicalizer` per RFC 8785 |
| 4 | PROOF-ID-0004 | DOING | Task 3 | Attestor Guild | Implement `IContentAddressedIdGenerator` for EvidenceID |
| 5 | PROOF-ID-0005 | DOING | Task 3 | Attestor Guild | Implement `IContentAddressedIdGenerator` for ReasoningID |
| 6 | PROOF-ID-0006 | DOING | Task 3 | Attestor Guild | Implement `IContentAddressedIdGenerator` for VEXVerdictID |
| 7 | PROOF-ID-0007 | DOING | Task 1 | Attestor Guild | Implement `IMerkleTreeBuilder` for deterministic merkle construction |
| 8 | PROOF-ID-0008 | DOING | Task 4-7 | Attestor Guild | Implement `IContentAddressedIdGenerator` for ProofBundleID |
| 9 | PROOF-ID-0009 | DOING | Task 7 | Attestor Guild | Implement `IContentAddressedIdGenerator` for GraphRevisionID |
| 10 | PROOF-ID-0010 | DOING | Task 3 | Attestor Guild | Implement `SbomEntryId` computation from SBOM + PURL |
| 11 | PROOF-ID-0011 | DOING | Task 1 | Attestor Guild | Implement `ISubjectExtractor` for CycloneDX SBOMs |
| 12 | PROOF-ID-0012 | DOING | Task 1 | Attestor Guild | Create all predicate record types (Evidence, Reasoning, VEX, ProofSpine) |
| 13 | PROOF-ID-0013 | TODO | Task 2-12 | QA Guild | Unit tests for all ID generation (determinism verification) |
| 14 | PROOF-ID-0014 | TODO | Task 13 | QA Guild | Property-based tests for canonicalization stability |
| 1 | PROOF-ID-0001 | DONE | None | Attestor Guild | Create `StellaOps.Attestor.ProofChain` library project structure |
| 2 | PROOF-ID-0002 | DONE | Task 1 | Attestor Guild | Implement `ContentAddressedId` base record and derived types |
| 3 | PROOF-ID-0003 | DONE | Task 1 | Attestor Guild | Implement `IJsonCanonicalizer` per RFC 8785 |
| 4 | PROOF-ID-0004 | DONE | Task 3 | Attestor Guild | Implement `IContentAddressedIdGenerator` for EvidenceID |
| 5 | PROOF-ID-0005 | DONE | Task 3 | Attestor Guild | Implement `IContentAddressedIdGenerator` for ReasoningID |
| 6 | PROOF-ID-0006 | DONE | Task 3 | Attestor Guild | Implement `IContentAddressedIdGenerator` for VEXVerdictID |
| 7 | PROOF-ID-0007 | DONE | Task 1 | Attestor Guild | Implement `IMerkleTreeBuilder` for deterministic merkle construction |
| 8 | PROOF-ID-0008 | DONE | Task 4-7 | Attestor Guild | Implement `IContentAddressedIdGenerator` for ProofBundleID |
| 9 | PROOF-ID-0009 | DONE | Task 7 | Attestor Guild | Implement `IContentAddressedIdGenerator` for GraphRevisionID |
| 10 | PROOF-ID-0010 | DONE | Task 3 | Attestor Guild | Implement `SbomEntryId` computation from SBOM + PURL |
| 11 | PROOF-ID-0011 | DONE | Task 1 | Attestor Guild | Implement `ISubjectExtractor` for CycloneDX SBOMs |
| 12 | PROOF-ID-0012 | DONE | Task 1 | Attestor Guild | Create all predicate record types (Evidence, Reasoning, VEX, ProofSpine) |
| 13 | PROOF-ID-0013 | DONE | Task 2-12 | QA Guild | Unit tests for all ID generation (determinism verification) |
| 14 | PROOF-ID-0014 | DONE | Task 13 | QA Guild | Property-based tests for canonicalization stability |
| 15 | PROOF-ID-0015 | TODO | Task 13 | Docs Guild | Document ID format specifications in module architecture |
## Test Specifications

6
docs/implplan/SPRINT_3401_0001_0001_determinism_scoring_foundations.md
View File

@@ -33,17 +33,17 @@ Implement high-value, low-effort scoring enhancements from the Determinism and R
|---|---------|--------|---------------------------|--------|-----------------|
| 1 | DET-3401-001 | DONE | None | Scoring Team | Define `FreshnessBucket` record and `FreshnessMultiplierConfig` in Policy.Scoring |
| 2 | DET-3401-002 | DONE | After #1 | Scoring Team | Implement `EvidenceFreshnessCalculator` service with basis-points multipliers |
| 3 | DET-3401-003 | TODO | After #2 | Scoring Team | Integrate freshness multiplier into existing evidence scoring pipeline |
| 3 | DET-3401-003 | DONE | After #2 | Scoring Team | Integrate freshness multiplier into existing evidence scoring pipeline |
| 4 | DET-3401-004 | DONE | After #3 | Scoring Team | Add unit tests for freshness buckets (7d, 30d, 90d, 180d, 365d, >365d) |
| 5 | DET-3401-005 | DONE | None | Telemetry Team | Define `ProofCoverageMetrics` class with Prometheus counters/gauges |
| 6 | DET-3401-006 | DONE | After #5 | Telemetry Team | Implement `proof_coverage_all`, `proof_coverage_vex`, `proof_coverage_reachable` gauges |
| 7 | DET-3401-007 | TODO | After #6 | Telemetry Team | Add proof coverage calculation to scan completion pipeline |
| 7 | DET-3401-007 | DONE | After #6 | Telemetry Team | Add proof coverage calculation to scan completion pipeline |
| 8 | DET-3401-008 | DONE | After #7 | Telemetry Team | Add unit tests for proof coverage ratio calculations |
| 9 | DET-3401-009 | DONE | None | Scoring Team | Define `ScoreExplanation` record with factor/value/reason structure |
| 10 | DET-3401-010 | DONE | After #9 | Scoring Team | Implement `ScoreExplainBuilder` to accumulate explanations during scoring |
| 11 | DET-3401-011 | DONE | After #10 | Scoring Team | Refactor `RiskScoringResult` to include `Explain` array |
| 12 | DET-3401-012 | DONE | After #11 | Scoring Team | Add unit tests for explanation generation |
| 13 | DET-3401-013 | TODO | After #4, #8, #12 | QA | Integration tests: freshness + proof coverage + explain in full scan |
| 13 | DET-3401-013 | DONE | After #4, #8, #12 | QA | Integration tests: freshness + proof coverage + explain in full scan |
## Wave Coordination

19
docs/implplan/SPRINT_3406_0001_0001_metrics_tables.md
View File

@@ -30,22 +30,23 @@ Implement relational PostgreSQL tables for scan metrics tracking (hybrid approac
| # | Task ID | Status | Key dependency / next step | Owners | Task Definition |
|---|---------|--------|---------------------------|--------|-----------------|
| 1 | METRICS-3406-001 | TODO | None | DB Team | Create `scan_metrics` table migration |
| 2 | METRICS-3406-002 | TODO | After #1 | DB Team | Create `execution_phases` table for timing breakdown |
| 3 | METRICS-3406-003 | TODO | After #1 | DB Team | Create `scan_tte` view for TTE calculation |
| 4 | METRICS-3406-004 | TODO | After #1 | DB Team | Create indexes for metrics queries |
| 5 | METRICS-3406-005 | TODO | None | Scanner Team | Define `ScanMetrics` entity and `ExecutionPhase` record |
| 6 | METRICS-3406-006 | TODO | After #1, #5 | Scanner Team | Implement `IScanMetricsRepository` interface |
| 7 | METRICS-3406-007 | TODO | After #6 | Scanner Team | Implement `PostgresScanMetricsRepository` |
| 8 | METRICS-3406-008 | TODO | After #7 | Scanner Team | Implement `ScanMetricsCollector` service |
| 1 | METRICS-3406-001 | DONE | None | DB Team | Create `scan_metrics` table migration |
| 2 | METRICS-3406-002 | DONE | After #1 | DB Team | Create `execution_phases` table for timing breakdown |
| 3 | METRICS-3406-003 | DONE | After #1 | DB Team | Create `scan_tte` view for TTE calculation |
| 4 | METRICS-3406-004 | DONE | After #1 | DB Team | Create indexes for metrics queries |
| 5 | METRICS-3406-005 | DONE | None | Scanner Team | Define `ScanMetrics` entity and `ExecutionPhase` record |
| 6 | METRICS-3406-006 | DONE | After #1, #5 | Scanner Team | Implement `IScanMetricsRepository` interface |
| 7 | METRICS-3406-007 | DONE | After #6 | Scanner Team | Implement `PostgresScanMetricsRepository` |
| 8 | METRICS-3406-008 | DONE | After #7 | Scanner Team | Implement `ScanMetricsCollector` service |
| 9 | METRICS-3406-009 | TODO | After #8 | Scanner Team | Integrate collector into scan completion pipeline |
| 10 | METRICS-3406-010 | TODO | After #3 | Telemetry Team | Export TTE percentiles to Prometheus |
| 11 | METRICS-3406-011 | TODO | After #7 | Scanner Team | Unit tests for repository operations |
| 12 | METRICS-3406-012 | TODO | After #9 | QA | Integration test: metrics captured on scan completion |
| 13 | METRICS-3406-013 | TODO | After #3 | Docs Guild | Document metrics schema in `docs/db/schemas/scan-metrics.md` |
| 13 | METRICS-3406-013 | DONE | After #3 | Docs Guild | Document metrics schema in `docs/db/schemas/scan-metrics.md` |
## Wave Coordination
- **Wave 1** (Parallel): Tasks #1-5 (Schema + Models)
- **Wave 2** (Sequential): Tasks #6-9 (Repository + Collector + Integration)
- **Wave 3** (Parallel): Tasks #10-13 (Telemetry + Tests + Docs)

4
docs/implplan/SPRINT_3423_0001_0001_generated_columns.md
View File

@@ -75,8 +75,8 @@ Benefits:
| 4.6 | Verify query plans | DONE | | |
| 4.7 | Integration tests | DONE | | Via runbook validation |
| **Phase 5: Documentation** |||||
| 5.1 | Update SPECIFICATION.md with generated column pattern | TODO | | |
| 5.2 | Add generated column guidelines to RULES.md | TODO | | |
| 5.1 | Update SPECIFICATION.md with generated column pattern | DONE | | Added Section 6.4 |
| 5.2 | Add generated column guidelines to RULES.md | DONE | | Added Section 5.3.1 |
| 5.3 | Document query optimization gains | DONE | | postgresql-patterns-runbook.md |
---