save checkpoint: save features

This commit is contained in:
master
2026-02-12 10:27:23 +02:00
parent dca86e1248
commit 5bca406787
8837 changed files with 1796879 additions and 5294 deletions

View File

@@ -0,0 +1,39 @@
# Materialized Views for Analytics
## Module
Platform
## Status
VERIFIED
## Description
Materialized views with indexes, VEX validity filters, and deterministic arrays are used for analytics with a dedicated maintenance service for refresh.
## Implementation Details
- **PlatformAnalyticsMaintenanceService**: `src/Platform/StellaOps.Platform.WebService/Services/PlatformAnalyticsMaintenanceService.cs` -- BackgroundService that periodically refreshes 4 materialized views (`mv_supplier_concentration`, `mv_license_distribution`, `mv_vuln_exposure`, `mv_attestation_coverage`) using `REFRESH MATERIALIZED VIEW CONCURRENTLY`; supports daily rollup backfill, configurable interval, and run-on-startup mode via `PlatformAnalyticsMaintenanceOptions`
- **PlatformAnalyticsQueryExecutor**: `src/Platform/StellaOps.Platform.WebService/Services/PlatformAnalyticsQueryExecutor.cs` -- executes analytical queries against materialized views
- **PlatformAnalyticsService**: `src/Platform/StellaOps.Platform.WebService/Services/PlatformAnalyticsService.cs` -- service layer for analytics queries (suppliers, licenses, vulnerabilities, backlog, attestation coverage, vulnerability/component trends)
- **AnalyticsEndpoints**: `src/Platform/StellaOps.Platform.WebService/Endpoints/AnalyticsEndpoints.cs` -- REST API at `/api/analytics` with 7 endpoints: suppliers, licenses, vulnerabilities, backlog, attestation-coverage, trends/vulnerabilities, trends/components; all require `AnalyticsRead` authorization
- **Source**: Feature matrix scan
## E2E Test Plan
- [ ] Verify materialized views refresh concurrently without blocking reads
- [ ] Test daily rollup backfill covers configured BackfillDays range
- [ ] Verify analytics endpoints return correct data from materialized views
- [ ] Test analytics service returns 503 when analytics storage is not configured
- [ ] Verify trend endpoints return daily data points for specified time windows
## Verification
- Verified on 2026-02-11 via `run-001`.
- Tier 0 source checks passed for maintenance, query executor, analytics service, and analytics endpoints.
- Tier 1 build + targeted test gates passed across webservice and analytics projects.
- Tier 2 behavioral checks passed with fresh user-surface/API and integration evidence:
- backfill + `REFRESH MATERIALIZED VIEW CONCURRENTLY` maintenance path (`PlatformAnalyticsMaintenanceServiceTests`)
- unconfigured endpoint path returns `503` (`AnalyticsEndpointsTests`, 7 routes)
- configured endpoint success payloads and trend points (`AnalyticsEndpointsSuccessTests`)
- analytics service cache/normalization behavior for suppliers, vulnerabilities, backlog, attestation coverage, and trend windows (`PlatformAnalyticsServiceTests`)
- materialized-view + stored-procedure integration behavior in Docker PostgreSQL (`AnalyticsSchemaIntegrationTests`, 22 tests)
- Evidence:
- `docs/qa/feature-checks/runs/platform/materialized-views-for-analytics/run-001/tier0-source-check.json`
- `docs/qa/feature-checks/runs/platform/materialized-views-for-analytics/run-001/tier1-build-check.json`
- `docs/qa/feature-checks/runs/platform/materialized-views-for-analytics/run-001/tier2-integration-check.json`

View File

@@ -0,0 +1,38 @@
# Platform Service Aggregation Layer
## Module
Platform
## Status
VERIFIED
## Description
Backend Platform Service acting as aggregation layer for health status, quotas, onboarding progress, user preferences, and global search across all modules.
## Implementation Details
- **PlatformEndpoints**: `src/Platform/StellaOps.Platform.WebService/Endpoints/PlatformEndpoints.cs` -- REST API at `/api/v1/platform` with 6 endpoint groups: health (summary/dependencies/incidents/metrics), quotas (summary/tenants/alerts), onboarding (status/complete/skip), preferences (dashboard/profiles), search, metadata; all with tenant-scoped authorization policies
- **PlatformHealthService**: `src/Platform/StellaOps.Platform.WebService/Services/PlatformHealthService.cs` -- aggregates health status from all platform services
- **PlatformQuotaService**: service for quota tracking with alert management (create alert thresholds per tenant)
- **PlatformOnboardingService**: `src/Platform/StellaOps.Platform.WebService/Services/PlatformOnboardingService.cs` -- tracks onboarding progress with step completion and skip support
- **PlatformPreferencesService**: `src/Platform/StellaOps.Platform.WebService/Services/PlatformPreferencesService.cs` -- user dashboard preferences with profile management (CRUD)
- **PlatformSearchService**: global search across all modules with source filtering, pagination
- **PlatformMetadataService**: `src/Platform/StellaOps.Platform.WebService/Services/PlatformMetadataService.cs` -- platform metadata retrieval
- **PlatformCache**: `src/Platform/StellaOps.Platform.WebService/Services/PlatformCache.cs` -- caching layer with TTL and data-as-of timestamps
- **Source**: SPRINT_20251229_043_PLATFORM_platform_service_foundation
## E2E Test Plan
- [ ] Verify health summary endpoint aggregates all service statuses
- [ ] Test quota alerts are created and returned with proper authorization
- [ ] Verify onboarding step completion transitions correctly
- [ ] Test global search returns results from multiple sources with pagination
- [ ] Verify dashboard preferences persist and load per tenant/user
## Verification
- Verified on 2026-02-11 with `run-001`.
- Tier 0 source checks confirmed aggregation endpoint + service/cache implementation parity.
- Tier 1 build and endpoint test suite passed (98/98) after adding quota alert and search pagination/filter coverage.
- Tier 2 API behavior checks passed for health, quota (positive + negative), onboarding, preferences, and search routes using live HTTP request/response transactions against `http://127.0.0.1:10011`.
- Evidence:
- `docs/qa/feature-checks/runs/platform/platform-service-aggregation-layer/run-001/tier0-source-check.json`
- `docs/qa/feature-checks/runs/platform/platform-service-aggregation-layer/run-001/tier1-build-check.json`
- `docs/qa/feature-checks/runs/platform/platform-service-aggregation-layer/run-001/tier2-api-check.json`

View File

@@ -0,0 +1,36 @@
# Platform Setup Wizard Backend API
## Module
Platform
## Status
VERIFIED
## Description
Real /api/v1/setup/* endpoints replacing UI mocks with deterministic session state (create, resume, execute, skip, finalize), tenant scoping, and offline-first "data as of" metadata.
## Implementation Details
- **SetupEndpoints**: `src/Platform/StellaOps.Platform.WebService/Endpoints/SetupEndpoints.cs` -- REST API at `/api/v1/setup` with 3 endpoint groups: sessions (GET current, POST create, POST resume, POST finalize), steps (POST execute, POST skip), definitions (GET step definitions); AllowAnonymous during initial setup, requires auth after completion
- **SetupStateDetector**: detects setup completion state from storage/DB settings; routes between bootstrap context and authenticated context
- **PlatformSetupService**: service layer for setup wizard operations (CreateSessionAsync, ResumeOrCreateSessionAsync, ExecuteStepAsync, SkipStepAsync, FinalizeSessionAsync, GetStepDefinitionsAsync)
- **SetupWizardModels**: `src/Platform/StellaOps.Platform.WebService/Contracts/SetupWizardModels.cs` -- request/response models (CreateSetupSessionRequest, SetupSessionResponse, ExecuteSetupStepRequest, SkipSetupStepRequest, FinalizeSetupSessionRequest, FinalizeSetupSessionResponse, SetupStepDefinitionsResponse)
- **Problem+JSON errors**: all endpoints return RFC 7807 ProblemDetails on errors
- **Source**: SPRINT_20260112_004_PLATFORM_setup_wizard_backend.md
## E2E Test Plan
- [ ] Verify setup session creates with bootstrap context when auth is unavailable
- [ ] Test session resume returns existing session or creates new one
- [ ] Verify step execution updates session state correctly
- [ ] Test step skip marks step as skipped and advances session
- [ ] Verify finalize completes setup and subsequent requests require authentication
- [ ] Test step definitions endpoint returns all available setup steps
## Verification
- Verified on 2026-02-11 with run-001 Tier 0/1/2 evidence.
- Tier 1 build/test pass: platform webservice + platform webservice tests (100/100).
- Tier 2 setup API behavior checks pass (7/7 setup-focused class tests) covering create/resume/execute/skip/finalize/definitions and required-step skip negative path.
- Evidence:
- `docs/qa/feature-checks/runs/platform/platform-setup-wizard-backend-api/run-001/tier0-source-check.json`
- `docs/qa/feature-checks/runs/platform/platform-setup-wizard-backend-api/run-001/tier1-build-check.json`
- `docs/qa/feature-checks/runs/platform/platform-setup-wizard-backend-api/run-001/tier2-api-check.json`

View File

@@ -0,0 +1,38 @@
# SBOM Analytics Lake (Star-Schema PostgreSQL)
## Module
Platform
## Status
VERIFIED
## Description
Star-schema PostgreSQL analytics layer for SBOM data with component registry, vulnerability correlation tables, attestation tracking, materialized views for trend analysis, and stored procedures for analytics queries. While "Materialized Views for Analytics" is in the known list, this is a much broader star-schema analytics subsystem with dedicated migration, ingestion services, and multi-table analytics design.
## Implementation Details
- **AnalyticsIngestionService**: `src/Platform/StellaOps.Platform.Analytics/Services/AnalyticsIngestionService.cs` -- BackgroundService subscribing to orchestrator scanner-report-ready events (`OrchestratorEventKinds.ScannerReportReady` / `scanner.event.report.ready`) via `IEventStream`; parses SBOM (CycloneDX/SPDX), resolves artifact digests, upserts into star-schema tables (`analytics.artifacts`, `analytics.raw_sboms`, `analytics.components`, `analytics.artifact_components`), and builds dependency paths via BFS from root component.
- **VulnerabilityCorrelationService**: `src/Platform/StellaOps.Platform.Analytics/Services/VulnerabilityCorrelationService.cs` -- correlates PURL-based components with known vulnerabilities; updates artifact vulnerability counts
- **AttestationIngestionService**: `src/Platform/StellaOps.Platform.Analytics/Services/AttestationIngestionService.cs` -- ingests attestation events into analytics
- **Utilities**: PurlParser (PURL normalization), LicenseExpressionRenderer (license aggregation), Sha256Hasher (digest computation), TenantNormalizer (tenant filtering), VersionRuleEvaluator, VulnerabilityCorrelationRules
- **AnalyticsIngestionOptions**: `src/Platform/StellaOps.Platform.Analytics/Options/AnalyticsIngestionOptions.cs` -- configurable stream names, tenant allowlists, ingest/schema versions
- **Tests**: `src/Platform/__Tests/StellaOps.Platform.Analytics.Tests/`
- **Source**: SPRINT_20260120_030_Platform_sbom_analytics_lake.md
## E2E Test Plan
- [ ] Verify SBOM ingestion from scanner scanner-report-ready events (`scanner.event.report.ready`) populates all star-schema tables
- [ ] Test component deduplication via (purl, hash_sha256) conflict resolution
- [ ] Verify dependency path BFS builds correct depth and introduced_via values
- [ ] Test vulnerability correlation updates component and artifact vulnerability counts
- [ ] Verify daily rollup stored procedure computes correct aggregates
- [ ] Test tenant filtering respects AllowedTenants configuration
## Verification
- Verified on 2026-02-11 with `run-001`.
- Tier 0 source checks confirmed ingestion services/options/test surfaces and dependency-path builders are present.
- Tier 1 build + focused analytics class checks passed (`64/64`).
- Tier 2 integration behavior checks passed (`171/171`) across schema integration, ingestion helpers, vulnerability-correlation rules, and tenant normalization behavior.
- Tier 0 reported `partial` only because utility source paths in this dossier still used `Services/` names while active implementations are under `src/Platform/StellaOps.Platform.Analytics/Utilities/`.
- Evidence:
- `docs/qa/feature-checks/runs/platform/sbom-analytics-lake/run-001/tier0-source-check.json`
- `docs/qa/feature-checks/runs/platform/sbom-analytics-lake/run-001/tier1-build-check.json`
- `docs/qa/feature-checks/runs/platform/sbom-analytics-lake/run-001/tier2-integration-check.json`

View File

@@ -0,0 +1,36 @@
# Scanner Platform Events (Redis Streams)
## Module
Platform
## Status
VERIFIED
## Description
Scanner WebService events are ingested by Platform Analytics from the configured stream. The ingestion path now supports `scanner.event.report.ready` and `scanner.scan.completed`, handles scanner DSSE payload decoding for report-ready content, and resumes from persisted stream checkpoints after restart.
## Implementation Details
- **AnalyticsIngestionService**: `src/Platform/StellaOps.Platform.Analytics/Services/AnalyticsIngestionService.cs` -- subscribes to scanner stream; supports both `OrchestratorEventKinds.ScannerReportReady` and `OrchestratorEventKinds.ScannerScanCompleted`; resolves report payloads from direct JSON, wrapped `reportReady`, and DSSE envelopes; persists and restores scanner stream checkpoints.
- **ScannerOrchestratorEvents**: `src/Platform/StellaOps.Platform.Analytics/Models/ScannerOrchestratorEvents.cs` -- scanner event models, including `ScanCompletedEventPayload` and scanner event-kind constants.
- **AnalyticsIngestionOptions**: `src/Platform/StellaOps.Platform.Analytics/Options/AnalyticsIngestionOptions.cs` -- stream settings include `ResumeFromCheckpoint` and optional `ScannerCheckpointFilePath`.
- **IEventStream/IEventStreamFactory**: `src/Router/__Libraries/StellaOps.Messaging/Abstractions/IEventStream.cs` -- stream subscription with `StreamPosition.After(entryId)` checkpoint resume support.
- **Tests**: `src/Platform/__Tests/StellaOps.Platform.Analytics.Tests/ScannerPlatformEventsBehaviorTests.cs`.
- **Source**: 2025-10-19-scanner-policy.md
## E2E Test Plan
- [ ] Verify scanner report-ready events (`scanner.event.report.ready`) are consumed and processed by analytics ingestion.
- [ ] Verify scanner scan-completed events (`scanner.scan.completed`) are accepted and mapped into report-ready ingestion payloads.
- [ ] Verify DSSE envelope payloads are decoded and parsed into report-ready scanner payloads.
- [ ] Verify scanner stream checkpoint normalization and subscription position resolution for restart resume.
- [ ] Verify tenant filtering skips events from non-allowed tenants.
## Verification
- Verified on 2026-02-11 with `run-003`.
- Tier 0 source checks passed with scanner event-kind, DSSE parser, and checkpoint resume declarations present.
- Tier 1 build + tests passed (`185/185`) with code-review parity confirming previously missing scanner-event claims are implemented.
- Tier 2 behavioral/e2e checks passed (`38/38`) across scanner event behavior tests, Docker-backed analytics schema integration, and tenant normalization checks.
- Evidence:
- `docs/qa/feature-checks/runs/platform/scanner-platform-events/run-003/tier0-source-check.json`
- `docs/qa/feature-checks/runs/platform/scanner-platform-events/run-003/tier1-build-check.json`
- `docs/qa/feature-checks/runs/platform/scanner-platform-events/run-003/tier2-integration-check.json`
- `docs/qa/feature-checks/runs/platform/scanner-platform-events/run-003/retest-result.json`