save checkpoint: save features

This commit is contained in:
master
2026-02-12 10:27:23 +02:00
parent dca86e1248
commit 5bca406787
8837 changed files with 1796879 additions and 5294 deletions

View File

@@ -1,31 +0,0 @@
# AI Governance Policy Loader for ML-BOM Scanning
## Module
Scanner
## Status
IMPLEMENTED
## Description
Configurable AI governance policies for scanner-level enforcement of model card requirements, training data lineage thresholds, and EU AI Act compliance categories during SBOM analysis.
## Implementation Details
- **Policy Loader**:
- `src/Scanner/__Libraries/StellaOps.Scanner.AiMlSecurity/Policy/AiGovernancePolicyLoader.cs` - Loads and validates AI governance policy configurations
- `src/Scanner/__Libraries/StellaOps.Scanner.AiMlSecurity/Policy/AiGovernancePolicy.cs` - Policy model defining model card requirements, training data lineage thresholds, and EU AI Act compliance categories
- **Enforcement Analyzers**:
- `src/Scanner/__Libraries/StellaOps.Scanner.AiMlSecurity/Analyzers/ModelCardCompletenessAnalyzer.cs` - Enforces model card completeness requirements from policy
- `src/Scanner/__Libraries/StellaOps.Scanner.AiMlSecurity/Analyzers/ModelCardScoring.cs` - Scores model cards against policy thresholds
- `src/Scanner/__Libraries/StellaOps.Scanner.AiMlSecurity/Analyzers/TrainingDataProvenanceAnalyzer.cs` - Validates training data lineage against policy thresholds
- `src/Scanner/__Libraries/StellaOps.Scanner.AiMlSecurity/Analyzers/AiSafetyRiskAnalyzer.cs` - EU AI Act risk classification
- **Worker Integration**:
- `src/Scanner/StellaOps.Scanner.Worker/Processing/AiMlSecurity/AiMlSecurityStageExecutor.cs` - Stage executor that loads governance policy and runs analyzers during scan
- **Models**: `src/Scanner/__Libraries/StellaOps.Scanner.AiMlSecurity/Models/AiMlSecurityModels.cs`
## E2E Test Plan
- [ ] Configure an AI governance policy with specific model card requirements (e.g., require description, intended use, limitations fields)
- [ ] Scan an image containing an ML model with incomplete model card metadata
- [ ] Verify the scan produces findings for missing model card fields per policy
- [ ] Configure training data lineage threshold and verify scan flags models below threshold
- [ ] Configure EU AI Act compliance category and verify classification is applied to findings
- [ ] Verify policy changes are picked up on subsequent scans without service restart

View File

@@ -1,43 +0,0 @@
# AI/ML Supply Chain Security Analysis Module
## Module
Scanner
## Status
IMPLEMENTED
## Description
Dedicated scanner module for AI/ML supply chain security including EU AI Act risk classification, model card completeness analysis, training data provenance verification, bias/fairness analysis, and AI governance policy enforcement. Distinct from the existing "AI Authority Classification Engine" which focuses on VEX/advisory AI classification, not ML-BOM supply chain scanning.
## Implementation Details
- **Core Analyzer**:
- `src/Scanner/__Libraries/StellaOps.Scanner.AiMlSecurity/AiMlSecurityAnalyzer.cs` - `IAiMlSecurityAnalyzer` / `AiMlSecurityAnalyzer` orchestrates all AI/ML security checks
- `src/Scanner/__Libraries/StellaOps.Scanner.AiMlSecurity/AiMlSecurityServiceCollectionExtensions.cs` - DI registration
- **Analysis Context & Results**:
- `src/Scanner/__Libraries/StellaOps.Scanner.AiMlSecurity/Analyzers/AiMlSecurityContext.cs` - `AiMlSecurityContext` input model
- `src/Scanner/__Libraries/StellaOps.Scanner.AiMlSecurity/Analyzers/AiMlSecurityResult.cs` - `AiMlSecurityResult`, `IAiMlSecurityCheck` interface for pluggable checks
- **Individual Analyzers** (each implements `IAiMlSecurityCheck`):
- `src/Scanner/__Libraries/StellaOps.Scanner.AiMlSecurity/Analyzers/AiSafetyRiskAnalyzer.cs` - EU AI Act risk classification
- `src/Scanner/__Libraries/StellaOps.Scanner.AiMlSecurity/Analyzers/ModelCardCompletenessAnalyzer.cs` - Model card completeness scoring
- `src/Scanner/__Libraries/StellaOps.Scanner.AiMlSecurity/Analyzers/ModelCardScoring.cs` - Scoring logic for model card fields
- `src/Scanner/__Libraries/StellaOps.Scanner.AiMlSecurity/Analyzers/TrainingDataProvenanceAnalyzer.cs` - Training data lineage verification
- `src/Scanner/__Libraries/StellaOps.Scanner.AiMlSecurity/Analyzers/BiasFairnessAnalyzer.cs` - Bias and fairness analysis
- `src/Scanner/__Libraries/StellaOps.Scanner.AiMlSecurity/Analyzers/ModelBinaryAnalyzer.cs` - Model binary format analysis
- `src/Scanner/__Libraries/StellaOps.Scanner.AiMlSecurity/Analyzers/ModelProvenanceVerifier.cs` - Model provenance verification
- `src/Scanner/__Libraries/StellaOps.Scanner.AiMlSecurity/Analyzers/AiModelInventoryGenerator.cs` - Generates inventory of discovered AI/ML models
- **Governance Policy**:
- `src/Scanner/__Libraries/StellaOps.Scanner.AiMlSecurity/Policy/AiGovernancePolicyLoader.cs` - Policy configuration loader
- `src/Scanner/__Libraries/StellaOps.Scanner.AiMlSecurity/Policy/AiGovernancePolicy.cs` - Policy model
- **Reporting**: `src/Scanner/__Libraries/StellaOps.Scanner.AiMlSecurity/Reporting/AiMlSecurityReportFormatter.cs`
- **Models**: `src/Scanner/__Libraries/StellaOps.Scanner.AiMlSecurity/Models/AiMlSecurityModels.cs`
- **Worker Stage**: `src/Scanner/StellaOps.Scanner.Worker/Processing/AiMlSecurity/AiMlSecurityStageExecutor.cs`
## E2E Test Plan
- [ ] Scan a container image containing ML model artifacts (e.g., ONNX, TensorFlow SavedModel, PyTorch)
- [ ] Verify `AiModelInventoryGenerator` discovers and lists all ML models in the scan results
- [ ] Verify `ModelCardCompletenessAnalyzer` produces findings for models with missing/incomplete model cards
- [ ] Verify `AiSafetyRiskAnalyzer` assigns EU AI Act risk classification (unacceptable, high, limited, minimal)
- [ ] Verify `TrainingDataProvenanceAnalyzer` flags models without verifiable training data lineage
- [ ] Verify `BiasFairnessAnalyzer` produces bias/fairness findings where applicable
- [ ] Verify `ModelBinaryAnalyzer` identifies model format and potential binary-level issues
- [ ] Verify all findings appear in the unified scan report and SARIF export

View File

@@ -1,31 +0,0 @@
# API Gateway Boundary Extractor (Kong, Envoy/Istio, AWS API Gateway, Traefik)
## Module
Scanner
## Status
IMPLEMENTED
## Description
Parses API gateway configurations from Kong, Envoy/Istio, AWS API Gateway, and Traefik to extract route-level boundary information for reachability analysis. Determines which internal services are exposed through gateway routes.
## Implementation Details
- **Core Extractor**:
- `src/Scanner/__Libraries/StellaOps.Scanner.Reachability/Boundary/GatewayBoundaryExtractor.cs` - `GatewayBoundaryExtractor` implements `IBoundaryProofExtractor`; parses Kong, Envoy/Istio, AWS API Gateway, and Traefik configurations
- Includes gateway-specific auth detection: `DetectKongAuth()`, `DetectEnvoyAuth()` (including Istio JWT/AuthorizationPolicy), `DetectTraefikAuth()`
- Identifies Istio mesh internal routes and external ingress routes
- **Supporting Infrastructure**:
- `src/Scanner/__Libraries/StellaOps.Scanner.Reachability/Boundary/IacBoundaryExtractor.cs` - Infrastructure-as-code boundary extraction
- `src/Scanner/__Libraries/StellaOps.Scanner.Reachability/Boundary/BoundaryServiceCollectionExtensions.cs` - DI registration for boundary extractors
- `src/Scanner/__Libraries/StellaOps.Scanner.Reachability/Boundary/BoundaryExtractionContext.cs` - Context model for boundary extraction
- `src/Scanner/__Libraries/StellaOps.Scanner.Reachability/Boundary/RichGraphBoundaryExtractor.cs` - Integrates boundary data into rich graphs
- **Tests**:
- `src/Scanner/__Tests/StellaOps.Scanner.Reachability.Tests/GatewayBoundaryExtractorTests.cs`
## E2E Test Plan
- [ ] Provide a Kong configuration file with routes exposing internal services and verify extracted boundary information includes route paths, upstream services, and auth requirements
- [ ] Provide an Envoy/Istio configuration with VirtualService and AuthorizationPolicy and verify correct route-to-service mappings and auth detection
- [ ] Provide an AWS API Gateway configuration and verify correct extraction of REST/HTTP API routes with Lambda/ECS integrations
- [ ] Provide a Traefik configuration with middleware auth and verify correct route extraction with authentication metadata
- [ ] Verify extracted boundaries are reflected in the reachability rich graph via `GET /api/v1/scans/{scanId}/reachability`
- [ ] Verify that gated gateway routes (auth-required) reduce reachability risk scores compared to ungated routes

View File

@@ -1,32 +0,0 @@
# Auto-VEX Generation from Smart-Diff
## Module
Scanner
## Status
IMPLEMENTED
## Description
VEX candidate emission from SmartDiff detection results, generating VEX statements backed by delta evidence.
## Implementation Details
- **VEX Candidate Emission**:
- `src/Scanner/__Libraries/StellaOps.Scanner.SmartDiff/Detection/VexCandidateEmitter.cs` - Emits VEX candidates from SmartDiff detection results
- `src/Scanner/__Libraries/StellaOps.Scanner.SmartDiff/Detection/VexCandidateModels.cs` - VEX candidate data models
- `src/Scanner/__Libraries/StellaOps.Scanner.SmartDiff/Detection/VexEvidence.cs` - Delta evidence backing VEX statements
- `src/Scanner/__Libraries/StellaOps.Scanner.SmartDiff/Detection/Repositories.cs` - Repository interfaces for VEX candidate persistence
- **SmartDiff Predicate**:
- `src/Scanner/__Libraries/StellaOps.Scanner.SmartDiff/SmartDiffPredicate.cs` - SmartDiff predicate model including VEX data
- `src/Scanner/__Libraries/StellaOps.Scanner.SmartDiff/Detection/ReachabilityGateBridge.cs` - Bridges reachability gate data into SmartDiff VEX candidates
- **SARIF Output**:
- `src/Scanner/__Libraries/StellaOps.Scanner.SmartDiff/Output/SarifOutputGenerator.cs` - SARIF export including VEX candidate data
- **API Endpoint**:
- `src/Scanner/StellaOps.Scanner.WebService/Endpoints/SmartDiffEndpoints.cs` - `SmartDiffEndpoints` exposes VEX candidates via REST (`VexCandidatesResponse`, `VexCandidateDto`, `ReviewRequest`)
## E2E Test Plan
- [ ] Scan two versions of the same image to produce a smart-diff delta
- [ ] Call `GET /api/v1/smart-diff/{scanId}/vex-candidates` and verify VEX candidates are returned
- [ ] Verify each VEX candidate includes evidence links referencing the specific delta changes
- [ ] Submit a review decision via `POST /api/v1/smart-diff/{scanId}/vex-candidates/review` and verify the candidate status updates
- [ ] Export the SARIF output and verify VEX candidate data is embedded in the SARIF report
- [ ] Verify VEX candidates include reachability gate context when gates are detected

View File

@@ -1,31 +0,0 @@
# Base Image Detection and Recommendations
## Module
Scanner
## Status
IMPLEMENTED
## Description
Base image detection via layer diffID fingerprinting with PostgreSQL-backed fingerprint database, in-memory index, exact layer match and fuzzy matching, and bulk detection support. Interface `IBaseImageDetector` with full `BaseImageDetector` implementation.
## Implementation Details
- **Core Detection**:
- `src/Scanner/__Libraries/StellaOps.Scanner.Manifest/Resolution/IBaseImageDetector.cs` - `IBaseImageDetector` interface
- `src/Scanner/__Libraries/StellaOps.Scanner.Manifest/Resolution/BaseImageDetector.cs` - `BaseImageDetector` with exact layer match and fuzzy matching, bulk detection support
- **Layer Resolution**:
- `src/Scanner/__Libraries/StellaOps.Scanner.Manifest/Resolution/ILayerDigestResolver.cs` - Interface for resolving layer digests
- `src/Scanner/__Libraries/StellaOps.Scanner.Manifest/Resolution/LayerDigestResolver.cs` - Resolves layer diffIDs for fingerprint matching
- `src/Scanner/__Libraries/StellaOps.Scanner.Manifest/Resolution/LayerProvenance.cs` - Layer provenance tracking for base image attribution
- **Layer Reuse Detection**:
- `src/Scanner/__Libraries/StellaOps.Scanner.Manifest/Reuse/ILayerReuseDetector.cs` - Interface for layer reuse detection
- `src/Scanner/__Libraries/StellaOps.Scanner.Manifest/Reuse/LayerReuseDetector.cs` - Detects shared layers between images for base image identification
- **DI Registration**: `src/Scanner/__Libraries/StellaOps.Scanner.Manifest/ManifestServiceCollectionExtensions.cs`
## E2E Test Plan
- [ ] Scan an image built on a known base image (e.g., `debian:bookworm`) and verify `IBaseImageDetector` identifies the correct base image
- [ ] Verify exact layer match identifies base images by diffID fingerprint comparison
- [ ] Test fuzzy matching with a slightly modified base image (e.g., additional layer) and verify partial match is returned with confidence score
- [ ] Test bulk detection by submitting multiple image references and verify all base images are identified in a single operation
- [ ] Verify base image detection results appear in the scan report and SBOM metadata
- [ ] Verify layer provenance tracking attributes vulnerability findings to base image vs application layers

View File

@@ -1,37 +0,0 @@
# Secret Detection and Credential Leak Guard
## Module
Scanner
## Status
IMPLEMENTED
## Description
Secret detection analyzer with leak evidence capture, alert emission, and integration into the scanner worker pipeline. Compatible with Grype credential leak test scenarios.
## Implementation Details
- **Secrets Analyzer**:
- `src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Secrets/SecretsAnalyzer.cs` - `SecretsAnalyzer` detecting exposed secrets (API keys, tokens, passwords, private keys) in container image layers using regex-based detection rules
- `src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Secrets/SecretsAnalyzerHost.cs` - `SecretsAnalyzerHost` managing the lifecycle and execution of the secrets analyzer
- `src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Secrets/SecretsAnalyzerOptions.cs` - Configuration options for detection rules, severity thresholds, and enabled categories
- `src/Scanner/__Libraries/StellaOps.Scanner.Analyzers.Secrets/ServiceCollectionExtensions.cs` - DI registration for secrets analysis
- **Pipeline Integration**:
- `src/Scanner/StellaOps.Scanner.Worker/Processing/Secrets/SecretsAnalyzerStageExecutor.cs` - `SecretsAnalyzerStageExecutor` scanner worker pipeline stage executing secrets analysis
- **Alert Emission**:
- `src/Scanner/__Libraries/StellaOps.Scanner.Core/Secrets/Alerts/SecretAlertEmitter.cs` - `SecretAlertEmitter` emitting alerts for discovered credential leaks
- `src/Scanner/__Libraries/StellaOps.Scanner.Core/Secrets/Alerts/ISecretAlertEmitter.cs` - Interface for alert emission
- **Exception Matching**:
- `src/Scanner/__Libraries/StellaOps.Scanner.Core/Secrets/Configuration/SecretExceptionMatcher.cs` - `SecretExceptionMatcher` matching findings against allowlist patterns to suppress known-safe secrets
- **Tests**:
- `src/Scanner/__Tests/StellaOps.Scanner.Analyzers.Secrets.Tests/SecretsAnalyzerTests.cs` - Unit tests for secret detection
- `src/Scanner/__Tests/StellaOps.Scanner.Analyzers.Secrets.Tests/SecretsAnalyzerIntegrationTests.cs` - Integration tests
- `src/Scanner/__Tests/StellaOps.Scanner.Analyzers.Secrets.Tests/SecretsAnalyzerHostTests.cs` - Host lifecycle tests
- `src/Scanner/__Tests/StellaOps.Scanner.Analyzers.Secrets.Tests/RegexDetectorTests.cs` - Regex detection rule tests
## E2E Test Plan
- [ ] Scan a container image containing known test secrets and verify `SecretsAnalyzer` detects API keys, tokens, and passwords with correct file paths and line numbers
- [ ] Verify `SecretAlertEmitter` emits alerts for each detected secret with severity classification
- [ ] Verify `SecretExceptionMatcher` suppresses findings matching allowlist patterns (e.g., placeholder values, test credentials)
- [ ] Verify the `SecretsAnalyzerStageExecutor` integrates into the scanner worker pipeline and produces findings in the unified finding format
- [ ] Verify the analyzer handles large files and binary content without performance degradation or false positives
- [ ] Verify compatibility with Grype credential leak test scenarios by running against the same test fixtures