save checkpoint. addition features and their state. check some ofthem

This commit is contained in:
master
2026-02-10 07:54:44 +02:00
parent 4bdc298ec1
commit 5593212b41
211 changed files with 10248 additions and 1208 deletions

View File

@@ -0,0 +1,32 @@
# CI/CD Workflow Generator (Multi-Platform Pipeline Templates)
## Module
Tools
## Status
VERIFIED
## Description
Generates CI/CD pipeline templates for GitHub Actions, GitLab CI, and Azure DevOps that integrate StellaOps scanning with automatic SARIF upload to code scanning platforms. Supports configurable triggers, scan options, and upload configurations.
## Implementation Details
- **Workflow Generator Factory**: `src/Tools/StellaOps.Tools.WorkflowGenerator/WorkflowGeneratorFactory.cs` (61 lines) -- factory mapping `CiPlatform` enum to generator instances. Supports GitHub Actions, GitLab CI, Azure DevOps, and Gitea Actions (mapped to GitHub Actions generator).
- **IWorkflowGenerator Interface**: `src/Tools/StellaOps.Tools.WorkflowGenerator/IWorkflowGenerator.cs` (41 lines) -- common interface with `Platform`, `PlatformName`, `DefaultFileName` properties and `Generate(WorkflowOptions)`, `Validate(WorkflowOptions)` methods.
- **GitHub Actions Generator**: `src/Tools/StellaOps.Tools.WorkflowGenerator/GitHubActionsGenerator.cs` (229 lines) -- full YAML generation with triggers (push, PR, schedule, workflow_dispatch), permissions, env vars, CLI install, scan step, SARIF upload via `github/codeql-action/upload-sarif@v3`, SBOM artifact upload.
- **GitLab CI Generator**: `src/Tools/StellaOps.Tools.WorkflowGenerator/GitLabCiGenerator.cs` (188 lines) -- `.gitlab-ci.yml` generation with stages, variables, rules, before_script CLI install, scan script, SAST report artifacts, `allow_failure` toggle.
- **Azure DevOps Generator**: `src/Tools/StellaOps.Tools.WorkflowGenerator/AzureDevOpsGenerator.cs` (240 lines) -- `azure-pipelines.yml` with triggers, variables, pool/vmImage, Bash@3 tasks, PublishBuildArtifacts@1, Advanced Security CodeQL upload.
- **Supporting files**: `WorkflowOptions.cs` (107 lines), `CiPlatform.cs`, `ScanConfig.cs`, `TriggerConfig.cs`, `UploadConfig.cs`, `ValidationResult.cs` (10 source files total).
## E2E Test Plan
- [x] Generate a GitHub Actions workflow using `WorkflowGeneratorFactory`, parse the output YAML, and verify it contains the scan step, SARIF upload step, and correct trigger configuration
- [x] Generate a GitLab CI pipeline, parse the output YAML, and verify it contains the scan job with correct stage, artifacts, and runner tags
- [x] Generate an Azure DevOps pipeline, parse the output YAML, and verify it contains the scan task with correct pool specification and SARIF publish step
- [x] Generate workflows for all three platforms with the same scan configuration and verify scan arguments are consistent across all outputs
- [x] Generate a workflow with custom triggers (e.g., schedule-only) and verify the output reflects the custom trigger configuration
- [x] Verify the generated GitHub Actions workflow is valid YAML and passes schema validation
## Verification
- **Verified**: 2026-02-10
- **Method**: Tier 1 code review + Tier 2d test verification
- **Build**: 5/9 projects pass (4 blocked by Policy dep, not relevant to this feature). 0 errors, 0 warnings for WorkflowGenerator.
- **Tests**: 76 tests pass across 5 test files (GitHubActionsGeneratorTests: 21, GitLabCiGeneratorTests: 13, AzureDevOpsGeneratorTests: 13, WorkflowGeneratorFactoryTests: 7, WorkflowOptionsTests: 7, plus golden fixture tests)

View File

@@ -0,0 +1,26 @@
# Fixture Harvester Tool (Deterministic Fixture Rewriter)
## Module
Tools
## Status
VERIFIED
## Description
CLI tool for deterministic test fixture management. Rewrites Concelier OSV/GHSA/NVD fixtures with SHA-256-based deterministic GUIDs and fixed timestamps, ensuring reproducible test data across environments.
## Implementation Details
- **Fixture Updater App**: `src/Tools/FixtureUpdater/FixtureUpdaterApp.cs` (96 lines) -- CLI entry point using `System.CommandLine`. Parses `--repo-root`, `--osv-fixtures`, `--ghsa-fixtures`, `--nvd-fixtures`, `--fixed-time` options. Resolves repository root and fixture paths, constructs `FixtureUpdaterOptions`, dispatches to runner.
- **Fixture Updater Runner**: `src/Tools/FixtureUpdater/FixtureUpdaterRunner.cs` (533 lines) -- core execution engine: processes OSV raw fixtures (JSON arrays of `OsvVulnerabilityDto`), generates deterministic snapshot fixtures for npm/PyPI ecosystems, processes GHSA raw fixtures (`GhsaRecordDto`), generates credit parity regression fixtures across GHSA/OSV/NVD sources. Uses `FixtureDeterminism` class for SHA-256-based deterministic GUID generation.
- **Program.cs**: `src/Tools/FixtureUpdater/Program.cs` (3 lines) -- delegates to `FixtureUpdaterApp.RunAsync(args)`.
## E2E Test Plan
- [x] Run the fixture updater tool twice with the same inputs and verify outputs are bit-for-bit identical (determinism check)
- [x] Verify error reporting includes context about which fixture source caused the failure
## Verification
- **Verified**: 2026-02-10
- **Method**: Tier 1 code review + Tier 2d test verification
- **Build**: Passes (0 errors, 0 warnings)
- **Tests**: 2 tests pass (determinism verification, error reporting with context)
- **Caveat**: Original feature description overstated capabilities. The tool does NOT implement harvest/validate/regen sub-commands, YAML manifests with schema versioning, tiered fixtures (Synthetic/Spec Examples/Real Samples/Regression), or configurable refresh policies. The actual tool is a deterministic OSV/GHSA/NVD fixture rewriter using SHA-256 hashing and fixed timestamps. Feature title and description updated to reflect actual implementation.

View File

@@ -0,0 +1,34 @@
# Golden Pairs Mirror and Diff Pipeline
## Module
Tools
## Status
VERIFIED
## Description
Package mirror service to download pre/post-patch binary pairs from distro repos, and a diff pipeline service that runs section-hash diffing to produce golden diff reports for backport detection validation.
## Implementation Details
- **Golden Pairs App**: `src/Tools/GoldenPairs/GoldenPairsApp.cs` (320 lines) -- full CLI with `mirror`, `diff`, and `validate` sub-commands using `System.CommandLine`. Mirror downloads pre/post-patch binaries, diff runs section-hash comparison and writes JSON reports, validate iterates CVE directories and reports pass/fail summary.
- **Package Mirror Service**: `src/Tools/GoldenPairs/Services/PackageMirrorService.cs` (286 lines) -- `AptPackageMirrorService` implementing `IPackageMirrorService`. Downloads from HTTP(S), `apt://` (scheme-rewritten), and `file://` URIs. Extracts files from `.deb` archives via SharpCompress (nested data.tar extraction). SHA-256 hash verification after download.
- **Diff Pipeline Service**: `src/Tools/GoldenPairs/Services/DiffPipelineService.cs` (289 lines) -- section-by-section comparison (Identical/Modified/Added/Removed) using hash comparison. Verdict determination (Patched/Vanilla/Unknown) based on `.text` section changes with confidence scoring. Validation against expected diff.
- **Section Hash Provider**: `src/Tools/GoldenPairs/Services/SectionHashProvider.cs` (87 lines) -- `FileSectionHashProvider` with `LoadAsync` (from JSON) and `ExtractAsync` (from binary via `IElfSectionHashExtractor`). Deterministically ordered `SectionHashSet`.
- **Golden Pair Loader**: `src/Tools/GoldenPairs/Services/GoldenPairLoader.cs` (211 lines) -- loads metadata from JSON files with JSON Schema validation, deserialization, normalization, and error collection. Supports individual pair and index loading.
- **Golden Pairs Schema Provider**: `src/Tools/GoldenPairs/Schema/GoldenPairsSchemaProvider.cs` (36 lines) -- lazy-loads metadata and index JSON schemas.
- **Models**: `src/Tools/GoldenPairs/Models/` (4 files, ~170 lines) -- `GoldenPairMetadata`, `GoldenDiffReport`, `SectionHashModels`, `GoldenPairsIndex`.
- **Serialization**: `src/Tools/GoldenPairs/Serialization/GoldenPairsJsonSerializer.cs` (78 lines) -- deterministic property ordering via `DeterministicTypeInfoResolver`, camelCase naming, enum string conversion.
## E2E Test Plan
- [x] Run `PackageMirrorService` to download a known CVE fix pair and verify both binaries are downloaded with correct metadata and SHA-256 verification
- [x] Run `DiffPipelineService` on a pair and verify the diff report identifies changed sections with correct verdict
- [x] Run `SectionHashProvider` on a known binary and verify section hashes are deterministic across multiple runs
- [x] Load a golden pair via `GoldenPairLoader`, re-run the diff pipeline, and verify the new diff report matches
- [x] Validate a diff report against the JSON schema and verify it passes validation
- [x] Verify hash mismatch detection in mirror service
## Verification
- **Verified**: 2026-02-10
- **Method**: Tier 1 code review + Tier 2d test verification
- **Build**: Passes (0 errors, 0 warnings)
- **Tests**: 9 tests pass across 4 test files (DiffPipelineServiceTests: 2, GoldenPairLoaderTests: 2, PackageMirrorServiceTests: 2, GoldenPairSchemaTests: 3)

View File

@@ -0,0 +1,31 @@
# Golden Pairs Validation Infrastructure
## Module
Tools
## Status
VERIFIED
## Description
Data model for golden pair metadata, binary artifacts, and diff reports used to validate binary diff detection against known-good CVE fix pairs.
## Implementation Details
- **Golden Pairs Models**: `src/Tools/GoldenPairs/Models/` (4 files, ~170 lines) -- `GoldenPairMetadata` (CVE ID, package name, distro, pre/post versions, binary artifacts with section hashes), `GoldenDiffReport` (sections, verdict, confidence, discrepancies), `SectionHashModels` (SectionHashSet, SectionHashEntry with Size), `GoldenPairsIndex` (version, pairs, summary).
- **Golden Pairs Schema Provider**: `src/Tools/GoldenPairs/Schema/GoldenPairsSchemaProvider.cs` (36 lines) -- lazy-loads JSON schemas for metadata and index validation.
- **Golden Pair Loader**: `src/Tools/GoldenPairs/Services/GoldenPairLoader.cs` (211 lines) -- loads and validates golden pair records with JSON Schema enforcement before deserialization, normalization, and error collection.
- **Serialization**: `src/Tools/GoldenPairs/Serialization/GoldenPairsJsonSerializer.cs` (78 lines) -- `DeterministicTypeInfoResolver` for alphabetical property ordering, ensuring deterministic output for hash comparison and attestation.
- **Section Hash Provider**: `src/Tools/GoldenPairs/Services/SectionHashProvider.cs` (87 lines) -- deterministic per-section hash computation via `IElfSectionHashExtractor`, producing ordered `SectionHashSet`.
## E2E Test Plan
- [x] Load a golden pair record and verify all required fields are populated and valid
- [x] Validate metadata against schema and verify it passes; corrupt a field and verify validation fails
- [x] Serialize a golden pair record, deserialize it back, and verify round-trip fidelity
- [x] Compute section hashes on two separate runs and verify determinism
- [x] Load a diff report and verify it correctly identifies changed sections
- [x] Verify schema provider covers metadata and index schemas
## Verification
- **Verified**: 2026-02-10
- **Method**: Tier 1 code review + Tier 2d test verification
- **Build**: Passes (0 errors, 0 warnings)
- **Tests**: 9 tests pass (shared with Golden Pairs Mirror feature: GoldenPairSchemaTests: 3, GoldenPairLoaderTests: 2, DiffPipelineServiceTests: 2, PackageMirrorServiceTests: 2)