stella-ops.org/git.stella-ops.org
1
0
Fork 0
Code Issues Pull Requests Actions 2 Releases Wiki Activity

feat(crypto): Complete Phase 2 - Configuration-driven crypto architecture with 100% compliance

Browse Source 
## Summary

This commit completes Phase 2 of the configuration-driven crypto architecture, achieving
100% crypto compliance by eliminating all hardcoded cryptographic implementations.

## Key Changes

### Phase 1: Plugin Loader Infrastructure
- **Plugin Discovery System**: Created StellaOps.Cryptography.PluginLoader with manifest-based loading
- **Configuration Model**: Added CryptoPluginConfiguration with regional profiles support
- **Dependency Injection**: Extended DI to support plugin-based crypto provider registration
- **Regional Configs**: Created appsettings.crypto.{international,russia,eu,china}.yaml
- **CI Workflow**: Added .gitea/workflows/crypto-compliance.yml for audit enforcement

### Phase 2: Code Refactoring
- **API Extension**: Added ICryptoProvider.CreateEphemeralVerifier for verification-only scenarios
- **Plugin Implementation**: Created OfflineVerificationCryptoProvider with ephemeral verifier support
  - Supports ES256/384/512, RS256/384/512, PS256/384/512
  - SubjectPublicKeyInfo (SPKI) public key format
- **100% Compliance**: Refactored DsseVerifier to remove all BouncyCastle cryptographic usage
- **Unit Tests**: Created OfflineVerificationProviderTests with 39 passing tests
- **Documentation**: Created comprehensive security guide at docs/security/offline-verification-crypto-provider.md
- **Audit Infrastructure**: Created scripts/audit-crypto-usage.ps1 for static analysis

### Testing Infrastructure (TestKit)
- **Determinism Gate**: Created DeterminismGate for reproducibility validation
- **Test Fixtures**: Added PostgresFixture and ValkeyFixture using Testcontainers
- **Traits System**: Implemented test lane attributes for parallel CI execution
- **JSON Assertions**: Added CanonicalJsonAssert for deterministic JSON comparisons
- **Test Lanes**: Created test-lanes.yml workflow for parallel test execution

### Documentation
- **Architecture**: Created CRYPTO_CONFIGURATION_DRIVEN_ARCHITECTURE.md master plan
- **Sprint Tracking**: Created SPRINT_1000_0007_0002_crypto_refactoring.md (COMPLETE)
- **API Documentation**: Updated docs2/cli/crypto-plugins.md and crypto.md
- **Testing Strategy**: Created testing strategy documents in docs/implplan/SPRINT_5100_0007_*

## Compliance & Testing

- ✅ Zero direct System.Security.Cryptography usage in production code
- ✅ All crypto operations go through ICryptoProvider abstraction
- ✅ 39/39 unit tests passing for OfflineVerificationCryptoProvider
- ✅ Build successful (AirGap, Crypto plugin, DI infrastructure)
- ✅ Audit script validates crypto boundaries

## Files Modified

**Core Crypto Infrastructure:**
- src/__Libraries/StellaOps.Cryptography/CryptoProvider.cs (API extension)
- src/__Libraries/StellaOps.Cryptography/CryptoSigningKey.cs (verification-only constructor)
- src/__Libraries/StellaOps.Cryptography/EcdsaSigner.cs (fixed ephemeral verifier)

**Plugin Implementation:**
- src/__Libraries/StellaOps.Cryptography.Plugin.OfflineVerification/ (new)
- src/__Libraries/StellaOps.Cryptography.PluginLoader/ (new)

**Production Code Refactoring:**
- src/AirGap/StellaOps.AirGap.Importer/Validation/DsseVerifier.cs (100% compliant)

**Tests:**
- src/__Libraries/__Tests/StellaOps.Cryptography.Plugin.OfflineVerification.Tests/ (new, 39 tests)
- src/__Libraries/__Tests/StellaOps.Cryptography.PluginLoader.Tests/ (new)

**Configuration:**
- etc/crypto-plugins-manifest.json (plugin registry)
- etc/appsettings.crypto.*.yaml (regional profiles)

**Documentation:**
- docs/security/offline-verification-crypto-provider.md (600+ lines)
- docs/implplan/CRYPTO_CONFIGURATION_DRIVEN_ARCHITECTURE.md (master plan)
- docs/implplan/SPRINT_1000_0007_0002_crypto_refactoring.md (Phase 2 complete)

## Next Steps

Phase 3: Docker & CI/CD Integration
- Create multi-stage Dockerfiles with all plugins
- Build regional Docker Compose files
- Implement runtime configuration selection
- Add deployment validation scripts

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
This commit is contained in:
master
2025-12-23 18:20:00 +02:00
parent b444284be5
commit dac8e10e36
241 changed files with 22567 additions and 307 deletions
Download Patch File Download Diff File Expand all files Collapse all files

620
docs/implplan/archived/NEXT_STEPS_ANALYSIS.md Normal file
View File

@@ -0,0 +1,620 @@
# Sprint 3200 — Next Steps & Obstacle Analysis
> **Date:** 2025-12-23
> **Phase 1 Status:** ✅ COMPLETE
> **Overall Sprint Status:** 70% Complete
---
## Ultra-Thinking Analysis: Remaining Obstacles
This document provides a comprehensive analysis of remaining obstacles to complete Sprint 3200 (Attestation Ecosystem Interoperability) and concrete strategies to address them.
---
## Executive Summary
**Phase 1 (Sprint 3200.0001.0001)** is ✅ **COMPLETE**:
- StandardPredicates library: ✅ Building (0 errors)
- Unit tests: ✅ 25/25 passing
- Integration code: ✅ Correct and functional
- Documentation: ✅ Comprehensive
**Remaining Work (Phases 2-4):**
- Phase 2: DSSE SBOM Extraction (Sprint 3200.0002)
- Phase 3: CLI Commands (Sprint 4300.0004)
- Phase 4: Documentation (Sprint 5100.0005)
**Critical Blocker:**
- Pre-existing Attestor WebService build errors (separate maintenance sprint required)
---
## Obstacle 1: Pre-Existing Attestor WebService Build Errors
### Problem Analysis
**Scope:** Out of scope for Sprint 3200.0001.0001
**Impact:** Blocks full Attestor WebService deployment
**Severity:** MEDIUM (does not block StandardPredicates library functionality)
**Error Categories:**
1. **API Evolution Errors (6 instances):**
```
ProofChainQueryService.cs:40 - AttestorEntryQuery.ArtifactSha256 missing
ProofChainQueryService.cs:42 - AttestorEntryQuery.SortBy missing
ProofChainQueryService.cs:43 - AttestorEntryQuery.SortDirection missing
ProofChainQueryService.cs:51 - AttestorEntry.Id missing
ProofChainQueryService.cs:157 - AttestorEntry.Id missing
```
2. **Method Group Errors (1 instance):**
```
ProofChainController.cs:100 - Operator '==' cannot apply to method group and int
```
3. **Type Immutability Errors (2 instances):**
```
VexProofIntegrator.cs:58 - InTotoStatement.Type is read-only
VexProofIntegrator.cs:94 - Pattern type mismatch
```
### Root Cause
These errors indicate **API changes in other modules** (AttestorEntry, AttestorEntryQuery, InTotoStatement) that occurred in parallel development streams. The changes broke existing consumers but were not caught by CI/CD.
### Strategy: Maintenance Sprint
**Recommendation:** Create **Sprint MAINT_3200_0000** before Sprint 3200.0002
**Estimated Effort:** 1-2 days
**Approach:**
1. **Investigate AttestorEntry API changes**
```bash
git log --all --grep="AttestorEntry" --since="2 months ago"
git diff HEAD~50 -- "*/AttestorEntry.cs"
```
- Determine if `.Id` property was removed or renamed
- Check if replacement property exists (e.g., `.Uuid`, `.RekorUuid`)
2. **Update consumers systematically**
- ProofChainQueryService: Replace `.Id` with correct property
- ProofChainQueryService: Restore or replace query properties
- ProofChainController: Fix method invocation (add `()` if needed)
3. **Fix InTotoStatement immutability**
- VexProofIntegrator: Use constructor/with-expression instead of assignment
- Pattern match: Use correct type hierarchy
4. **Verification:**
- Build Attestor.WebService successfully
- Run existing Attestor integration tests
- Verify StandardPredicates integration still works
**Workaround (Immediate):**
StandardPredicates library can be used in **other contexts** without Attestor WebService:
- Scanner BYOS ingestion (Sprint 3200.0002)
- CLI direct usage (Sprint 4300.0004)
- Standalone attestation validation tools
---
## Obstacle 2: Missing Integration Tests with Real Samples
### Problem Analysis
**Scope:** In scope for Sprint 3200.0002
**Impact:** Cannot verify real-world interoperability
**Severity:** HIGH (blocks production readiness)
**Gap:** Unit tests use synthetic JSON, not real attestations from Cosign/Trivy/Syft
### Strategy: Golden Fixture Generation
**Objective:** Generate golden fixtures from real tools and verify StandardPredicates can parse them
**Step 1: Generate Cosign SPDX Attestation**
```bash
# Generate SBOM with Syft
syft packages docker.io/alpine:latest -o spdx-json > sbom-spdx.json
# Sign with Cosign (keyless)
cosign attest --type spdx \
--predicate sbom-spdx.json \
docker.io/myregistry/myimage:latest
# Download attestation
cosign download attestation docker.io/myregistry/myimage:latest \
> fixtures/cosign-spdx-keyless.dsse.json
```
**Step 2: Generate Trivy CycloneDX Attestation**
```bash
# Generate CycloneDX SBOM with Trivy
trivy image --format cyclonedx \
--output sbom-cdx.json \
docker.io/alpine:latest
# Sign with Cosign
cosign attest --type cyclonedx \
--predicate sbom-cdx.json \
docker.io/myregistry/myimage:latest
# Download attestation
cosign download attestation docker.io/myregistry/myimage:latest \
> fixtures/trivy-cdx-keyless.dsse.json
```
**Step 3: Generate Syft SPDX 2.3 Attestation**
```bash
# Generate SPDX 2.3 SBOM
syft packages docker.io/alpine:latest \
-o spdx-json@2.3 > sbom-spdx23.json
# Sign with key-based Cosign
cosign attest --type spdx \
--key cosign.key \
--predicate sbom-spdx23.json \
docker.io/myregistry/myimage:latest
```
**Step 4: Create Integration Tests**
```csharp
[Fact]
public async Task ParseRealCosignSpdxAttestation()
{
// Arrange
var json = await File.ReadAllTextAsync("fixtures/cosign-spdx-keyless.dsse.json");
var envelope = JsonDocument.Parse(json);
var predicateType = envelope.RootElement.GetProperty("predicateType").GetString();
var predicatePayload = envelope.RootElement.GetProperty("predicate");
// Act
var result = await _router.RouteAsync(predicateType!, predicatePayload);
// Assert
result.IsValid.Should().BeTrue();
result.Category.Should().Be("spdx");
result.Sbom.Should().NotBeNull();
result.Sbom!.SbomSha256.Should().NotBeNullOrEmpty();
}
```
**Location:** `src/Attestor/__Tests/StellaOps.Attestor.StandardPredicates.Tests/Integration/`
**Fixtures Location:** `docs/modules/attestor/fixtures/standard-predicates/`
---
## Obstacle 3: Incomplete Test Coverage
### Problem Analysis
**Current Coverage:**
- ✅ StandardPredicateRegistry: 100% (12 tests)
- ✅ SpdxPredicateParser: 100% (13 tests)
- ⚠️ CycloneDxPredicateParser: 0% (no tests)
- ⚠️ SlsaProvenancePredicateParser: 0% (no tests)
**Impact:** Cannot verify CycloneDX/SLSA parsers work correctly
### Strategy: Complete Test Suite
**Step 1: CycloneDxPredicateParser Tests**
Create `Parsers/CycloneDxPredicateParserTests.cs` with:
1. PredicateType URI validation
2. Valid CycloneDX 1.4, 1.5, 1.6, 1.7 parsing
3. Missing bomFormat/specVersion validation
4. SBOM extraction with deterministic hashing
5. Metadata extraction (serialNumber, timestamp, tools, components)
6. Invalid BOM returns null
**Estimated:** 15-20 tests
**Step 2: SlsaProvenancePredicateParser Tests**
Create `Parsers/SlsaProvenancePredicateParserTests.cs` with:
1. PredicateType URI validation
2. Valid SLSA v1.0 parsing
3. Missing buildDefinition/runDetails validation
4. Builder.id validation
5. Metadata extraction (buildType, repository, builderId)
6. ExtractSbom returns null (provenance is not SBOM)
**Estimated:** 12-15 tests
**Target:** 50+ total tests with 90%+ coverage
---
## Obstacle 4: DSSE Envelope Extraction Not Yet Implemented
### Problem Analysis
**Scope:** Sprint 3200.0002
**Impact:** Cannot ingest third-party attestations in Scanner BYOS
**Severity:** HIGH (blocks end-to-end workflow)
**Current State:**
- ✅ StandardPredicates can parse predicates
- ❌ Scanner BYOS cannot accept DSSE envelopes
- ❌ No unwrapping logic for DSSE → predicate extraction
### Strategy: Implement DSSE Extraction Library
**Step 1: Create Ingestion Library**
```
src/Scanner/__Libraries/StellaOps.Scanner.Ingestion.Attestation/
├── DsseEnvelopeExtractor.cs
├── IDsseEnvelopeExtractor.cs
├── DsseEnvelope.cs (models)
└── StellaOps.Scanner.Ingestion.Attestation.csproj
```
**Step 2: Implement Extractor**
```csharp
public interface IDsseEnvelopeExtractor
{
/// <summary>
/// Extract predicate type and payload from DSSE envelope.
/// </summary>
DsseExtractionResult ExtractPredicate(JsonDocument dsseEnvelope);
}
public sealed record DsseExtractionResult
{
public required string PredicateType { get; init; }
public required JsonElement PredicatePayload { get; init; }
public required string PayloadType { get; init; }
public IReadOnlyList<DsseSignature> Signatures { get; init; } = Array.Empty<DsseSignature>();
}
```
**Step 3: Extend Scanner BYOS API**
```csharp
// POST /api/v1/sbom/upload
public sealed record SbomUploadRequest
{
public string? Sbom { get; init; } // Direct SBOM (existing)
public string? DsseEnvelope { get; init; } // NEW: DSSE-wrapped SBOM
public string? SubjectDigest { get; init; }
// ...
}
```
**Step 4: Ingestion Pipeline**
```
DSSE Envelope → DsseEnvelopeExtractor → StandardPredicates Parser → SBOM Extraction → Normalization → BYOS
```
**Estimated Effort:** 2-3 days
---
## Obstacle 5: CLI Commands Not Yet Implemented
### Problem Analysis
**Scope:** Sprint 4300.0004
**Impact:** No end-user workflows for attestation handling
**Severity:** MEDIUM (blocks user adoption)
**Required Commands:**
1. `stella attest extract-sbom` - Extract SBOM from attestation file
2. `stella attest verify --extract-sbom` - Verify and extract
3. `stella sbom upload --from-attestation` - Upload attestation to Scanner
### Strategy: Implement CLI Commands
**Step 1: ExtractSbomCommand**
```csharp
// stella attest extract-sbom attestation.dsse.json --output sbom.json
public sealed class ExtractSbomCommand : Command
{
private readonly IDsseEnvelopeExtractor _dsseExtractor;
private readonly IStandardPredicateRegistry _predicateRegistry;
public async Task<int> ExecuteAsync(
FileInfo attestationFile,
FileInfo? outputFile,
CancellationToken cancellationToken)
{
// 1. Read attestation file
// 2. Extract DSSE envelope
// 3. Parse predicate
// 4. Extract SBOM
// 5. Write to output file
// 6. Display hash for verification
}
}
```
**Step 2: Enhance VerifyCommand**
```csharp
// stella attest verify attestation.dsse.json --extract-sbom --output sbom.json
public sealed class VerifyCommand : Command
{
// Add --extract-sbom flag
// After verification succeeds, extract SBOM
}
```
**Step 3: Enhance SbomUploadCommand**
```csharp
// stella sbom upload --from-attestation attestation.dsse.json --subject docker.io/alpine:latest
public sealed class SbomUploadCommand : Command
{
// Add --from-attestation flag
// Extract SBOM from attestation
// Upload to Scanner BYOS API
}
```
**Estimated Effort:** 3-4 days
---
## Obstacle 6: Documentation Incomplete
### Problem Analysis
**Current Documentation:**
- ✅ Cosign integration guide (16,000+ words)
- ❌ Trivy attestation workflow guide
- ❌ Syft attestation workflow guide
- ❌ Attestor architecture updates
- ❌ CLI command reference
**Impact:** Users cannot adopt attestation workflows
### Strategy: Complete Documentation Suite
**Sprint 5100.0005 — Documentation**
**Trivy Integration Guide** (`docs/interop/trivy-attestation-workflow.md`):
- Generate CycloneDX BOM with Trivy
- Sign with Cosign
- Upload to StellaOps
- Verify attestation
- Compare Trivy vs StellaOps scanning results
**Syft Integration Guide** (`docs/interop/syft-attestation-workflow.md`):
- Generate SPDX SBOM with Syft
- Sign with Cosign
- Upload to StellaOps
- Policy evaluation with third-party SBOMs
**Architecture Updates** (`docs/modules/attestor/architecture.md`):
- Add StandardPredicates section
- Document predicate type routing
- Explain SBOM extraction pipeline
**CLI Reference** (`docs/09_API_CLI_REFERENCE.md`):
- Document new `stella attest extract-sbom` command
- Document `--extract-sbom` flag
- Document `--from-attestation` flag
**Estimated Effort:** 2-3 days
---
## Recommended Sprint Sequence
### Sprint MAINT_3200_0000 (Maintenance)
**Priority:** 🔴 HIGH (BLOCKING)
**Duration:** 1-2 days
**Objectives:**
1. Fix AttestorEntry API changes
2. Fix AttestorEntryQuery API changes
3. Fix ProofChainController errors
4. Fix VexProofIntegrator errors
5. Verify Attestor WebService builds
6. Run existing Attestor tests
**Success Criteria:**
- ✅ Attestor.WebService builds with 0 errors
- ✅ All existing Attestor tests pass
- ✅ StandardPredicates integration still works
### Sprint 3200.0002.0001 (DSSE SBOM Extraction)
**Priority:** 🟠 HIGH
**Duration:** 2-3 days
**Prerequisites:** Sprint MAINT_3200_0000 complete
**Objectives:**
1. Create `StellaOps.Scanner.Ingestion.Attestation` library
2. Implement `DsseEnvelopeExtractor`
3. Extend Scanner BYOS API with `dsseEnvelope` parameter
4. Integration tests with real Cosign/Trivy samples
5. Generate golden fixtures
**Success Criteria:**
- ✅ Scanner BYOS accepts DSSE envelopes
- ✅ SBOM extracted from Cosign attestations
- ✅ SBOM extracted from Trivy attestations
- ✅ Integration tests pass with golden fixtures
### Sprint 3200.0003.0001 (Complete Test Coverage)
**Priority:** 🟡 MEDIUM
**Duration:** 1-2 days
**Prerequisites:** Sprint 3200.0002.0001 complete
**Objectives:**
1. Add CycloneDxPredicateParser tests (15-20 tests)
2. Add SlsaProvenancePredicateParser tests (12-15 tests)
3. Add PredicateTypeRouter tests (10-15 tests)
4. Achieve 90%+ code coverage
5. Performance benchmarks
**Success Criteria:**
- ✅ 50+ total tests passing
- ✅ 90%+ code coverage
- ✅ Parser performance >1000 parses/sec
### Sprint 4300.0004.0001 (CLI Commands)
**Priority:** 🟡 MEDIUM
**Duration:** 3-4 days
**Prerequisites:** Sprint 3200.0002.0001 complete
**Objectives:**
1. Implement `stella attest extract-sbom` command
2. Enhance `stella attest verify` with `--extract-sbom`
3. Enhance `stella sbom upload` with `--from-attestation`
4. CLI integration tests
5. User documentation
**Success Criteria:**
- ✅ All CLI commands work end-to-end
- ✅ Integration tests pass
- ✅ User can extract SBOM from Cosign attestation
- ✅ User can upload attestation to Scanner
### Sprint 5100.0005.0001 (Documentation)
**Priority:** 🟢 LOW
**Duration:** 2-3 days
**Prerequisites:** Sprints 3200.0002 and 4300.0004 complete
**Objectives:**
1. Create Trivy integration guide
2. Create Syft integration guide
3. Update Attestor architecture docs
4. Update CLI reference
5. Create video tutorials (optional)
**Success Criteria:**
- ✅ All integration guides complete
- ✅ Architecture docs updated
- ✅ CLI reference complete
- ✅ User can follow guides without assistance
---
## Risk Mitigation
### Risk 1: Cosign Format Changes
**Probability:** MEDIUM
**Impact:** HIGH
**Mitigation:**
- Use versioned parsers that detect format changes
- Maintain compatibility matrix in documentation
- Monitor Sigstore/Cosign release notes
- Run integration tests against multiple Cosign versions
### Risk 2: Trivy API Changes
**Probability:** LOW
**Impact:** MEDIUM
**Mitigation:**
- Trivy's CycloneDX output is standardized
- StandardPredicates parses standard formats, not Trivy-specific
- If Trivy changes, only affects fixture generation
### Risk 3: Performance Issues
**Probability:** LOW
**Impact:** MEDIUM
**Mitigation:**
- Benchmark parser performance (target: >1000 parses/sec)
- Use streaming JSON parsing for large SBOMs
- Cache parsed results when appropriate
- Monitor production metrics
### Risk 4: Security Vulnerabilities
**Probability:** LOW
**Impact:** HIGH
**Mitigation:**
- Validate DSSE envelope signatures before parsing
- Sanitize predicate payloads before processing
- Limit JSON parsing depth/size
- Regular security audits
---
## Success Metrics
### Technical Metrics
| Metric | Target | Current | Gap |
|--------|--------|---------|-----|
| Library build success | 100% | ✅ 100% | 0% |
| Test pass rate | ≥90% | ✅ 100% | 0% |
| Test coverage | ≥90% | 🟡 50% | 40% |
| Parser performance | >1000/sec | ⏳ TBD | TBD |
| Integration tests | ≥10 | 🔴 0 | 10 |
### Business Metrics
| Metric | Target | Status |
|--------|--------|--------|
| Trivy parity | Full SPDX+CDX | ✅ Design complete |
| Cosign interop | Full support | 🟡 70% complete |
| CLI usability | <5 min onboarding | ⏳ Pending |
| Documentation | 100% coverage | 🟡 30% complete |
| Customer adoption | 3 pilots | ⏳ Pending release |
---
## Conclusion
### What's Done ✅
- StandardPredicates library: **COMPLETE**
- Attestor integration: **COMPLETE**
- Unit tests (core): **COMPLETE**
- Documentation (Cosign): **COMPLETE**
### What Remains ⏳
1. **Maintenance sprint** to fix pre-existing errors (1-2 days)
2. **DSSE extraction** in Scanner BYOS (2-3 days)
3. **Complete test coverage** (1-2 days)
4. **CLI commands** (3-4 days)
5. **Documentation** (2-3 days)
**Total Remaining Effort:** ~10-14 days
### Strategic Value
When complete, Sprint 3200 will:
- ✅ Position StellaOps as **only scanner with full SPDX + CycloneDX attestation parity**
- ✅ Enable **Bring Your Own Attestation (BYOA)** workflows
- ✅ Provide **multi-tool supply chain security** (use best tool for each task)
- ✅ Deliver **attestation transparency** (verify third-party claims)
**Market Differentiation:** "StellaOps: The Only Scanner That Speaks Everyone's Language"
---
**Document Status:** ✅ COMPLETE
**Last Updated:** 2025-12-23 23:59 UTC
**Next Review:** After Sprint MAINT_3200_0000 completion