Commit Graph

18 Commits

Author SHA1 Message Date
master
dac8e10e36 feat(crypto): Complete Phase 2 - Configuration-driven crypto architecture with 100% compliance
## Summary

This commit completes Phase 2 of the configuration-driven crypto architecture, achieving
100% crypto compliance by eliminating all hardcoded cryptographic implementations.

## Key Changes

### Phase 1: Plugin Loader Infrastructure
- **Plugin Discovery System**: Created StellaOps.Cryptography.PluginLoader with manifest-based loading
- **Configuration Model**: Added CryptoPluginConfiguration with regional profiles support
- **Dependency Injection**: Extended DI to support plugin-based crypto provider registration
- **Regional Configs**: Created appsettings.crypto.{international,russia,eu,china}.yaml
- **CI Workflow**: Added .gitea/workflows/crypto-compliance.yml for audit enforcement

### Phase 2: Code Refactoring
- **API Extension**: Added ICryptoProvider.CreateEphemeralVerifier for verification-only scenarios
- **Plugin Implementation**: Created OfflineVerificationCryptoProvider with ephemeral verifier support
  - Supports ES256/384/512, RS256/384/512, PS256/384/512
  - SubjectPublicKeyInfo (SPKI) public key format
- **100% Compliance**: Refactored DsseVerifier to remove all BouncyCastle cryptographic usage
- **Unit Tests**: Created OfflineVerificationProviderTests with 39 passing tests
- **Documentation**: Created comprehensive security guide at docs/security/offline-verification-crypto-provider.md
- **Audit Infrastructure**: Created scripts/audit-crypto-usage.ps1 for static analysis

### Testing Infrastructure (TestKit)
- **Determinism Gate**: Created DeterminismGate for reproducibility validation
- **Test Fixtures**: Added PostgresFixture and ValkeyFixture using Testcontainers
- **Traits System**: Implemented test lane attributes for parallel CI execution
- **JSON Assertions**: Added CanonicalJsonAssert for deterministic JSON comparisons
- **Test Lanes**: Created test-lanes.yml workflow for parallel test execution

### Documentation
- **Architecture**: Created CRYPTO_CONFIGURATION_DRIVEN_ARCHITECTURE.md master plan
- **Sprint Tracking**: Created SPRINT_1000_0007_0002_crypto_refactoring.md (COMPLETE)
- **API Documentation**: Updated docs2/cli/crypto-plugins.md and crypto.md
- **Testing Strategy**: Created testing strategy documents in docs/implplan/SPRINT_5100_0007_*

## Compliance & Testing

-  Zero direct System.Security.Cryptography usage in production code
-  All crypto operations go through ICryptoProvider abstraction
-  39/39 unit tests passing for OfflineVerificationCryptoProvider
-  Build successful (AirGap, Crypto plugin, DI infrastructure)
-  Audit script validates crypto boundaries

## Files Modified

**Core Crypto Infrastructure:**
- src/__Libraries/StellaOps.Cryptography/CryptoProvider.cs (API extension)
- src/__Libraries/StellaOps.Cryptography/CryptoSigningKey.cs (verification-only constructor)
- src/__Libraries/StellaOps.Cryptography/EcdsaSigner.cs (fixed ephemeral verifier)

**Plugin Implementation:**
- src/__Libraries/StellaOps.Cryptography.Plugin.OfflineVerification/ (new)
- src/__Libraries/StellaOps.Cryptography.PluginLoader/ (new)

**Production Code Refactoring:**
- src/AirGap/StellaOps.AirGap.Importer/Validation/DsseVerifier.cs (100% compliant)

**Tests:**
- src/__Libraries/__Tests/StellaOps.Cryptography.Plugin.OfflineVerification.Tests/ (new, 39 tests)
- src/__Libraries/__Tests/StellaOps.Cryptography.PluginLoader.Tests/ (new)

**Configuration:**
- etc/crypto-plugins-manifest.json (plugin registry)
- etc/appsettings.crypto.*.yaml (regional profiles)

**Documentation:**
- docs/security/offline-verification-crypto-provider.md (600+ lines)
- docs/implplan/CRYPTO_CONFIGURATION_DRIVEN_ARCHITECTURE.md (master plan)
- docs/implplan/SPRINT_1000_0007_0002_crypto_refactoring.md (Phase 2 complete)

## Next Steps

Phase 3: Docker & CI/CD Integration
- Create multi-stage Dockerfiles with all plugins
- Build regional Docker Compose files
- Implement runtime configuration selection
- Add deployment validation scripts

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2025-12-23 18:20:00 +02:00
master
396e9b75a4 docs: Add comprehensive component architecture documentation
Created detailed architectural documentation showing component interactions,
communication patterns, and data flows across all StellaOps services.

## New Documentation

**docs/ARCHITECTURE_DETAILED.md** - Comprehensive architecture guide:
- Component topology diagram (all 36+ services)
- Infrastructure layer details (PostgreSQL, Valkey, RustFS, NATS)
- Service-by-service catalog with responsibilities
- Communication patterns with WHY (business purpose)
- 5 detailed data flow diagrams:
  1. Scan Request Flow (CLI → Scanner → Worker → Policy → Signer → Attestor → Notify)
  2. Advisory Update Flow (Concelier → Scheduler → Scanner re-evaluation)
  3. VEX Update Flow (Excititor → IssuerDirectory → Scheduler → Policy)
  4. Notification Delivery Flow (Scanner → Valkey → Notify → Slack/Teams/Email)
  5. Policy Evaluation Flow (Scanner → Policy.Gateway → OPA → PostgreSQL replication)
- Database schema isolation details per service
- Security boundaries and authentication flows

## Updated Documentation

**docs/DEVELOPER_ONBOARDING.md**:
- Added link to detailed architecture
- Simplified overview with component categories
- Quick reference topology tree

**docs/07_HIGH_LEVEL_ARCHITECTURE.md**:
- Updated infrastructure requirements section
- Clarified PostgreSQL as ONLY database
- Emphasized Valkey as REQUIRED (not optional)
- Marked NATS as optional (Valkey is default transport)

**docs/README.md**:
- Added link to detailed architecture in navigation

## Key Architectural Insights Documented

**Communication Patterns:**
- 11 communication steps in scan flow (Gateway → Scanner → Valkey → Worker → Concelier → Policy → Signer → Attestor → Valkey → Notify → Slack)
- PostgreSQL logical replication (advisory_raw_stream, vex_raw_stream → Policy Engine)
- Valkey Streams for async job queuing (XADD/XREADGROUP pattern)
- HTTP webhooks for delta events (Concelier/Excititor → Scheduler)

**Security Boundaries:**
- Authority issues OpToks with DPoP binding (RFC 9449)
- Signer enforces PoE validation + scanner digest verification
- All services validate JWT + DPoP on every request
- Tenant isolation via tenant_id in all PostgreSQL queries

**Database Patterns:**
- 8 dedicated PostgreSQL schemas (authority, scanner, vuln, vex, scheduler, notify, policy, orchestrator)
- Append-only advisory/VEX storage (AOC - Aggregation-Only Contract)
- BOM-Index for impact selection (CVE → PURL → image mapping)

This documentation provides complete visibility into who calls who, why they
communicate, what data flows through the system, and how security is enforced
at every layer.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2025-12-23 11:05:55 +02:00
master
342c35f8ce Deprecate MongoDB support in AOC verification CLI
Removes legacy MongoDB options and code paths from the AOC verification command, enforcing PostgreSQL as the required backend. Updates environment examples and documentation to reflect Valkey and RustFS as defaults, replacing Redis and MinIO references.
2025-12-23 10:21:02 +02:00
StellaOps Bot
4b3db9ca85 docs(ops): Complete operations runbooks for Epic 3500
Sprint 3500.0004.0004 (Documentation & Handoff) - T2 DONE

Operations Runbooks Added:
- score-replay-runbook.md: Deterministic replay procedures
- proof-verification-runbook.md: DSSE/Merkle verification ops
- airgap-operations-runbook.md: Offline kit management

CLI Reference Docs:
- reachability-cli-reference.md
- score-proofs-cli-reference.md
- unknowns-cli-reference.md

Air-Gap Guides:
- score-proofs-reachability-airgap-runbook.md

Training Materials:
- score-proofs-concept-guide.md

UI API Clients:
- proof.client.ts
- reachability.client.ts
- unknowns.client.ts

All 5 operations runbooks now complete (reachability, unknowns-queue,
score-replay, proof-verification, airgap-operations).
2025-12-20 22:30:02 +02:00
master
3a2100aa78 Add unit and integration tests for VexCandidateEmitter and SmartDiff repositories
- Implemented comprehensive unit tests for VexCandidateEmitter to validate candidate emission logic based on various scenarios including absent and present APIs, confidence thresholds, and rate limiting.
- Added integration tests for SmartDiff PostgreSQL repositories, covering snapshot storage and retrieval, candidate storage, and material risk change handling.
- Ensured tests validate correct behavior for storing, retrieving, and querying snapshots and candidates, including edge cases and expected outcomes.
2025-12-16 19:00:43 +02:00
StellaOps Bot
233873f620 up
Some checks failed
Docs CI / lint-and-preview (push) Has been cancelled
Signals CI & Image / signals-ci (push) Has been cancelled
Signals Reachability Scoring & Events / reachability-smoke (push) Has been cancelled
Signals Reachability Scoring & Events / sign-and-upload (push) Has been cancelled
AOC Guard CI / aoc-guard (push) Has been cancelled
AOC Guard CI / aoc-verify (push) Has been cancelled
Reachability Corpus Validation / validate-corpus (push) Has been cancelled
Reachability Corpus Validation / validate-ground-truths (push) Has been cancelled
Scanner Analyzers / Discover Analyzers (push) Has been cancelled
Scanner Analyzers / Validate Test Fixtures (push) Has been cancelled
Reachability Corpus Validation / determinism-check (push) Has been cancelled
Scanner Analyzers / Build Analyzers (push) Has been cancelled
Scanner Analyzers / Test Language Analyzers (push) Has been cancelled
Scanner Analyzers / Verify Deterministic Output (push) Has been cancelled
Notify Smoke Test / Notify Unit Tests (push) Has been cancelled
Notify Smoke Test / Notifier Service Tests (push) Has been cancelled
Notify Smoke Test / Notification Smoke Test (push) Has been cancelled
Policy Lint & Smoke / policy-lint (push) Has been cancelled
2025-12-14 15:50:38 +02:00
StellaOps Bot
564df71bfb up
Some checks failed
Docs CI / lint-and-preview (push) Has been cancelled
AOC Guard CI / aoc-guard (push) Has been cancelled
AOC Guard CI / aoc-verify (push) Has been cancelled
Concelier Attestation Tests / attestation-tests (push) Has been cancelled
Export Center CI / export-ci (push) Has been cancelled
Notify Smoke Test / Notify Unit Tests (push) Has been cancelled
Notify Smoke Test / Notifier Service Tests (push) Has been cancelled
Notify Smoke Test / Notification Smoke Test (push) Has been cancelled
Policy Lint & Smoke / policy-lint (push) Has been cancelled
Scanner Analyzers / Discover Analyzers (push) Has been cancelled
Scanner Analyzers / Build Analyzers (push) Has been cancelled
Scanner Analyzers / Test Language Analyzers (push) Has been cancelled
Scanner Analyzers / Validate Test Fixtures (push) Has been cancelled
Scanner Analyzers / Verify Deterministic Output (push) Has been cancelled
Signals CI & Image / signals-ci (push) Has been cancelled
Signals Reachability Scoring & Events / reachability-smoke (push) Has been cancelled
Signals Reachability Scoring & Events / sign-and-upload (push) Has been cancelled
2025-12-13 00:20:26 +02:00
master
b7059d523e Refactor and update test projects, remove obsolete tests, and upgrade dependencies
- Deleted obsolete test files for SchedulerAuditService and SchedulerMongoSessionFactory.
- Removed unused TestDataFactory class.
- Updated project files for Mongo.Tests to remove references to deleted files.
- Upgraded BouncyCastle.Cryptography package to version 2.6.2 across multiple projects.
- Replaced Microsoft.Extensions.Http.Polly with Microsoft.Extensions.Http.Resilience in Zastava.Webhook project.
- Updated NetEscapades.Configuration.Yaml package to version 3.1.0 in Configuration library.
- Upgraded Pkcs11Interop package to version 5.1.2 in Cryptography libraries.
- Refactored Argon2idPasswordHasher to use BouncyCastle for hashing instead of Konscious.
- Updated JsonSchema.Net package to version 7.3.2 in Microservice project.
- Updated global.json to use .NET SDK version 10.0.101.
2025-12-10 19:13:29 +02:00
master
75f6942769 Add integration tests for migration categories and execution
Some checks failed
Docs CI / lint-and-preview (push) Has been cancelled
Policy Lint & Smoke / policy-lint (push) Has been cancelled
Concelier Attestation Tests / attestation-tests (push) Has been cancelled
AOC Guard CI / aoc-guard (push) Has been cancelled
AOC Guard CI / aoc-verify (push) Has been cancelled
- Implemented MigrationCategoryTests to validate migration categorization for startup, release, seed, and data migrations.
- Added tests for edge cases, including null, empty, and whitespace migration names.
- Created StartupMigrationHostTests to verify the behavior of the migration host with real PostgreSQL instances using Testcontainers.
- Included tests for migration execution, schema creation, and handling of pending release migrations.
- Added SQL migration files for testing: creating a test table, adding a column, a release migration, and seeding data.
2025-12-04 19:10:54 +02:00
2de8d1784b new advisories
Some checks failed
Docs CI / lint-and-preview (push) Has been cancelled
2025-11-23 23:38:25 +02:00
master
e91da22836 feat: Add new provenance and crypto registry documentation
Some checks failed
api-governance / spectral-lint (push) Has been cancelled
Docs CI / lint-and-preview (push) Has been cancelled
- Introduced attestation inventory and subject-rekor mapping files for tracking Docker packages.
- Added a comprehensive crypto registry decision document outlining defaults and required follow-ups.
- Created an offline feeds manifest for bundling air-gap resources.
- Implemented a script to generate and update binary manifests for curated binaries.
- Added a verification script to ensure binary artefacts are located in approved directories.
- Defined new schemas for AdvisoryEvidenceBundle, OrchestratorEnvelope, ScannerReportReadyPayload, and ScannerScanCompletedPayload.
- Established project files for StellaOps.Orchestrator.Schemas and StellaOps.PolicyAuthoritySignals.Contracts.
- Updated vendor manifest to track pinned binaries for integrity.
2025-11-18 23:47:13 +02:00
master
15b4a1de6a feat: Document completed tasks for KMS, Cryptography, and Plugin Libraries
- Added detailed task completion records for KMS interface implementation and CLI support for file-based keys.
- Documented security enhancements including Argon2id password hashing, audit event contracts, and rate limiting configurations.
- Included scoped service support and integration updates for the Plugin platform, ensuring proper DI handling and testing coverage.
2025-10-31 14:37:45 +02:00
master
70d7fb529e feat: Add RustFS artifact object store and migration tool
- Implemented RustFsArtifactObjectStore for managing artifacts in RustFS.
- Added unit tests for RustFsArtifactObjectStore functionality.
- Created a RustFS migrator tool to transfer objects from S3 to RustFS.
- Introduced policy preview and report models for API integration.
- Added fixtures and tests for policy preview and report functionality.
- Included necessary metadata and scripts for cache_pkg package.
2025-10-23 18:53:18 +03:00
master
8dc7273e27 FUll implementation plan (first draft) 2025-10-19 00:28:48 +03:00
master
89ede53cc3 Rename Feedser to Concelier 2025-10-18 20:46:16 +03:00
master
dd66f58b00 Rename Vexer to Excititor 2025-10-18 20:44:59 +03:00
master
e2672ba968 Rewrite architecture docs and add Vexer connector template 2025-10-18 20:44:16 +03:00
master
016c5a3fe7 Initial commit (history squashed) 2025-10-11 23:28:35 +03:00