feat: Implement BerkeleyDB reader for RPM databases
Some checks failed
AOC Guard CI / aoc-guard (push) Has been cancelled
AOC Guard CI / aoc-verify (push) Has been cancelled
Concelier Attestation Tests / attestation-tests (push) Has been cancelled
Docs CI / lint-and-preview (push) Has been cancelled
Policy Lint & Smoke / policy-lint (push) Has been cancelled
Scanner Analyzers / Discover Analyzers (push) Has been cancelled
Scanner Analyzers / Build Analyzers (push) Has been cancelled
Scanner Analyzers / Test Language Analyzers (push) Has been cancelled
Scanner Analyzers / Validate Test Fixtures (push) Has been cancelled
Scanner Analyzers / Verify Deterministic Output (push) Has been cancelled
console-runner-image / build-runner-image (push) Has been cancelled
wine-csp-build / Build Wine CSP Image (push) Has been cancelled
wine-csp-build / Integration Tests (push) Has been cancelled
wine-csp-build / Security Scan (push) Has been cancelled
wine-csp-build / Generate SBOM (push) Has been cancelled
wine-csp-build / Publish Image (push) Has been cancelled
wine-csp-build / Air-Gap Bundle (push) Has been cancelled
wine-csp-build / Test Summary (push) Has been cancelled

- Added BerkeleyDbReader class to read and extract RPM header blobs from BerkeleyDB hash databases.
- Implemented methods to detect BerkeleyDB format and extract values, including handling of page sizes and magic numbers.
- Added tests for BerkeleyDbReader to ensure correct functionality and header extraction.

feat: Add Yarn PnP data tests

- Created YarnPnpDataTests to validate package resolution and data loading from Yarn PnP cache.
- Implemented tests for resolved keys, package presence, and loading from cache structure.

test: Add egg-info package fixtures for Python tests

- Created egg-info package fixtures for testing Python analyzers.
- Included PKG-INFO, entry_points.txt, and installed-files.txt for comprehensive coverage.

test: Enhance RPM database reader tests

- Added tests for RpmDatabaseReader to validate fallback to legacy packages when SQLite is missing.
- Implemented helper methods to create legacy package files and RPM headers for testing.

test: Implement dual signing tests

- Added DualSignTests to validate secondary signature addition when configured.
- Created stub implementations for crypto providers and key resolvers to facilitate testing.

chore: Update CI script for Playwright Chromium installation

- Modified ci-console-exports.sh to ensure deterministic Chromium binary installation for console exports tests.
- Added checks for Windows compatibility and environment variable setups for Playwright browsers.
This commit is contained in:
StellaOps Bot
2025-12-07 16:24:45 +02:00
parent e3f28a21ab
commit 11597679ed
199 changed files with 9809 additions and 4404 deletions

View File

@@ -0,0 +1,60 @@
# Mongo Removal Plan — Phase 7 (Scheduler, Notify, Policy, Concelier, Excititor)
## Purpose
Provide a phased, module-by-module plan to delete remaining Mongo storage/projects and unblock PG-T7.1.2T7.1.6. Each plan includes replacements, sequencing, test strategy, and rollback.
## Global guardrails
- No dual-write: switch callers to Postgres equivalents first, then delete Mongo projects.
- Determinism: reuse existing ordered queries; ensure migrations are idempotent and timestamps are UTC.
- Tests: add Postgres-backed unit/integration slices before removing Mongo tests; keep fixtures deterministic.
- Rollback: retain Mongo projects on a feature branch until each modules Postgres tests are green.
## Scheduler (PG-T7.1.2)
1) Replace repos:
- Implement Postgres `IGraphJobStore`, `IOverlayStore`, `IPolicyRunStore` (schema per `docs/db/reports/scheduler-graphjobs-postgres-plan.md`).
- Wire DI in WebService/Worker/Backfill to Postgres stores only.
2) Delete Mongo refs:
- Remove `StellaOps.Scheduler.Storage.Mongo` project and package refs from csproj/solutions.
- Drop Mongo options/session code paths.
3) Tests:
- Add Postgres integration fixtures for graph jobs and overlays.
- Update Backfill tests to use Postgres schema/data.
4) Rollback:
- Keep Mongo project on a branch; if Postgres regression is found, revert DI to Mongo and re-run tests.
## Notify (PG-T7.1.3)
1) Inventory usages in import/backfill hooks; create Postgres equivalents for notification state and history.
2) Switch DI to Postgres stores; add migrations for notification messages/outbox.
3) Update tests to Postgres fixtures; remove Mongo helpers.
4) Delete `StellaOps.Notify.Storage.Mongo` project and solution entries.
## Policy (PG-T7.1.4)
1) Add Postgres storage for any remaining policy documents relying on Mongo (check registries/legacy surfaces).
2) Ensure migrations exist and are applied; switch DI to Postgres stores.
3) Update tests to Postgres fixtures; remove Mongo mocks.
4) Delete `StellaOps.Policy.Storage.Mongo` project and solution references.
## Concelier (PG-T7.1.5)
1) Finish Postgres document/raw storage + state repositories (tasks T7.1.5a/b); wire all connectors/exporters/tests to Postgres (T7.1.5c/d).
2) Add migrations for document/state/export tables and include in offline kit.
3) Remove Mongo packages, `StellaOps.Concelier.Storage.Mongo` project, solution references (T7.1.5e).
4) Tests: Postgres-backed connector/exporter tests; replace Mongo fixtures with deterministic Postgres fixtures.
## Excititor (PG-T7.1.6)
1) Identify Mongo test harness references in Excititor; add Postgres test harness equivalents.
2) Switch any lingering storage abstractions to Postgres (if any remain); otherwise drop Mongo-only test helpers.
3) Remove `StellaOps.Excititor.Storage.Mongo` project and solution entries.
4) Tests: run WebService/Core/Worker tests with Postgres harness; replace Mongo fixtures.
## Promotion & cleanup
- After each modules tests are green, delete the corresponding Mongo project and solution entries in a single PR per module.
- Update module AGENTS.md to remove Mongo references and point to Postgres fixtures.
- Add Execution Log entries in `SPRINT_3407_0001_0001_postgres_cleanup.md` as each module switches to TODO/DOING with this plan.
## Owners
- Scheduler: Scheduler Guild
- Notify: Notify Guild
- Policy: Policy Guild
- Concelier: Concelier Guild
- Excititor: Excititor Guild
- Coordination: Infrastructure Guild