Files
git.stella-ops.org/docs/db/reports/mongo-removal-plan-20251207.md
StellaOps Bot 11597679ed
Some checks failed
AOC Guard CI / aoc-guard (push) Has been cancelled
AOC Guard CI / aoc-verify (push) Has been cancelled
Concelier Attestation Tests / attestation-tests (push) Has been cancelled
Docs CI / lint-and-preview (push) Has been cancelled
Policy Lint & Smoke / policy-lint (push) Has been cancelled
Scanner Analyzers / Discover Analyzers (push) Has been cancelled
Scanner Analyzers / Build Analyzers (push) Has been cancelled
Scanner Analyzers / Test Language Analyzers (push) Has been cancelled
Scanner Analyzers / Validate Test Fixtures (push) Has been cancelled
Scanner Analyzers / Verify Deterministic Output (push) Has been cancelled
console-runner-image / build-runner-image (push) Has been cancelled
wine-csp-build / Build Wine CSP Image (push) Has been cancelled
wine-csp-build / Integration Tests (push) Has been cancelled
wine-csp-build / Security Scan (push) Has been cancelled
wine-csp-build / Generate SBOM (push) Has been cancelled
wine-csp-build / Publish Image (push) Has been cancelled
wine-csp-build / Air-Gap Bundle (push) Has been cancelled
wine-csp-build / Test Summary (push) Has been cancelled
feat: Implement BerkeleyDB reader for RPM databases
- Added BerkeleyDbReader class to read and extract RPM header blobs from BerkeleyDB hash databases.
- Implemented methods to detect BerkeleyDB format and extract values, including handling of page sizes and magic numbers.
- Added tests for BerkeleyDbReader to ensure correct functionality and header extraction.

feat: Add Yarn PnP data tests

- Created YarnPnpDataTests to validate package resolution and data loading from Yarn PnP cache.
- Implemented tests for resolved keys, package presence, and loading from cache structure.

test: Add egg-info package fixtures for Python tests

- Created egg-info package fixtures for testing Python analyzers.
- Included PKG-INFO, entry_points.txt, and installed-files.txt for comprehensive coverage.

test: Enhance RPM database reader tests

- Added tests for RpmDatabaseReader to validate fallback to legacy packages when SQLite is missing.
- Implemented helper methods to create legacy package files and RPM headers for testing.

test: Implement dual signing tests

- Added DualSignTests to validate secondary signature addition when configured.
- Created stub implementations for crypto providers and key resolvers to facilitate testing.

chore: Update CI script for Playwright Chromium installation

- Modified ci-console-exports.sh to ensure deterministic Chromium binary installation for console exports tests.
- Added checks for Windows compatibility and environment variable setups for Playwright browsers.
2025-12-07 16:24:45 +02:00

3.4 KiB
Raw Blame History

Mongo Removal Plan — Phase 7 (Scheduler, Notify, Policy, Concelier, Excititor)

Purpose

Provide a phased, module-by-module plan to delete remaining Mongo storage/projects and unblock PG-T7.1.2T7.1.6. Each plan includes replacements, sequencing, test strategy, and rollback.

Global guardrails

  • No dual-write: switch callers to Postgres equivalents first, then delete Mongo projects.
  • Determinism: reuse existing ordered queries; ensure migrations are idempotent and timestamps are UTC.
  • Tests: add Postgres-backed unit/integration slices before removing Mongo tests; keep fixtures deterministic.
  • Rollback: retain Mongo projects on a feature branch until each modules Postgres tests are green.

Scheduler (PG-T7.1.2)

  1. Replace repos:
    • Implement Postgres IGraphJobStore, IOverlayStore, IPolicyRunStore (schema per docs/db/reports/scheduler-graphjobs-postgres-plan.md).
    • Wire DI in WebService/Worker/Backfill to Postgres stores only.
  2. Delete Mongo refs:
    • Remove StellaOps.Scheduler.Storage.Mongo project and package refs from csproj/solutions.
    • Drop Mongo options/session code paths.
  3. Tests:
    • Add Postgres integration fixtures for graph jobs and overlays.
    • Update Backfill tests to use Postgres schema/data.
  4. Rollback:
    • Keep Mongo project on a branch; if Postgres regression is found, revert DI to Mongo and re-run tests.

Notify (PG-T7.1.3)

  1. Inventory usages in import/backfill hooks; create Postgres equivalents for notification state and history.
  2. Switch DI to Postgres stores; add migrations for notification messages/outbox.
  3. Update tests to Postgres fixtures; remove Mongo helpers.
  4. Delete StellaOps.Notify.Storage.Mongo project and solution entries.

Policy (PG-T7.1.4)

  1. Add Postgres storage for any remaining policy documents relying on Mongo (check registries/legacy surfaces).
  2. Ensure migrations exist and are applied; switch DI to Postgres stores.
  3. Update tests to Postgres fixtures; remove Mongo mocks.
  4. Delete StellaOps.Policy.Storage.Mongo project and solution references.

Concelier (PG-T7.1.5)

  1. Finish Postgres document/raw storage + state repositories (tasks T7.1.5a/b); wire all connectors/exporters/tests to Postgres (T7.1.5c/d).
  2. Add migrations for document/state/export tables and include in offline kit.
  3. Remove Mongo packages, StellaOps.Concelier.Storage.Mongo project, solution references (T7.1.5e).
  4. Tests: Postgres-backed connector/exporter tests; replace Mongo fixtures with deterministic Postgres fixtures.

Excititor (PG-T7.1.6)

  1. Identify Mongo test harness references in Excititor; add Postgres test harness equivalents.
  2. Switch any lingering storage abstractions to Postgres (if any remain); otherwise drop Mongo-only test helpers.
  3. Remove StellaOps.Excititor.Storage.Mongo project and solution entries.
  4. Tests: run WebService/Core/Worker tests with Postgres harness; replace Mongo fixtures.

Promotion & cleanup

  • After each modules tests are green, delete the corresponding Mongo project and solution entries in a single PR per module.
  • Update module AGENTS.md to remove Mongo references and point to Postgres fixtures.
  • Add Execution Log entries in SPRINT_3407_0001_0001_postgres_cleanup.md as each module switches to TODO/DOING with this plan.

Owners

  • Scheduler: Scheduler Guild
  • Notify: Notify Guild
  • Policy: Policy Guild
  • Concelier: Concelier Guild
  • Excititor: Excititor Guild
  • Coordination: Infrastructure Guild