git.stella-ops.org/docs/db/MIGRATION_CONSOLIDATION_PLAN.md

# Migration Consolidation Plan (On-Prem Upgradeable)

Date: 2026-02-22 (UTC)  
Status: Active sprint plan for `MGC-02`, `MGC-03`, and `MGC-12`

## Objective

Consolidate StellaOps database migrations to one canonical mechanism and one operational runner entrypoint so upgrades are deterministic for self-hosted and long-lived on-prem installations.

## Canonical Migration Mechanism

The canonical mechanism is the shared PostgreSQL migration stack in `src/__Libraries/StellaOps.Infrastructure.Postgres/Migrations/`:

- `MigrationRunner` for execution, advisory lock coordination, checksum validation, and migration history.
- `StartupMigrationHost` for automatic startup-time migration execution.
- `MigrationStatusService` for status inspection.

Module discovery for this runner is plug-in based:

- One migration module plug-in per web service implementing `src/Platform/__Libraries/StellaOps.Platform.Database/IMigrationModulePlugin.cs`.
- Consolidated module registry auto-discovers plug-ins through `src/Platform/__Libraries/StellaOps.Platform.Database/MigrationModulePluginDiscovery.cs`.
- Each service plug-in may flatten multiple migration sources (assembly + resource prefix) into one service-level runner module.
- Current built-in plug-ins are in `src/Platform/__Libraries/StellaOps.Platform.Database/MigrationModulePlugins.cs`.
- Optional external plug-in directories can be injected with `STELLAOPS_MIGRATION_PLUGIN_DIR` (path-list separated by OS path separator).
- Consolidated execution behavior:
  - When `<schema>.schema_migrations` is empty, CLI/API runner paths execute one synthesized per-service migration (`100_consolidated_<service>.sql`) built from the plug-in source set.
  - After a successful non-dry-run consolidated execution, legacy per-file history rows are backfilled so future incremental upgrades remain compatible.
  - If consolidated history exists but legacy backfill is partially missing, CLI/API runner paths auto-backfill missing legacy rows before source-set execution.
  - Therefore, bootstrap execution is one migration per service module, but the resulting history table intentionally retains per-file entries for compatibility.

Canonical history table format is:

- `<schema>.schema_migrations`
- Required columns: `migration_name`, `category`, `checksum`, `applied_at`, `applied_by`, `duration_ms`

## Canonical Migration Types and Numbering

Only these categories are valid:

- `startup`: automatic; prefix `001-099`
- `release`: manual gate; prefix `100+`
- `seed`: automatic; prefix `S001-S999`
- `data`: manual/background; prefix `DM001-DM999`

Naming standard:

- `NNN_description.sql`
- `SNNN_description.sql`
- `DMNNN_description.sql`

Legacy naming is allowed only for already shipped files. New migrations must follow the canonical format.

## Canonical Runner Entrypoint Policy

Upgrade and deployment gate entrypoint:

- `stella system migrations-status --module all`
- `stella system migrations-verify --module all`
- `stella system migrations-run --module all --category release --dry-run`
- `stella system migrations-run --module all --category release --force`

Startup behavior:

- Services may run only `startup` and `seed` categories automatically through `StartupMigrationHost` / `AddStartupMigrations(...)`.
- Pending `release` or `data` migrations must block startup when startup migration hosts are enabled.

Compose/bootstrap behavior:

- `devops/compose/postgres-init` remains bootstrap-only (first initialization).
- Release upgrades must never rely on `docker-entrypoint-initdb.d`; they must use the canonical CLI entrypoint.

UI/API execution path (implemented):

- Module registry ownership is platform-level: `src/Platform/__Libraries/StellaOps.Platform.Database/MigrationModuleRegistry.cs`.
- UI-triggered migration execution must call Platform WebService administrative APIs (no browser-direct database execution).
- Platform service applies the same consolidated-empty-db behavior as CLI using `MigrationModuleConsolidation`.
- Platform endpoint contract:
  - `GET /api/v1/admin/migrations/modules`
  - `GET /api/v1/admin/migrations/status?module=<name|all>`
  - `GET /api/v1/admin/migrations/verify?module=<name|all>`
  - `POST /api/v1/admin/migrations/run`
- Endpoint authorization policy: `platform.setup.admin` (`PlatformPolicies.SetupAdmin`).
- Endpoint implementation and server-side runner path:
  - `src/Platform/StellaOps.Platform.WebService/Endpoints/MigrationAdminEndpoints.cs`
  - `src/Platform/StellaOps.Platform.WebService/Services/PlatformMigrationAdminService.cs`

## Legacy Compatibility Mapping

The following non-canonical history tables must be migrated to `<schema>.schema_migrations` via one-time compatibility scripts/adapters:

| Module | Legacy history table | Mapping target |
| --- | --- | --- |
| EvidenceLocker | `evidence_locker.evidence_schema_version` | `evidence_locker.schema_migrations` |
| ExportCenter | `export_center.export_schema_version` | `export_center.schema_migrations` |
| BinaryIndex | `binaries.schema_migrations` (`name`, `applied_at`) | `binaries.schema_migrations` canonical columns |
| Plugin Registry | `<schema>.plugin_migrations` | `<schema>.schema_migrations` |

Compatibility rules:

- Preserve applied order and timestamps.
- Preserve or derive deterministic checksums for already-applied scripts.
- Never rewrite already-applied migration file contents.

## Cutover Waves (Runner Entrypoint Consolidation)

| Wave | Goal | Module Set | Current Mechanism | Target Mechanism | Rollback Marker |
| --- | --- | --- | --- | --- | --- |
| W0 | Governance baseline | Docs + CLI contract | Mixed | Canonical policy published | Docs and runbooks merged |
| W1 | Expand CLI/shared-runner coverage | Authority, Scheduler, Concelier, Policy, Notify, Excititor, Scanner, AirGap, TimelineIndexer, ReleaseOrchestrator | Shared runner partial coverage | Canonical CLI entrypoint for all shared-runner modules | `migrations-status` parity before/after |
| W2 | Convert custom history runners | EvidenceLocker, ExportCenter, BinaryIndex, Plugin Registry | Custom runners/history tables | Shared `MigrationRunner` with compatibility mapping | Legacy table snapshot and compare report |
| W3 | Wire currently unwired migration folders | Modules marked unwired in `docs/db/MIGRATION_INVENTORY.md` | Embedded SQL without runtime invocation | Shared runner via startup/CLI registration | Per-module dry-run evidence |
| W4 | Count consolidation/baselining | High-count modules (for example Platform release scripts) | Long historical chains | Approved baseline/squash strategy | Rehearsed upgrade from previous GA |
| W5 | On-prem rehearsal gate | Full product stack | Mixed in-flight state | Canonical entrypoint only | Deterministic replay report |

## Migration Count Consolidation Rules

- Baseline/squash is allowed only after:
  - at least one release where compatibility mappings are shipped,
  - deterministic replay tests pass for clean install and upgrade paths,
  - installed-version compatibility window is documented.
- A baseline migration must not remove upgradeability from supported in-field versions.
- Squash output must retain checksum determinism and clear mapping to superseded migration ranges.

## EF Core v10 Phase Gate

The Dapper-to-EF Core v10 transition starts only after waves W1-W5 complete and gate evidence is accepted.

Exit criteria before EF phase opens:

- One canonical migration mechanism in production paths.
- One canonical operational entrypoint in runbooks and CI/CD automation.
- Legacy history tables mapped and validated.
- Migration replay determinism proven for clean install and upgrade scenarios.

Gate decision (2026-02-22 UTC): `GO`

- Gate evidence accepted:
  - `docs/db/rehearsals/20260222_mgc06_retry_seq_after_fix6/` (clean install + idempotent rerun evidence)
  - `docs/db/rehearsals/20260222_mgc06_rollback_retry_seq3/` (sequential rollback/retry rehearsal from partial state)
- EF transition implementation sprint opened:
  - `docs/implplan/SPRINT_20260222_062_DOCS_efcore_v10_dapper_transition_phase_gate.md`

Governance boundary for EF phase:

- Migration registry ownership remains platform/infrastructure-owned in `src/Platform/__Libraries/StellaOps.Platform.Database/`.
- Migration execution from UI must continue through Platform migration admin APIs; UI must not execute database operations directly.
- Consolidated migration policy (categories, numbering, checksum/history compatibility) remains authoritative and cannot be relaxed by ORM refactors.