stela ops usage fixes roles propagation and timoeut, one account to support multi tenants, migrations consolidation, search to support documentation, doctor and open api vector db search

2026-02-22 19:27:54 +02:00
parent a29f438f53
commit bd8fee6ed8
373 changed files with 832097 additions and 3369 deletions
--- a/docs/db/MIGRATION_CONSOLIDATION_PLAN.md
+++ b/docs/db/MIGRATION_CONSOLIDATION_PLAN.md
@@ -0,0 +1,126 @@
+# Migration Consolidation Plan (On-Prem Upgradeable)
+
+Date: 2026-02-22 (UTC)  
+Status: Active sprint plan for `MGC-02`, `MGC-03`, and `MGC-12`
+
+## Objective
+
+Consolidate StellaOps database migrations to one canonical mechanism and one operational runner entrypoint so upgrades are deterministic for self-hosted and long-lived on-prem installations.
+
+## Canonical Migration Mechanism
+
+The canonical mechanism is the shared PostgreSQL migration stack in `src/__Libraries/StellaOps.Infrastructure.Postgres/Migrations/`:
+
+- `MigrationRunner` for execution, advisory lock coordination, checksum validation, and migration history.
+- `StartupMigrationHost` for automatic startup-time migration execution.
+- `MigrationStatusService` for status inspection.
+
+Module discovery for this runner is plug-in based:
+
+- One migration module plug-in per web service implementing `src/Platform/__Libraries/StellaOps.Platform.Database/IMigrationModulePlugin.cs`.
+- Consolidated module registry auto-discovers plug-ins through `src/Platform/__Libraries/StellaOps.Platform.Database/MigrationModulePluginDiscovery.cs`.
+- Current built-in plug-ins are in `src/Platform/__Libraries/StellaOps.Platform.Database/MigrationModulePlugins.cs`.
+- Optional external plug-in directories can be injected with `STELLAOPS_MIGRATION_PLUGIN_DIR` (path-list separated by OS path separator).
+
+Canonical history table format is:
+
+- `<schema>.schema_migrations`
+- Required columns: `migration_name`, `category`, `checksum`, `applied_at`, `applied_by`, `duration_ms`
+
+## Canonical Migration Types and Numbering
+
+Only these categories are valid:
+
+- `startup`: automatic; prefix `001-099`
+- `release`: manual gate; prefix `100+`
+- `seed`: automatic; prefix `S001-S999`
+- `data`: manual/background; prefix `DM001-DM999`
+
+Naming standard:
+
+- `NNN_description.sql`
+- `SNNN_description.sql`
+- `DMNNN_description.sql`
+
+Legacy naming is allowed only for already shipped files. New migrations must follow the canonical format.
+
+## Canonical Runner Entrypoint Policy
+
+Upgrade and deployment gate entrypoint:
+
+- `stella system migrations-status --module all`
+- `stella system migrations-verify --module all`
+- `stella system migrations-run --module all --category release --dry-run`
+- `stella system migrations-run --module all --category release --force`
+
+Startup behavior:
+
+- Services may run only `startup` and `seed` categories automatically through `StartupMigrationHost` / `AddStartupMigrations(...)`.
+- Pending `release` or `data` migrations must block startup when startup migration hosts are enabled.
+
+Compose/bootstrap behavior:
+
+- `devops/compose/postgres-init` remains bootstrap-only (first initialization).
+- Release upgrades must never rely on `docker-entrypoint-initdb.d`; they must use the canonical CLI entrypoint.
+
+UI/API execution path (implemented):
+
+- Module registry ownership is platform-level: `src/Platform/__Libraries/StellaOps.Platform.Database/MigrationModuleRegistry.cs`.
+- UI-triggered migration execution must call Platform WebService administrative APIs (no browser-direct database execution).
+- Platform endpoint contract:
+  - `GET /api/v1/admin/migrations/modules`
+  - `GET /api/v1/admin/migrations/status?module=<name|all>`
+  - `GET /api/v1/admin/migrations/verify?module=<name|all>`
+  - `POST /api/v1/admin/migrations/run`
+- Endpoint authorization policy: `platform.setup.admin` (`PlatformPolicies.SetupAdmin`).
+- Endpoint implementation and server-side runner path:
+  - `src/Platform/StellaOps.Platform.WebService/Endpoints/MigrationAdminEndpoints.cs`
+  - `src/Platform/StellaOps.Platform.WebService/Services/PlatformMigrationAdminService.cs`
+
+## Legacy Compatibility Mapping
+
+The following non-canonical history tables must be migrated to `<schema>.schema_migrations` via one-time compatibility scripts/adapters:
+
+| Module | Legacy history table | Mapping target |
+| --- | --- | --- |
+| EvidenceLocker | `evidence_locker.evidence_schema_version` | `evidence_locker.schema_migrations` |
+| ExportCenter | `export_center.export_schema_version` | `export_center.schema_migrations` |
+| BinaryIndex | `binaries.schema_migrations` (`name`, `applied_at`) | `binaries.schema_migrations` canonical columns |
+| Plugin Registry | `<schema>.plugin_migrations` | `<schema>.schema_migrations` |
+
+Compatibility rules:
+
+- Preserve applied order and timestamps.
+- Preserve or derive deterministic checksums for already-applied scripts.
+- Never rewrite already-applied migration file contents.
+
+## Cutover Waves (Runner Entrypoint Consolidation)
+
+| Wave | Goal | Module Set | Current Mechanism | Target Mechanism | Rollback Marker |
+| --- | --- | --- | --- | --- | --- |
+| W0 | Governance baseline | Docs + CLI contract | Mixed | Canonical policy published | Docs and runbooks merged |
+| W1 | Expand CLI/shared-runner coverage | Authority, Scheduler, Concelier, Policy, Notify, Excititor, Scanner, AirGap, TimelineIndexer, ReleaseOrchestrator | Shared runner partial coverage | Canonical CLI entrypoint for all shared-runner modules | `migrations-status` parity before/after |
+| W2 | Convert custom history runners | EvidenceLocker, ExportCenter, BinaryIndex, Plugin Registry | Custom runners/history tables | Shared `MigrationRunner` with compatibility mapping | Legacy table snapshot and compare report |
+| W3 | Wire currently unwired migration folders | Modules marked unwired in `docs/db/MIGRATION_INVENTORY.md` | Embedded SQL without runtime invocation | Shared runner via startup/CLI registration | Per-module dry-run evidence |
+| W4 | Count consolidation/baselining | High-count modules (for example Platform release scripts) | Long historical chains | Approved baseline/squash strategy | Rehearsed upgrade from previous GA |
+| W5 | On-prem rehearsal gate | Full product stack | Mixed in-flight state | Canonical entrypoint only | Deterministic replay report |
+
+## Migration Count Consolidation Rules
+
+- Baseline/squash is allowed only after:
+  - at least one release where compatibility mappings are shipped,
+  - deterministic replay tests pass for clean install and upgrade paths,
+  - installed-version compatibility window is documented.
+- A baseline migration must not remove upgradeability from supported in-field versions.
+- Squash output must retain checksum determinism and clear mapping to superseded migration ranges.
+
+## EF Core v10 Phase Gate
+
+The Dapper-to-EF Core v10 transition starts only after waves W1-W5 complete and gate evidence is accepted.
+
+Exit criteria before EF phase opens:
+
+- One canonical migration mechanism in production paths.
+- One canonical operational entrypoint in runbooks and CI/CD automation.
+- Legacy history tables mapped and validated.
+- Migration replay determinism proven for clean install and upgrade scenarios.
--- a/docs/db/MIGRATION_CONVENTIONS.md
+++ b/docs/db/MIGRATION_CONVENTIONS.md
@@ -2,6 +2,42 @@

 This document defines the standard conventions for database migrations in StellaOps.

+## Consolidation Baseline (2026-02-22)
+
+The repository currently contains multiple migration naming styles and execution paths. Use:
+
+- `docs/db/MIGRATION_INVENTORY.md`
+- `docs/db/MIGRATION_CONSOLIDATION_PLAN.md`
+
+as the authoritative current-state baseline while this conventions document defines the target standard.
+
+During the consolidation sprint:
+
+- New migrations must follow this document's naming rules.
+- Legacy folders that do not yet conform are tracked as remediation items, not exceptions to the target standard.
+- Any procedure docs (setup, CLI, compose, upgrade runbooks) must be updated in lockstep with migration mechanism changes.
+
+## Canonical Runner Entrypoint Convention
+
+For release and upgrade operations, the only supported runner entrypoint is the `stella system migrations-*` command group:
+
+```bash
+stella system migrations-status --module all
+stella system migrations-verify --module all
+stella system migrations-run --module all --category release --dry-run
+stella system migrations-run --module all --category release --force
+```
+
+Startup-hosted migration execution is limited to automatic categories (`startup`, `seed`) and must route through the shared startup migration host contract.
+
+## Legacy Compatibility Convention
+
+During consolidation, non-canonical history tables must be adapter-mapped to the canonical `<schema>.schema_migrations` structure without rewriting already-applied migration files.
+
+Current mandatory mappings are tracked in:
+
+- `docs/db/MIGRATION_CONSOLIDATION_PLAN.md` (Legacy Compatibility Mapping table)
+
 ## File Naming

 All migration files must follow the naming pattern:
--- a/docs/db/MIGRATION_INVENTORY.md
+++ b/docs/db/MIGRATION_INVENTORY.md
@@ -0,0 +1,109 @@
+# Migration Inventory (Consolidation Baseline)
+
+Date: 2026-02-22 (UTC)
+Scope: `src/**/Migrations/**/*.sql` and `src/**/migrations/**/*.sql`, excluding `__Tests` and `_archived`.
+
+## Service Migration Matrix
+
+| Service / Module | DAL Type | Migration Locations | Migration Count | Migration Mechanism Type | Runner Entrypoint (Current) |
+| --- | --- | --- | --- | --- | --- |
+| Authority | Npgsql repositories (no Dapper usage observed in module) | `src/Authority/__Libraries/StellaOps.Authority.Persistence/Migrations` | 2 | Shared `MigrationRunner` resources | `CLI+PlatformAdminApi+SeedOnly`; startup migration host not wired |
+| Scheduler | Dapper/Npgsql | `src/Scheduler/__Libraries/StellaOps.Scheduler.Persistence/Migrations` | 4 | Shared `MigrationRunner` resources | `CLI+PlatformAdminApi+SeedOnly`; startup migration host not wired |
+| Concelier | Dapper/Npgsql | `src/Concelier/__Libraries/StellaOps.Concelier.Persistence/Migrations`, `src/Concelier/__Libraries/StellaOps.Concelier.ProofService.Postgres/Migrations` | 7 | Shared `MigrationRunner` resources | `CLI+PlatformAdminApi+SeedOnly`; startup migration host not wired |
+| Policy | Mixed Npgsql + Dapper (module-level) | `src/Policy/__Libraries/StellaOps.Policy.Persistence/Migrations` | 6 | Shared `MigrationRunner` resources | `CLI+PlatformAdminApi+SeedOnly`; `PolicyMigrator` is data conversion, not schema runner |
+| Notify | Npgsql repositories (no Dapper usage observed in module) | `src/Notify/__Libraries/StellaOps.Notify.Persistence/Migrations` | 2 | Shared `MigrationRunner` resources | `CLI+PlatformAdminApi+SeedOnly`; startup migration host not wired |
+| Excititor | Npgsql repositories (no Dapper usage observed in module) | `src/Excititor/__Libraries/StellaOps.Excititor.Persistence/Migrations` | 3 | Shared `MigrationRunner` resources | `CLI+PlatformAdminApi+SeedOnly`; startup migration host not wired |
+| Scanner | Dapper/Npgsql | `src/Scanner/__Libraries/StellaOps.Scanner.Storage/Postgres/Migrations`, `src/Scanner/__Libraries/StellaOps.Scanner.Triage/Migrations` | 35 | Shared `StartupMigrationHost` + `MigrationRunner` | `ScannerStartupHost + CLI + PlatformAdminApi` |
+| AirGap | Npgsql repositories (no Dapper usage observed in module) | `src/AirGap/__Libraries/StellaOps.AirGap.Persistence/Migrations` | 1 | Shared `StartupMigrationHost` + `MigrationRunner` | `AirGapStartupHost + CLI + PlatformAdminApi` |
+| TimelineIndexer | Npgsql repositories (no Dapper usage observed in module) | `src/TimelineIndexer/StellaOps.TimelineIndexer/StellaOps.TimelineIndexer.Infrastructure/Db/Migrations` | 1 | Shared `MigrationRunner` via module wrapper | `TimelineIndexerMigrationHostedService + CLI + PlatformAdminApi` |
+| EvidenceLocker | Dapper/Npgsql | `src/EvidenceLocker/StellaOps.EvidenceLocker/StellaOps.EvidenceLocker.Infrastructure/Db/Migrations`, `src/EvidenceLocker/StellaOps.EvidenceLocker/Migrations` | 5 | Custom SQL runner with custom history table | `EvidenceLockerMigrationHostedService` (`evidence_schema_version`) |
+| ExportCenter | Npgsql repositories (no Dapper usage observed in module) | `src/ExportCenter/StellaOps.ExportCenter/StellaOps.ExportCenter.Infrastructure/Db/Migrations` | 1 | Custom SQL runner with custom history table | `ExportCenterMigrationHostedService` (`export_schema_version`) |
+| BinaryIndex | Dapper/Npgsql | `src/BinaryIndex/__Libraries/StellaOps.BinaryIndex.Persistence/Migrations`, `src/BinaryIndex/__Libraries/StellaOps.BinaryIndex.GoldenSet/Migrations` | 6 | Custom SQL runner with custom history table | Runner class exists; no runtime invocation found in non-test code |
+| Plugin Registry | Npgsql repositories (no Dapper usage observed in module) | `src/Plugin/StellaOps.Plugin.Registry/Migrations` | 1 | Custom SQL runner with custom history table | Runner registered in DI; no runtime invocation found in non-test code |
+| Platform | Npgsql repositories (no Dapper usage observed in module) | `src/Platform/__Libraries/StellaOps.Platform.Database/Migrations/Release` | 56 | Shared `MigrationRunner` via module wrapper | `CLI+PlatformAdminApi`; no automatic runtime invocation found in non-test code |
+| Graph | Npgsql repositories (no Dapper usage observed in module) | `src/Graph/__Libraries/StellaOps.Graph.Indexer.Persistence/Migrations`, `src/Graph/__Libraries/StellaOps.Graph.Core/migrations` | 2 | Embedded SQL files only | No runtime invocation found in non-test code |
+| IssuerDirectory | Npgsql repositories (no Dapper usage observed in module) | `src/IssuerDirectory/__Libraries/StellaOps.IssuerDirectory.Persistence/Migrations` | 1 | Embedded SQL files only | No runtime invocation found in non-test code |
+| Findings Ledger | Npgsql repositories (no Dapper usage observed in module) | `src/Findings/StellaOps.Findings.Ledger/migrations` | 12 | Embedded SQL files only | No runtime invocation found in non-test code |
+| Orchestrator | Npgsql repositories (no Dapper usage observed in module) | `src/Orchestrator/StellaOps.Orchestrator/StellaOps.Orchestrator.Infrastructure/migrations` | 8 | Embedded SQL files only | No runtime invocation found in non-test code |
+| Attestor | Npgsql repositories (no Dapper usage observed in module) | `src/Attestor/__Libraries/StellaOps.Attestor.Persistence/Migrations`, `src/Attestor/__Libraries/StellaOps.Attestor.TrustVerdict/Migrations`, `src/Attestor/StellaOps.Attestor/StellaOps.Attestor.Infrastructure/Migrations` | 7 | Embedded SQL files only | No runtime invocation found in non-test code |
+| Signer | Npgsql repositories (no Dapper usage observed in module) | `src/Signer/__Libraries/StellaOps.Signer.KeyManagement/Migrations` | 1 | Embedded SQL files only | No runtime invocation found in non-test code |
+| Signals | Npgsql repositories (no Dapper usage observed in module) | `src/Signals/__Libraries/StellaOps.Signals.Persistence/Migrations` | 2 | Embedded SQL files only | No runtime invocation found in non-test code |
+| Unknowns | Npgsql repositories (no Dapper usage observed in module) | `src/Unknowns/__Libraries/StellaOps.Unknowns.Persistence/Migrations` | 2 | Embedded SQL files only | No runtime invocation found in non-test code |
+| VexHub | Dapper/Npgsql | `src/VexHub/__Libraries/StellaOps.VexHub.Persistence/Migrations` | 1 | Embedded SQL files only | No runtime invocation found in non-test code |
+| VexLens | Npgsql repositories (no Dapper usage observed in module) | `src/VexLens/StellaOps.VexLens.Persistence/Migrations` | 1 | Embedded SQL files only | No runtime invocation found in non-test code |
+| Remediation | Npgsql repositories (no Dapper usage observed in module) | `src/Remediation/StellaOps.Remediation.Persistence/Migrations` | 1 | Embedded SQL files only | No runtime invocation found in non-test code |
+| SbomService Lineage | Npgsql repositories (no Dapper usage observed in module) | `src/SbomService/__Libraries/StellaOps.SbomService.Lineage/Persistence/Migrations` | 1 | Embedded SQL files only | No runtime invocation found in non-test code |
+| AdvisoryAI Storage | Npgsql repositories (no Dapper usage observed in module) | `src/AdvisoryAI/StellaOps.AdvisoryAI/Storage/Migrations` | 1 | Embedded SQL files only | No runtime invocation found in non-test code |
+| Timeline Core | Npgsql repositories (no Dapper usage observed in module) | `src/Timeline/__Libraries/StellaOps.Timeline.Core/Migrations` | 1 | Embedded SQL files only | No runtime invocation found in non-test code |
+| ReachGraph Persistence (shared lib) | Dapper/Npgsql | `src/__Libraries/StellaOps.ReachGraph.Persistence/Migrations` | 1 | Embedded SQL files only | No runtime invocation found in non-test code |
+| Artifact Infrastructure (shared lib) | Npgsql repositories (no Dapper usage observed in module) | `src/__Libraries/StellaOps.Artifact.Infrastructure/Migrations` | 1 | Embedded SQL files only | No runtime invocation found in non-test code |
+| Evidence Persistence (shared lib) | Npgsql repositories (no Dapper usage observed in module) | `src/__Libraries/StellaOps.Evidence.Persistence/Migrations` | 1 | Embedded SQL files only | No runtime invocation found in non-test code |
+| Eventing (shared lib) | Npgsql repositories (no Dapper usage observed in module) | `src/__Libraries/StellaOps.Eventing/Migrations` | 1 | Embedded SQL files only | No runtime invocation found in non-test code |
+| Verdict Persistence (shared lib) | Npgsql repositories (no Dapper usage observed in module) | `src/__Libraries/StellaOps.Verdict/Persistence/Migrations` | 1 | Embedded SQL files only | No runtime invocation found in non-test code |
+
+## Operational SQL Mechanisms (Non-assembly)
+
+| Service / Mechanism | DAL Type | Migration Locations | Migration Count | Migration Mechanism Type | Runner Entrypoint (Current) |
+| --- | --- | --- | --- | --- | --- |
+| Compose bootstrap init scripts | PostgreSQL container init SQL | `devops/compose/postgres-init` | 12 | `docker-entrypoint-initdb.d` bootstrap scripts | `docker-compose.dev.yml`, `docker-compose.stella-ops.yml` Postgres service mount |
+| DevOps migration scripts pack | Manual SQL script set | `devops/database/migrations` | 10 | Out-of-band SQL scripts | No direct runtime invocation found in non-test code |
+
+## Entrypoint Legend
+
+- `CLI+PlatformAdminApi+SeedOnly`:
+  - CLI: `src/Cli/StellaOps.Cli/Commands/SystemCommandBuilder.cs`
+  - Plug-in contract: `src/Platform/__Libraries/StellaOps.Platform.Database/IMigrationModulePlugin.cs`
+  - Plug-in discovery: `src/Platform/__Libraries/StellaOps.Platform.Database/MigrationModulePluginDiscovery.cs`
+  - Platform API: `src/Platform/StellaOps.Platform.WebService/Endpoints/MigrationAdminEndpoints.cs`
+  - Platform migration registry: `src/Platform/__Libraries/StellaOps.Platform.Database/MigrationModuleRegistry.cs`
+  - Seed endpoint (category seed path): `src/Platform/StellaOps.Platform.WebService/Endpoints/SeedEndpoints.cs`
+- `CLI+PlatformAdminApi`:
+  - CLI: `src/Cli/StellaOps.Cli/Commands/SystemCommandBuilder.cs`
+  - Plug-in contract: `src/Platform/__Libraries/StellaOps.Platform.Database/IMigrationModulePlugin.cs`
+  - Plug-in discovery: `src/Platform/__Libraries/StellaOps.Platform.Database/MigrationModulePluginDiscovery.cs`
+  - Platform API: `src/Platform/StellaOps.Platform.WebService/Endpoints/MigrationAdminEndpoints.cs`
+  - Platform migration registry: `src/Platform/__Libraries/StellaOps.Platform.Database/MigrationModuleRegistry.cs`
+- `ScannerStartupHost + CLI + PlatformAdminApi`:
+  - Startup host: `src/Scanner/__Libraries/StellaOps.Scanner.Storage/Extensions/ServiceCollectionExtensions.cs`
+  - Plug-in discovery: `src/Platform/__Libraries/StellaOps.Platform.Database/MigrationModulePluginDiscovery.cs`
+  - Platform API: `src/Platform/StellaOps.Platform.WebService/Endpoints/MigrationAdminEndpoints.cs`
+  - Platform migration registry: `src/Platform/__Libraries/StellaOps.Platform.Database/MigrationModuleRegistry.cs`
+- `AirGapStartupHost + CLI + PlatformAdminApi`:
+  - Startup host: `src/AirGap/__Libraries/StellaOps.AirGap.Persistence/Postgres/AirGapStartupMigrationHost.cs`
+  - Plug-in discovery: `src/Platform/__Libraries/StellaOps.Platform.Database/MigrationModulePluginDiscovery.cs`
+  - Platform API: `src/Platform/StellaOps.Platform.WebService/Endpoints/MigrationAdminEndpoints.cs`
+  - Platform migration registry: `src/Platform/__Libraries/StellaOps.Platform.Database/MigrationModuleRegistry.cs`
+- `TimelineIndexerMigrationHostedService + CLI + PlatformAdminApi`:
+  - Startup host: `src/TimelineIndexer/StellaOps.TimelineIndexer/StellaOps.TimelineIndexer.Infrastructure/DependencyInjection/TimelineIndexerMigrationHostedService.cs`
+  - Plug-in discovery: `src/Platform/__Libraries/StellaOps.Platform.Database/MigrationModulePluginDiscovery.cs`
+  - Platform API: `src/Platform/StellaOps.Platform.WebService/Endpoints/MigrationAdminEndpoints.cs`
+  - Platform migration registry: `src/Platform/__Libraries/StellaOps.Platform.Database/MigrationModuleRegistry.cs`
+- `ScannerStartupHost`: `src/Scanner/__Libraries/StellaOps.Scanner.Storage/Extensions/ServiceCollectionExtensions.cs`
+- `AirGapStartupHost`: `src/AirGap/__Libraries/StellaOps.AirGap.Persistence/Postgres/AirGapStartupMigrationHost.cs`
+- `TimelineIndexerMigrationHostedService`: `src/TimelineIndexer/StellaOps.TimelineIndexer/StellaOps.TimelineIndexer.Infrastructure/DependencyInjection/TimelineIndexerMigrationHostedService.cs`
+- `EvidenceLockerMigrationHostedService`: `src/EvidenceLocker/StellaOps.EvidenceLocker/StellaOps.EvidenceLocker.Infrastructure/DependencyInjection/EvidenceLockerMigrationHostedService.cs`
+- `ExportCenterMigrationHostedService`: `src/ExportCenter/StellaOps.ExportCenter/StellaOps.ExportCenter.Infrastructure/Db/ExportCenterDbServiceExtensions.cs`
+
+## Consolidation Notes (Current State)
+
+- Active migration mechanism types currently in use:
+  - Shared `MigrationRunner` + `schema_migrations` history
+  - Shared `StartupMigrationHost` wrappers
+  - Custom runners with custom history tables (`evidence_schema_version`, `export_schema_version`, `plugin_migrations`, `binaries.schema_migrations`)
+  - Compose bootstrap init SQL (`docker-entrypoint-initdb.d`)
+  - Unwired embedded SQL migration folders
+- Primary consolidation objective for this sprint:
+  - Reduce to one canonical runner contract and one canonical runtime entrypoint policy across startup, CLI, and compose/upgrade workflows.
+  - Execute UI-triggered migration flows through Platform WebService administrative APIs that consume the platform-owned migration registry.
+
+## Target Wave Assignment (Consolidation)
+
+Reference policy: `docs/db/MIGRATION_CONSOLIDATION_PLAN.md`
+
+| Wave | Focus | Modules / Mechanisms |
+| --- | --- | --- |
+| W1 | Shared runner entrypoint expansion | Authority, Scheduler, Concelier, Policy, Notify, Excititor, Scanner, AirGap, TimelineIndexer, Platform/ReleaseOrchestrator |
+| W2 | Custom history-table compatibility cutover | EvidenceLocker, ExportCenter, BinaryIndex, Plugin Registry |
+| W3 | Wire currently unwired embedded SQL folders | Graph, IssuerDirectory, Findings Ledger, Orchestrator, Attestor, Signer, Signals, Unknowns, VexHub, VexLens, Remediation, SbomService Lineage, AdvisoryAI Storage, Timeline Core, ReachGraph Persistence, Artifact Infrastructure, Evidence Persistence, Eventing, Verdict Persistence |
+| W4 | Migration count baseline/squash strategy | High-count chains (notably Platform release chain and any module above approved threshold) |
+| W5 | On-prem rehearsal and gate | Full compose/helm deployment upgrade path with deterministic replay evidence |
--- a/docs/db/MIGRATION_STRATEGY.md
+++ b/docs/db/MIGRATION_STRATEGY.md
@@ -1,13 +1,50 @@
 # PostgreSQL Migration Strategy

-**Version:** 1.0
-**Last Updated:** 2025-12-03
+**Version:** 1.1
+**Last Updated:** 2026-02-22
 **Status:** Active

 ## Overview

 This document defines the migration strategy for StellaOps PostgreSQL databases. It covers initial setup, per-release migrations, multi-instance coordination, and air-gapped operation.

+## Consolidation Baseline (2026-02-22)
+
+This strategy is under active consolidation. The current repository baseline is tracked in:
+
+- `docs/db/MIGRATION_INVENTORY.md`
+- `docs/db/MIGRATION_CONSOLIDATION_PLAN.md`
+
+Current-state realities that must be accounted for in operations:
+
+- Multiple migration mechanisms are active (shared `MigrationRunner`, `StartupMigrationHost` wrappers, custom runners with custom history tables, compose bootstrap init SQL, and unwired migration folders).
+- CLI migration coverage is currently limited to the modules registered in `src/Platform/__Libraries/StellaOps.Platform.Database/MigrationModuleRegistry.cs`.
+- Registry module population is plug-in based (`IMigrationModulePlugin`) with one migration plug-in per web service.
+- Platform migration admin endpoints (`/api/v1/admin/migrations/*`) use the same platform-owned registry for UI/backend orchestration.
+- Several services contain migration SQL but have no verified runtime invocation path in non-test code.
+
+Until consolidation is complete, treat this document as the target policy and `docs/db/MIGRATION_INVENTORY.md` as the source of truth for current implementation state.
+
+## Canonical Mechanism and Entrypoint (Target)
+
+For on-prem upgradeable deployments, the target state is:
+
+- One migration mechanism:
+  - `src/__Libraries/StellaOps.Infrastructure.Postgres/Migrations/MigrationRunner.cs`
+  - `src/__Libraries/StellaOps.Infrastructure.Postgres/Migrations/StartupMigrationHost.cs`
+- One migration history table format: `<schema>.schema_migrations`
+- One operational release gate with one shared module catalog:
+  - `stella system migrations-status --module all`
+  - `stella system migrations-verify --module all`
+  - `stella system migrations-run --module all --category release --dry-run`
+  - `stella system migrations-run --module all --category release --force`
+  - `GET /api/v1/admin/migrations/status?module=all`
+  - `GET /api/v1/admin/migrations/verify?module=all`
+  - `POST /api/v1/admin/migrations/run`
+
+Service startup paths may run only automatic categories (`startup`, `seed`) through `StartupMigrationHost`.
+Pending `release` or `data` migrations are handled through the canonical CLI/API execution path before service rollout.
+
 ## Principles

 1. **Forward-Only**: No down migrations. Fixes are applied as new forward migrations.
@@ -40,9 +77,9 @@ Run automatically when application starts. Must complete within 60 seconds.
 - Large data migrations (> 10,000 rows affected)
 - Any operation requiring `ACCESS EXCLUSIVE` lock for extended periods

-### Category B: Release Migrations (Manual/CLI)
+### Category B: Release Migrations (Manual/CLI/API)

-Require explicit execution via CLI before deployment. Used for breaking changes.
+Require explicit execution through CLI or Platform migration admin API before deployment. Used for breaking changes.

 **Typical Operations:**
 - Dropping deprecated columns/tables
@@ -128,7 +165,7 @@ src/<Module>/__Libraries/StellaOps.<Module>.Storage.Postgres/
 ┌─────────────────────────────────────────────────────────────┐
 │  5. Check for pending Category B migrations                  │
 │     - If any 100+ migrations are pending: FAIL STARTUP      │
-│     - Log: "Run 'stellaops migrate' before deployment"      │
+│     - Log: "Run 'stella system migrations-run ...' before deployment" │
 └─────────────────────────────────────────────────────────────┘
                              │
                              ▼
@@ -155,17 +192,28 @@ src/<Module>/__Libraries/StellaOps.<Module>.Storage.Postgres/
 └─────────────────────────────────────────────────────────────┘
 ```

-### Release Migration (CLI)
+### Release Migration (CLI/API)

 ```bash
 # Before deployment - run breaking migrations
-stellaops system migrations-run --module Authority --category release
+stella system migrations-run --module Authority --category release --force

 # Verify migration state
-stellaops system migrations-status --module Authority
+stella system migrations-status --module Authority

 # Dry run (show what would be executed)
-stellaops system migrations-run --module Authority --dry-run
+stella system migrations-run --module Authority --category release --dry-run
+```
+
+```bash
+# API equivalent (used by Platform UI/backend orchestration)
+curl -sk -H "Authorization: Bearer ${TOKEN}" \
+  "https://platform.stella-ops.local/api/v1/admin/migrations/status?module=Authority"
+
+curl -sk -H "Authorization: Bearer ${TOKEN}" \
+  -H "Content-Type: application/json" \
+  -d '{"module":"Authority","category":"release","dryRun":true}' \
+  "https://platform.stella-ops.local/api/v1/admin/migrations/run"
 ```

 ## Multi-Instance Coordination
@@ -285,16 +333,16 @@ At startup, migrations are validated for:

 ```bash
 # 1. Review pending migrations
-stellaops system migrations-status --module all
+stella system migrations-status --module all

 # 2. Backup database (if required)
 pg_dump -Fc stellaops > backup_$(date +%Y%m%d).dump

 # 3. Run release migrations in maintenance window
-stellaops system migrations-run --category release --module all
+stella system migrations-run --category release --module all --force

 # 4. Verify schema state
-stellaops system migrations-verify --module all
+stella system migrations-verify --module all
 ```

 ### Deployment
@@ -307,10 +355,10 @@ stellaops system migrations-verify --module all

 ```bash
 # Check migration status
-stellaops system migrations-status --module all
+stella system migrations-status --module all

 # Run any data migrations (background)
-stellaops system migrations-run --category data --module all
+stella system migrations-run --category data --module all
 ```

 ## Rollback Strategy
@@ -401,7 +449,7 @@ EOF
 dotnet run --project src/Authority/StellaOps.Authority.WebService

 # 4. Verify migration applied
-stellaops system migrations-status --module Authority
+stella system migrations-status --module Authority
 ```

 ### Testing Migrations
@@ -411,8 +459,8 @@ stellaops system migrations-status --module Authority
 dotnet test --filter "Category=Migration"

 # Test idempotency (run twice)
-stellaops system migrations-run --module Authority
-stellaops system migrations-run --module Authority  # Should be no-op
+stella system migrations-run --module Authority
+stella system migrations-run --module Authority  # Should be no-op
 ```

 ## Troubleshooting
@@ -454,7 +502,7 @@ ERROR: Migration checksum mismatch for '003_add_audit_columns.sql'
 ```
 ERROR: Cannot start application - pending release migrations require manual execution
  Pending: 100_drop_legacy_columns.sql
-  Run: stellaops system migrations-run --module Authority --category release
+  Run: stella system migrations-run --module Authority --category release --force
 ```

 **Resolution**: Run CLI migration command before deployment.
--- a/docs/db/README.md
+++ b/docs/db/README.md
@@ -12,6 +12,10 @@ This directory contains all documentation related to the StellaOps database arch
 | [RULES.md](./RULES.md) | Database coding rules, patterns, and constraints for all developers |
 | [CONVERSION_PLAN.md](./CONVERSION_PLAN.md) | Strategic plan for MongoDB to PostgreSQL conversion |
 | [VERIFICATION.md](./VERIFICATION.md) | Testing and verification requirements for database changes |
+| [MIGRATION_STRATEGY.md](./MIGRATION_STRATEGY.md) | Canonical migration strategy and category semantics |
+| [MIGRATION_CONVENTIONS.md](./MIGRATION_CONVENTIONS.md) | Migration naming, numbering, and runner conventions |
+| [MIGRATION_INVENTORY.md](./MIGRATION_INVENTORY.md) | Current-state service migration matrix (DAL, locations, counts, runner) |
+| [MIGRATION_CONSOLIDATION_PLAN.md](./MIGRATION_CONSOLIDATION_PLAN.md) | Runner consolidation waves and legacy compatibility mapping for on-prem upgrades |

 ## Task Definitions