stela ops usage fixes roles propagation and timoeut, one account to support multi tenants, migrations consolidation, search to support documentation, doctor and open api vector db search
This commit is contained in:
@@ -1,13 +1,50 @@
|
||||
# PostgreSQL Migration Strategy
|
||||
|
||||
**Version:** 1.0
|
||||
**Last Updated:** 2025-12-03
|
||||
**Version:** 1.1
|
||||
**Last Updated:** 2026-02-22
|
||||
**Status:** Active
|
||||
|
||||
## Overview
|
||||
|
||||
This document defines the migration strategy for StellaOps PostgreSQL databases. It covers initial setup, per-release migrations, multi-instance coordination, and air-gapped operation.
|
||||
|
||||
## Consolidation Baseline (2026-02-22)
|
||||
|
||||
This strategy is under active consolidation. The current repository baseline is tracked in:
|
||||
|
||||
- `docs/db/MIGRATION_INVENTORY.md`
|
||||
- `docs/db/MIGRATION_CONSOLIDATION_PLAN.md`
|
||||
|
||||
Current-state realities that must be accounted for in operations:
|
||||
|
||||
- Multiple migration mechanisms are active (shared `MigrationRunner`, `StartupMigrationHost` wrappers, custom runners with custom history tables, compose bootstrap init SQL, and unwired migration folders).
|
||||
- CLI migration coverage is currently limited to the modules registered in `src/Platform/__Libraries/StellaOps.Platform.Database/MigrationModuleRegistry.cs`.
|
||||
- Registry module population is plug-in based (`IMigrationModulePlugin`) with one migration plug-in per web service.
|
||||
- Platform migration admin endpoints (`/api/v1/admin/migrations/*`) use the same platform-owned registry for UI/backend orchestration.
|
||||
- Several services contain migration SQL but have no verified runtime invocation path in non-test code.
|
||||
|
||||
Until consolidation is complete, treat this document as the target policy and `docs/db/MIGRATION_INVENTORY.md` as the source of truth for current implementation state.
|
||||
|
||||
## Canonical Mechanism and Entrypoint (Target)
|
||||
|
||||
For on-prem upgradeable deployments, the target state is:
|
||||
|
||||
- One migration mechanism:
|
||||
- `src/__Libraries/StellaOps.Infrastructure.Postgres/Migrations/MigrationRunner.cs`
|
||||
- `src/__Libraries/StellaOps.Infrastructure.Postgres/Migrations/StartupMigrationHost.cs`
|
||||
- One migration history table format: `<schema>.schema_migrations`
|
||||
- One operational release gate with one shared module catalog:
|
||||
- `stella system migrations-status --module all`
|
||||
- `stella system migrations-verify --module all`
|
||||
- `stella system migrations-run --module all --category release --dry-run`
|
||||
- `stella system migrations-run --module all --category release --force`
|
||||
- `GET /api/v1/admin/migrations/status?module=all`
|
||||
- `GET /api/v1/admin/migrations/verify?module=all`
|
||||
- `POST /api/v1/admin/migrations/run`
|
||||
|
||||
Service startup paths may run only automatic categories (`startup`, `seed`) through `StartupMigrationHost`.
|
||||
Pending `release` or `data` migrations are handled through the canonical CLI/API execution path before service rollout.
|
||||
|
||||
## Principles
|
||||
|
||||
1. **Forward-Only**: No down migrations. Fixes are applied as new forward migrations.
|
||||
@@ -40,9 +77,9 @@ Run automatically when application starts. Must complete within 60 seconds.
|
||||
- Large data migrations (> 10,000 rows affected)
|
||||
- Any operation requiring `ACCESS EXCLUSIVE` lock for extended periods
|
||||
|
||||
### Category B: Release Migrations (Manual/CLI)
|
||||
### Category B: Release Migrations (Manual/CLI/API)
|
||||
|
||||
Require explicit execution via CLI before deployment. Used for breaking changes.
|
||||
Require explicit execution through CLI or Platform migration admin API before deployment. Used for breaking changes.
|
||||
|
||||
**Typical Operations:**
|
||||
- Dropping deprecated columns/tables
|
||||
@@ -128,7 +165,7 @@ src/<Module>/__Libraries/StellaOps.<Module>.Storage.Postgres/
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ 5. Check for pending Category B migrations │
|
||||
│ - If any 100+ migrations are pending: FAIL STARTUP │
|
||||
│ - Log: "Run 'stellaops migrate' before deployment" │
|
||||
│ - Log: "Run 'stella system migrations-run ...' before deployment" │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
│
|
||||
▼
|
||||
@@ -155,17 +192,28 @@ src/<Module>/__Libraries/StellaOps.<Module>.Storage.Postgres/
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
### Release Migration (CLI)
|
||||
### Release Migration (CLI/API)
|
||||
|
||||
```bash
|
||||
# Before deployment - run breaking migrations
|
||||
stellaops system migrations-run --module Authority --category release
|
||||
stella system migrations-run --module Authority --category release --force
|
||||
|
||||
# Verify migration state
|
||||
stellaops system migrations-status --module Authority
|
||||
stella system migrations-status --module Authority
|
||||
|
||||
# Dry run (show what would be executed)
|
||||
stellaops system migrations-run --module Authority --dry-run
|
||||
stella system migrations-run --module Authority --category release --dry-run
|
||||
```
|
||||
|
||||
```bash
|
||||
# API equivalent (used by Platform UI/backend orchestration)
|
||||
curl -sk -H "Authorization: Bearer ${TOKEN}" \
|
||||
"https://platform.stella-ops.local/api/v1/admin/migrations/status?module=Authority"
|
||||
|
||||
curl -sk -H "Authorization: Bearer ${TOKEN}" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"module":"Authority","category":"release","dryRun":true}' \
|
||||
"https://platform.stella-ops.local/api/v1/admin/migrations/run"
|
||||
```
|
||||
|
||||
## Multi-Instance Coordination
|
||||
@@ -285,16 +333,16 @@ At startup, migrations are validated for:
|
||||
|
||||
```bash
|
||||
# 1. Review pending migrations
|
||||
stellaops system migrations-status --module all
|
||||
stella system migrations-status --module all
|
||||
|
||||
# 2. Backup database (if required)
|
||||
pg_dump -Fc stellaops > backup_$(date +%Y%m%d).dump
|
||||
|
||||
# 3. Run release migrations in maintenance window
|
||||
stellaops system migrations-run --category release --module all
|
||||
stella system migrations-run --category release --module all --force
|
||||
|
||||
# 4. Verify schema state
|
||||
stellaops system migrations-verify --module all
|
||||
stella system migrations-verify --module all
|
||||
```
|
||||
|
||||
### Deployment
|
||||
@@ -307,10 +355,10 @@ stellaops system migrations-verify --module all
|
||||
|
||||
```bash
|
||||
# Check migration status
|
||||
stellaops system migrations-status --module all
|
||||
stella system migrations-status --module all
|
||||
|
||||
# Run any data migrations (background)
|
||||
stellaops system migrations-run --category data --module all
|
||||
stella system migrations-run --category data --module all
|
||||
```
|
||||
|
||||
## Rollback Strategy
|
||||
@@ -401,7 +449,7 @@ EOF
|
||||
dotnet run --project src/Authority/StellaOps.Authority.WebService
|
||||
|
||||
# 4. Verify migration applied
|
||||
stellaops system migrations-status --module Authority
|
||||
stella system migrations-status --module Authority
|
||||
```
|
||||
|
||||
### Testing Migrations
|
||||
@@ -411,8 +459,8 @@ stellaops system migrations-status --module Authority
|
||||
dotnet test --filter "Category=Migration"
|
||||
|
||||
# Test idempotency (run twice)
|
||||
stellaops system migrations-run --module Authority
|
||||
stellaops system migrations-run --module Authority # Should be no-op
|
||||
stella system migrations-run --module Authority
|
||||
stella system migrations-run --module Authority # Should be no-op
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
@@ -454,7 +502,7 @@ ERROR: Migration checksum mismatch for '003_add_audit_columns.sql'
|
||||
```
|
||||
ERROR: Cannot start application - pending release migrations require manual execution
|
||||
Pending: 100_drop_legacy_columns.sql
|
||||
Run: stellaops system migrations-run --module Authority --category release
|
||||
Run: stella system migrations-run --module Authority --category release --force
|
||||
```
|
||||
|
||||
**Resolution**: Run CLI migration command before deployment.
|
||||
|
||||
Reference in New Issue
Block a user