stella-ops.org/git.stella-ops.org
1
0
Fork 0
Code Issues Pull Requests Actions 2 Releases Wiki Activity

stela ops usage fixes roles propagation and timoeut, one account to support multi tenants, migrations consolidation, search to support documentation, doctor and open api vector db search

Browse Source
This commit is contained in:
master
2026-02-22 19:27:54 +02:00
parent a29f438f53
commit bd8fee6ed8
373 changed files with 832097 additions and 3369 deletions
Download Patch File Download Diff File Expand all files Collapse all files

109
docs/db/MIGRATION_INVENTORY.md Normal file
View File

@@ -0,0 +1,109 @@
# Migration Inventory (Consolidation Baseline)
Date: 2026-02-22 (UTC)
Scope: `src/**/Migrations/**/*.sql` and `src/**/migrations/**/*.sql`, excluding `__Tests` and `_archived`.
## Service Migration Matrix
| Service / Module | DAL Type | Migration Locations | Migration Count | Migration Mechanism Type | Runner Entrypoint (Current) |
| --- | --- | --- | --- | --- | --- |
| Authority | Npgsql repositories (no Dapper usage observed in module) | `src/Authority/__Libraries/StellaOps.Authority.Persistence/Migrations` | 2 | Shared `MigrationRunner` resources | `CLI+PlatformAdminApi+SeedOnly`; startup migration host not wired |
| Scheduler | Dapper/Npgsql | `src/Scheduler/__Libraries/StellaOps.Scheduler.Persistence/Migrations` | 4 | Shared `MigrationRunner` resources | `CLI+PlatformAdminApi+SeedOnly`; startup migration host not wired |
| Concelier | Dapper/Npgsql | `src/Concelier/__Libraries/StellaOps.Concelier.Persistence/Migrations`, `src/Concelier/__Libraries/StellaOps.Concelier.ProofService.Postgres/Migrations` | 7 | Shared `MigrationRunner` resources | `CLI+PlatformAdminApi+SeedOnly`; startup migration host not wired |
| Policy | Mixed Npgsql + Dapper (module-level) | `src/Policy/__Libraries/StellaOps.Policy.Persistence/Migrations` | 6 | Shared `MigrationRunner` resources | `CLI+PlatformAdminApi+SeedOnly`; `PolicyMigrator` is data conversion, not schema runner |
| Notify | Npgsql repositories (no Dapper usage observed in module) | `src/Notify/__Libraries/StellaOps.Notify.Persistence/Migrations` | 2 | Shared `MigrationRunner` resources | `CLI+PlatformAdminApi+SeedOnly`; startup migration host not wired |
| Excititor | Npgsql repositories (no Dapper usage observed in module) | `src/Excititor/__Libraries/StellaOps.Excititor.Persistence/Migrations` | 3 | Shared `MigrationRunner` resources | `CLI+PlatformAdminApi+SeedOnly`; startup migration host not wired |
| Scanner | Dapper/Npgsql | `src/Scanner/__Libraries/StellaOps.Scanner.Storage/Postgres/Migrations`, `src/Scanner/__Libraries/StellaOps.Scanner.Triage/Migrations` | 35 | Shared `StartupMigrationHost` + `MigrationRunner` | `ScannerStartupHost + CLI + PlatformAdminApi` |
| AirGap | Npgsql repositories (no Dapper usage observed in module) | `src/AirGap/__Libraries/StellaOps.AirGap.Persistence/Migrations` | 1 | Shared `StartupMigrationHost` + `MigrationRunner` | `AirGapStartupHost + CLI + PlatformAdminApi` |
| TimelineIndexer | Npgsql repositories (no Dapper usage observed in module) | `src/TimelineIndexer/StellaOps.TimelineIndexer/StellaOps.TimelineIndexer.Infrastructure/Db/Migrations` | 1 | Shared `MigrationRunner` via module wrapper | `TimelineIndexerMigrationHostedService + CLI + PlatformAdminApi` |
| EvidenceLocker | Dapper/Npgsql | `src/EvidenceLocker/StellaOps.EvidenceLocker/StellaOps.EvidenceLocker.Infrastructure/Db/Migrations`, `src/EvidenceLocker/StellaOps.EvidenceLocker/Migrations` | 5 | Custom SQL runner with custom history table | `EvidenceLockerMigrationHostedService` (`evidence_schema_version`) |
| ExportCenter | Npgsql repositories (no Dapper usage observed in module) | `src/ExportCenter/StellaOps.ExportCenter/StellaOps.ExportCenter.Infrastructure/Db/Migrations` | 1 | Custom SQL runner with custom history table | `ExportCenterMigrationHostedService` (`export_schema_version`) |
| BinaryIndex | Dapper/Npgsql | `src/BinaryIndex/__Libraries/StellaOps.BinaryIndex.Persistence/Migrations`, `src/BinaryIndex/__Libraries/StellaOps.BinaryIndex.GoldenSet/Migrations` | 6 | Custom SQL runner with custom history table | Runner class exists; no runtime invocation found in non-test code |
| Plugin Registry | Npgsql repositories (no Dapper usage observed in module) | `src/Plugin/StellaOps.Plugin.Registry/Migrations` | 1 | Custom SQL runner with custom history table | Runner registered in DI; no runtime invocation found in non-test code |
| Platform | Npgsql repositories (no Dapper usage observed in module) | `src/Platform/__Libraries/StellaOps.Platform.Database/Migrations/Release` | 56 | Shared `MigrationRunner` via module wrapper | `CLI+PlatformAdminApi`; no automatic runtime invocation found in non-test code |
| Graph | Npgsql repositories (no Dapper usage observed in module) | `src/Graph/__Libraries/StellaOps.Graph.Indexer.Persistence/Migrations`, `src/Graph/__Libraries/StellaOps.Graph.Core/migrations` | 2 | Embedded SQL files only | No runtime invocation found in non-test code |
| IssuerDirectory | Npgsql repositories (no Dapper usage observed in module) | `src/IssuerDirectory/__Libraries/StellaOps.IssuerDirectory.Persistence/Migrations` | 1 | Embedded SQL files only | No runtime invocation found in non-test code |
| Findings Ledger | Npgsql repositories (no Dapper usage observed in module) | `src/Findings/StellaOps.Findings.Ledger/migrations` | 12 | Embedded SQL files only | No runtime invocation found in non-test code |
| Orchestrator | Npgsql repositories (no Dapper usage observed in module) | `src/Orchestrator/StellaOps.Orchestrator/StellaOps.Orchestrator.Infrastructure/migrations` | 8 | Embedded SQL files only | No runtime invocation found in non-test code |
| Attestor | Npgsql repositories (no Dapper usage observed in module) | `src/Attestor/__Libraries/StellaOps.Attestor.Persistence/Migrations`, `src/Attestor/__Libraries/StellaOps.Attestor.TrustVerdict/Migrations`, `src/Attestor/StellaOps.Attestor/StellaOps.Attestor.Infrastructure/Migrations` | 7 | Embedded SQL files only | No runtime invocation found in non-test code |
| Signer | Npgsql repositories (no Dapper usage observed in module) | `src/Signer/__Libraries/StellaOps.Signer.KeyManagement/Migrations` | 1 | Embedded SQL files only | No runtime invocation found in non-test code |
| Signals | Npgsql repositories (no Dapper usage observed in module) | `src/Signals/__Libraries/StellaOps.Signals.Persistence/Migrations` | 2 | Embedded SQL files only | No runtime invocation found in non-test code |
| Unknowns | Npgsql repositories (no Dapper usage observed in module) | `src/Unknowns/__Libraries/StellaOps.Unknowns.Persistence/Migrations` | 2 | Embedded SQL files only | No runtime invocation found in non-test code |
| VexHub | Dapper/Npgsql | `src/VexHub/__Libraries/StellaOps.VexHub.Persistence/Migrations` | 1 | Embedded SQL files only | No runtime invocation found in non-test code |
| VexLens | Npgsql repositories (no Dapper usage observed in module) | `src/VexLens/StellaOps.VexLens.Persistence/Migrations` | 1 | Embedded SQL files only | No runtime invocation found in non-test code |
| Remediation | Npgsql repositories (no Dapper usage observed in module) | `src/Remediation/StellaOps.Remediation.Persistence/Migrations` | 1 | Embedded SQL files only | No runtime invocation found in non-test code |
| SbomService Lineage | Npgsql repositories (no Dapper usage observed in module) | `src/SbomService/__Libraries/StellaOps.SbomService.Lineage/Persistence/Migrations` | 1 | Embedded SQL files only | No runtime invocation found in non-test code |
| AdvisoryAI Storage | Npgsql repositories (no Dapper usage observed in module) | `src/AdvisoryAI/StellaOps.AdvisoryAI/Storage/Migrations` | 1 | Embedded SQL files only | No runtime invocation found in non-test code |
| Timeline Core | Npgsql repositories (no Dapper usage observed in module) | `src/Timeline/__Libraries/StellaOps.Timeline.Core/Migrations` | 1 | Embedded SQL files only | No runtime invocation found in non-test code |
| ReachGraph Persistence (shared lib) | Dapper/Npgsql | `src/__Libraries/StellaOps.ReachGraph.Persistence/Migrations` | 1 | Embedded SQL files only | No runtime invocation found in non-test code |
| Artifact Infrastructure (shared lib) | Npgsql repositories (no Dapper usage observed in module) | `src/__Libraries/StellaOps.Artifact.Infrastructure/Migrations` | 1 | Embedded SQL files only | No runtime invocation found in non-test code |
| Evidence Persistence (shared lib) | Npgsql repositories (no Dapper usage observed in module) | `src/__Libraries/StellaOps.Evidence.Persistence/Migrations` | 1 | Embedded SQL files only | No runtime invocation found in non-test code |
| Eventing (shared lib) | Npgsql repositories (no Dapper usage observed in module) | `src/__Libraries/StellaOps.Eventing/Migrations` | 1 | Embedded SQL files only | No runtime invocation found in non-test code |
| Verdict Persistence (shared lib) | Npgsql repositories (no Dapper usage observed in module) | `src/__Libraries/StellaOps.Verdict/Persistence/Migrations` | 1 | Embedded SQL files only | No runtime invocation found in non-test code |
## Operational SQL Mechanisms (Non-assembly)
| Service / Mechanism | DAL Type | Migration Locations | Migration Count | Migration Mechanism Type | Runner Entrypoint (Current) |
| --- | --- | --- | --- | --- | --- |
| Compose bootstrap init scripts | PostgreSQL container init SQL | `devops/compose/postgres-init` | 12 | `docker-entrypoint-initdb.d` bootstrap scripts | `docker-compose.dev.yml`, `docker-compose.stella-ops.yml` Postgres service mount |
| DevOps migration scripts pack | Manual SQL script set | `devops/database/migrations` | 10 | Out-of-band SQL scripts | No direct runtime invocation found in non-test code |
## Entrypoint Legend
- `CLI+PlatformAdminApi+SeedOnly`:
- CLI: `src/Cli/StellaOps.Cli/Commands/SystemCommandBuilder.cs`
- Plug-in contract: `src/Platform/__Libraries/StellaOps.Platform.Database/IMigrationModulePlugin.cs`
- Plug-in discovery: `src/Platform/__Libraries/StellaOps.Platform.Database/MigrationModulePluginDiscovery.cs`
- Platform API: `src/Platform/StellaOps.Platform.WebService/Endpoints/MigrationAdminEndpoints.cs`
- Platform migration registry: `src/Platform/__Libraries/StellaOps.Platform.Database/MigrationModuleRegistry.cs`
- Seed endpoint (category seed path): `src/Platform/StellaOps.Platform.WebService/Endpoints/SeedEndpoints.cs`
- `CLI+PlatformAdminApi`:
- CLI: `src/Cli/StellaOps.Cli/Commands/SystemCommandBuilder.cs`
- Plug-in contract: `src/Platform/__Libraries/StellaOps.Platform.Database/IMigrationModulePlugin.cs`
- Plug-in discovery: `src/Platform/__Libraries/StellaOps.Platform.Database/MigrationModulePluginDiscovery.cs`
- Platform API: `src/Platform/StellaOps.Platform.WebService/Endpoints/MigrationAdminEndpoints.cs`
- Platform migration registry: `src/Platform/__Libraries/StellaOps.Platform.Database/MigrationModuleRegistry.cs`
- `ScannerStartupHost + CLI + PlatformAdminApi`:
- Startup host: `src/Scanner/__Libraries/StellaOps.Scanner.Storage/Extensions/ServiceCollectionExtensions.cs`
- Plug-in discovery: `src/Platform/__Libraries/StellaOps.Platform.Database/MigrationModulePluginDiscovery.cs`
- Platform API: `src/Platform/StellaOps.Platform.WebService/Endpoints/MigrationAdminEndpoints.cs`
- Platform migration registry: `src/Platform/__Libraries/StellaOps.Platform.Database/MigrationModuleRegistry.cs`
- `AirGapStartupHost + CLI + PlatformAdminApi`:
- Startup host: `src/AirGap/__Libraries/StellaOps.AirGap.Persistence/Postgres/AirGapStartupMigrationHost.cs`
- Plug-in discovery: `src/Platform/__Libraries/StellaOps.Platform.Database/MigrationModulePluginDiscovery.cs`
- Platform API: `src/Platform/StellaOps.Platform.WebService/Endpoints/MigrationAdminEndpoints.cs`
- Platform migration registry: `src/Platform/__Libraries/StellaOps.Platform.Database/MigrationModuleRegistry.cs`
- `TimelineIndexerMigrationHostedService + CLI + PlatformAdminApi`:
- Startup host: `src/TimelineIndexer/StellaOps.TimelineIndexer/StellaOps.TimelineIndexer.Infrastructure/DependencyInjection/TimelineIndexerMigrationHostedService.cs`
- Plug-in discovery: `src/Platform/__Libraries/StellaOps.Platform.Database/MigrationModulePluginDiscovery.cs`
- Platform API: `src/Platform/StellaOps.Platform.WebService/Endpoints/MigrationAdminEndpoints.cs`
- Platform migration registry: `src/Platform/__Libraries/StellaOps.Platform.Database/MigrationModuleRegistry.cs`
- `ScannerStartupHost`: `src/Scanner/__Libraries/StellaOps.Scanner.Storage/Extensions/ServiceCollectionExtensions.cs`
- `AirGapStartupHost`: `src/AirGap/__Libraries/StellaOps.AirGap.Persistence/Postgres/AirGapStartupMigrationHost.cs`
- `TimelineIndexerMigrationHostedService`: `src/TimelineIndexer/StellaOps.TimelineIndexer/StellaOps.TimelineIndexer.Infrastructure/DependencyInjection/TimelineIndexerMigrationHostedService.cs`
- `EvidenceLockerMigrationHostedService`: `src/EvidenceLocker/StellaOps.EvidenceLocker/StellaOps.EvidenceLocker.Infrastructure/DependencyInjection/EvidenceLockerMigrationHostedService.cs`
- `ExportCenterMigrationHostedService`: `src/ExportCenter/StellaOps.ExportCenter/StellaOps.ExportCenter.Infrastructure/Db/ExportCenterDbServiceExtensions.cs`
## Consolidation Notes (Current State)
- Active migration mechanism types currently in use:
- Shared `MigrationRunner` + `schema_migrations` history
- Shared `StartupMigrationHost` wrappers
- Custom runners with custom history tables (`evidence_schema_version`, `export_schema_version`, `plugin_migrations`, `binaries.schema_migrations`)
- Compose bootstrap init SQL (`docker-entrypoint-initdb.d`)
- Unwired embedded SQL migration folders
- Primary consolidation objective for this sprint:
- Reduce to one canonical runner contract and one canonical runtime entrypoint policy across startup, CLI, and compose/upgrade workflows.
- Execute UI-triggered migration flows through Platform WebService administrative APIs that consume the platform-owned migration registry.
## Target Wave Assignment (Consolidation)
Reference policy: `docs/db/MIGRATION_CONSOLIDATION_PLAN.md`
| Wave | Focus | Modules / Mechanisms |
| --- | --- | --- |
| W1 | Shared runner entrypoint expansion | Authority, Scheduler, Concelier, Policy, Notify, Excititor, Scanner, AirGap, TimelineIndexer, Platform/ReleaseOrchestrator |
| W2 | Custom history-table compatibility cutover | EvidenceLocker, ExportCenter, BinaryIndex, Plugin Registry |
| W3 | Wire currently unwired embedded SQL folders | Graph, IssuerDirectory, Findings Ledger, Orchestrator, Attestor, Signer, Signals, Unknowns, VexHub, VexLens, Remediation, SbomService Lineage, AdvisoryAI Storage, Timeline Core, ReachGraph Persistence, Artifact Infrastructure, Evidence Persistence, Eventing, Verdict Persistence |
| W4 | Migration count baseline/squash strategy | High-count chains (notably Platform release chain and any module above approved threshold) |
| W5 | On-prem rehearsal and gate | Full compose/helm deployment upgrade path with deterministic replay evidence |