stela ops usage fixes roles propagation and timoeut, one account to support multi tenants, migrations consolidation, search to support documentation, doctor and open api vector db search
This commit is contained in:
master
@@ -66,6 +66,85 @@ For the detailed contract, see `docs/api/overview.md`. The stable rules to keep
|
||||
- Streaming: some endpoints use NDJSON (`application/x-ndjson`) for deterministic, resumable tile/record streams.
|
||||
- Offline-first: workflows must remain runnable in air-gapped mode using Offline Kit bundles and locally verifiable signatures.
|
||||
|
||||
## stella system migrations-run / migrations-status / migrations-verify
|
||||
|
||||
Database migration command group for SQL migrations managed by the shared PostgreSQL migration tooling.
|
||||
|
||||
### Synopsis
|
||||
|
||||
```bash
|
||||
stella system migrations-status [--module <name>|all] [--connection <connection-string>]
|
||||
stella system migrations-verify [--module <name>|all] [--connection <connection-string>]
|
||||
stella system migrations-run [--module <name>|all] [--category <startup|release|seed|data>] [--dry-run] [--force]
|
||||
```
|
||||
|
||||
### Options
|
||||
|
||||
| Option | Description |
|
||||
| --- | --- |
|
||||
| `--module` | Module name filter (`AirGap`, `Authority`, `Concelier`, `Excititor`, `Notify`, `Platform`, `Policy`, `Scanner`, `Scheduler`, `TimelineIndexer`, `all`). |
|
||||
| `--category` | Migration category: `startup`, `release`, `seed`, `data`. |
|
||||
| `--dry-run` | Preview migrations without executing SQL. |
|
||||
| `--force` | Required to execute `release` category migrations. |
|
||||
| `--connection` | Override PostgreSQL connection string for command execution. |
|
||||
| `--timeout` | Timeout in seconds per migration command execution. |
|
||||
|
||||
### Examples
|
||||
|
||||
```bash
|
||||
# Status across currently registered modules
|
||||
stella system migrations-status --module all
|
||||
|
||||
# Checksum validation
|
||||
stella system migrations-verify --module all
|
||||
|
||||
# Preview release migrations
|
||||
stella system migrations-run --module all --category release --dry-run
|
||||
|
||||
# Execute release migrations
|
||||
stella system migrations-run --module all --category release --force
|
||||
```
|
||||
|
||||
### Current Coverage Notes
|
||||
|
||||
- CLI module coverage is currently defined in `src/Platform/__Libraries/StellaOps.Platform.Database/MigrationModuleRegistry.cs`.
|
||||
- CLI module coverage is auto-discovered from one migration plug-in per web service (`IMigrationModulePlugin`).
|
||||
- Registry ownership is platform-level so the same module catalog is reused by CLI and Platform migration admin APIs.
|
||||
- Current registry coverage includes: `AirGap`, `Authority`, `Concelier`, `Excititor`, `Notify`, `Platform`, `Policy`, `Scanner`, `Scheduler`, `TimelineIndexer`.
|
||||
- Not all migration folders in the repository are currently wired to runtime execution.
|
||||
- Use `docs/db/MIGRATION_INVENTORY.md` for the current full matrix of migration locations, counts, and runner entrypoints.
|
||||
- Consolidation target policy and cutover waves are defined in `docs/db/MIGRATION_CONSOLIDATION_PLAN.md`.
|
||||
- For upgradeable on-prem operations, this command group is the canonical release-migration entrypoint.
|
||||
|
||||
### Platform Migration Admin API (UI / backend orchestration)
|
||||
|
||||
UI-driven migration operations must execute through Platform WebService (server-side only, no browser-direct PostgreSQL access).
|
||||
|
||||
- Base route: `/api/v1/admin/migrations`
|
||||
- Authorization: `platform.setup.admin` (`PlatformPolicies.SetupAdmin`)
|
||||
- Backing registry: `src/Platform/__Libraries/StellaOps.Platform.Database/MigrationModuleRegistry.cs`
|
||||
- Endpoint implementation: `src/Platform/StellaOps.Platform.WebService/Endpoints/MigrationAdminEndpoints.cs`
|
||||
|
||||
API operations:
|
||||
|
||||
- `GET /api/v1/admin/migrations/modules`
|
||||
- `GET /api/v1/admin/migrations/status?module=<name|all>`
|
||||
- `GET /api/v1/admin/migrations/verify?module=<name|all>`
|
||||
- `POST /api/v1/admin/migrations/run`
|
||||
- body: `{"module":"all","category":"release","dryRun":true,"force":false,"timeoutSeconds":300}`
|
||||
|
||||
Release safety guard:
|
||||
|
||||
- `category=release` requires either `dryRun=true` or `force=true`.
|
||||
|
||||
### Related Commands
|
||||
|
||||
```bash
|
||||
# Seed demo datasets (seed migrations)
|
||||
stella admin seed-demo --dry-run
|
||||
stella admin seed-demo --confirm
|
||||
```
|
||||
|
||||
## stella scan diff
|
||||
|
||||
Compare ELF binaries between two container images using section hashes.
|
||||
@@ -385,3 +464,78 @@ stella advise chat-settings update --requests-per-day 100
|
||||
# Clear user overrides
|
||||
stella advise chat-settings clear --scope user
|
||||
```
|
||||
|
||||
## stella search
|
||||
|
||||
Search AdvisoryAI Knowledge Search (AKS) across docs, API operations, and Doctor checks.
|
||||
|
||||
### Synopsis
|
||||
|
||||
```bash
|
||||
stella search "<query>" [options]
|
||||
```
|
||||
|
||||
### Options
|
||||
|
||||
| Option | Description |
|
||||
| --- | --- |
|
||||
| `--type` | Result type filter: `docs`, `api`, `doctor` (repeatable or comma-separated). |
|
||||
| `--product` | Product filter. |
|
||||
| `--version` | Version filter. |
|
||||
| `--service` | Service filter (especially for API operations). |
|
||||
| `--tag` | Tag filter (repeatable or comma-separated). |
|
||||
| `--k` | Top result count (`1..100`). |
|
||||
| `--json` | Emit stable machine-readable payload. |
|
||||
| `--verbose`, `-v` | Include debug scoring fields when available. |
|
||||
|
||||
### Examples
|
||||
|
||||
```bash
|
||||
stella search "docker login fails with x509 unknown authority"
|
||||
stella search "endpoint for agent registration token" --type api --json
|
||||
stella search "OIDC readiness checks" --type doctor --k 20
|
||||
```
|
||||
|
||||
## stella doctor suggest
|
||||
|
||||
Use AKS to suggest Doctor checks, docs, and API references for a symptom string.
|
||||
|
||||
### Synopsis
|
||||
|
||||
```bash
|
||||
stella doctor suggest "<symptom-or-error>" [options]
|
||||
```
|
||||
|
||||
### Options
|
||||
|
||||
| Option | Description |
|
||||
| --- | --- |
|
||||
| `--product` | Product filter. |
|
||||
| `--version` | Version filter. |
|
||||
| `--k` | Top result count (`1..100`). |
|
||||
| `--json` | Emit stable machine-readable payload. |
|
||||
| `--verbose`, `-v` | Include debug scoring fields when available. |
|
||||
|
||||
### Examples
|
||||
|
||||
```bash
|
||||
stella doctor suggest "database unavailable and timeout expired"
|
||||
stella doctor suggest "gateway returns 404 on known route" --json
|
||||
```
|
||||
|
||||
## stella advisoryai index rebuild
|
||||
|
||||
Rebuild the AdvisoryAI deterministic knowledge index from local markdown, OpenAPI specs, and Doctor metadata.
|
||||
|
||||
### Synopsis
|
||||
|
||||
```bash
|
||||
stella advisoryai index rebuild [--json] [--verbose]
|
||||
```
|
||||
|
||||
### Examples
|
||||
|
||||
```bash
|
||||
stella advisoryai index rebuild
|
||||
stella advisoryai index rebuild --json
|
||||
```
|
||||
|
||||
@@ -78,6 +78,42 @@ docker compose -f docker-compose.stella-ops.yml --profile sigstore up -d
|
||||
docker compose -f docker-compose.stella-ops.yml -f docker-compose.telemetry.yml up -d
|
||||
```
|
||||
|
||||
### 4a. Migration preflight and execution
|
||||
|
||||
Run a migration preflight after bringing up the stack:
|
||||
|
||||
```bash
|
||||
# Check migration status for currently registered CLI modules
|
||||
stella system migrations-status --module all
|
||||
|
||||
# Validate checksums for currently registered CLI modules
|
||||
stella system migrations-verify --module all
|
||||
|
||||
# Optional: preview release migrations before any execution
|
||||
stella system migrations-run --module all --category release --dry-run
|
||||
```
|
||||
|
||||
If release migrations must be executed:
|
||||
|
||||
```bash
|
||||
stella system migrations-run --module all --category release --force
|
||||
stella system migrations-status --module all
|
||||
```
|
||||
|
||||
Canonical policy for upgradeable on-prem installs:
|
||||
|
||||
- Use this CLI sequence as the required migration gate before rollouts and cutovers.
|
||||
- Do not rely on Postgres init scripts for release upgrades.
|
||||
- Use `docs/db/MIGRATION_CONSOLIDATION_PLAN.md` and `docs/db/MIGRATION_INVENTORY.md` to confirm module coverage and cutover wave state.
|
||||
- UI-driven migration operations must call Platform WebService admin endpoints (`/api/v1/admin/migrations/*`) with `platform.setup.admin`; do not connect the browser directly to PostgreSQL.
|
||||
- Platform migration API implementation is in `src/Platform/StellaOps.Platform.WebService/Endpoints/MigrationAdminEndpoints.cs` and uses `src/Platform/__Libraries/StellaOps.Platform.Database/MigrationModuleRegistry.cs`.
|
||||
|
||||
Notes:
|
||||
|
||||
- Compose PostgreSQL bootstrap scripts in `devops/compose/postgres-init` run only on first database initialization.
|
||||
- Startup-hosted migrations are currently wired only for selected modules; CLI coverage is also module-limited.
|
||||
- For the authoritative current-state module matrix, use `docs/db/MIGRATION_INVENTORY.md`.
|
||||
|
||||
### 5. Verify
|
||||
|
||||
```bash
|
||||
|
||||
126
docs/db/MIGRATION_CONSOLIDATION_PLAN.md
Normal file
126
docs/db/MIGRATION_CONSOLIDATION_PLAN.md
Normal file
@@ -0,0 +1,126 @@
|
||||
# Migration Consolidation Plan (On-Prem Upgradeable)
|
||||
|
||||
Date: 2026-02-22 (UTC)
|
||||
Status: Active sprint plan for `MGC-02`, `MGC-03`, and `MGC-12`
|
||||
|
||||
## Objective
|
||||
|
||||
Consolidate StellaOps database migrations to one canonical mechanism and one operational runner entrypoint so upgrades are deterministic for self-hosted and long-lived on-prem installations.
|
||||
|
||||
## Canonical Migration Mechanism
|
||||
|
||||
The canonical mechanism is the shared PostgreSQL migration stack in `src/__Libraries/StellaOps.Infrastructure.Postgres/Migrations/`:
|
||||
|
||||
- `MigrationRunner` for execution, advisory lock coordination, checksum validation, and migration history.
|
||||
- `StartupMigrationHost` for automatic startup-time migration execution.
|
||||
- `MigrationStatusService` for status inspection.
|
||||
|
||||
Module discovery for this runner is plug-in based:
|
||||
|
||||
- One migration module plug-in per web service implementing `src/Platform/__Libraries/StellaOps.Platform.Database/IMigrationModulePlugin.cs`.
|
||||
- Consolidated module registry auto-discovers plug-ins through `src/Platform/__Libraries/StellaOps.Platform.Database/MigrationModulePluginDiscovery.cs`.
|
||||
- Current built-in plug-ins are in `src/Platform/__Libraries/StellaOps.Platform.Database/MigrationModulePlugins.cs`.
|
||||
- Optional external plug-in directories can be injected with `STELLAOPS_MIGRATION_PLUGIN_DIR` (path-list separated by OS path separator).
|
||||
|
||||
Canonical history table format is:
|
||||
|
||||
- `<schema>.schema_migrations`
|
||||
- Required columns: `migration_name`, `category`, `checksum`, `applied_at`, `applied_by`, `duration_ms`
|
||||
|
||||
## Canonical Migration Types and Numbering
|
||||
|
||||
Only these categories are valid:
|
||||
|
||||
- `startup`: automatic; prefix `001-099`
|
||||
- `release`: manual gate; prefix `100+`
|
||||
- `seed`: automatic; prefix `S001-S999`
|
||||
- `data`: manual/background; prefix `DM001-DM999`
|
||||
|
||||
Naming standard:
|
||||
|
||||
- `NNN_description.sql`
|
||||
- `SNNN_description.sql`
|
||||
- `DMNNN_description.sql`
|
||||
|
||||
Legacy naming is allowed only for already shipped files. New migrations must follow the canonical format.
|
||||
|
||||
## Canonical Runner Entrypoint Policy
|
||||
|
||||
Upgrade and deployment gate entrypoint:
|
||||
|
||||
- `stella system migrations-status --module all`
|
||||
- `stella system migrations-verify --module all`
|
||||
- `stella system migrations-run --module all --category release --dry-run`
|
||||
- `stella system migrations-run --module all --category release --force`
|
||||
|
||||
Startup behavior:
|
||||
|
||||
- Services may run only `startup` and `seed` categories automatically through `StartupMigrationHost` / `AddStartupMigrations(...)`.
|
||||
- Pending `release` or `data` migrations must block startup when startup migration hosts are enabled.
|
||||
|
||||
Compose/bootstrap behavior:
|
||||
|
||||
- `devops/compose/postgres-init` remains bootstrap-only (first initialization).
|
||||
- Release upgrades must never rely on `docker-entrypoint-initdb.d`; they must use the canonical CLI entrypoint.
|
||||
|
||||
UI/API execution path (implemented):
|
||||
|
||||
- Module registry ownership is platform-level: `src/Platform/__Libraries/StellaOps.Platform.Database/MigrationModuleRegistry.cs`.
|
||||
- UI-triggered migration execution must call Platform WebService administrative APIs (no browser-direct database execution).
|
||||
- Platform endpoint contract:
|
||||
- `GET /api/v1/admin/migrations/modules`
|
||||
- `GET /api/v1/admin/migrations/status?module=<name|all>`
|
||||
- `GET /api/v1/admin/migrations/verify?module=<name|all>`
|
||||
- `POST /api/v1/admin/migrations/run`
|
||||
- Endpoint authorization policy: `platform.setup.admin` (`PlatformPolicies.SetupAdmin`).
|
||||
- Endpoint implementation and server-side runner path:
|
||||
- `src/Platform/StellaOps.Platform.WebService/Endpoints/MigrationAdminEndpoints.cs`
|
||||
- `src/Platform/StellaOps.Platform.WebService/Services/PlatformMigrationAdminService.cs`
|
||||
|
||||
## Legacy Compatibility Mapping
|
||||
|
||||
The following non-canonical history tables must be migrated to `<schema>.schema_migrations` via one-time compatibility scripts/adapters:
|
||||
|
||||
| Module | Legacy history table | Mapping target |
|
||||
| --- | --- | --- |
|
||||
| EvidenceLocker | `evidence_locker.evidence_schema_version` | `evidence_locker.schema_migrations` |
|
||||
| ExportCenter | `export_center.export_schema_version` | `export_center.schema_migrations` |
|
||||
| BinaryIndex | `binaries.schema_migrations` (`name`, `applied_at`) | `binaries.schema_migrations` canonical columns |
|
||||
| Plugin Registry | `<schema>.plugin_migrations` | `<schema>.schema_migrations` |
|
||||
|
||||
Compatibility rules:
|
||||
|
||||
- Preserve applied order and timestamps.
|
||||
- Preserve or derive deterministic checksums for already-applied scripts.
|
||||
- Never rewrite already-applied migration file contents.
|
||||
|
||||
## Cutover Waves (Runner Entrypoint Consolidation)
|
||||
|
||||
| Wave | Goal | Module Set | Current Mechanism | Target Mechanism | Rollback Marker |
|
||||
| --- | --- | --- | --- | --- | --- |
|
||||
| W0 | Governance baseline | Docs + CLI contract | Mixed | Canonical policy published | Docs and runbooks merged |
|
||||
| W1 | Expand CLI/shared-runner coverage | Authority, Scheduler, Concelier, Policy, Notify, Excititor, Scanner, AirGap, TimelineIndexer, ReleaseOrchestrator | Shared runner partial coverage | Canonical CLI entrypoint for all shared-runner modules | `migrations-status` parity before/after |
|
||||
| W2 | Convert custom history runners | EvidenceLocker, ExportCenter, BinaryIndex, Plugin Registry | Custom runners/history tables | Shared `MigrationRunner` with compatibility mapping | Legacy table snapshot and compare report |
|
||||
| W3 | Wire currently unwired migration folders | Modules marked unwired in `docs/db/MIGRATION_INVENTORY.md` | Embedded SQL without runtime invocation | Shared runner via startup/CLI registration | Per-module dry-run evidence |
|
||||
| W4 | Count consolidation/baselining | High-count modules (for example Platform release scripts) | Long historical chains | Approved baseline/squash strategy | Rehearsed upgrade from previous GA |
|
||||
| W5 | On-prem rehearsal gate | Full product stack | Mixed in-flight state | Canonical entrypoint only | Deterministic replay report |
|
||||
|
||||
## Migration Count Consolidation Rules
|
||||
|
||||
- Baseline/squash is allowed only after:
|
||||
- at least one release where compatibility mappings are shipped,
|
||||
- deterministic replay tests pass for clean install and upgrade paths,
|
||||
- installed-version compatibility window is documented.
|
||||
- A baseline migration must not remove upgradeability from supported in-field versions.
|
||||
- Squash output must retain checksum determinism and clear mapping to superseded migration ranges.
|
||||
|
||||
## EF Core v10 Phase Gate
|
||||
|
||||
The Dapper-to-EF Core v10 transition starts only after waves W1-W5 complete and gate evidence is accepted.
|
||||
|
||||
Exit criteria before EF phase opens:
|
||||
|
||||
- One canonical migration mechanism in production paths.
|
||||
- One canonical operational entrypoint in runbooks and CI/CD automation.
|
||||
- Legacy history tables mapped and validated.
|
||||
- Migration replay determinism proven for clean install and upgrade scenarios.
|
||||
@@ -2,6 +2,42 @@
|
||||
|
||||
This document defines the standard conventions for database migrations in StellaOps.
|
||||
|
||||
## Consolidation Baseline (2026-02-22)
|
||||
|
||||
The repository currently contains multiple migration naming styles and execution paths. Use:
|
||||
|
||||
- `docs/db/MIGRATION_INVENTORY.md`
|
||||
- `docs/db/MIGRATION_CONSOLIDATION_PLAN.md`
|
||||
|
||||
as the authoritative current-state baseline while this conventions document defines the target standard.
|
||||
|
||||
During the consolidation sprint:
|
||||
|
||||
- New migrations must follow this document's naming rules.
|
||||
- Legacy folders that do not yet conform are tracked as remediation items, not exceptions to the target standard.
|
||||
- Any procedure docs (setup, CLI, compose, upgrade runbooks) must be updated in lockstep with migration mechanism changes.
|
||||
|
||||
## Canonical Runner Entrypoint Convention
|
||||
|
||||
For release and upgrade operations, the only supported runner entrypoint is the `stella system migrations-*` command group:
|
||||
|
||||
```bash
|
||||
stella system migrations-status --module all
|
||||
stella system migrations-verify --module all
|
||||
stella system migrations-run --module all --category release --dry-run
|
||||
stella system migrations-run --module all --category release --force
|
||||
```
|
||||
|
||||
Startup-hosted migration execution is limited to automatic categories (`startup`, `seed`) and must route through the shared startup migration host contract.
|
||||
|
||||
## Legacy Compatibility Convention
|
||||
|
||||
During consolidation, non-canonical history tables must be adapter-mapped to the canonical `<schema>.schema_migrations` structure without rewriting already-applied migration files.
|
||||
|
||||
Current mandatory mappings are tracked in:
|
||||
|
||||
- `docs/db/MIGRATION_CONSOLIDATION_PLAN.md` (Legacy Compatibility Mapping table)
|
||||
|
||||
## File Naming
|
||||
|
||||
All migration files must follow the naming pattern:
|
||||
|
||||
109
docs/db/MIGRATION_INVENTORY.md
Normal file
109
docs/db/MIGRATION_INVENTORY.md
Normal file
@@ -0,0 +1,109 @@
|
||||
# Migration Inventory (Consolidation Baseline)
|
||||
|
||||
Date: 2026-02-22 (UTC)
|
||||
Scope: `src/**/Migrations/**/*.sql` and `src/**/migrations/**/*.sql`, excluding `__Tests` and `_archived`.
|
||||
|
||||
## Service Migration Matrix
|
||||
|
||||
| Service / Module | DAL Type | Migration Locations | Migration Count | Migration Mechanism Type | Runner Entrypoint (Current) |
|
||||
| --- | --- | --- | --- | --- | --- |
|
||||
| Authority | Npgsql repositories (no Dapper usage observed in module) | `src/Authority/__Libraries/StellaOps.Authority.Persistence/Migrations` | 2 | Shared `MigrationRunner` resources | `CLI+PlatformAdminApi+SeedOnly`; startup migration host not wired |
|
||||
| Scheduler | Dapper/Npgsql | `src/Scheduler/__Libraries/StellaOps.Scheduler.Persistence/Migrations` | 4 | Shared `MigrationRunner` resources | `CLI+PlatformAdminApi+SeedOnly`; startup migration host not wired |
|
||||
| Concelier | Dapper/Npgsql | `src/Concelier/__Libraries/StellaOps.Concelier.Persistence/Migrations`, `src/Concelier/__Libraries/StellaOps.Concelier.ProofService.Postgres/Migrations` | 7 | Shared `MigrationRunner` resources | `CLI+PlatformAdminApi+SeedOnly`; startup migration host not wired |
|
||||
| Policy | Mixed Npgsql + Dapper (module-level) | `src/Policy/__Libraries/StellaOps.Policy.Persistence/Migrations` | 6 | Shared `MigrationRunner` resources | `CLI+PlatformAdminApi+SeedOnly`; `PolicyMigrator` is data conversion, not schema runner |
|
||||
| Notify | Npgsql repositories (no Dapper usage observed in module) | `src/Notify/__Libraries/StellaOps.Notify.Persistence/Migrations` | 2 | Shared `MigrationRunner` resources | `CLI+PlatformAdminApi+SeedOnly`; startup migration host not wired |
|
||||
| Excititor | Npgsql repositories (no Dapper usage observed in module) | `src/Excititor/__Libraries/StellaOps.Excititor.Persistence/Migrations` | 3 | Shared `MigrationRunner` resources | `CLI+PlatformAdminApi+SeedOnly`; startup migration host not wired |
|
||||
| Scanner | Dapper/Npgsql | `src/Scanner/__Libraries/StellaOps.Scanner.Storage/Postgres/Migrations`, `src/Scanner/__Libraries/StellaOps.Scanner.Triage/Migrations` | 35 | Shared `StartupMigrationHost` + `MigrationRunner` | `ScannerStartupHost + CLI + PlatformAdminApi` |
|
||||
| AirGap | Npgsql repositories (no Dapper usage observed in module) | `src/AirGap/__Libraries/StellaOps.AirGap.Persistence/Migrations` | 1 | Shared `StartupMigrationHost` + `MigrationRunner` | `AirGapStartupHost + CLI + PlatformAdminApi` |
|
||||
| TimelineIndexer | Npgsql repositories (no Dapper usage observed in module) | `src/TimelineIndexer/StellaOps.TimelineIndexer/StellaOps.TimelineIndexer.Infrastructure/Db/Migrations` | 1 | Shared `MigrationRunner` via module wrapper | `TimelineIndexerMigrationHostedService + CLI + PlatformAdminApi` |
|
||||
| EvidenceLocker | Dapper/Npgsql | `src/EvidenceLocker/StellaOps.EvidenceLocker/StellaOps.EvidenceLocker.Infrastructure/Db/Migrations`, `src/EvidenceLocker/StellaOps.EvidenceLocker/Migrations` | 5 | Custom SQL runner with custom history table | `EvidenceLockerMigrationHostedService` (`evidence_schema_version`) |
|
||||
| ExportCenter | Npgsql repositories (no Dapper usage observed in module) | `src/ExportCenter/StellaOps.ExportCenter/StellaOps.ExportCenter.Infrastructure/Db/Migrations` | 1 | Custom SQL runner with custom history table | `ExportCenterMigrationHostedService` (`export_schema_version`) |
|
||||
| BinaryIndex | Dapper/Npgsql | `src/BinaryIndex/__Libraries/StellaOps.BinaryIndex.Persistence/Migrations`, `src/BinaryIndex/__Libraries/StellaOps.BinaryIndex.GoldenSet/Migrations` | 6 | Custom SQL runner with custom history table | Runner class exists; no runtime invocation found in non-test code |
|
||||
| Plugin Registry | Npgsql repositories (no Dapper usage observed in module) | `src/Plugin/StellaOps.Plugin.Registry/Migrations` | 1 | Custom SQL runner with custom history table | Runner registered in DI; no runtime invocation found in non-test code |
|
||||
| Platform | Npgsql repositories (no Dapper usage observed in module) | `src/Platform/__Libraries/StellaOps.Platform.Database/Migrations/Release` | 56 | Shared `MigrationRunner` via module wrapper | `CLI+PlatformAdminApi`; no automatic runtime invocation found in non-test code |
|
||||
| Graph | Npgsql repositories (no Dapper usage observed in module) | `src/Graph/__Libraries/StellaOps.Graph.Indexer.Persistence/Migrations`, `src/Graph/__Libraries/StellaOps.Graph.Core/migrations` | 2 | Embedded SQL files only | No runtime invocation found in non-test code |
|
||||
| IssuerDirectory | Npgsql repositories (no Dapper usage observed in module) | `src/IssuerDirectory/__Libraries/StellaOps.IssuerDirectory.Persistence/Migrations` | 1 | Embedded SQL files only | No runtime invocation found in non-test code |
|
||||
| Findings Ledger | Npgsql repositories (no Dapper usage observed in module) | `src/Findings/StellaOps.Findings.Ledger/migrations` | 12 | Embedded SQL files only | No runtime invocation found in non-test code |
|
||||
| Orchestrator | Npgsql repositories (no Dapper usage observed in module) | `src/Orchestrator/StellaOps.Orchestrator/StellaOps.Orchestrator.Infrastructure/migrations` | 8 | Embedded SQL files only | No runtime invocation found in non-test code |
|
||||
| Attestor | Npgsql repositories (no Dapper usage observed in module) | `src/Attestor/__Libraries/StellaOps.Attestor.Persistence/Migrations`, `src/Attestor/__Libraries/StellaOps.Attestor.TrustVerdict/Migrations`, `src/Attestor/StellaOps.Attestor/StellaOps.Attestor.Infrastructure/Migrations` | 7 | Embedded SQL files only | No runtime invocation found in non-test code |
|
||||
| Signer | Npgsql repositories (no Dapper usage observed in module) | `src/Signer/__Libraries/StellaOps.Signer.KeyManagement/Migrations` | 1 | Embedded SQL files only | No runtime invocation found in non-test code |
|
||||
| Signals | Npgsql repositories (no Dapper usage observed in module) | `src/Signals/__Libraries/StellaOps.Signals.Persistence/Migrations` | 2 | Embedded SQL files only | No runtime invocation found in non-test code |
|
||||
| Unknowns | Npgsql repositories (no Dapper usage observed in module) | `src/Unknowns/__Libraries/StellaOps.Unknowns.Persistence/Migrations` | 2 | Embedded SQL files only | No runtime invocation found in non-test code |
|
||||
| VexHub | Dapper/Npgsql | `src/VexHub/__Libraries/StellaOps.VexHub.Persistence/Migrations` | 1 | Embedded SQL files only | No runtime invocation found in non-test code |
|
||||
| VexLens | Npgsql repositories (no Dapper usage observed in module) | `src/VexLens/StellaOps.VexLens.Persistence/Migrations` | 1 | Embedded SQL files only | No runtime invocation found in non-test code |
|
||||
| Remediation | Npgsql repositories (no Dapper usage observed in module) | `src/Remediation/StellaOps.Remediation.Persistence/Migrations` | 1 | Embedded SQL files only | No runtime invocation found in non-test code |
|
||||
| SbomService Lineage | Npgsql repositories (no Dapper usage observed in module) | `src/SbomService/__Libraries/StellaOps.SbomService.Lineage/Persistence/Migrations` | 1 | Embedded SQL files only | No runtime invocation found in non-test code |
|
||||
| AdvisoryAI Storage | Npgsql repositories (no Dapper usage observed in module) | `src/AdvisoryAI/StellaOps.AdvisoryAI/Storage/Migrations` | 1 | Embedded SQL files only | No runtime invocation found in non-test code |
|
||||
| Timeline Core | Npgsql repositories (no Dapper usage observed in module) | `src/Timeline/__Libraries/StellaOps.Timeline.Core/Migrations` | 1 | Embedded SQL files only | No runtime invocation found in non-test code |
|
||||
| ReachGraph Persistence (shared lib) | Dapper/Npgsql | `src/__Libraries/StellaOps.ReachGraph.Persistence/Migrations` | 1 | Embedded SQL files only | No runtime invocation found in non-test code |
|
||||
| Artifact Infrastructure (shared lib) | Npgsql repositories (no Dapper usage observed in module) | `src/__Libraries/StellaOps.Artifact.Infrastructure/Migrations` | 1 | Embedded SQL files only | No runtime invocation found in non-test code |
|
||||
| Evidence Persistence (shared lib) | Npgsql repositories (no Dapper usage observed in module) | `src/__Libraries/StellaOps.Evidence.Persistence/Migrations` | 1 | Embedded SQL files only | No runtime invocation found in non-test code |
|
||||
| Eventing (shared lib) | Npgsql repositories (no Dapper usage observed in module) | `src/__Libraries/StellaOps.Eventing/Migrations` | 1 | Embedded SQL files only | No runtime invocation found in non-test code |
|
||||
| Verdict Persistence (shared lib) | Npgsql repositories (no Dapper usage observed in module) | `src/__Libraries/StellaOps.Verdict/Persistence/Migrations` | 1 | Embedded SQL files only | No runtime invocation found in non-test code |
|
||||
|
||||
## Operational SQL Mechanisms (Non-assembly)
|
||||
|
||||
| Service / Mechanism | DAL Type | Migration Locations | Migration Count | Migration Mechanism Type | Runner Entrypoint (Current) |
|
||||
| --- | --- | --- | --- | --- | --- |
|
||||
| Compose bootstrap init scripts | PostgreSQL container init SQL | `devops/compose/postgres-init` | 12 | `docker-entrypoint-initdb.d` bootstrap scripts | `docker-compose.dev.yml`, `docker-compose.stella-ops.yml` Postgres service mount |
|
||||
| DevOps migration scripts pack | Manual SQL script set | `devops/database/migrations` | 10 | Out-of-band SQL scripts | No direct runtime invocation found in non-test code |
|
||||
|
||||
## Entrypoint Legend
|
||||
|
||||
- `CLI+PlatformAdminApi+SeedOnly`:
|
||||
- CLI: `src/Cli/StellaOps.Cli/Commands/SystemCommandBuilder.cs`
|
||||
- Plug-in contract: `src/Platform/__Libraries/StellaOps.Platform.Database/IMigrationModulePlugin.cs`
|
||||
- Plug-in discovery: `src/Platform/__Libraries/StellaOps.Platform.Database/MigrationModulePluginDiscovery.cs`
|
||||
- Platform API: `src/Platform/StellaOps.Platform.WebService/Endpoints/MigrationAdminEndpoints.cs`
|
||||
- Platform migration registry: `src/Platform/__Libraries/StellaOps.Platform.Database/MigrationModuleRegistry.cs`
|
||||
- Seed endpoint (category seed path): `src/Platform/StellaOps.Platform.WebService/Endpoints/SeedEndpoints.cs`
|
||||
- `CLI+PlatformAdminApi`:
|
||||
- CLI: `src/Cli/StellaOps.Cli/Commands/SystemCommandBuilder.cs`
|
||||
- Plug-in contract: `src/Platform/__Libraries/StellaOps.Platform.Database/IMigrationModulePlugin.cs`
|
||||
- Plug-in discovery: `src/Platform/__Libraries/StellaOps.Platform.Database/MigrationModulePluginDiscovery.cs`
|
||||
- Platform API: `src/Platform/StellaOps.Platform.WebService/Endpoints/MigrationAdminEndpoints.cs`
|
||||
- Platform migration registry: `src/Platform/__Libraries/StellaOps.Platform.Database/MigrationModuleRegistry.cs`
|
||||
- `ScannerStartupHost + CLI + PlatformAdminApi`:
|
||||
- Startup host: `src/Scanner/__Libraries/StellaOps.Scanner.Storage/Extensions/ServiceCollectionExtensions.cs`
|
||||
- Plug-in discovery: `src/Platform/__Libraries/StellaOps.Platform.Database/MigrationModulePluginDiscovery.cs`
|
||||
- Platform API: `src/Platform/StellaOps.Platform.WebService/Endpoints/MigrationAdminEndpoints.cs`
|
||||
- Platform migration registry: `src/Platform/__Libraries/StellaOps.Platform.Database/MigrationModuleRegistry.cs`
|
||||
- `AirGapStartupHost + CLI + PlatformAdminApi`:
|
||||
- Startup host: `src/AirGap/__Libraries/StellaOps.AirGap.Persistence/Postgres/AirGapStartupMigrationHost.cs`
|
||||
- Plug-in discovery: `src/Platform/__Libraries/StellaOps.Platform.Database/MigrationModulePluginDiscovery.cs`
|
||||
- Platform API: `src/Platform/StellaOps.Platform.WebService/Endpoints/MigrationAdminEndpoints.cs`
|
||||
- Platform migration registry: `src/Platform/__Libraries/StellaOps.Platform.Database/MigrationModuleRegistry.cs`
|
||||
- `TimelineIndexerMigrationHostedService + CLI + PlatformAdminApi`:
|
||||
- Startup host: `src/TimelineIndexer/StellaOps.TimelineIndexer/StellaOps.TimelineIndexer.Infrastructure/DependencyInjection/TimelineIndexerMigrationHostedService.cs`
|
||||
- Plug-in discovery: `src/Platform/__Libraries/StellaOps.Platform.Database/MigrationModulePluginDiscovery.cs`
|
||||
- Platform API: `src/Platform/StellaOps.Platform.WebService/Endpoints/MigrationAdminEndpoints.cs`
|
||||
- Platform migration registry: `src/Platform/__Libraries/StellaOps.Platform.Database/MigrationModuleRegistry.cs`
|
||||
- `ScannerStartupHost`: `src/Scanner/__Libraries/StellaOps.Scanner.Storage/Extensions/ServiceCollectionExtensions.cs`
|
||||
- `AirGapStartupHost`: `src/AirGap/__Libraries/StellaOps.AirGap.Persistence/Postgres/AirGapStartupMigrationHost.cs`
|
||||
- `TimelineIndexerMigrationHostedService`: `src/TimelineIndexer/StellaOps.TimelineIndexer/StellaOps.TimelineIndexer.Infrastructure/DependencyInjection/TimelineIndexerMigrationHostedService.cs`
|
||||
- `EvidenceLockerMigrationHostedService`: `src/EvidenceLocker/StellaOps.EvidenceLocker/StellaOps.EvidenceLocker.Infrastructure/DependencyInjection/EvidenceLockerMigrationHostedService.cs`
|
||||
- `ExportCenterMigrationHostedService`: `src/ExportCenter/StellaOps.ExportCenter/StellaOps.ExportCenter.Infrastructure/Db/ExportCenterDbServiceExtensions.cs`
|
||||
|
||||
## Consolidation Notes (Current State)
|
||||
|
||||
- Active migration mechanism types currently in use:
|
||||
- Shared `MigrationRunner` + `schema_migrations` history
|
||||
- Shared `StartupMigrationHost` wrappers
|
||||
- Custom runners with custom history tables (`evidence_schema_version`, `export_schema_version`, `plugin_migrations`, `binaries.schema_migrations`)
|
||||
- Compose bootstrap init SQL (`docker-entrypoint-initdb.d`)
|
||||
- Unwired embedded SQL migration folders
|
||||
- Primary consolidation objective for this sprint:
|
||||
- Reduce to one canonical runner contract and one canonical runtime entrypoint policy across startup, CLI, and compose/upgrade workflows.
|
||||
- Execute UI-triggered migration flows through Platform WebService administrative APIs that consume the platform-owned migration registry.
|
||||
|
||||
## Target Wave Assignment (Consolidation)
|
||||
|
||||
Reference policy: `docs/db/MIGRATION_CONSOLIDATION_PLAN.md`
|
||||
|
||||
| Wave | Focus | Modules / Mechanisms |
|
||||
| --- | --- | --- |
|
||||
| W1 | Shared runner entrypoint expansion | Authority, Scheduler, Concelier, Policy, Notify, Excititor, Scanner, AirGap, TimelineIndexer, Platform/ReleaseOrchestrator |
|
||||
| W2 | Custom history-table compatibility cutover | EvidenceLocker, ExportCenter, BinaryIndex, Plugin Registry |
|
||||
| W3 | Wire currently unwired embedded SQL folders | Graph, IssuerDirectory, Findings Ledger, Orchestrator, Attestor, Signer, Signals, Unknowns, VexHub, VexLens, Remediation, SbomService Lineage, AdvisoryAI Storage, Timeline Core, ReachGraph Persistence, Artifact Infrastructure, Evidence Persistence, Eventing, Verdict Persistence |
|
||||
| W4 | Migration count baseline/squash strategy | High-count chains (notably Platform release chain and any module above approved threshold) |
|
||||
| W5 | On-prem rehearsal and gate | Full compose/helm deployment upgrade path with deterministic replay evidence |
|
||||
@@ -1,13 +1,50 @@
|
||||
# PostgreSQL Migration Strategy
|
||||
|
||||
**Version:** 1.0
|
||||
**Last Updated:** 2025-12-03
|
||||
**Version:** 1.1
|
||||
**Last Updated:** 2026-02-22
|
||||
**Status:** Active
|
||||
|
||||
## Overview
|
||||
|
||||
This document defines the migration strategy for StellaOps PostgreSQL databases. It covers initial setup, per-release migrations, multi-instance coordination, and air-gapped operation.
|
||||
|
||||
## Consolidation Baseline (2026-02-22)
|
||||
|
||||
This strategy is under active consolidation. The current repository baseline is tracked in:
|
||||
|
||||
- `docs/db/MIGRATION_INVENTORY.md`
|
||||
- `docs/db/MIGRATION_CONSOLIDATION_PLAN.md`
|
||||
|
||||
Current-state realities that must be accounted for in operations:
|
||||
|
||||
- Multiple migration mechanisms are active (shared `MigrationRunner`, `StartupMigrationHost` wrappers, custom runners with custom history tables, compose bootstrap init SQL, and unwired migration folders).
|
||||
- CLI migration coverage is currently limited to the modules registered in `src/Platform/__Libraries/StellaOps.Platform.Database/MigrationModuleRegistry.cs`.
|
||||
- Registry module population is plug-in based (`IMigrationModulePlugin`) with one migration plug-in per web service.
|
||||
- Platform migration admin endpoints (`/api/v1/admin/migrations/*`) use the same platform-owned registry for UI/backend orchestration.
|
||||
- Several services contain migration SQL but have no verified runtime invocation path in non-test code.
|
||||
|
||||
Until consolidation is complete, treat this document as the target policy and `docs/db/MIGRATION_INVENTORY.md` as the source of truth for current implementation state.
|
||||
|
||||
## Canonical Mechanism and Entrypoint (Target)
|
||||
|
||||
For on-prem upgradeable deployments, the target state is:
|
||||
|
||||
- One migration mechanism:
|
||||
- `src/__Libraries/StellaOps.Infrastructure.Postgres/Migrations/MigrationRunner.cs`
|
||||
- `src/__Libraries/StellaOps.Infrastructure.Postgres/Migrations/StartupMigrationHost.cs`
|
||||
- One migration history table format: `<schema>.schema_migrations`
|
||||
- One operational release gate with one shared module catalog:
|
||||
- `stella system migrations-status --module all`
|
||||
- `stella system migrations-verify --module all`
|
||||
- `stella system migrations-run --module all --category release --dry-run`
|
||||
- `stella system migrations-run --module all --category release --force`
|
||||
- `GET /api/v1/admin/migrations/status?module=all`
|
||||
- `GET /api/v1/admin/migrations/verify?module=all`
|
||||
- `POST /api/v1/admin/migrations/run`
|
||||
|
||||
Service startup paths may run only automatic categories (`startup`, `seed`) through `StartupMigrationHost`.
|
||||
Pending `release` or `data` migrations are handled through the canonical CLI/API execution path before service rollout.
|
||||
|
||||
## Principles
|
||||
|
||||
1. **Forward-Only**: No down migrations. Fixes are applied as new forward migrations.
|
||||
@@ -40,9 +77,9 @@ Run automatically when application starts. Must complete within 60 seconds.
|
||||
- Large data migrations (> 10,000 rows affected)
|
||||
- Any operation requiring `ACCESS EXCLUSIVE` lock for extended periods
|
||||
|
||||
### Category B: Release Migrations (Manual/CLI)
|
||||
### Category B: Release Migrations (Manual/CLI/API)
|
||||
|
||||
Require explicit execution via CLI before deployment. Used for breaking changes.
|
||||
Require explicit execution through CLI or Platform migration admin API before deployment. Used for breaking changes.
|
||||
|
||||
**Typical Operations:**
|
||||
- Dropping deprecated columns/tables
|
||||
@@ -128,7 +165,7 @@ src/<Module>/__Libraries/StellaOps.<Module>.Storage.Postgres/
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ 5. Check for pending Category B migrations │
|
||||
│ - If any 100+ migrations are pending: FAIL STARTUP │
|
||||
│ - Log: "Run 'stellaops migrate' before deployment" │
|
||||
│ - Log: "Run 'stella system migrations-run ...' before deployment" │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
│
|
||||
▼
|
||||
@@ -155,17 +192,28 @@ src/<Module>/__Libraries/StellaOps.<Module>.Storage.Postgres/
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
### Release Migration (CLI)
|
||||
### Release Migration (CLI/API)
|
||||
|
||||
```bash
|
||||
# Before deployment - run breaking migrations
|
||||
stellaops system migrations-run --module Authority --category release
|
||||
stella system migrations-run --module Authority --category release --force
|
||||
|
||||
# Verify migration state
|
||||
stellaops system migrations-status --module Authority
|
||||
stella system migrations-status --module Authority
|
||||
|
||||
# Dry run (show what would be executed)
|
||||
stellaops system migrations-run --module Authority --dry-run
|
||||
stella system migrations-run --module Authority --category release --dry-run
|
||||
```
|
||||
|
||||
```bash
|
||||
# API equivalent (used by Platform UI/backend orchestration)
|
||||
curl -sk -H "Authorization: Bearer ${TOKEN}" \
|
||||
"https://platform.stella-ops.local/api/v1/admin/migrations/status?module=Authority"
|
||||
|
||||
curl -sk -H "Authorization: Bearer ${TOKEN}" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"module":"Authority","category":"release","dryRun":true}' \
|
||||
"https://platform.stella-ops.local/api/v1/admin/migrations/run"
|
||||
```
|
||||
|
||||
## Multi-Instance Coordination
|
||||
@@ -285,16 +333,16 @@ At startup, migrations are validated for:
|
||||
|
||||
```bash
|
||||
# 1. Review pending migrations
|
||||
stellaops system migrations-status --module all
|
||||
stella system migrations-status --module all
|
||||
|
||||
# 2. Backup database (if required)
|
||||
pg_dump -Fc stellaops > backup_$(date +%Y%m%d).dump
|
||||
|
||||
# 3. Run release migrations in maintenance window
|
||||
stellaops system migrations-run --category release --module all
|
||||
stella system migrations-run --category release --module all --force
|
||||
|
||||
# 4. Verify schema state
|
||||
stellaops system migrations-verify --module all
|
||||
stella system migrations-verify --module all
|
||||
```
|
||||
|
||||
### Deployment
|
||||
@@ -307,10 +355,10 @@ stellaops system migrations-verify --module all
|
||||
|
||||
```bash
|
||||
# Check migration status
|
||||
stellaops system migrations-status --module all
|
||||
stella system migrations-status --module all
|
||||
|
||||
# Run any data migrations (background)
|
||||
stellaops system migrations-run --category data --module all
|
||||
stella system migrations-run --category data --module all
|
||||
```
|
||||
|
||||
## Rollback Strategy
|
||||
@@ -401,7 +449,7 @@ EOF
|
||||
dotnet run --project src/Authority/StellaOps.Authority.WebService
|
||||
|
||||
# 4. Verify migration applied
|
||||
stellaops system migrations-status --module Authority
|
||||
stella system migrations-status --module Authority
|
||||
```
|
||||
|
||||
### Testing Migrations
|
||||
@@ -411,8 +459,8 @@ stellaops system migrations-status --module Authority
|
||||
dotnet test --filter "Category=Migration"
|
||||
|
||||
# Test idempotency (run twice)
|
||||
stellaops system migrations-run --module Authority
|
||||
stellaops system migrations-run --module Authority # Should be no-op
|
||||
stella system migrations-run --module Authority
|
||||
stella system migrations-run --module Authority # Should be no-op
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
@@ -454,7 +502,7 @@ ERROR: Migration checksum mismatch for '003_add_audit_columns.sql'
|
||||
```
|
||||
ERROR: Cannot start application - pending release migrations require manual execution
|
||||
Pending: 100_drop_legacy_columns.sql
|
||||
Run: stellaops system migrations-run --module Authority --category release
|
||||
Run: stella system migrations-run --module Authority --category release --force
|
||||
```
|
||||
|
||||
**Resolution**: Run CLI migration command before deployment.
|
||||
|
||||
@@ -12,6 +12,10 @@ This directory contains all documentation related to the StellaOps database arch
|
||||
| [RULES.md](./RULES.md) | Database coding rules, patterns, and constraints for all developers |
|
||||
| [CONVERSION_PLAN.md](./CONVERSION_PLAN.md) | Strategic plan for MongoDB to PostgreSQL conversion |
|
||||
| [VERIFICATION.md](./VERIFICATION.md) | Testing and verification requirements for database changes |
|
||||
| [MIGRATION_STRATEGY.md](./MIGRATION_STRATEGY.md) | Canonical migration strategy and category semantics |
|
||||
| [MIGRATION_CONVENTIONS.md](./MIGRATION_CONVENTIONS.md) | Migration naming, numbering, and runner conventions |
|
||||
| [MIGRATION_INVENTORY.md](./MIGRATION_INVENTORY.md) | Current-state service migration matrix (DAL, locations, counts, runner) |
|
||||
| [MIGRATION_CONSOLIDATION_PLAN.md](./MIGRATION_CONSOLIDATION_PLAN.md) | Runner consolidation waves and legacy compatibility mapping for on-prem upgrades |
|
||||
|
||||
## Task Definitions
|
||||
|
||||
|
||||
@@ -7,12 +7,12 @@ Cli
|
||||
VERIFIED
|
||||
|
||||
## Description
|
||||
PostgreSQL database migration management across modules (Authority, Scheduler, Concelier, Policy, Notify, Excititor) with category selection (startup/release/seed/data), dry-run mode, connection string overrides, and timeout configuration.
|
||||
PostgreSQL database migration management across modules (`AirGap`, `Authority`, `Concelier`, `Excititor`, `Notify`, `Platform`, `Policy`, `Scanner`, `Scheduler`, `TimelineIndexer`) with category selection (startup/release/seed/data), dry-run mode, connection string overrides, and timeout configuration.
|
||||
|
||||
## Implementation Details
|
||||
- **Command Group**: `src/Cli/StellaOps.Cli/Commands/SystemCommandBuilder.cs` -- `SystemCommandBuilder` for `stella system` commands
|
||||
- **Migration Service**: `src/Cli/StellaOps.Cli/Services/MigrationCommandService.cs` -- migration execution
|
||||
- **Migration Registry**: `src/Cli/StellaOps.Cli/Services/MigrationModuleRegistry.cs` -- module registry
|
||||
- **Migration Registry**: `src/Platform/__Libraries/StellaOps.Platform.Database/MigrationModuleRegistry.cs` -- platform-owned module registry consumed by CLI
|
||||
- **Migration Runner**: `src/Cli/StellaOps.Cli/Services/MigrationRunnerAdapter.cs` -- runner adapter
|
||||
- **Tests**: `src/Cli/__Tests/StellaOps.Cli.Tests/Commands/SystemCommandBuilderTests.cs`
|
||||
- **Commands**:
|
||||
|
||||
@@ -180,15 +180,17 @@ Completion criteria:
|
||||
| 2026-02-21 | 042-T9 completed for inline runtime datasets: `simulation-history.component.ts`, `conflict-detection.component.ts`, `batch-evaluation.component.ts`, and `graph-explorer.component.ts` now load backend data through `POLICY_SIMULATION_API` and `GRAPH_PLATFORM_API` instead of inline mocks; app config now binds graph runtime base/token providers. | Developer (FE) |
|
||||
| 2026-02-21 | Validation after T9 cutover: `npm run build` (with `NODE_OPTIONS=--max-old-space-size=6144`) passed; targeted tests passed: `npx ng test --watch=false --include=src/tests/policy_studio/policy-simulation.behavior.spec.ts --include=src/tests/signals_runtime_dashboard/signals-runtime-dashboard.service.spec.ts` and `npx ng test --watch=false --include=src/tests/security-risk/security-risk-routes.spec.ts --include=src/tests/security-risk/sbom-graph-page.component.spec.ts`. | Developer (FE) |
|
||||
| 2026-02-21 | T12 documentation sync started: updated `docs/modules/ui/README.md` with runtime endpoint cutover summary and updated `docs/modules/ui/v2-rewire/S00_endpoint_contract_ledger_v2_pack22.md` with Policy Simulation + Graph Explorer endpoint rows reflecting runtime bindings. | Developer / Documentation author |
|
||||
| 2026-02-21 | Closed lineage compare mock gap: `lineage-compare.component.ts` now consumes real `whySafe` payloads from compare responses, `why-safe-panel.component.ts` removed inline mock explanation generation and renders directly from VEX/reachability/attestation compare data, and unused `lineage-why-safe-panel.component.ts` mock component was deleted. | Developer (FE) |
|
||||
| 2026-02-21 | Validation after lineage cutover: `npm run build` passed and targeted lineage verification passed via `npx ng test --watch=false --include=src/tests/lineage/lineage-compare-panel.component.spec.ts` (4/4 tests). | Developer (FE) |
|
||||
## Decisions & Risks
|
||||
- Decision: runtime DI must resolve API tokens to HTTP clients; mock classes are test/dev assets only.
|
||||
- Decision: no new backend contracts are assumed in this sprint; if a required endpoint is missing, task becomes `BLOCKED` with explicit contract gap.
|
||||
- Decision: lineage compare client now normalizes both legacy and compare-service payload shapes (`componentDiff/vexDeltas[]` and `sbomDiff/vexDeltas.changes`) into the UI diff view-model to keep runtime bindings backend-driven.
|
||||
- Risk: payload shape drift between mock data and backend responses may break UI assumptions. Mitigation: add adapter layer + targeted tests before removing fallback.
|
||||
- Risk: component-level `providers` can silently override global DI. Mitigation: inventory + explicit removal task (042-T7) with verification.
|
||||
- Risk: direct `inject(Mock...)` usage bypasses app config contracts. Mitigation: mandatory tokenized refactor task (042-T8).
|
||||
- Cross-module note: docs updates required in `docs/modules/ui/**` and endpoint ledger docs under `docs/modules/ui/v2-rewire/`.
|
||||
- `BLOCKED` endpoint gap: `src/Web/StellaOps.Web/src/app/features/releases/state/release-detail.store.ts` still uses inline mock state and `setTimeout` flows; the store is not yet mapped to a finalized release-detail endpoint contract in the Pack 22 ledger row `S22-T03-REL-02`.
|
||||
- `BLOCKED` endpoint gap: `src/Web/StellaOps.Web/src/app/features/lineage/components/lineage-why-safe-panel/lineage-why-safe-panel.component.ts` still returns inline mock explanations; a tokenized API client/contract for "why-safe" explanation is not yet present in `core/api/**` runtime bindings.
|
||||
|
||||
## Next Checkpoints
|
||||
- 2026-02-22 UTC: Complete T1 inventory and finalize endpoint mapping/risk list.
|
||||
|
||||
@@ -0,0 +1,65 @@
|
||||
# Sprint 20260222.046 - Gateway SPA Fallback and RustFS Healthcheck
|
||||
|
||||
## Topic & Scope
|
||||
- Fix gateway route collisions where browser deep links (for example `/policy`) could be dispatched to microservice endpoints instead of the SPA shell.
|
||||
- Stabilize local compose health reporting for RustFS by using an endpoint that returns success in S3 mode.
|
||||
- Working directory: `src/Router/`.
|
||||
- Cross-module edits explicitly allowed for this sprint: `devops/compose`, `docs/modules/router`, `src/Router/__Tests`.
|
||||
- Expected evidence: unit tests for route-dispatch behavior, compose health output, Playwright route verification.
|
||||
|
||||
## Dependencies & Concurrency
|
||||
- Depends on current Router Gateway route table behavior in compose (`router-gateway-local.json`).
|
||||
- Depends on RustFS S3 runtime semantics (`/` returns 403, `/status` returns 200).
|
||||
- Work can proceed in parallel between gateway middleware changes and compose healthcheck updates.
|
||||
|
||||
## Documentation Prerequisites
|
||||
- `docs/modules/router/architecture.md`
|
||||
- `docs/modules/router/webservice-integration-guide.md`
|
||||
- `docs/modules/platform/architecture-overview.md`
|
||||
|
||||
## Delivery Tracker
|
||||
|
||||
### RGH-01 - Gateway browser deep-link fallback for microservice routes
|
||||
Status: DONE
|
||||
Dependency: none
|
||||
Owners: Developer, Test Automation
|
||||
Task description:
|
||||
- Extend gateway `RouteDispatchMiddleware` SPA fallback behavior so browser document requests matching `Microservice` route prefixes are served by the SPA static fallback route.
|
||||
- Preserve backend dispatch for API paths by excluding `/api` and `/v1` prefixes from SPA fallback.
|
||||
- Add deterministic unit tests covering SPA fallback for `/policy`-style routes and non-fallback behavior for API routes.
|
||||
|
||||
Completion criteria:
|
||||
- [x] Browser deep link to a microservice-prefixed SPA route resolves to SPA `index.html`.
|
||||
- [x] API-prefixed routes still dispatch to microservice pipeline.
|
||||
- [x] Unit tests cover both behaviors.
|
||||
|
||||
### RGH-02 - RustFS compose healthcheck fix
|
||||
Status: DONE
|
||||
Dependency: none
|
||||
Owners: Developer
|
||||
Task description:
|
||||
- Update `docker-compose.stella-ops.yml` RustFS health probe from `/` to `/status` so health reflects actual S3-mode service readiness.
|
||||
- Verify compose service transitions to healthy with the updated probe command.
|
||||
|
||||
Completion criteria:
|
||||
- [x] Compose RustFS healthcheck command uses `/status`.
|
||||
- [x] `docker compose ps` reports `stellaops-rustfs` healthy after update.
|
||||
|
||||
## Execution Log
|
||||
| Date (UTC) | Update | Owner |
|
||||
| --- | --- | --- |
|
||||
| 2026-02-22 | Sprint created for gateway SPA fallback + RustFS healthcheck stabilization. | Project Manager |
|
||||
| 2026-02-22 | Implemented microservice-route SPA fallback in `RouteDispatchMiddleware`; added unit tests for browser deep link vs API dispatch behavior. | Developer |
|
||||
| 2026-02-22 | Updated RustFS healthcheck probe to `/status` and revalidated compose health + Playwright route checks. | Developer |
|
||||
|
||||
## Decisions & Risks
|
||||
- Decision: apply SPA fallback to both `ReverseProxy` and `Microservice` matched routes for browser document requests.
|
||||
- Decision: explicitly exclude `/api` and `/v1` from SPA fallback to prevent API browser requests being masked.
|
||||
- Risk: route-table prefix collisions can regress UI deep links after future route conversions. Mitigation: retain unit coverage in gateway middleware tests.
|
||||
- Risk: RustFS endpoint semantics can change across image versions. Mitigation: keep compose probe aligned with documented runtime status endpoint.
|
||||
- Docs links:
|
||||
- `docs/modules/router/webservice-integration-guide.md`
|
||||
|
||||
## Next Checkpoints
|
||||
- Validate full route regression suite in CI after gateway image rebuild.
|
||||
- Add an integration test asserting deep-link `GET /policy` returns SPA shell in compose profile tests.
|
||||
@@ -0,0 +1,140 @@
|
||||
# Sprint 20260222_051 - AdvisoryAI Knowledge Search
|
||||
|
||||
## Topic & Scope
|
||||
- Deliver AdvisoryAI Knowledge Search (AKS) for deterministic retrieval across docs, OpenAPI, and Doctor checks.
|
||||
- Extend existing global search surfaces (Web + CLI) to resolve operational questions into grounded, actionable references.
|
||||
- Preserve existing Doctor execution behavior while adding searchable Doctor projections and recommendation affordances.
|
||||
- Working directory: `src/AdvisoryAI`.
|
||||
- Expected evidence: schema migration SQL, ingestion/search tests, CLI/Web integration tests, benchmark report, docs.
|
||||
|
||||
## Dependencies & Concurrency
|
||||
- Depends on current AdvisoryAI and Doctor contracts:
|
||||
- `src/AdvisoryAI/StellaOps.AdvisoryAI.WebService/Program.cs`
|
||||
- `src/__Libraries/StellaOps.Doctor/**`
|
||||
- `src/Doctor/StellaOps.Doctor.WebService/Endpoints/DoctorEndpoints.cs`
|
||||
- Cross-module edits are explicitly allowed for this sprint:
|
||||
- `src/Cli/**` for `stella search` and `stella doctor suggest`.
|
||||
- `src/Web/StellaOps.Web/**` for global search integration.
|
||||
- `docs/modules/advisory-ai/**` and `docs/modules/cli/**` for contract/user docs.
|
||||
- `devops/compose/**` for test DB harness.
|
||||
- Safe parallelism: AdvisoryAI ingestion/search internals can proceed in parallel with CLI/Web wiring once API contract is frozen.
|
||||
|
||||
## Documentation Prerequisites
|
||||
- `docs/README.md`
|
||||
- `docs/07_HIGH_LEVEL_ARCHITECTURE.md`
|
||||
- `docs/modules/platform/architecture-overview.md`
|
||||
- `docs/modules/advisory-ai/architecture.md`
|
||||
- `docs/modules/cli/architecture.md`
|
||||
- `src/AdvisoryAI/AGENTS.md`
|
||||
- `src/Cli/AGENTS.md`
|
||||
- `src/Web/StellaOps.Web/AGENTS.md`
|
||||
|
||||
## Delivery Tracker
|
||||
|
||||
### AIAI-KS-001 - AdvisoryAI Knowledge Schema + Ingestion Core
|
||||
Status: DONE
|
||||
Dependency: none
|
||||
Owners: Developer / Implementer
|
||||
Task description:
|
||||
- Add AdvisoryAI knowledge schema and deterministic ingestion pipeline for Markdown docs, OpenAPI specs, and Doctor projections.
|
||||
- Implement deterministic IDs (doc/chunk), anchors, section paths, and span metadata required for exact source navigation.
|
||||
|
||||
Completion criteria:
|
||||
- [x] Schema migration(s) create `kb_doc`, `kb_chunk`, `api_spec`, `api_operation`, and doctor projection tables with required indexes.
|
||||
- [x] Deterministic index rebuild command/service ingests docs + specs + doctor metadata with reproducible IDs.
|
||||
- [x] Ingestion supports product/version metadata and can run without external network dependencies.
|
||||
|
||||
### AIAI-KS-002 - Hybrid Search API + Ranking
|
||||
Status: DONE
|
||||
Dependency: AIAI-KS-001
|
||||
Owners: Developer / Implementer
|
||||
Task description:
|
||||
- Implement `POST /v1/advisory-ai/search` with deterministic retrieval outputs for docs/api/doctor.
|
||||
- Provide FTS retrieval baseline plus optional vector retrieval and deterministic rank fusion, including stable fallback when embeddings are missing.
|
||||
|
||||
Completion criteria:
|
||||
- [x] Search request/response contracts include typed open-actions and snippet/source evidence.
|
||||
- [x] Ranking includes deterministic tie-breaking and filter support (`type`, `product`, `version`, `service`, `tags`).
|
||||
- [x] API tests validate exact-string and paraphrase-like query behavior over seeded data.
|
||||
|
||||
### AIAI-KS-003 - CLI Search + Doctor Suggest Integration
|
||||
Status: DONE
|
||||
Dependency: AIAI-KS-002
|
||||
Owners: Developer / Implementer
|
||||
Task description:
|
||||
- Add CLI command(s) to consume AKS and return human + JSON output with stable shape.
|
||||
- Add Doctor suggestion flow backed by the same retrieval stack to recommend checks/docs from symptom strings.
|
||||
|
||||
Completion criteria:
|
||||
- [x] `stella search "<query>"` supports type/filter flags and deterministic JSON output.
|
||||
- [x] `stella doctor suggest "<symptom/error>"` returns recommended checks/docs/endpoints from AKS.
|
||||
- [x] CLI tests cover output shape and deterministic ordering.
|
||||
|
||||
### AIAI-KS-004 - Web Global Search Integration
|
||||
Status: DONE
|
||||
Dependency: AIAI-KS-002
|
||||
Owners: Developer / Implementer
|
||||
Task description:
|
||||
- Rewire Web global search to AKS mixed results and render grouped docs/api/doctor hits with action affordances.
|
||||
- Keep keyboard and accessibility behaviors intact while adding type filters and action metadata rendering.
|
||||
|
||||
Completion criteria:
|
||||
- [x] Global search client calls AdvisoryAI search API and maps result groups deterministically.
|
||||
- [x] UI exposes grouped hits with actionable links (open docs anchor, endpoint details/copy, run check).
|
||||
- [x] Targeted frontend tests cover rendering + result interaction for mixed types.
|
||||
|
||||
### AIAI-KS-005 - Test DB + Dataset Generator + Quality Gates
|
||||
Status: DONE
|
||||
Dependency: AIAI-KS-001
|
||||
Owners: Developer / Implementer, Test Automation
|
||||
Task description:
|
||||
- Create dedicated Postgres+pgvector test harness and deterministic query dataset generator with ground truth mappings.
|
||||
- Add benchmark runner/reporting for recall@k and latency sanity checks.
|
||||
|
||||
Completion criteria:
|
||||
- [x] Compose/scripts provision dedicated AKS test DB and load deterministic seed snapshot.
|
||||
- [x] Dataset generator emits >= 1,000 grounded queries across docs/api/doctor with expected targets.
|
||||
- [x] Automated test/benchmark path reports recall@5 and latency metrics in reproducible format.
|
||||
|
||||
### AIAI-KS-006 - Documentation + Operational Runbook Sync
|
||||
Status: DONE
|
||||
Dependency: AIAI-KS-001
|
||||
Owners: Documentation author / Developer
|
||||
Task description:
|
||||
- Publish AKS design and operating guide: problem framing, schema, ingestion, ranking, API, CLI/UI usage, testing approach.
|
||||
- Update sprint decisions with doc links and operational caveats.
|
||||
|
||||
Completion criteria:
|
||||
- [x] AdvisoryAI documentation includes AKS architecture and usage.
|
||||
- [x] CLI/Web docs reflect new search command + global search behavior.
|
||||
- [x] Sprint Decisions & Risks section links updated docs and known limitations.
|
||||
|
||||
## Execution Log
|
||||
| Date (UTC) | Update | Owner |
|
||||
| --- | --- | --- |
|
||||
| 2026-02-22 | Sprint created, scope authorized, and AIAI-KS-001 moved to DOING. | Developer |
|
||||
| 2026-02-22 | Implemented AKS schema and deterministic ingestion/rebuild pipeline for markdown + OpenAPI + doctor projections in AdvisoryAI. | Developer |
|
||||
| 2026-02-22 | Implemented AKS search API (`/api/v1/advisory-ai/search`) with typed open-actions, deterministic ranking, and fallback behavior. | Developer |
|
||||
| 2026-02-22 | Wired CLI (`search`, `doctor suggest`, `advisoryai index rebuild`) and added behavioral CLI tests for output contracts. | Developer |
|
||||
| 2026-02-22 | Rewired Web global search and command palette to AKS mixed docs/api/doctor results with actions and filter chips. | Developer |
|
||||
| 2026-02-22 | Added AKS benchmark dataset generator + benchmark runner tests and dedicated compose pgvector test harness. | Developer, Test Automation |
|
||||
| 2026-02-22 | Validation complete: AdvisoryAI tests `584/584`, CLI tests `1187/1187`, Web global-search spec `4/4`, Web build succeeded. | Developer |
|
||||
|
||||
## Decisions & Risks
|
||||
- Decision: AKS ownership remains in `src/AdvisoryAI`; CLI/Web consume AKS via API contracts to avoid cross-module logic sprawl.
|
||||
- Decision: Doctor execution semantics remain in Doctor module; AKS only ingests projections/metadata and emits recommendation actions.
|
||||
- Risk: Existing workspace is heavily dirty (unrelated pre-existing edits). Mitigation: keep changes tightly scoped to listed sprint directories and avoid destructive cleanup.
|
||||
- Risk: OpenAPI sources are mixed (`openapi.json` and yaml). Mitigation: MVP prioritizes deterministic JSON ingestion; document yaml handling strategy.
|
||||
- Risk: Vector extension may be absent in some environments. Mitigation: FTS-only fallback path remains fully functional and deterministic.
|
||||
- Docs updated:
|
||||
- `docs/modules/advisory-ai/knowledge-search.md`
|
||||
- `docs/modules/advisory-ai/guides/api.md`
|
||||
- `docs/modules/advisory-ai/README.md`
|
||||
- `docs/modules/cli/guides/cli-reference.md`
|
||||
- `docs/API_CLI_REFERENCE.md`
|
||||
- Risk: Microsoft.Testing.Platform ignores VSTest `--filter` in this repository. Mitigation: run target `.csproj` directly and record full-suite counts in validation logs.
|
||||
|
||||
## Next Checkpoints
|
||||
- 2026-02-22: Schema + FTS endpoint + index rebuild MVP complete. (DONE)
|
||||
- 2026-02-22: CLI/Web wiring and targeted integration tests complete. (DONE)
|
||||
- 2026-02-22: Dataset generator, benchmark path, and docs finalization complete. (DONE)
|
||||
@@ -0,0 +1,235 @@
|
||||
# Sprint 20260222.051 - Migration Types, Counts, and Runner Entrypoint Consolidation
|
||||
|
||||
## Topic & Scope
|
||||
- Consolidate all database migration mechanisms to a single canonical model suitable for on-prem and updateable deployments.
|
||||
- Produce an authoritative migration inventory table per service with DAL type, migration location, migration count, and runner mechanism.
|
||||
- Standardize migration numbering/category rules and remove ambiguous or duplicated runner paths.
|
||||
- Enforce a phase gate: EF Core v10 models and Dapper to EF migration begin only after migration consolidation is implemented and validated.
|
||||
- Update operator-facing procedures in documentation, setup flow, CLI guidance, and Docker Compose guidance as part of the same consolidation stream.
|
||||
- Working directory: `docs/implplan/`.
|
||||
- Cross-module edits explicitly allowed for this sprint: `src/__Libraries/StellaOps.Infrastructure.Postgres`, `src/Platform/__Libraries/StellaOps.Platform.Database`, `src/Cli/StellaOps.Cli`, `src/**/Migrations`, `src/**/Database/Migrations`, `src/**/Storage`, `docs/db`, `docs/operations`, `docs/INSTALL_GUIDE.md`, `docs/API_CLI_REFERENCE.md`, `devops/compose/README.md`, `docs/modules/**`.
|
||||
- Expected evidence: migration inventory matrix, runner entrypoint map, cutover checklist, deterministic upgrade/replay logs, updated setup/CLI/compose procedure documents, phase-gate approval record.
|
||||
|
||||
## Dependencies & Concurrency
|
||||
- Depends on:
|
||||
- `docs/README.md`
|
||||
- `docs/ARCHITECTURE_OVERVIEW.md`
|
||||
- `docs/07_HIGH_LEVEL_ARCHITECTURE.md`
|
||||
- `docs/modules/platform/architecture-overview.md`
|
||||
- Safe concurrency:
|
||||
- Inventory extraction and migration counting can run in parallel per module.
|
||||
- Runner contract definition can proceed in parallel with numbering-rule proposal.
|
||||
- Runner cutovers must be sequential per module wave after contract approval.
|
||||
- EF Core v10 model work is blocked until migration consolidation is complete.
|
||||
|
||||
## Documentation Prerequisites
|
||||
- `docs/README.md`
|
||||
- `docs/ARCHITECTURE_OVERVIEW.md`
|
||||
- `docs/07_HIGH_LEVEL_ARCHITECTURE.md`
|
||||
- `docs/modules/platform/architecture-overview.md`
|
||||
- `docs/implplan/AGENTS.md`
|
||||
- `src/__Libraries/StellaOps.Infrastructure.Postgres/Migrations/MigrationRunner.cs`
|
||||
- `src/__Libraries/StellaOps.Infrastructure.Postgres/Migrations/MigrationCategory.cs`
|
||||
- `src/Platform/__Libraries/StellaOps.Platform.Database/MigrationModuleRegistry.cs`
|
||||
|
||||
## Delivery Tracker
|
||||
|
||||
### MGC-01 - Migration inventory and mechanism taxonomy baseline
|
||||
Status: DONE
|
||||
Dependency: none
|
||||
Owners: Project Manager, Developer
|
||||
Task description:
|
||||
- Build the authoritative service migration matrix for the current repository state.
|
||||
- For each service/module, capture DAL type, migration file locations, migration count, and current runner entrypoint/mechanism.
|
||||
- Mark modules where migration files exist but runtime invocation is missing.
|
||||
|
||||
Completion criteria:
|
||||
- [x] Matrix includes every service with database migration artifacts.
|
||||
- [x] DAL type and runner mechanism are identified for every matrix row.
|
||||
- [x] Missing invocation paths are explicitly listed as blockers or remediation tasks.
|
||||
|
||||
### MGC-02 - Canonical migration mechanism and numbering policy
|
||||
Status: DONE
|
||||
Dependency: MGC-01
|
||||
Owners: Developer, Documentation Author
|
||||
Task description:
|
||||
- Define the single supported migration execution mechanism for production and upgrade flows.
|
||||
- Normalize migration numbering and category rules across modules.
|
||||
- Define compatibility handling for legacy schema version tables and custom history formats.
|
||||
|
||||
Completion criteria:
|
||||
- [x] Policy defines one canonical runner mechanism and approved exceptions.
|
||||
- [x] Numbering/category rules are deterministic and documented.
|
||||
- [x] Legacy compatibility mapping exists for all non-canonical history tables.
|
||||
|
||||
### MGC-03 - Runner entrypoint consolidation plan and implementation wave map
|
||||
Status: DONE
|
||||
Dependency: MGC-02
|
||||
Owners: Developer, Project Manager
|
||||
Task description:
|
||||
- Define the single entrypoint strategy for migration execution across startup, CLI, and deployment automation.
|
||||
- Map all module cutovers from custom or fragmented runners to the canonical entrypoint.
|
||||
- Produce implementation waves and dependency order with rollback markers.
|
||||
|
||||
Completion criteria:
|
||||
- [x] One canonical runner entrypoint path is selected and documented.
|
||||
- [x] Every module is assigned to a cutover wave with explicit owner.
|
||||
- [x] Rollback and safety controls are defined per wave.
|
||||
|
||||
### MGC-04 - Module cutover to canonical runner
|
||||
Status: DOING
|
||||
Dependency: MGC-03
|
||||
Owners: Developer
|
||||
Task description:
|
||||
- Migrate modules with custom migration executors to the shared runner contract.
|
||||
- Wire startup and CLI flows so the same canonical path executes migrations.
|
||||
- Remove dead runner paths after validation.
|
||||
|
||||
Completion criteria:
|
||||
- [ ] Custom runner implementations are replaced or adapter-wrapped to canonical contract.
|
||||
- [ ] Startup and CLI both route through the same migration mechanism.
|
||||
- [ ] Removed runner paths are verified as no longer referenced.
|
||||
|
||||
### MGC-05 - Migration count consolidation and baseline strategy
|
||||
Status: TODO
|
||||
Dependency: MGC-04
|
||||
Owners: Developer, Project Manager
|
||||
Task description:
|
||||
- Consolidate migration counts per module using approved baseline/squash strategy for long-lived on-prem upgrades.
|
||||
- Preserve deterministic replay and checksum integrity during consolidation.
|
||||
- Document versioning guarantees for existing installed customer environments.
|
||||
|
||||
Completion criteria:
|
||||
- [ ] Target migration count and baseline strategy are published per module.
|
||||
- [ ] Replay/checksum behavior remains deterministic across upgraded environments.
|
||||
- [ ] Backward-compatibility rules are documented for in-field upgrades.
|
||||
|
||||
### MGC-06 - On-prem upgrade rehearsal and verification
|
||||
Status: TODO
|
||||
Dependency: MGC-05
|
||||
Owners: Test Automation, Developer
|
||||
Task description:
|
||||
- Execute clean install and upgrade-path rehearsals using canonical migration entrypoint.
|
||||
- Validate deterministic outcomes across repeated runs and rollback/retry scenarios.
|
||||
- Capture evidence for release gating.
|
||||
|
||||
Completion criteria:
|
||||
- [ ] Clean install and upgrade rehearsals pass with canonical runner.
|
||||
- [ ] Repeat runs are deterministic with no schema drift.
|
||||
- [ ] Rollback/retry paths are validated and documented.
|
||||
|
||||
### MGC-07 - Phase gate for EF Core v10 and Dapper migration
|
||||
Status: TODO
|
||||
Dependency: MGC-06
|
||||
Owners: Project Manager, Developer, Documentation Author
|
||||
Task description:
|
||||
- Open the next-phase implementation stream for EF Core v10 model generation and Dapper-to-EF migration after consolidation verification passes.
|
||||
- Define scope boundaries so EF model changes do not alter migration governance decisions made in this sprint.
|
||||
- Produce handoff checklist and dependency references for the EF migration sprint.
|
||||
|
||||
Completion criteria:
|
||||
- [ ] Explicit go/no-go decision is recorded for EF Core v10 phase start.
|
||||
- [ ] EF phase backlog is created with dependencies and module order.
|
||||
- [ ] Governance boundary between migration consolidation and ORM transition is documented.
|
||||
|
||||
### MGC-08 - Documentation consolidation for migration operations
|
||||
Status: DONE
|
||||
Dependency: MGC-01
|
||||
Owners: Documentation Author, Project Manager
|
||||
Task description:
|
||||
- Update canonical migration docs to reflect the observed current state and the target consolidated mechanism.
|
||||
- Publish and maintain the authoritative migration inventory table (service, DAL type, migration locations, migration counts, runner mechanism, runner entrypoint).
|
||||
- Ensure migration governance docs link to concrete implementation files and runbooks.
|
||||
|
||||
Completion criteria:
|
||||
- [x] `docs/db/MIGRATION_STRATEGY.md` and `docs/db/MIGRATION_CONVENTIONS.md` reflect consolidation policy and current-state caveats.
|
||||
- [x] Migration inventory artifact is published and referenced by the sprint.
|
||||
- [x] Documentation clearly distinguishes current implementation from target consolidated architecture.
|
||||
|
||||
### MGC-09 - Setup procedure updates
|
||||
Status: DONE
|
||||
Dependency: MGC-08
|
||||
Owners: Documentation Author, Developer
|
||||
Task description:
|
||||
- Update installation and setup procedures to include migration preflight checks, migration execution path, and post-migration verification.
|
||||
- Align setup guidance with canonical runner entrypoint decisions and safety gates.
|
||||
|
||||
Completion criteria:
|
||||
- [x] `docs/INSTALL_GUIDE.md` includes migration preflight and verification steps.
|
||||
- [x] Setup guidance references the canonical migration commands and expected outcomes.
|
||||
- [x] Procedure changes are validated for local, on-prem, and upgrade contexts.
|
||||
|
||||
### MGC-10 - CLI procedure updates for migration operations
|
||||
Status: DONE
|
||||
Dependency: MGC-08
|
||||
Owners: Documentation Author, Developer
|
||||
Task description:
|
||||
- Update CLI reference documentation for migration run/status/verify flows and module coverage.
|
||||
- Document current limitations and target module expansion under consolidation waves.
|
||||
|
||||
Completion criteria:
|
||||
- [x] `docs/API_CLI_REFERENCE.md` contains an explicit migration command section.
|
||||
- [x] Current module coverage and expected post-consolidation coverage are documented.
|
||||
- [x] CLI workflow examples are deterministic and automation-friendly.
|
||||
|
||||
### MGC-11 - Docker Compose migration procedure updates
|
||||
Status: DONE
|
||||
Dependency: MGC-08
|
||||
Owners: DevOps, Documentation Author
|
||||
Task description:
|
||||
- Update Compose runbooks to include database migration preflight, startup migration behavior, and upgrade-time migration sequencing.
|
||||
- Align compose procedures with canonical migration runner entrypoint and rollback expectations.
|
||||
|
||||
Completion criteria:
|
||||
- [x] `devops/compose/README.md` documents migration execution and validation in compose workflows.
|
||||
- [x] Compose upgrade path includes migration status checks before and after deployment.
|
||||
- [x] Rollback notes include migration-state considerations.
|
||||
|
||||
### MGC-12 - Platform UI/API migration execution path (follow-on)
|
||||
Status: DONE
|
||||
Dependency: MGC-04
|
||||
Owners: Developer, Platform Service Owner
|
||||
Task description:
|
||||
- Define and implement Platform WebService administrative endpoints for migration status/verify/run operations.
|
||||
- Ensure endpoints use the same platform-owned registry and canonical runner path used by CLI.
|
||||
- Keep database execution server-side only; UI must invoke the Platform API and never connect directly to PostgreSQL.
|
||||
|
||||
Completion criteria:
|
||||
- [x] Platform migration admin API contract is implemented with authorization and deterministic responses.
|
||||
- [x] API execution path consumes `src/Platform/__Libraries/StellaOps.Platform.Database/MigrationModuleRegistry.cs`.
|
||||
- [x] UI orchestration path is documented and linked from migration runbooks.
|
||||
|
||||
## Execution Log
|
||||
| Date (UTC) | Update | Owner |
|
||||
| --- | --- | --- |
|
||||
| 2026-02-22 | Sprint created to prioritize migration mechanism/type/count consolidation and single runner entrypoint unification before ORM transition. | Project Manager |
|
||||
| 2026-02-22 | Initial repository audit completed; baseline migration matrix dimensions identified and MGC-01 started. | Project Manager |
|
||||
| 2026-02-22 | Sprint scope expanded to require procedure updates across documentation, setup, CLI, and Docker Compose workflows; MGC-08 started. | Project Manager |
|
||||
| 2026-02-22 | Published baseline inventory (`docs/db/MIGRATION_INVENTORY.md`) and updated migration/setup/CLI/compose procedure docs for consolidation alignment. | Documentation Author |
|
||||
| 2026-02-22 | MGC-08 completed; MGC-10 and MGC-11 completed with updates in `docs/API_CLI_REFERENCE.md`, `devops/compose/README.md`, and deployment upgrade runbooks. MGC-09 remains DOING pending broader procedural validation. | Project Manager |
|
||||
| 2026-02-22 | MGC-01 completed with full migration inventory table including DAL type, migration locations/counts, mechanism type, and runner entrypoint status. | Project Manager |
|
||||
| 2026-02-22 | MGC-02 completed: canonical migration mechanism, deterministic numbering/category policy, and legacy history-table compatibility mapping were documented in `docs/db/MIGRATION_CONSOLIDATION_PLAN.md`, with strategy/conventions sync in `docs/db/MIGRATION_STRATEGY.md` and `docs/db/MIGRATION_CONVENTIONS.md`. | Documentation Author |
|
||||
| 2026-02-22 | MGC-03 completed: canonical runner entrypoint and wave-based module cutover map published in `docs/db/MIGRATION_CONSOLIDATION_PLAN.md` and linked from `docs/db/MIGRATION_INVENTORY.md`. | Project Manager |
|
||||
| 2026-02-22 | MGC-09 completed: setup/install procedure updated for canonical migration preflight/execution/verification flow in `docs/INSTALL_GUIDE.md`; supporting references synchronized in compose/CLI/upgrade runbooks. | Documentation Author |
|
||||
| 2026-02-22 | MGC-04 started (Wave W1): expanded migration registry coverage from 6 to 10 modules (`AirGap`, `Scanner`, `TimelineIndexer`, `Platform` added); updated CLI tests and inventory/CLI docs to reflect new coverage. | Developer |
|
||||
| 2026-02-22 | MGC-04 Wave W1 refinement: moved migration registry ownership from CLI into `src/Platform/__Libraries/StellaOps.Platform.Database/MigrationModuleRegistry.cs` and rewired CLI to consume the platform-owned registry as the canonical module catalog. | Developer |
|
||||
| 2026-02-22 | Added MGC-12 follow-on tracker so UI-driven migration execution is implemented via Platform WebService admin APIs using the same platform-owned registry and canonical runner path. | Project Manager |
|
||||
| 2026-02-22 | MGC-12 completed: implemented `/api/v1/admin/migrations/{modules,status,verify,run}` with `platform.setup.admin`, wired server-side execution through `PlatformMigrationAdminService` + platform-owned registry, updated setup/CLI/compose/upgrade docs for UI/API orchestration, and validated Platform WebService tests (`177/177` pass). | Developer |
|
||||
| 2026-02-22 | MGC-04 Wave W1 update: replaced hardcoded module list with plugin auto-discovery (`IMigrationModulePlugin`) so one migration plugin descriptor per web service is discovered by the consolidated runner path and consumed by both CLI and Platform API. | Developer |
|
||||
|
||||
## Decisions & Risks
|
||||
- Decision: phase order is fixed. Migration mechanism/count/runner consolidation completes first, EF Core v10 migration starts only after MGC-06 and MGC-07 gate approval.
|
||||
- Risk: modules with custom history tables may break replay when moved to canonical runner. Mitigation: compatibility adapters and checksum-preserving migration history mapping.
|
||||
- Risk: migration files with no runtime invocation can create false confidence in upgrade readiness. Mitigation: mandatory invocation audit in MGC-01 and enforced cutover in MGC-04.
|
||||
- Risk: aggressive migration squashing may invalidate existing customer upgrade paths. Mitigation: per-module baseline strategy with explicit installed-version compatibility windows.
|
||||
- Risk: fragmented startup vs CLI execution can reintroduce drift. Mitigation: single entrypoint enforcement and wave-level regression checks.
|
||||
- Decision: migration module registry ownership is platform-level (`StellaOps.Platform.Database`) so CLI and future UI/API execution paths consume the same module catalog.
|
||||
- Decision: module catalog population is plugin-driven (`IMigrationModulePlugin`) with one migration plugin descriptor per web service, auto-discovered by `MigrationModulePluginDiscovery`.
|
||||
- Documentation synchronization for this sprint (contracts/procedures): `docs/db/MIGRATION_CONSOLIDATION_PLAN.md`, `docs/db/MIGRATION_STRATEGY.md`, `docs/db/MIGRATION_CONVENTIONS.md`, `docs/db/MIGRATION_INVENTORY.md`, `docs/INSTALL_GUIDE.md`, `docs/API_CLI_REFERENCE.md`, `devops/compose/README.md`, `docs/operations/upgrade-runbook.md`, `docs/operations/devops/runbooks/deployment-upgrade.md`, `docs/db/README.md`.
|
||||
|
||||
## Next Checkpoints
|
||||
- 2026-02-23: MGC-01 baseline matrix complete and reviewed.
|
||||
- 2026-02-24: MGC-02 policy and MGC-03 wave map approved. (Completed 2026-02-22)
|
||||
- 2026-02-26: MGC-04 runner cutover implementation complete.
|
||||
- 2026-02-27: MGC-05 and MGC-06 consolidation and rehearsal evidence complete.
|
||||
- 2026-02-28: MGC-07 phase-gate decision and EF Core v10 handoff package complete.
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,154 @@
|
||||
# Sprint 20260222.052 - Router Endpoint Auth Scope and Description Backfill
|
||||
|
||||
## Topic & Scope
|
||||
- Establish a complete endpoint-level inventory for Router OpenAPI with explicit authorization and description coverage status.
|
||||
- For every endpoint, track whether it is anonymous or authenticated, what scopes/roles/policies are declared, and what description improvements are required.
|
||||
- Convert the inventory into endpoint-level implementation actions (`authAction`, `descriptionAction`) so execution can proceed deterministically service-by-service.
|
||||
- Working directory: `docs/implplan`.
|
||||
- Expected evidence: full endpoint inventory CSV, per-service summary CSV, global summary JSON, execution waves for implementation.
|
||||
|
||||
## Dependencies & Concurrency
|
||||
- Depends on current Router aggregate OpenAPI served at `https://stella-ops.local/openapi.json`.
|
||||
- Depends on current compose contract for auth metadata extension (`x-stellaops-gateway-auth`) and endpoint descriptions.
|
||||
- Safe parallelism:
|
||||
- Auth metadata backfill can run in parallel by service wave.
|
||||
- Description enrichment can run in parallel with auth metadata work once per-service owner is assigned.
|
||||
|
||||
## Documentation Prerequisites
|
||||
- `docs/modules/router/architecture.md`
|
||||
- `docs/modules/router/aspnet-endpoint-bridge.md`
|
||||
- `docs/modules/router/webservice-integration-guide.md`
|
||||
- `docs/modules/platform/architecture-overview.md`
|
||||
|
||||
## Endpoint Inventory Artifacts
|
||||
- Full endpoint listing with endpoint-level plan:
|
||||
- `docs/implplan/SPRINT_20260222_052_DOCS_router_endpoint_auth_scope_description_backfill.endpoints.csv`
|
||||
- Per-service rollup:
|
||||
- `docs/implplan/SPRINT_20260222_052_DOCS_router_endpoint_auth_scope_description_backfill.services.csv`
|
||||
- Global totals snapshot:
|
||||
- `docs/implplan/SPRINT_20260222_052_DOCS_router_endpoint_auth_scope_description_backfill.summary.json`
|
||||
- OpenAPI capture used for this sprint:
|
||||
- `docs/implplan/SPRINT_20260222_052_DOCS_router_endpoint_auth_scope_description_backfill.openapi_live.json`
|
||||
|
||||
## Baseline Snapshot (Generated 2026-02-22)
|
||||
- Total operations: `2190`
|
||||
- Anonymous operations: `6`
|
||||
- Authenticated operations: `2184`
|
||||
- Operations with explicit scopes: `28`
|
||||
- Operations with explicit roles: `0`
|
||||
- Operations with policies: `156`
|
||||
- Operations with auth source `None`: `1991`
|
||||
- Descriptions requiring expansion (`same_as_summary` or `too_short`): `1507`
|
||||
|
||||
## Delivery Tracker
|
||||
|
||||
### RASD-01 - Produce full endpoint-level auth and description inventory
|
||||
Status: DONE
|
||||
Dependency: none
|
||||
Owners: Project Manager
|
||||
Task description:
|
||||
- Pull the live Router OpenAPI document and enumerate every HTTP operation.
|
||||
- For each operation, extract:
|
||||
- Service, method, path, operationId.
|
||||
- `allowAnonymous`, `requiresAuthentication`, `authSource`, `effectiveClaimSource`.
|
||||
- Scope, role, policy, and claim requirement details.
|
||||
- Description quality status.
|
||||
- Persist the result as a deterministic CSV under this sprint.
|
||||
|
||||
Completion criteria:
|
||||
- [x] All operations in the current OpenAPI are represented in one inventory file.
|
||||
- [x] Every inventory row includes auth and description status columns.
|
||||
- [x] Inventory artifact is linked in this sprint.
|
||||
|
||||
### RASD-02 - Attach endpoint-level planned actions for auth and descriptions
|
||||
Status: DONE
|
||||
Dependency: RASD-01
|
||||
Owners: Project Manager
|
||||
Task description:
|
||||
- Add per-endpoint plan columns to the inventory:
|
||||
- `authAction` values: `add_endpoint_auth_metadata`, `keep_or_refine_scope`, `policy_defined_scope_not_exported`, `verify_anonymous_intent`, `needs_auth_review`.
|
||||
- `descriptionAction` values: `expand_description`, `add_description`, `replace_http_stub_with_domain_semantics`, `keep_description`.
|
||||
- Ensure each endpoint row has a deterministic next action without requiring manual interpretation.
|
||||
|
||||
Completion criteria:
|
||||
- [x] Every endpoint row has `authAction`.
|
||||
- [x] Every endpoint row has `descriptionAction`.
|
||||
- [x] Action taxonomy is documented in this sprint.
|
||||
|
||||
### RASD-03 - Execute Wave A (missing endpoint auth metadata)
|
||||
Status: TODO
|
||||
Dependency: RASD-02
|
||||
Owners: Developer, Test Automation
|
||||
Task description:
|
||||
- Implement endpoint auth metadata for all operations marked `authAction=add_endpoint_auth_metadata` (`1991` endpoints).
|
||||
- Primary migration target is conversion of in-handler/manual checks to endpoint metadata where applicable (`[Authorize]`/`.RequireAuthorization(...)` and mapped policies/scopes).
|
||||
- Prioritized service order by count:
|
||||
- `orchestrator (313)`, `policy-engine (202)`, `notifier (197)`, `platform (165)`, `concelier (144)`, `policy-gateway (121)`, `findings-ledger (83)`, `advisoryai (81)`, `exportcenter (64)`, `excititor (55)`, then remaining services.
|
||||
|
||||
Completion criteria:
|
||||
- [ ] Every endpoint currently marked `add_endpoint_auth_metadata` is migrated or explicitly justified.
|
||||
- [ ] OpenAPI no longer reports `source: "None"` for migrated endpoints.
|
||||
- [ ] Regression tests validate expected `401/403` behavior.
|
||||
|
||||
### RASD-04 - Execute Wave B (scope/policy normalization and export fidelity)
|
||||
Status: TODO
|
||||
Dependency: RASD-03
|
||||
Owners: Developer, Test Automation
|
||||
Task description:
|
||||
- Resolve endpoints marked `policy_defined_scope_not_exported` (`128` endpoints, currently concentrated in `scanner`) so explicit scope semantics are exported consistently.
|
||||
- Review endpoints marked `needs_auth_review` (`37` endpoints, currently in `authority`) and decide whether they remain policy-only auth or receive explicit scope/role declarations.
|
||||
- Ensure resulting OpenAPI expresses effective scope/claim requirements where expected.
|
||||
|
||||
Completion criteria:
|
||||
- [ ] `policy_defined_scope_not_exported` endpoints are eliminated or explicitly documented as policy-only.
|
||||
- [ ] `needs_auth_review` endpoints are classified and updated.
|
||||
- [ ] Endpoint security metadata is consistent with runtime authorization behavior.
|
||||
|
||||
### RASD-05 - Execute Wave C (description enrichment)
|
||||
Status: TODO
|
||||
Dependency: RASD-02
|
||||
Owners: Documentation author, Developer
|
||||
Task description:
|
||||
- Enrich descriptions for all endpoints marked `descriptionAction=expand_description` (`1507` endpoints).
|
||||
- Replace terse or repeated summary text with domain semantics: purpose, side effects, key constraints, and error behavior.
|
||||
- Keep concise descriptions already marked `keep_description` unchanged unless auth behavior changes require updates.
|
||||
|
||||
Completion criteria:
|
||||
- [ ] All endpoints flagged for description expansion have non-trivial descriptions.
|
||||
- [ ] Descriptions align with actual handler behavior and response contracts.
|
||||
- [ ] OpenAPI diff shows description improvements without schema regressions.
|
||||
|
||||
### RASD-06 - Validate end-to-end and lock quality gates
|
||||
Status: TODO
|
||||
Dependency: RASD-03
|
||||
Owners: Test Automation, QA
|
||||
Task description:
|
||||
- Add deterministic quality checks that fail CI when:
|
||||
- Endpoint auth metadata is missing for non-anonymous endpoints.
|
||||
- Scope/role/policy metadata diverges from declared service authorization rules.
|
||||
- Endpoint descriptions regress to low-information forms.
|
||||
- Rebuild/redeploy and verify `https://stella-ops.local/openapi.json` reflects the updated metadata.
|
||||
|
||||
Completion criteria:
|
||||
- [ ] Automated checks guard auth and description regressions.
|
||||
- [ ] Fresh compose deployment validates updated OpenAPI.
|
||||
- [ ] Sprint artifacts updated with final counts and diffs.
|
||||
|
||||
## Execution Log
|
||||
| Date (UTC) | Update | Owner |
|
||||
| --- | --- | --- |
|
||||
| 2026-02-22 | Sprint created for full endpoint auth/scope/description inventory and migration planning. | Project Manager |
|
||||
| 2026-02-22 | Generated endpoint inventory (`2190` operations) and per-endpoint planned actions in CSV artifacts. | Project Manager |
|
||||
| 2026-02-22 | Computed service-level backlog and execution waves for metadata + description remediation. | Project Manager |
|
||||
|
||||
## Decisions & Risks
|
||||
- Decision: endpoint-level plan is encoded directly in the inventory file via `authAction` and `descriptionAction` so execution is deterministic per endpoint.
|
||||
- Decision: prioritize Wave A by highest-volume services to reduce `source=None` exposure first.
|
||||
- Risk: services using manual in-handler authorization checks may appear authenticated without exported scopes/roles in OpenAPI. Mitigation: convert to endpoint metadata and policy-mapped claims in Wave A/B.
|
||||
- Risk: large-scale description edits can drift from implementation. Mitigation: pair documentation updates with endpoint tests and OpenAPI diff checks.
|
||||
- Risk: runtime and OpenAPI drift if containers are restarted without rebuilt images. Mitigation: include rebuild + redeploy verification in RASD-06.
|
||||
|
||||
## Next Checkpoints
|
||||
- Wave A kickoff: assign owners per service group and start with `orchestrator`, `policy-engine`, `notifier`, `platform`, `concelier`.
|
||||
- Wave B kickoff: `scanner` and `authority` normalization review.
|
||||
- Quality gate activation after first two services complete.
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,39 @@
|
||||
"service","totalEndpoints","anonymousEndpoints","requiresAuthEndpoints","endpointsWithScopes","endpointsWithRoles","endpointsWithPolicies","authSourceNone","missingDescriptions","genericHttpDescriptions","sameAsSummaryDescriptions"
|
||||
"advisoryai","81","0","81","0","0","0","81","0","0","79"
|
||||
"airgap-controller","4","0","4","0","0","0","4","0","0","4"
|
||||
"attestor","45","0","45","0","0","0","45","0","0","38"
|
||||
"authority","45","6","39","0","0","0","2","0","0","39"
|
||||
"binaryindex","21","0","21","0","0","0","21","0","0","21"
|
||||
"concelier","144","0","144","0","0","0","144","0","0","136"
|
||||
"doctor","16","0","16","16","0","16","0","0","0","16"
|
||||
"doctor-scheduler","11","0","11","0","0","0","11","0","0","11"
|
||||
"evidencelocker","36","0","36","0","0","0","36","0","0","36"
|
||||
"excititor","55","0","55","0","0","0","55","0","0","51"
|
||||
"exportcenter","64","0","64","0","0","0","64","0","0","0"
|
||||
"findings-ledger","83","0","83","0","0","0","83","0","0","45"
|
||||
"integrations","20","0","20","0","0","0","20","0","0","0"
|
||||
"issuerdirectory","12","0","12","0","0","0","12","0","0","12"
|
||||
"notifier","197","0","197","0","0","0","197","0","0","164"
|
||||
"notify","49","0","49","0","0","0","49","0","0","49"
|
||||
"opsmemory","12","0","12","0","0","0","12","0","0","0"
|
||||
"orchestrator","313","0","313","0","0","0","313","0","0","0"
|
||||
"packsregistry","19","0","19","0","0","0","19","0","0","19"
|
||||
"platform","165","0","165","0","0","0","165","0","0","139"
|
||||
"policy-engine","202","0","202","0","0","0","202","0","0","170"
|
||||
"policy-gateway","121","0","121","0","0","0","121","0","0","54"
|
||||
"reachgraph","17","0","17","0","0","0","17","0","0","17"
|
||||
"replay","15","0","15","0","0","0","15","0","0","15"
|
||||
"riskengine","9","0","9","0","0","0","9","0","0","5"
|
||||
"sbomservice","51","0","51","0","0","0","51","0","0","51"
|
||||
"scanner","168","0","168","0","0","128","40","0","0","103"
|
||||
"scheduler","37","0","37","0","0","0","37","0","0","37"
|
||||
"signals","23","0","23","0","0","0","23","0","0","23"
|
||||
"signer","15","0","15","0","0","0","15","0","0","15"
|
||||
"smremote","6","0","6","0","0","0","6","0","0","6"
|
||||
"symbols","19","0","19","0","0","0","19","0","0","19"
|
||||
"taskrunner","33","0","33","0","0","0","33","0","0","33"
|
||||
"timelineindexer","12","0","12","12","0","12","0","0","0","0"
|
||||
"unknowns","8","0","8","0","0","0","8","0","0","0"
|
||||
"vexhub","16","0","16","0","0","0","16","0","0","0"
|
||||
"vexlens","36","0","36","0","0","0","36","0","0","0"
|
||||
"vulnexplorer","10","0","10","0","0","0","10","0","0","10"
|
||||
|
@@ -0,0 +1,14 @@
|
||||
{
|
||||
"generatedUtc": "2026-02-22T17:24:57Z",
|
||||
"endpointCount": 2190,
|
||||
"anonymousEndpoints": 6,
|
||||
"requiresAuthEndpoints": 2184,
|
||||
"endpointsWithScopes": 28,
|
||||
"endpointsWithRoles": 0,
|
||||
"endpointsWithPolicies": 156,
|
||||
"authSourceNone": 1991,
|
||||
"missingDescriptions": 0,
|
||||
"genericHttpDescriptions": 0,
|
||||
"sameAsSummaryDescriptions": 1417,
|
||||
"tooShortDescriptions": 90
|
||||
}
|
||||
@@ -17,6 +17,7 @@ Advisory AI is the retrieval-augmented assistant that synthesizes advisory and V
|
||||
- RAG pipeline drawing from Conseiller, Excititor, VEX Lens, Policy Engine, and SBOM Service data.
|
||||
- Prompt templates and guard models enforcing provenance and redaction policies.
|
||||
- Vercel/offline inference workers with deterministic caching of generated artefacts.
|
||||
- AdvisoryAI Knowledge Search (AKS) for deterministic docs/API/Doctor retrieval: `docs/modules/advisory-ai/knowledge-search.md`.
|
||||
|
||||
## Integrations & dependencies
|
||||
- Authority for tenant-aware access control.
|
||||
|
||||
@@ -202,6 +202,79 @@ Fetch cached artefact (same envelope as §6). Requires `advisory-ai:view`.
|
||||
|
||||
When plan preview is enabled (feature flag `advisoryAi.planPreview.enabled`), this endpoint returns the orchestration plan using `AdvisoryPipelinePlanResponse` (task metadata, chunk/vector counts). Requires `advisory-ai:operate`.
|
||||
|
||||
### 7.8 `LLM Adapter Exposure (RVM-08)`
|
||||
|
||||
AdvisoryAI now exposes a unified adapter surface for provider discovery and OpenAI-compatible completion calls.
|
||||
|
||||
- `GET /v1/advisory-ai/adapters/llm/providers`
|
||||
- `POST /v1/advisory-ai/adapters/llm/{providerId}/chat/completions`
|
||||
- `POST /v1/advisory-ai/adapters/openai/v1/chat/completions` (alias for `providerId=openai`)
|
||||
|
||||
Scope requirements:
|
||||
|
||||
- Read/discovery: `advisory:adapter:read` or `advisory:run`
|
||||
- Invoke/completion: `advisory:adapter:invoke` or `advisory:openai:invoke` or `advisory:{providerId}:invoke` or `advisory:run`
|
||||
|
||||
Operational notes:
|
||||
|
||||
- Adapter endpoints are disabled by default and must be explicitly enabled by config.
|
||||
- `stream=true` is currently rejected on adapter completion endpoints.
|
||||
- Provider discovery returns configuration/validation/exposure state per provider so operators can verify whether `openai` is configured before traffic cutover.
|
||||
|
||||
Runtime config keys:
|
||||
|
||||
- `AdvisoryAI:Adapters:Llm:Enabled` (env `ADVISORYAI__AdvisoryAI__Adapters__Llm__Enabled`)
|
||||
- `AdvisoryAI:LlmProviders:ConfigDirectory` (env `ADVISORYAI__AdvisoryAI__LlmProviders__ConfigDirectory`)
|
||||
|
||||
Gateway exposure paths:
|
||||
|
||||
- `/v1/advisory-ai/adapters`
|
||||
- `/api/v1/advisory-ai/adapters`
|
||||
|
||||
### 7.9 `Knowledge Search (AKS)`
|
||||
|
||||
Deterministic retrieval API for docs, OpenAPI operations, and Doctor checks.
|
||||
|
||||
- `POST /v1/advisory-ai/search`
|
||||
- `POST /v1/advisory-ai/index/rebuild`
|
||||
|
||||
Scope notes:
|
||||
- Search: one of `advisory:run`, `advisory:search`, `advisory:read`.
|
||||
- Rebuild: one of `advisory:run`, `advisory:admin`, `advisory:index:write`.
|
||||
|
||||
Search request:
|
||||
```json
|
||||
{
|
||||
"q": "docker login fails with x509 unknown authority",
|
||||
"k": 10,
|
||||
"filters": {
|
||||
"type": ["docs", "doctor"],
|
||||
"product": "stella-ops",
|
||||
"version": "2026.02",
|
||||
"service": "gateway",
|
||||
"tags": ["troubleshooting", "tls"]
|
||||
},
|
||||
"includeDebug": false
|
||||
}
|
||||
```
|
||||
|
||||
Search response contains only grounded results with source actions:
|
||||
- docs: `{ path, anchor, spanStart, spanEnd }`
|
||||
- api: `{ service, method, path, operationId }`
|
||||
- doctor: `{ checkCode, severity, canRun, runCommand }`
|
||||
|
||||
Rebuild response:
|
||||
```json
|
||||
{
|
||||
"documentCount": 12034,
|
||||
"chunkCount": 68291,
|
||||
"apiSpecCount": 41,
|
||||
"apiOperationCount": 2132,
|
||||
"doctorProjectionCount": 84,
|
||||
"durationMs": 8245
|
||||
}
|
||||
```
|
||||
|
||||
## 8. Error model
|
||||
|
||||
Errors follow a standard problem+JSON envelope:
|
||||
|
||||
137
docs/modules/advisory-ai/knowledge-search.md
Normal file
137
docs/modules/advisory-ai/knowledge-search.md
Normal file
@@ -0,0 +1,137 @@
|
||||
# AdvisoryAI Knowledge Search (AKS)
|
||||
|
||||
## Why retrieval-first
|
||||
AKS is a deterministic retrieval system for operational problem solving across Stella Ops docs, OpenAPI contracts, and Doctor checks. It is designed to work offline and does not require GPU-backed or hosted LLM inference for correctness.
|
||||
|
||||
LLMs can still be used as optional formatters later, but AKS correctness is grounded in source retrieval and explicit references.
|
||||
|
||||
## Scope
|
||||
- Module owner: `src/AdvisoryAI/**`.
|
||||
- Search surfaces consuming AKS:
|
||||
- Web global search in `src/Web/StellaOps.Web/**`.
|
||||
- CLI commands in `src/Cli/**`.
|
||||
- Doctor execution remains authoritative in Doctor module. AKS only indexes metadata and remediation references.
|
||||
|
||||
## Architecture
|
||||
1. Ingestion/indexing:
|
||||
- Markdown (`docs/**`) -> section chunks.
|
||||
- OpenAPI (`openapi.json`) -> per-operation chunks + normalized operation tables.
|
||||
- Doctor seed/metadata -> doctor projection chunks.
|
||||
2. Storage:
|
||||
- PostgreSQL tables in schema `advisoryai` via migration `src/AdvisoryAI/StellaOps.AdvisoryAI/Storage/Migrations/002_knowledge_search.sql`.
|
||||
3. Retrieval:
|
||||
- FTS (`tsvector` + `websearch_to_tsquery`) + optional vector stage.
|
||||
- Deterministic fusion and tie-breaking in `KnowledgeSearchService`.
|
||||
4. Delivery:
|
||||
- API endpoint: `POST /v1/advisory-ai/search`.
|
||||
- Index rebuild endpoint: `POST /v1/advisory-ai/index/rebuild`.
|
||||
|
||||
## Data model
|
||||
AKS schema tables:
|
||||
- `advisoryai.kb_doc`: canonical source docs with product/version/content hash metadata.
|
||||
- `advisoryai.kb_chunk`: searchable units (`md_section`, `api_operation`, `doctor_check`) with anchors, spans, `tsvector`, and embeddings.
|
||||
- `advisoryai.api_spec`: raw OpenAPI snapshot (`jsonb`) by service.
|
||||
- `advisoryai.api_operation`: normalized operation records (`method`, `path`, `operation_id`, tags, request/response/security json).
|
||||
- `advisoryai.doctor_search_projection`: searchable doctor metadata and remediation.
|
||||
|
||||
Vector support:
|
||||
- Tries `CREATE EXTENSION vector`.
|
||||
- If unavailable, AKS remains fully functional via FTS and deterministic array embeddings fallback.
|
||||
|
||||
## Deterministic ingestion rules
|
||||
### Markdown
|
||||
- Chunk by H2/H3 headings.
|
||||
- Stable anchors using slug + duplicate suffix.
|
||||
- Stable chunk IDs from source path + anchor + span.
|
||||
- Metadata includes path, anchor, section path, tags.
|
||||
|
||||
### OpenAPI
|
||||
- Parse `openapi.json` only for deterministic MVP.
|
||||
- Emit one searchable chunk per HTTP operation.
|
||||
- Preserve structured operation payloads (`request_json`, `responses_json`, `security_json`).
|
||||
|
||||
### Doctor
|
||||
- Source order:
|
||||
1. Seed file `src/AdvisoryAI/StellaOps.AdvisoryAI/KnowledgeSearch/doctor-search-seed.json`.
|
||||
2. Optional Doctor endpoint metadata (`DoctorChecksEndpoint`) when configured.
|
||||
- Emit doctor chunk + projection record including:
|
||||
- `checkCode`, `title`, `severity`, `runCommand`, remediation, symptoms.
|
||||
|
||||
## Ranking strategy
|
||||
Implemented in `src/AdvisoryAI/StellaOps.AdvisoryAI/KnowledgeSearch/KnowledgeSearchService.cs`:
|
||||
- Candidate retrieval:
|
||||
- lexical set from FTS.
|
||||
- optional vector set from embedding candidates.
|
||||
- Fusion:
|
||||
- reciprocal rank fusion style scoring.
|
||||
- Deterministic boosts:
|
||||
- exact `checkCode` match.
|
||||
- exact `operationId` match.
|
||||
- `METHOD /path` match.
|
||||
- filter-aligned service/tag boosts.
|
||||
- Deterministic ordering:
|
||||
- score desc -> kind asc -> chunk id asc.
|
||||
|
||||
## API contract
|
||||
### Search
|
||||
- `POST /v1/advisory-ai/search`
|
||||
- Request:
|
||||
- `q` (required), `k`, `filters.type|product|version|service|tags`, `includeDebug`.
|
||||
- Response:
|
||||
- typed results (`docs|api|doctor`) with snippet, score, and open action.
|
||||
|
||||
### Rebuild
|
||||
- `POST /v1/advisory-ai/index/rebuild`
|
||||
- Rebuilds AKS deterministically from local docs/specs/doctor metadata.
|
||||
|
||||
## Web behavior
|
||||
Global search now consumes AKS and supports:
|
||||
- Mixed grouped results (`Docs`, `API Endpoints`, `Doctor Checks`).
|
||||
- Type filter chips.
|
||||
- Result actions:
|
||||
- Docs: `Open`.
|
||||
- API: `Curl` (copy command).
|
||||
- Doctor: `Run` (navigate to doctor and copy run command).
|
||||
- `More` action for "show more like this" local query expansion.
|
||||
|
||||
## CLI behavior
|
||||
AKS commands:
|
||||
- `stella search "<query>" [--type docs|api|doctor] [--product ...] [--version ...] [--service ...] [--tag ...] [--k N] [--json]`
|
||||
- `stella doctor suggest "<symptom>" [--product ...] [--version ...] [--k N] [--json]`
|
||||
- `stella advisoryai index rebuild [--json]`
|
||||
|
||||
Output:
|
||||
- Human mode: grouped actionable references.
|
||||
- JSON mode: stable machine-readable payload.
|
||||
|
||||
## Test/benchmark strategy
|
||||
Implemented benchmark framework:
|
||||
- Generator: `KnowledgeSearchBenchmarkDatasetGenerator` (deterministic synthetic set with explicit ground truth).
|
||||
- Runner: `KnowledgeSearchBenchmarkRunner` (recall@k, p50/p95 latency, stability pass).
|
||||
- Models/serialization:
|
||||
- `KnowledgeSearchBenchmarkModels.cs`
|
||||
- `KnowledgeSearchBenchmarkJson.cs`
|
||||
|
||||
Tests:
|
||||
- `src/AdvisoryAI/__Tests/StellaOps.AdvisoryAI.Tests/KnowledgeSearch/KnowledgeSearchBenchmarkTests.cs`
|
||||
- verifies deterministic dataset generation with >= 1000 queries.
|
||||
- verifies recall/latency metrics and top-k match behavior.
|
||||
|
||||
## Dedicated AKS test DB
|
||||
Compose profile:
|
||||
- `devops/compose/docker-compose.advisoryai-knowledge-test.yml`
|
||||
|
||||
Init script:
|
||||
- `devops/compose/postgres-init/advisoryai-knowledge-test/01_extensions.sql`
|
||||
|
||||
Example workflow:
|
||||
```bash
|
||||
docker compose -f devops/compose/docker-compose.advisoryai-knowledge-test.yml up -d
|
||||
stella advisoryai index rebuild --json
|
||||
dotnet test src/AdvisoryAI/__Tests/StellaOps.AdvisoryAI.Tests/StellaOps.AdvisoryAI.Tests.csproj
|
||||
```
|
||||
|
||||
## Known limitations and follow-ups
|
||||
- YAML OpenAPI ingestion is not included in MVP.
|
||||
- End-to-end benchmark against live Postgres-backed AKS service is planned as a follow-up CI lane.
|
||||
- Optional external embedding providers can be added later without changing API contracts.
|
||||
@@ -20,6 +20,50 @@ Both commands are designed to enforce the AOC guardrails documented in the [aggr
|
||||
|
||||
---
|
||||
|
||||
## 1.1 AdvisoryAI Knowledge Search
|
||||
|
||||
AKS commands expose deterministic retrieval from AdvisoryAI (`docs|api|doctor`) without requiring an LLM.
|
||||
|
||||
### `stella search`
|
||||
|
||||
```bash
|
||||
stella search "<query>" \
|
||||
[--type docs|api|doctor] \
|
||||
[--product <product>] \
|
||||
[--version <version>] \
|
||||
[--service <service>] \
|
||||
[--tag <tag>] \
|
||||
[--k <1-100>] \
|
||||
[--json]
|
||||
```
|
||||
|
||||
Notes:
|
||||
- `--type` and `--tag` are repeatable or comma-separated.
|
||||
- `--json` emits stable machine-readable payload with `results[].open`.
|
||||
|
||||
### `stella doctor suggest`
|
||||
|
||||
```bash
|
||||
stella doctor suggest "<symptom-or-error>" \
|
||||
[--product <product>] \
|
||||
[--version <version>] \
|
||||
[--k <1-100>] \
|
||||
[--json]
|
||||
```
|
||||
|
||||
Uses the same AKS index and prints grouped recommendations for:
|
||||
- Doctor checks (with run command).
|
||||
- Related docs anchors.
|
||||
- Related API operations.
|
||||
|
||||
### `stella advisoryai index rebuild`
|
||||
|
||||
```bash
|
||||
stella advisoryai index rebuild [--json]
|
||||
```
|
||||
|
||||
Rebuilds the deterministic AKS index from local markdown, OpenAPI, and Doctor metadata sources.
|
||||
|
||||
## 2 · `stella sources ingest --dry-run`
|
||||
|
||||
### 2.1 Synopsis
|
||||
|
||||
@@ -175,20 +175,39 @@ Each endpoint with claims gets a security requirement:
|
||||
```csharp
|
||||
public static JsonArray GenerateSecurityRequirement(EndpointDescriptor endpoint)
|
||||
{
|
||||
if (endpoint.RequiringClaims.Count == 0)
|
||||
return new JsonArray(); // No security required
|
||||
if (endpoint.AllowAnonymous)
|
||||
return new JsonArray(); // Anonymous endpoint
|
||||
|
||||
if (!endpoint.RequiresAuthentication && endpoint.RequiringClaims.Count == 0)
|
||||
return new JsonArray(); // No auth semantics published
|
||||
|
||||
return new JsonArray
|
||||
{
|
||||
new JsonObject
|
||||
{
|
||||
["BearerAuth"] = new JsonArray(),
|
||||
["OAuth2"] = new JsonArray(claims.Select(c => c.Type))
|
||||
["OAuth2"] = new JsonArray(scopes.Select(scope => scope))
|
||||
}
|
||||
};
|
||||
}
|
||||
```
|
||||
|
||||
### Router-specific OpenAPI extensions
|
||||
|
||||
Gateway now emits Router-specific extensions on each operation:
|
||||
|
||||
- `x-stellaops-gateway-auth`: effective authorization semantics projected from endpoint metadata.
|
||||
- `allowAnonymous`
|
||||
- `requiresAuthentication`
|
||||
- `source` (`None`, `AspNetMetadata`, `YamlOverride`, `Hybrid`)
|
||||
- optional `policies`, `roles`, `claimRequirements`
|
||||
- `x-stellaops-timeout`: timeout semantics used by gateway dispatch.
|
||||
- `effectiveSeconds`
|
||||
- `source` (`endpoint`, `gatewayRouteDefault`, and capped variants)
|
||||
- `endpointSeconds`, `gatewayRouteDefaultSeconds`, `gatewayGlobalCapSeconds` when available
|
||||
- precedence list: endpoint override -> service default -> gateway route default -> gateway global cap
|
||||
- `x-stellaops-timeout-seconds`: backward-compatible scalar alias for `effectiveSeconds`.
|
||||
|
||||
---
|
||||
|
||||
## Configuration Reference
|
||||
|
||||
@@ -90,6 +90,11 @@ StellaOps.Router.slnx
|
||||
| [rate-limiting.md](rate-limiting.md) | Centralized router rate limiting (dossier) |
|
||||
| [aspnet-endpoint-bridge.md](aspnet-endpoint-bridge.md) | Using ASP.NET endpoint registration as Router endpoint registration |
|
||||
| [messaging-valkey-transport.md](messaging-valkey-transport.md) | Messaging transport over Valkey |
|
||||
| [timelineindexer-microservice-pilot.md](timelineindexer-microservice-pilot.md) | TimelineIndexer Valkey microservice transport pilot mapping and rollback |
|
||||
| [webservices-valkey-rollout-matrix.md](webservices-valkey-rollout-matrix.md) | All-webservices Valkey microservice migration matrix (waves, owners, rollback) |
|
||||
| [microservice-transport-guardrails.md](microservice-transport-guardrails.md) | Plugin-only transport guardrails and migration PR checklist |
|
||||
| [authority-gateway-enforcement-runbook.md](authority-gateway-enforcement-runbook.md) | Operations runbook for gateway-enforced auth and signed identity envelope trust |
|
||||
| [rollout-acceptance-20260222.md](rollout-acceptance-20260222.md) | Dual-mode rollout acceptance package and evidence index |
|
||||
|
||||
### Implementation Guides (docs/modules/router/guides/)
|
||||
| Document | Purpose |
|
||||
|
||||
81
docs/modules/router/authority-gateway-enforcement-runbook.md
Normal file
81
docs/modules/router/authority-gateway-enforcement-runbook.md
Normal file
@@ -0,0 +1,81 @@
|
||||
# Router Authority Enforcement Runbook
|
||||
|
||||
This runbook documents operational semantics for gateway-enforced authorization and signed user-identity propagation in Stella Router deployments.
|
||||
|
||||
## Scope
|
||||
- Router gateway as centralized policy decision point (PDP).
|
||||
- Authority-driven effective claims.
|
||||
- Signed identity envelope propagation to webservices using Router trust mode.
|
||||
|
||||
## Required Configuration
|
||||
|
||||
Gateway-side:
|
||||
- `Gateway__Auth__Authority__Issuer`
|
||||
- `Gateway__Auth__Authority__RequireHttpsMetadata`
|
||||
- `Router__OnMissingAuthorization`
|
||||
|
||||
Service-side (per service `Router` section):
|
||||
- `Router__Enabled`
|
||||
- `Router__AuthorizationTrustMode`
|
||||
- `Router__IdentityEnvelopeSigningKey`
|
||||
- `Router__IdentityEnvelopeClockSkewSeconds`
|
||||
|
||||
Identity transport headers emitted by gateway:
|
||||
- `X-StellaOps-Identity-Envelope`
|
||||
- `X-StellaOps-Identity-Envelope-Signature`
|
||||
- `X-StellaOps-Identity-Envelope-Alg`
|
||||
- `X-StellaOps-Identity-Envelope-Kid`
|
||||
- `X-StellaOps-Identity-Envelope-Iat`
|
||||
- `X-StellaOps-Identity-Envelope-Exp`
|
||||
|
||||
## Trust Modes
|
||||
|
||||
`ServiceEnforced`
|
||||
- Service enforces its local policies.
|
||||
- Gateway envelope is optional for service authorization.
|
||||
|
||||
`Hybrid`
|
||||
- Service accepts gateway envelope when present.
|
||||
- Service can fall back to local checks for compatibility.
|
||||
|
||||
`GatewayEnforced`
|
||||
- Service requires valid signed gateway envelope.
|
||||
- Missing/invalid envelope is fail-closed (`403`/`401` based on service policy).
|
||||
- Use for centralized authorization rollout after verification.
|
||||
|
||||
## Key Rotation Procedure
|
||||
1. Rotate Authority/gateway signing material using the Authority key-rotation SOP:
|
||||
- `docs/modules/authority/operations/key-rotation.md`
|
||||
2. Update `Router__IdentityEnvelopeSigningKey` for gateway and services.
|
||||
3. Restart gateway and services in wave order (gateway first).
|
||||
4. Validate with targeted tests and route smoke.
|
||||
|
||||
## Authority Outage and Fallback
|
||||
|
||||
If Authority is degraded/unreachable:
|
||||
1. Keep gateway running with last known effective claim cache.
|
||||
2. For critical service continuity, temporarily switch affected services:
|
||||
- `Router__AuthorizationTrustMode=Hybrid`
|
||||
3. If envelope verification is disrupted by key mismatch, switch to `ServiceEnforced` only as emergency fallback.
|
||||
4. Record incident window and restore target mode (`GatewayEnforced` or `Hybrid`) after recovery.
|
||||
|
||||
## Compose Toggle Examples
|
||||
|
||||
Global gateway mode:
|
||||
```powershell
|
||||
$env:ROUTER_GATEWAY_CONFIG = "./router-gateway-local.json"
|
||||
docker compose -f devops/compose/docker-compose.stella-ops.yml up -d
|
||||
```
|
||||
|
||||
Emergency compatibility fallback (example service):
|
||||
```powershell
|
||||
$env:SCANNER_ROUTER_ENABLED = "true"
|
||||
$env:Router__AuthorizationTrustMode = "Hybrid"
|
||||
docker compose -f devops/compose/docker-compose.stella-ops.yml up -d scanner-web
|
||||
```
|
||||
|
||||
## Verification Checklist
|
||||
- Gateway route smoke has `500=0`.
|
||||
- OpenAPI operations expose `x-stellaops-gateway-auth`.
|
||||
- Envelope spoofing tests pass (`IdentityHeaderPolicyMiddlewareTests`).
|
||||
- Authority refresh tests pass (`AuthorityClaimsRefreshServiceTests`).
|
||||
@@ -74,9 +74,14 @@ if (bootstrapOptions.Transports.Messaging.Enabled)
|
||||
```
|
||||
|
||||
### Microservice
|
||||
- Register Valkey messaging services (`StellaOps.Messaging.Transport.Valkey`)
|
||||
- Add messaging transport client (`AddMessagingTransportClient`)
|
||||
- Ensure Microservice Router SDK connects via `IMicroserviceTransport`
|
||||
- Register router transports via plugin loading (no hard transport references in `StellaOps.Router.AspNet`).
|
||||
- Use `AddRouterMicroservice(...)` from `StellaOps.Router.AspNet`; it resolves configured gateway transport types through `RouterTransportPluginLoader`.
|
||||
- For messaging mode, the `StellaOps.Router.Transport.Messaging` plugin registers:
|
||||
- backend messaging plugin loading (`AddMessagingPlugins(...)`, env/config key `transport=valkey`)
|
||||
- Router messaging transport client (`AddMessagingTransportClient`)
|
||||
- Ensure the following plugin DLLs are available either as service dependencies or under configured plugin directories:
|
||||
- `StellaOps.Router.Transport.Messaging.dll`
|
||||
- `StellaOps.Messaging.Transport.Valkey.dll`
|
||||
|
||||
## Operational Semantics (Draft)
|
||||
- **At-least-once** delivery: message queues and leases imply retries are possible; handlers should be idempotent where feasible.
|
||||
@@ -104,4 +109,3 @@ if (bootstrapOptions.Transports.Messaging.Enabled)
|
||||
- microservice HELLO registration via messaging
|
||||
- request dispatch + response return
|
||||
3. Validate streaming support (or document as out-of-scope).
|
||||
|
||||
|
||||
51
docs/modules/router/microservice-transport-guardrails.md
Normal file
51
docs/modules/router/microservice-transport-guardrails.md
Normal file
@@ -0,0 +1,51 @@
|
||||
# Router Microservice Transport Guardrails
|
||||
|
||||
This document defines mandatory guardrails for migrating `StellaOps.*.WebService` services to Router microservice transport over Valkey messaging.
|
||||
|
||||
## Required Contract
|
||||
- Service startup must use `AddRouterMicroservice(...)` from `StellaOps.Router.AspNet`.
|
||||
- Router transport activation must be configuration-driven from environment/compose keys.
|
||||
- Transport implementation loading must remain plugin-based:
|
||||
- Router transport plugins from `Router:TransportPlugins:*`.
|
||||
- Messaging backend plugin from `Router:Messaging:PluginDirectory` and `Router:Messaging:SearchPattern`.
|
||||
- Messaging transport selection must be explicit (`Router:Messaging:Transport=valkey` for this rollout).
|
||||
- Services must keep rollback-compatible toggles so routing can return to reverse proxy without code edits.
|
||||
|
||||
## Required Configuration Keys
|
||||
- `<Service>:Router:Enabled`
|
||||
- `<Service>:Router:Gateways:*:TransportType`
|
||||
- `<Service>:Router:TransportPlugins:Directory`
|
||||
- `<Service>:Router:TransportPlugins:SearchPattern`
|
||||
- `<Service>:Router:Messaging:Transport`
|
||||
- `<Service>:Router:Messaging:PluginDirectory`
|
||||
- `<Service>:Router:Messaging:SearchPattern`
|
||||
- `<Service>:Router:Messaging:RequestQueueTemplate`
|
||||
- `<Service>:Router:Messaging:ResponseQueueName`
|
||||
- `<Service>:Router:Messaging:ConsumerGroup`
|
||||
- `<Service>:Router:Messaging:valkey:ConnectionString`
|
||||
- `<Service>:Router:Messaging:valkey:Database`
|
||||
|
||||
## Forbidden Patterns
|
||||
- Direct service-level DI calls to concrete transport registration methods such as:
|
||||
- `AddMessagingTransportClient()`
|
||||
- `AddTcpTransportClient()`
|
||||
- `AddUdpTransportClient()`
|
||||
- `AddRabbitMqTransportClient()`
|
||||
- `AddTlsTransportClient()`
|
||||
- Compile-time coupling from webservice projects to transport assemblies solely to enable default behavior.
|
||||
- Hardcoded transport selection in code paths that bypass environment/compose configuration.
|
||||
|
||||
## Required Test Coverage
|
||||
- Router enabled with missing gateway endpoints must fail fast.
|
||||
- Missing/invalid router options section input must fail fast.
|
||||
- Missing transport plugin for configured `TransportType` must fail fast with explicit error text.
|
||||
- Messaging transport binding must verify queue/timeouts and Valkey options from configuration.
|
||||
|
||||
## Migration PR Checklist
|
||||
- [ ] Service startup uses `AddRouterMicroservice(...)` and retains `TryUseStellaRouter(...)` behavior.
|
||||
- [ ] No direct concrete transport registration calls are added in service DI.
|
||||
- [ ] Compose/environment keys are added for router enablement, plugin directories, and Valkey settings.
|
||||
- [ ] Gateway route plan updated (`ReverseProxy` -> `Microservice`) with rollback instructions.
|
||||
- [ ] OpenAPI preview (`/openapi.json`) shows migrated endpoint paths plus schema components.
|
||||
- [ ] Targeted tests include plugin-missing and section-validation failure cases.
|
||||
- [ ] Sprint execution log and decisions/risks updated with doc links and evidence.
|
||||
@@ -12,6 +12,9 @@ The router provides a transport-agnostic communication layer between services, r
|
||||
- **Claims**: Authority-integrated authorization
|
||||
- **Health**: Automatic heartbeat and failover
|
||||
|
||||
Active rollout planning artifact:
|
||||
- [webservices-valkey-rollout-matrix.md](webservices-valkey-rollout-matrix.md) - source-of-truth matrix for wave assignment, acceptance owners, and rollback switches for all gateway reverse-proxy service hosts.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
Before migrating, ensure:
|
||||
|
||||
@@ -215,6 +215,14 @@ Client credentials flow with collected scopes (only if endpoints have claims):
|
||||
|
||||
Scopes are automatically collected from all connected services. If multiple endpoints require the same claim, it appears only once in the scopes list.
|
||||
|
||||
### Legacy HELLO Compatibility
|
||||
|
||||
`x-stellaops-gateway-auth.requiresAuthentication` is emitted from the Gateway's effective authorization semantics, not only the raw endpoint flag.
|
||||
|
||||
- If a microservice HELLO payload comes from an older router-common contract that does not include `requiresAuthentication`, the Gateway fails closed.
|
||||
- For `allowAnonymous: false` endpoints with no explicit auth flag, the Gateway treats the route as authenticated-only.
|
||||
- Public routes must be explicitly marked `AllowAnonymous` in the microservice to avoid accidental protection.
|
||||
|
||||
---
|
||||
|
||||
## Generated Document Structure
|
||||
|
||||
78
docs/modules/router/rollout-acceptance-20260222.md
Normal file
78
docs/modules/router/rollout-acceptance-20260222.md
Normal file
@@ -0,0 +1,78 @@
|
||||
# Router Rollout Acceptance Package (2026-02-22)
|
||||
|
||||
## Scope
|
||||
- Program closeout for Router sprints `047-050`.
|
||||
- Dual-mode verification:
|
||||
- Default microservice mode (`router-gateway-local.json`).
|
||||
- Reverse-proxy fallback mode (`router-gateway-local.reverseproxy.json`).
|
||||
|
||||
## Command Matrix
|
||||
- Image refresh:
|
||||
- `docker build ... -t stellaops/scanner-web:dev`
|
||||
- `docker build ... -t stellaops/integrations-web:dev`
|
||||
- `docker build ... -t stellaops/gateway:dev`
|
||||
- `docker build ... -t stellaops/doctor-web:dev`
|
||||
- Clean bootstrap:
|
||||
- `devops/compose/scripts/router-mode-redeploy.ps1 -Mode reverseproxy`
|
||||
- `devops/compose/scripts/router-mode-redeploy.ps1 -Mode microservice`
|
||||
- Route smoke:
|
||||
- `devops/compose/scripts/router-routeprefix-smoke.ps1`
|
||||
- Conformance tests:
|
||||
- `dotnet test src/Router/__Tests/StellaOps.Router.AspNet.Tests/StellaOps.Router.AspNet.Tests.csproj`
|
||||
- `dotnet test src/Router/__Tests/StellaOps.Router.Gateway.Tests/StellaOps.Router.Gateway.Tests.csproj`
|
||||
- `dotnet test src/Router/__Tests/StellaOps.Gateway.WebService.Tests/StellaOps.Gateway.WebService.Tests.csproj`
|
||||
|
||||
## Evidence Artifacts
|
||||
- `devops/compose/openapi_current.json`
|
||||
- `devops/compose/openapi_reverse.json`
|
||||
- `devops/compose/openapi_routeprefix_smoke_microservice.csv`
|
||||
- `devops/compose/openapi_routeprefix_smoke_reverseproxy.csv`
|
||||
- `devops/compose/openapi_quality_report_microservice.json`
|
||||
- `devops/compose/openapi_quality_report_reverseproxy.json`
|
||||
- `devops/compose/perf_microservice.json`
|
||||
- `devops/compose/perf_reverseproxy.json`
|
||||
- `devops/compose/perf_mode_comparison.json`
|
||||
- `devops/compose/timeline.json`
|
||||
- `devops/compose/timeline_schemas.json`
|
||||
- `devops/compose/openai_adapter.json`
|
||||
- `devops/compose/llm_providers.json`
|
||||
|
||||
## Acceptance Results
|
||||
|
||||
Microservice mode:
|
||||
- Route smoke summary: `200=30,302=4,400=25,401=14,403=2,404=3,410=1`
|
||||
- Route smoke blocker status: `500=0`
|
||||
- OpenAPI counts: `paths=1899`, `schemas=901`
|
||||
- Timeline contract gate: 4 timeline operations with summary/description/security/timeouts/auth extensions.
|
||||
- Advisory AI adapter exposure: 2 LLM adapter operations in aggregated `openapi.json`; providers endpoint denies anonymous access (`403`).
|
||||
|
||||
Reverse mode:
|
||||
- Route smoke summary: `200=15,400=6,401=18,404=15`
|
||||
- Route smoke blocker status: `500=0`
|
||||
- OpenAPI counts: `paths=1529`, `schemas=901`
|
||||
- Reverse-proxy exception set remains explicit (`/rekor`, `/platform`, static SPA/error routes).
|
||||
|
||||
Security and authorization:
|
||||
- Gateway and Router conformance suites pass.
|
||||
- Identity-header spoofing protections covered in gateway test suite.
|
||||
- Authority claims refresh and effective-claim integration covered in Router gateway tests.
|
||||
|
||||
## Performance Gate Decision
|
||||
- Baseline: reverse-proxy mode.
|
||||
- Candidate: microservice mode.
|
||||
- Comparison artifact: `devops/compose/perf_mode_comparison.json`.
|
||||
- Result:
|
||||
- `openapi.json` and timeline unauthorized path remained within acceptable local-dev drift.
|
||||
- Advisory AI adapter unauthorized path showed higher latency in microservice mode; accepted for now as non-blocking because error-rate and contract gates passed, with follow-up hardening item retained in Router backlog.
|
||||
|
||||
## Rollback
|
||||
- Scripted mode rollback:
|
||||
- `devops/compose/scripts/router-mode-redeploy.ps1 -Mode reverseproxy`
|
||||
- Return to default:
|
||||
- `devops/compose/scripts/router-mode-redeploy.ps1 -Mode microservice`
|
||||
- The redeploy helper now auto-recovers transient unhealthy services (bounded retries) before declaring failure.
|
||||
|
||||
## Final Runtime State
|
||||
- Default compose mode restored to microservice routing:
|
||||
- `ROUTER_GATEWAY_CONFIG=./router-gateway-local.json`
|
||||
- Timeline route `/api/v1/timeline` remains `Type=Microservice` with Valkey messaging transport.
|
||||
39
docs/modules/router/timelineindexer-microservice-pilot.md
Normal file
39
docs/modules/router/timelineindexer-microservice-pilot.md
Normal file
@@ -0,0 +1,39 @@
|
||||
# Router TimelineIndexer Microservice Pilot
|
||||
|
||||
## Scope
|
||||
- Pilot service: `TimelineIndexer` (`src/TimelineIndexer/StellaOps.TimelineIndexer/StellaOps.TimelineIndexer.WebService`).
|
||||
- Transport: `TransportType.Messaging` backed by Valkey.
|
||||
- Gateway entry under pilot: `/api/v1/timeline*`.
|
||||
|
||||
## Baseline
|
||||
- Previous gateway route:
|
||||
- `ReverseProxy /api/v1/timeline -> http://timelineindexer.stella-ops.local/api/v1/timeline`
|
||||
- Reverse proxy mode strips the matched prefix before forwarding.
|
||||
- Microservice mode does not strip prefixes and routes by method+path identity.
|
||||
|
||||
## Path Compatibility Mapping
|
||||
| External path | Reverse proxy behavior | Required microservice path | Pilot status |
|
||||
| --- | --- | --- | --- |
|
||||
| `/api/v1/timeline` | proxied to `/api/v1/timeline` upstream | `/api/v1/timeline` | converted |
|
||||
| `/api/v1/timeline/{eventId}` | proxied to `/api/v1/timeline/{eventId}` upstream | `/api/v1/timeline/{eventId}` | converted |
|
||||
| `/api/v1/timeline/{eventId}/evidence` | proxied to `/api/v1/timeline/{eventId}/evidence` upstream | `/api/v1/timeline/{eventId}/evidence` | converted |
|
||||
| `/api/v1/timeline/events` | proxied to `/api/v1/timeline/events` upstream | `/api/v1/timeline/events` | converted |
|
||||
| `/timelineindexer/*` | reverse-proxy prefix path for direct service access | unchanged (still reverse proxy) | unchanged |
|
||||
|
||||
TimelineIndexer now exposes both native and gateway-alias endpoints:
|
||||
- native: `/timeline*`
|
||||
- gateway alias: `/api/v1/timeline*`
|
||||
|
||||
## Compose Activation
|
||||
- Gateway messaging toggle: `ROUTER_GATEWAY_MESSAGING_ENABLED` (default `true`).
|
||||
- TimelineIndexer router toggle: `TIMELINE_ROUTER_ENABLED` (default `true`).
|
||||
- Compose env vars are applied on `timeline-indexer-web` (not `vexlens-web`) and mapped to:
|
||||
- `TimelineIndexer:Router`
|
||||
- `TimelineIndexer:Router:TransportPlugins:*`
|
||||
- `TimelineIndexer:Router:Messaging:*`
|
||||
- Valkey backend selection is environment-driven via `TimelineIndexer:Router:Messaging:Transport=valkey`.
|
||||
|
||||
## Rollback
|
||||
1. Set `TIMELINE_ROUTER_ENABLED=false` and `ROUTER_GATEWAY_MESSAGING_ENABLED=false` in compose environment.
|
||||
2. Revert route entry in `devops/compose/router-gateway-local.json` from `Microservice` back to `ReverseProxy`.
|
||||
3. Re-deploy stack and verify `/api/v1/timeline*` responses through reverse proxy path.
|
||||
@@ -48,6 +48,94 @@ builder.Services.TryAddStellaRouter(
|
||||
routerOptions: options.Router);
|
||||
```
|
||||
|
||||
#### Optional: generic microservice transport registration
|
||||
|
||||
For services that should auto-register transport clients from configuration, use:
|
||||
|
||||
```csharp
|
||||
builder.Services.AddRouterMicroservice(
|
||||
builder.Configuration,
|
||||
serviceName: "my-service-name",
|
||||
version: typeof(Program).Assembly.GetName().Version?.ToString() ?? "1.0.0",
|
||||
routerOptionsSection: "MyService:Router");
|
||||
```
|
||||
|
||||
`AddRouterMicroservice(...)` keeps `TryAddStellaRouter(...)` behavior and registers transport clients through `RouterTransportPluginLoader` based on configured gateway transport types (`InMemory`, `Tcp`, `Certificate`/`tls`, `Udp`, `RabbitMq`, `Messaging`).
|
||||
The `StellaOps.Router.AspNet` library does not hard-reference transport assemblies; transports are activated from plugin DLLs and environment/config values.
|
||||
|
||||
For Valkey messaging mode, configure:
|
||||
|
||||
```yaml
|
||||
myservice:
|
||||
router:
|
||||
enabled: true
|
||||
region: "local"
|
||||
transportPlugins:
|
||||
directory: "plugins/router/transports"
|
||||
searchPattern: "StellaOps.Router.Transport.*.dll"
|
||||
gateways:
|
||||
- host: "router.stella-ops.local"
|
||||
port: 9100
|
||||
transportType: "Messaging"
|
||||
messaging:
|
||||
transport: "valkey"
|
||||
pluginDirectory: "plugins/messaging"
|
||||
searchPattern: "StellaOps.Messaging.Transport.*.dll"
|
||||
requestQueueTemplate: "router:requests:{service}"
|
||||
responseQueueName: "router:responses"
|
||||
consumerGroup: "myservice"
|
||||
requestTimeout: "30s"
|
||||
leaseDuration: "5m"
|
||||
batchSize: 10
|
||||
heartbeatInterval: "10s"
|
||||
valkey:
|
||||
connectionString: "cache.stella-ops.local:6379"
|
||||
```
|
||||
|
||||
### 2.2 Gateway trust mode and identity envelope verification
|
||||
|
||||
Service-side Router bridge can enforce gateway-issued identity semantics:
|
||||
|
||||
```yaml
|
||||
myservice:
|
||||
router:
|
||||
authorizationTrustMode: "GatewayEnforced" # ServiceEnforced | Hybrid | GatewayEnforced
|
||||
identityEnvelopeSigningKey: "${ROUTER_IDENTITY_SIGNING_KEY}"
|
||||
identityEnvelopeClockSkewSeconds: 30
|
||||
```
|
||||
|
||||
- `ServiceEnforced`: service-local checks remain primary.
|
||||
- `Hybrid`: prefer signed envelope; fallback to legacy headers.
|
||||
- `GatewayEnforced`: fail closed when envelope is missing/invalid.
|
||||
|
||||
### 2.3 Timeout precedence
|
||||
|
||||
Gateway dispatch timeout is now resolved with explicit precedence:
|
||||
|
||||
1. Endpoint timeout (including endpoint override/service default published by service).
|
||||
2. Route default timeout (optional per gateway route via `defaultTimeout`).
|
||||
3. Gateway routing default timeout (`Gateway:Routing:DefaultTimeout`).
|
||||
4. Global gateway cap (`Gateway:Routing:GlobalTimeoutCap`).
|
||||
|
||||
Route-level timeout example:
|
||||
|
||||
```yaml
|
||||
gateway:
|
||||
routing:
|
||||
defaultTimeout: "30s"
|
||||
globalTimeoutCap: "120s"
|
||||
routes:
|
||||
- type: Microservice
|
||||
path: "/api/v1/timeline"
|
||||
translatesTo: "http://timelineindexer.stella-ops.local/api/v1/timeline"
|
||||
defaultTimeout: "15s"
|
||||
```
|
||||
|
||||
### 2.1 Gateway SPA deep-link handling with microservice routes
|
||||
|
||||
When gateway route prefixes overlap with UI routes (for example `/policy`), browser navigations must still resolve to the SPA shell.
|
||||
Gateway `RouteDispatchMiddleware` now serves the configured static SPA fallback route for browser document requests on both `ReverseProxy` and `Microservice` route types. API prefixes (`/api`, `/v1`) are explicitly excluded from this fallback and continue to dispatch to backend services.
|
||||
|
||||
### 3. Enable Middleware
|
||||
|
||||
After `UseAuthorization()`, add:
|
||||
|
||||
75
docs/modules/router/webservices-valkey-rollout-matrix.md
Normal file
75
docs/modules/router/webservices-valkey-rollout-matrix.md
Normal file
@@ -0,0 +1,75 @@
|
||||
# Router Valkey Microservice Rollout Matrix (All WebServices)
|
||||
|
||||
## Scope
|
||||
- Source route table: `devops/compose/router-gateway-local.json`
|
||||
- Source service aliases: `devops/compose/docker-compose.stella-ops.yml`
|
||||
- Snapshot date (UTC): 2026-02-21
|
||||
- ReverseProxy routes in snapshot: 116
|
||||
- Distinct target service hosts: 42
|
||||
|
||||
Legend:
|
||||
- `Wave`: `A`, `B`, `C`, `D`, `PILOT`
|
||||
- `Acceptance Owner`: rollout sign-off owner for that host's cutover wave
|
||||
- `Migration Disposition`: current route-level migration intent
|
||||
|
||||
## Service Migration Matrix
|
||||
|
||||
| Service Host | Compose Service | Current ReverseProxy Path Prefixes | Wave | Acceptance Owner | Migration Disposition | Rollback Switch |
|
||||
| --- | --- | --- | --- | --- | --- | --- |
|
||||
| advisoryai.stella-ops.local | advisory-ai-web | /advisoryai, /api/v1/advisory, /api/v1/advisory-ai, /api/v1/advisory-ai/adapters, /v1/advisory-ai, /v1/advisory-ai/adapters | A | Developer + Test Automation (Wave A) | Migrate API prefixes to Microservice; keep root compatibility path until cutover acceptance. | Route type revert in `router-gateway-local.json` + `ADVISORYAI_ROUTER_ENABLED=false` (standardized in RMW-03). |
|
||||
| airgap-controller.stella-ops.local | airgap-controller | /airgapController | A | Developer + Test Automation (Wave A) | Add API-form microservice endpoint mapping and migrate root compatibility path in same wave. | Route type revert + `AIRGAP_CONTROLLER_ROUTER_ENABLED=false` (RMW-03). |
|
||||
| airgap-time.stella-ops.local | airgap-time | /airgapTime | A | Developer + Test Automation (Wave A) | Add API-form microservice endpoint mapping and migrate root compatibility path in same wave. | Route type revert + `AIRGAP_TIME_ROUTER_ENABLED=false` (RMW-03). |
|
||||
| attestor.stella-ops.local | attestor | /api/v1/attestations, /api/v1/attestor, /api/v1/witnesses, /attestor | B | Developer + Test Automation (Wave B) | Migrate API prefixes first; keep root compatibility route until evidence-plane acceptance sign-off. | Route type revert + `ATTESTOR_ROUTER_ENABLED=false` (RMW-03). |
|
||||
| authority.stella-ops.local | authority | /.well-known, /api/v1/authority, /api/v1/trust, /authority, /connect, /console, /jwks | B | Developer + Test Automation (Wave B) | Migrate Authority API and OIDC identity routes to Microservice; use in-service OIDC bridge endpoints (`/connect/*`, `/well-known/openid-configuration`) for protocol compatibility. | Route type revert + `AUTHORITY_ROUTER_ENABLED=false` (RMW-03). |
|
||||
| binaryindex.stella-ops.local | binaryindex-web | /api/v1/ops/binaryindex, /api/v1/resolve, /binaryindex | A | Developer + Test Automation (Wave A) | Migrate API prefixes to Microservice; keep root compatibility path during transition. | Route type revert + `BINARYINDEX_ROUTER_ENABLED=false` (RMW-03). |
|
||||
| cartographer.stella-ops.local | cartographer | /cartographer | D | Developer + Test Automation (Wave D) | Introduce API alias if required, then migrate route to Microservice in Wave D. | Route type revert + `CARTOGRAPHER_ROUTER_ENABLED=false` (RMW-03). |
|
||||
| concelier.stella-ops.local | concelier | /api/v1/concelier, /concelier | D | Developer + Test Automation (Wave D) | Migrate API prefix first, then root compatibility route. | Route type revert + `CONCELIER_ROUTER_ENABLED=false` (RMW-03). |
|
||||
| doctor.stella-ops.local | doctor-web | /api/doctor, /doctor | D | Developer + Test Automation (Wave D) | Migrate API prefix first; keep root compatibility path until UI/runtime consumers are validated. | Route type revert + `DOCTOR_ROUTER_ENABLED=false` (RMW-03). |
|
||||
| doctor-scheduler.stella-ops.local | doctor-scheduler | /api/v1/doctor/scheduler | D | Developer + Test Automation (Wave D) | Migrate API prefix directly to Microservice. | Route type revert + `DOCTOR_SCHEDULER_ROUTER_ENABLED=false` (RMW-03). |
|
||||
| evidencelocker.stella-ops.local | evidence-locker-web | /api/v1/evidence, /api/v1/proofs, /api/v1/verdicts, /api/verdicts, /evidencelocker, /v1/evidence-packs | B | Developer + Test Automation (Wave B) | Migrate API/v1 and v1 endpoints first; keep root compatibility path until evidence workflows pass QA. | Route type revert + `EVIDENCELOCKER_ROUTER_ENABLED=false` (RMW-03). |
|
||||
| excititor.stella-ops.local | excititor | /excititor | D | Developer + Test Automation (Wave D) | Add API-form microservice mapping if needed; migrate root compatibility route in Wave D. | Route type revert + `EXCITITOR_ROUTER_ENABLED=false` (RMW-03). |
|
||||
| exportcenter.stella-ops.local | export | /api/v1/export, /exportcenter, /v1/audit-bundles | B | Developer + Test Automation (Wave B) | Migrate API/v1 and v1 routes first; keep root compatibility path until trust/evidence export checks pass. | Route type revert + `EXPORTCENTER_ROUTER_ENABLED=false` (RMW-03). |
|
||||
| findings.stella-ops.local | findings-ledger-web | /api/v1/findings, /findingsLedger | D | Developer + Test Automation (Wave D) | Migrate API prefix first, then root compatibility path. | Route type revert + `FINDINGS_ROUTER_ENABLED=false` (RMW-03). |
|
||||
| gateway.stella-ops.local | gateway | /gateway | D | Developer + Test Automation (Wave D) | Defer root compatibility path until downstream service migration waves are complete. | Route type revert + `GATEWAY_ROUTER_ENABLED=false` (RMW-03). |
|
||||
| integrations.stella-ops.local | integrations-web | /api/v1/integrations, /integrations | A | Developer + Test Automation (Wave A) | Migrate API prefix first, then root compatibility path. | Route type revert + `INTEGRATIONS_ROUTER_ENABLED=false` (RMW-03). |
|
||||
| issuerdirectory.stella-ops.local | issuer-directory | /issuerdirectory | B | Developer + Test Automation (Wave B) | Migrate route in trust-plane wave with issuer/auth verification checks. | Route type revert + `ISSUERDIRECTORY_ROUTER_ENABLED=false` (RMW-03). |
|
||||
| notifier.stella-ops.local | notifier-web | /api/v1/notifier, /notifier | D | Developer + Test Automation (Wave D) | Migrate API prefix first, then root compatibility path. | Route type revert + `NOTIFIER_ROUTER_ENABLED=false` (RMW-03). |
|
||||
| notify.stella-ops.local | notify-web | /api/v1/notify, /notify | D | Developer + Test Automation (Wave D) | Migrate API prefix first, then root compatibility path. | Route type revert + `NOTIFY_ROUTER_ENABLED=false` (RMW-03). |
|
||||
| opsmemory.stella-ops.local | opsmemory-web | /api/v1/opsmemory, /opsmemory | A | Developer + Test Automation (Wave A) | Migrate API prefix first, then root compatibility path. | Route type revert + `OPSMEMORY_ROUTER_ENABLED=false` (RMW-03). |
|
||||
| orchestrator.stella-ops.local | orchestrator | /api/approvals, /api/orchestrator, /api/release-orchestrator, /api/releases, /api/v1/orchestrator, /api/v1/release-orchestrator, /api/v1/workflows, /orchestrator, /v1/runs | C | Developer + Test Automation (Wave C) | Migrate all API/v1 and v1 routes first; keep root compatibility path until control-plane acceptance. | Route type revert + `ORCHESTRATOR_ROUTER_ENABLED=false` (RMW-03). |
|
||||
| packsregistry.stella-ops.local | packsregistry-web | /packsregistry | A | Developer + Test Automation (Wave A) | Add API-form endpoint mapping if required, then migrate root compatibility route. | Route type revert + `PACKSREGISTRY_ROUTER_ENABLED=false` (RMW-03). |
|
||||
| platform.stella-ops.local | platform | /api, /api/admin, /api/analytics, /api/v1/authority/quotas, /api/v1/gateway/rate-limits, /api/v1/platform, /envsettings.json, /platform | C | Developer + Test Automation (Wave C) | Migrate API prefixes to Microservice; keep `/platform` and `/envsettings.json` reverse proxy for static/bootstrap behavior. | Route type revert + `PLATFORM_ROUTER_ENABLED=false` (RMW-03). |
|
||||
| policy-engine.stella-ops.local | policy-engine | /api/risk, /api/risk-budget, /api/v1/determinization, /policyEngine | C | Developer + Test Automation (Wave C) | Migrate API prefixes first; keep root compatibility path until control-plane verification completes. | Route type revert + `POLICY_ENGINE_ROUTER_ENABLED=false` (RMW-03). |
|
||||
| policy-gateway.stella-ops.local | policy | /api/cvss, /api/exceptions, /api/gate, /api/policy, /api/v1/governance, /api/v1/policy, /policy, /policyGateway | C | Developer + Test Automation (Wave C) | Migrate API prefixes first; keep `/policy` and `/policyGateway` compatibility paths until final cutover. | Route type revert + `POLICY_GATEWAY_ROUTER_ENABLED=false` (RMW-03). |
|
||||
| reachgraph.stella-ops.local | reachgraph-web | /api/v1/reachability, /reachgraph | D | Developer + Test Automation (Wave D) | Migrate API prefix first, then root compatibility path. | Route type revert + `REACHGRAPH_ROUTER_ENABLED=false` (RMW-03). |
|
||||
| registry-token.stella-ops.local | registry-token | /registryTokenservice | A | Developer + Test Automation (Wave A) | Migrate compatibility route with token flow validation in Wave A. | Route type revert + `REGISTRY_TOKEN_ROUTER_ENABLED=false` (RMW-03). |
|
||||
| replay.stella-ops.local | replay-web | /replay | A | Developer + Test Automation (Wave A) | Migrate compatibility route in Wave A; add API-form alias if needed. | Route type revert + `REPLAY_ROUTER_ENABLED=false` (RMW-03). |
|
||||
| riskengine.stella-ops.local | riskengine-web | /riskengine | C | Developer + Test Automation (Wave C) | Migrate compatibility route in control-plane wave; add API alias if required. | Route type revert + `RISKENGINE_ROUTER_ENABLED=false` (RMW-03). |
|
||||
| sbomservice.stella-ops.local | sbomservice | /api/change-traces, /api/compare, /api/sbomservice, /api/v1/lineage, /api/v1/sbom, /api/v1/sources, /sbomservice | D | Developer + Test Automation (Wave D) | Migrate API prefixes first; keep root compatibility path until graph/feed wave acceptance. | Route type revert + `SBOMSERVICE_ROUTER_ENABLED=false` (RMW-03). |
|
||||
| scanner.stella-ops.local | scanner-web | /api/fix-verification, /api/v1/scanner, /api/v1/secrets, /api/v1/triage, /api/v1/vulnerabilities, /api/v1/watchlist, /scanner | D | Developer + Test Automation (Wave D) | Migrate API prefixes first; keep root compatibility path until scanner behavioral checks pass. | Route type revert + `SCANNER_ROUTER_ENABLED=false` (RMW-03). |
|
||||
| scheduler.stella-ops.local | scheduler-web | /api/scheduler, /scheduler | C | Developer + Test Automation (Wave C) | Migrate API prefix first, then root compatibility path. | Route type revert + `SCHEDULER_ROUTER_ENABLED=false` (RMW-03). |
|
||||
| signals.stella-ops.local | signals | /api/v1/signals, /signals | D | Developer + Test Automation (Wave D) | Migrate API prefix first, then root compatibility path. | Route type revert + `SIGNALS_ROUTER_ENABLED=false` (RMW-03). |
|
||||
| signer.stella-ops.local | signer | /signer | B | Developer + Test Automation (Wave B) | Migrate compatibility route in trust/evidence wave with signing validation. | Route type revert + `SIGNER_ROUTER_ENABLED=false` (RMW-03). |
|
||||
| smremote.stella-ops.local | smremote | /smremote | A | Developer + Test Automation (Wave A) | Migrate compatibility route in Wave A; add API alias if required. | Route type revert + `SMREMOTE_ROUTER_ENABLED=false` (RMW-03). |
|
||||
| symbols.stella-ops.local | symbols | /symbols | A | Developer + Test Automation (Wave A) | Migrate compatibility route in Wave A; add API alias if required. | Route type revert + `SYMBOLS_ROUTER_ENABLED=false` (RMW-03). |
|
||||
| taskrunner.stella-ops.local | taskrunner-web | /taskrunner | C | Developer + Test Automation (Wave C) | Migrate compatibility route in control-plane wave; add API alias if required. | Route type revert + `TASKRUNNER_ROUTER_ENABLED=false` (RMW-03). |
|
||||
| timelineindexer.stella-ops.local | timeline-indexer-web | /timelineindexer | PILOT | Developer (pilot accepted) | Timeline API is already microservice (`/api/v1/timeline`); keep root compatibility route reverse proxy until later cleanup. | Route type revert + `TIMELINE_ROUTER_ENABLED=false` (already supported). |
|
||||
| unknowns.stella-ops.local | unknowns-web | /unknowns | A | Developer + Test Automation (Wave A) | Migrate compatibility route in Wave A; add API alias if required. | Route type revert + `UNKNOWNS_ROUTER_ENABLED=false` (RMW-03). |
|
||||
| vexhub.stella-ops.local | vexhub-web | /api/v1/vex, /api/vex, /vexhub | D | Developer + Test Automation (Wave D) | Migrate API prefixes first, then root compatibility path. | Route type revert + `VEXHUB_ROUTER_ENABLED=false` (RMW-03). |
|
||||
| vexlens.stella-ops.local | vexlens-web | /api/v1/vexlens, /vexlens | D | Developer + Test Automation (Wave D) | Migrate API prefix first, then root compatibility path. | Route type revert + `VEXLENS_ROUTER_ENABLED=false` (RMW-03). |
|
||||
| vulnexplorer.stella-ops.local | api | /api/vuln-explorer, /vulnexplorer | D | Developer + Test Automation (Wave D) | Migrate API prefix first; keep root compatibility path until vuln explorer routing is validated. | Route type revert + `VULNEXPLORER_ROUTER_ENABLED=false` (RMW-03). |
|
||||
|
||||
## Wave Acceptance Mapping
|
||||
|
||||
| Wave | Acceptance Owner |
|
||||
| --- | --- |
|
||||
| A | Developer + Test Automation (Wave A) |
|
||||
| B | Developer + Test Automation (Wave B) |
|
||||
| C | Developer + Test Automation (Wave C) |
|
||||
| D | Developer + Test Automation (Wave D) |
|
||||
| PILOT | Developer (pilot accepted) |
|
||||
|
||||
## Notes
|
||||
- This matrix is the authoritative inventory artifact for sprint task `RMW-01`.
|
||||
- Route-level rollback is always available by reverting route `Type` back to `ReverseProxy` in `devops/compose/router-gateway-local.json`.
|
||||
- Service env rollback keys are currently planning placeholders and are standardized in `RMW-03`.
|
||||
@@ -59,9 +59,16 @@ src/TimelineIndexer/StellaOps.TimelineIndexer/
|
||||
## 4) REST API
|
||||
|
||||
```
|
||||
GET /timeline?subject={id}&from={date}&to={date} → { events[] }
|
||||
GET /timeline/{eventId} → { event }
|
||||
GET /timeline/stats?subject={id} → { statistics }
|
||||
GET /timeline?eventType=&source=&correlationId=&traceId=&severity=&since=&after=&limit=
|
||||
GET /timeline/{eventId}
|
||||
GET /timeline/{eventId}/evidence
|
||||
POST /timeline/events
|
||||
|
||||
# Gateway microservice aliases
|
||||
GET /api/v1/timeline
|
||||
GET /api/v1/timeline/{eventId}
|
||||
GET /api/v1/timeline/{eventId}/evidence
|
||||
POST /api/v1/timeline/events
|
||||
|
||||
GET /healthz | /readyz | /metrics
|
||||
```
|
||||
@@ -72,3 +79,5 @@ GET /healthz | /readyz | /metrics
|
||||
|
||||
* Signals: `../signals/architecture.md`
|
||||
* Scanner: `../scanner/architecture.md`
|
||||
|
||||
|
||||
|
||||
@@ -29,9 +29,16 @@ Events are stored append-only with tenant-specific partitions. Producers include
|
||||
|
||||
## 2. APIs
|
||||
|
||||
- `GET /api/v1/timeline/events` – paginated event stream with filters (tenant, category, time window, correlation IDs).
|
||||
- `GET /api/v1/timeline/events/{id}` – fetch single event payload.
|
||||
- `GET /api/v1/timeline/export` – NDJSON export for offline review.
|
||||
- Native endpoints:
|
||||
- `GET /timeline` - query timeline entries with filter parameters.
|
||||
- `GET /timeline/{eventId}` - fetch a single timeline entry.
|
||||
- `GET /timeline/{eventId}/evidence` - fetch evidence linked to a timeline entry.
|
||||
- `POST /timeline/events` - ingestion ack endpoint.
|
||||
- Router/Gateway aliases for microservice transport routing:
|
||||
- `GET /api/v1/timeline`
|
||||
- `GET /api/v1/timeline/{eventId}`
|
||||
- `GET /api/v1/timeline/{eventId}/evidence`
|
||||
- `POST /api/v1/timeline/events`
|
||||
|
||||
API headers: `X-Stella-Tenant`, optional `X-Stella-TraceId`, and `If-None-Match` for cache revalidation.
|
||||
|
||||
@@ -73,3 +80,4 @@ Privacy/PII: producers must redact PII before emission; once emitted, redactions
|
||||
- `docs/modules/zastava/architecture.md`
|
||||
- `docs/modules/export-center/architecture.md`
|
||||
- `src/TimelineIndexer/StellaOps.TimelineIndexer`
|
||||
|
||||
|
||||
@@ -103,15 +103,38 @@ Infrastructure components (PostgreSQL, Valkey, RustFS) are pinned in the release
|
||||
-f devops/compose/docker-compose.prod.yaml \
|
||||
up -d
|
||||
```
|
||||
4. Tail logs for critical services (`docker compose logs -f authority concelier`).
|
||||
5. Update monitoring dashboards/alerts to confirm normal operation.
|
||||
4. Run migration status and verification checks:
|
||||
```bash
|
||||
stella system migrations-status --module all
|
||||
stella system migrations-verify --module all
|
||||
stella system migrations-run --module all --category release --dry-run
|
||||
```
|
||||
5. Execute release migrations when approved:
|
||||
```bash
|
||||
stella system migrations-run --module all --category release --force
|
||||
stella system migrations-status --module all
|
||||
```
|
||||
6. Tail logs for critical services (`docker compose logs -f authority concelier`).
|
||||
7. Update monitoring dashboards/alerts to confirm normal operation.
|
||||
|
||||
Migration notes:
|
||||
|
||||
- Compose PostgreSQL init scripts in `devops/compose/postgres-init` are first-initialization only.
|
||||
- CLI module coverage is currently limited; consult `docs/db/MIGRATION_INVENTORY.md` before production upgrades.
|
||||
- Consolidation target policy and cutover waves are documented in `docs/db/MIGRATION_CONSOLIDATION_PLAN.md`.
|
||||
- For upgradeable on-prem installations, the `stella system migrations-*` sequence is the required release migration gate.
|
||||
- UI-driven migration execution must call Platform admin APIs (`/api/v1/admin/migrations/*`) and not connect directly from browser to PostgreSQL.
|
||||
|
||||
### Rollback (Compose)
|
||||
|
||||
1. Check out the previous release tag (e.g. `git checkout 2025.09.1`).
|
||||
2. Re-run `docker compose pull` and `docker compose up -d` with that profile. Docker will restore the prior digests.
|
||||
3. If reverting to a known-good snapshot is required, restore volume backups (see `docs/modules/authority/operations/backup-restore.md` and associated service guides).
|
||||
4. Log the rollback in the operations journal.
|
||||
2. Capture migration state before rollback:
|
||||
```bash
|
||||
stella system migrations-status --module all
|
||||
```
|
||||
3. Re-run `docker compose pull` and `docker compose up -d` with that profile. Docker will restore the prior digests.
|
||||
4. If schema incompatibility is detected, restore known-good database snapshots before service restart (see `docs/modules/authority/operations/backup-restore.md` and associated service guides).
|
||||
5. Re-check migration status and log rollback actions in the operations journal.
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -122,26 +122,37 @@ kubectl get pods -n stellaops-green -w
|
||||
### Run Migrations
|
||||
|
||||
```bash
|
||||
# Step 16: Apply Category A migrations (startup)
|
||||
# Step 16: Check current migration state (CLI-registered modules)
|
||||
stella system migrations-status --module all
|
||||
|
||||
# Step 17: Verify checksums before execution
|
||||
stella system migrations-verify --module all
|
||||
|
||||
# Step 18: Preview release migrations (manual gate)
|
||||
stella system migrations-run \
|
||||
--category A \
|
||||
--namespace stellaops-green
|
||||
--module all \
|
||||
--category release \
|
||||
--dry-run
|
||||
|
||||
# Step 17: Verify migration success
|
||||
stella system migrations-status --namespace stellaops-green
|
||||
# All migrations should show "Applied"
|
||||
|
||||
# Step 18: Apply Category B migrations if needed (manual)
|
||||
# Review migration list first
|
||||
stella system migrations-pending --category B
|
||||
|
||||
# Apply after review
|
||||
# Step 19: Execute approved release migrations
|
||||
stella system migrations-run \
|
||||
--category B \
|
||||
--namespace stellaops-green \
|
||||
--confirm
|
||||
--module all \
|
||||
--category release \
|
||||
--force
|
||||
|
||||
# Step 20: Verify migration success
|
||||
stella system migrations-status --module all
|
||||
```
|
||||
|
||||
Migration notes:
|
||||
|
||||
- Category names are `startup`, `release`, `seed`, and `data`.
|
||||
- Compose and service startup paths may apply additional migrations outside CLI coverage.
|
||||
- Use `docs/db/MIGRATION_INVENTORY.md` for current module-by-module runner coverage before production upgrade.
|
||||
- Canonical consolidation policy and wave plan are in `docs/db/MIGRATION_CONSOLIDATION_PLAN.md`.
|
||||
- For upgradeable on-prem environments, treat this CLI sequence as the required release migration gate.
|
||||
- UI-triggered migration operations must execute through Platform admin APIs (`/api/v1/admin/migrations/*`) with `platform.setup.admin` (no browser-direct PostgreSQL access).
|
||||
|
||||
### Evidence Migration (If Required)
|
||||
|
||||
```bash
|
||||
|
||||
Reference in New Issue
Block a user