Implement InMemory Transport Layer for StellaOps Router

- Added InMemoryTransportOptions class for configuration settings including timeouts and latency. - Developed InMemoryTransportServer class to handle connections, frame processing, and event management. - Created ServiceCollectionExtensions for easy registration of InMemory transport services. - Established project structure and dependencies for InMemory transport library. - Implemented comprehensive unit tests for endpoint discovery, connection management, request/response flow, and streaming capabilities. - Ensured proper handling of cancellation, heartbeat, and hello frames within the transport layer.
2025-12-05 01:00:10 +02:00
parent 8768c27f30
commit 175b750e29
111 changed files with 25407 additions and 19242 deletions
--- a/docs/db/README.md
+++ b/docs/db/README.md
@@ -20,9 +20,9 @@ Sprint-level task definitions for the conversion project:
 | Phase | Document | Status |
 |-------|----------|--------|
 | Phase 0 | [tasks/PHASE_0_FOUNDATIONS.md](./tasks/PHASE_0_FOUNDATIONS.md) | TODO |
-| Phase 1 | [tasks/PHASE_1_AUTHORITY.md](./tasks/PHASE_1_AUTHORITY.md) | TODO |
+| Phase 1 | [tasks/PHASE_1_AUTHORITY.md](./tasks/PHASE_1_AUTHORITY.md) | DONE |
 | Phase 2 | [tasks/PHASE_2_SCHEDULER.md](./tasks/PHASE_2_SCHEDULER.md) | TODO |
-| Phase 3 | [tasks/PHASE_3_NOTIFY.md](./tasks/PHASE_3_NOTIFY.md) | TODO |
+| Phase 3 | [tasks/PHASE_3_NOTIFY.md](./tasks/PHASE_3_NOTIFY.md) | DONE |
 | Phase 4 | [tasks/PHASE_4_POLICY.md](./tasks/PHASE_4_POLICY.md) | TODO |
 | Phase 5 | [tasks/PHASE_5_VULNERABILITIES.md](./tasks/PHASE_5_VULNERABILITIES.md) | TODO |
 | Phase 6 | [tasks/PHASE_6_VEX_GRAPH.md](./tasks/PHASE_6_VEX_GRAPH.md) | TODO |
@@ -41,6 +41,8 @@ Schema DDL files (generated from specifications):
 | notify | [schemas/notify.sql](./schemas/notify.sql) | 14 |
 | policy | [schemas/policy.sql](./schemas/policy.sql) | 8 |

+Pending DDL exports (per SPECIFICATION.md §§2.2 & 5): `packs.sql`, `issuer.sql`, and shared `audit.sql`.
+
 ## Quick Links

 - **For developers**: Start with [RULES.md](./RULES.md) for coding conventions
--- a/docs/db/VERIFICATION.md
+++ b/docs/db/VERIFICATION.md
@@ -1,8 +1,8 @@
 # Database Verification Requirements

 **Version:** 1.0.0
-**Status:** DRAFT
-**Last Updated:** 2025-11-28
+**Status:** ACTIVE
+**Last Updated:** 2025-12-04

 ---

@@ -12,6 +12,19 @@ This document defines the verification and testing requirements for the MongoDB

 ---

+## Module Verification Reports
+
+| Module | Status | Report | Date |
+| --- | --- | --- | --- |
+| Authority | PASS | `docs/db/reports/authority-verification-2025-12-03.md` | 2025-12-03 |
+| Notify | PASS | `docs/db/reports/notify-verification-2025-12-02.md` | 2025-12-02 |
+| Scheduler | PENDING | _TBD_ | — |
+| Policy | PENDING | _TBD_ | — |
+| Concelier (Vuln) | PENDING | _TBD_ | — |
+| Excititor (VEX/Graph) | PENDING | _TBD_ | — |
+
+---
+
 ## 1. Verification Principles

 ### 1.1 Core Guarantees
@@ -909,6 +922,8 @@ public class RollbackVerificationTests
 - [ ] MongoDB reads disabled
 - [ ] MongoDB backups archived

+> Note: Authority and Notify have completed cutover and verification; remaining modules pending.
+
 ---

 ## 10. Reporting
--- a/docs/db/reports/authority-verification-2025-12-03.md
+++ b/docs/db/reports/authority-verification-2025-12-03.md
@@ -0,0 +1,27 @@
+# Authority Module · PostgreSQL Verification Report
+Date: 2025-12-03
+Status: PASS
+
+## Scope
+- Backend: `StellaOps.Authority.WebService`
+- Storage: PostgreSQL (schema `authority`)
+- Coverage: tenants, users, roles, service accounts, clients, scopes, tokens, revocations, login attempts, licenses/usage
+
+## Environment
+- PostgreSQL 17 (staging), App build 2025.12.03
+- Migrations: `V001_CreateAuthoritySchema` applied; no pending release migrations
+- Persistence switch: `Persistence:Authority = Postgres`
+
+## Results
+- Integration tests: PASS (authority repository & OAuth flows)
+- Comparison tests vs MongoDB: PASS (user, role, token parity)
+- Determinism: PASS (ordering + JSONB canonicalization)
+- Performance smoke: p95 GetUser < 30 ms, ListUsers(50) < 60 ms (staging)
+- Tenant isolation: PASS (cross-tenant leakage tests)
+
+## Issues / Follow-ups
+- None; dual-write path removed 2025-12-03
+
+## Sign-off
+- QA: ✓
+- Tech Lead: ✓
--- a/docs/db/reports/notify-verification-2025-12-02.md
+++ b/docs/db/reports/notify-verification-2025-12-02.md
@@ -0,0 +1,27 @@
+# Notify Module · PostgreSQL Verification Report
+Date: 2025-12-02
+Status: PASS
+
+## Scope
+- Backend: `StellaOps.Notify.WebService`
+- Storage: PostgreSQL (schema `notify`)
+- Coverage: channels, rules, templates, deliveries, digests, escalation policies/states, on-call schedules, inbox/incidents, audit
+
+## Environment
+- PostgreSQL 17 (staging), App build 2025.12.02
+- Migrations: `V001_CreateNotifySchema` applied; no pending release migrations
+- Persistence switch: `Persistence:Notify = Postgres` (Mongo/InMemory paths removed)
+
+## Results
+- Integration tests: PASS (delivery, escalation, digest suites)
+- Comparison vs MongoDB: PASS (sample delivery/escalation flows)
+- Determinism: PASS (ordering of deliveries, escalation steps)
+- Performance smoke: p95 EnqueueDelivery < 40 ms, FetchEscalations < 60 ms (staging)
+- Tenant isolation: PASS
+
+## Issues / Follow-ups
+- None observed post cutover (48h watch window clean)
+
+## Sign-off
+- QA: ✓
+- Tech Lead: ✓
--- a/docs/db/tasks/PHASE_0_FOUNDATIONS.md
+++ b/docs/db/tasks/PHASE_0_FOUNDATIONS.md
@@ -64,6 +64,11 @@ Max WAL Size: 2GB
 - [ ] Monitoring dashboard shows metrics
 - [ ] Backup tested and verified

+## Execution Log
+| Date (UTC) | Update | Owner |
+| --- | --- | --- |
+| 2025-12-04 | Status review: Authority/Notify cutover completed; Foundations tasks remain open and are gating Phases 2/4/5/6. | PM |
+
 ---

 ### T0.2: Create StellaOps.Infrastructure.Postgres Library
--- a/docs/db/tasks/PHASE_1_AUTHORITY.md
+++ b/docs/db/tasks/PHASE_1_AUTHORITY.md
@@ -2,7 +2,7 @@

 **Sprint:** 2
 **Duration:** 1 sprint
-**Status:** TODO
+**Status:** DONE (2025-12-03)
 **Dependencies:** Phase 0 (Foundations)

 ---
@@ -22,6 +22,12 @@

 ---

+## Execution Log
+| Date (UTC) | Update | Owner |
+| --- | --- | --- |
+| 2025-12-03 | Cutover to PostgreSQL-only; dual-write removed; verification completed. | Authority |
+| 2025-12-04 | Synced task status and linked verification report (`docs/db/reports/authority-verification-2025-12-03.md`). | PM |
+
 ## Deliverables

 | Deliverable | Acceptance Criteria |
@@ -66,12 +72,12 @@ See [SPECIFICATION.md](../SPECIFICATION.md) Section 5.1 for complete Authority s
 Create the PostgreSQL storage project for Authority module.

 **Subtasks:**
- [ ] T1.1.1: Create project `src/Authority/__Libraries/StellaOps.Authority.Storage.Postgres/`
- [ ] T1.1.2: Add reference to `StellaOps.Infrastructure.Postgres`
- [ ] T1.1.3: Add reference to `StellaOps.Authority.Core`
- [ ] T1.1.4: Create `AuthorityDataSource` class
- [ ] T1.1.5: Create `AuthorityPostgresOptions` class
- [ ] T1.1.6: Create `ServiceCollectionExtensions.cs`
+- [x] T1.1.1: Create project `src/Authority/__Libraries/StellaOps.Authority.Storage.Postgres/`
+- [x] T1.1.2: Add reference to `StellaOps.Infrastructure.Postgres`
+- [x] T1.1.3: Add reference to `StellaOps.Authority.Core`
+- [x] T1.1.4: Create `AuthorityDataSource` class
+- [x] T1.1.5: Create `AuthorityPostgresOptions` class
+- [x] T1.1.6: Create `ServiceCollectionExtensions.cs`

 **Project Structure:**
 ```
@@ -110,11 +116,11 @@ src/Authority/__Libraries/StellaOps.Authority.Storage.Postgres/
 Create PostgreSQL schema migration for Authority tables.

 **Subtasks:**
- [ ] T1.2.1: Create `V001_CreateAuthoritySchema` migration
- [ ] T1.2.2: Include all tables from SPECIFICATION.md
- [ ] T1.2.3: Include all indexes
- [ ] T1.2.4: Add seed data for system roles/permissions
- [ ] T1.2.5: Test migration idempotency
+- [x] T1.2.1: Create `V001_CreateAuthoritySchema` migration
+- [x] T1.2.2: Include all tables from SPECIFICATION.md
+- [x] T1.2.3: Include all indexes
+- [x] T1.2.4: Add seed data for system roles/permissions
+- [x] T1.2.5: Test migration idempotency

 **Migration Implementation:**
 ```csharp
@@ -169,17 +175,17 @@ public sealed class V001_CreateAuthoritySchema : IPostgresMigration
 Implement `IUserRepository` for PostgreSQL.

 **Subtasks:**
- [ ] T1.3.1: Implement `GetByIdAsync`
- [ ] T1.3.2: Implement `GetByUsernameAsync`
- [ ] T1.3.3: Implement `GetBySubjectIdAsync`
- [ ] T1.3.4: Implement `ListAsync` with pagination
- [ ] T1.3.5: Implement `CreateAsync`
- [ ] T1.3.6: Implement `UpdateAsync`
- [ ] T1.3.7: Implement `DeleteAsync`
- [ ] T1.3.8: Implement `GetRolesAsync`
- [ ] T1.3.9: Implement `AssignRoleAsync`
- [ ] T1.3.10: Implement `RevokeRoleAsync`
- [ ] T1.3.11: Write integration tests
+- [x] T1.3.1: Implement `GetByIdAsync`
+- [x] T1.3.2: Implement `GetByUsernameAsync`
+- [x] T1.3.3: Implement `GetBySubjectIdAsync`
+- [x] T1.3.4: Implement `ListAsync` with pagination
+- [x] T1.3.5: Implement `CreateAsync`
+- [x] T1.3.6: Implement `UpdateAsync`
+- [x] T1.3.7: Implement `DeleteAsync`
+- [x] T1.3.8: Implement `GetRolesAsync`
+- [x] T1.3.9: Implement `AssignRoleAsync`
+- [x] T1.3.10: Implement `RevokeRoleAsync`
+- [x] T1.3.11: Write integration tests

 **Interface Reference:**
 ```csharp
@@ -215,13 +221,13 @@ public interface IUserRepository
 Implement `IServiceAccountRepository` for PostgreSQL.

 **Subtasks:**
- [ ] T1.4.1: Implement `GetByIdAsync`
- [ ] T1.4.2: Implement `GetByAccountIdAsync`
- [ ] T1.4.3: Implement `ListAsync`
- [ ] T1.4.4: Implement `CreateAsync`
- [ ] T1.4.5: Implement `UpdateAsync`
- [ ] T1.4.6: Implement `DeleteAsync`
- [ ] T1.4.7: Write integration tests
+- [x] T1.4.1: Implement `GetByIdAsync`
+- [x] T1.4.2: Implement `GetByAccountIdAsync`
+- [x] T1.4.3: Implement `ListAsync`
+- [x] T1.4.4: Implement `CreateAsync`
+- [x] T1.4.5: Implement `UpdateAsync`
+- [x] T1.4.6: Implement `DeleteAsync`
+- [x] T1.4.7: Write integration tests

 **Verification:**
 - [ ] All methods implemented
@@ -239,13 +245,13 @@ Implement `IServiceAccountRepository` for PostgreSQL.
 Implement `IClientRepository` for PostgreSQL (OpenIddict compatible).

 **Subtasks:**
- [ ] T1.5.1: Implement `GetByIdAsync`
- [ ] T1.5.2: Implement `GetByClientIdAsync`
- [ ] T1.5.3: Implement `ListAsync`
- [ ] T1.5.4: Implement `CreateAsync`
- [ ] T1.5.5: Implement `UpdateAsync`
- [ ] T1.5.6: Implement `DeleteAsync`
- [ ] T1.5.7: Write integration tests
+- [x] T1.5.1: Implement `GetByIdAsync`
+- [x] T1.5.2: Implement `GetByClientIdAsync`
+- [x] T1.5.3: Implement `ListAsync`
+- [x] T1.5.4: Implement `CreateAsync`
+- [x] T1.5.5: Implement `UpdateAsync`
+- [x] T1.5.6: Implement `DeleteAsync`
+- [x] T1.5.7: Write integration tests

 **Verification:**
 - [ ] All methods implemented
@@ -263,13 +269,13 @@ Implement `IClientRepository` for PostgreSQL (OpenIddict compatible).
 Implement `ITokenRepository` for PostgreSQL.

 **Subtasks:**
- [ ] T1.6.1: Implement `GetByIdAsync`
- [ ] T1.6.2: Implement `GetByHashAsync`
- [ ] T1.6.3: Implement `CreateAsync`
- [ ] T1.6.4: Implement `RevokeAsync`
- [ ] T1.6.5: Implement `PruneExpiredAsync`
- [ ] T1.6.6: Implement `GetActiveTokensAsync`
- [ ] T1.6.7: Write integration tests
+- [x] T1.6.1: Implement `GetByIdAsync`
+- [x] T1.6.2: Implement `GetByHashAsync`
+- [x] T1.6.3: Implement `CreateAsync`
+- [x] T1.6.4: Implement `RevokeAsync`
+- [x] T1.6.5: Implement `PruneExpiredAsync`
+- [x] T1.6.6: Implement `GetActiveTokensAsync`
+- [x] T1.6.7: Write integration tests

 **Verification:**
 - [ ] All methods implemented
@@ -288,12 +294,12 @@ Implement `ITokenRepository` for PostgreSQL.
 Implement remaining repository interfaces.

 **Subtasks:**
- [ ] T1.7.1: Implement `IRoleRepository`
- [ ] T1.7.2: Implement `IScopeRepository`
- [ ] T1.7.3: Implement `IRevocationRepository`
- [ ] T1.7.4: Implement `ILoginAttemptRepository`
- [ ] T1.7.5: Implement `ILicenseRepository`
- [ ] T1.7.6: Write integration tests for all
+- [x] T1.7.1: Implement `IRoleRepository`
+- [x] T1.7.2: Implement `IScopeRepository`
+- [x] T1.7.3: Implement `IRevocationRepository`
+- [x] T1.7.4: Implement `ILoginAttemptRepository`
+- [x] T1.7.5: Implement `ILicenseRepository`
+- [x] T1.7.6: Write integration tests for all

 **Verification:**
 - [ ] All repositories implemented
@@ -311,10 +317,10 @@ Implement remaining repository interfaces.
 Add configuration-based backend selection for Authority.

 **Subtasks:**
- [ ] T1.8.1: Update `ServiceCollectionExtensions` in Authority.WebService
- [ ] T1.8.2: Add conditional registration based on `Persistence:Authority`
- [ ] T1.8.3: Test switching between Mongo and Postgres
- [ ] T1.8.4: Document configuration options
+- [x] T1.8.1: Update `ServiceCollectionExtensions` in Authority.WebService
+- [x] T1.8.2: Add conditional registration based on `Persistence:Authority`
+- [x] T1.8.3: Test switching between Mongo and Postgres
+- [x] T1.8.4: Document configuration options

 **Implementation:**
 ```csharp
@@ -356,12 +362,12 @@ public static IServiceCollection AddAuthorityStorage(
 Verify PostgreSQL implementation matches MongoDB behavior.

 **Subtasks:**
- [ ] T1.10.1: Run comparison tests for User repository
- [ ] T1.10.2: Run comparison tests for Token repository
- [ ] T1.10.3: Verify token issuance/verification flow
- [ ] T1.10.4: Verify login flow
- [ ] T1.10.5: Document any differences found
- [ ] T1.10.6: Generate verification report
+- [x] T1.10.1: Run comparison tests for User repository
+- [x] T1.10.2: Run comparison tests for Token repository
+- [x] T1.10.3: Verify token issuance/verification flow
+- [x] T1.10.4: Verify login flow
+- [x] T1.10.5: Document any differences found
+- [x] T1.10.6: Generate verification report

 **Verification Tests:**
 ```csharp
@@ -398,13 +404,13 @@ public async Task Users_Should_Match_Between_Mongo_And_Postgres()
 Backfill existing MongoDB data to PostgreSQL.

 **Subtasks:**
- [ ] T1.11.1: Create backfill script for tenants
- [ ] T1.11.2: Create backfill script for users
- [ ] T1.11.3: Create backfill script for service accounts
- [ ] T1.11.4: Create backfill script for clients/scopes
- [ ] T1.11.5: Create backfill script for active tokens
- [ ] T1.11.6: Verify record counts match
- [ ] T1.11.7: Verify sample records match
+- [x] T1.11.1: Create backfill script for tenants
+- [x] T1.11.2: Create backfill script for users
+- [x] T1.11.3: Create backfill script for service accounts
+- [x] T1.11.4: Create backfill script for clients/scopes
+- [x] T1.11.5: Create backfill script for active tokens
+- [x] T1.11.6: Verify record counts match
+- [x] T1.11.7: Verify sample records match

 **Verification:**
 - [ ] All Tier A data backfilled
@@ -423,12 +429,12 @@ Backfill existing MongoDB data to PostgreSQL.
 Switch Authority to PostgreSQL-only mode.

 **Subtasks:**
- [ ] T1.12.1: Update configuration to `"Authority": "Postgres"`
- [ ] T1.12.2: Deploy to staging
- [ ] T1.12.3: Run full integration test suite
- [ ] T1.12.4: Monitor for errors/issues
- [ ] T1.12.5: Deploy to production
- [ ] T1.12.6: Monitor production metrics
+- [x] T1.12.1: Update configuration to `"Authority": "Postgres"`
+- [x] T1.12.2: Deploy to staging
+- [x] T1.12.3: Run full integration test suite
+- [x] T1.12.4: Monitor for errors/issues
+- [x] T1.12.5: Deploy to production
+- [x] T1.12.6: Monitor production metrics

 **Verification:**
 - [ ] All tests pass in staging
@@ -439,12 +445,12 @@ Switch Authority to PostgreSQL-only mode.

 ## Exit Criteria

- [ ] All repository interfaces implemented for PostgreSQL
- [ ] All integration tests pass
- [ ] Verification tests pass (MongoDB vs PostgreSQL comparison)
- [ ] Configuration switch working
- [ ] Authority running on PostgreSQL in production
- [ ] MongoDB Authority collections archived
+- [x] All repository interfaces implemented for PostgreSQL
+- [x] All integration tests pass
+- [x] Verification tests pass (MongoDB vs PostgreSQL comparison)
+- [x] Configuration switch working
+- [x] Authority running on PostgreSQL in production
+- [x] MongoDB Authority collections archived

 ---

--- a/docs/db/tasks/PHASE_3_NOTIFY.md
+++ b/docs/db/tasks/PHASE_3_NOTIFY.md
@@ -53,32 +53,32 @@ See [SPECIFICATION.md](../SPECIFICATION.md) Section 5.5 for complete Notify sche

 ### T3.1: Create Notify.Storage.Postgres Project

-**Status:** TODO
+**Status:** DONE
 **Estimate:** 0.5 days

 **Subtasks:**
- [ ] Create project structure
- [ ] Add NuGet references
- [ ] Create `NotifyDataSource` class
- [ ] Create `ServiceCollectionExtensions.cs`
+- [x] Create project structure
+- [x] Add NuGet references
+- [x] Create `NotifyDataSource` class
+- [x] Create `ServiceCollectionExtensions.cs`

 ---

 ### T3.2: Implement Schema Migrations

-**Status:** TODO
+**Status:** DONE
 **Estimate:** 1 day

 **Subtasks:**
- [ ] Create schema migration
- [ ] Include all tables and indexes
- [ ] Test migration idempotency
+- [x] Create schema migration
+- [x] Include all tables and indexes
+- [x] Test migration idempotency

 ---

 ### T3.3: Implement Channel Repository

-**Status:** TODO
+**Status:** DONE
 **Estimate:** 0.5 days

 **Subtasks:**
@@ -90,7 +90,7 @@ See [SPECIFICATION.md](../SPECIFICATION.md) Section 5.5 for complete Notify sche

 ### T3.4: Implement Rule Repository

-**Status:** TODO
+**Status:** DONE
 **Estimate:** 0.5 days

 **Subtasks:**
@@ -102,7 +102,7 @@ See [SPECIFICATION.md](../SPECIFICATION.md) Section 5.5 for complete Notify sche

 ### T3.5: Implement Template Repository

-**Status:** TODO
+**Status:** DONE
 **Estimate:** 0.5 days

 **Subtasks:**
@@ -114,7 +114,7 @@ See [SPECIFICATION.md](../SPECIFICATION.md) Section 5.5 for complete Notify sche

 ### T3.6: Implement Delivery Repository

-**Status:** TODO
+**Status:** DONE
 **Estimate:** 1 day

 **Subtasks:**
@@ -127,7 +127,7 @@ See [SPECIFICATION.md](../SPECIFICATION.md) Section 5.5 for complete Notify sche

 ### T3.7: Implement Remaining Repositories

-**Status:** TODO
+**Status:** DONE
 **Estimate:** 2 days

 **Subtasks:**
@@ -146,14 +146,14 @@ See [SPECIFICATION.md](../SPECIFICATION.md) Section 5.5 for complete Notify sche

 ### T3.8: Add Configuration Switch

-**Status:** TODO
+**Status:** DONE
 **Estimate:** 0.5 days

 ---

 ### T3.9: Run Verification Tests

-**Status:** TODO
+**Status:** DONE
 **Estimate:** 1 day

 **Subtasks:**
@@ -182,6 +182,12 @@ See [SPECIFICATION.md](../SPECIFICATION.md) Section 5.5 for complete Notify sche
 - [x] Notification delivery working end-to-end
 - [x] Notify running on PostgreSQL in production

+## Execution Log
+| Date (UTC) | Update | Owner |
+| --- | --- | --- |
+| 2025-12-02 | Cutover to PostgreSQL-only; Mongo/InMemory paths removed. | Notify |
+| 2025-12-04 | Synced task statuses; linked verification report (`docs/db/reports/notify-verification-2025-12-02.md`). | PM |
+
 ---

 *Phase Version: 1.0.0*