up

2025-11-28 20:55:22 +02:00
parent d040c001ac
commit 2548abc56f
231 changed files with 47468 additions and 68 deletions
--- a/docs/db/tasks/PHASE_0_FOUNDATIONS.md
+++ b/docs/db/tasks/PHASE_0_FOUNDATIONS.md
@@ -0,0 +1,404 @@
+# Phase 0: Foundations
+
+**Sprint:** 1
+**Duration:** 1 sprint
+**Status:** TODO
+**Dependencies:** None
+
+---
+
+## Objectives
+
+1. Provision PostgreSQL cluster for staging and production
+2. Create shared infrastructure library (`StellaOps.Infrastructure.Postgres`)
+3. Set up CI/CD pipeline for PostgreSQL migrations
+4. Establish Testcontainers-based integration testing
+
+---
+
+## Deliverables
+
+| Deliverable | Acceptance Criteria |
+|-------------|---------------------|
+| PostgreSQL cluster | Running in staging with proper configuration |
+| Shared library | DataSource, migrations, extensions implemented |
+| CI pipeline | PostgreSQL tests running on every PR |
+| Documentation | SPECIFICATION.md, RULES.md reviewed and approved |
+
+---
+
+## Task Breakdown
+
+### T0.1: PostgreSQL Cluster Provisioning
+
+**Status:** TODO
+**Assignee:** TBD
+**Estimate:** 2 days
+
+**Description:**
+Provision PostgreSQL 16+ cluster with appropriate configuration for StellaOps workload.
+
+**Subtasks:**
+- [ ] T0.1.1: Select PostgreSQL hosting (managed vs self-hosted)
+- [ ] T0.1.2: Create staging cluster with single primary
+- [ ] T0.1.3: Configure connection pooling (PgBouncer or built-in)
+- [ ] T0.1.4: Set up backup and restore procedures
+- [ ] T0.1.5: Configure monitoring (pg_stat_statements, Prometheus exporter)
+- [ ] T0.1.6: Document connection strings and access credentials
+- [ ] T0.1.7: Configure SSL/TLS for connections
+
+**Configuration Requirements:**
+```
+PostgreSQL Version: 16+
+Max Connections: 100 (via pooler: 500)
+Shared Buffers: 25% of RAM
+Work Mem: 64MB
+Maintenance Work Mem: 512MB
+WAL Level: replica
+Max WAL Size: 2GB
+```
+
+**Verification:**
+- [ ] Can connect from development machines
+- [ ] Can connect from CI/CD runners
+- [ ] Monitoring dashboard shows metrics
+- [ ] Backup tested and verified
+
+---
+
+### T0.2: Create StellaOps.Infrastructure.Postgres Library
+
+**Status:** TODO
+**Assignee:** TBD
+**Estimate:** 3 days
+
+**Description:**
+Create shared library with reusable PostgreSQL infrastructure components.
+
+**Subtasks:**
+- [ ] T0.2.1: Create project `src/Shared/StellaOps.Infrastructure.Postgres/`
+- [ ] T0.2.2: Add Npgsql NuGet package reference
+- [ ] T0.2.3: Implement `DataSourceBase` abstract class
+- [ ] T0.2.4: Implement `IPostgresMigration` interface
+- [ ] T0.2.5: Implement `PostgresMigrationRunner` class
+- [ ] T0.2.6: Implement `NpgsqlExtensions` helper methods
+- [ ] T0.2.7: Implement `ServiceCollectionExtensions` for DI
+- [ ] T0.2.8: Add XML documentation to all public APIs
+- [ ] T0.2.9: Add unit tests for migration runner
+
+**Files to Create:**
+```
+src/Shared/StellaOps.Infrastructure.Postgres/
+├── StellaOps.Infrastructure.Postgres.csproj
+├── DataSourceBase.cs
+├── PostgresOptions.cs
+├── Migrations/
+│   ├── IPostgresMigration.cs
+│   └── PostgresMigrationRunner.cs
+├── Extensions/
+│   ├── NpgsqlExtensions.cs
+│   └── NpgsqlCommandExtensions.cs
+└── ServiceCollectionExtensions.cs
+```
+
+**DataSourceBase Implementation:**
+```csharp
+public abstract class DataSourceBase : IAsyncDisposable
+{
+    protected readonly NpgsqlDataSource DataSource;
+    protected readonly PostgresOptions Options;
+
+    protected DataSourceBase(IOptions<PostgresOptions> options)
+    {
+        Options = options.Value;
+        var builder = new NpgsqlDataSourceBuilder(Options.ConnectionString);
+        ConfigureDataSource(builder);
+        DataSource = builder.Build();
+    }
+
+    protected virtual void ConfigureDataSource(NpgsqlDataSourceBuilder builder)
+    {
+        // Override in derived classes for module-specific config
+    }
+
+    public async Task<NpgsqlConnection> OpenConnectionAsync(
+        string tenantId,
+        CancellationToken cancellationToken = default)
+    {
+        var connection = await DataSource.OpenConnectionAsync(cancellationToken);
+        await ConfigureSessionAsync(connection, tenantId, cancellationToken);
+        return connection;
+    }
+
+    protected virtual async Task ConfigureSessionAsync(
+        NpgsqlConnection connection,
+        string tenantId,
+        CancellationToken cancellationToken)
+    {
+        await using var cmd = connection.CreateCommand();
+        cmd.CommandText = $"""
+            SET app.tenant_id = '{tenantId}';
+            SET timezone = 'UTC';
+            SET statement_timeout = '{Options.CommandTimeoutSeconds}s';
+            """;
+        await cmd.ExecuteNonQueryAsync(cancellationToken);
+    }
+
+    public async ValueTask DisposeAsync()
+    {
+        await DataSource.DisposeAsync();
+        GC.SuppressFinalize(this);
+    }
+}
+```
+
+**Verification:**
+- [ ] Project builds without errors
+- [ ] Unit tests pass
+- [ ] Can be referenced from module projects
+
+---
+
+### T0.3: Migration Framework Implementation
+
+**Status:** TODO
+**Assignee:** TBD
+**Estimate:** 2 days
+
+**Description:**
+Implement idempotent migration framework for schema management.
+
+**Subtasks:**
+- [ ] T0.3.1: Define `IPostgresMigration` interface
+- [ ] T0.3.2: Implement `PostgresMigrationRunner` with transaction support
+- [ ] T0.3.3: Implement migration tracking table (`_migrations`)
+- [ ] T0.3.4: Add `IHostedService` for automatic migration on startup
+- [ ] T0.3.5: Add CLI command for manual migration execution
+- [ ] T0.3.6: Add migration rollback support (optional)
+
+**Migration Interface:**
+```csharp
+public interface IPostgresMigration
+{
+    /// <summary>
+    /// Unique migration identifier (e.g., "V001_CreateAuthoritySchema")
+    /// </summary>
+    string Id { get; }
+
+    /// <summary>
+    /// Human-readable description
+    /// </summary>
+    string Description { get; }
+
+    /// <summary>
+    /// Apply the migration
+    /// </summary>
+    Task UpAsync(NpgsqlConnection connection, CancellationToken cancellationToken);
+
+    /// <summary>
+    /// Rollback the migration (optional)
+    /// </summary>
+    Task DownAsync(NpgsqlConnection connection, CancellationToken cancellationToken);
+}
+```
+
+**Verification:**
+- [ ] Migrations run idempotently (can run multiple times)
+- [ ] Migration state tracked correctly
+- [ ] Failed migrations roll back cleanly
+
+---
+
+### T0.4: CI/CD Pipeline Configuration
+
+**Status:** TODO
+**Assignee:** TBD
+**Estimate:** 2 days
+
+**Description:**
+Add PostgreSQL integration testing to CI/CD pipeline.
+
+**Subtasks:**
+- [ ] T0.4.1: Add Testcontainers.PostgreSql NuGet package to test projects
+- [ ] T0.4.2: Create `PostgresTestFixture` base class
+- [ ] T0.4.3: Update CI workflow to support PostgreSQL containers
+- [ ] T0.4.4: Add parallel test execution configuration
+- [ ] T0.4.5: Add test coverage reporting for PostgreSQL code
+
+**PostgresTestFixture:**
+```csharp
+public sealed class PostgresTestFixture : IAsyncLifetime
+{
+    private readonly PostgreSqlContainer _container;
+    private NpgsqlDataSource? _dataSource;
+
+    public PostgresTestFixture()
+    {
+        _container = new PostgreSqlBuilder()
+            .WithImage("postgres:16-alpine")
+            .WithDatabase("stellaops_test")
+            .WithUsername("test")
+            .WithPassword("test")
+            .WithWaitStrategy(Wait.ForUnixContainer()
+                .UntilPortIsAvailable(5432))
+            .Build();
+    }
+
+    public string ConnectionString => _container.GetConnectionString();
+    public NpgsqlDataSource DataSource => _dataSource
+        ?? throw new InvalidOperationException("Fixture not initialized");
+
+    public async Task InitializeAsync()
+    {
+        await _container.StartAsync();
+        _dataSource = NpgsqlDataSource.Create(ConnectionString);
+    }
+
+    public async Task DisposeAsync()
+    {
+        if (_dataSource is not null)
+            await _dataSource.DisposeAsync();
+        await _container.DisposeAsync();
+    }
+}
+```
+
+**CI Workflow Update:**
+```yaml
+# .gitea/workflows/build-test-deploy.yml
+- name: Run PostgreSQL Integration Tests
+  run: |
+    dotnet test src/StellaOps.sln \
+      --filter "Category=PostgresIntegration" \
+      --logger "trx;LogFileName=postgres-test-results.trx"
+  env:
+    TESTCONTAINERS_RYUK_DISABLED: true
+```
+
+**Verification:**
+- [ ] CI pipeline runs PostgreSQL tests
+- [ ] Tests can run in parallel without conflicts
+- [ ] Test results reported correctly
+
+---
+
+### T0.5: Persistence Configuration
+
+**Status:** TODO
+**Assignee:** TBD
+**Estimate:** 1 day
+
+**Description:**
+Add persistence backend configuration to all services.
+
+**Subtasks:**
+- [ ] T0.5.1: Define `PersistenceOptions` class
+- [ ] T0.5.2: Add configuration section to `appsettings.json`
+- [ ] T0.5.3: Update service registration to read persistence config
+- [ ] T0.5.4: Add configuration validation on startup
+
+**PersistenceOptions:**
+```csharp
+public sealed class PersistenceOptions
+{
+    public const string SectionName = "Persistence";
+
+    public string Authority { get; set; } = "Mongo";
+    public string Scheduler { get; set; } = "Mongo";
+    public string Concelier { get; set; } = "Mongo";
+    public string Excititor { get; set; } = "Mongo";
+    public string Notify { get; set; } = "Mongo";
+    public string Policy { get; set; } = "Mongo";
+}
+```
+
+**Configuration Template:**
+```json
+{
+  "Persistence": {
+    "Authority": "Mongo",
+    "Scheduler": "Mongo",
+    "Concelier": "Mongo",
+    "Excititor": "Mongo",
+    "Notify": "Mongo",
+    "Policy": "Mongo"
+  },
+  "Postgres": {
+    "ConnectionString": "Host=localhost;Database=stellaops;Username=stellaops;Password=secret",
+    "CommandTimeoutSeconds": 30,
+    "ConnectionTimeoutSeconds": 15
+  }
+}
+```
+
+**Verification:**
+- [ ] Configuration loads correctly
+- [ ] Invalid configuration throws on startup
+- [ ] Environment variables can override settings
+
+---
+
+### T0.6: Documentation Review
+
+**Status:** TODO
+**Assignee:** TBD
+**Estimate:** 1 day
+
+**Description:**
+Review and finalize database documentation.
+
+**Subtasks:**
+- [ ] T0.6.1: Review SPECIFICATION.md for completeness
+- [ ] T0.6.2: Review RULES.md for clarity
+- [ ] T0.6.3: Review VERIFICATION.md for test coverage
+- [ ] T0.6.4: Get Architecture Team sign-off
+- [ ] T0.6.5: Publish to team wiki/docs site
+
+**Verification:**
+- [ ] All documents reviewed by 2+ team members
+- [ ] No outstanding questions or TODOs
+- [ ] Architecture Team approval received
+
+---
+
+## Exit Criteria
+
+- [ ] PostgreSQL cluster running and accessible
+- [ ] `StellaOps.Infrastructure.Postgres` library implemented and tested
+- [ ] CI pipeline running PostgreSQL integration tests
+- [ ] Persistence configuration framework in place
+- [ ] Documentation reviewed and approved
+
+---
+
+## Risks & Mitigations
+
+| Risk | Likelihood | Impact | Mitigation |
+|------|------------|--------|------------|
+| PostgreSQL provisioning delays | Medium | High | Start early, have backup plan |
+| Testcontainers compatibility issues | Low | Medium | Test on CI runners early |
+| Configuration complexity | Low | Low | Use existing patterns from Orchestrator |
+
+---
+
+## Dependencies on Later Phases
+
+Phase 0 must complete before any module conversion (Phases 1-6) can begin. The following are required:
+
+1. PostgreSQL cluster operational
+2. Shared library published
+3. CI pipeline validated
+4. Configuration framework deployed
+
+---
+
+## Notes
+
+- Use Orchestrator module as reference for all patterns
+- Prioritize getting CI pipeline working early
+- Document all configuration decisions
+
+---
+
+*Phase Version: 1.0.0*
+*Last Updated: 2025-11-28*