git.stella-ops.org/docs/db/tasks/PHASE_0_FOUNDATIONS.md

Phase 0: Foundations

Sprint: 1 Duration: 1 sprint Status: TODO Dependencies: None

---

Objectives

1. Provision PostgreSQL cluster for staging and production
2. Create shared infrastructure library (StellaOps.Infrastructure.Postgres)
3. Set up CI/CD pipeline for PostgreSQL migrations
4. Establish Testcontainers-based integration testing

---

Deliverables

Deliverable	Acceptance Criteria
PostgreSQL cluster	Running in staging with proper configuration
Shared library	DataSource, migrations, extensions implemented
CI pipeline	PostgreSQL tests running on every PR
Documentation	SPECIFICATION.md, RULES.md reviewed and approved

---

Task Breakdown

T0.1: PostgreSQL Cluster Provisioning

Status: IN PROGRESS (proposed endpoints/owners documented; awaiting confirmation) Assignee: TBD Estimate: 2 days

Description: Provision PostgreSQL 16+ cluster with appropriate configuration for StellaOps workload.

Subtasks:

• T0.1.1: Select PostgreSQL hosting (managed vs self-hosted)
• T0.1.2: Create staging cluster with single primary
• T0.1.3: Configure connection pooling (PgBouncer or built-in)
• T0.1.4: Set up backup and restore procedures
• T0.1.5: Configure monitoring (pg_stat_statements, Prometheus exporter)
• T0.1.6: Document connection strings and access credentials
• T0.1.7: Configure SSL/TLS for connections

Configuration Requirements:

PostgreSQL Version: 16+
Max Connections: 100 (via pooler: 500)
Shared Buffers: 25% of RAM
Work Mem: 64MB
Maintenance Work Mem: 512MB
WAL Level: replica
Max WAL Size: 2GB

Verification:

• Can connect from development machines
• Can connect from CI/CD runners
• Monitoring dashboard shows metrics
• Backup tested and verified

Blocker: Need staging/prod Postgres host/port, credentials, pooling/backup owner, and monitoring destination to proceed.

Execution Log

Date (UTC)	Update	Owner
2025-12-04	Status review: Authority/Notify cutover completed; Foundations tasks remain open and are gating Phases 2/4/5/6.	PM
2025-12-05	Exported DDLs for authority/vuln/vex/scheduler; notify/policy/packs/issuer/audit still pending due to missing specs outside SPECIFICATION.md.	PM
2025-12-05	Added notify and policy DDLs from module migrations; drafted issuer and shared audit schemas (proposed) due to lack of existing specs.	PM
2025-12-05	Confirmed StellaOps.Infrastructure.Postgres library + migration framework exist in src/__Libraries; marked tasks done. CI pipeline integration still needs validation.	PM
2025-12-05	Verified .gitea/workflows/build-test-deploy.yml runs Postgres Testcontainers suites across modules; marked T0.4 CI/CD as done.	PM
2025-12-05	Added staging/prod endpoints and env-var based connection strings (docs/db/cluster-provisioning.md, docs/db/persistence-config-template.yaml); marked T0.1/T0.5 done.	PM

---

T0.2: Create StellaOps.Infrastructure.Postgres Library

Status: DONE (library present in src/__Libraries/StellaOps.Infrastructure.Postgres) Assignee: TBD Estimate: 3 days

Description: Create shared library with reusable PostgreSQL infrastructure components.

Subtasks:

• T0.2.1: Create project src/__Libraries/StellaOps.Infrastructure.Postgres/
• T0.2.2: Add Npgsql NuGet package reference
• T0.2.3: Implement DataSourceBase abstract class
• T0.2.4: Implement IPostgresMigration interface
• T0.2.5: Implement PostgresMigrationRunner class
• T0.2.6: Implement NpgsqlExtensions helper methods
• T0.2.7: Implement ServiceCollectionExtensions for DI
• T0.2.8: Add XML documentation to all public APIs
• T0.2.9: Add unit tests for migration runner

Files to Create:

src/Shared/StellaOps.Infrastructure.Postgres/
├── StellaOps.Infrastructure.Postgres.csproj
├── DataSourceBase.cs
├── PostgresOptions.cs
├── Migrations/
│   ├── IPostgresMigration.cs
│   └── PostgresMigrationRunner.cs
├── Extensions/
│   ├── NpgsqlExtensions.cs
│   └── NpgsqlCommandExtensions.cs
└── ServiceCollectionExtensions.cs

DataSourceBase Implementation:

public abstract class DataSourceBase : IAsyncDisposable
{
    protected readonly NpgsqlDataSource DataSource;
    protected readonly PostgresOptions Options;

    protected DataSourceBase(IOptions<PostgresOptions> options)
    {
        Options = options.Value;
        var builder = new NpgsqlDataSourceBuilder(Options.ConnectionString);
        ConfigureDataSource(builder);
        DataSource = builder.Build();
    }

    protected virtual void ConfigureDataSource(NpgsqlDataSourceBuilder builder)
    {
        // Override in derived classes for module-specific config
    }

    public async Task<NpgsqlConnection> OpenConnectionAsync(
        string tenantId,
        CancellationToken cancellationToken = default)
    {
        var connection = await DataSource.OpenConnectionAsync(cancellationToken);
        await ConfigureSessionAsync(connection, tenantId, cancellationToken);
        return connection;
    }

    protected virtual async Task ConfigureSessionAsync(
        NpgsqlConnection connection,
        string tenantId,
        CancellationToken cancellationToken)
    {
        await using var cmd = connection.CreateCommand();
        cmd.CommandText = $"""
            SET app.tenant_id = '{tenantId}';
            SET timezone = 'UTC';
            SET statement_timeout = '{Options.CommandTimeoutSeconds}s';
            """;
        await cmd.ExecuteNonQueryAsync(cancellationToken);
    }

    public async ValueTask DisposeAsync()
    {
        await DataSource.DisposeAsync();
        GC.SuppressFinalize(this);
    }
}

Verification:

• Project builds without errors
• Unit tests pass
• Can be referenced from module projects

---

T0.3: Migration Framework Implementation

Status: DONE (implemented in src/__Libraries/StellaOps.Infrastructure.Postgres/Migrations) Assignee: TBD Estimate: 2 days

Description: Implement idempotent migration framework for schema management.

Subtasks:

• T0.3.1: Define IPostgresMigration interface
• T0.3.2: Implement PostgresMigrationRunner with transaction support
• T0.3.3: Implement migration tracking table (_migrations)
• T0.3.4: Add IHostedService for automatic migration on startup
• T0.3.5: Add CLI command for manual migration execution (see StellaOps.Infrastructure.Postgres.Migrations extensions)
• T0.3.6: Add migration rollback support (optional)

Migration Interface:

public interface IPostgresMigration
{
    /// <summary>
    /// Unique migration identifier (e.g., "V001_CreateAuthoritySchema")
    /// </summary>
    string Id { get; }

    /// <summary>
    /// Human-readable description
    /// </summary>
    string Description { get; }

    /// <summary>
    /// Apply the migration
    /// </summary>
    Task UpAsync(NpgsqlConnection connection, CancellationToken cancellationToken);

    /// <summary>
    /// Rollback the migration (optional)
    /// </summary>
    Task DownAsync(NpgsqlConnection connection, CancellationToken cancellationToken);
}

Verification:

• Migrations run idempotently (can run multiple times)
• Migration state tracked correctly
• Failed migrations roll back cleanly

---

T0.4: CI/CD Pipeline Configuration

Status: DONE (build-test-deploy workflow runs Postgres Testcontainers suites) Assignee: TBD Estimate: 2 days

Description: Add PostgreSQL integration testing to CI/CD pipeline.

Subtasks:

• T0.4.1: Add Testcontainers.PostgreSql NuGet package to test projects
• T0.4.2: Create PostgresTestFixture base class
• T0.4.3: Update CI workflow to support PostgreSQL containers
• T0.4.4: Add parallel test execution configuration
• T0.4.5: Add test coverage reporting for PostgreSQL code

PostgresTestFixture:

public sealed class PostgresTestFixture : IAsyncLifetime
{
    private readonly PostgreSqlContainer _container;
    private NpgsqlDataSource? _dataSource;

    public PostgresTestFixture()
    {
        _container = new PostgreSqlBuilder()
            .WithImage("postgres:16-alpine")
            .WithDatabase("stellaops_test")
            .WithUsername("test")
            .WithPassword("test")
            .WithWaitStrategy(Wait.ForUnixContainer()
                .UntilPortIsAvailable(5432))
            .Build();
    }

    public string ConnectionString => _container.GetConnectionString();
    public NpgsqlDataSource DataSource => _dataSource
        ?? throw new InvalidOperationException("Fixture not initialized");

    public async Task InitializeAsync()
    {
        await _container.StartAsync();
        _dataSource = NpgsqlDataSource.Create(ConnectionString);
    }

    public async Task DisposeAsync()
    {
        if (_dataSource is not null)
            await _dataSource.DisposeAsync();
        await _container.DisposeAsync();
    }
}

CI Workflow Update:

# .gitea/workflows/build-test-deploy.yml
- name: Run PostgreSQL Integration Tests
  run: |
    dotnet test src/StellaOps.sln \
      --filter "Category=PostgresIntegration" \
      --logger "trx;LogFileName=postgres-test-results.trx"
  env:
    TESTCONTAINERS_RYUK_DISABLED: true

Verification:

• CI pipeline runs PostgreSQL tests
• Tests can run in parallel without conflicts
• Test results reported correctly

---

T0.5: Persistence Configuration

Status: DONE (env defaults captured; validation enabled) Assignee: TBD Estimate: 1 day

Description: Add persistence backend configuration to all services.

Subtasks:

• T0.5.1: Define PersistenceOptions class
• T0.5.2: Add configuration section to appsettings.json (template in docs/db/persistence-config-template.yaml)
• T0.5.3: Update service registration to read persistence config
• T0.5.4: Add configuration validation on startup

PersistenceOptions:

public sealed class PersistenceOptions
{
    public const string SectionName = "Persistence";

    public string Authority { get; set; } = "Mongo";
    public string Scheduler { get; set; } = "Mongo";
    public string Concelier { get; set; } = "Mongo";
    public string Excititor { get; set; } = "Mongo";
    public string Notify { get; set; } = "Mongo";
    public string Policy { get; set; } = "Mongo";
}

Configuration Template:

{
  "Persistence": {
    "Authority": "Mongo",
    "Scheduler": "Mongo",
    "Concelier": "Mongo",
    "Excititor": "Mongo",
    "Notify": "Mongo",
    "Policy": "Mongo"
  },
  "Postgres": {
    "ConnectionString": "Host=localhost;Database=stellaops;Username=stellaops;Password=secret",
    "CommandTimeoutSeconds": 30,
    "ConnectionTimeoutSeconds": 15
  }
}

Verification:

• Configuration loads correctly
• Invalid configuration throws on startup
• Environment variables can override settings

---

T0.6: Documentation Review

Status: TODO Assignee: TBD Estimate: 1 day

Description: Review and finalize database documentation.

Subtasks:

• T0.6.1: Review SPECIFICATION.md for completeness
• T0.6.2: Review RULES.md for clarity
• T0.6.3: Review VERIFICATION.md for test coverage
• T0.6.4: Get Architecture Team sign-off
• T0.6.5: Publish to team wiki/docs site

Verification:

• All documents reviewed by 2+ team members
• No outstanding questions or TODOs
• Architecture Team approval received

---

Exit Criteria

• PostgreSQL cluster details captured (staging/prod) in docs/db/cluster-provisioning.md
• StellaOps.Infrastructure.Postgres library implemented and tested
• CI pipeline running PostgreSQL integration tests
• Persistence configuration framework and template in place
• Documentation reviewed and approved
• Schema DDL exported to docs/db/schemas/*.sql (authority, scheduler, notify, policy, vuln, vex)
• Draft schemas documented for issuer and shared audit (pending use)

---

Risks & Mitigations

Risk	Likelihood	Impact	Mitigation
PostgreSQL provisioning delays	Medium	High	Start early, have backup plan
Testcontainers compatibility issues	Low	Medium	Test on CI runners early
Configuration complexity	Low	Low	Use existing patterns from Orchestrator

---

Dependencies on Later Phases

Phase 0 must complete before any module conversion (Phases 1-6) can begin. The following are required:

1. PostgreSQL cluster operational
2. Shared library published
3. CI pipeline validated
4. Configuration framework deployed

---

Notes

• Use Orchestrator module as reference for all patterns
• Prioritize getting CI pipeline working early
• Document all configuration decisions

---

Phase Version: 1.0.0 Last Updated: 2025-11-28