stella-ops.org/git.stella-ops.org
1
0
Fork 0
Code Issues Pull Requests Actions 3 Releases Wiki Activity

Add integration tests for migration categories and execution
Some checks failed
Docs CI / lint-and-preview (push) Has been cancelled
Details
Policy Lint & Smoke / policy-lint (push) Has been cancelled
Details
Concelier Attestation Tests / attestation-tests (push) Has been cancelled
Details
AOC Guard CI / aoc-guard (push) Has been cancelled
Details
AOC Guard CI / aoc-verify (push) Has been cancelled
Details

Browse Source 
- Implemented MigrationCategoryTests to validate migration categorization for startup, release, seed, and data migrations.
- Added tests for edge cases, including null, empty, and whitespace migration names.
- Created StartupMigrationHostTests to verify the behavior of the migration host with real PostgreSQL instances using Testcontainers.
- Included tests for migration execution, schema creation, and handling of pending release migrations.
- Added SQL migration files for testing: creating a test table, adding a column, a release migration, and seeding data.
This commit is contained in:
master
2025-12-04 19:10:54 +02:00
parent 600f3a7a3c
commit 75f6942769
301 changed files with 32810 additions and 1128 deletions
Download Patch File Download Diff File Expand all files Collapse all files

714
docs/router/19-Step.md Normal file
View File

@@ -0,0 +1,714 @@
# Step 19: Microservice Host Builder
**Phase 5: Microservice SDK**
**Estimated Complexity:** High
**Dependencies:** Step 14 (TCP Transport), Step 15 (TLS Transport)
---
## Overview
The Microservice Host Builder provides a fluent API for building microservices that connect to the Stella Router. It handles transport configuration, endpoint registration, graceful shutdown, and integration with ASP.NET Core's hosting infrastructure.
---
## Goals
1. Provide fluent builder API for microservice configuration
2. Support both standalone and ASP.NET Core integrated hosting
3. Handle transport lifecycle (connect, reconnect, disconnect)
4. Support multiple transport configurations
5. Enable dual-exposure mode (gateway + direct HTTP)
---
## Core Architecture
```
┌────────────────────────────────────────────────────────────────┐
│ Microservice Host Builder │
├────────────────────────────────────────────────────────────────┤
│ │
│ ┌─────────────────────────────────────────────────────────┐ │
│ │ StellaMicroserviceHost │ │
│ │ ┌───────────────┐ ┌───────────────┐ ┌─────────────┐ │ │
│ │ │Transport Layer│ │Endpoint Registry│ │ Request │ │ │
│ │ │ (TCP/TLS/etc) │ │(Discovery/Reg) │ │ Dispatcher │ │ │
│ │ └───────────────┘ └───────────────┘ └─────────────┘ │ │
│ └─────────────────────────────────────────────────────────┘ │
│ │
│ ┌─────────────────────────────────────────────────────────┐ │
│ │ Optional: ASP.NET Core Host │ │
│ │ (Kestrel for direct HTTP access + default claims) │ │
│ └─────────────────────────────────────────────────────────┘ │
│ │
└────────────────────────────────────────────────────────────────┘
```
---
## Configuration
```csharp
namespace StellaOps.Microservice;
public class StellaMicroserviceOptions
{
/// <summary>Service name for registration.</summary>
public required string ServiceName { get; set; }
/// <summary>Unique instance identifier (auto-generated if not set).</summary>
public string InstanceId { get; set; } = Guid.NewGuid().ToString("N")[..8];
/// <summary>Service version for routing.</summary>
public string Version { get; set; } = "1.0.0";
/// <summary>Region for routing affinity.</summary>
public string? Region { get; set; }
/// <summary>Tags for routing metadata.</summary>
public Dictionary<string, string> Tags { get; set; } = new();
/// <summary>Router connection pool.</summary>
public List<RouterConnectionConfig> Routers { get; set; } = new();
/// <summary>Transport configuration.</summary>
public TransportConfig Transport { get; set; } = new();
/// <summary>Endpoint discovery configuration.</summary>
public EndpointDiscoveryConfig Discovery { get; set; } = new();
/// <summary>Heartbeat configuration.</summary>
public HeartbeatConfig Heartbeat { get; set; } = new();
/// <summary>Dual exposure mode configuration.</summary>
public DualExposureConfig? DualExposure { get; set; }
/// <summary>Graceful shutdown timeout.</summary>
public TimeSpan ShutdownTimeout { get; set; } = TimeSpan.FromSeconds(30);
}
public class RouterConnectionConfig
{
public string Host { get; set; } = "localhost";
public int Port { get; set; } = 9500;
public string Transport { get; set; } = "TCP"; // TCP, TLS, InMemory
public int Priority { get; set; } = 1;
public bool Enabled { get; set; } = true;
}
public class TransportConfig
{
public string Default { get; set; } = "TCP";
public TcpClientConfig? Tcp { get; set; }
public TlsClientConfig? Tls { get; set; }
public int MaxReconnectAttempts { get; set; } = -1; // -1 = unlimited
public TimeSpan ReconnectDelay { get; set; } = TimeSpan.FromSeconds(5);
}
public class EndpointDiscoveryConfig
{
/// <summary>Assemblies to scan for endpoints.</summary>
public List<string> ScanAssemblies { get; set; } = new();
/// <summary>Path to YAML overrides file.</summary>
public string? ConfigFilePath { get; set; }
/// <summary>Base path prefix for all endpoints.</summary>
public string? BasePath { get; set; }
/// <summary>Whether to auto-discover endpoints via reflection.</summary>
public bool AutoDiscover { get; set; } = true;
}
public class HeartbeatConfig
{
public TimeSpan Interval { get; set; } = TimeSpan.FromSeconds(10);
public TimeSpan Timeout { get; set; } = TimeSpan.FromSeconds(5);
public int MissedHeartbeatsThreshold { get; set; } = 3;
}
public class DualExposureConfig
{
/// <summary>Enable direct HTTP access.</summary>
public bool Enabled { get; set; } = false;
/// <summary>HTTP port for direct access.</summary>
public int HttpPort { get; set; } = 8080;
/// <summary>Default claims for direct access (no JWT).</summary>
public Dictionary<string, string> DefaultClaims { get; set; } = new();
/// <summary>Whether to require JWT for direct access.</summary>
public bool RequireAuthentication { get; set; } = false;
}
```
---
## Host Builder Implementation
```csharp
namespace StellaOps.Microservice;
public interface IStellaMicroserviceBuilder
{
IStellaMicroserviceBuilder ConfigureServices(Action<IServiceCollection> configure);
IStellaMicroserviceBuilder ConfigureTransport(Action<TransportConfig> configure);
IStellaMicroserviceBuilder ConfigureEndpoints(Action<EndpointDiscoveryConfig> configure);
IStellaMicroserviceBuilder AddRouter(string host, int port, string transport = "TCP");
IStellaMicroserviceBuilder EnableDualExposure(Action<DualExposureConfig>? configure = null);
IStellaMicroserviceBuilder UseYamlConfig(string path);
IStellaMicroserviceHost Build();
}
public sealed class StellaMicroserviceBuilder : IStellaMicroserviceBuilder
{
private readonly StellaMicroserviceOptions _options;
private readonly IServiceCollection _services;
private readonly List<Action<IServiceCollection>> _configureActions = new();
public StellaMicroserviceBuilder(string serviceName)
{
_options = new StellaMicroserviceOptions { ServiceName = serviceName };
_services = new ServiceCollection();
// Add default services
_services.AddLogging(b => b.AddConsole());
_services.AddSingleton(_options);
}
public static IStellaMicroserviceBuilder Create(string serviceName)
{
return new StellaMicroserviceBuilder(serviceName);
}
public IStellaMicroserviceBuilder ConfigureServices(Action<IServiceCollection> configure)
{
_configureActions.Add(configure);
return this;
}
public IStellaMicroserviceBuilder ConfigureTransport(Action<TransportConfig> configure)
{
configure(_options.Transport);
return this;
}
public IStellaMicroserviceBuilder ConfigureEndpoints(Action<EndpointDiscoveryConfig> configure)
{
configure(_options.Discovery);
return this;
}
public IStellaMicroserviceBuilder AddRouter(string host, int port, string transport = "TCP")
{
_options.Routers.Add(new RouterConnectionConfig
{
Host = host,
Port = port,
Transport = transport,
Priority = _options.Routers.Count + 1
});
return this;
}
public IStellaMicroserviceBuilder EnableDualExposure(Action<DualExposureConfig>? configure = null)
{
_options.DualExposure = new DualExposureConfig { Enabled = true };
configure?.Invoke(_options.DualExposure);
return this;
}
public IStellaMicroserviceBuilder UseYamlConfig(string path)
{
_options.Discovery.ConfigFilePath = path;
return this;
}
public IStellaMicroserviceHost Build()
{
// Apply custom service configuration
foreach (var action in _configureActions)
{
action(_services);
}
// Add core services
AddCoreServices();
// Add transport services
AddTransportServices();
// Add endpoint services
AddEndpointServices();
var serviceProvider = _services.BuildServiceProvider();
return serviceProvider.GetRequiredService<IStellaMicroserviceHost>();
}
private void AddCoreServices()
{
_services.AddSingleton<IStellaMicroserviceHost, StellaMicroserviceHost>();
_services.AddSingleton<IEndpointRegistry, EndpointRegistry>();
_services.AddSingleton<IRequestDispatcher, RequestDispatcher>();
_services.AddSingleton<IPayloadSerializer, MessagePackPayloadSerializer>();
}
private void AddTransportServices()
{
_services.AddSingleton<TcpFrameCodec>();
switch (_options.Transport.Default.ToUpper())
{
case "TCP":
_services.AddSingleton<ITransportServer, TcpTransportClient>();
break;
case "TLS":
_services.AddSingleton<ICertificateProvider, CertificateProvider>();
_services.AddSingleton<ITransportServer, TlsTransportClient>();
break;
case "INMEMORY":
// InMemory requires hub to be provided externally
_services.AddSingleton<ITransportServer, InMemoryTransportServer>();
break;
}
}
private void AddEndpointServices()
{
_services.AddSingleton<IEndpointDiscovery, ReflectionEndpointDiscovery>();
if (!string.IsNullOrEmpty(_options.Discovery.ConfigFilePath))
{
_services.AddSingleton<IEndpointOverrideProvider, YamlEndpointOverrideProvider>();
}
}
}
```
---
## Microservice Host Implementation
```csharp
namespace StellaOps.Microservice;
public interface IStellaMicroserviceHost : IAsyncDisposable
{
StellaMicroserviceOptions Options { get; }
bool IsConnected { get; }
Task StartAsync(CancellationToken cancellationToken = default);
Task StopAsync(CancellationToken cancellationToken = default);
Task WaitForShutdownAsync(CancellationToken cancellationToken = default);
}
public sealed class StellaMicroserviceHost : IStellaMicroserviceHost, IHostedService
{
private readonly StellaMicroserviceOptions _options;
private readonly ITransportServer _transport;
private readonly IEndpointRegistry _endpointRegistry;
private readonly IRequestDispatcher _dispatcher;
private readonly ILogger<StellaMicroserviceHost> _logger;
private readonly CancellationTokenSource _shutdownCts = new();
private readonly TaskCompletionSource _shutdownComplete = new();
private Timer? _heartbeatTimer;
private IHost? _httpHost;
public StellaMicroserviceOptions Options => _options;
public bool IsConnected => _transport.IsConnected;
public StellaMicroserviceHost(
StellaMicroserviceOptions options,
ITransportServer transport,
IEndpointRegistry endpointRegistry,
IRequestDispatcher dispatcher,
ILogger<StellaMicroserviceHost> logger)
{
_options = options;
_transport = transport;
_endpointRegistry = endpointRegistry;
_dispatcher = dispatcher;
_logger = logger;
}
public async Task StartAsync(CancellationToken cancellationToken = default)
{
_logger.LogInformation(
"Starting microservice {ServiceName}/{InstanceId}",
_options.ServiceName, _options.InstanceId);
// Discover endpoints
var endpoints = await _endpointRegistry.DiscoverEndpointsAsync(cancellationToken);
_logger.LogInformation("Discovered {Count} endpoints", endpoints.Length);
// Wire up request handler
_transport.OnRequest += HandleRequestAsync;
_transport.OnCancel += HandleCancelAsync;
// Connect to router
var router = _options.Routers.OrderBy(r => r.Priority).FirstOrDefault()
?? throw new InvalidOperationException("No routers configured");
await _transport.ConnectAsync(
_options.ServiceName,
_options.InstanceId,
endpoints,
cancellationToken);
_logger.LogInformation(
"Connected to router at {Host}:{Port}",
router.Host, router.Port);
// Start heartbeat
_heartbeatTimer = new Timer(
SendHeartbeatAsync,
null,
_options.Heartbeat.Interval,
_options.Heartbeat.Interval);
// Start dual exposure HTTP if enabled
if (_options.DualExposure?.Enabled == true)
{
await StartHttpHostAsync(cancellationToken);
}
_logger.LogInformation(
"Microservice {ServiceName} started successfully",
_options.ServiceName);
}
private async Task<ResponsePayload> HandleRequestAsync(
RequestPayload request,
CancellationToken cancellationToken)
{
using var activity = Activity.StartActivity("HandleRequest");
activity?.SetTag("http.method", request.Method);
activity?.SetTag("http.path", request.Path);
try
{
return await _dispatcher.DispatchAsync(request, cancellationToken);
}
catch (Exception ex)
{
_logger.LogError(ex, "Error handling request {Path}", request.Path);
return new ResponsePayload
{
StatusCode = 500,
Headers = new Dictionary<string, string>(),
Body = Encoding.UTF8.GetBytes($"{{\"error\": \"{ex.Message}\"}}"),
IsFinalChunk = true
};
}
}
private Task HandleCancelAsync(string correlationId, CancellationToken cancellationToken)
{
_logger.LogDebug("Request {CorrelationId} cancelled", correlationId);
// Propagate cancellation to active request handling
return Task.CompletedTask;
}
private async void SendHeartbeatAsync(object? state)
{
try
{
await _transport.SendHeartbeatAsync(_shutdownCts.Token);
}
catch (Exception ex)
{
_logger.LogWarning(ex, "Failed to send heartbeat");
}
}
private async Task StartHttpHostAsync(CancellationToken cancellationToken)
{
var config = _options.DualExposure!;
_httpHost = Host.CreateDefaultBuilder()
.ConfigureWebHostDefaults(web =>
{
web.UseKestrel(k => k.ListenAnyIP(config.HttpPort));
web.Configure(app =>
{
app.UseRouting();
app.UseEndpoints(endpoints =>
{
endpoints.MapFallback(async context =>
{
// Inject default claims for direct access
var claims = config.DefaultClaims;
var request = new RequestPayload
{
Method = context.Request.Method,
Path = context.Request.Path + context.Request.QueryString,
Host = context.Request.Host.Value,
Headers = context.Request.Headers
.ToDictionary(h => h.Key, h => h.Value.ToString()),
Claims = claims,
ClientIp = context.Connection.RemoteIpAddress?.ToString(),
TraceId = context.TraceIdentifier
};
// Read body if present
if (context.Request.ContentLength > 0)
{
using var ms = new MemoryStream();
await context.Request.Body.CopyToAsync(ms);
request = request with { Body = ms.ToArray() };
}
var response = await _dispatcher.DispatchAsync(request, context.RequestAborted);
context.Response.StatusCode = response.StatusCode;
foreach (var (key, value) in response.Headers)
{
context.Response.Headers[key] = value;
}
if (response.Body != null)
{
await context.Response.Body.WriteAsync(response.Body);
}
});
});
});
})
.Build();
await _httpHost.StartAsync(cancellationToken);
_logger.LogInformation(
"Direct HTTP access enabled on port {Port}",
config.HttpPort);
}
public async Task StopAsync(CancellationToken cancellationToken = default)
{
_logger.LogInformation(
"Stopping microservice {ServiceName}",
_options.ServiceName);
_shutdownCts.Cancel();
_heartbeatTimer?.Dispose();
if (_httpHost != null)
{
await _httpHost.StopAsync(cancellationToken);
}
await _transport.DisconnectAsync();
_logger.LogInformation(
"Microservice {ServiceName} stopped",
_options.ServiceName);
_shutdownComplete.TrySetResult();
}
public Task WaitForShutdownAsync(CancellationToken cancellationToken = default)
{
return _shutdownComplete.Task.WaitAsync(cancellationToken);
}
public async ValueTask DisposeAsync()
{
await StopAsync();
_shutdownCts.Dispose();
}
// IHostedService implementation for ASP.NET Core integration
Task IHostedService.StartAsync(CancellationToken cancellationToken) => StartAsync(cancellationToken);
Task IHostedService.StopAsync(CancellationToken cancellationToken) => StopAsync(cancellationToken);
}
```
---
## ASP.NET Core Integration
```csharp
namespace StellaOps.Microservice;
public static class StellaMicroserviceExtensions
{
/// <summary>
/// Adds Stella microservice to an existing ASP.NET Core host.
/// </summary>
public static IServiceCollection AddStellaMicroservice(
this IServiceCollection services,
Action<StellaMicroserviceOptions> configure)
{
var options = new StellaMicroserviceOptions { ServiceName = "unknown" };
configure(options);
services.AddSingleton(options);
services.AddSingleton<IEndpointRegistry, EndpointRegistry>();
services.AddSingleton<IRequestDispatcher, RequestDispatcher>();
services.AddSingleton<IPayloadSerializer, MessagePackPayloadSerializer>();
services.AddSingleton<TcpFrameCodec>();
// Add transport based on configuration
switch (options.Transport.Default.ToUpper())
{
case "TCP":
services.AddSingleton<ITransportServer, TcpTransportClient>();
break;
case "TLS":
services.AddSingleton<ICertificateProvider, CertificateProvider>();
services.AddSingleton<ITransportServer, TlsTransportClient>();
break;
}
services.AddSingleton<IStellaMicroserviceHost, StellaMicroserviceHost>();
services.AddHostedService(sp => (StellaMicroserviceHost)sp.GetRequiredService<IStellaMicroserviceHost>());
return services;
}
/// <summary>
/// Configures an endpoint handler for the microservice.
/// </summary>
public static IServiceCollection AddEndpointHandler<THandler>(
this IServiceCollection services)
where THandler : class, IEndpointHandler
{
services.AddScoped<IEndpointHandler, THandler>();
return services;
}
}
```
---
## Usage Examples
### Standalone Microservice
```csharp
var host = StellaMicroserviceBuilder
.Create("billing-service")
.AddRouter("gateway.internal", 9500, "TLS")
.ConfigureTransport(t =>
{
t.Tls = new TlsClientConfig
{
ClientCertificatePath = "/etc/certs/billing.pfx",
ClientCertificatePassword = Environment.GetEnvironmentVariable("CERT_PASSWORD")
};
})
.ConfigureEndpoints(e =>
{
e.BasePath = "/billing";
e.ScanAssemblies.Add("BillingService.Handlers");
})
.ConfigureServices(services =>
{
services.AddScoped<BillingContext>();
services.AddScoped<InvoiceHandler>();
})
.Build();
await host.StartAsync();
await host.WaitForShutdownAsync();
```
### ASP.NET Core Integration
```csharp
var builder = WebApplication.CreateBuilder(args);
builder.Services.AddStellaMicroservice(options =>
{
options.ServiceName = "user-service";
options.Region = "us-east-1";
options.Routers.Add(new RouterConnectionConfig
{
Host = "gateway.internal",
Port = 9500
});
options.DualExposure = new DualExposureConfig
{
Enabled = true,
HttpPort = 8080,
DefaultClaims = new Dictionary<string, string>
{
["tier"] = "free"
}
};
});
builder.Services.AddEndpointHandler<UserEndpointHandler>();
var app = builder.Build();
await app.RunAsync();
```
---
## YAML Configuration
```yaml
Microservice:
ServiceName: "billing-service"
Version: "1.0.0"
Region: "us-east-1"
Tags:
team: "payments"
tier: "critical"
Routers:
- Host: "gateway-primary.internal"
Port: 9500
Transport: "TLS"
Priority: 1
- Host: "gateway-secondary.internal"
Port: 9500
Transport: "TLS"
Priority: 2
Transport:
Default: "TLS"
Tls:
ClientCertificatePath: "/etc/certs/service.pfx"
ClientCertificatePassword: "${CERT_PASSWORD}"
Discovery:
AutoDiscover: true
BasePath: "/billing"
ConfigFilePath: "/etc/stellaops/endpoints.yaml"
Heartbeat:
Interval: "00:00:10"
Timeout: "00:00:05"
DualExposure:
Enabled: true
HttpPort: 8080
DefaultClaims:
tier: "free"
ShutdownTimeout: "00:00:30"
```
---
## Deliverables
1. `StellaOps.Microservice/StellaMicroserviceOptions.cs`
2. `StellaOps.Microservice/IStellaMicroserviceBuilder.cs`
3. `StellaOps.Microservice/StellaMicroserviceBuilder.cs`
4. `StellaOps.Microservice/IStellaMicroserviceHost.cs`
5. `StellaOps.Microservice/StellaMicroserviceHost.cs`
6. `StellaOps.Microservice/StellaMicroserviceExtensions.cs`
7. Builder pattern tests
8. Lifecycle tests (start/stop/reconnect)
9. Dual exposure mode tests
---
## Next Step
Proceed to [Step 20: Endpoint Discovery & Registration](20-Step.md) to implement automatic endpoint discovery.