Fix build and code structure improvements. New but essential UI functionality. CI improvements. Documentation improvements. AI module improvements.

This commit is contained in:
StellaOps Bot
2025-12-26 21:54:17 +02:00
parent 335ff7da16
commit c2b9cd8d1f
3717 changed files with 264714 additions and 48202 deletions

View File

@@ -0,0 +1,203 @@
# Router Transport Plugins
StellaOps Router uses a **plugin-based transport architecture** that enables runtime loading of transport implementations. This allows operators to deploy only the transports they need and swap implementations without recompiling the Gateway.
## Available Transports
| Transport | Plugin Assembly | Use Case | Status |
|-----------|-----------------|----------|--------|
| [TCP](./tcp.md) | `StellaOps.Router.Transport.Tcp.dll` | Internal services, same datacenter | Stable |
| [TLS](./tls.md) | `StellaOps.Router.Transport.Tls.dll` | Cross-datacenter, mTLS | Stable |
| [UDP](./udp.md) | `StellaOps.Router.Transport.Udp.dll` | Fire-and-forget, broadcast | Stable |
| [RabbitMQ](./rabbitmq.md) | `StellaOps.Router.Transport.RabbitMq.dll` | Async processing, fan-out | Stable |
| [InMemory](./inmemory.md) | `StellaOps.Router.Transport.InMemory.dll` | Development, testing | Stable |
| Valkey | `StellaOps.Messaging.Transport.Valkey.dll` | Distributed, pub/sub | Stable |
| PostgreSQL | `StellaOps.Messaging.Transport.Postgres.dll` | Transactional, LISTEN/NOTIFY | Stable |
## Plugin Architecture
### Loading Model
Transport plugins are loaded at Gateway startup via `RouterTransportPluginLoader`:
```
┌─────────────────────────────────────────────────────────────────┐
│ Gateway Startup │
│ │
│ ┌────────────────────────────────────────────────────────────┐ │
│ │ RouterTransportPluginLoader │ │
│ │ │ │
│ │ 1. Scan plugins/router/transports/ │ │
│ │ 2. Load assemblies in isolation (AssemblyLoadContext) │ │
│ │ 3. Discover IRouterTransportPlugin implementations │ │
│ │ 4. Register configured transport with DI │ │
│ └────────────────────────────────────────────────────────────┘ │
│ │ │
│ ▼ │
│ ┌────────────────────────────────────────────────────────────┐ │
│ │ Transport Plugin (e.g., TLS) │ │
│ │ │ │
│ │ - TransportName: "tls" │ │
│ │ - DisplayName: "TLS Transport" │ │
│ │ - IsAvailable(): Check dependencies │ │
│ │ - Register(): Wire up ITransportServer/ITransportClient │ │
│ └────────────────────────────────────────────────────────────┘ │
│ │
└──────────────────────────────────────────────────────────────────┘
```
### Directory Structure
```
plugins/
└── router/
└── transports/
├── StellaOps.Router.Transport.Tcp.dll
├── StellaOps.Router.Transport.Tls.dll
├── StellaOps.Router.Transport.Udp.dll
├── StellaOps.Router.Transport.RabbitMq.dll
└── StellaOps.Router.Transport.InMemory.dll
```
### Configuration
Transport selection and options are configured in `router.yaml` or environment variables:
```yaml
Router:
Transport:
Type: tls # Which transport plugin to use
Tls: # Transport-specific options
Port: 5101
CertificatePath: /certs/server.pfx
RequireClientCertificate: true
Tcp:
Port: 5100
MaxConnections: 1000
```
Environment override:
```bash
ROUTER__TRANSPORT__TYPE=tcp
ROUTER__TRANSPORT__TCP__PORT=5100
```
## Using Plugins in Gateway
### Programmatic Loading
```csharp
using StellaOps.Router.Common.Plugins;
var builder = WebApplication.CreateBuilder(args);
// Load transport plugins from directory
var pluginLoader = new RouterTransportPluginLoader(
builder.Services.BuildServiceProvider().GetService<ILogger<RouterTransportPluginLoader>>());
var pluginsPath = Path.Combine(AppContext.BaseDirectory, "plugins", "router", "transports");
pluginLoader.LoadFromDirectory(pluginsPath);
// Register the configured transport (reads Router:Transport:Type from config)
pluginLoader.RegisterConfiguredTransport(
builder.Services,
builder.Configuration,
RouterTransportMode.Both); // Register both server and client
var app = builder.Build();
// ...
```
### Gateway Integration
The Gateway automatically loads transport plugins during startup. Configure in `router.yaml`:
```yaml
gateway:
name: api-gateway
plugins:
transports:
directory: plugins/router/transports
searchPattern: "StellaOps.Router.Transport.*.dll"
```
## Creating Custom Transports
See the [Transport Plugin Development Guide](./development.md) for creating custom transport implementations.
### Minimal Plugin Example
```csharp
using StellaOps.Router.Common.Plugins;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.DependencyInjection;
namespace MyCompany.Router.Transport.Custom;
public sealed class CustomTransportPlugin : IRouterTransportPlugin
{
public string TransportName => "custom";
public string DisplayName => "Custom Transport";
public bool IsAvailable(IServiceProvider services) => true;
public void Register(RouterTransportRegistrationContext context)
{
var configSection = context.Configuration.GetSection("Router:Transport:Custom");
context.Services.Configure<CustomTransportOptions>(options =>
{
configSection.Bind(options);
});
if (context.Mode.HasFlag(RouterTransportMode.Server))
{
context.Services.AddSingleton<CustomTransportServer>();
context.Services.AddSingleton<ITransportServer>(sp =>
sp.GetRequiredService<CustomTransportServer>());
}
if (context.Mode.HasFlag(RouterTransportMode.Client))
{
context.Services.AddSingleton<CustomTransportClient>();
context.Services.AddSingleton<ITransportClient>(sp =>
sp.GetRequiredService<CustomTransportClient>());
context.Services.AddSingleton<IMicroserviceTransport>(sp =>
sp.GetRequiredService<CustomTransportClient>());
}
}
}
```
## Transport Selection Guide
| Scenario | Recommended Transport | Configuration |
|----------|----------------------|---------------|
| Development/Testing | InMemory | `Type: inmemory` |
| Same-datacenter | TCP | `Type: tcp` |
| Cross-datacenter secure | TLS | `Type: tls` with mTLS |
| High-volume async | RabbitMQ | `Type: rabbitmq` |
| Broadcast/fire-and-forget | UDP | `Type: udp` |
| Distributed with replay | Valkey | Via Messaging plugins |
| Transactional messaging | PostgreSQL | Via Messaging plugins |
## Air-Gap Deployment
For offline/air-gapped deployments:
1. Pre-package transport plugins with your deployment
2. Configure the plugin directory path
3. No external network access required
```yaml
gateway:
plugins:
transports:
directory: /opt/stellaops/plugins/router/transports
```
## See Also
- [Router Architecture](../ARCHITECTURE.md)
- [Plugin SDK Guide](../../10_PLUGIN_SDK_GUIDE.md)
- [Unified Plugin System](../../plugins/README.md)

View File

@@ -0,0 +1,534 @@
# Transport Plugin Development Guide
This guide explains how to create custom router transport plugins for StellaOps.
## Overview
Router transport plugins implement the `IRouterTransportPlugin` interface to provide custom communication protocols. The plugin system enables:
- Runtime loading of transport implementations
- Isolation from the Gateway codebase
- Hot-swappable transports (restart required)
- Third-party transport extensions
## Prerequisites
- .NET 10 SDK
- Understanding of async socket programming
- Familiarity with dependency injection
## Creating a Transport Plugin
### Step 1: Create Project
```bash
mkdir MyCompany.Router.Transport.Custom
cd MyCompany.Router.Transport.Custom
dotnet new classlib -f net10.0
dotnet add package Microsoft.Extensions.Configuration.Binder
dotnet add package Microsoft.Extensions.DependencyInjection.Abstractions
dotnet add package Microsoft.Extensions.Logging.Abstractions
dotnet add package Microsoft.Extensions.Options
```
Add project reference to Router.Common:
```xml
<ItemGroup>
<ProjectReference Include="path/to/StellaOps.Router.Common/StellaOps.Router.Common.csproj" />
</ItemGroup>
```
### Step 2: Create Options Class
```csharp
namespace MyCompany.Router.Transport.Custom;
/// <summary>
/// Configuration options for the custom transport.
/// </summary>
public sealed class CustomTransportOptions
{
/// <summary>
/// Host address to bind/connect to.
/// </summary>
public string Host { get; set; } = "0.0.0.0";
/// <summary>
/// Port number.
/// </summary>
public int Port { get; set; } = 5200;
/// <summary>
/// Connection timeout.
/// </summary>
public TimeSpan ConnectTimeout { get; set; } = TimeSpan.FromSeconds(30);
/// <summary>
/// Maximum concurrent connections.
/// </summary>
public int MaxConnections { get; set; } = 1000;
}
```
### Step 3: Implement Transport Server
```csharp
using Microsoft.Extensions.Logging;
using Microsoft.Extensions.Options;
using StellaOps.Router.Common.Abstractions;
namespace MyCompany.Router.Transport.Custom;
public sealed class CustomTransportServer : ITransportServer
{
private readonly CustomTransportOptions _options;
private readonly ILogger<CustomTransportServer> _logger;
public CustomTransportServer(
IOptions<CustomTransportOptions> options,
ILogger<CustomTransportServer> logger)
{
_options = options.Value;
_logger = logger;
}
public async Task StartAsync(CancellationToken cancellationToken)
{
_logger.LogInformation("Starting custom transport server on {Host}:{Port}",
_options.Host, _options.Port);
// Initialize your transport server (socket, listener, etc.)
// ...
}
public async Task StopAsync(CancellationToken cancellationToken)
{
_logger.LogInformation("Stopping custom transport server");
// Graceful shutdown
// ...
}
public async Task<ITransportConnection> AcceptAsync(CancellationToken cancellationToken)
{
// Accept incoming connection
// Return ITransportConnection implementation
throw new NotImplementedException();
}
public void Dispose()
{
// Cleanup resources
}
}
```
### Step 4: Implement Transport Client
```csharp
using Microsoft.Extensions.Logging;
using Microsoft.Extensions.Options;
using StellaOps.Router.Common.Abstractions;
namespace MyCompany.Router.Transport.Custom;
public sealed class CustomTransportClient : ITransportClient, IMicroserviceTransport
{
private readonly CustomTransportOptions _options;
private readonly ILogger<CustomTransportClient> _logger;
public CustomTransportClient(
IOptions<CustomTransportOptions> options,
ILogger<CustomTransportClient> logger)
{
_options = options.Value;
_logger = logger;
}
public async Task<ITransportConnection> ConnectAsync(
string host,
int port,
CancellationToken cancellationToken)
{
_logger.LogDebug("Connecting to {Host}:{Port}", host, port);
// Establish connection
// Return ITransportConnection implementation
throw new NotImplementedException();
}
public async Task DisconnectAsync(CancellationToken cancellationToken)
{
// Disconnect from server
}
public void Dispose()
{
// Cleanup resources
}
}
```
### Step 5: Implement Connection
```csharp
using StellaOps.Router.Common.Abstractions;
namespace MyCompany.Router.Transport.Custom;
public sealed class CustomTransportConnection : ITransportConnection
{
public string ConnectionId { get; } = Guid.NewGuid().ToString("N");
public bool IsConnected { get; private set; }
public EndPoint? RemoteEndPoint { get; private set; }
public async Task<int> SendAsync(
ReadOnlyMemory<byte> data,
CancellationToken cancellationToken)
{
// Send data over transport
throw new NotImplementedException();
}
public async Task<int> ReceiveAsync(
Memory<byte> buffer,
CancellationToken cancellationToken)
{
// Receive data from transport
throw new NotImplementedException();
}
public async Task CloseAsync(CancellationToken cancellationToken)
{
IsConnected = false;
// Close connection
}
public void Dispose()
{
// Cleanup
}
}
```
### Step 6: Implement Plugin
```csharp
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.DependencyInjection;
using StellaOps.Router.Common.Abstractions;
using StellaOps.Router.Common.Plugins;
namespace MyCompany.Router.Transport.Custom;
/// <summary>
/// Plugin implementation for custom transport.
/// </summary>
public sealed class CustomTransportPlugin : IRouterTransportPlugin
{
/// <inheritdoc />
public string TransportName => "custom";
/// <inheritdoc />
public string DisplayName => "Custom Transport";
/// <inheritdoc />
public bool IsAvailable(IServiceProvider services)
{
// Check if required dependencies are available
// Return false if transport cannot be used in current environment
return true;
}
/// <inheritdoc />
public void Register(RouterTransportRegistrationContext context)
{
var services = context.Services;
var configuration = context.Configuration;
// Bind configuration
var configSection = context.ConfigurationSection is not null
? configuration.GetSection(context.ConfigurationSection)
: configuration.GetSection("Router:Transport:Custom");
services.AddOptions<CustomTransportOptions>();
if (configSection.GetChildren().Any())
{
services.Configure<CustomTransportOptions>(options =>
{
configSection.Bind(options);
});
}
// Register server if requested
if (context.Mode.HasFlag(RouterTransportMode.Server))
{
services.AddSingleton<CustomTransportServer>();
services.AddSingleton<ITransportServer>(sp =>
sp.GetRequiredService<CustomTransportServer>());
}
// Register client if requested
if (context.Mode.HasFlag(RouterTransportMode.Client))
{
services.AddSingleton<CustomTransportClient>();
services.AddSingleton<ITransportClient>(sp =>
sp.GetRequiredService<CustomTransportClient>());
services.AddSingleton<IMicroserviceTransport>(sp =>
sp.GetRequiredService<CustomTransportClient>());
}
}
}
```
## Building and Packaging
### Build Configuration
Add to your `.csproj`:
```xml
<PropertyGroup>
<TargetFramework>net10.0</TargetFramework>
<ImplicitUsings>enable</ImplicitUsings>
<Nullable>enable</Nullable>
<RootNamespace>MyCompany.Router.Transport.Custom</RootNamespace>
<!-- Plugin assembly attributes -->
<GenerateAssemblyInfo>true</GenerateAssemblyInfo>
</PropertyGroup>
<!-- Output to plugins directory -->
<PropertyGroup Condition="'$(Configuration)' == 'Release'">
<OutputPath>$(SolutionDir)plugins\router\transports\</OutputPath>
<AppendTargetFrameworkToOutputPath>false</AppendTargetFrameworkToOutputPath>
</PropertyGroup>
```
### Build Commands
```bash
# Debug build
dotnet build
# Release build to plugins directory
dotnet build -c Release
# Publish with all dependencies
dotnet publish -c Release -o ./publish
```
## Configuration Schema
Create `plugin.json` manifest:
```json
{
"schemaVersion": "2.0",
"id": "mycompany.router.transport.custom",
"name": "Custom Transport",
"version": "1.0.0",
"assembly": {
"path": "MyCompany.Router.Transport.Custom.dll",
"entryType": "MyCompany.Router.Transport.Custom.CustomTransportPlugin"
},
"capabilities": ["server", "client", "streaming"],
"platforms": ["linux-x64", "win-x64", "osx-arm64"],
"enabled": true,
"priority": 100
}
```
Create `config.yaml` for runtime configuration:
```yaml
id: mycompany.router.transport.custom
name: Custom Transport
enabled: true
config:
host: "0.0.0.0"
port: 5200
maxConnections: 1000
connectTimeout: "00:00:30"
```
## Testing
### Unit Tests
```csharp
public class CustomTransportPluginTests
{
[Fact]
public void TransportName_ReturnsCustom()
{
var plugin = new CustomTransportPlugin();
Assert.Equal("custom", plugin.TransportName);
}
[Fact]
public void Register_AddsServerServices()
{
var plugin = new CustomTransportPlugin();
var services = new ServiceCollection();
var config = new ConfigurationBuilder().Build();
var context = new RouterTransportRegistrationContext(
services, config, RouterTransportMode.Server);
plugin.Register(context);
var provider = services.BuildServiceProvider();
Assert.NotNull(provider.GetService<ITransportServer>());
}
}
```
### Integration Tests
```csharp
public class CustomTransportIntegrationTests
{
[Fact]
public async Task Server_AcceptsConnections()
{
var services = new ServiceCollection();
services.AddLogging();
services.Configure<CustomTransportOptions>(opts =>
{
opts.Port = 15200; // Test port
});
services.AddSingleton<CustomTransportServer>();
var provider = services.BuildServiceProvider();
var server = provider.GetRequiredService<CustomTransportServer>();
await server.StartAsync(CancellationToken.None);
// Connect client and verify
// ...
await server.StopAsync(CancellationToken.None);
}
}
```
## Best Practices
### Error Handling
```csharp
public async Task<ITransportConnection> ConnectAsync(
string host,
int port,
CancellationToken cancellationToken)
{
try
{
// Attempt connection
return await ConnectInternalAsync(host, port, cancellationToken);
}
catch (SocketException ex) when (ex.SocketErrorCode == SocketError.ConnectionRefused)
{
_logger.LogWarning("Connection refused to {Host}:{Port}", host, port);
throw new TransportConnectionException($"Connection refused to {host}:{port}", ex);
}
catch (OperationCanceledException)
{
_logger.LogDebug("Connection attempt cancelled");
throw;
}
catch (Exception ex)
{
_logger.LogError(ex, "Failed to connect to {Host}:{Port}", host, port);
throw new TransportConnectionException($"Failed to connect to {host}:{port}", ex);
}
}
```
### Resource Management
```csharp
public sealed class CustomTransportServer : ITransportServer, IAsyncDisposable
{
private readonly SemaphoreSlim _connectionLock = new(1, 1);
private readonly List<ITransportConnection> _connections = [];
private bool _disposed;
public async ValueTask DisposeAsync()
{
if (_disposed) return;
_disposed = true;
await _connectionLock.WaitAsync();
try
{
foreach (var conn in _connections)
{
await conn.CloseAsync(CancellationToken.None);
conn.Dispose();
}
_connections.Clear();
}
finally
{
_connectionLock.Release();
_connectionLock.Dispose();
}
}
}
```
### Logging
```csharp
// Use structured logging
_logger.LogInformation(
"Connection established {ConnectionId} from {RemoteEndPoint}",
connection.ConnectionId,
connection.RemoteEndPoint);
// Include correlation IDs
using (_logger.BeginScope(new Dictionary<string, object>
{
["ConnectionId"] = connectionId,
["TraceId"] = Activity.Current?.TraceId.ToString() ?? "N/A"
}))
{
await ProcessConnectionAsync(connection, cancellationToken);
}
```
## Deployment
### Copy to Plugins Directory
```bash
cp ./publish/*.dll /opt/stellaops/plugins/router/transports/
```
### Verify Plugin Loading
```bash
# Check logs for plugin discovery
grep "Loaded router transport plugin" /var/log/stellaops/gateway.log
```
### Configuration
```yaml
# router.yaml
Router:
Transport:
Type: custom
Custom:
Host: "0.0.0.0"
Port: 5200
```
## See Also
- [Transport Overview](./README.md)
- [Plugin SDK Guide](../../10_PLUGIN_SDK_GUIDE.md)
- [IRouterTransportPlugin API](../../../src/Router/__Libraries/StellaOps.Router.Common/Plugins/IRouterTransportPlugin.cs)

View File

@@ -0,0 +1,233 @@
# InMemory Transport
The InMemory transport provides zero-latency, in-process communication for development, testing, and scenarios where services run in the same process.
## Overview
| Property | Value |
|----------|-------|
| Plugin Assembly | `StellaOps.Router.Transport.InMemory.dll` |
| Transport Name | `inmemory` |
| Latency | Sub-microsecond |
| Use Case | Development, testing, embedded scenarios |
## Configuration
### router.yaml
```yaml
Router:
Transport:
Type: inmemory
InMemory:
MaxPendingMessages: 10000
MessageTimeout: "00:01:00"
```
### microservice.yaml
```yaml
routers:
- host: localhost
port: 0 # Port ignored for InMemory
transportType: InMemory
priority: 1
```
### Environment Variables
```bash
ROUTER__TRANSPORT__TYPE=inmemory
```
## Options Reference
| Option | Type | Default | Description |
|--------|------|---------|-------------|
| `MaxPendingMessages` | int | `10000` | Maximum queued messages before backpressure |
| `MessageTimeout` | TimeSpan | `00:01:00` | Timeout for pending messages |
| `PreserveMessageOrder` | bool | `true` | Guarantee message ordering |
## Architecture
The InMemory transport uses a shared `InMemoryConnectionRegistry` singleton that enables direct in-process communication:
```
┌─────────────────────────────────────────────────────────────────┐
│ Single Process │
│ │
│ ┌─────────────────────────────────────────────────────────────┐│
│ │ InMemoryConnectionRegistry ││
│ │ (Singleton) ││
│ │ ││
│ │ ┌────────────────┐ ┌─────────────────────────────┐ ││
│ │ │ Service A │ │ Service B │ ││
│ │ │ (InMemoryClient│◄────►│ (InMemoryServer) │ ││
│ │ │ endpoints) │ │ │ ││
│ │ └────────────────┘ └─────────────────────────────┘ ││
│ │ ││
│ │ Messages passed directly via ││
│ │ ConcurrentQueue<T> - no serialization ││
│ └──────────────────────────────────────────────────────────────┘│
│ │
└──────────────────────────────────────────────────────────────────┘
```
## Use Cases
### Development
Run Gateway and microservices in the same process:
```csharp
var builder = WebApplication.CreateBuilder(args);
// Register InMemory transport (shared between gateway and services)
builder.Services.AddInMemoryTransport();
// Add gateway
builder.Services.AddRouterGateway(builder.Configuration);
// Add microservice in same process
builder.Services.AddStellaMicroservice(options =>
{
options.ServiceName = "order-service";
options.Version = "1.0.0";
});
var app = builder.Build();
app.UseRouterGateway();
app.Run();
```
### Integration Testing
```csharp
public class OrderServiceTests : IClassFixture<WebApplicationFactory<Program>>
{
private readonly WebApplicationFactory<Program> _factory;
public OrderServiceTests(WebApplicationFactory<Program> factory)
{
_factory = factory.WithWebHostBuilder(builder =>
{
builder.ConfigureServices(services =>
{
// Use InMemory transport for tests
services.Configure<RouterOptions>(opts =>
opts.Transport.Type = "inmemory");
});
});
}
[Fact]
public async Task CreateOrder_ReturnsOrderId()
{
var client = _factory.CreateClient();
var response = await client.PostAsJsonAsync("/orders", new
{
CustomerId = "CUST-001",
Amount = 99.99m
});
response.EnsureSuccessStatusCode();
var result = await response.Content.ReadFromJsonAsync<OrderResponse>();
Assert.NotNull(result?.OrderId);
}
}
```
### Embedded Scenarios
For single-binary deployments:
```csharp
// All services compiled into one executable
var host = Host.CreateDefaultBuilder(args)
.ConfigureServices(services =>
{
services.AddInMemoryTransport();
// Register all microservice handlers
services.AddScoped<CreateOrderEndpoint>();
services.AddScoped<GetOrderEndpoint>();
services.AddScoped<CreateInvoiceEndpoint>();
})
.Build();
```
## Performance Characteristics
| Metric | Typical Value |
|--------|---------------|
| Latency (p50) | < 0.1ms |
| Latency (p99) | < 0.5ms |
| Throughput | 500,000+ rps |
| Memory overhead | Minimal |
*Zero serialization, direct object passing*
## Limitations
1. **Single process only**: Cannot communicate across process boundaries
2. **No persistence**: Messages lost on process termination
3. **No distribution**: Cannot scale to multiple nodes
4. **Shared memory**: Large messages consume process memory
## When to Use InMemory
| Scenario | Use InMemory? |
|----------|---------------|
| Local development | Yes |
| Unit testing | Yes |
| Integration testing | Yes |
| Single-binary deployment | Yes |
| Multi-node deployment | No - use TCP/TLS |
| Production load testing | No - use production transport |
## Transitioning to Production
When moving from development to production:
```yaml
# Development (appsettings.Development.json)
Router:
Transport:
Type: inmemory
# Production (appsettings.Production.json)
Router:
Transport:
Type: tls
Tls:
Port: 5101
CertificatePath: /certs/server.pfx
```
No code changes required - just configuration.
## Troubleshooting
### Messages Not Being Delivered
1. Verify both client and server use InMemory transport
2. Check `InMemoryConnectionRegistry` is registered as singleton
3. Ensure services are registered in same DI container
### Memory Growing
1. Check `MaxPendingMessages` limit
2. Verify consumers are processing messages
3. Monitor for message timeout (messages queued too long)
### Order Not Preserved
1. Set `PreserveMessageOrder: true`
2. Ensure single consumer per endpoint
3. Don't use parallel processing in handlers
## See Also
- [TCP Transport](./tcp.md) - For multi-process development
- [Transport Overview](./README.md)
- [Testing Guide](../../19_TEST_SUITE_OVERVIEW.md)

View File

@@ -0,0 +1,241 @@
# RabbitMQ Transport
The RabbitMQ transport provides durable, asynchronous message delivery using AMQP 0.9.1 protocol. Ideal for high-volume async processing, fan-out patterns, and scenarios requiring message persistence.
## Overview
| Property | Value |
|----------|-------|
| Plugin Assembly | `StellaOps.Router.Transport.RabbitMq.dll` |
| Transport Name | `rabbitmq` |
| Protocol | AMQP 0.9.1 (RabbitMQ.Client 7.x) |
| Security | TLS, SASL authentication |
| Use Case | Async processing, fan-out, durable messaging |
## Configuration
### router.yaml
```yaml
Router:
Transport:
Type: rabbitmq
RabbitMq:
HostName: rabbitmq.internal
Port: 5672
VirtualHost: /stellaops
UserName: stellaops
Password: ${RABBITMQ_PASSWORD:-}
Ssl:
Enabled: true
ServerName: rabbitmq.internal
CertPath: /certs/client.pfx
Exchange:
Name: stellaops.router
Type: topic
Durable: true
Queue:
Durable: true
AutoDelete: false
PrefetchCount: 100
```
### microservice.yaml
```yaml
routers:
- host: rabbitmq.internal
port: 5672
transportType: RabbitMq
priority: 1
rabbitmq:
virtualHost: /stellaops
exchange: stellaops.router
routingKeyPrefix: orders
```
### Environment Variables
```bash
ROUTER__TRANSPORT__TYPE=rabbitmq
ROUTER__TRANSPORT__RABBITMQ__HOSTNAME=rabbitmq.internal
ROUTER__TRANSPORT__RABBITMQ__PORT=5672
ROUTER__TRANSPORT__RABBITMQ__USERNAME=stellaops
ROUTER__TRANSPORT__RABBITMQ__PASSWORD=secret
```
## Options Reference
### Connection Options
| Option | Type | Default | Description |
|--------|------|---------|-------------|
| `HostName` | string | `localhost` | RabbitMQ server hostname |
| `Port` | int | `5672` | AMQP port (5671 for TLS) |
| `VirtualHost` | string | `/` | RabbitMQ virtual host |
| `UserName` | string | `guest` | Authentication username |
| `Password` | string | `guest` | Authentication password |
| `ConnectionTimeout` | TimeSpan | `00:00:30` | Connection timeout |
| `RequestedHeartbeat` | TimeSpan | `00:01:00` | Heartbeat interval |
### SSL Options
| Option | Type | Default | Description |
|--------|------|---------|-------------|
| `Ssl:Enabled` | bool | `false` | Enable TLS |
| `Ssl:ServerName` | string | - | Expected server certificate name |
| `Ssl:CertPath` | string | - | Client certificate path |
| `Ssl:CertPassphrase` | string | - | Client certificate password |
| `Ssl:Version` | SslProtocols | `Tls13` | TLS version |
### Exchange Options
| Option | Type | Default | Description |
|--------|------|---------|-------------|
| `Exchange:Name` | string | `stellaops.router` | Exchange name |
| `Exchange:Type` | string | `topic` | Exchange type (direct, topic, fanout, headers) |
| `Exchange:Durable` | bool | `true` | Survive broker restart |
| `Exchange:AutoDelete` | bool | `false` | Delete when unused |
### Queue Options
| Option | Type | Default | Description |
|--------|------|---------|-------------|
| `Queue:Durable` | bool | `true` | Persist messages to disk |
| `Queue:AutoDelete` | bool | `false` | Delete when all consumers disconnect |
| `Queue:Exclusive` | bool | `false` | Single-consumer queue |
| `Queue:PrefetchCount` | int | `100` | Prefetch limit per consumer |
## Message Flow
```
┌───────────────┐ ┌─────────────────────────────────────────────┐
│ Gateway │ │ RabbitMQ Broker │
│ │ │ │
│ ─────────────►│ AMQP │ ┌───────────────┐ ┌──────────────────┐ │
│ Publish │────────►│ │ Exchange │───►│ Service Queue │ │
│ │ │ │ (topic/fanout)│ │ (orders.*) │ │
└───────────────┘ │ └───────────────┘ └────────┬─────────┘ │
│ │ │
└─────────────────────────────────│──────────┘
┌───────────────────────────────────────────┐
│ Microservices │
│ ┌─────────────┐ ┌─────────────────────┐ │
│ │ OrderCreate │ │ OrderNotification │ │
│ │ Consumer │ │ Consumer │ │
│ └─────────────┘ └─────────────────────┘ │
└───────────────────────────────────────────┘
```
## Routing Patterns
### Topic Routing
```yaml
Exchange:
Type: topic
# Microservice A binds to: orders.create.#
# Microservice B binds to: orders.*.notify
# Gateway publishes to: orders.create.premium → matches A only
# Gateway publishes to: orders.cancel.notify → matches B only
```
### Fan-Out Pattern
```yaml
Exchange:
Type: fanout
# All bound queues receive every message
# Good for broadcasting events to all services
```
### Direct Routing
```yaml
Exchange:
Type: direct
# Exact routing key match required
# Gateway publishes to: order-service
# Only queue bound with key "order-service" receives
```
## Performance Characteristics
| Metric | Typical Value |
|--------|---------------|
| Latency (p50) | < 5ms |
| Latency (p99) | < 20ms |
| Throughput | 50,000+ mps |
| Memory per connection | ~16KB |
*Persistent messages with acknowledgments on dedicated broker*
## High Availability
### Clustered RabbitMQ
```yaml
RabbitMq:
Endpoints:
- Host: rabbit1.internal
Port: 5672
- Host: rabbit2.internal
Port: 5672
- Host: rabbit3.internal
Port: 5672
Queue:
Arguments:
x-ha-policy: all # Mirror to all nodes
x-queue-type: quorum # Quorum queue (RabbitMQ 3.8+)
```
### Dead Letter Handling
```yaml
Queue:
Arguments:
x-dead-letter-exchange: stellaops.dlx
x-dead-letter-routing-key: failed
x-message-ttl: 3600000 # 1 hour TTL
```
## Troubleshooting
### Connection Refused
```
Error: Failed to connect to rabbitmq.internal:5672
```
1. Verify RabbitMQ is running: `rabbitmqctl status`
2. Check firewall allows AMQP port
3. Verify virtual host exists: `rabbitmqctl list_vhosts`
4. Confirm user has permissions: `rabbitmqctl list_user_permissions stellaops`
### Authentication Failed
```
Error: ACCESS_REFUSED - Login was refused
```
1. Check username/password are correct
2. Verify user exists: `rabbitmqctl list_users`
3. Grant permissions: `rabbitmqctl set_permissions -p /stellaops stellaops ".*" ".*" ".*"`
### Messages Not Being Consumed
1. Check queue exists: `rabbitmqctl list_queues`
2. Verify binding: `rabbitmqctl list_bindings -p /stellaops`
3. Check consumer is connected: `rabbitmqctl list_consumers`
4. Monitor unacked messages: `rabbitmqctl list_queues messages_unacknowledged`
## See Also
- [RabbitMQ Documentation](https://www.rabbitmq.com/documentation.html)
- [Transport Overview](./README.md)
- [Messaging Transports](../../messaging/)

View File

@@ -0,0 +1,135 @@
# TCP Transport
The TCP transport provides high-performance binary communication for internal microservices within the same datacenter or trusted network.
## Overview
| Property | Value |
|----------|-------|
| Plugin Assembly | `StellaOps.Router.Transport.Tcp.dll` |
| Transport Name | `tcp` |
| Default Port | 5100 |
| Security | Network isolation (no encryption) |
| Use Case | Internal services, low-latency communication |
## Configuration
### router.yaml
```yaml
Router:
Transport:
Type: tcp
Tcp:
Host: "0.0.0.0"
Port: 5100
MaxConnections: 1000
ReceiveBufferSize: 65536
SendBufferSize: 65536
KeepAlive: true
NoDelay: true
```
### microservice.yaml
```yaml
routers:
- host: gateway.internal
port: 5100
transportType: Tcp
priority: 1
```
### Environment Variables
```bash
ROUTER__TRANSPORT__TYPE=tcp
ROUTER__TRANSPORT__TCP__HOST=0.0.0.0
ROUTER__TRANSPORT__TCP__PORT=5100
ROUTER__TRANSPORT__TCP__MAXCONNECTIONS=1000
```
## Options Reference
| Option | Type | Default | Description |
|--------|------|---------|-------------|
| `Host` | string | `0.0.0.0` | Bind address for server |
| `Port` | int | `5100` | TCP port number |
| `MaxConnections` | int | `1000` | Maximum concurrent connections |
| `ReceiveBufferSize` | int | `65536` | Socket receive buffer size in bytes |
| `SendBufferSize` | int | `65536` | Socket send buffer size in bytes |
| `KeepAlive` | bool | `true` | Enable TCP keep-alive probes |
| `NoDelay` | bool | `true` | Disable Nagle's algorithm (lower latency) |
| `ConnectTimeout` | TimeSpan | `00:00:30` | Connection timeout |
| `ReadTimeout` | TimeSpan | `00:02:00` | Socket read timeout |
| `WriteTimeout` | TimeSpan | `00:02:00` | Socket write timeout |
## Performance Characteristics
| Metric | Typical Value |
|--------|---------------|
| Latency (p50) | < 1ms |
| Latency (p99) | < 5ms |
| Throughput | 100,000+ rps |
| Memory per connection | ~2KB |
*Benchmarks on 10Gbps network with small payloads (<1KB)*
## Security Considerations
TCP transport does **not** provide encryption. Use only in:
- Private networks with proper network segmentation
- Same-datacenter deployments with firewalled traffic
- Container orchestration networks (Kubernetes pod network)
For encrypted communication, use [TLS transport](./tls.md).
## Framing Protocol
The TCP transport uses the standard Router binary framing protocol:
```
┌────────────────────────────────────────────────────────────────┐
│ Frame Header (24 bytes) │
├────────────┬────────────┬────────────┬────────────┬────────────┤
│ Magic (4) │ Version(2) │ Type (2) │ Flags (4) │ Length (8) │
├────────────┴────────────┴────────────┴────────────┴────────────┤
│ Correlation ID (4) │
├─────────────────────────────────────────────────────────────────┤
│ Frame Payload (variable) │
└─────────────────────────────────────────────────────────────────┘
```
## Troubleshooting
### Connection Refused
```
Error: Connection refused to gateway.internal:5100
```
1. Verify Gateway is running and listening on port 5100
2. Check firewall rules allow traffic on port 5100
3. Verify DNS resolution of hostname
### Connection Timeout
```
Error: Connection to gateway.internal:5100 timed out
```
1. Increase `ConnectTimeout` value
2. Check network connectivity between services
3. Verify no network segmentation blocking traffic
### Performance Issues
1. Enable `NoDelay: true` for latency-sensitive workloads
2. Tune buffer sizes based on payload sizes
3. Monitor connection pool exhaustion
## See Also
- [TLS Transport](./tls.md) - Encrypted variant
- [Transport Overview](./README.md)
- [Router Architecture](../ARCHITECTURE.md)

View File

@@ -0,0 +1,223 @@
# TLS Transport
The TLS transport provides encrypted communication with optional mutual TLS (mTLS) authentication for secure cross-datacenter and external service communication.
## Overview
| Property | Value |
|----------|-------|
| Plugin Assembly | `StellaOps.Router.Transport.Tls.dll` |
| Transport Name | `tls` |
| Default Port | 5101 |
| Security | TLS 1.3, optional mTLS |
| Use Case | Cross-datacenter, external services, compliance-required environments |
## Configuration
### router.yaml (Gateway/Server)
```yaml
Router:
Transport:
Type: tls
Tls:
Host: "0.0.0.0"
Port: 5101
CertificatePath: /certs/server.pfx
CertificatePassword: ${TLS_CERT_PASSWORD:-}
RequireClientCertificate: true # Enable mTLS
AllowedClientCertificates:
- /certs/trusted/client1.cer
- /certs/trusted/client2.cer
TlsProtocols: Tls13 # TLS 1.3 only
CheckCertificateRevocation: true
```
### microservice.yaml (Client)
```yaml
routers:
- host: gateway.external.company.com
port: 5101
transportType: Tls
priority: 1
tls:
clientCertificatePath: /certs/client.pfx
clientCertificatePassword: ${CLIENT_CERT_PASSWORD:-}
validateServerCertificate: true
serverCertificateThumbprints:
- "A1B2C3D4E5F6..."
```
### Environment Variables
```bash
ROUTER__TRANSPORT__TYPE=tls
ROUTER__TRANSPORT__TLS__PORT=5101
ROUTER__TRANSPORT__TLS__CERTIFICATEPATH=/certs/server.pfx
ROUTER__TRANSPORT__TLS__CERTIFICATEPASSWORD=secret
ROUTER__TRANSPORT__TLS__REQUIRECLIENTCERTIFICATE=true
```
## Options Reference
### Server Options
| Option | Type | Default | Description |
|--------|------|---------|-------------|
| `Host` | string | `0.0.0.0` | Bind address |
| `Port` | int | `5101` | TLS port number |
| `CertificatePath` | string | - | Path to server certificate (PFX/P12) |
| `CertificatePassword` | string | - | Password for certificate file |
| `RequireClientCertificate` | bool | `false` | Enable mutual TLS |
| `AllowedClientCertificates` | string[] | - | Paths to trusted client certs |
| `TlsProtocols` | TlsProtocols | `Tls12,Tls13` | Allowed TLS versions |
| `CheckCertificateRevocation` | bool | `true` | Check CRL/OCSP |
| `CipherSuites` | string[] | - | Allowed cipher suites (TLS 1.3) |
### Client Options
| Option | Type | Default | Description |
|--------|------|---------|-------------|
| `ClientCertificatePath` | string | - | Path to client certificate (PFX/P12) |
| `ClientCertificatePassword` | string | - | Password for client certificate |
| `ValidateServerCertificate` | bool | `true` | Validate server certificate |
| `ServerCertificateThumbprints` | string[] | - | Pinned server cert thumbprints |
| `AllowUntrustedCertificates` | bool | `false` | Allow self-signed certs (dev only) |
## Mutual TLS (mTLS)
For zero-trust environments, enable mutual TLS authentication:
```
┌──────────────────┐ ┌──────────────────┐
│ Microservice │ │ Gateway │
│ │ │ │
│ Client Cert │◄───── TLS ────────►│ Server Cert │
│ (identity) │ Handshake │ (identity) │
│ │ │ │
│ Validates: │ │ Validates: │
│ - Server cert │ │ - Client cert │
│ - Thumbprint │ │ - Allowlist │
└──────────────────┘ └──────────────────┘
```
### Certificate Requirements
**Server Certificate:**
- Extended Key Usage: `Server Authentication (1.3.6.1.5.5.7.3.1)`
- Subject Alternative Name: Include all DNS names clients will connect to
**Client Certificate:**
- Extended Key Usage: `Client Authentication (1.3.6.1.5.5.7.3.2)`
- Common Name or SAN identifying the service
## Performance Characteristics
| Metric | Typical Value |
|--------|---------------|
| Latency (p50) | < 2ms |
| Latency (p99) | < 10ms |
| Throughput | 80,000+ rps |
| Memory per connection | ~8KB |
*TLS 1.3 with session resumption on 10Gbps network*
## Certificate Management
### Generating Certificates
```bash
# Generate CA
openssl genrsa -out ca.key 4096
openssl req -new -x509 -days 3650 -key ca.key -out ca.crt \
-subj "/CN=StellaOps Internal CA"
# Generate server certificate
openssl genrsa -out server.key 2048
openssl req -new -key server.key -out server.csr \
-subj "/CN=gateway.internal" \
-addext "subjectAltName=DNS:gateway.internal,DNS:localhost"
openssl x509 -req -days 365 -in server.csr -CA ca.crt -CAkey ca.key \
-CAcreateserial -out server.crt \
-extfile <(echo "subjectAltName=DNS:gateway.internal,DNS:localhost")
# Package as PFX
openssl pkcs12 -export -out server.pfx -inkey server.key -in server.crt \
-certfile ca.crt -passout pass:changeit
```
### Certificate Rotation
1. Generate new certificate before expiry
2. Update `CertificatePath` in configuration
3. Restart Gateway (no connection interruption with graceful shutdown)
4. Update client thumbprint pins if using certificate pinning
## Air-Gap Deployment
For offline environments:
1. Pre-provision all certificates
2. Disable CRL/OCSP checks: `CheckCertificateRevocation: false`
3. Use certificate pinning instead of chain validation
```yaml
Router:
Transport:
Type: tls
Tls:
CheckCertificateRevocation: false
AllowedClientCertificates:
- /certs/trusted/client1.cer
```
## Troubleshooting
### Certificate Validation Failed
```
Error: The remote certificate is invalid according to the validation procedure
```
1. Verify certificate is not expired: `openssl x509 -in cert.pem -noout -dates`
2. Check certificate chain is complete
3. Verify CA is trusted by the system or explicitly configured
### mTLS Handshake Failed
```
Error: The client certificate is not provided
```
1. Ensure client certificate is configured with correct path
2. Verify certificate has Client Authentication EKU
3. Check certificate is in Gateway's allowlist
### TLS Protocol Mismatch
```
Error: A call to SSPI failed, TLS version mismatch
```
1. Ensure both sides support compatible TLS versions
2. Update `TlsProtocols` to include common version
3. TLS 1.3 recommended for new deployments
## Compliance
The TLS transport supports compliance requirements:
| Standard | Configuration |
|----------|---------------|
| PCI-DSS | TLS 1.2+, strong ciphers, certificate validation |
| HIPAA | TLS 1.2+, mTLS for service-to-service |
| FedRAMP | TLS 1.3, FIPS-validated crypto modules |
For FIPS mode, ensure .NET is configured for FIPS compliance and use FIPS-approved cipher suites.
## See Also
- [TCP Transport](./tcp.md) - Unencrypted variant for internal use
- [Transport Overview](./README.md)
- [Security Hardening Guide](../../17_SECURITY_HARDENING_GUIDE.md)

View File

@@ -0,0 +1,173 @@
# UDP Transport
The UDP transport provides connectionless, fire-and-forget messaging suitable for broadcast notifications and scenarios where delivery guarantees are not critical.
## Overview
| Property | Value |
|----------|-------|
| Plugin Assembly | `StellaOps.Router.Transport.Udp.dll` |
| Transport Name | `udp` |
| Default Port | 5102 |
| Security | Network isolation (no encryption) |
| Use Case | Broadcast, metrics, fire-and-forget events |
## Configuration
### router.yaml
```yaml
Router:
Transport:
Type: udp
Udp:
Host: "0.0.0.0"
Port: 5102
ReceiveBufferSize: 65536
SendBufferSize: 65536
MulticastGroup: null # Optional multicast group
MulticastTtl: 1
EnableBroadcast: false
```
### microservice.yaml
```yaml
routers:
- host: gateway.internal
port: 5102
transportType: Udp
priority: 1
```
### Environment Variables
```bash
ROUTER__TRANSPORT__TYPE=udp
ROUTER__TRANSPORT__UDP__HOST=0.0.0.0
ROUTER__TRANSPORT__UDP__PORT=5102
```
## Options Reference
| Option | Type | Default | Description |
|--------|------|---------|-------------|
| `Host` | string | `0.0.0.0` | Bind address |
| `Port` | int | `5102` | UDP port number |
| `ReceiveBufferSize` | int | `65536` | Socket receive buffer size |
| `SendBufferSize` | int | `65536` | Socket send buffer size |
| `MulticastGroup` | string | `null` | Multicast group address (e.g., `239.1.2.3`) |
| `MulticastTtl` | int | `1` | Multicast time-to-live |
| `EnableBroadcast` | bool | `false` | Allow broadcast to `255.255.255.255` |
| `MaxDatagramSize` | int | `65507` | Maximum UDP datagram size |
## Use Cases
### Fire-and-Forget Events
For events where acknowledgment is not required:
```csharp
[StellaEndpoint("POST", "/events/metrics", FireAndForget = true)]
public sealed class MetricsEndpoint : IStellaEndpoint<MetricsBatch, EmptyResponse>
{
public Task<EmptyResponse> HandleAsync(MetricsBatch request, CancellationToken ct)
{
// Process metrics - no response expected
return Task.FromResult(new EmptyResponse());
}
}
```
### Multicast Broadcasting
For broadcasting to multiple listeners:
```yaml
Router:
Transport:
Type: udp
Udp:
MulticastGroup: "239.0.1.1"
MulticastTtl: 2
```
Services join the multicast group to receive broadcasts:
```csharp
// All services with this config receive broadcasts
routers:
- host: "239.0.1.1"
port: 5102
transportType: Udp
```
## Performance Characteristics
| Metric | Typical Value |
|--------|---------------|
| Latency (p50) | < 0.5ms |
| Latency (p99) | < 2ms |
| Throughput | 150,000+ pps |
| Memory per socket | ~1KB |
*Note: No delivery guarantees; packets may be lost under load*
## Limitations
1. **No delivery guarantees**: Packets may be dropped
2. **No ordering guarantees**: Packets may arrive out of order
3. **Size limits**: Maximum datagram size ~65KB (64KB practical limit)
4. **No acknowledgments**: Sender doesn't know if message was received
5. **No fragmentation handling**: Large messages must fit in single datagram
## When to Use UDP
**Good fit:**
- Real-time metrics and telemetry
- Service discovery announcements
- Health check broadcasts
- Low-latency gaming or streaming metadata
**Not recommended:**
- Transaction processing
- Data that must not be lost
- Large payloads (use TCP/TLS instead)
## Security Considerations
UDP transport does **not** provide encryption or authentication. Consider:
- Network segmentation
- Firewall rules to restrict UDP traffic
- Application-level message signing if integrity is needed
- DTLS wrapper for encrypted UDP (not built-in)
## Troubleshooting
### Messages Not Received
1. Check firewall allows UDP traffic on the configured port
2. Verify receiver is bound to correct address/port
3. Check for buffer overflow (increase `ReceiveBufferSize`)
4. Monitor for packet loss with network tools
### Multicast Not Working
1. Verify multicast routing is enabled on network
2. Check `MulticastTtl` is sufficient for network topology
3. Ensure IGMP snooping is properly configured
4. Verify all receivers joined the multicast group
### High Packet Loss
1. Reduce message rate
2. Increase buffer sizes
3. Check for network congestion
4. Consider switching to TCP for reliable delivery
## See Also
- [TCP Transport](./tcp.md) - Reliable delivery
- [Transport Overview](./README.md)
- [Router Architecture](../ARCHITECTURE.md)