stella-ops.org/git.stella-ops.org
1
0
Fork 0
Code Issues Pull Requests Actions 2 Releases Wiki Activity

docs consolidation and others

Browse Source
This commit is contained in:
master
2026-01-06 19:02:21 +02:00
parent d7bdca6d97
commit 4789027317
849 changed files with 16551 additions and 66770 deletions
Download Patch File Download Diff File Expand all files Collapse all files

402
docs-archived/router/ARCHITECTURE.md Normal file
View File

@@ -0,0 +1,402 @@
# StellaOps Router Architecture
This document describes the internal architecture of the StellaOps Router framework.
## Core Components
### 1. Router Gateway
The Gateway is the HTTP ingress point that accepts client requests and routes them to appropriate microservices.
```
┌─────────────────────────────────────────────┐
│ Router Gateway │
│ │
HTTP Request ──────►│ ┌────────────────────────────────────────┐ │
│ │ Middleware Pipeline │ │
│ │ ├─ Authentication │ │
│ │ ├─ Authorization │ │
│ │ └─ Rate Limiting │ │
│ └────────────────────────────────────────┘ │
│ │ │
│ ▼ │
│ ┌────────────────────────────────────────┐ │
│ │ Route Resolution │ │
│ │ - Path matching │ │
│ │ - Service discovery │ │
│ │ - Load balancing │ │
│ └────────────────────────────────────────┘ │
│ │ │
│ ▼ │
│ ┌────────────────────────────────────────┐ │
│ │ Request Dispatch │ │
│ │ - Serialize to binary │ │
│ │ - Send via transport │ │
│ │ - Await response │ │
│ └────────────────────────────────────────┘ │
│ │ │
│ ▼ │
│ ┌────────────────────────────────────────┐ │
│ │ Response Processing │ │
│ │ - Deserialize from binary │ │
│ │ - Transform headers │ │
│ │ - Stream body │ │
│ └────────────────────────────────────────┘ │
│ │
└──────────────────────────────────────────────┘
│
▼ Binary Protocol
┌─────────────────────────────────────────────┐
│ Microservices │
└─────────────────────────────────────────────┘
```
**Key Classes:**
- `RouterGatewayMiddleware` - ASP.NET Core middleware for request handling
- `ServiceRegistry` - Tracks registered microservices and their endpoints
- `RouteResolver` - Matches incoming paths to service endpoints
- `RequestDispatcher` - Sends requests via appropriate transport
### 2. Microservice SDK
The SDK provides the programming model for building microservices with typed endpoints.
```
┌──────────────────────────────────────────────────────────────────┐
│ Microservice Process │
│ │
│ ┌────────────────────────────────────────────────────────────┐ │
│ │ Endpoint Handlers (User Code) │ │
│ │ │ │
│ │ [StellaEndpoint("POST", "/orders")] │ │
│ │ class CreateOrderEndpoint : IStellaEndpoint<Req, Res> │ │
│ │ │ │
│ │ [StellaEndpoint("GET", "/stream", SupportsStreaming=true)]│ │
│ │ class StreamEndpoint : IRawStellaEndpoint │ │
│ └────────────────────────────────────────────────────────────┘ │
│ │ │
│ ▼ │
│ ┌────────────────────────────────────────────────────────────┐ │
│ │ EndpointDescriptorProvider (Source Generated) │ │
│ │ - Collects all [StellaEndpoint] in assembly │ │
│ │ - Provides routing metadata at startup │ │
│ └────────────────────────────────────────────────────────────┘ │
│ │ │
│ ▼ │
│ ┌────────────────────────────────────────────────────────────┐ │
│ │ MicroserviceHost │ │
│ │ - Connects to gateway │ │
│ │ - Registers endpoints │ │
│ │ - Dispatches incoming requests to handlers │ │
│ │ - Manages heartbeat and reconnection │ │
│ └────────────────────────────────────────────────────────────┘ │
│ │ │
│ ▼ │
│ ┌────────────────────────────────────────────────────────────┐ │
│ │ Transport Client │ │
│ │ (InMemory, TCP, TLS, RabbitMQ, etc.) │ │
│ └────────────────────────────────────────────────────────────┘ │
│ │
└───────────────────────────────────────────────────────────────────┘
```
**Key Classes:**
- `StellaEndpointAttribute` - Marks endpoint classes with routing metadata
- `IStellaEndpoint<TRequest, TResponse>` - Typed endpoint interface
- `IRawStellaEndpoint` - Raw/streaming endpoint interface
- `MicroserviceHost` - Manages service lifecycle and gateway connection
### 3. Transport Layer
The transport layer provides pluggable communication protocols.
```
┌────────────────────────────────────────────────────────────────────┐
│ Transport Abstraction │
│ │
│ ┌────────────────────────────────────────────────────────────┐ │
│ │ ITransportServer / ITransportClient │ │
│ │ - ConnectAsync / AcceptAsync │ │
│ │ - SendAsync / ReceiveAsync │ │
│ │ - Disconnect / Dispose │ │
│ └────────────────────────────────────────────────────────────┘ │
│ │ │
│ ┌──────────────────┼──────────────────┐ │
│ ▼ ▼ ▼ │
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │
│ │ InMemory │ │ TCP/TLS │ │ RabbitMQ │ │
│ │ Transport │ │ Transport │ │ Transport │ │
│ │ │ │ │ │ │ │
│ │ - Zero-copy │ │ - Binary │ │ - Async │ │
│ │ - Dev/test │ │ - Framing │ │ - Pub/Sub │ │
│ │ │ │ - Backpress │ │ - Durable │ │
│ └──────────────┘ └──────────────┘ └──────────────┘ │
│ │
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │
│ │ UDP │ │ Valkey │ │ PostgreSQL │ │
│ │ Transport │ │ Transport │ │ Transport │ │
│ │ │ │ │ │ │ │
│ │ - Broadcast │ │ - Streams │ │ - LISTEN/ │ │
│ │ - Fire&Forg │ │ - Consumer │ │ NOTIFY │ │
│ │ │ │ Groups │ │ - Txn queue │ │
│ └──────────────┘ └──────────────┘ └──────────────┘ │
│ │
└─────────────────────────────────────────────────────────────────────┘
```
**Transport Selection Guidelines:**
| Scenario | Recommended Transport |
|----------|----------------------|
| Development/Testing | InMemory |
| Same-datacenter services | TCP |
| Cross-datacenter secure | TLS with mTLS |
| High-volume async | RabbitMQ |
| Broadcast notifications | UDP |
| Distributed with replay | Valkey Streams |
| Transactional messaging | PostgreSQL |
### 4. Binary Protocol
All transports use a common binary frame format:
```
┌────────────────────────────────────────────────────────────────┐
│ Frame Header (24 bytes) │
├────────────┬────────────┬────────────┬────────────┬────────────┤
│ Magic (4) │ Version(2) │ Type (2) │ Flags (4) │ Length (8) │
├────────────┴────────────┴────────────┴────────────┴────────────┤
│ Correlation ID (4) │
├─────────────────────────────────────────────────────────────────┤
│ Frame Payload │
│ │
│ For Request: │
│ ┌─────────────────────────────────────────────────────────┐ │
│ │ Method (1) │ Path Length (2) │ Path (variable) │ │
│ ├─────────────────────────────────────────────────────────┤ │
│ │ Header Count (2) │ Headers (key-length-key-val pairs) │ │
│ ├─────────────────────────────────────────────────────────┤ │
│ │ Body (remaining bytes) │ │
│ └─────────────────────────────────────────────────────────┘ │
│ │
│ For Response: │
│ ┌─────────────────────────────────────────────────────────┐ │
│ │ Status Code (2) │ Header Count (2) │ Headers (...) │ │
│ ├─────────────────────────────────────────────────────────┤ │
│ │ Body (remaining bytes) │ │
│ └─────────────────────────────────────────────────────────┘ │
│ │
└─────────────────────────────────────────────────────────────────┘
```
**Frame Types:**
- `0x01` - Request
- `0x02` - Response
- `0x03` - Heartbeat
- `0x04` - Registration
- `0x05` - Acknowledgment
- `0x10` - Stream Chunk
- `0x11` - Stream End
### 5. Source Generator
The `StellaOps.Microservice.SourceGen` package generates code at compile time:
```csharp
// Input: User-defined endpoint
[StellaEndpoint("POST", "/orders")]
public sealed class CreateOrderEndpoint : IStellaEndpoint<CreateOrderRequest, CreateOrderResponse>
{
// ...
}
// Generated: Endpoint descriptor
[GeneratedCode("StellaOps.Microservice.SourceGen", "1.0.0")]
internal static class StellaEndpointDescriptors
{
public static IEnumerable<EndpointDescriptor> GetDescriptors()
{
yield return new EndpointDescriptor
{
Method = HttpMethod.Post,
Path = "/orders",
TimeoutSeconds = 30,
SupportsStreaming = false,
HandlerType = typeof(CreateOrderEndpoint),
RequestType = typeof(CreateOrderRequest),
ResponseType = typeof(CreateOrderResponse)
};
}
}
// Generated: JSON Schema provider (if [ValidateSchema] present)
[GeneratedCode("StellaOps.Microservice.SourceGen", "1.0.0")]
internal static class CreateOrderRequestSchemaProvider
{
public static JsonSchema GetSchema() => JsonSchema.FromText("""
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"type": "object",
"properties": {
"customerId": { "type": "string" },
"amount": { "type": "number" }
},
"required": ["customerId", "amount"]
}
""");
}
```
## Request Flow
### 1. Service Registration
```
Microservice Gateway
│ │
│─────── Connect ─────────────►│
│ │
│─────── Register ────────────►│
│ (service name, version, │ ┌───────────────┐
│ endpoint descriptors) │ │ ServiceReg │
│ │ │ Registry │
│◄────── Ack ─────────────────│ └───────────────┘
│ │
│◄─────► Heartbeat ───────────►│ (every 30s)
│ │
```
### 2. Request Dispatch
```
Client Gateway Microservice
│ │ │
│── HTTP Req ──►│ │
│ │ │
│ │── Route Resolve ─► │
│ │ (path matching) │
│ │ │
│ │── Binary Frame ───────────►│
│ │ (via transport) │
│ │ │
│ │ │── Handle
│ │ │ Request
│ │ │
│ │◄── Binary Response ────────│
│ │ │
│◄─ HTTP Resp ──│ │
│ │ │
```
### 3. Streaming
```
Client Gateway Microservice
│ │ │
│── HTTP POST ─►│ │
│ (chunked) │ │
│ │── Stream Start ───────────►│
│ │ │
│ ┌─ chunk 1 ─►│── Stream Chunk ───────────►│
│ │ │ │
│ │ chunk 2 ─►│── Stream Chunk ───────────►│
│ │ │ │
│ └ chunk N ─►│── Stream End ─────────────►│
│ │ │
│ │◄── Response ───────────────│
│◄─ HTTP 200 ──│ │
│ │ │
```
## Configuration Loading
```
┌─────────────────────────────────────────────────────────────────┐
│ Configuration Sources │
├─────────────────────────────────────────────────────────────────┤
│ │
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────────────┐ │
│ │ YAML Files │ │ Environment │ │ IConfiguration │ │
│ │ │ │ Variables │ │ (ASP.NET) │ │
│ │ - router.yaml│ │ │ │ │ │
│ │ - microserv. │ │ STELLA_* │ │ appsettings.json │ │
│ │ .yaml │ │ │ │ │ │
│ └──────────────┘ └──────────────┘ └──────────────────────┘ │
│ │ │ │ │
│ └────────────────┬────────────────────┘ │
│ ▼ │
│ ┌──────────────────┐ │
│ │ RouterConfig │ │
│ │ Provider │ │
│ │ │ │
│ │ - Hot reload │ │
│ │ - Validation │ │
│ │ - Defaults │ │
│ └──────────────────┘ │
│ │
└──────────────────────────────────────────────────────────────────┘
```
## Error Handling
### Transport Errors
- Connection failures trigger automatic reconnection with exponential backoff
- Failed requests are retried based on idempotency metadata
- Circuit breaker prevents cascading failures
### Request Errors
- Timeout errors return 504 Gateway Timeout
- Validation errors return 400 Bad Request with details
- Handler exceptions return 500 with sanitized message
### Resilience Patterns
```csharp
// Retry policy (configured in microservice.yaml)
retryPolicy:
maxAttempts: 3
initialDelay: 100ms
maxDelay: 5s
retryableStatuses: [502, 503, 504]
// Circuit breaker (configured in router.yaml)
circuitBreaker:
failureThreshold: 5
samplingDuration: 10s
breakDuration: 30s
```
## Security
### Transport Security
- TLS 1.3 for encrypted communication
- mTLS for mutual authentication
- Certificate validation with configurable CA trust
### Gateway Security
- Integration with Authority module for OAuth/OIDC
- Claims-based authorization at route level
- Rate limiting per client/tenant
### Message Security
- Optional message signing for integrity
- Tenant isolation in multi-tenant deployments
- Audit logging of all requests
## Performance Characteristics
| Metric | InMemory | TCP | TLS | RabbitMQ |
|--------|----------|-----|-----|----------|
| Latency (p50) | < 0.1ms | < 1ms | < 2ms | < 5ms |
| Latency (p99) | < 0.5ms | < 5ms | < 10ms | < 20ms |
| Throughput | 500K rps | 100K rps | 80K rps | 50K rps |
| Memory/conn | N/A | 2KB | 8KB | 16KB |
*Benchmarks run on AMD EPYC with 10GB network, small payloads (<1KB)*
## See Also
- [Getting Started Guide](./GETTING_STARTED.md)
- [Transport Configuration](./transports/)
- [Examples](./examples/README.md)
- [API Reference](../api/router/)

298
docs-archived/router/README.md Normal file
View File

@@ -0,0 +1,298 @@
# StellaOps Router
**Transport-agnostic microservice communication framework with plugin-based transport loading**
The StellaOps Router is a high-performance, production-grade framework for building distributed microservice architectures. It provides a unified API for service-to-service communication across multiple transport protocols, with automatic service discovery, OpenAPI aggregation, compile-time endpoint validation, and **runtime transport plugin loading**.
## Key Features
- **Multi-Transport Support**: InMemory, TCP, TLS/mTLS, RabbitMQ, UDP, Valkey/Redis, PostgreSQL
- **Plugin-Based Transports**: Transports loaded at runtime via `IRouterTransportPlugin` interface
- **Compile-Time Code Generation**: Source generators create endpoint descriptors and schema providers at build time
- **Gateway Pattern**: HTTP ingress with automatic routing to backend microservices
- **OpenAPI Aggregation**: Unified Swagger documentation from all connected services
- **Streaming Support**: First-class support for large payloads and server-sent events
- **JSON Schema Validation**: Optional request/response validation with `[ValidateSchema]`
- **Zero-Configuration Discovery**: Services auto-register with the gateway on startup
## Architecture
```
┌─────────────────────────────────────────────────────────────────────┐
│ HTTP Clients │
│ (Browser, CLI, Mobile) │
└─────────────────────────────┬───────────────────────────────────────┘
│ HTTPS
▼
┌─────────────────────────────────────────────────────────────────────┐
│ Router Gateway │
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────────────────┐ │
│ │ OpenAPI Agg │ │ Auth/AuthZ │ │ Route Resolution │ │
│ └──────────────┘ └──────────────┘ └──────────────────────────┘ │
└─────────────────────────────┬───────────────────────────────────────┘
│ Binary Transport (TCP/TLS/RabbitMQ)
┌──────────────────┼──────────────────┐
▼ ▼ ▼
┌───────────────────┐ ┌───────────────────┐ ┌───────────────────┐
│ Billing │ │ Inventory │ │ Notification │
│ Microservice │ │ Microservice │ │ Microservice │
│ │ │ │ │ │
│ [StellaEndpoint] │ │ [StellaEndpoint] │ │ [StellaEndpoint] │
└───────────────────┘ └───────────────────┘ └───────────────────┘
```
## Quick Start
### 1. Create a Microservice
```csharp
using StellaOps.Microservice;
using StellaOps.Router.Transport.InMemory;
var builder = Host.CreateApplicationBuilder(args);
builder.Services.AddStellaMicroservice(options =>
{
options.ServiceName = "my-service";
options.Version = "1.0.0";
options.ConfigFilePath = "microservice.yaml";
});
builder.Services.AddInMemoryTransport();
builder.Services.AddScoped<MyEndpoint>();
var host = builder.Build();
await host.RunAsync();
```
### 2. Define Endpoints
```csharp
using StellaOps.Microservice;
// Typed endpoint with automatic JSON serialization
[StellaEndpoint("POST", "/orders", TimeoutSeconds = 30)]
public sealed class CreateOrderEndpoint : IStellaEndpoint<CreateOrderRequest, CreateOrderResponse>
{
public Task<CreateOrderResponse> HandleAsync(
CreateOrderRequest request,
CancellationToken cancellationToken)
{
return Task.FromResult(new CreateOrderResponse
{
OrderId = $"ORD-{Guid.NewGuid():N}"[..16]
});
}
}
// Raw endpoint for streaming
[StellaEndpoint("POST", "/files/{id}", SupportsStreaming = true)]
public sealed class UploadFileEndpoint : IRawStellaEndpoint
{
public async Task<RawResponse> HandleAsync(
RawRequestContext context,
CancellationToken cancellationToken)
{
var fileId = context.PathParameters["id"];
await using var stream = context.Body;
// Process stream...
var headers = new HeaderCollection();
headers.Set("Content-Type", "application/json");
return new RawResponse
{
StatusCode = 201,
Headers = headers,
Body = new MemoryStream("""{"status":"uploaded"}"""u8.ToArray())
};
}
}
```
### 3. Configure the Gateway
```csharp
using StellaOps.Router.Gateway;
using StellaOps.Router.Gateway.DependencyInjection;
using StellaOps.Router.Transport.InMemory;
var builder = WebApplication.CreateBuilder(args);
builder.Services.AddRouterGateway(builder.Configuration);
builder.Services.AddInMemoryTransport();
builder.Services.AddAuthentication();
var app = builder.Build();
app.UseAuthentication();
app.UseClaimsAuthorization();
app.MapRouterOpenApi();
app.UseRouterGateway();
app.Run();
```
## Transport Plugins
Transport implementations are loaded at runtime via the plugin system. See [Transport Plugins](./transports/README.md) for full documentation.
| Transport | Plugin Assembly | Use Case | Performance |
|-----------|-----------------|----------|-------------|
| **InMemory** | `StellaOps.Router.Transport.InMemory.dll` | Development, testing | Sub-microsecond |
| **TCP** | `StellaOps.Router.Transport.Tcp.dll` | Internal services | < 1ms latency |
| **TLS** | `StellaOps.Router.Transport.Tls.dll` | Cross-datacenter, mTLS | < 2ms latency |
| **RabbitMQ** | `StellaOps.Router.Transport.RabbitMq.dll` | Async processing | Queue-based |
| **UDP** | `StellaOps.Router.Transport.Udp.dll` | Fire-and-forget | Best effort |
| **Valkey** | `StellaOps.Messaging.Transport.Valkey.dll` | Distributed, pub/sub | Memory-fast |
| **PostgreSQL** | `StellaOps.Messaging.Transport.Postgres.dll` | Transactional | Persistent |
### Loading Transport Plugins
```csharp
using StellaOps.Router.Common.Plugins;
// Load transport plugins from directory
var pluginLoader = new RouterTransportPluginLoader(logger);
pluginLoader.LoadFromDirectory("plugins/router/transports");
// Register the configured transport (reads Router:Transport:Type from config)
pluginLoader.RegisterConfiguredTransport(services, configuration);
```
### Configuring Transport
```yaml
# router.yaml
Router:
Transport:
Type: tls # Which transport plugin to use
Tls:
Port: 5101
CertificatePath: /certs/server.pfx
RequireClientCertificate: true
```
## Project Structure
```
src/Router/
├── StellaOps.Gateway.WebService/ # Gateway HTTP entrypoint
├── __Libraries/
│ ├── StellaOps.Router.Gateway/ # Gateway core logic
│ ├── StellaOps.Router.Common/ # Shared models and enums
│ ├── StellaOps.Router.Config/ # YAML configuration
│ ├── StellaOps.Router.AspNet/ # ASP.NET Core integration
│ ├── StellaOps.Microservice/ # Microservice SDK
│ ├── StellaOps.Microservice.SourceGen/ # Compile-time generator
│ ├── StellaOps.Messaging/ # Queue abstractions
│ └── StellaOps.Router.Transport.*/ # Transport implementations
├── __Tests/ # Unit and integration tests
└── examples/ # Working examples
├── Examples.OrderService/
├── Examples.NotificationService/
└── Examples.MultiTransport.Gateway/
```
## Configuration
### microservice.yaml
```yaml
service:
name: billing-service
version: 1.0.0
region: us-east-1
instanceId: billing-001
endpoints:
- path: /invoices
method: POST
timeoutSeconds: 30
- path: /invoices/{id}
method: GET
timeoutSeconds: 10
routers:
- host: gateway.internal
port: 5100
transportType: Tcp
priority: 1
- host: gateway-backup.internal
port: 5100
transportType: Tcp
priority: 2
heartbeat:
intervalSeconds: 30
reconnect:
initialDelayMs: 1000
maxDelayMs: 60000
```
### router.yaml (Gateway)
```yaml
gateway:
name: api-gateway
region: us-east-1
transports:
tcp:
port: 5100
maxConnections: 1000
tls:
port: 5101
certificatePath: /certs/server.pfx
requireClientCertificate: true
routing:
defaultTimeout: 30
maxRequestBodyBytes: 10485760
services:
billing:
routes:
- path: /invoices
methods: [GET, POST]
- path: /invoices/{id}
methods: [GET, PUT, DELETE]
inventory:
routes:
- path: /items
methods: [GET]
```
## Building
```bash
# Build the Router solution
dotnet build src/Router/StellaOps.Router.sln
# Run tests
dotnet test src/Router/StellaOps.Router.sln
# Build specific example
dotnet run --project src/Router/examples/Examples.OrderService
```
## Documentation
### Implementation Guides (this directory)
- [Architecture Overview](./ARCHITECTURE.md)
- [Getting Started Guide](./GETTING_STARTED.md)
- [Transport Configuration](./transports/)
- [Rate Limiting](./rate-limiting.md)
- [Examples](./examples/README.md)
### Module Dossier
For architectural decisions, invariants, and module context, see the [Router Module Dossier](../modules/router/README.md).
## Related Modules
- **Authority**: OAuth/OIDC integration for gateway authentication
- **Policy**: Route-level authorization policies
- **Telemetry**: Distributed tracing across microservices
## License
AGPL-3.0-or-later