Add unit tests for RabbitMq and Udp transport servers and clients
Some checks failed
Docs CI / lint-and-preview (push) Has been cancelled
Some checks failed
Docs CI / lint-and-preview (push) Has been cancelled
- Implemented comprehensive unit tests for RabbitMqTransportServer, covering constructor, disposal, connection management, event handlers, and exception handling. - Added configuration tests for RabbitMqTransportServer to validate SSL, durable queues, auto-recovery, and custom virtual host options. - Created unit tests for UdpFrameProtocol, including frame parsing and serialization, header size validation, and round-trip data preservation. - Developed tests for UdpTransportClient, focusing on connection handling, event subscriptions, and exception scenarios. - Established tests for UdpTransportServer, ensuring proper start/stop behavior, connection state management, and event handling. - Included tests for UdpTransportOptions to verify default values and modification capabilities. - Enhanced service registration tests for Udp transport services in the dependency injection container.
This commit is contained in:
325
docs/modules/gateway/openapi.md
Normal file
325
docs/modules/gateway/openapi.md
Normal file
@@ -0,0 +1,325 @@
|
||||
# Gateway OpenAPI Implementation
|
||||
|
||||
This document describes the implementation architecture of OpenAPI document aggregation in the StellaOps Router Gateway.
|
||||
|
||||
## Architecture
|
||||
|
||||
The Gateway generates OpenAPI 3.1.0 documentation by aggregating schemas and endpoint metadata from connected microservices.
|
||||
|
||||
### Component Overview
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────────┐
|
||||
│ Gateway │
|
||||
├─────────────────────────────────────────────────────────────────────┤
|
||||
│ │
|
||||
│ ┌──────────────────┐ ┌────────────────────┐ │
|
||||
│ │ ConnectionManager │───►│ InMemoryRoutingState│ │
|
||||
│ │ │ │ │ │
|
||||
│ │ - OnHelloReceived │ │ - Connections[] │ │
|
||||
│ │ - OnConnClosed │ │ - Endpoints │ │
|
||||
│ └──────────────────┘ │ - Schemas │ │
|
||||
│ │ └─────────┬──────────┘ │
|
||||
│ │ │ │
|
||||
│ ▼ ▼ │
|
||||
│ ┌──────────────────┐ ┌────────────────────┐ │
|
||||
│ │ OpenApiDocument │◄───│ GatewayOpenApi │ │
|
||||
│ │ Cache │ │ DocumentCache │ │
|
||||
│ │ │ │ │ │
|
||||
│ │ - Invalidate() │ │ - TTL expiration │ │
|
||||
│ └──────────────────┘ │ - ETag generation │ │
|
||||
│ └─────────┬──────────┘ │
|
||||
│ │ │
|
||||
│ ▼ │
|
||||
│ ┌────────────────────┐ │
|
||||
│ │ OpenApiDocument │ │
|
||||
│ │ Generator │ │
|
||||
│ │ │ │
|
||||
│ │ - GenerateInfo() │ │
|
||||
│ │ - GeneratePaths() │ │
|
||||
│ │ - GenerateTags() │ │
|
||||
│ │ - GenerateSchemas()│ │
|
||||
│ └─────────┬──────────┘ │
|
||||
│ │ │
|
||||
│ ▼ │
|
||||
│ ┌────────────────────┐ │
|
||||
│ │ ClaimSecurity │ │
|
||||
│ │ Mapper │ │
|
||||
│ │ │ │
|
||||
│ │ - SecuritySchemes │ │
|
||||
│ │ - SecurityRequire │ │
|
||||
│ └────────────────────┘ │
|
||||
│ │
|
||||
└─────────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
### Components
|
||||
|
||||
| Component | File | Responsibility |
|
||||
|-----------|------|----------------|
|
||||
| `IOpenApiDocumentGenerator` | `OpenApi/IOpenApiDocumentGenerator.cs` | Interface for document generation |
|
||||
| `OpenApiDocumentGenerator` | `OpenApi/OpenApiDocumentGenerator.cs` | Builds OpenAPI 3.1.0 JSON |
|
||||
| `IGatewayOpenApiDocumentCache` | `OpenApi/IGatewayOpenApiDocumentCache.cs` | Interface for document caching |
|
||||
| `GatewayOpenApiDocumentCache` | `OpenApi/GatewayOpenApiDocumentCache.cs` | TTL + invalidation caching |
|
||||
| `ClaimSecurityMapper` | `OpenApi/ClaimSecurityMapper.cs` | Maps claims to OAuth2 scopes |
|
||||
| `OpenApiEndpoints` | `OpenApi/OpenApiEndpoints.cs` | HTTP endpoint handlers |
|
||||
| `OpenApiAggregationOptions` | `OpenApi/OpenApiAggregationOptions.cs` | Configuration options |
|
||||
|
||||
---
|
||||
|
||||
## OpenApiDocumentGenerator
|
||||
|
||||
Generates the complete OpenAPI 3.1.0 document from routing state.
|
||||
|
||||
### Process Flow
|
||||
|
||||
1. **Collect connections** from `IGlobalRoutingState`
|
||||
2. **Generate info** section from `OpenApiAggregationOptions`
|
||||
3. **Generate paths** by iterating all endpoints across connections
|
||||
4. **Generate components** including schemas and security schemes
|
||||
5. **Generate tags** from unique service names
|
||||
|
||||
### Schema Handling
|
||||
|
||||
Schemas are prefixed with service name to avoid naming conflicts:
|
||||
|
||||
```csharp
|
||||
var prefixedId = $"{conn.Instance.ServiceName}_{schemaId}";
|
||||
// billing_CreateInvoiceRequest
|
||||
```
|
||||
|
||||
### Operation ID Generation
|
||||
|
||||
Operation IDs follow a consistent pattern:
|
||||
|
||||
```csharp
|
||||
var operationId = $"{serviceName}_{path}_{method}";
|
||||
// billing_invoices_POST
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## GatewayOpenApiDocumentCache
|
||||
|
||||
Implements caching with TTL expiration and content-based ETags.
|
||||
|
||||
### Cache Behavior
|
||||
|
||||
| Trigger | Action |
|
||||
|---------|--------|
|
||||
| First request | Generate and cache document |
|
||||
| Subsequent requests (within TTL) | Return cached document |
|
||||
| TTL expired | Regenerate document |
|
||||
| Connection added/removed | Invalidate cache |
|
||||
|
||||
### ETag Generation
|
||||
|
||||
ETags are computed from SHA256 hash of document content:
|
||||
|
||||
```csharp
|
||||
var hash = SHA256.HashData(Encoding.UTF8.GetBytes(documentJson));
|
||||
var etag = $"\"{Convert.ToHexString(hash)[..16]}\"";
|
||||
```
|
||||
|
||||
### Thread Safety
|
||||
|
||||
The cache uses locking to ensure thread-safe regeneration:
|
||||
|
||||
```csharp
|
||||
lock (_lock)
|
||||
{
|
||||
if (_cachedDocument is null || IsExpired())
|
||||
{
|
||||
RegenerateDocument();
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## ClaimSecurityMapper
|
||||
|
||||
Maps endpoint claim requirements to OpenAPI security schemes.
|
||||
|
||||
### Security Scheme Generation
|
||||
|
||||
Always generates `BearerAuth` scheme. Generates `OAuth2` scheme only when endpoints have claim requirements:
|
||||
|
||||
```csharp
|
||||
public static JsonObject GenerateSecuritySchemes(
|
||||
IEnumerable<EndpointDescriptor> endpoints,
|
||||
string tokenUrl)
|
||||
{
|
||||
var schemes = new JsonObject();
|
||||
|
||||
// Always add BearerAuth
|
||||
schemes["BearerAuth"] = new JsonObject { ... };
|
||||
|
||||
// Collect scopes from all endpoints
|
||||
var scopes = CollectScopes(endpoints);
|
||||
|
||||
// Add OAuth2 only if scopes exist
|
||||
if (scopes.Count > 0)
|
||||
{
|
||||
schemes["OAuth2"] = GenerateOAuth2Scheme(tokenUrl, scopes);
|
||||
}
|
||||
|
||||
return schemes;
|
||||
}
|
||||
```
|
||||
|
||||
### Per-Operation Security
|
||||
|
||||
Each endpoint with claims gets a security requirement:
|
||||
|
||||
```csharp
|
||||
public static JsonArray GenerateSecurityRequirement(EndpointDescriptor endpoint)
|
||||
{
|
||||
if (endpoint.RequiringClaims.Count == 0)
|
||||
return new JsonArray(); // No security required
|
||||
|
||||
return new JsonArray
|
||||
{
|
||||
new JsonObject
|
||||
{
|
||||
["BearerAuth"] = new JsonArray(),
|
||||
["OAuth2"] = new JsonArray(claims.Select(c => c.Type))
|
||||
}
|
||||
};
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Configuration Reference
|
||||
|
||||
### OpenApiAggregationOptions
|
||||
|
||||
| Property | Type | Default | Description |
|
||||
|----------|------|---------|-------------|
|
||||
| `Title` | `string` | `"StellaOps Gateway API"` | API title |
|
||||
| `Description` | `string` | `"Unified API..."` | API description |
|
||||
| `Version` | `string` | `"1.0.0"` | API version |
|
||||
| `ServerUrl` | `string` | `"/"` | Base server URL |
|
||||
| `CacheTtlSeconds` | `int` | `60` | Cache TTL |
|
||||
| `Enabled` | `bool` | `true` | Enable/disable |
|
||||
| `LicenseName` | `string` | `"AGPL-3.0-or-later"` | License name |
|
||||
| `ContactName` | `string?` | `null` | Contact name |
|
||||
| `ContactEmail` | `string?` | `null` | Contact email |
|
||||
| `TokenUrl` | `string` | `"/auth/token"` | OAuth2 token URL |
|
||||
|
||||
### YAML Configuration
|
||||
|
||||
```yaml
|
||||
OpenApi:
|
||||
Title: "My Gateway API"
|
||||
Description: "Unified API for all microservices"
|
||||
Version: "2.0.0"
|
||||
ServerUrl: "https://api.example.com"
|
||||
CacheTtlSeconds: 60
|
||||
Enabled: true
|
||||
LicenseName: "AGPL-3.0-or-later"
|
||||
ContactName: "API Team"
|
||||
ContactEmail: "api@example.com"
|
||||
TokenUrl: "/auth/token"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Service Registration
|
||||
|
||||
Services are registered via dependency injection in `ServiceCollectionExtensions`:
|
||||
|
||||
```csharp
|
||||
services.Configure<OpenApiAggregationOptions>(
|
||||
configuration.GetSection("OpenApi"));
|
||||
services.AddSingleton<IOpenApiDocumentGenerator, OpenApiDocumentGenerator>();
|
||||
services.AddSingleton<IGatewayOpenApiDocumentCache, GatewayOpenApiDocumentCache>();
|
||||
```
|
||||
|
||||
Endpoints are mapped in `ApplicationBuilderExtensions`:
|
||||
|
||||
```csharp
|
||||
app.MapGatewayOpenApiEndpoints();
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Cache Invalidation
|
||||
|
||||
The `ConnectionManager` invalidates the cache on connection changes:
|
||||
|
||||
```csharp
|
||||
private Task HandleHelloReceivedAsync(ConnectionState state, HelloPayload payload)
|
||||
{
|
||||
_routingState.AddConnection(state);
|
||||
_openApiCache?.Invalidate(); // Invalidate on new connection
|
||||
return Task.CompletedTask;
|
||||
}
|
||||
|
||||
private Task HandleConnectionClosedAsync(string connectionId)
|
||||
{
|
||||
_routingState.RemoveConnection(connectionId);
|
||||
_openApiCache?.Invalidate(); // Invalidate on disconnect
|
||||
return Task.CompletedTask;
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Extension Points
|
||||
|
||||
### Custom Routing Plugins
|
||||
|
||||
The Gateway supports custom routing plugins via `IRoutingPlugin`. While not directly related to OpenAPI, routing decisions can affect which endpoints are exposed.
|
||||
|
||||
### Future Enhancements
|
||||
|
||||
Potential extension points for future development:
|
||||
|
||||
- **Schema Transformers**: Modify schemas before aggregation
|
||||
- **Tag Customization**: Custom tag generation logic
|
||||
- **Response Examples**: Include example responses from connected services
|
||||
- **Webhooks**: Notify external systems on document changes
|
||||
|
||||
---
|
||||
|
||||
## Testing
|
||||
|
||||
Unit tests are located in `src/Gateway/__Tests/StellaOps.Gateway.WebService.Tests/OpenApi/`:
|
||||
|
||||
| Test File | Coverage |
|
||||
|-----------|----------|
|
||||
| `OpenApiDocumentGeneratorTests.cs` | Document structure, schema merging, tag generation |
|
||||
| `GatewayOpenApiDocumentCacheTests.cs` | TTL expiry, invalidation, ETag consistency |
|
||||
| `ClaimSecurityMapperTests.cs` | Security scheme generation from claims |
|
||||
|
||||
### Test Patterns
|
||||
|
||||
```csharp
|
||||
[Fact]
|
||||
public void GenerateDocument_WithConnections_GeneratesPaths()
|
||||
{
|
||||
// Arrange
|
||||
var endpoint = new EndpointDescriptor { ... };
|
||||
var connection = CreateConnection("inventory", "1.0.0", endpoint);
|
||||
_routingState.Setup(x => x.GetAllConnections()).Returns([connection]);
|
||||
|
||||
// Act
|
||||
var document = _sut.GenerateDocument();
|
||||
|
||||
// Assert
|
||||
var doc = JsonDocument.Parse(document);
|
||||
doc.RootElement.GetProperty("paths")
|
||||
.TryGetProperty("/api/items", out _)
|
||||
.Should().BeTrue();
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## See Also
|
||||
|
||||
- [Schema Validation](../router/schema-validation.md) - JSON Schema validation in microservices
|
||||
- [OpenAPI Aggregation](../router/openapi-aggregation.md) - Configuration and usage guide
|
||||
- [API Overview](../../api/overview.md) - General API conventions
|
||||
165
docs/modules/router/README.md
Normal file
165
docs/modules/router/README.md
Normal file
@@ -0,0 +1,165 @@
|
||||
# Router Module
|
||||
|
||||
The StellaOps Router is the internal communication infrastructure that enables microservices to communicate through a central gateway using efficient binary protocols.
|
||||
|
||||
## Why Another Gateway?
|
||||
|
||||
StellaOps already has HTTP-based services. The Router exists because:
|
||||
|
||||
1. **Performance**: Binary framing eliminates HTTP overhead for internal traffic
|
||||
2. **Streaming**: First-class support for large payloads (SBOMs, scan results, evidence bundles)
|
||||
3. **Cancellation**: Request abortion propagates across service boundaries
|
||||
4. **Health-aware Routing**: Automatic failover based on heartbeat and latency
|
||||
5. **Claims-based Auth**: Unified authorization via Authority integration
|
||||
6. **Transport Flexibility**: UDP for small payloads, TCP/TLS for streams, RabbitMQ for queuing
|
||||
|
||||
The Router replaces the Serdica HTTP-to-RabbitMQ pattern with a simpler, generic design.
|
||||
|
||||
## Architecture Overview
|
||||
|
||||
```
|
||||
┌─────────────────────────────────┐
|
||||
│ StellaOps.Gateway.WebService│
|
||||
HTTP Clients ────────────────────► (HTTP ingress) │
|
||||
│ │
|
||||
│ ┌─────────────────────────────┐│
|
||||
│ │ Endpoint Resolution ││
|
||||
│ │ Authorization (Claims) ││
|
||||
│ │ Routing Decision ││
|
||||
│ │ Transport Dispatch ││
|
||||
│ └─────────────────────────────┘│
|
||||
└──────────────┬──────────────────┘
|
||||
│
|
||||
┌─────────────────────────┼─────────────────────────┐
|
||||
│ │ │
|
||||
▼ ▼ ▼
|
||||
┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐
|
||||
│ Billing │ │ Inventory │ │ Scanner │
|
||||
│ Microservice │ │ Microservice │ │ Microservice │
|
||||
│ │ │ │ │ │
|
||||
│ TCP/TLS │ │ InMemory │ │ RabbitMQ │
|
||||
└─────────────────┘ └─────────────────┘ └─────────────────┘
|
||||
```
|
||||
|
||||
## Components
|
||||
|
||||
| Component | Project | Purpose |
|
||||
|-----------|---------|---------|
|
||||
| Gateway | `StellaOps.Gateway.WebService` | HTTP ingress, routing, authorization |
|
||||
| Microservice SDK | `StellaOps.Microservice` | SDK for building microservices |
|
||||
| Source Generator | `StellaOps.Microservice.SourceGen` | Compile-time endpoint discovery |
|
||||
| Common | `StellaOps.Router.Common` | Shared types, frames, interfaces |
|
||||
| Config | `StellaOps.Router.Config` | Configuration models, YAML binding |
|
||||
| InMemory Transport | `StellaOps.Router.Transport.InMemory` | Testing transport |
|
||||
| TCP Transport | `StellaOps.Router.Transport.Tcp` | Production TCP transport |
|
||||
| TLS Transport | `StellaOps.Router.Transport.Tls` | Encrypted TCP transport |
|
||||
| UDP Transport | `StellaOps.Router.Transport.Udp` | Small payload transport |
|
||||
| RabbitMQ Transport | `StellaOps.Router.Transport.RabbitMQ` | Message queue transport |
|
||||
|
||||
## Solution Structure
|
||||
|
||||
```
|
||||
StellaOps.Router.slnx
|
||||
├── src/__Libraries/
|
||||
│ ├── StellaOps.Router.Common/
|
||||
│ ├── StellaOps.Router.Config/
|
||||
│ ├── StellaOps.Router.Transport.InMemory/
|
||||
│ ├── StellaOps.Router.Transport.Tcp/
|
||||
│ ├── StellaOps.Router.Transport.Tls/
|
||||
│ ├── StellaOps.Router.Transport.Udp/
|
||||
│ ├── StellaOps.Router.Transport.RabbitMQ/
|
||||
│ ├── StellaOps.Microservice/
|
||||
│ └── StellaOps.Microservice.SourceGen/
|
||||
├── src/Gateway/
|
||||
│ └── StellaOps.Gateway.WebService/
|
||||
└── tests/
|
||||
└── (test projects)
|
||||
```
|
||||
|
||||
## Key Documents
|
||||
|
||||
| Document | Purpose |
|
||||
|----------|---------|
|
||||
| [architecture.md](architecture.md) | Canonical specification and requirements |
|
||||
| [schema-validation.md](schema-validation.md) | JSON Schema validation feature |
|
||||
| [openapi-aggregation.md](openapi-aggregation.md) | OpenAPI document generation |
|
||||
| [migration-guide.md](migration-guide.md) | WebService to Microservice migration |
|
||||
|
||||
## Quick Start
|
||||
|
||||
### Gateway
|
||||
|
||||
```csharp
|
||||
var builder = WebApplication.CreateBuilder(args);
|
||||
|
||||
// Add router services
|
||||
builder.Services.AddGatewayServices(builder.Configuration);
|
||||
builder.Services.AddInMemoryTransport(); // or TCP, TLS, etc.
|
||||
|
||||
var app = builder.Build();
|
||||
|
||||
// Configure pipeline
|
||||
app.UseGatewayMiddleware();
|
||||
|
||||
await app.RunAsync();
|
||||
```
|
||||
|
||||
### Microservice
|
||||
|
||||
```csharp
|
||||
var builder = Host.CreateApplicationBuilder(args);
|
||||
|
||||
builder.Services.AddStellaMicroservice(options =>
|
||||
{
|
||||
options.ServiceName = "billing";
|
||||
options.Version = "1.0.0";
|
||||
options.Region = "us-east-1";
|
||||
});
|
||||
|
||||
builder.Services.AddInMemoryTransportClient();
|
||||
|
||||
await builder.Build().RunAsync();
|
||||
```
|
||||
|
||||
### Endpoint Definition
|
||||
|
||||
```csharp
|
||||
[StellaEndpoint("POST", "/invoices")]
|
||||
[ValidateSchema(Summary = "Create invoice")]
|
||||
public sealed class CreateInvoiceEndpoint : IStellaEndpoint<CreateInvoiceRequest, CreateInvoiceResponse>
|
||||
{
|
||||
public Task<CreateInvoiceResponse> HandleAsync(
|
||||
CreateInvoiceRequest request,
|
||||
CancellationToken ct)
|
||||
{
|
||||
return Task.FromResult(new CreateInvoiceResponse
|
||||
{
|
||||
InvoiceId = Guid.NewGuid().ToString()
|
||||
});
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Invariants
|
||||
|
||||
These are non-negotiable design constraints:
|
||||
|
||||
- **Method + Path** is the endpoint identity
|
||||
- **Strict semver** for version matching
|
||||
- **Region from GatewayNodeConfig** (not headers/host)
|
||||
- **No HTTP transport** between gateway and microservices
|
||||
- **RequiringClaims** (not AllowedRoles) for authorization
|
||||
- **Opaque body handling** (router doesn't interpret payloads)
|
||||
|
||||
## Building
|
||||
|
||||
```bash
|
||||
# Build router solution
|
||||
dotnet build StellaOps.Router.slnx
|
||||
|
||||
# Run tests
|
||||
dotnet test StellaOps.Router.slnx
|
||||
|
||||
# Run gateway
|
||||
dotnet run --project src/Gateway/StellaOps.Gateway.WebService
|
||||
```
|
||||
519
docs/modules/router/architecture.md
Normal file
519
docs/modules/router/architecture.md
Normal file
@@ -0,0 +1,519 @@
|
||||
# Router Architecture
|
||||
|
||||
This document is the canonical specification for the StellaOps Router system.
|
||||
|
||||
## System Architecture
|
||||
|
||||
### Scope
|
||||
|
||||
- A single HTTP ingress service (`StellaOps.Gateway.WebService`) handles all external HTTP traffic
|
||||
- Microservices communicate with the Gateway using binary transports (TCP, TLS, UDP, RabbitMQ)
|
||||
- HTTP is not used for internal microservice-to-gateway traffic
|
||||
- Request/response bodies are opaque to the router (raw bytes/streams)
|
||||
|
||||
### Transport Architecture
|
||||
|
||||
Each transport connection carries:
|
||||
|
||||
- Initial registration (HELLO) and endpoint configuration
|
||||
- Ongoing heartbeats
|
||||
- Request/response data frames
|
||||
- Streaming data frames
|
||||
- Cancellation frames
|
||||
|
||||
```
|
||||
┌─────────────────┐ ┌─────────────────┐
|
||||
│ Microservice │ │ Gateway │
|
||||
│ │ HELLO │ │
|
||||
│ Endpoints: │ ─────────────────────────►│ Routing │
|
||||
│ - POST /items │ HEARTBEAT │ State │
|
||||
│ - GET /items │ ◄────────────────────────►│ │
|
||||
│ │ │ Connections[] │
|
||||
│ │ REQUEST / RESPONSE │ │
|
||||
│ │ ◄────────────────────────►│ │
|
||||
│ │ │ │
|
||||
│ │ STREAM_DATA / CANCEL │ │
|
||||
│ │ ◄────────────────────────►│ │
|
||||
└─────────────────┘ └─────────────────┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Service Identity
|
||||
|
||||
### Instance Identity
|
||||
|
||||
Each microservice instance is identified by:
|
||||
|
||||
| Field | Type | Description |
|
||||
|-------|------|-------------|
|
||||
| `ServiceName` | string | Logical service name (e.g., "billing") |
|
||||
| `Version` | string | Semantic version (`major.minor.patch`) |
|
||||
| `Region` | string | Deployment region (e.g., "us-east-1") |
|
||||
| `InstanceId` | string | Unique instance identifier |
|
||||
|
||||
### Version Matching
|
||||
|
||||
- Version matching is strict semver equality
|
||||
- Router only routes to instances with exact version match
|
||||
- Default version used when client doesn't specify
|
||||
|
||||
### Region Configuration
|
||||
|
||||
Gateway region comes from `GatewayNodeConfig`:
|
||||
|
||||
```csharp
|
||||
public sealed class GatewayNodeConfig
|
||||
{
|
||||
public required string Region { get; init; } // e.g., "eu1"
|
||||
public required string NodeId { get; init; } // e.g., "gw-eu1-01"
|
||||
public required string Environment { get; init; } // e.g., "prod"
|
||||
}
|
||||
```
|
||||
|
||||
Region is never derived from HTTP headers or URL hostnames.
|
||||
|
||||
---
|
||||
|
||||
## Endpoint Model
|
||||
|
||||
### Endpoint Identity
|
||||
|
||||
Endpoint identity is `(HTTP Method, Path)`:
|
||||
|
||||
| Field | Example |
|
||||
|-------|---------|
|
||||
| Method | `GET`, `POST`, `PUT`, `PATCH`, `DELETE` |
|
||||
| Path | `/invoices`, `/items/{id}`, `/users/{userId}/orders` |
|
||||
|
||||
### Endpoint Descriptor
|
||||
|
||||
Each endpoint includes:
|
||||
|
||||
```csharp
|
||||
public sealed class EndpointDescriptor
|
||||
{
|
||||
public required string Method { get; init; }
|
||||
public required string Path { get; init; }
|
||||
public required string ServiceName { get; init; }
|
||||
public required string Version { get; init; }
|
||||
public TimeSpan DefaultTimeout { get; init; }
|
||||
public bool SupportsStreaming { get; init; }
|
||||
public IReadOnlyList<ClaimRequirement> RequiringClaims { get; init; } = [];
|
||||
public EndpointSchemaInfo? SchemaInfo { get; init; }
|
||||
}
|
||||
```
|
||||
|
||||
### Path Matching
|
||||
|
||||
- ASP.NET-style route templates
|
||||
- Parameter segments: `{id}`, `{userId}`
|
||||
- Case sensitivity and trailing slash handling follow ASP.NET conventions
|
||||
|
||||
---
|
||||
|
||||
## Routing Algorithm
|
||||
|
||||
### Instance Selection
|
||||
|
||||
Given `(ServiceName, Version, Method, Path)`:
|
||||
|
||||
1. **Filter candidates**:
|
||||
- Match `ServiceName` exactly
|
||||
- Match `Version` exactly (strict semver)
|
||||
- Health status in acceptable set (`Healthy` or `Degraded`)
|
||||
|
||||
2. **Region preference**:
|
||||
- Prefer instances where `Region == GatewayNodeConfig.Region`
|
||||
- Fall back to configured neighbor regions
|
||||
- Fall back to all other regions
|
||||
|
||||
3. **Within region tier**:
|
||||
- Prefer lower `AveragePingMs`
|
||||
- If tied, prefer more recent `LastHeartbeatUtc`
|
||||
- If still tied, use round-robin balancing
|
||||
|
||||
### Instance Health
|
||||
|
||||
```csharp
|
||||
public enum InstanceHealthStatus
|
||||
{
|
||||
Unknown,
|
||||
Healthy,
|
||||
Degraded,
|
||||
Draining,
|
||||
Unhealthy
|
||||
}
|
||||
```
|
||||
|
||||
Health metadata per connection:
|
||||
|
||||
| Field | Type | Description |
|
||||
|-------|------|-------------|
|
||||
| `Status` | enum | Current health status |
|
||||
| `LastHeartbeatUtc` | DateTime | Last heartbeat timestamp |
|
||||
| `AveragePingMs` | double | Average round-trip latency |
|
||||
|
||||
---
|
||||
|
||||
## Transport Layer
|
||||
|
||||
### Transport Types
|
||||
|
||||
| Transport | Use Case | Streaming | Notes |
|
||||
|-----------|----------|-----------|-------|
|
||||
| InMemory | Testing | Yes | In-process channels |
|
||||
| TCP | Production | Yes | Length-prefixed frames |
|
||||
| TLS | Secure | Yes | Certificate-based encryption |
|
||||
| UDP | Small payloads | No | Single datagram per frame |
|
||||
| RabbitMQ | Queuing | Yes | Exchange/queue routing |
|
||||
|
||||
### Transport Plugin Interface
|
||||
|
||||
```csharp
|
||||
public interface ITransportServer
|
||||
{
|
||||
Task StartAsync(CancellationToken ct);
|
||||
Task StopAsync(CancellationToken ct);
|
||||
event Func<ConnectionState, HelloPayload, Task> OnHelloReceived;
|
||||
event Func<ConnectionState, HeartbeatPayload, Task> OnHeartbeatReceived;
|
||||
event Func<string, Task> OnConnectionClosed;
|
||||
}
|
||||
|
||||
public interface ITransportClient
|
||||
{
|
||||
Task ConnectAsync(CancellationToken ct);
|
||||
Task DisconnectAsync(CancellationToken ct);
|
||||
Task SendFrameAsync(Frame frame, CancellationToken ct);
|
||||
}
|
||||
```
|
||||
|
||||
### Frame Types
|
||||
|
||||
```csharp
|
||||
public enum FrameType : byte
|
||||
{
|
||||
Hello = 1,
|
||||
Heartbeat = 2,
|
||||
Request = 3,
|
||||
Response = 4,
|
||||
RequestStreamData = 5,
|
||||
ResponseStreamData = 6,
|
||||
Cancel = 7
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Gateway Pipeline
|
||||
|
||||
### HTTP Middleware Stack
|
||||
|
||||
```
|
||||
Request ─►│ ForwardedHeaders │
|
||||
│ RequestLogging │
|
||||
│ ErrorHandling │
|
||||
│ Authentication │
|
||||
│ EndpointResolution │ ◄── (Method, Path) → EndpointDescriptor
|
||||
│ Authorization │ ◄── RequiringClaims check
|
||||
│ RoutingDecision │ ◄── Select connection/instance
|
||||
│ TransportDispatch │ ◄── Send to microservice
|
||||
▼
|
||||
```
|
||||
|
||||
### Connection State
|
||||
|
||||
Per-connection state maintained by Gateway:
|
||||
|
||||
```csharp
|
||||
public sealed class ConnectionState
|
||||
{
|
||||
public required string ConnectionId { get; init; }
|
||||
public required InstanceDescriptor Instance { get; init; }
|
||||
public InstanceHealthStatus Status { get; set; }
|
||||
public DateTime? LastHeartbeatUtc { get; set; }
|
||||
public double AveragePingMs { get; set; }
|
||||
public TransportType TransportType { get; init; }
|
||||
public Dictionary<(string Method, string Path), EndpointDescriptor> Endpoints { get; } = new();
|
||||
public IReadOnlyDictionary<string, SchemaDefinition> Schemas { get; init; } = new Dictionary<string, SchemaDefinition>();
|
||||
}
|
||||
```
|
||||
|
||||
### Payload Handling
|
||||
|
||||
The Gateway treats bodies as opaque byte sequences:
|
||||
|
||||
- No deserialization or schema interpretation
|
||||
- Headers and bytes forwarded as-is
|
||||
- Schema validation is microservice responsibility
|
||||
|
||||
### Payload Limits
|
||||
|
||||
Configurable limits protect against resource exhaustion:
|
||||
|
||||
| Limit | Scope |
|
||||
|-------|-------|
|
||||
| `MaxRequestBytesPerCall` | Single request |
|
||||
| `MaxRequestBytesPerConnection` | All requests on connection |
|
||||
| `MaxAggregateInflightBytes` | All in-flight across gateway |
|
||||
|
||||
Exceeded limits result in:
|
||||
- Early rejection (HTTP 413) if `Content-Length` known
|
||||
- Mid-stream abort with CANCEL frame
|
||||
- Appropriate error response (413 or 503)
|
||||
|
||||
---
|
||||
|
||||
## Microservice SDK
|
||||
|
||||
### Configuration
|
||||
|
||||
```csharp
|
||||
services.AddStellaMicroservice(options =>
|
||||
{
|
||||
options.ServiceName = "billing";
|
||||
options.Version = "1.0.0";
|
||||
options.Region = "us-east-1";
|
||||
options.InstanceId = Guid.NewGuid().ToString();
|
||||
options.ServiceDescription = "Invoice processing service";
|
||||
});
|
||||
```
|
||||
|
||||
### Endpoint Declaration
|
||||
|
||||
Attributes:
|
||||
|
||||
```csharp
|
||||
[StellaEndpoint("POST", "/invoices")]
|
||||
public sealed class CreateInvoiceEndpoint : IStellaEndpoint<CreateInvoiceRequest, CreateInvoiceResponse>
|
||||
```
|
||||
|
||||
### Handler Interfaces
|
||||
|
||||
**Typed handler** (JSON serialization):
|
||||
|
||||
```csharp
|
||||
public interface IStellaEndpoint<TRequest, TResponse>
|
||||
{
|
||||
Task<TResponse> HandleAsync(TRequest request, CancellationToken ct);
|
||||
}
|
||||
|
||||
public interface IStellaEndpoint<TResponse>
|
||||
{
|
||||
Task<TResponse> HandleAsync(CancellationToken ct);
|
||||
}
|
||||
```
|
||||
|
||||
**Raw handler** (streaming):
|
||||
|
||||
```csharp
|
||||
public interface IRawStellaEndpoint
|
||||
{
|
||||
Task<RawResponse> HandleAsync(RawRequestContext ctx, CancellationToken ct);
|
||||
}
|
||||
```
|
||||
|
||||
### Endpoint Discovery
|
||||
|
||||
Two mechanisms:
|
||||
|
||||
1. **Source Generator** (preferred): Compile-time discovery via Roslyn
|
||||
2. **Reflection** (fallback): Runtime assembly scanning
|
||||
|
||||
### Connection Behavior
|
||||
|
||||
On connection:
|
||||
1. Send HELLO with instance info and endpoints
|
||||
2. Start heartbeat timer
|
||||
3. Listen for REQUEST frames
|
||||
|
||||
HELLO payload:
|
||||
|
||||
```csharp
|
||||
public sealed class HelloPayload
|
||||
{
|
||||
public required InstanceDescriptor Instance { get; init; }
|
||||
public required IReadOnlyList<EndpointDescriptor> Endpoints { get; init; }
|
||||
public IReadOnlyDictionary<string, SchemaDefinition> Schemas { get; init; } = new Dictionary<string, SchemaDefinition>();
|
||||
public ServiceOpenApiInfo? OpenApiInfo { get; init; }
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Authorization
|
||||
|
||||
### Claims-based Model
|
||||
|
||||
Authorization uses `RequiringClaims`, not roles:
|
||||
|
||||
```csharp
|
||||
public sealed class ClaimRequirement
|
||||
{
|
||||
public required string Type { get; init; }
|
||||
public string? Value { get; init; }
|
||||
}
|
||||
```
|
||||
|
||||
### Precedence
|
||||
|
||||
1. Microservice provides defaults in HELLO
|
||||
2. Authority can override centrally
|
||||
3. Gateway enforces final effective claims
|
||||
|
||||
### Enforcement
|
||||
|
||||
Gateway `AuthorizationMiddleware`:
|
||||
- Validates user principal has all required claims
|
||||
- Empty claims list = authenticated access only
|
||||
- Missing claim = 403 Forbidden
|
||||
|
||||
---
|
||||
|
||||
## Cancellation
|
||||
|
||||
### CANCEL Frame
|
||||
|
||||
```csharp
|
||||
public sealed class CancelPayload
|
||||
{
|
||||
public required string Reason { get; init; }
|
||||
// Values: "ClientDisconnected", "Timeout", "PayloadLimitExceeded", "Shutdown"
|
||||
}
|
||||
```
|
||||
|
||||
### Gateway sends CANCEL when:
|
||||
|
||||
- HTTP client disconnects (`HttpContext.RequestAborted`)
|
||||
- Request timeout elapses
|
||||
- Payload limit exceeded
|
||||
- Gateway shutdown
|
||||
|
||||
### Microservice handles CANCEL:
|
||||
|
||||
- Maps correlation ID to `CancellationTokenSource`
|
||||
- Calls `Cancel()` on the source
|
||||
- Handler receives cancellation via `CancellationToken`
|
||||
|
||||
---
|
||||
|
||||
## Streaming
|
||||
|
||||
### Buffered vs Streaming
|
||||
|
||||
| Mode | Request Body | Response Body | Use Case |
|
||||
|------|--------------|---------------|----------|
|
||||
| Buffered | Full in memory | Full in memory | Small payloads |
|
||||
| Streaming | Chunked frames | Chunked frames | Large payloads |
|
||||
|
||||
### Frame Flow (Streaming)
|
||||
|
||||
```
|
||||
Gateway Microservice
|
||||
│ │
|
||||
│ REQUEST (headers only) │
|
||||
│ ────────────────────────────────────►│
|
||||
│ │
|
||||
│ REQUEST_STREAM_DATA (chunk 1) │
|
||||
│ ────────────────────────────────────►│
|
||||
│ │
|
||||
│ REQUEST_STREAM_DATA (chunk n) │
|
||||
│ ────────────────────────────────────►│
|
||||
│ │
|
||||
│ REQUEST_STREAM_DATA (final=true) │
|
||||
│ ────────────────────────────────────►│
|
||||
│ │
|
||||
│ RESPONSE │
|
||||
│◄────────────────────────────────────│
|
||||
│ │
|
||||
│ RESPONSE_STREAM_DATA │
|
||||
│◄────────────────────────────────────│
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Heartbeat & Health
|
||||
|
||||
### Heartbeat Frame
|
||||
|
||||
Sent at regular intervals over the same connection as requests:
|
||||
|
||||
```csharp
|
||||
public sealed class HeartbeatPayload
|
||||
{
|
||||
public required InstanceHealthStatus Status { get; init; }
|
||||
public int InflightRequests { get; init; }
|
||||
public double ErrorRate { get; init; }
|
||||
}
|
||||
```
|
||||
|
||||
### Health Tracking
|
||||
|
||||
Gateway tracks:
|
||||
- `LastHeartbeatUtc` per connection
|
||||
- Derives status from heartbeat recency
|
||||
- Marks stale instances as Unhealthy
|
||||
- Uses health in routing decisions
|
||||
|
||||
---
|
||||
|
||||
## Configuration
|
||||
|
||||
### Router YAML
|
||||
|
||||
```yaml
|
||||
# router.yaml
|
||||
Gateway:
|
||||
Region: "us-east-1"
|
||||
NodeId: "gw-east-01"
|
||||
Environment: "production"
|
||||
|
||||
PayloadLimits:
|
||||
MaxRequestBytesPerCall: 10485760 # 10 MB
|
||||
MaxRequestBytesPerConnection: 104857600 # 100 MB
|
||||
MaxAggregateInflightBytes: 1073741824 # 1 GB
|
||||
|
||||
Services:
|
||||
- ServiceName: billing
|
||||
DefaultVersion: "1.0.0"
|
||||
DefaultTransport: Tcp
|
||||
Endpoints:
|
||||
- Method: POST
|
||||
Path: /invoices
|
||||
TimeoutSeconds: 30
|
||||
RequiringClaims:
|
||||
- Type: "invoices:write"
|
||||
|
||||
OpenApi:
|
||||
Title: "StellaOps Gateway API"
|
||||
CacheTtlSeconds: 60
|
||||
```
|
||||
|
||||
### Hot Reload
|
||||
|
||||
- YAML changes picked up at runtime
|
||||
- Routing state updated without restart
|
||||
- New services/endpoints added dynamically
|
||||
|
||||
---
|
||||
|
||||
## Error Mapping
|
||||
|
||||
| Condition | HTTP Status |
|
||||
|-----------|-------------|
|
||||
| Version not found | 404 Not Found |
|
||||
| No healthy instance | 503 Service Unavailable |
|
||||
| Request timeout | 504 Gateway Timeout |
|
||||
| Payload too large | 413 Payload Too Large |
|
||||
| Unauthorized | 401 Unauthorized |
|
||||
| Missing claims | 403 Forbidden |
|
||||
| Validation error | 422 Unprocessable Entity |
|
||||
| Internal error | 500 Internal Server Error |
|
||||
|
||||
---
|
||||
|
||||
## See Also
|
||||
|
||||
- [schema-validation.md](schema-validation.md) - JSON Schema validation
|
||||
- [openapi-aggregation.md](openapi-aggregation.md) - OpenAPI document generation
|
||||
- [migration-guide.md](migration-guide.md) - WebService to Microservice migration
|
||||
462
docs/modules/router/migration-guide.md
Normal file
462
docs/modules/router/migration-guide.md
Normal file
@@ -0,0 +1,462 @@
|
||||
# StellaOps Router Migration Guide
|
||||
|
||||
This guide describes how to migrate existing `StellaOps.*.WebService` projects to the new microservice pattern with the StellaOps Router.
|
||||
|
||||
## Overview
|
||||
|
||||
The router provides a transport-agnostic communication layer between services, replacing direct HTTP calls with efficient binary protocols (TCP, TLS, UDP, RabbitMQ). Benefits include:
|
||||
|
||||
- **Performance**: Binary framing vs HTTP overhead
|
||||
- **Streaming**: First-class support for large payloads
|
||||
- **Cancellation**: Propagated across service boundaries
|
||||
- **Claims**: Authority-integrated authorization
|
||||
- **Health**: Automatic heartbeat and failover
|
||||
|
||||
## Prerequisites
|
||||
|
||||
Before migrating, ensure:
|
||||
|
||||
1. Router infrastructure is deployed (Gateway, transports)
|
||||
2. Authority is configured with endpoint claims
|
||||
3. Local development environment has router.yaml configured
|
||||
|
||||
## Migration Strategies
|
||||
|
||||
### Strategy A: In-Place Adaptation
|
||||
|
||||
Best for services that need to maintain HTTP compatibility during transition.
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────┐
|
||||
│ StellaOps.*.WebService │
|
||||
│ ┌─────────────────────────────────┐│
|
||||
│ │ Existing HTTP Controllers ││◄── HTTP clients (legacy)
|
||||
│ └─────────────────────────────────┘│
|
||||
│ ┌─────────────────────────────────┐│
|
||||
│ │ [StellaEndpoint] Handlers ││◄── Router (new)
|
||||
│ └─────────────────────────────────┘│
|
||||
│ ┌─────────────────────────────────┐│
|
||||
│ │ Shared Domain Logic ││
|
||||
│ └─────────────────────────────────┘│
|
||||
└─────────────────────────────────────┘
|
||||
```
|
||||
|
||||
**Steps:**
|
||||
|
||||
1. Add `StellaOps.Microservice` package reference
|
||||
2. Create handler classes for each HTTP route
|
||||
3. Handlers call existing service layer
|
||||
4. Register with router alongside HTTP
|
||||
5. Test via router
|
||||
6. Shift traffic gradually
|
||||
7. Remove HTTP controllers when ready
|
||||
|
||||
**Pros:**
|
||||
- Gradual migration
|
||||
- No downtime
|
||||
- Can roll back easily
|
||||
|
||||
**Cons:**
|
||||
- Dual maintenance during transition
|
||||
- May delay cleanup
|
||||
|
||||
### Strategy B: Clean Split
|
||||
|
||||
Best for major refactoring or when HTTP compatibility is not needed.
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────┐
|
||||
│ StellaOps.*.Domain │ ◄── Shared library
|
||||
│ (extracted business logic) │
|
||||
└─────────────────────────────────────┘
|
||||
▲ ▲
|
||||
│ │
|
||||
┌─────────┴───────┐ ┌───────┴─────────┐
|
||||
│ (Legacy) │ │ (New) │
|
||||
│ *.WebService │ │ *.Microservice │
|
||||
│ HTTP only │ │ Router only │
|
||||
└─────────────────┘ └─────────────────┘
|
||||
```
|
||||
|
||||
**Steps:**
|
||||
|
||||
1. Extract domain logic to `.Domain` library
|
||||
2. Create new `.Microservice` project
|
||||
3. Implement handlers using domain library
|
||||
4. Deploy alongside WebService
|
||||
5. Shift traffic to router
|
||||
6. Deprecate WebService
|
||||
|
||||
**Pros:**
|
||||
- Clean architecture
|
||||
- No legacy code in new project
|
||||
- Clear separation of concerns
|
||||
|
||||
**Cons:**
|
||||
- More upfront work
|
||||
- Requires domain extraction
|
||||
|
||||
## Controller to Handler Mapping
|
||||
|
||||
### Before (ASP.NET Controller)
|
||||
|
||||
```csharp
|
||||
[ApiController]
|
||||
[Route("api/invoices")]
|
||||
public class InvoicesController : ControllerBase
|
||||
{
|
||||
private readonly IInvoiceService _service;
|
||||
|
||||
[HttpPost]
|
||||
[Authorize(Roles = "billing-admin")]
|
||||
public async Task<IActionResult> Create(
|
||||
[FromBody] CreateInvoiceRequest request,
|
||||
CancellationToken ct)
|
||||
{
|
||||
var invoice = await _service.CreateAsync(request);
|
||||
return Ok(new { invoice.Id });
|
||||
}
|
||||
|
||||
[HttpGet("{id}")]
|
||||
public async Task<IActionResult> Get(string id)
|
||||
{
|
||||
var invoice = await _service.GetAsync(id);
|
||||
if (invoice == null) return NotFound();
|
||||
return Ok(invoice);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### After (Microservice Handler)
|
||||
|
||||
```csharp
|
||||
// Handler for POST /api/invoices
|
||||
[StellaEndpoint("POST", "/api/invoices", RequiredClaims = ["invoices:write"])]
|
||||
public sealed class CreateInvoiceEndpoint : IStellaEndpoint<CreateInvoiceRequest, CreateInvoiceResponse>
|
||||
{
|
||||
private readonly IInvoiceService _service;
|
||||
|
||||
public CreateInvoiceEndpoint(IInvoiceService service) => _service = service;
|
||||
|
||||
public async Task<CreateInvoiceResponse> HandleAsync(
|
||||
CreateInvoiceRequest request,
|
||||
CancellationToken ct)
|
||||
{
|
||||
var invoice = await _service.CreateAsync(request, ct);
|
||||
return new CreateInvoiceResponse { InvoiceId = invoice.Id };
|
||||
}
|
||||
}
|
||||
|
||||
// Handler for GET /api/invoices/{id}
|
||||
[StellaEndpoint("GET", "/api/invoices/{id}", RequiredClaims = ["invoices:read"])]
|
||||
public sealed class GetInvoiceEndpoint : IStellaEndpoint<GetInvoiceRequest, GetInvoiceResponse>
|
||||
{
|
||||
private readonly IInvoiceService _service;
|
||||
|
||||
public GetInvoiceEndpoint(IInvoiceService service) => _service = service;
|
||||
|
||||
public async Task<GetInvoiceResponse> HandleAsync(
|
||||
GetInvoiceRequest request,
|
||||
CancellationToken ct)
|
||||
{
|
||||
var invoice = await _service.GetAsync(request.Id, ct);
|
||||
return new GetInvoiceResponse
|
||||
{
|
||||
InvoiceId = invoice?.Id,
|
||||
Found = invoice != null
|
||||
};
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## CancellationToken Wiring
|
||||
|
||||
**This is the #1 source of migration bugs.** Every async operation must receive and respect the cancellation token.
|
||||
|
||||
### Checklist
|
||||
|
||||
For each migrated handler, verify:
|
||||
|
||||
- [ ] Handler accepts CancellationToken parameter (automatic with IStellaEndpoint)
|
||||
- [ ] Token passed to all database calls
|
||||
- [ ] Token passed to all HTTP client calls
|
||||
- [ ] Token passed to all file I/O operations
|
||||
- [ ] Long-running loops check `ct.IsCancellationRequested`
|
||||
- [ ] Token passed to `Task.Delay`, `WaitAsync`, etc.
|
||||
|
||||
### Example: Before (missing tokens)
|
||||
|
||||
```csharp
|
||||
public async Task<Invoice> CreateAsync(CreateInvoiceRequest request)
|
||||
{
|
||||
var invoice = new Invoice(request);
|
||||
await _db.Invoices.AddAsync(invoice); // Missing token!
|
||||
await _db.SaveChangesAsync(); // Missing token!
|
||||
await _notifier.SendAsync(invoice); // Missing token!
|
||||
return invoice;
|
||||
}
|
||||
```
|
||||
|
||||
### Example: After (proper wiring)
|
||||
|
||||
```csharp
|
||||
public async Task<Invoice> CreateAsync(CreateInvoiceRequest request, CancellationToken ct)
|
||||
{
|
||||
ct.ThrowIfCancellationRequested();
|
||||
|
||||
var invoice = new Invoice(request);
|
||||
await _db.Invoices.AddAsync(invoice, ct);
|
||||
await _db.SaveChangesAsync(ct);
|
||||
await _notifier.SendAsync(invoice, ct);
|
||||
return invoice;
|
||||
}
|
||||
```
|
||||
|
||||
## Streaming Migration
|
||||
|
||||
### File Upload: Before
|
||||
|
||||
```csharp
|
||||
[HttpPost("upload")]
|
||||
public async Task<IActionResult> Upload(IFormFile file)
|
||||
{
|
||||
using var stream = file.OpenReadStream();
|
||||
await _storage.SaveAsync(stream);
|
||||
return Ok();
|
||||
}
|
||||
```
|
||||
|
||||
### File Upload: After
|
||||
|
||||
```csharp
|
||||
[StellaEndpoint("POST", "/upload", SupportsStreaming = true)]
|
||||
public sealed class UploadEndpoint : IRawStellaEndpoint
|
||||
{
|
||||
private readonly IStorageService _storage;
|
||||
|
||||
public UploadEndpoint(IStorageService storage) => _storage = storage;
|
||||
|
||||
public async Task<RawResponse> HandleAsync(RawRequestContext ctx, CancellationToken ct)
|
||||
{
|
||||
// ctx.Body is already a stream - no buffering needed
|
||||
var path = await _storage.SaveAsync(ctx.Body, ct);
|
||||
return RawResponse.Ok($"{{\"path\":\"{path}\"}}");
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### File Download: Before
|
||||
|
||||
```csharp
|
||||
[HttpGet("download/{id}")]
|
||||
public async Task<IActionResult> Download(string id)
|
||||
{
|
||||
var stream = await _storage.GetAsync(id);
|
||||
return File(stream, "application/octet-stream");
|
||||
}
|
||||
```
|
||||
|
||||
### File Download: After
|
||||
|
||||
```csharp
|
||||
[StellaEndpoint("GET", "/download/{id}", SupportsStreaming = true)]
|
||||
public sealed class DownloadEndpoint : IRawStellaEndpoint
|
||||
{
|
||||
private readonly IStorageService _storage;
|
||||
|
||||
public DownloadEndpoint(IStorageService storage) => _storage = storage;
|
||||
|
||||
public async Task<RawResponse> HandleAsync(RawRequestContext ctx, CancellationToken ct)
|
||||
{
|
||||
var id = ctx.PathParameters["id"];
|
||||
var stream = await _storage.GetAsync(id, ct);
|
||||
return RawResponse.Stream(stream, "application/octet-stream");
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Authorization Migration
|
||||
|
||||
### Before: [Authorize] Attribute
|
||||
|
||||
```csharp
|
||||
[Authorize(Roles = "admin,billing-manager")]
|
||||
public async Task<IActionResult> Delete(string id) { ... }
|
||||
```
|
||||
|
||||
### After: RequiredClaims
|
||||
|
||||
```csharp
|
||||
[StellaEndpoint("DELETE", "/invoices/{id}", RequiredClaims = ["invoices:delete"])]
|
||||
public sealed class DeleteInvoiceEndpoint : IStellaEndpoint<...> { ... }
|
||||
```
|
||||
|
||||
Claims are configured in Authority and enforced by the Gateway's AuthorizationMiddleware.
|
||||
|
||||
## Migration Checklist Template
|
||||
|
||||
Use this checklist for each service migration:
|
||||
|
||||
```markdown
|
||||
# Migration Checklist: [ServiceName]
|
||||
|
||||
## Inventory
|
||||
- [ ] List all HTTP routes (Method + Path)
|
||||
- [ ] Identify streaming endpoints
|
||||
- [ ] Identify authorization requirements
|
||||
- [ ] Document external dependencies
|
||||
|
||||
## Preparation
|
||||
- [ ] Add StellaOps.Microservice package
|
||||
- [ ] Add StellaOps.Router.Transport.* package(s)
|
||||
- [ ] Configure router connection in Program.cs
|
||||
- [ ] Set up local gateway for testing
|
||||
|
||||
## Per-Route Migration
|
||||
For each route:
|
||||
- [ ] Create [StellaEndpoint] handler class
|
||||
- [ ] Define request/response record types
|
||||
- [ ] Map path parameters
|
||||
- [ ] Wire CancellationToken throughout
|
||||
- [ ] Convert to IRawStellaEndpoint if streaming
|
||||
- [ ] Add RequiredClaims
|
||||
- [ ] Write unit tests
|
||||
- [ ] Write integration tests
|
||||
|
||||
## Cutover
|
||||
- [ ] Deploy alongside existing WebService
|
||||
- [ ] Verify via router routing
|
||||
- [ ] Shift percentage of traffic
|
||||
- [ ] Monitor for errors
|
||||
- [ ] Full cutover
|
||||
- [ ] Remove WebService HTTP listeners
|
||||
|
||||
## Cleanup
|
||||
- [ ] Remove unused controller code
|
||||
- [ ] Remove HTTP pipeline configuration
|
||||
- [ ] Update OpenAPI documentation
|
||||
- [ ] Update client SDKs
|
||||
```
|
||||
|
||||
## Service Inventory
|
||||
|
||||
| Module | WebService Project | Priority | Complexity | Notes |
|
||||
|--------|-------------------|----------|------------|-------|
|
||||
| Gateway | StellaOps.Gateway.WebService | N/A | N/A | IS the router |
|
||||
| Concelier | StellaOps.Concelier.WebService | High | Medium | Advisory ingestion |
|
||||
| Scanner | StellaOps.Scanner.WebService | High | High | Streaming scans |
|
||||
| Attestor | StellaOps.Attestor.WebService | Medium | Medium | Attestation gen |
|
||||
| Excititor | StellaOps.Excititor.WebService | Medium | Low | VEX processing |
|
||||
| Orchestrator | StellaOps.Orchestrator.WebService | Medium | Medium | Job coordination |
|
||||
| Scheduler | StellaOps.Scheduler.WebService | Low | Low | Job scheduling |
|
||||
| Notify | StellaOps.Notify.WebService | Low | Low | Notifications |
|
||||
| Notifier | StellaOps.Notifier.WebService | Low | Low | Alert dispatch |
|
||||
| Signer | StellaOps.Signer.WebService | Medium | Low | Crypto signing |
|
||||
| Findings | StellaOps.Findings.Ledger.WebService | Medium | Medium | Results storage |
|
||||
| EvidenceLocker | StellaOps.EvidenceLocker.WebService | Low | Medium | Blob storage |
|
||||
| ExportCenter | StellaOps.ExportCenter.WebService | Low | Medium | Report generation |
|
||||
| IssuerDirectory | StellaOps.IssuerDirectory.WebService | Low | Low | Issuer lookup |
|
||||
| PacksRegistry | StellaOps.PacksRegistry.WebService | Low | Low | Pack management |
|
||||
| RiskEngine | StellaOps.RiskEngine.WebService | Medium | Medium | Risk calculation |
|
||||
| TaskRunner | StellaOps.TaskRunner.WebService | Low | Medium | Task execution |
|
||||
| TimelineIndexer | StellaOps.TimelineIndexer.WebService | Low | Low | Event indexing |
|
||||
| AdvisoryAI | StellaOps.AdvisoryAI.WebService | Low | Medium | AI assistance |
|
||||
|
||||
## Testing During Migration
|
||||
|
||||
### Unit Tests
|
||||
|
||||
Test handlers in isolation using mocked dependencies:
|
||||
|
||||
```csharp
|
||||
[Fact]
|
||||
public async Task CreateInvoice_ValidRequest_ReturnsInvoiceId()
|
||||
{
|
||||
// Arrange
|
||||
var mockService = new Mock<IInvoiceService>();
|
||||
mockService.Setup(s => s.CreateAsync(It.IsAny<CreateInvoiceRequest>(), It.IsAny<CancellationToken>()))
|
||||
.ReturnsAsync(new Invoice { Id = "INV-123" });
|
||||
|
||||
var endpoint = new CreateInvoiceEndpoint(mockService.Object);
|
||||
|
||||
// Act
|
||||
var response = await endpoint.HandleAsync(
|
||||
new CreateInvoiceRequest { Amount = 100 },
|
||||
CancellationToken.None);
|
||||
|
||||
// Assert
|
||||
response.InvoiceId.Should().Be("INV-123");
|
||||
}
|
||||
```
|
||||
|
||||
### Integration Tests
|
||||
|
||||
Use WebApplicationFactory for the Gateway and actual microservice instances:
|
||||
|
||||
```csharp
|
||||
public sealed class InvoiceTests : IClassFixture<GatewayFixture>
|
||||
{
|
||||
private readonly GatewayFixture _fixture;
|
||||
|
||||
[Fact]
|
||||
public async Task CreateAndGetInvoice_WorksEndToEnd()
|
||||
{
|
||||
var createResponse = await _fixture.Client.PostAsJsonAsync("/api/invoices",
|
||||
new { Amount = 100 });
|
||||
createResponse.StatusCode.Should().Be(HttpStatusCode.OK);
|
||||
|
||||
var created = await createResponse.Content.ReadFromJsonAsync<CreateInvoiceResponse>();
|
||||
|
||||
var getResponse = await _fixture.Client.GetAsync($"/api/invoices/{created.InvoiceId}");
|
||||
getResponse.StatusCode.Should().Be(HttpStatusCode.OK);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Common Migration Issues
|
||||
|
||||
### 1. Missing CancellationToken Propagation
|
||||
|
||||
**Symptom:** Requests continue processing after client disconnects.
|
||||
|
||||
**Fix:** Pass `CancellationToken` to all async operations.
|
||||
|
||||
### 2. IFormFile Not Available
|
||||
|
||||
**Symptom:** Compilation error on `IFormFile` parameter.
|
||||
|
||||
**Fix:** Convert to `IRawStellaEndpoint` for streaming.
|
||||
|
||||
### 3. HttpContext Not Available
|
||||
|
||||
**Symptom:** Code references `HttpContext` for headers, claims.
|
||||
|
||||
**Fix:** Use `RawRequestContext` for raw endpoints, or inject claims via Authority.
|
||||
|
||||
### 4. Return Type Mismatch
|
||||
|
||||
**Symptom:** Handler returns `IActionResult`.
|
||||
|
||||
**Fix:** Define proper response record type, return that instead.
|
||||
|
||||
### 5. Route Parameter Not Extracted
|
||||
|
||||
**Symptom:** Path parameters like `{id}` not populated.
|
||||
|
||||
**Fix:** For `IStellaEndpoint`, add property to request type. For `IRawStellaEndpoint`, use `ctx.PathParameters["id"]`.
|
||||
|
||||
## Next Steps
|
||||
|
||||
1. Choose a low-risk service for pilot migration (Scheduler recommended)
|
||||
2. Follow the Migration Checklist
|
||||
3. Document lessons learned
|
||||
4. Proceed with higher-priority services
|
||||
5. Eventually merge all to use router exclusively
|
||||
|
||||
---
|
||||
|
||||
## See Also
|
||||
|
||||
- [Router Architecture](architecture.md) - System specification
|
||||
- [Schema Validation](schema-validation.md) - JSON Schema validation
|
||||
- [OpenAPI Aggregation](openapi-aggregation.md) - OpenAPI document generation
|
||||
503
docs/modules/router/openapi-aggregation.md
Normal file
503
docs/modules/router/openapi-aggregation.md
Normal file
@@ -0,0 +1,503 @@
|
||||
# OpenAPI Aggregation
|
||||
|
||||
This document describes how the StellaOps Gateway aggregates OpenAPI documentation from connected microservices into a unified specification.
|
||||
|
||||
## Overview
|
||||
|
||||
The Gateway automatically generates a single OpenAPI 3.1.0 document that aggregates all endpoints from connected microservices. This provides:
|
||||
|
||||
- **Unified API documentation**: All services documented in one place
|
||||
- **Dynamic updates**: Document regenerates when services connect/disconnect
|
||||
- **Standard compliance**: OpenAPI 3.1.0 with native JSON Schema draft 2020-12 support
|
||||
- **Multiple formats**: Available as JSON or YAML
|
||||
- **Efficient caching**: ETag-based caching with configurable TTL
|
||||
|
||||
### How It Works
|
||||
|
||||
```
|
||||
┌──────────────┐ HELLO ┌──────────────┐ GET /openapi.json ┌──────────────┐
|
||||
│ Billing │ ──────────► │ │ ◄───────────────────── │ Client │
|
||||
│ Service │ + schemas │ Gateway │ │ │
|
||||
└──────────────┘ │ │ OpenAPI 3.1.0 │ │
|
||||
│ │ ─────────────────────► │ │
|
||||
┌──────────────┐ HELLO │ │ unified document └──────────────┘
|
||||
│ Inventory │ ──────────► │ │
|
||||
│ Service │ + schemas └──────────────┘
|
||||
└──────────────┘
|
||||
```
|
||||
|
||||
1. Microservices send schemas and endpoint metadata via HELLO payload
|
||||
2. Gateway stores this information in routing state
|
||||
3. OpenAPI generator aggregates all connected services
|
||||
4. Document is cached and served via HTTP endpoints
|
||||
|
||||
---
|
||||
|
||||
## Configuration
|
||||
|
||||
### OpenApiAggregationOptions
|
||||
|
||||
Configure OpenAPI aggregation in your Gateway configuration:
|
||||
|
||||
```yaml
|
||||
# router.yaml or appsettings.yaml
|
||||
OpenApi:
|
||||
Title: "My API Gateway"
|
||||
Description: "Unified API for all microservices"
|
||||
Version: "2.0.0"
|
||||
ServerUrl: "https://api.example.com"
|
||||
CacheTtlSeconds: 60
|
||||
Enabled: true
|
||||
LicenseName: "AGPL-3.0-or-later"
|
||||
ContactName: "API Team"
|
||||
ContactEmail: "api@example.com"
|
||||
TokenUrl: "/auth/token"
|
||||
```
|
||||
|
||||
### Configuration Reference
|
||||
|
||||
| Property | Type | Default | Description |
|
||||
|----------|------|---------|-------------|
|
||||
| `Title` | `string` | `"StellaOps Gateway API"` | API title in OpenAPI info section |
|
||||
| `Description` | `string` | `"Unified API aggregating all connected microservices."` | API description |
|
||||
| `Version` | `string` | `"1.0.0"` | API version number |
|
||||
| `ServerUrl` | `string` | `"/"` | Base server URL |
|
||||
| `CacheTtlSeconds` | `int` | `60` | Cache time-to-live in seconds |
|
||||
| `Enabled` | `bool` | `true` | Enable/disable OpenAPI aggregation |
|
||||
| `LicenseName` | `string` | `"AGPL-3.0-or-later"` | License name in OpenAPI info |
|
||||
| `ContactName` | `string?` | `null` | Contact name (optional) |
|
||||
| `ContactEmail` | `string?` | `null` | Contact email (optional) |
|
||||
| `TokenUrl` | `string` | `"/auth/token"` | OAuth2 token endpoint URL |
|
||||
|
||||
### Disabling OpenAPI
|
||||
|
||||
To disable OpenAPI aggregation entirely:
|
||||
|
||||
```yaml
|
||||
OpenApi:
|
||||
Enabled: false
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Endpoints
|
||||
|
||||
### Discovery Endpoint
|
||||
|
||||
```http
|
||||
GET /.well-known/openapi
|
||||
```
|
||||
|
||||
Returns metadata about the OpenAPI document:
|
||||
|
||||
**Response:**
|
||||
```json
|
||||
{
|
||||
"openapi_json": "/openapi.json",
|
||||
"openapi_yaml": "/openapi.yaml",
|
||||
"etag": "\"5d41402abc4b2a76b9719d911017c592\"",
|
||||
"generated_at": "2025-01-15T10:30:00.0000000Z"
|
||||
}
|
||||
```
|
||||
|
||||
### OpenAPI JSON
|
||||
|
||||
```http
|
||||
GET /openapi.json
|
||||
```
|
||||
|
||||
Returns the full OpenAPI 3.1.0 specification in JSON format.
|
||||
|
||||
**Headers:**
|
||||
- `Cache-Control: public, max-age=60`
|
||||
- `ETag: "<content-hash>"`
|
||||
- `Content-Type: application/json; charset=utf-8`
|
||||
|
||||
**Conditional Request:**
|
||||
```http
|
||||
GET /openapi.json
|
||||
If-None-Match: "5d41402abc4b2a76b9719d911017c592"
|
||||
```
|
||||
|
||||
Returns `304 Not Modified` if content unchanged.
|
||||
|
||||
### OpenAPI YAML
|
||||
|
||||
```http
|
||||
GET /openapi.yaml
|
||||
```
|
||||
|
||||
Returns the full OpenAPI 3.1.0 specification in YAML format.
|
||||
|
||||
**Headers:**
|
||||
- `Cache-Control: public, max-age=60`
|
||||
- `ETag: "<content-hash>"`
|
||||
- `Content-Type: application/yaml; charset=utf-8`
|
||||
|
||||
---
|
||||
|
||||
## Security Mapping
|
||||
|
||||
The Gateway automatically maps claim requirements to OpenAPI security schemes.
|
||||
|
||||
### Claim to Scope Mapping
|
||||
|
||||
When endpoints define `RequiringClaims`, these are converted to OAuth2 scopes:
|
||||
|
||||
```csharp
|
||||
// Endpoint with claim requirements
|
||||
[StellaEndpoint("POST", "/invoices")]
|
||||
[RequireClaim("billing:write")]
|
||||
public sealed class CreateInvoiceEndpoint : IStellaEndpoint<...>
|
||||
```
|
||||
|
||||
Becomes in OpenAPI:
|
||||
|
||||
```json
|
||||
{
|
||||
"paths": {
|
||||
"/invoices": {
|
||||
"post": {
|
||||
"security": [
|
||||
{
|
||||
"BearerAuth": [],
|
||||
"OAuth2": ["billing:write"]
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Security Schemes
|
||||
|
||||
The Gateway generates two security schemes:
|
||||
|
||||
#### BearerAuth
|
||||
|
||||
HTTP Bearer token authentication (always present):
|
||||
|
||||
```json
|
||||
{
|
||||
"BearerAuth": {
|
||||
"type": "http",
|
||||
"scheme": "bearer",
|
||||
"bearerFormat": "JWT",
|
||||
"description": "JWT Bearer token authentication"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
#### OAuth2
|
||||
|
||||
Client credentials flow with collected scopes (only if endpoints have claims):
|
||||
|
||||
```json
|
||||
{
|
||||
"OAuth2": {
|
||||
"type": "oauth2",
|
||||
"flows": {
|
||||
"clientCredentials": {
|
||||
"tokenUrl": "/auth/token",
|
||||
"scopes": {
|
||||
"billing:write": "Access scope: billing:write",
|
||||
"billing:read": "Access scope: billing:read",
|
||||
"inventory:read": "Access scope: inventory:read"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Scope Collection
|
||||
|
||||
Scopes are automatically collected from all connected services. If multiple endpoints require the same claim, it appears only once in the scopes list.
|
||||
|
||||
---
|
||||
|
||||
## Generated Document Structure
|
||||
|
||||
The aggregated OpenAPI document follows this structure:
|
||||
|
||||
```json
|
||||
{
|
||||
"openapi": "3.1.0",
|
||||
"info": {
|
||||
"title": "StellaOps Gateway API",
|
||||
"version": "1.0.0",
|
||||
"description": "Unified API aggregating all connected microservices.",
|
||||
"license": {
|
||||
"name": "AGPL-3.0-or-later"
|
||||
},
|
||||
"contact": {
|
||||
"name": "API Team",
|
||||
"email": "api@example.com"
|
||||
}
|
||||
},
|
||||
"servers": [
|
||||
{
|
||||
"url": "/"
|
||||
}
|
||||
],
|
||||
"paths": {
|
||||
"/invoices": {
|
||||
"post": {
|
||||
"operationId": "billing_invoices_POST",
|
||||
"tags": ["billing"],
|
||||
"summary": "Create invoice",
|
||||
"description": "Creates a new draft invoice",
|
||||
"security": [
|
||||
{
|
||||
"BearerAuth": [],
|
||||
"OAuth2": ["billing:write"]
|
||||
}
|
||||
],
|
||||
"requestBody": {
|
||||
"required": true,
|
||||
"content": {
|
||||
"application/json": {
|
||||
"schema": {
|
||||
"$ref": "#/components/schemas/billing_CreateInvoiceRequest"
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"responses": {
|
||||
"200": {
|
||||
"description": "Success",
|
||||
"content": {
|
||||
"application/json": {
|
||||
"schema": {
|
||||
"$ref": "#/components/schemas/billing_CreateInvoiceResponse"
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"400": { "description": "Bad Request" },
|
||||
"401": { "description": "Unauthorized" },
|
||||
"404": { "description": "Not Found" },
|
||||
"422": { "description": "Validation Error" },
|
||||
"500": { "description": "Internal Server Error" }
|
||||
}
|
||||
}
|
||||
},
|
||||
"/items": {
|
||||
"get": {
|
||||
"operationId": "inventory_items_GET",
|
||||
"tags": ["inventory"],
|
||||
"summary": "List items",
|
||||
"responses": {
|
||||
"200": { "description": "Success" }
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"components": {
|
||||
"schemas": {
|
||||
"billing_CreateInvoiceRequest": {
|
||||
"type": "object",
|
||||
"required": ["customerId", "amount"],
|
||||
"properties": {
|
||||
"customerId": { "type": "string" },
|
||||
"amount": { "type": "number" },
|
||||
"description": { "type": ["string", "null"] },
|
||||
"lineItems": {
|
||||
"type": "array",
|
||||
"items": { "$ref": "#/components/schemas/billing_LineItem" }
|
||||
}
|
||||
}
|
||||
},
|
||||
"billing_CreateInvoiceResponse": {
|
||||
"type": "object",
|
||||
"required": ["invoiceId", "createdAt", "status"],
|
||||
"properties": {
|
||||
"invoiceId": { "type": "string" },
|
||||
"createdAt": { "type": "string", "format": "date-time" },
|
||||
"status": { "type": "string" }
|
||||
}
|
||||
},
|
||||
"billing_LineItem": {
|
||||
"type": "object",
|
||||
"required": ["description", "amount"],
|
||||
"properties": {
|
||||
"description": { "type": "string" },
|
||||
"amount": { "type": "number" },
|
||||
"quantity": { "type": "integer", "default": 1 }
|
||||
}
|
||||
}
|
||||
},
|
||||
"securitySchemes": {
|
||||
"BearerAuth": {
|
||||
"type": "http",
|
||||
"scheme": "bearer",
|
||||
"bearerFormat": "JWT",
|
||||
"description": "JWT Bearer token authentication"
|
||||
},
|
||||
"OAuth2": {
|
||||
"type": "oauth2",
|
||||
"flows": {
|
||||
"clientCredentials": {
|
||||
"tokenUrl": "/auth/token",
|
||||
"scopes": {
|
||||
"billing:write": "Access scope: billing:write"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"tags": [
|
||||
{
|
||||
"name": "billing",
|
||||
"description": "billing microservice (v1.0.0)"
|
||||
},
|
||||
{
|
||||
"name": "inventory",
|
||||
"description": "inventory microservice (v2.0.0)"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### Schema Prefixing
|
||||
|
||||
Schemas are prefixed with the service name to prevent naming conflicts:
|
||||
|
||||
| Service | Original Type | Prefixed Schema ID |
|
||||
|---------|--------------|-------------------|
|
||||
| `billing` | `CreateInvoiceRequest` | `billing_CreateInvoiceRequest` |
|
||||
| `inventory` | `GetItemResponse` | `inventory_GetItemResponse` |
|
||||
|
||||
### Tag Generation
|
||||
|
||||
Tags are automatically generated from connected services:
|
||||
|
||||
- Tag name: Service name (lowercase)
|
||||
- Tag description: Service description from `OpenApiInfo` or auto-generated
|
||||
|
||||
### Operation IDs
|
||||
|
||||
Operation IDs follow the pattern: `{serviceName}_{path}_{method}`
|
||||
|
||||
Example: `billing_invoices_POST`
|
||||
|
||||
---
|
||||
|
||||
## Cache Behavior
|
||||
|
||||
### TTL-Based Expiration
|
||||
|
||||
The document cache expires based on `CacheTtlSeconds` (default: 60 seconds):
|
||||
|
||||
```yaml
|
||||
OpenApi:
|
||||
CacheTtlSeconds: 30 # More frequent regeneration
|
||||
```
|
||||
|
||||
Setting `CacheTtlSeconds: 0` regenerates the document on every request (not recommended for production).
|
||||
|
||||
### Connection-Based Invalidation
|
||||
|
||||
The cache is automatically invalidated when:
|
||||
|
||||
1. A new microservice connects (HELLO received)
|
||||
2. A microservice disconnects (connection closed)
|
||||
|
||||
This ensures the OpenAPI document always reflects currently connected services.
|
||||
|
||||
### ETag Consistency
|
||||
|
||||
The ETag is computed from the document content hash (SHA256). This ensures:
|
||||
|
||||
- Same content = same ETag
|
||||
- Content changes = new ETag
|
||||
- Clients can use conditional requests to avoid re-downloading unchanged documents
|
||||
|
||||
### Recommended Client Strategy
|
||||
|
||||
```javascript
|
||||
// Store ETag from previous response
|
||||
let cachedETag = localStorage.getItem('openapi-etag');
|
||||
|
||||
const response = await fetch('/openapi.json', {
|
||||
headers: cachedETag ? { 'If-None-Match': cachedETag } : {}
|
||||
});
|
||||
|
||||
if (response.status === 304) {
|
||||
// Use cached document
|
||||
return getCachedDocument();
|
||||
}
|
||||
|
||||
// Store new ETag and document
|
||||
localStorage.setItem('openapi-etag', response.headers.get('ETag'));
|
||||
const document = await response.json();
|
||||
cacheDocument(document);
|
||||
return document;
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Service Registration
|
||||
|
||||
### Microservice Options
|
||||
|
||||
Configure service metadata that appears in OpenAPI:
|
||||
|
||||
```csharp
|
||||
services.AddStellaMicroservice(options =>
|
||||
{
|
||||
options.ServiceName = "billing";
|
||||
options.ServiceDescription = "Invoice and payment processing service";
|
||||
options.ContactInfo = "billing-team@example.com";
|
||||
});
|
||||
```
|
||||
|
||||
### Service OpenAPI Info
|
||||
|
||||
The `ServiceOpenApiInfo` is sent in the HELLO payload:
|
||||
|
||||
```json
|
||||
{
|
||||
"instance": { ... },
|
||||
"endpoints": [ ... ],
|
||||
"schemas": { ... },
|
||||
"openApiInfo": {
|
||||
"title": "billing",
|
||||
"description": "Invoice and payment processing service"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
This description appears in the tag entry for the service.
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Document Not Updating
|
||||
|
||||
1. Check `CacheTtlSeconds` - may need to wait for TTL expiration
|
||||
2. Verify service connected successfully (check Gateway logs)
|
||||
3. Force refresh by restarting the Gateway
|
||||
|
||||
### Missing Schemas
|
||||
|
||||
1. Ensure `[ValidateSchema]` attribute is applied to endpoints
|
||||
2. Check for schema parsing errors in Gateway logs
|
||||
3. Verify endpoint implements `IStellaEndpoint<TRequest, TResponse>`
|
||||
|
||||
### Security Schemes Not Appearing
|
||||
|
||||
1. OAuth2 scheme only appears if endpoints have claim requirements
|
||||
2. Check `RequiringClaims` is populated on endpoint descriptors
|
||||
3. Verify claim types are being transmitted correctly
|
||||
|
||||
---
|
||||
|
||||
## See Also
|
||||
|
||||
- [Schema Validation](schema-validation.md) - JSON Schema validation reference
|
||||
- [API Overview](../../api/overview.md) - General API conventions
|
||||
- [Gateway OpenAPI](../gateway/openapi.md) - Gateway OpenAPI implementation details
|
||||
426
docs/modules/router/schema-validation.md
Normal file
426
docs/modules/router/schema-validation.md
Normal file
@@ -0,0 +1,426 @@
|
||||
# JSON Schema Validation
|
||||
|
||||
This document describes the JSON Schema validation feature in the StellaOps Router/Microservice SDK.
|
||||
|
||||
## Overview
|
||||
|
||||
The StellaOps Microservice SDK provides compile-time JSON Schema generation and runtime request/response validation. Schemas are automatically generated from your C# types using a source generator, then transmitted to the Gateway via the HELLO payload where they power both runtime validation and OpenAPI documentation.
|
||||
|
||||
### Key Features
|
||||
|
||||
- **Compile-time schema generation**: JSON Schema draft 2020-12 generated from C# types
|
||||
- **Runtime validation**: Request bodies validated against schema before reaching handlers
|
||||
- **OpenAPI 3.1.0 compatibility**: Native JSON Schema support in OpenAPI 3.1.0
|
||||
- **Automatic documentation**: Schemas flow to Gateway for unified OpenAPI documentation
|
||||
- **External schema support**: Override generated schemas with embedded resource files
|
||||
|
||||
### Benefits
|
||||
|
||||
| Benefit | Description |
|
||||
|---------|-------------|
|
||||
| Type safety | Contract enforcement between clients and services |
|
||||
| Early error detection | Invalid requests rejected with 422 before handler execution |
|
||||
| Documentation automation | No manual schema maintenance required |
|
||||
| Interoperability | Standard JSON Schema works with any tooling |
|
||||
|
||||
---
|
||||
|
||||
## Quick Start
|
||||
|
||||
### 1. Add the ValidateSchema Attribute
|
||||
|
||||
```csharp
|
||||
using StellaOps.Microservice;
|
||||
|
||||
[StellaEndpoint("POST", "/invoices")]
|
||||
[ValidateSchema]
|
||||
public sealed class CreateInvoiceEndpoint : IStellaEndpoint<CreateInvoiceRequest, CreateInvoiceResponse>
|
||||
{
|
||||
public Task<CreateInvoiceResponse> HandleAsync(
|
||||
CreateInvoiceRequest request,
|
||||
CancellationToken cancellationToken)
|
||||
{
|
||||
// Request is already validated against JSON Schema
|
||||
return Task.FromResult(new CreateInvoiceResponse
|
||||
{
|
||||
InvoiceId = Guid.NewGuid().ToString(),
|
||||
CreatedAt = DateTime.UtcNow,
|
||||
Status = "draft"
|
||||
});
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### 2. Define Your Request/Response Types
|
||||
|
||||
```csharp
|
||||
public sealed record CreateInvoiceRequest
|
||||
{
|
||||
public required string CustomerId { get; init; }
|
||||
public required decimal Amount { get; init; }
|
||||
public string? Description { get; init; }
|
||||
public List<LineItem> LineItems { get; init; } = [];
|
||||
}
|
||||
|
||||
public sealed record CreateInvoiceResponse
|
||||
{
|
||||
public required string InvoiceId { get; init; }
|
||||
public required DateTime CreatedAt { get; init; }
|
||||
public required string Status { get; init; }
|
||||
}
|
||||
```
|
||||
|
||||
### 3. Build Your Project
|
||||
|
||||
The source generator automatically creates JSON Schema definitions at compile time. These schemas are included in the HELLO payload when your microservice connects to the Gateway.
|
||||
|
||||
---
|
||||
|
||||
## Attribute Reference
|
||||
|
||||
### ValidateSchemaAttribute
|
||||
|
||||
Enables JSON Schema validation for an endpoint.
|
||||
|
||||
```csharp
|
||||
[AttributeUsage(AttributeTargets.Class, AllowMultiple = false, Inherited = false)]
|
||||
public sealed class ValidateSchemaAttribute : Attribute
|
||||
```
|
||||
|
||||
### Properties
|
||||
|
||||
| Property | Type | Default | Description |
|
||||
|----------|------|---------|-------------|
|
||||
| `ValidateRequest` | `bool` | `true` | Enable request body validation |
|
||||
| `ValidateResponse` | `bool` | `false` | Enable response body validation |
|
||||
| `RequestSchemaResource` | `string?` | `null` | Embedded resource path to external request schema |
|
||||
| `ResponseSchemaResource` | `string?` | `null` | Embedded resource path to external response schema |
|
||||
| `Summary` | `string?` | `null` | OpenAPI operation summary |
|
||||
| `Description` | `string?` | `null` | OpenAPI operation description |
|
||||
| `Tags` | `string[]?` | `null` | OpenAPI tags for grouping endpoints |
|
||||
| `Deprecated` | `bool` | `false` | Mark endpoint as deprecated in OpenAPI |
|
||||
|
||||
### Validation Properties
|
||||
|
||||
#### ValidateRequest (default: true)
|
||||
|
||||
Controls whether incoming request bodies are validated against the generated schema.
|
||||
|
||||
```csharp
|
||||
// Request validation enabled (default)
|
||||
[ValidateSchema(ValidateRequest = true)]
|
||||
|
||||
// Disable request validation
|
||||
[ValidateSchema(ValidateRequest = false)]
|
||||
```
|
||||
|
||||
#### ValidateResponse (default: false)
|
||||
|
||||
Enables response body validation. Useful for debugging or strict contract enforcement, but adds overhead.
|
||||
|
||||
```csharp
|
||||
// Enable response validation
|
||||
[ValidateSchema(ValidateResponse = true)]
|
||||
```
|
||||
|
||||
#### External Schema Files
|
||||
|
||||
Override generated schemas with embedded resource files when you need custom schema definitions:
|
||||
|
||||
```csharp
|
||||
[ValidateSchema(RequestSchemaResource = "Schemas.create-order.json")]
|
||||
public sealed class CreateOrderEndpoint : IStellaEndpoint<CreateOrderRequest, CreateOrderResponse>
|
||||
```
|
||||
|
||||
The schema file must be embedded as a resource in your assembly.
|
||||
|
||||
### Documentation Properties
|
||||
|
||||
#### Summary and Description
|
||||
|
||||
Provide OpenAPI documentation for the operation:
|
||||
|
||||
```csharp
|
||||
[ValidateSchema(
|
||||
Summary = "Create a new invoice",
|
||||
Description = "Creates a draft invoice for the specified customer. The invoice must be finalized before it can be sent.")]
|
||||
public sealed class CreateInvoiceEndpoint : IStellaEndpoint<CreateInvoiceRequest, CreateInvoiceResponse>
|
||||
```
|
||||
|
||||
#### Tags
|
||||
|
||||
Override the default service-based tag grouping:
|
||||
|
||||
```csharp
|
||||
[ValidateSchema(Tags = ["Billing", "Invoices"])]
|
||||
public sealed class CreateInvoiceEndpoint : IStellaEndpoint<CreateInvoiceRequest, CreateInvoiceResponse>
|
||||
```
|
||||
|
||||
#### Deprecated
|
||||
|
||||
Mark an endpoint as deprecated in OpenAPI documentation:
|
||||
|
||||
```csharp
|
||||
[ValidateSchema(Deprecated = true, Description = "Use /v2/invoices instead")]
|
||||
public sealed class CreateInvoiceV1Endpoint : IStellaEndpoint<CreateInvoiceRequest, CreateInvoiceResponse>
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Schema Discovery Endpoints
|
||||
|
||||
The Gateway exposes endpoints to discover and retrieve schemas from connected microservices.
|
||||
|
||||
### Discovery Endpoint
|
||||
|
||||
```http
|
||||
GET /.well-known/openapi
|
||||
```
|
||||
|
||||
Returns metadata about the OpenAPI document including available format URLs:
|
||||
|
||||
```json
|
||||
{
|
||||
"openapi_json": "/openapi.json",
|
||||
"openapi_yaml": "/openapi.yaml",
|
||||
"etag": "\"a1b2c3d4\"",
|
||||
"generated_at": "2025-01-15T10:30:00.000Z"
|
||||
}
|
||||
```
|
||||
|
||||
### OpenAPI Document (JSON)
|
||||
|
||||
```http
|
||||
GET /openapi.json
|
||||
Accept: application/json
|
||||
```
|
||||
|
||||
Returns the full OpenAPI 3.1.0 specification in JSON format. Supports ETag-based caching:
|
||||
|
||||
```http
|
||||
GET /openapi.json
|
||||
If-None-Match: "a1b2c3d4"
|
||||
```
|
||||
|
||||
Returns `304 Not Modified` if the document hasn't changed.
|
||||
|
||||
### OpenAPI Document (YAML)
|
||||
|
||||
```http
|
||||
GET /openapi.yaml
|
||||
Accept: application/yaml
|
||||
```
|
||||
|
||||
Returns the full OpenAPI 3.1.0 specification in YAML format.
|
||||
|
||||
### Caching Behavior
|
||||
|
||||
All OpenAPI endpoints support HTTP caching:
|
||||
|
||||
| Header | Value | Description |
|
||||
|--------|-------|-------------|
|
||||
| `Cache-Control` | `public, max-age=60` | Client-side caching for 60 seconds |
|
||||
| `ETag` | `"<hash>"` | Content hash for conditional requests |
|
||||
|
||||
---
|
||||
|
||||
## Examples
|
||||
|
||||
### Basic Endpoint with Validation
|
||||
|
||||
```csharp
|
||||
[StellaEndpoint("POST", "/invoices")]
|
||||
[ValidateSchema]
|
||||
public sealed class CreateInvoiceEndpoint : IStellaEndpoint<CreateInvoiceRequest, CreateInvoiceResponse>
|
||||
{
|
||||
private readonly ILogger<CreateInvoiceEndpoint> _logger;
|
||||
|
||||
public CreateInvoiceEndpoint(ILogger<CreateInvoiceEndpoint> logger)
|
||||
{
|
||||
_logger = logger;
|
||||
}
|
||||
|
||||
public Task<CreateInvoiceResponse> HandleAsync(
|
||||
CreateInvoiceRequest request,
|
||||
CancellationToken cancellationToken)
|
||||
{
|
||||
_logger.LogInformation(
|
||||
"Creating invoice for customer {CustomerId} with amount {Amount}",
|
||||
request.CustomerId,
|
||||
request.Amount);
|
||||
|
||||
return Task.FromResult(new CreateInvoiceResponse
|
||||
{
|
||||
InvoiceId = $"INV-{Guid.NewGuid():N}"[..16].ToUpperInvariant(),
|
||||
CreatedAt = DateTime.UtcNow,
|
||||
Status = "draft"
|
||||
});
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Endpoint with Full Documentation
|
||||
|
||||
```csharp
|
||||
[StellaEndpoint("POST", "/invoices", TimeoutSeconds = 30)]
|
||||
[ValidateSchema(
|
||||
Summary = "Create invoice",
|
||||
Description = "Creates a new draft invoice for the specified customer. Line items are optional but recommended for itemized billing.",
|
||||
Tags = ["Billing", "Invoices"])]
|
||||
public sealed class CreateInvoiceEndpoint : IStellaEndpoint<CreateInvoiceRequest, CreateInvoiceResponse>
|
||||
{
|
||||
// Implementation...
|
||||
}
|
||||
```
|
||||
|
||||
### Deprecated Endpoint
|
||||
|
||||
```csharp
|
||||
[StellaEndpoint("POST", "/v1/invoices")]
|
||||
[ValidateSchema(
|
||||
Deprecated = true,
|
||||
Summary = "Create invoice (deprecated)",
|
||||
Description = "This endpoint is deprecated. Use POST /v2/invoices instead.")]
|
||||
public sealed class CreateInvoiceV1Endpoint : IStellaEndpoint<CreateInvoiceRequest, CreateInvoiceResponse>
|
||||
{
|
||||
// Implementation...
|
||||
}
|
||||
```
|
||||
|
||||
### Request with Complex Types
|
||||
|
||||
```csharp
|
||||
public sealed record CreateInvoiceRequest
|
||||
{
|
||||
/// <summary>
|
||||
/// The customer identifier.
|
||||
/// </summary>
|
||||
public required string CustomerId { get; init; }
|
||||
|
||||
/// <summary>
|
||||
/// The invoice amount in the default currency.
|
||||
/// </summary>
|
||||
public required decimal Amount { get; init; }
|
||||
|
||||
/// <summary>
|
||||
/// Optional description for the invoice.
|
||||
/// </summary>
|
||||
public string? Description { get; init; }
|
||||
|
||||
/// <summary>
|
||||
/// Line items for itemized billing.
|
||||
/// </summary>
|
||||
public List<LineItem> LineItems { get; init; } = [];
|
||||
}
|
||||
|
||||
public sealed record LineItem
|
||||
{
|
||||
public required string Description { get; init; }
|
||||
public required decimal Amount { get; init; }
|
||||
public int Quantity { get; init; } = 1;
|
||||
}
|
||||
```
|
||||
|
||||
The source generator produces JSON Schema that includes:
|
||||
- Required property validation (`required` keyword)
|
||||
- Type validation (`type` keyword)
|
||||
- Nullable property handling (`null` in type union)
|
||||
- Nested object schemas
|
||||
|
||||
---
|
||||
|
||||
## Architecture
|
||||
|
||||
### Schema Flow Diagram
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────────┐
|
||||
│ COMPILE TIME │
|
||||
├─────────────────────────────────────────────────────────────────────┤
|
||||
│ │
|
||||
│ C# Types Source Generator Generated Code │
|
||||
│ ───────── ──────────────── ────────────── │
|
||||
│ CreateInvoice ──► Analyzes types ──► JSON Schema defs │
|
||||
│ Request Extracts metadata GetSchemaDefinitions│
|
||||
│ CreateInvoice Generates schemas EndpointDescriptor │
|
||||
│ Response with SchemaInfo │
|
||||
│ │
|
||||
└─────────────────────────────────────────────────────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────────────┐
|
||||
│ RUNTIME │
|
||||
├─────────────────────────────────────────────────────────────────────┤
|
||||
│ │
|
||||
│ Microservice Start HELLO Payload Gateway Storage │
|
||||
│ ───────────────── ───────────── ─────────────── │
|
||||
│ RouterConnection ──► Instance info ──► ConnectionState │
|
||||
│ Manager Endpoints[] Schemas dictionary │
|
||||
│ Schemas{} OpenApiInfo │
|
||||
│ OpenApiInfo │
|
||||
│ │
|
||||
└─────────────────────────────────────────────────────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────────────┐
|
||||
│ HTTP EXPOSURE │
|
||||
├─────────────────────────────────────────────────────────────────────┤
|
||||
│ │
|
||||
│ OpenApiDocument Document Cache HTTP Endpoints │
|
||||
│ Generator ───────────── ────────────── │
|
||||
│ ────────────── │
|
||||
│ Aggregates all ──► TTL-based cache ──► GET /openapi.json │
|
||||
│ connected services ETag generation GET /openapi.yaml │
|
||||
│ Prefixes schemas Invalidation on GET /.well-known/ │
|
||||
│ by service name connection change openapi │
|
||||
│ │
|
||||
└─────────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
### Schema Naming Convention
|
||||
|
||||
Schemas are prefixed with the service name to avoid conflicts:
|
||||
|
||||
- Service: `Billing`
|
||||
- Type: `CreateInvoiceRequest`
|
||||
- Schema ID: `Billing_CreateInvoiceRequest`
|
||||
|
||||
This allows multiple services to define types with the same name without collisions.
|
||||
|
||||
---
|
||||
|
||||
## Error Handling
|
||||
|
||||
### Validation Errors
|
||||
|
||||
When request validation fails, the endpoint returns a `422 Unprocessable Entity` response with details:
|
||||
|
||||
```json
|
||||
{
|
||||
"type": "https://tools.ietf.org/html/rfc4918#section-11.2",
|
||||
"title": "Validation Error",
|
||||
"status": 422,
|
||||
"errors": [
|
||||
{
|
||||
"path": "$.customerId",
|
||||
"message": "Required property 'customerId' is missing"
|
||||
},
|
||||
{
|
||||
"path": "$.amount",
|
||||
"message": "Value must be a number"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### Invalid Schemas
|
||||
|
||||
If a schema cannot be parsed (e.g., malformed JSON in external schema file), the schema is skipped and a warning is logged. The endpoint will still function but without schema validation.
|
||||
|
||||
---
|
||||
|
||||
## See Also
|
||||
|
||||
- [OpenAPI Aggregation](openapi-aggregation.md) - How schemas become OpenAPI documentation
|
||||
- [API Overview](../../api/overview.md) - General API conventions
|
||||
- [Router Architecture](architecture.md) - Router system overview
|
||||
Reference in New Issue
Block a user