Merge remote-tracking branch 'origin/main' into feature/docs-mdx-skeletons
Some checks failed
AOC Guard CI / aoc-guard (push) Has been cancelled
AOC Guard CI / aoc-verify (push) Has been cancelled
Concelier Attestation Tests / attestation-tests (push) Has been cancelled
Docs CI / lint-and-preview (push) Has been cancelled
Export Center CI / export-ci (push) Has been cancelled
Policy Lint & Smoke / policy-lint (push) Has been cancelled
Policy Simulation / policy-simulate (push) Has been cancelled
SDK Publish & Sign / sdk-publish (push) Has been cancelled
Signals CI & Image / signals-ci (push) Has been cancelled
sdk-generator-smoke / sdk-smoke (push) Has been cancelled
Airgap Sealed CI Smoke / sealed-smoke (push) Has been cancelled
Console CI / console-ci (push) Has been cancelled
Symbols Server CI / symbols-smoke (push) Has been cancelled
VEX Proof Bundles / verify-bundles (push) Has been cancelled

This commit is contained in:
StellaOps Bot
2025-12-05 23:14:58 +02:00
7590 changed files with 22444 additions and 7465469 deletions

View File

@@ -0,0 +1,325 @@
# Gateway OpenAPI Implementation
This document describes the implementation architecture of OpenAPI document aggregation in the StellaOps Router Gateway.
## Architecture
The Gateway generates OpenAPI 3.1.0 documentation by aggregating schemas and endpoint metadata from connected microservices.
### Component Overview
```
┌─────────────────────────────────────────────────────────────────────┐
│ Gateway │
├─────────────────────────────────────────────────────────────────────┤
│ │
│ ┌──────────────────┐ ┌────────────────────┐ │
│ │ ConnectionManager │───►│ InMemoryRoutingState│ │
│ │ │ │ │ │
│ │ - OnHelloReceived │ │ - Connections[] │ │
│ │ - OnConnClosed │ │ - Endpoints │ │
│ └──────────────────┘ │ - Schemas │ │
│ │ └─────────┬──────────┘ │
│ │ │ │
│ ▼ ▼ │
│ ┌──────────────────┐ ┌────────────────────┐ │
│ │ OpenApiDocument │◄───│ GatewayOpenApi │ │
│ │ Cache │ │ DocumentCache │ │
│ │ │ │ │ │
│ │ - Invalidate() │ │ - TTL expiration │ │
│ └──────────────────┘ │ - ETag generation │ │
│ └─────────┬──────────┘ │
│ │ │
│ ▼ │
│ ┌────────────────────┐ │
│ │ OpenApiDocument │ │
│ │ Generator │ │
│ │ │ │
│ │ - GenerateInfo() │ │
│ │ - GeneratePaths() │ │
│ │ - GenerateTags() │ │
│ │ - GenerateSchemas()│ │
│ └─────────┬──────────┘ │
│ │ │
│ ▼ │
│ ┌────────────────────┐ │
│ │ ClaimSecurity │ │
│ │ Mapper │ │
│ │ │ │
│ │ - SecuritySchemes │ │
│ │ - SecurityRequire │ │
│ └────────────────────┘ │
│ │
└─────────────────────────────────────────────────────────────────────┘
```
### Components
| Component | File | Responsibility |
|-----------|------|----------------|
| `IOpenApiDocumentGenerator` | `OpenApi/IOpenApiDocumentGenerator.cs` | Interface for document generation |
| `OpenApiDocumentGenerator` | `OpenApi/OpenApiDocumentGenerator.cs` | Builds OpenAPI 3.1.0 JSON |
| `IGatewayOpenApiDocumentCache` | `OpenApi/IGatewayOpenApiDocumentCache.cs` | Interface for document caching |
| `GatewayOpenApiDocumentCache` | `OpenApi/GatewayOpenApiDocumentCache.cs` | TTL + invalidation caching |
| `ClaimSecurityMapper` | `OpenApi/ClaimSecurityMapper.cs` | Maps claims to OAuth2 scopes |
| `OpenApiEndpoints` | `OpenApi/OpenApiEndpoints.cs` | HTTP endpoint handlers |
| `OpenApiAggregationOptions` | `OpenApi/OpenApiAggregationOptions.cs` | Configuration options |
---
## OpenApiDocumentGenerator
Generates the complete OpenAPI 3.1.0 document from routing state.
### Process Flow
1. **Collect connections** from `IGlobalRoutingState`
2. **Generate info** section from `OpenApiAggregationOptions`
3. **Generate paths** by iterating all endpoints across connections
4. **Generate components** including schemas and security schemes
5. **Generate tags** from unique service names
### Schema Handling
Schemas are prefixed with service name to avoid naming conflicts:
```csharp
var prefixedId = $"{conn.Instance.ServiceName}_{schemaId}";
// billing_CreateInvoiceRequest
```
### Operation ID Generation
Operation IDs follow a consistent pattern:
```csharp
var operationId = $"{serviceName}_{path}_{method}";
// billing_invoices_POST
```
---
## GatewayOpenApiDocumentCache
Implements caching with TTL expiration and content-based ETags.
### Cache Behavior
| Trigger | Action |
|---------|--------|
| First request | Generate and cache document |
| Subsequent requests (within TTL) | Return cached document |
| TTL expired | Regenerate document |
| Connection added/removed | Invalidate cache |
### ETag Generation
ETags are computed from SHA256 hash of document content:
```csharp
var hash = SHA256.HashData(Encoding.UTF8.GetBytes(documentJson));
var etag = $"\"{Convert.ToHexString(hash)[..16]}\"";
```
### Thread Safety
The cache uses locking to ensure thread-safe regeneration:
```csharp
lock (_lock)
{
if (_cachedDocument is null || IsExpired())
{
RegenerateDocument();
}
}
```
---
## ClaimSecurityMapper
Maps endpoint claim requirements to OpenAPI security schemes.
### Security Scheme Generation
Always generates `BearerAuth` scheme. Generates `OAuth2` scheme only when endpoints have claim requirements:
```csharp
public static JsonObject GenerateSecuritySchemes(
IEnumerable<EndpointDescriptor> endpoints,
string tokenUrl)
{
var schemes = new JsonObject();
// Always add BearerAuth
schemes["BearerAuth"] = new JsonObject { ... };
// Collect scopes from all endpoints
var scopes = CollectScopes(endpoints);
// Add OAuth2 only if scopes exist
if (scopes.Count > 0)
{
schemes["OAuth2"] = GenerateOAuth2Scheme(tokenUrl, scopes);
}
return schemes;
}
```
### Per-Operation Security
Each endpoint with claims gets a security requirement:
```csharp
public static JsonArray GenerateSecurityRequirement(EndpointDescriptor endpoint)
{
if (endpoint.RequiringClaims.Count == 0)
return new JsonArray(); // No security required
return new JsonArray
{
new JsonObject
{
["BearerAuth"] = new JsonArray(),
["OAuth2"] = new JsonArray(claims.Select(c => c.Type))
}
};
}
```
---
## Configuration Reference
### OpenApiAggregationOptions
| Property | Type | Default | Description |
|----------|------|---------|-------------|
| `Title` | `string` | `"StellaOps Gateway API"` | API title |
| `Description` | `string` | `"Unified API..."` | API description |
| `Version` | `string` | `"1.0.0"` | API version |
| `ServerUrl` | `string` | `"/"` | Base server URL |
| `CacheTtlSeconds` | `int` | `60` | Cache TTL |
| `Enabled` | `bool` | `true` | Enable/disable |
| `LicenseName` | `string` | `"AGPL-3.0-or-later"` | License name |
| `ContactName` | `string?` | `null` | Contact name |
| `ContactEmail` | `string?` | `null` | Contact email |
| `TokenUrl` | `string` | `"/auth/token"` | OAuth2 token URL |
### YAML Configuration
```yaml
OpenApi:
Title: "My Gateway API"
Description: "Unified API for all microservices"
Version: "2.0.0"
ServerUrl: "https://api.example.com"
CacheTtlSeconds: 60
Enabled: true
LicenseName: "AGPL-3.0-or-later"
ContactName: "API Team"
ContactEmail: "api@example.com"
TokenUrl: "/auth/token"
```
---
## Service Registration
Services are registered via dependency injection in `ServiceCollectionExtensions`:
```csharp
services.Configure<OpenApiAggregationOptions>(
configuration.GetSection("OpenApi"));
services.AddSingleton<IOpenApiDocumentGenerator, OpenApiDocumentGenerator>();
services.AddSingleton<IGatewayOpenApiDocumentCache, GatewayOpenApiDocumentCache>();
```
Endpoints are mapped in `ApplicationBuilderExtensions`:
```csharp
app.MapGatewayOpenApiEndpoints();
```
---
## Cache Invalidation
The `ConnectionManager` invalidates the cache on connection changes:
```csharp
private Task HandleHelloReceivedAsync(ConnectionState state, HelloPayload payload)
{
_routingState.AddConnection(state);
_openApiCache?.Invalidate(); // Invalidate on new connection
return Task.CompletedTask;
}
private Task HandleConnectionClosedAsync(string connectionId)
{
_routingState.RemoveConnection(connectionId);
_openApiCache?.Invalidate(); // Invalidate on disconnect
return Task.CompletedTask;
}
```
---
## Extension Points
### Custom Routing Plugins
The Gateway supports custom routing plugins via `IRoutingPlugin`. While not directly related to OpenAPI, routing decisions can affect which endpoints are exposed.
### Future Enhancements
Potential extension points for future development:
- **Schema Transformers**: Modify schemas before aggregation
- **Tag Customization**: Custom tag generation logic
- **Response Examples**: Include example responses from connected services
- **Webhooks**: Notify external systems on document changes
---
## Testing
Unit tests are located in `src/Gateway/__Tests/StellaOps.Gateway.WebService.Tests/OpenApi/`:
| Test File | Coverage |
|-----------|----------|
| `OpenApiDocumentGeneratorTests.cs` | Document structure, schema merging, tag generation |
| `GatewayOpenApiDocumentCacheTests.cs` | TTL expiry, invalidation, ETag consistency |
| `ClaimSecurityMapperTests.cs` | Security scheme generation from claims |
### Test Patterns
```csharp
[Fact]
public void GenerateDocument_WithConnections_GeneratesPaths()
{
// Arrange
var endpoint = new EndpointDescriptor { ... };
var connection = CreateConnection("inventory", "1.0.0", endpoint);
_routingState.Setup(x => x.GetAllConnections()).Returns([connection]);
// Act
var document = _sut.GenerateDocument();
// Assert
var doc = JsonDocument.Parse(document);
doc.RootElement.GetProperty("paths")
.TryGetProperty("/api/items", out _)
.Should().BeTrue();
}
```
---
## See Also
- [Schema Validation](../router/schema-validation.md) - JSON Schema validation in microservices
- [OpenAPI Aggregation](../router/openapi-aggregation.md) - Configuration and usage guide
- [API Overview](../../api/overview.md) - General API conventions

View File

@@ -0,0 +1,165 @@
# Router Module
The StellaOps Router is the internal communication infrastructure that enables microservices to communicate through a central gateway using efficient binary protocols.
## Why Another Gateway?
StellaOps already has HTTP-based services. The Router exists because:
1. **Performance**: Binary framing eliminates HTTP overhead for internal traffic
2. **Streaming**: First-class support for large payloads (SBOMs, scan results, evidence bundles)
3. **Cancellation**: Request abortion propagates across service boundaries
4. **Health-aware Routing**: Automatic failover based on heartbeat and latency
5. **Claims-based Auth**: Unified authorization via Authority integration
6. **Transport Flexibility**: UDP for small payloads, TCP/TLS for streams, RabbitMQ for queuing
The Router replaces the Serdica HTTP-to-RabbitMQ pattern with a simpler, generic design.
## Architecture Overview
```
┌─────────────────────────────────┐
│ StellaOps.Gateway.WebService│
HTTP Clients ────────────────────► (HTTP ingress) │
│ │
│ ┌─────────────────────────────┐│
│ │ Endpoint Resolution ││
│ │ Authorization (Claims) ││
│ │ Routing Decision ││
│ │ Transport Dispatch ││
│ └─────────────────────────────┘│
└──────────────┬──────────────────┘
┌─────────────────────────┼─────────────────────────┐
│ │ │
▼ ▼ ▼
┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐
│ Billing │ │ Inventory │ │ Scanner │
│ Microservice │ │ Microservice │ │ Microservice │
│ │ │ │ │ │
│ TCP/TLS │ │ InMemory │ │ RabbitMQ │
└─────────────────┘ └─────────────────┘ └─────────────────┘
```
## Components
| Component | Project | Purpose |
|-----------|---------|---------|
| Gateway | `StellaOps.Gateway.WebService` | HTTP ingress, routing, authorization |
| Microservice SDK | `StellaOps.Microservice` | SDK for building microservices |
| Source Generator | `StellaOps.Microservice.SourceGen` | Compile-time endpoint discovery |
| Common | `StellaOps.Router.Common` | Shared types, frames, interfaces |
| Config | `StellaOps.Router.Config` | Configuration models, YAML binding |
| InMemory Transport | `StellaOps.Router.Transport.InMemory` | Testing transport |
| TCP Transport | `StellaOps.Router.Transport.Tcp` | Production TCP transport |
| TLS Transport | `StellaOps.Router.Transport.Tls` | Encrypted TCP transport |
| UDP Transport | `StellaOps.Router.Transport.Udp` | Small payload transport |
| RabbitMQ Transport | `StellaOps.Router.Transport.RabbitMQ` | Message queue transport |
## Solution Structure
```
StellaOps.Router.slnx
├── src/__Libraries/
│ ├── StellaOps.Router.Common/
│ ├── StellaOps.Router.Config/
│ ├── StellaOps.Router.Transport.InMemory/
│ ├── StellaOps.Router.Transport.Tcp/
│ ├── StellaOps.Router.Transport.Tls/
│ ├── StellaOps.Router.Transport.Udp/
│ ├── StellaOps.Router.Transport.RabbitMQ/
│ ├── StellaOps.Microservice/
│ └── StellaOps.Microservice.SourceGen/
├── src/Gateway/
│ └── StellaOps.Gateway.WebService/
└── tests/
└── (test projects)
```
## Key Documents
| Document | Purpose |
|----------|---------|
| [architecture.md](architecture.md) | Canonical specification and requirements |
| [schema-validation.md](schema-validation.md) | JSON Schema validation feature |
| [openapi-aggregation.md](openapi-aggregation.md) | OpenAPI document generation |
| [migration-guide.md](migration-guide.md) | WebService to Microservice migration |
## Quick Start
### Gateway
```csharp
var builder = WebApplication.CreateBuilder(args);
// Add router services
builder.Services.AddGatewayServices(builder.Configuration);
builder.Services.AddInMemoryTransport(); // or TCP, TLS, etc.
var app = builder.Build();
// Configure pipeline
app.UseGatewayMiddleware();
await app.RunAsync();
```
### Microservice
```csharp
var builder = Host.CreateApplicationBuilder(args);
builder.Services.AddStellaMicroservice(options =>
{
options.ServiceName = "billing";
options.Version = "1.0.0";
options.Region = "us-east-1";
});
builder.Services.AddInMemoryTransportClient();
await builder.Build().RunAsync();
```
### Endpoint Definition
```csharp
[StellaEndpoint("POST", "/invoices")]
[ValidateSchema(Summary = "Create invoice")]
public sealed class CreateInvoiceEndpoint : IStellaEndpoint<CreateInvoiceRequest, CreateInvoiceResponse>
{
public Task<CreateInvoiceResponse> HandleAsync(
CreateInvoiceRequest request,
CancellationToken ct)
{
return Task.FromResult(new CreateInvoiceResponse
{
InvoiceId = Guid.NewGuid().ToString()
});
}
}
```
## Invariants
These are non-negotiable design constraints:
- **Method + Path** is the endpoint identity
- **Strict semver** for version matching
- **Region from GatewayNodeConfig** (not headers/host)
- **No HTTP transport** between gateway and microservices
- **RequiringClaims** (not AllowedRoles) for authorization
- **Opaque body handling** (router doesn't interpret payloads)
## Building
```bash
# Build router solution
dotnet build StellaOps.Router.slnx
# Run tests
dotnet test StellaOps.Router.slnx
# Run gateway
dotnet run --project src/Gateway/StellaOps.Gateway.WebService
```

View File

@@ -0,0 +1,519 @@
# Router Architecture
This document is the canonical specification for the StellaOps Router system.
## System Architecture
### Scope
- A single HTTP ingress service (`StellaOps.Gateway.WebService`) handles all external HTTP traffic
- Microservices communicate with the Gateway using binary transports (TCP, TLS, UDP, RabbitMQ)
- HTTP is not used for internal microservice-to-gateway traffic
- Request/response bodies are opaque to the router (raw bytes/streams)
### Transport Architecture
Each transport connection carries:
- Initial registration (HELLO) and endpoint configuration
- Ongoing heartbeats
- Request/response data frames
- Streaming data frames
- Cancellation frames
```
┌─────────────────┐ ┌─────────────────┐
│ Microservice │ │ Gateway │
│ │ HELLO │ │
│ Endpoints: │ ─────────────────────────►│ Routing │
│ - POST /items │ HEARTBEAT │ State │
│ - GET /items │ ◄────────────────────────►│ │
│ │ │ Connections[] │
│ │ REQUEST / RESPONSE │ │
│ │ ◄────────────────────────►│ │
│ │ │ │
│ │ STREAM_DATA / CANCEL │ │
│ │ ◄────────────────────────►│ │
└─────────────────┘ └─────────────────┘
```
---
## Service Identity
### Instance Identity
Each microservice instance is identified by:
| Field | Type | Description |
|-------|------|-------------|
| `ServiceName` | string | Logical service name (e.g., "billing") |
| `Version` | string | Semantic version (`major.minor.patch`) |
| `Region` | string | Deployment region (e.g., "us-east-1") |
| `InstanceId` | string | Unique instance identifier |
### Version Matching
- Version matching is strict semver equality
- Router only routes to instances with exact version match
- Default version used when client doesn't specify
### Region Configuration
Gateway region comes from `GatewayNodeConfig`:
```csharp
public sealed class GatewayNodeConfig
{
public required string Region { get; init; } // e.g., "eu1"
public required string NodeId { get; init; } // e.g., "gw-eu1-01"
public required string Environment { get; init; } // e.g., "prod"
}
```
Region is never derived from HTTP headers or URL hostnames.
---
## Endpoint Model
### Endpoint Identity
Endpoint identity is `(HTTP Method, Path)`:
| Field | Example |
|-------|---------|
| Method | `GET`, `POST`, `PUT`, `PATCH`, `DELETE` |
| Path | `/invoices`, `/items/{id}`, `/users/{userId}/orders` |
### Endpoint Descriptor
Each endpoint includes:
```csharp
public sealed class EndpointDescriptor
{
public required string Method { get; init; }
public required string Path { get; init; }
public required string ServiceName { get; init; }
public required string Version { get; init; }
public TimeSpan DefaultTimeout { get; init; }
public bool SupportsStreaming { get; init; }
public IReadOnlyList<ClaimRequirement> RequiringClaims { get; init; } = [];
public EndpointSchemaInfo? SchemaInfo { get; init; }
}
```
### Path Matching
- ASP.NET-style route templates
- Parameter segments: `{id}`, `{userId}`
- Case sensitivity and trailing slash handling follow ASP.NET conventions
---
## Routing Algorithm
### Instance Selection
Given `(ServiceName, Version, Method, Path)`:
1. **Filter candidates**:
- Match `ServiceName` exactly
- Match `Version` exactly (strict semver)
- Health status in acceptable set (`Healthy` or `Degraded`)
2. **Region preference**:
- Prefer instances where `Region == GatewayNodeConfig.Region`
- Fall back to configured neighbor regions
- Fall back to all other regions
3. **Within region tier**:
- Prefer lower `AveragePingMs`
- If tied, prefer more recent `LastHeartbeatUtc`
- If still tied, use round-robin balancing
### Instance Health
```csharp
public enum InstanceHealthStatus
{
Unknown,
Healthy,
Degraded,
Draining,
Unhealthy
}
```
Health metadata per connection:
| Field | Type | Description |
|-------|------|-------------|
| `Status` | enum | Current health status |
| `LastHeartbeatUtc` | DateTime | Last heartbeat timestamp |
| `AveragePingMs` | double | Average round-trip latency |
---
## Transport Layer
### Transport Types
| Transport | Use Case | Streaming | Notes |
|-----------|----------|-----------|-------|
| InMemory | Testing | Yes | In-process channels |
| TCP | Production | Yes | Length-prefixed frames |
| TLS | Secure | Yes | Certificate-based encryption |
| UDP | Small payloads | No | Single datagram per frame |
| RabbitMQ | Queuing | Yes | Exchange/queue routing |
### Transport Plugin Interface
```csharp
public interface ITransportServer
{
Task StartAsync(CancellationToken ct);
Task StopAsync(CancellationToken ct);
event Func<ConnectionState, HelloPayload, Task> OnHelloReceived;
event Func<ConnectionState, HeartbeatPayload, Task> OnHeartbeatReceived;
event Func<string, Task> OnConnectionClosed;
}
public interface ITransportClient
{
Task ConnectAsync(CancellationToken ct);
Task DisconnectAsync(CancellationToken ct);
Task SendFrameAsync(Frame frame, CancellationToken ct);
}
```
### Frame Types
```csharp
public enum FrameType : byte
{
Hello = 1,
Heartbeat = 2,
Request = 3,
Response = 4,
RequestStreamData = 5,
ResponseStreamData = 6,
Cancel = 7
}
```
---
## Gateway Pipeline
### HTTP Middleware Stack
```
Request ─►│ ForwardedHeaders │
│ RequestLogging │
│ ErrorHandling │
│ Authentication │
│ EndpointResolution │ ◄── (Method, Path) → EndpointDescriptor
│ Authorization │ ◄── RequiringClaims check
│ RoutingDecision │ ◄── Select connection/instance
│ TransportDispatch │ ◄── Send to microservice
```
### Connection State
Per-connection state maintained by Gateway:
```csharp
public sealed class ConnectionState
{
public required string ConnectionId { get; init; }
public required InstanceDescriptor Instance { get; init; }
public InstanceHealthStatus Status { get; set; }
public DateTime? LastHeartbeatUtc { get; set; }
public double AveragePingMs { get; set; }
public TransportType TransportType { get; init; }
public Dictionary<(string Method, string Path), EndpointDescriptor> Endpoints { get; } = new();
public IReadOnlyDictionary<string, SchemaDefinition> Schemas { get; init; } = new Dictionary<string, SchemaDefinition>();
}
```
### Payload Handling
The Gateway treats bodies as opaque byte sequences:
- No deserialization or schema interpretation
- Headers and bytes forwarded as-is
- Schema validation is microservice responsibility
### Payload Limits
Configurable limits protect against resource exhaustion:
| Limit | Scope |
|-------|-------|
| `MaxRequestBytesPerCall` | Single request |
| `MaxRequestBytesPerConnection` | All requests on connection |
| `MaxAggregateInflightBytes` | All in-flight across gateway |
Exceeded limits result in:
- Early rejection (HTTP 413) if `Content-Length` known
- Mid-stream abort with CANCEL frame
- Appropriate error response (413 or 503)
---
## Microservice SDK
### Configuration
```csharp
services.AddStellaMicroservice(options =>
{
options.ServiceName = "billing";
options.Version = "1.0.0";
options.Region = "us-east-1";
options.InstanceId = Guid.NewGuid().ToString();
options.ServiceDescription = "Invoice processing service";
});
```
### Endpoint Declaration
Attributes:
```csharp
[StellaEndpoint("POST", "/invoices")]
public sealed class CreateInvoiceEndpoint : IStellaEndpoint<CreateInvoiceRequest, CreateInvoiceResponse>
```
### Handler Interfaces
**Typed handler** (JSON serialization):
```csharp
public interface IStellaEndpoint<TRequest, TResponse>
{
Task<TResponse> HandleAsync(TRequest request, CancellationToken ct);
}
public interface IStellaEndpoint<TResponse>
{
Task<TResponse> HandleAsync(CancellationToken ct);
}
```
**Raw handler** (streaming):
```csharp
public interface IRawStellaEndpoint
{
Task<RawResponse> HandleAsync(RawRequestContext ctx, CancellationToken ct);
}
```
### Endpoint Discovery
Two mechanisms:
1. **Source Generator** (preferred): Compile-time discovery via Roslyn
2. **Reflection** (fallback): Runtime assembly scanning
### Connection Behavior
On connection:
1. Send HELLO with instance info and endpoints
2. Start heartbeat timer
3. Listen for REQUEST frames
HELLO payload:
```csharp
public sealed class HelloPayload
{
public required InstanceDescriptor Instance { get; init; }
public required IReadOnlyList<EndpointDescriptor> Endpoints { get; init; }
public IReadOnlyDictionary<string, SchemaDefinition> Schemas { get; init; } = new Dictionary<string, SchemaDefinition>();
public ServiceOpenApiInfo? OpenApiInfo { get; init; }
}
```
---
## Authorization
### Claims-based Model
Authorization uses `RequiringClaims`, not roles:
```csharp
public sealed class ClaimRequirement
{
public required string Type { get; init; }
public string? Value { get; init; }
}
```
### Precedence
1. Microservice provides defaults in HELLO
2. Authority can override centrally
3. Gateway enforces final effective claims
### Enforcement
Gateway `AuthorizationMiddleware`:
- Validates user principal has all required claims
- Empty claims list = authenticated access only
- Missing claim = 403 Forbidden
---
## Cancellation
### CANCEL Frame
```csharp
public sealed class CancelPayload
{
public required string Reason { get; init; }
// Values: "ClientDisconnected", "Timeout", "PayloadLimitExceeded", "Shutdown"
}
```
### Gateway sends CANCEL when:
- HTTP client disconnects (`HttpContext.RequestAborted`)
- Request timeout elapses
- Payload limit exceeded
- Gateway shutdown
### Microservice handles CANCEL:
- Maps correlation ID to `CancellationTokenSource`
- Calls `Cancel()` on the source
- Handler receives cancellation via `CancellationToken`
---
## Streaming
### Buffered vs Streaming
| Mode | Request Body | Response Body | Use Case |
|------|--------------|---------------|----------|
| Buffered | Full in memory | Full in memory | Small payloads |
| Streaming | Chunked frames | Chunked frames | Large payloads |
### Frame Flow (Streaming)
```
Gateway Microservice
│ │
│ REQUEST (headers only) │
│ ────────────────────────────────────►│
│ │
│ REQUEST_STREAM_DATA (chunk 1) │
│ ────────────────────────────────────►│
│ │
│ REQUEST_STREAM_DATA (chunk n) │
│ ────────────────────────────────────►│
│ │
│ REQUEST_STREAM_DATA (final=true) │
│ ────────────────────────────────────►│
│ │
│ RESPONSE │
│◄────────────────────────────────────│
│ │
│ RESPONSE_STREAM_DATA │
│◄────────────────────────────────────│
```
---
## Heartbeat & Health
### Heartbeat Frame
Sent at regular intervals over the same connection as requests:
```csharp
public sealed class HeartbeatPayload
{
public required InstanceHealthStatus Status { get; init; }
public int InflightRequests { get; init; }
public double ErrorRate { get; init; }
}
```
### Health Tracking
Gateway tracks:
- `LastHeartbeatUtc` per connection
- Derives status from heartbeat recency
- Marks stale instances as Unhealthy
- Uses health in routing decisions
---
## Configuration
### Router YAML
```yaml
# router.yaml
Gateway:
Region: "us-east-1"
NodeId: "gw-east-01"
Environment: "production"
PayloadLimits:
MaxRequestBytesPerCall: 10485760 # 10 MB
MaxRequestBytesPerConnection: 104857600 # 100 MB
MaxAggregateInflightBytes: 1073741824 # 1 GB
Services:
- ServiceName: billing
DefaultVersion: "1.0.0"
DefaultTransport: Tcp
Endpoints:
- Method: POST
Path: /invoices
TimeoutSeconds: 30
RequiringClaims:
- Type: "invoices:write"
OpenApi:
Title: "StellaOps Gateway API"
CacheTtlSeconds: 60
```
### Hot Reload
- YAML changes picked up at runtime
- Routing state updated without restart
- New services/endpoints added dynamically
---
## Error Mapping
| Condition | HTTP Status |
|-----------|-------------|
| Version not found | 404 Not Found |
| No healthy instance | 503 Service Unavailable |
| Request timeout | 504 Gateway Timeout |
| Payload too large | 413 Payload Too Large |
| Unauthorized | 401 Unauthorized |
| Missing claims | 403 Forbidden |
| Validation error | 422 Unprocessable Entity |
| Internal error | 500 Internal Server Error |
---
## See Also
- [schema-validation.md](schema-validation.md) - JSON Schema validation
- [openapi-aggregation.md](openapi-aggregation.md) - OpenAPI document generation
- [migration-guide.md](migration-guide.md) - WebService to Microservice migration

View File

@@ -0,0 +1,462 @@
# StellaOps Router Migration Guide
This guide describes how to migrate existing `StellaOps.*.WebService` projects to the new microservice pattern with the StellaOps Router.
## Overview
The router provides a transport-agnostic communication layer between services, replacing direct HTTP calls with efficient binary protocols (TCP, TLS, UDP, RabbitMQ). Benefits include:
- **Performance**: Binary framing vs HTTP overhead
- **Streaming**: First-class support for large payloads
- **Cancellation**: Propagated across service boundaries
- **Claims**: Authority-integrated authorization
- **Health**: Automatic heartbeat and failover
## Prerequisites
Before migrating, ensure:
1. Router infrastructure is deployed (Gateway, transports)
2. Authority is configured with endpoint claims
3. Local development environment has router.yaml configured
## Migration Strategies
### Strategy A: In-Place Adaptation
Best for services that need to maintain HTTP compatibility during transition.
```
┌─────────────────────────────────────┐
│ StellaOps.*.WebService │
│ ┌─────────────────────────────────┐│
│ │ Existing HTTP Controllers ││◄── HTTP clients (legacy)
│ └─────────────────────────────────┘│
│ ┌─────────────────────────────────┐│
│ │ [StellaEndpoint] Handlers ││◄── Router (new)
│ └─────────────────────────────────┘│
│ ┌─────────────────────────────────┐│
│ │ Shared Domain Logic ││
│ └─────────────────────────────────┘│
└─────────────────────────────────────┘
```
**Steps:**
1. Add `StellaOps.Microservice` package reference
2. Create handler classes for each HTTP route
3. Handlers call existing service layer
4. Register with router alongside HTTP
5. Test via router
6. Shift traffic gradually
7. Remove HTTP controllers when ready
**Pros:**
- Gradual migration
- No downtime
- Can roll back easily
**Cons:**
- Dual maintenance during transition
- May delay cleanup
### Strategy B: Clean Split
Best for major refactoring or when HTTP compatibility is not needed.
```
┌─────────────────────────────────────┐
│ StellaOps.*.Domain │ ◄── Shared library
│ (extracted business logic) │
└─────────────────────────────────────┘
▲ ▲
│ │
┌─────────┴───────┐ ┌───────┴─────────┐
│ (Legacy) │ │ (New) │
│ *.WebService │ │ *.Microservice │
│ HTTP only │ │ Router only │
└─────────────────┘ └─────────────────┘
```
**Steps:**
1. Extract domain logic to `.Domain` library
2. Create new `.Microservice` project
3. Implement handlers using domain library
4. Deploy alongside WebService
5. Shift traffic to router
6. Deprecate WebService
**Pros:**
- Clean architecture
- No legacy code in new project
- Clear separation of concerns
**Cons:**
- More upfront work
- Requires domain extraction
## Controller to Handler Mapping
### Before (ASP.NET Controller)
```csharp
[ApiController]
[Route("api/invoices")]
public class InvoicesController : ControllerBase
{
private readonly IInvoiceService _service;
[HttpPost]
[Authorize(Roles = "billing-admin")]
public async Task<IActionResult> Create(
[FromBody] CreateInvoiceRequest request,
CancellationToken ct)
{
var invoice = await _service.CreateAsync(request);
return Ok(new { invoice.Id });
}
[HttpGet("{id}")]
public async Task<IActionResult> Get(string id)
{
var invoice = await _service.GetAsync(id);
if (invoice == null) return NotFound();
return Ok(invoice);
}
}
```
### After (Microservice Handler)
```csharp
// Handler for POST /api/invoices
[StellaEndpoint("POST", "/api/invoices", RequiredClaims = ["invoices:write"])]
public sealed class CreateInvoiceEndpoint : IStellaEndpoint<CreateInvoiceRequest, CreateInvoiceResponse>
{
private readonly IInvoiceService _service;
public CreateInvoiceEndpoint(IInvoiceService service) => _service = service;
public async Task<CreateInvoiceResponse> HandleAsync(
CreateInvoiceRequest request,
CancellationToken ct)
{
var invoice = await _service.CreateAsync(request, ct);
return new CreateInvoiceResponse { InvoiceId = invoice.Id };
}
}
// Handler for GET /api/invoices/{id}
[StellaEndpoint("GET", "/api/invoices/{id}", RequiredClaims = ["invoices:read"])]
public sealed class GetInvoiceEndpoint : IStellaEndpoint<GetInvoiceRequest, GetInvoiceResponse>
{
private readonly IInvoiceService _service;
public GetInvoiceEndpoint(IInvoiceService service) => _service = service;
public async Task<GetInvoiceResponse> HandleAsync(
GetInvoiceRequest request,
CancellationToken ct)
{
var invoice = await _service.GetAsync(request.Id, ct);
return new GetInvoiceResponse
{
InvoiceId = invoice?.Id,
Found = invoice != null
};
}
}
```
## CancellationToken Wiring
**This is the #1 source of migration bugs.** Every async operation must receive and respect the cancellation token.
### Checklist
For each migrated handler, verify:
- [ ] Handler accepts CancellationToken parameter (automatic with IStellaEndpoint)
- [ ] Token passed to all database calls
- [ ] Token passed to all HTTP client calls
- [ ] Token passed to all file I/O operations
- [ ] Long-running loops check `ct.IsCancellationRequested`
- [ ] Token passed to `Task.Delay`, `WaitAsync`, etc.
### Example: Before (missing tokens)
```csharp
public async Task<Invoice> CreateAsync(CreateInvoiceRequest request)
{
var invoice = new Invoice(request);
await _db.Invoices.AddAsync(invoice); // Missing token!
await _db.SaveChangesAsync(); // Missing token!
await _notifier.SendAsync(invoice); // Missing token!
return invoice;
}
```
### Example: After (proper wiring)
```csharp
public async Task<Invoice> CreateAsync(CreateInvoiceRequest request, CancellationToken ct)
{
ct.ThrowIfCancellationRequested();
var invoice = new Invoice(request);
await _db.Invoices.AddAsync(invoice, ct);
await _db.SaveChangesAsync(ct);
await _notifier.SendAsync(invoice, ct);
return invoice;
}
```
## Streaming Migration
### File Upload: Before
```csharp
[HttpPost("upload")]
public async Task<IActionResult> Upload(IFormFile file)
{
using var stream = file.OpenReadStream();
await _storage.SaveAsync(stream);
return Ok();
}
```
### File Upload: After
```csharp
[StellaEndpoint("POST", "/upload", SupportsStreaming = true)]
public sealed class UploadEndpoint : IRawStellaEndpoint
{
private readonly IStorageService _storage;
public UploadEndpoint(IStorageService storage) => _storage = storage;
public async Task<RawResponse> HandleAsync(RawRequestContext ctx, CancellationToken ct)
{
// ctx.Body is already a stream - no buffering needed
var path = await _storage.SaveAsync(ctx.Body, ct);
return RawResponse.Ok($"{{\"path\":\"{path}\"}}");
}
}
```
### File Download: Before
```csharp
[HttpGet("download/{id}")]
public async Task<IActionResult> Download(string id)
{
var stream = await _storage.GetAsync(id);
return File(stream, "application/octet-stream");
}
```
### File Download: After
```csharp
[StellaEndpoint("GET", "/download/{id}", SupportsStreaming = true)]
public sealed class DownloadEndpoint : IRawStellaEndpoint
{
private readonly IStorageService _storage;
public DownloadEndpoint(IStorageService storage) => _storage = storage;
public async Task<RawResponse> HandleAsync(RawRequestContext ctx, CancellationToken ct)
{
var id = ctx.PathParameters["id"];
var stream = await _storage.GetAsync(id, ct);
return RawResponse.Stream(stream, "application/octet-stream");
}
}
```
## Authorization Migration
### Before: [Authorize] Attribute
```csharp
[Authorize(Roles = "admin,billing-manager")]
public async Task<IActionResult> Delete(string id) { ... }
```
### After: RequiredClaims
```csharp
[StellaEndpoint("DELETE", "/invoices/{id}", RequiredClaims = ["invoices:delete"])]
public sealed class DeleteInvoiceEndpoint : IStellaEndpoint<...> { ... }
```
Claims are configured in Authority and enforced by the Gateway's AuthorizationMiddleware.
## Migration Checklist Template
Use this checklist for each service migration:
```markdown
# Migration Checklist: [ServiceName]
## Inventory
- [ ] List all HTTP routes (Method + Path)
- [ ] Identify streaming endpoints
- [ ] Identify authorization requirements
- [ ] Document external dependencies
## Preparation
- [ ] Add StellaOps.Microservice package
- [ ] Add StellaOps.Router.Transport.* package(s)
- [ ] Configure router connection in Program.cs
- [ ] Set up local gateway for testing
## Per-Route Migration
For each route:
- [ ] Create [StellaEndpoint] handler class
- [ ] Define request/response record types
- [ ] Map path parameters
- [ ] Wire CancellationToken throughout
- [ ] Convert to IRawStellaEndpoint if streaming
- [ ] Add RequiredClaims
- [ ] Write unit tests
- [ ] Write integration tests
## Cutover
- [ ] Deploy alongside existing WebService
- [ ] Verify via router routing
- [ ] Shift percentage of traffic
- [ ] Monitor for errors
- [ ] Full cutover
- [ ] Remove WebService HTTP listeners
## Cleanup
- [ ] Remove unused controller code
- [ ] Remove HTTP pipeline configuration
- [ ] Update OpenAPI documentation
- [ ] Update client SDKs
```
## Service Inventory
| Module | WebService Project | Priority | Complexity | Notes |
|--------|-------------------|----------|------------|-------|
| Gateway | StellaOps.Gateway.WebService | N/A | N/A | IS the router |
| Concelier | StellaOps.Concelier.WebService | High | Medium | Advisory ingestion |
| Scanner | StellaOps.Scanner.WebService | High | High | Streaming scans |
| Attestor | StellaOps.Attestor.WebService | Medium | Medium | Attestation gen |
| Excititor | StellaOps.Excititor.WebService | Medium | Low | VEX processing |
| Orchestrator | StellaOps.Orchestrator.WebService | Medium | Medium | Job coordination |
| Scheduler | StellaOps.Scheduler.WebService | Low | Low | Job scheduling |
| Notify | StellaOps.Notify.WebService | Low | Low | Notifications |
| Notifier | StellaOps.Notifier.WebService | Low | Low | Alert dispatch |
| Signer | StellaOps.Signer.WebService | Medium | Low | Crypto signing |
| Findings | StellaOps.Findings.Ledger.WebService | Medium | Medium | Results storage |
| EvidenceLocker | StellaOps.EvidenceLocker.WebService | Low | Medium | Blob storage |
| ExportCenter | StellaOps.ExportCenter.WebService | Low | Medium | Report generation |
| IssuerDirectory | StellaOps.IssuerDirectory.WebService | Low | Low | Issuer lookup |
| PacksRegistry | StellaOps.PacksRegistry.WebService | Low | Low | Pack management |
| RiskEngine | StellaOps.RiskEngine.WebService | Medium | Medium | Risk calculation |
| TaskRunner | StellaOps.TaskRunner.WebService | Low | Medium | Task execution |
| TimelineIndexer | StellaOps.TimelineIndexer.WebService | Low | Low | Event indexing |
| AdvisoryAI | StellaOps.AdvisoryAI.WebService | Low | Medium | AI assistance |
## Testing During Migration
### Unit Tests
Test handlers in isolation using mocked dependencies:
```csharp
[Fact]
public async Task CreateInvoice_ValidRequest_ReturnsInvoiceId()
{
// Arrange
var mockService = new Mock<IInvoiceService>();
mockService.Setup(s => s.CreateAsync(It.IsAny<CreateInvoiceRequest>(), It.IsAny<CancellationToken>()))
.ReturnsAsync(new Invoice { Id = "INV-123" });
var endpoint = new CreateInvoiceEndpoint(mockService.Object);
// Act
var response = await endpoint.HandleAsync(
new CreateInvoiceRequest { Amount = 100 },
CancellationToken.None);
// Assert
response.InvoiceId.Should().Be("INV-123");
}
```
### Integration Tests
Use WebApplicationFactory for the Gateway and actual microservice instances:
```csharp
public sealed class InvoiceTests : IClassFixture<GatewayFixture>
{
private readonly GatewayFixture _fixture;
[Fact]
public async Task CreateAndGetInvoice_WorksEndToEnd()
{
var createResponse = await _fixture.Client.PostAsJsonAsync("/api/invoices",
new { Amount = 100 });
createResponse.StatusCode.Should().Be(HttpStatusCode.OK);
var created = await createResponse.Content.ReadFromJsonAsync<CreateInvoiceResponse>();
var getResponse = await _fixture.Client.GetAsync($"/api/invoices/{created.InvoiceId}");
getResponse.StatusCode.Should().Be(HttpStatusCode.OK);
}
}
```
## Common Migration Issues
### 1. Missing CancellationToken Propagation
**Symptom:** Requests continue processing after client disconnects.
**Fix:** Pass `CancellationToken` to all async operations.
### 2. IFormFile Not Available
**Symptom:** Compilation error on `IFormFile` parameter.
**Fix:** Convert to `IRawStellaEndpoint` for streaming.
### 3. HttpContext Not Available
**Symptom:** Code references `HttpContext` for headers, claims.
**Fix:** Use `RawRequestContext` for raw endpoints, or inject claims via Authority.
### 4. Return Type Mismatch
**Symptom:** Handler returns `IActionResult`.
**Fix:** Define proper response record type, return that instead.
### 5. Route Parameter Not Extracted
**Symptom:** Path parameters like `{id}` not populated.
**Fix:** For `IStellaEndpoint`, add property to request type. For `IRawStellaEndpoint`, use `ctx.PathParameters["id"]`.
## Next Steps
1. Choose a low-risk service for pilot migration (Scheduler recommended)
2. Follow the Migration Checklist
3. Document lessons learned
4. Proceed with higher-priority services
5. Eventually merge all to use router exclusively
---
## See Also
- [Router Architecture](architecture.md) - System specification
- [Schema Validation](schema-validation.md) - JSON Schema validation
- [OpenAPI Aggregation](openapi-aggregation.md) - OpenAPI document generation

View File

@@ -0,0 +1,503 @@
# OpenAPI Aggregation
This document describes how the StellaOps Gateway aggregates OpenAPI documentation from connected microservices into a unified specification.
## Overview
The Gateway automatically generates a single OpenAPI 3.1.0 document that aggregates all endpoints from connected microservices. This provides:
- **Unified API documentation**: All services documented in one place
- **Dynamic updates**: Document regenerates when services connect/disconnect
- **Standard compliance**: OpenAPI 3.1.0 with native JSON Schema draft 2020-12 support
- **Multiple formats**: Available as JSON or YAML
- **Efficient caching**: ETag-based caching with configurable TTL
### How It Works
```
┌──────────────┐ HELLO ┌──────────────┐ GET /openapi.json ┌──────────────┐
│ Billing │ ──────────► │ │ ◄───────────────────── │ Client │
│ Service │ + schemas │ Gateway │ │ │
└──────────────┘ │ │ OpenAPI 3.1.0 │ │
│ │ ─────────────────────► │ │
┌──────────────┐ HELLO │ │ unified document └──────────────┘
│ Inventory │ ──────────► │ │
│ Service │ + schemas └──────────────┘
└──────────────┘
```
1. Microservices send schemas and endpoint metadata via HELLO payload
2. Gateway stores this information in routing state
3. OpenAPI generator aggregates all connected services
4. Document is cached and served via HTTP endpoints
---
## Configuration
### OpenApiAggregationOptions
Configure OpenAPI aggregation in your Gateway configuration:
```yaml
# router.yaml or appsettings.yaml
OpenApi:
Title: "My API Gateway"
Description: "Unified API for all microservices"
Version: "2.0.0"
ServerUrl: "https://api.example.com"
CacheTtlSeconds: 60
Enabled: true
LicenseName: "AGPL-3.0-or-later"
ContactName: "API Team"
ContactEmail: "api@example.com"
TokenUrl: "/auth/token"
```
### Configuration Reference
| Property | Type | Default | Description |
|----------|------|---------|-------------|
| `Title` | `string` | `"StellaOps Gateway API"` | API title in OpenAPI info section |
| `Description` | `string` | `"Unified API aggregating all connected microservices."` | API description |
| `Version` | `string` | `"1.0.0"` | API version number |
| `ServerUrl` | `string` | `"/"` | Base server URL |
| `CacheTtlSeconds` | `int` | `60` | Cache time-to-live in seconds |
| `Enabled` | `bool` | `true` | Enable/disable OpenAPI aggregation |
| `LicenseName` | `string` | `"AGPL-3.0-or-later"` | License name in OpenAPI info |
| `ContactName` | `string?` | `null` | Contact name (optional) |
| `ContactEmail` | `string?` | `null` | Contact email (optional) |
| `TokenUrl` | `string` | `"/auth/token"` | OAuth2 token endpoint URL |
### Disabling OpenAPI
To disable OpenAPI aggregation entirely:
```yaml
OpenApi:
Enabled: false
```
---
## Endpoints
### Discovery Endpoint
```http
GET /.well-known/openapi
```
Returns metadata about the OpenAPI document:
**Response:**
```json
{
"openapi_json": "/openapi.json",
"openapi_yaml": "/openapi.yaml",
"etag": "\"5d41402abc4b2a76b9719d911017c592\"",
"generated_at": "2025-01-15T10:30:00.0000000Z"
}
```
### OpenAPI JSON
```http
GET /openapi.json
```
Returns the full OpenAPI 3.1.0 specification in JSON format.
**Headers:**
- `Cache-Control: public, max-age=60`
- `ETag: "<content-hash>"`
- `Content-Type: application/json; charset=utf-8`
**Conditional Request:**
```http
GET /openapi.json
If-None-Match: "5d41402abc4b2a76b9719d911017c592"
```
Returns `304 Not Modified` if content unchanged.
### OpenAPI YAML
```http
GET /openapi.yaml
```
Returns the full OpenAPI 3.1.0 specification in YAML format.
**Headers:**
- `Cache-Control: public, max-age=60`
- `ETag: "<content-hash>"`
- `Content-Type: application/yaml; charset=utf-8`
---
## Security Mapping
The Gateway automatically maps claim requirements to OpenAPI security schemes.
### Claim to Scope Mapping
When endpoints define `RequiringClaims`, these are converted to OAuth2 scopes:
```csharp
// Endpoint with claim requirements
[StellaEndpoint("POST", "/invoices")]
[RequireClaim("billing:write")]
public sealed class CreateInvoiceEndpoint : IStellaEndpoint<...>
```
Becomes in OpenAPI:
```json
{
"paths": {
"/invoices": {
"post": {
"security": [
{
"BearerAuth": [],
"OAuth2": ["billing:write"]
}
]
}
}
}
}
```
### Security Schemes
The Gateway generates two security schemes:
#### BearerAuth
HTTP Bearer token authentication (always present):
```json
{
"BearerAuth": {
"type": "http",
"scheme": "bearer",
"bearerFormat": "JWT",
"description": "JWT Bearer token authentication"
}
}
```
#### OAuth2
Client credentials flow with collected scopes (only if endpoints have claims):
```json
{
"OAuth2": {
"type": "oauth2",
"flows": {
"clientCredentials": {
"tokenUrl": "/auth/token",
"scopes": {
"billing:write": "Access scope: billing:write",
"billing:read": "Access scope: billing:read",
"inventory:read": "Access scope: inventory:read"
}
}
}
}
}
```
### Scope Collection
Scopes are automatically collected from all connected services. If multiple endpoints require the same claim, it appears only once in the scopes list.
---
## Generated Document Structure
The aggregated OpenAPI document follows this structure:
```json
{
"openapi": "3.1.0",
"info": {
"title": "StellaOps Gateway API",
"version": "1.0.0",
"description": "Unified API aggregating all connected microservices.",
"license": {
"name": "AGPL-3.0-or-later"
},
"contact": {
"name": "API Team",
"email": "api@example.com"
}
},
"servers": [
{
"url": "/"
}
],
"paths": {
"/invoices": {
"post": {
"operationId": "billing_invoices_POST",
"tags": ["billing"],
"summary": "Create invoice",
"description": "Creates a new draft invoice",
"security": [
{
"BearerAuth": [],
"OAuth2": ["billing:write"]
}
],
"requestBody": {
"required": true,
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/billing_CreateInvoiceRequest"
}
}
}
},
"responses": {
"200": {
"description": "Success",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/billing_CreateInvoiceResponse"
}
}
}
},
"400": { "description": "Bad Request" },
"401": { "description": "Unauthorized" },
"404": { "description": "Not Found" },
"422": { "description": "Validation Error" },
"500": { "description": "Internal Server Error" }
}
}
},
"/items": {
"get": {
"operationId": "inventory_items_GET",
"tags": ["inventory"],
"summary": "List items",
"responses": {
"200": { "description": "Success" }
}
}
}
},
"components": {
"schemas": {
"billing_CreateInvoiceRequest": {
"type": "object",
"required": ["customerId", "amount"],
"properties": {
"customerId": { "type": "string" },
"amount": { "type": "number" },
"description": { "type": ["string", "null"] },
"lineItems": {
"type": "array",
"items": { "$ref": "#/components/schemas/billing_LineItem" }
}
}
},
"billing_CreateInvoiceResponse": {
"type": "object",
"required": ["invoiceId", "createdAt", "status"],
"properties": {
"invoiceId": { "type": "string" },
"createdAt": { "type": "string", "format": "date-time" },
"status": { "type": "string" }
}
},
"billing_LineItem": {
"type": "object",
"required": ["description", "amount"],
"properties": {
"description": { "type": "string" },
"amount": { "type": "number" },
"quantity": { "type": "integer", "default": 1 }
}
}
},
"securitySchemes": {
"BearerAuth": {
"type": "http",
"scheme": "bearer",
"bearerFormat": "JWT",
"description": "JWT Bearer token authentication"
},
"OAuth2": {
"type": "oauth2",
"flows": {
"clientCredentials": {
"tokenUrl": "/auth/token",
"scopes": {
"billing:write": "Access scope: billing:write"
}
}
}
}
}
},
"tags": [
{
"name": "billing",
"description": "billing microservice (v1.0.0)"
},
{
"name": "inventory",
"description": "inventory microservice (v2.0.0)"
}
]
}
```
### Schema Prefixing
Schemas are prefixed with the service name to prevent naming conflicts:
| Service | Original Type | Prefixed Schema ID |
|---------|--------------|-------------------|
| `billing` | `CreateInvoiceRequest` | `billing_CreateInvoiceRequest` |
| `inventory` | `GetItemResponse` | `inventory_GetItemResponse` |
### Tag Generation
Tags are automatically generated from connected services:
- Tag name: Service name (lowercase)
- Tag description: Service description from `OpenApiInfo` or auto-generated
### Operation IDs
Operation IDs follow the pattern: `{serviceName}_{path}_{method}`
Example: `billing_invoices_POST`
---
## Cache Behavior
### TTL-Based Expiration
The document cache expires based on `CacheTtlSeconds` (default: 60 seconds):
```yaml
OpenApi:
CacheTtlSeconds: 30 # More frequent regeneration
```
Setting `CacheTtlSeconds: 0` regenerates the document on every request (not recommended for production).
### Connection-Based Invalidation
The cache is automatically invalidated when:
1. A new microservice connects (HELLO received)
2. A microservice disconnects (connection closed)
This ensures the OpenAPI document always reflects currently connected services.
### ETag Consistency
The ETag is computed from the document content hash (SHA256). This ensures:
- Same content = same ETag
- Content changes = new ETag
- Clients can use conditional requests to avoid re-downloading unchanged documents
### Recommended Client Strategy
```javascript
// Store ETag from previous response
let cachedETag = localStorage.getItem('openapi-etag');
const response = await fetch('/openapi.json', {
headers: cachedETag ? { 'If-None-Match': cachedETag } : {}
});
if (response.status === 304) {
// Use cached document
return getCachedDocument();
}
// Store new ETag and document
localStorage.setItem('openapi-etag', response.headers.get('ETag'));
const document = await response.json();
cacheDocument(document);
return document;
```
---
## Service Registration
### Microservice Options
Configure service metadata that appears in OpenAPI:
```csharp
services.AddStellaMicroservice(options =>
{
options.ServiceName = "billing";
options.ServiceDescription = "Invoice and payment processing service";
options.ContactInfo = "billing-team@example.com";
});
```
### Service OpenAPI Info
The `ServiceOpenApiInfo` is sent in the HELLO payload:
```json
{
"instance": { ... },
"endpoints": [ ... ],
"schemas": { ... },
"openApiInfo": {
"title": "billing",
"description": "Invoice and payment processing service"
}
}
```
This description appears in the tag entry for the service.
---
## Troubleshooting
### Document Not Updating
1. Check `CacheTtlSeconds` - may need to wait for TTL expiration
2. Verify service connected successfully (check Gateway logs)
3. Force refresh by restarting the Gateway
### Missing Schemas
1. Ensure `[ValidateSchema]` attribute is applied to endpoints
2. Check for schema parsing errors in Gateway logs
3. Verify endpoint implements `IStellaEndpoint<TRequest, TResponse>`
### Security Schemes Not Appearing
1. OAuth2 scheme only appears if endpoints have claim requirements
2. Check `RequiringClaims` is populated on endpoint descriptors
3. Verify claim types are being transmitted correctly
---
## See Also
- [Schema Validation](schema-validation.md) - JSON Schema validation reference
- [API Overview](../../api/overview.md) - General API conventions
- [Gateway OpenAPI](../gateway/openapi.md) - Gateway OpenAPI implementation details

View File

@@ -0,0 +1,426 @@
# JSON Schema Validation
This document describes the JSON Schema validation feature in the StellaOps Router/Microservice SDK.
## Overview
The StellaOps Microservice SDK provides compile-time JSON Schema generation and runtime request/response validation. Schemas are automatically generated from your C# types using a source generator, then transmitted to the Gateway via the HELLO payload where they power both runtime validation and OpenAPI documentation.
### Key Features
- **Compile-time schema generation**: JSON Schema draft 2020-12 generated from C# types
- **Runtime validation**: Request bodies validated against schema before reaching handlers
- **OpenAPI 3.1.0 compatibility**: Native JSON Schema support in OpenAPI 3.1.0
- **Automatic documentation**: Schemas flow to Gateway for unified OpenAPI documentation
- **External schema support**: Override generated schemas with embedded resource files
### Benefits
| Benefit | Description |
|---------|-------------|
| Type safety | Contract enforcement between clients and services |
| Early error detection | Invalid requests rejected with 422 before handler execution |
| Documentation automation | No manual schema maintenance required |
| Interoperability | Standard JSON Schema works with any tooling |
---
## Quick Start
### 1. Add the ValidateSchema Attribute
```csharp
using StellaOps.Microservice;
[StellaEndpoint("POST", "/invoices")]
[ValidateSchema]
public sealed class CreateInvoiceEndpoint : IStellaEndpoint<CreateInvoiceRequest, CreateInvoiceResponse>
{
public Task<CreateInvoiceResponse> HandleAsync(
CreateInvoiceRequest request,
CancellationToken cancellationToken)
{
// Request is already validated against JSON Schema
return Task.FromResult(new CreateInvoiceResponse
{
InvoiceId = Guid.NewGuid().ToString(),
CreatedAt = DateTime.UtcNow,
Status = "draft"
});
}
}
```
### 2. Define Your Request/Response Types
```csharp
public sealed record CreateInvoiceRequest
{
public required string CustomerId { get; init; }
public required decimal Amount { get; init; }
public string? Description { get; init; }
public List<LineItem> LineItems { get; init; } = [];
}
public sealed record CreateInvoiceResponse
{
public required string InvoiceId { get; init; }
public required DateTime CreatedAt { get; init; }
public required string Status { get; init; }
}
```
### 3. Build Your Project
The source generator automatically creates JSON Schema definitions at compile time. These schemas are included in the HELLO payload when your microservice connects to the Gateway.
---
## Attribute Reference
### ValidateSchemaAttribute
Enables JSON Schema validation for an endpoint.
```csharp
[AttributeUsage(AttributeTargets.Class, AllowMultiple = false, Inherited = false)]
public sealed class ValidateSchemaAttribute : Attribute
```
### Properties
| Property | Type | Default | Description |
|----------|------|---------|-------------|
| `ValidateRequest` | `bool` | `true` | Enable request body validation |
| `ValidateResponse` | `bool` | `false` | Enable response body validation |
| `RequestSchemaResource` | `string?` | `null` | Embedded resource path to external request schema |
| `ResponseSchemaResource` | `string?` | `null` | Embedded resource path to external response schema |
| `Summary` | `string?` | `null` | OpenAPI operation summary |
| `Description` | `string?` | `null` | OpenAPI operation description |
| `Tags` | `string[]?` | `null` | OpenAPI tags for grouping endpoints |
| `Deprecated` | `bool` | `false` | Mark endpoint as deprecated in OpenAPI |
### Validation Properties
#### ValidateRequest (default: true)
Controls whether incoming request bodies are validated against the generated schema.
```csharp
// Request validation enabled (default)
[ValidateSchema(ValidateRequest = true)]
// Disable request validation
[ValidateSchema(ValidateRequest = false)]
```
#### ValidateResponse (default: false)
Enables response body validation. Useful for debugging or strict contract enforcement, but adds overhead.
```csharp
// Enable response validation
[ValidateSchema(ValidateResponse = true)]
```
#### External Schema Files
Override generated schemas with embedded resource files when you need custom schema definitions:
```csharp
[ValidateSchema(RequestSchemaResource = "Schemas.create-order.json")]
public sealed class CreateOrderEndpoint : IStellaEndpoint<CreateOrderRequest, CreateOrderResponse>
```
The schema file must be embedded as a resource in your assembly.
### Documentation Properties
#### Summary and Description
Provide OpenAPI documentation for the operation:
```csharp
[ValidateSchema(
Summary = "Create a new invoice",
Description = "Creates a draft invoice for the specified customer. The invoice must be finalized before it can be sent.")]
public sealed class CreateInvoiceEndpoint : IStellaEndpoint<CreateInvoiceRequest, CreateInvoiceResponse>
```
#### Tags
Override the default service-based tag grouping:
```csharp
[ValidateSchema(Tags = ["Billing", "Invoices"])]
public sealed class CreateInvoiceEndpoint : IStellaEndpoint<CreateInvoiceRequest, CreateInvoiceResponse>
```
#### Deprecated
Mark an endpoint as deprecated in OpenAPI documentation:
```csharp
[ValidateSchema(Deprecated = true, Description = "Use /v2/invoices instead")]
public sealed class CreateInvoiceV1Endpoint : IStellaEndpoint<CreateInvoiceRequest, CreateInvoiceResponse>
```
---
## Schema Discovery Endpoints
The Gateway exposes endpoints to discover and retrieve schemas from connected microservices.
### Discovery Endpoint
```http
GET /.well-known/openapi
```
Returns metadata about the OpenAPI document including available format URLs:
```json
{
"openapi_json": "/openapi.json",
"openapi_yaml": "/openapi.yaml",
"etag": "\"a1b2c3d4\"",
"generated_at": "2025-01-15T10:30:00.000Z"
}
```
### OpenAPI Document (JSON)
```http
GET /openapi.json
Accept: application/json
```
Returns the full OpenAPI 3.1.0 specification in JSON format. Supports ETag-based caching:
```http
GET /openapi.json
If-None-Match: "a1b2c3d4"
```
Returns `304 Not Modified` if the document hasn't changed.
### OpenAPI Document (YAML)
```http
GET /openapi.yaml
Accept: application/yaml
```
Returns the full OpenAPI 3.1.0 specification in YAML format.
### Caching Behavior
All OpenAPI endpoints support HTTP caching:
| Header | Value | Description |
|--------|-------|-------------|
| `Cache-Control` | `public, max-age=60` | Client-side caching for 60 seconds |
| `ETag` | `"<hash>"` | Content hash for conditional requests |
---
## Examples
### Basic Endpoint with Validation
```csharp
[StellaEndpoint("POST", "/invoices")]
[ValidateSchema]
public sealed class CreateInvoiceEndpoint : IStellaEndpoint<CreateInvoiceRequest, CreateInvoiceResponse>
{
private readonly ILogger<CreateInvoiceEndpoint> _logger;
public CreateInvoiceEndpoint(ILogger<CreateInvoiceEndpoint> logger)
{
_logger = logger;
}
public Task<CreateInvoiceResponse> HandleAsync(
CreateInvoiceRequest request,
CancellationToken cancellationToken)
{
_logger.LogInformation(
"Creating invoice for customer {CustomerId} with amount {Amount}",
request.CustomerId,
request.Amount);
return Task.FromResult(new CreateInvoiceResponse
{
InvoiceId = $"INV-{Guid.NewGuid():N}"[..16].ToUpperInvariant(),
CreatedAt = DateTime.UtcNow,
Status = "draft"
});
}
}
```
### Endpoint with Full Documentation
```csharp
[StellaEndpoint("POST", "/invoices", TimeoutSeconds = 30)]
[ValidateSchema(
Summary = "Create invoice",
Description = "Creates a new draft invoice for the specified customer. Line items are optional but recommended for itemized billing.",
Tags = ["Billing", "Invoices"])]
public sealed class CreateInvoiceEndpoint : IStellaEndpoint<CreateInvoiceRequest, CreateInvoiceResponse>
{
// Implementation...
}
```
### Deprecated Endpoint
```csharp
[StellaEndpoint("POST", "/v1/invoices")]
[ValidateSchema(
Deprecated = true,
Summary = "Create invoice (deprecated)",
Description = "This endpoint is deprecated. Use POST /v2/invoices instead.")]
public sealed class CreateInvoiceV1Endpoint : IStellaEndpoint<CreateInvoiceRequest, CreateInvoiceResponse>
{
// Implementation...
}
```
### Request with Complex Types
```csharp
public sealed record CreateInvoiceRequest
{
/// <summary>
/// The customer identifier.
/// </summary>
public required string CustomerId { get; init; }
/// <summary>
/// The invoice amount in the default currency.
/// </summary>
public required decimal Amount { get; init; }
/// <summary>
/// Optional description for the invoice.
/// </summary>
public string? Description { get; init; }
/// <summary>
/// Line items for itemized billing.
/// </summary>
public List<LineItem> LineItems { get; init; } = [];
}
public sealed record LineItem
{
public required string Description { get; init; }
public required decimal Amount { get; init; }
public int Quantity { get; init; } = 1;
}
```
The source generator produces JSON Schema that includes:
- Required property validation (`required` keyword)
- Type validation (`type` keyword)
- Nullable property handling (`null` in type union)
- Nested object schemas
---
## Architecture
### Schema Flow Diagram
```
┌─────────────────────────────────────────────────────────────────────┐
│ COMPILE TIME │
├─────────────────────────────────────────────────────────────────────┤
│ │
│ C# Types Source Generator Generated Code │
│ ───────── ──────────────── ────────────── │
│ CreateInvoice ──► Analyzes types ──► JSON Schema defs │
│ Request Extracts metadata GetSchemaDefinitions│
│ CreateInvoice Generates schemas EndpointDescriptor │
│ Response with SchemaInfo │
│ │
└─────────────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────────┐
│ RUNTIME │
├─────────────────────────────────────────────────────────────────────┤
│ │
│ Microservice Start HELLO Payload Gateway Storage │
│ ───────────────── ───────────── ─────────────── │
│ RouterConnection ──► Instance info ──► ConnectionState │
│ Manager Endpoints[] Schemas dictionary │
│ Schemas{} OpenApiInfo │
│ OpenApiInfo │
│ │
└─────────────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────────┐
│ HTTP EXPOSURE │
├─────────────────────────────────────────────────────────────────────┤
│ │
│ OpenApiDocument Document Cache HTTP Endpoints │
│ Generator ───────────── ────────────── │
│ ────────────── │
│ Aggregates all ──► TTL-based cache ──► GET /openapi.json │
│ connected services ETag generation GET /openapi.yaml │
│ Prefixes schemas Invalidation on GET /.well-known/ │
│ by service name connection change openapi │
│ │
└─────────────────────────────────────────────────────────────────────┘
```
### Schema Naming Convention
Schemas are prefixed with the service name to avoid conflicts:
- Service: `Billing`
- Type: `CreateInvoiceRequest`
- Schema ID: `Billing_CreateInvoiceRequest`
This allows multiple services to define types with the same name without collisions.
---
## Error Handling
### Validation Errors
When request validation fails, the endpoint returns a `422 Unprocessable Entity` response with details:
```json
{
"type": "https://tools.ietf.org/html/rfc4918#section-11.2",
"title": "Validation Error",
"status": 422,
"errors": [
{
"path": "$.customerId",
"message": "Required property 'customerId' is missing"
},
{
"path": "$.amount",
"message": "Value must be a number"
}
]
}
```
### Invalid Schemas
If a schema cannot be parsed (e.g., malformed JSON in external schema file), the schema is skipped and a warning is logged. The endpoint will still function but without schema validation.
---
## See Also
- [OpenAPI Aggregation](openapi-aggregation.md) - How schemas become OpenAPI documentation
- [API Overview](../../api/overview.md) - General API conventions
- [Router Architecture](architecture.md) - Router system overview