Add unit tests for PackRunAttestation and SealedInstallEnforcer
Some checks failed
Docs CI / lint-and-preview (push) Has been cancelled
AOC Guard CI / aoc-guard (push) Has been cancelled
AOC Guard CI / aoc-verify (push) Has been cancelled
Concelier Attestation Tests / attestation-tests (push) Has been cancelled
Policy Lint & Smoke / policy-lint (push) Has been cancelled
release-manifest-verify / verify (push) Has been cancelled
Some checks failed
Docs CI / lint-and-preview (push) Has been cancelled
AOC Guard CI / aoc-guard (push) Has been cancelled
AOC Guard CI / aoc-verify (push) Has been cancelled
Concelier Attestation Tests / attestation-tests (push) Has been cancelled
Policy Lint & Smoke / policy-lint (push) Has been cancelled
release-manifest-verify / verify (push) Has been cancelled
- Implement comprehensive tests for PackRunAttestationService, covering attestation generation, verification, and event emission. - Add tests for SealedInstallEnforcer to validate sealed install requirements and enforcement logic. - Introduce a MonacoLoaderService stub for testing purposes to prevent Monaco workers/styles from loading during Karma runs.
This commit is contained in:
3
docs/router/archived/README.md
Normal file
3
docs/router/archived/README.md
Normal file
@@ -0,0 +1,3 @@
|
||||
# Router Sprint Archives
|
||||
|
||||
These sprint plans were deleted on 2025-12-05 during test refactors. They have been restored from commit `53508ceccb2884bd15bf02104e5af48fd570e456` and placed here as archives (do not reactivate without review).
|
||||
121
docs/router/archived/SPRINT_7000_0001_0001_router_skeleton.md
Normal file
121
docs/router/archived/SPRINT_7000_0001_0001_router_skeleton.md
Normal file
@@ -0,0 +1,121 @@
|
||||
# Sprint 7000-0001-0001 · Router Foundation · Project Skeleton
|
||||
|
||||
## Topic & Scope
|
||||
|
||||
Phase 1 of Router implementation: establish the project skeleton with all required directories, solution files, and empty stubs. This sprint creates the structural foundation that all subsequent router sprints depend on.
|
||||
|
||||
**Goal:** Get a clean, compiling skeleton in place that matches the spec and folder conventions, with zero real logic and minimal dependencies.
|
||||
|
||||
**Working directories:**
|
||||
- `src/__Libraries/StellaOps.Router.Common/`
|
||||
- `src/__Libraries/StellaOps.Router.Config/`
|
||||
- `src/__Libraries/StellaOps.Microservice/`
|
||||
- `src/__Libraries/StellaOps.Microservice.SourceGen/`
|
||||
- `src/Gateway/StellaOps.Gateway.WebService/`
|
||||
- `tests/StellaOps.Router.Common.Tests/`
|
||||
- `tests/StellaOps.Gateway.WebService.Tests/`
|
||||
- `tests/StellaOps.Microservice.Tests/`
|
||||
|
||||
**Isolation strategy:** Router uses a separate `StellaOps.Router.sln` solution file to enable fully independent building and testing. This prevents any impact on the main `StellaOps.sln` until the migration phase.
|
||||
|
||||
## Dependencies & Concurrency
|
||||
|
||||
- **Upstream:** None. This is the first router sprint.
|
||||
- **Downstream:** All other router sprints depend on this skeleton.
|
||||
- **Parallel work:** None possible until this sprint completes.
|
||||
- **Cross-module impact:** None. All work is in new directories.
|
||||
|
||||
## Documentation Prerequisites
|
||||
|
||||
- `docs/router/specs.md` (canonical specification - READ FIRST)
|
||||
- `docs/router/implplan.md` (implementation plan overview)
|
||||
- `docs/router/01-Step.md` (detailed task breakdown for this sprint)
|
||||
|
||||
> **BLOCKED Tasks:** Before working on BLOCKED tasks, review [../implplan/BLOCKED_DEPENDENCY_TREE.md](../implplan/BLOCKED_DEPENDENCY_TREE.md) for root blockers and dependencies.
|
||||
|
||||
## Invariants (from specs.md)
|
||||
|
||||
Before coding, acknowledge these non-negotiables:
|
||||
- Method + Path identity for endpoints
|
||||
- Strict semver for versions
|
||||
- Region from `GatewayNodeConfig.Region` (no host/header derivation)
|
||||
- No HTTP transport for microservice-to-router communications
|
||||
- Single connection carrying HELLO + HEARTBEAT + REQUEST/RESPONSE + CANCEL
|
||||
- Router treats body as opaque bytes/streams
|
||||
- `RequiringClaims` replaces any form of `AllowedRoles`
|
||||
|
||||
## Delivery Tracker
|
||||
|
||||
| # | Task ID | Status | Description | Working Directory |
|
||||
|---|---------|--------|-------------|-------------------|
|
||||
| 1 | SKEL-001 | DONE | Create directory structure (`src/__Libraries/`, `src/Gateway/`, `tests/`) | repo root |
|
||||
| 2 | SKEL-002 | DONE | Create `StellaOps.Router.slnx` solution file at repo root | repo root |
|
||||
| 3 | SKEL-003 | DONE | Create `StellaOps.Router.Common` classlib project | `src/__Libraries/StellaOps.Router.Common/` |
|
||||
| 4 | SKEL-004 | DONE | Create `StellaOps.Router.Config` classlib project | `src/__Libraries/StellaOps.Router.Config/` |
|
||||
| 5 | SKEL-005 | DONE | Create `StellaOps.Microservice` classlib project | `src/__Libraries/StellaOps.Microservice/` |
|
||||
| 6 | SKEL-006 | DONE | Create `StellaOps.Microservice.SourceGen` classlib stub | `src/__Libraries/StellaOps.Microservice.SourceGen/` |
|
||||
| 7 | SKEL-007 | DONE | Create `StellaOps.Gateway.WebService` webapi project | `src/Gateway/StellaOps.Gateway.WebService/` |
|
||||
| 8 | SKEL-008 | DONE | Create xunit test projects for Common, Gateway, Microservice | `tests/` |
|
||||
| 9 | SKEL-009 | DONE | Wire project references per dependency graph | all projects |
|
||||
| 10 | SKEL-010 | DONE | Add common settings (net10.0, nullable, LangVersion) to each csproj | all projects |
|
||||
| 11 | SKEL-011 | DONE | Stub empty placeholder types in each project (no logic) | all projects |
|
||||
| 12 | SKEL-012 | DONE | Add dummy smoke tests so CI passes | `tests/` |
|
||||
| 13 | SKEL-013 | DONE | Verify `dotnet build StellaOps.Router.slnx` succeeds | repo root |
|
||||
| 14 | SKEL-014 | DONE | Verify `dotnet test StellaOps.Router.slnx` passes | repo root |
|
||||
| 15 | SKEL-015 | DONE | Update `docs/router/README.md` with solution overview | `docs/router/` |
|
||||
|
||||
## Project Reference Graph
|
||||
|
||||
```
|
||||
StellaOps.Gateway.WebService
|
||||
├── StellaOps.Router.Common
|
||||
└── StellaOps.Router.Config
|
||||
└── StellaOps.Router.Common
|
||||
|
||||
StellaOps.Microservice
|
||||
└── StellaOps.Router.Common
|
||||
|
||||
StellaOps.Microservice.SourceGen
|
||||
(no references yet - stub only)
|
||||
|
||||
Test projects reference their corresponding main projects.
|
||||
```
|
||||
|
||||
## Stub Types to Create
|
||||
|
||||
### StellaOps.Router.Common
|
||||
- Enums: `TransportType`, `FrameType`, `InstanceHealthStatus`
|
||||
- Models: `ClaimRequirement`, `EndpointDescriptor`, `InstanceDescriptor`, `ConnectionState`, `Frame`
|
||||
- Interfaces: `IGlobalRoutingState`, `IRoutingPlugin`, `ITransportServer`, `ITransportClient`
|
||||
|
||||
### StellaOps.Router.Config
|
||||
- `RouterConfig`, `ServiceConfig`, `PayloadLimits` (property-only classes)
|
||||
|
||||
### StellaOps.Microservice
|
||||
- `StellaMicroserviceOptions`, `RouterEndpointConfig`
|
||||
- `ServiceCollectionExtensions.AddStellaMicroservice()` (empty body)
|
||||
|
||||
### StellaOps.Gateway.WebService
|
||||
- `GatewayNodeConfig` with Region, NodeId, Environment
|
||||
- Minimal `Program.cs` that builds and runs (no logic)
|
||||
|
||||
## Exit Criteria
|
||||
|
||||
Before marking this sprint DONE:
|
||||
1. [x] `dotnet build StellaOps.Router.slnx` succeeds with zero warnings
|
||||
2. [x] `dotnet test StellaOps.Router.slnx` passes (even with dummy tests)
|
||||
3. [x] All project names match spec: `StellaOps.Gateway.WebService`, `StellaOps.Router.Common`, `StellaOps.Router.Config`, `StellaOps.Microservice`
|
||||
4. [x] No real business logic exists (no transport logic, no routing decisions, no YAML parsing)
|
||||
5. [x] `docs/router/README.md` exists and points to `specs.md`
|
||||
|
||||
## Execution Log
|
||||
|
||||
| Date (UTC) | Update | Owner |
|
||||
|------------|--------|-------|
|
||||
| 2024-12-04 | Sprint completed: all skeleton projects created, build and tests passing | Claude |
|
||||
|
||||
## Decisions & Risks
|
||||
|
||||
- Router uses a separate solution file (`StellaOps.Router.sln`) to enable isolated development. This will be merged into main `StellaOps.sln` during the migration phase.
|
||||
- Target framework is `net10.0` to match the rest of StellaOps.
|
||||
- `StellaOps.Microservice.SourceGen` is created as a plain classlib for now; it will be converted to a Source Generator project in a later sprint.
|
||||
157
docs/router/archived/SPRINT_7000_0001_0002_router_common.md
Normal file
157
docs/router/archived/SPRINT_7000_0001_0002_router_common.md
Normal file
@@ -0,0 +1,157 @@
|
||||
# Sprint 7000-0001-0002 · Router Foundation · Common Library Models
|
||||
|
||||
## Topic & Scope
|
||||
|
||||
Phase 2 of Router implementation: implement the shared core model in `StellaOps.Router.Common`. This sprint makes Common the single, stable contract layer that Gateway, Microservice SDK, and transports all depend on.
|
||||
|
||||
**Goal:** Lock down the domain vocabulary. Implement all data types and interfaces with **no behavior** - just shapes that match `specs.md`.
|
||||
|
||||
**Working directory:** `src/__Libraries/StellaOps.Router.Common/`
|
||||
|
||||
**Key principle:** Changes to `StellaOps.Router.Common` after this sprint must be rare and reviewed. Everything else depends on it.
|
||||
|
||||
## Dependencies & Concurrency
|
||||
|
||||
- **Upstream:** SPRINT_7000_0001_0001 (skeleton must be complete)
|
||||
- **Downstream:** All other router sprints depend on these contracts
|
||||
- **Parallel work:** None possible until this sprint completes
|
||||
- **Cross-module impact:** None. All work is in `StellaOps.Router.Common`
|
||||
|
||||
## Documentation Prerequisites
|
||||
|
||||
- `docs/router/specs.md` (canonical specification - READ FIRST, sections 2-13)
|
||||
- `docs/router/02-Step.md` (detailed task breakdown for this sprint)
|
||||
|
||||
> **BLOCKED Tasks:** Before working on BLOCKED tasks, review [../implplan/BLOCKED_DEPENDENCY_TREE.md](../implplan/BLOCKED_DEPENDENCY_TREE.md) for root blockers and dependencies.
|
||||
|
||||
## Delivery Tracker
|
||||
|
||||
| # | Task ID | Status | Description | Notes |
|
||||
|---|---------|--------|-------------|-------|
|
||||
| 1 | CMN-001 | DONE | Create `/Enums/TransportType.cs` with `[Udp, Tcp, Certificate, RabbitMq]` | No HTTP type per spec |
|
||||
| 2 | CMN-002 | DONE | Create `/Enums/FrameType.cs` with Hello, Heartbeat, EndpointsUpdate, Request, RequestStreamData, Response, ResponseStreamData, Cancel | |
|
||||
| 3 | CMN-003 | DONE | Create `/Enums/InstanceHealthStatus.cs` with Unknown, Healthy, Degraded, Draining, Unhealthy | |
|
||||
| 4 | CMN-010 | DONE | Create `/Models/ClaimRequirement.cs` with Type (required) and Value (optional) | Replaces AllowedRoles |
|
||||
| 5 | CMN-011 | DONE | Create `/Models/EndpointDescriptor.cs` with ServiceName, Version, Method, Path, DefaultTimeout, SupportsStreaming, RequiringClaims | |
|
||||
| 6 | CMN-012 | DONE | Create `/Models/InstanceDescriptor.cs` with InstanceId, ServiceName, Version, Region | |
|
||||
| 7 | CMN-013 | DONE | Create `/Models/ConnectionState.cs` with ConnectionId, Instance, Status, LastHeartbeatUtc, AveragePingMs, TransportType, Endpoints | |
|
||||
| 8 | CMN-014 | DONE | Create `/Models/RoutingContext.cs` matching spec (neutral context, no ASP.NET dependency) | |
|
||||
| 9 | CMN-015 | DONE | Create `/Models/RoutingDecision.cs` with Endpoint, Connection, TransportType, EffectiveTimeout | |
|
||||
| 10 | CMN-016 | DONE | Create `/Models/PayloadLimits.cs` with MaxRequestBytesPerCall, MaxRequestBytesPerConnection, MaxAggregateInflightBytes | |
|
||||
| 11 | CMN-020 | DONE | Create `/Models/Frame.cs` with Type, CorrelationId, Payload | |
|
||||
| 12 | CMN-021 | DONE | Create `/Models/HelloPayload.cs` with InstanceDescriptor and list of EndpointDescriptors | |
|
||||
| 13 | CMN-022 | DONE | Create `/Models/HeartbeatPayload.cs` with InstanceId, Status, metrics | |
|
||||
| 14 | CMN-023 | DONE | Create `/Models/CancelPayload.cs` with Reason | |
|
||||
| 15 | CMN-030 | DONE | Create `/Abstractions/IGlobalRoutingState.cs` interface | |
|
||||
| 16 | CMN-031 | DONE | Create `/Abstractions/IRoutingPlugin.cs` interface | |
|
||||
| 17 | CMN-032 | DONE | Create `/Abstractions/ITransportServer.cs` interface | |
|
||||
| 18 | CMN-033 | DONE | Create `/Abstractions/ITransportClient.cs` interface | |
|
||||
| 19 | CMN-034 | DONE | Create `/Abstractions/IRegionProvider.cs` interface (optional, if spec requires) | |
|
||||
| 20 | CMN-040 | DONE | Write shape tests for EndpointDescriptor, ConnectionState | Already covered in existing tests |
|
||||
| 21 | CMN-041 | DONE | Write enum completeness tests for FrameType | |
|
||||
| 22 | CMN-042 | DONE | Verify Common compiles with zero warnings (nullable enabled) | |
|
||||
| 23 | CMN-043 | DONE | Verify Common only references BCL (no ASP.NET, no serializers) | |
|
||||
|
||||
## File Layout
|
||||
|
||||
```
|
||||
/src/__Libraries/StellaOps.Router.Common/
|
||||
/Enums/
|
||||
TransportType.cs
|
||||
FrameType.cs
|
||||
InstanceHealthStatus.cs
|
||||
/Models/
|
||||
ClaimRequirement.cs
|
||||
EndpointDescriptor.cs
|
||||
InstanceDescriptor.cs
|
||||
ConnectionState.cs
|
||||
RoutingContext.cs
|
||||
RoutingDecision.cs
|
||||
PayloadLimits.cs
|
||||
Frame.cs
|
||||
HelloPayload.cs
|
||||
HeartbeatPayload.cs
|
||||
CancelPayload.cs
|
||||
/Abstractions/
|
||||
IGlobalRoutingState.cs
|
||||
IRoutingPlugin.cs
|
||||
ITransportClient.cs
|
||||
ITransportServer.cs
|
||||
IRegionProvider.cs
|
||||
```
|
||||
|
||||
## Interface Signatures (from specs.md)
|
||||
|
||||
### IGlobalRoutingState
|
||||
```csharp
|
||||
public interface IGlobalRoutingState
|
||||
{
|
||||
EndpointDescriptor? ResolveEndpoint(string method, string path);
|
||||
IReadOnlyList<ConnectionState> GetConnectionsFor(
|
||||
string serviceName, string version, string method, string path);
|
||||
}
|
||||
```
|
||||
|
||||
### IRoutingPlugin
|
||||
```csharp
|
||||
public interface IRoutingPlugin
|
||||
{
|
||||
Task<RoutingDecision?> ChooseInstanceAsync(
|
||||
RoutingContext context, CancellationToken cancellationToken);
|
||||
}
|
||||
```
|
||||
|
||||
### ITransportServer
|
||||
```csharp
|
||||
public interface ITransportServer
|
||||
{
|
||||
Task StartAsync(CancellationToken cancellationToken);
|
||||
Task StopAsync(CancellationToken cancellationToken);
|
||||
}
|
||||
```
|
||||
|
||||
### ITransportClient
|
||||
```csharp
|
||||
public interface ITransportClient
|
||||
{
|
||||
Task<Frame> SendRequestAsync(
|
||||
ConnectionState connection, Frame requestFrame,
|
||||
TimeSpan timeout, CancellationToken cancellationToken);
|
||||
Task SendCancelAsync(
|
||||
ConnectionState connection, Guid correlationId, string? reason = null);
|
||||
Task SendStreamingAsync(
|
||||
ConnectionState connection, Frame requestHeader, Stream requestBody,
|
||||
Func<Stream, Task> readResponseBody, PayloadLimits limits,
|
||||
CancellationToken cancellationToken);
|
||||
}
|
||||
```
|
||||
|
||||
## Design Constraints
|
||||
|
||||
1. **No behavior:** Only shapes - no LINQ-heavy methods, no routing algorithms, no network code
|
||||
2. **No serialization:** No JSON/MessagePack references; Common only defines shapes
|
||||
3. **Immutability preferred:** Use `init` properties for descriptors; `ConnectionState` health fields may be mutable
|
||||
4. **BCL only:** No ASP.NET or third-party package dependencies
|
||||
5. **Nullable enabled:** All code must compile with zero nullable warnings
|
||||
|
||||
## Exit Criteria
|
||||
|
||||
Before marking this sprint DONE:
|
||||
1. [x] All types from `specs.md` Common section exist with matching names and properties
|
||||
2. [x] Common compiles with zero warnings
|
||||
3. [x] Common only references BCL (verify no package references in .csproj)
|
||||
4. [x] No behavior/logic in any type (pure DTOs and interfaces)
|
||||
5. [x] `StellaOps.Router.Common.Tests` runs and passes
|
||||
6. [x] `docs/router/specs.md` is updated if any discrepancy found (or code matches spec)
|
||||
|
||||
## Execution Log
|
||||
|
||||
| Date (UTC) | Update | Owner |
|
||||
|------------|--------|-------|
|
||||
| 2024-12-04 | Sprint completed: all models and interfaces implemented per spec | Claude |
|
||||
|
||||
## Decisions & Risks
|
||||
|
||||
- `RoutingContext` uses a neutral model (not ASP.NET `HttpContext`) to keep Common free of web dependencies. Gateway will adapt from `HttpContext` to this neutral model.
|
||||
- `ConnectionState.Endpoints` uses `(string Method, string Path)` tuple as key for dictionary lookups.
|
||||
- Frame payloads are `byte[]` - serialization happens at the transport layer, not in Common.
|
||||
121
docs/router/archived/SPRINT_7000_0002_0001_inmemory_transport.md
Normal file
121
docs/router/archived/SPRINT_7000_0002_0001_inmemory_transport.md
Normal file
@@ -0,0 +1,121 @@
|
||||
# Sprint 7000-0002-0001 · Router Transport · InMemory Plugin
|
||||
|
||||
## Topic & Scope
|
||||
|
||||
Build a fake "in-memory" transport plugin for development and testing. This transport proves the HELLO/HEARTBEAT/REQUEST/RESPONSE/CANCEL semantics and routing logic **without** dealing with sockets and RabbitMQ yet.
|
||||
|
||||
**Goal:** Enable unit and integration testing of the router and SDK by providing an in-process transport where frames are passed via channels/queues in memory.
|
||||
|
||||
**Working directory:** `src/__Libraries/StellaOps.Router.Transport.InMemory/`
|
||||
|
||||
**Key principle:** This plugin will never ship to production; it's only for dev tests and CI. It must fully implement all transport abstractions so that switching to real transports later requires zero changes to Gateway or Microservice SDK code.
|
||||
|
||||
## Dependencies & Concurrency
|
||||
|
||||
- **Upstream:** SPRINT_7000_0001_0002 (Common models must be complete)
|
||||
- **Downstream:** SDK and Gateway sprints depend on this for testing
|
||||
- **Parallel work:** Can run in parallel with CMN-040/041/042/043 test tasks if Common models are done
|
||||
- **Cross-module impact:** None. Creates new directory only.
|
||||
|
||||
## Documentation Prerequisites
|
||||
|
||||
- `docs/router/specs.md` (sections 5, 10 - Transport and Cancellation requirements)
|
||||
- `docs/router/03-Step.md` (detailed task breakdown)
|
||||
- `docs/router/implplan.md` (phase 3 guidance)
|
||||
|
||||
> **BLOCKED Tasks:** Before working on BLOCKED tasks, review [../implplan/BLOCKED_DEPENDENCY_TREE.md](../implplan/BLOCKED_DEPENDENCY_TREE.md) for root blockers and dependencies.
|
||||
|
||||
## Delivery Tracker
|
||||
|
||||
| # | Task ID | Status | Description | Notes |
|
||||
|---|---------|--------|-------------|-------|
|
||||
| 1 | MEM-001 | DONE | Create `StellaOps.Router.Transport.InMemory` classlib project | Add to StellaOps.Router.sln |
|
||||
| 2 | MEM-002 | DONE | Add project reference to `StellaOps.Router.Common` | |
|
||||
| 3 | MEM-010 | DONE | Implement `InMemoryTransportServer` : `ITransportServer` | Gateway side |
|
||||
| 4 | MEM-011 | DONE | Implement `InMemoryTransportClient` : `ITransportClient` | Microservice side |
|
||||
| 5 | MEM-012 | DONE | Create shared `InMemoryConnectionRegistry` (concurrent dictionary keyed by ConnectionId) | Thread-safe |
|
||||
| 6 | MEM-013 | DONE | Create `InMemoryChannel` for bidirectional frame passing | Use System.Threading.Channels |
|
||||
| 7 | MEM-020 | DONE | Implement HELLO frame handling (client → server) | |
|
||||
| 8 | MEM-021 | DONE | Implement HEARTBEAT frame handling (client → server) | |
|
||||
| 9 | MEM-022 | DONE | Implement REQUEST frame handling (server → client) | |
|
||||
| 10 | MEM-023 | DONE | Implement RESPONSE frame handling (client → server) | |
|
||||
| 11 | MEM-024 | DONE | Implement CANCEL frame handling (bidirectional) | |
|
||||
| 12 | MEM-025 | DONE | Implement REQUEST_STREAM_DATA / RESPONSE_STREAM_DATA frame handling | For streaming support |
|
||||
| 13 | MEM-030 | DONE | Create `InMemoryTransportOptions` for configuration | Timeouts, buffer sizes |
|
||||
| 14 | MEM-031 | DONE | Create DI registration extension `AddInMemoryTransport()` | |
|
||||
| 15 | MEM-040 | DONE | Write integration tests for HELLO/HEARTBEAT flow | |
|
||||
| 16 | MEM-041 | DONE | Write integration tests for REQUEST/RESPONSE flow | |
|
||||
| 17 | MEM-042 | DONE | Write integration tests for CANCEL flow | |
|
||||
| 18 | MEM-043 | DONE | Write integration tests for streaming flow | |
|
||||
| 19 | MEM-050 | DONE | Create test project `StellaOps.Router.Transport.InMemory.Tests` | |
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
┌──────────────────────┐ InMemoryConnectionRegistry ┌──────────────────────┐
|
||||
│ Gateway │ (ConcurrentDictionary<ConnectionId, │ Microservice │
|
||||
│ (InMemoryTransport │◄──── InMemoryChannel>) ────►│ (InMemoryTransport │
|
||||
│ Server) │ │ Client) │
|
||||
└──────────────────────┘ └──────────────────────┘
|
||||
│ │
|
||||
│ Channel<Frame> ToMicroservice ─────────────────────────────────────►│
|
||||
│◄─────────────────────────────────────────────── Channel<Frame> ToGateway
|
||||
│ │
|
||||
```
|
||||
|
||||
## InMemoryChannel Design
|
||||
|
||||
```csharp
|
||||
internal sealed class InMemoryChannel
|
||||
{
|
||||
public string ConnectionId { get; }
|
||||
public Channel<Frame> ToMicroservice { get; } // Gateway writes, SDK reads
|
||||
public Channel<Frame> ToGateway { get; } // SDK writes, Gateway reads
|
||||
public InstanceDescriptor? Instance { get; set; }
|
||||
public CancellationTokenSource LifetimeToken { get; }
|
||||
}
|
||||
```
|
||||
|
||||
## Frame Flow Examples
|
||||
|
||||
### HELLO Flow
|
||||
1. Microservice SDK calls `InMemoryTransportClient.ConnectAsync()`
|
||||
2. Client creates `InMemoryChannel`, registers in `InMemoryConnectionRegistry`
|
||||
3. Client sends HELLO frame via `ToGateway` channel
|
||||
4. Server reads from `ToGateway`, processes HELLO, updates `ConnectionState`
|
||||
|
||||
### REQUEST/RESPONSE Flow
|
||||
1. Gateway receives HTTP request
|
||||
2. Gateway sends REQUEST frame via `ToMicroservice` channel
|
||||
3. SDK reads from `ToMicroservice`, invokes handler
|
||||
4. SDK sends RESPONSE frame via `ToGateway` channel
|
||||
5. Gateway reads from `ToGateway`, returns HTTP response
|
||||
|
||||
### CANCEL Flow
|
||||
1. HTTP client disconnects (or timeout)
|
||||
2. Gateway sends CANCEL frame via `ToMicroservice` channel
|
||||
3. SDK reads CANCEL, cancels handler's CancellationToken
|
||||
4. SDK optionally sends partial RESPONSE or no response
|
||||
|
||||
## Exit Criteria
|
||||
|
||||
Before marking this sprint DONE:
|
||||
1. [x] `InMemoryTransportServer` fully implements `ITransportServer`
|
||||
2. [x] `InMemoryTransportClient` fully implements `ITransportClient`
|
||||
3. [x] All frame types (HELLO, HEARTBEAT, REQUEST, RESPONSE, STREAM_DATA, CANCEL) are handled
|
||||
4. [x] Thread-safe concurrent access to `InMemoryConnectionRegistry`
|
||||
5. [x] All integration tests pass
|
||||
6. [x] No external dependencies (only BCL + Router.Common + DI/Options/Logging abstractions)
|
||||
|
||||
## Execution Log
|
||||
|
||||
| Date (UTC) | Update | Owner |
|
||||
|------------|--------|-------|
|
||||
| 2024-12-04 | Sprint completed: all InMemory transport components implemented and tested | Claude |
|
||||
|
||||
## Decisions & Risks
|
||||
|
||||
- Uses `System.Threading.Channels` for async frame passing (unbounded by default, can add backpressure later)
|
||||
- InMemory transport simulates latency only if explicitly configured (default: instant)
|
||||
- Connection lifetime is tied to `CancellationTokenSource`; disposing triggers cleanup
|
||||
- This transport is explicitly excluded from production deployments via conditional compilation or package separation
|
||||
@@ -0,0 +1,135 @@
|
||||
# Sprint 7000-0003-0001 · Microservice SDK · Core Infrastructure
|
||||
|
||||
## Topic & Scope
|
||||
|
||||
Implement the core infrastructure of the Microservice SDK: options, endpoint discovery, and router connection management. After this sprint, a microservice can connect to a router and send HELLO with its endpoint list.
|
||||
|
||||
**Goal:** "Connect and say HELLO" - microservice connects to router(s) and registers its identity and endpoints.
|
||||
|
||||
**Working directory:** `src/__Libraries/StellaOps.Microservice/`
|
||||
|
||||
**Parallel track:** This sprint can run in parallel with Gateway sprints (7000-0004-*) once the InMemory transport is complete.
|
||||
|
||||
## Dependencies & Concurrency
|
||||
|
||||
- **Upstream:** SPRINT_7000_0001_0002 (Common), SPRINT_7000_0002_0001 (InMemory transport)
|
||||
- **Downstream:** SPRINT_7000_0003_0002 (request handling)
|
||||
- **Parallel work:** Can run in parallel with Gateway core sprint
|
||||
- **Cross-module impact:** None. All work in `src/__Libraries/StellaOps.Microservice/`
|
||||
|
||||
## Documentation Prerequisites
|
||||
|
||||
- `docs/router/specs.md` (section 7 - Microservice SDK requirements)
|
||||
- `docs/router/04-Step.md` (detailed task breakdown)
|
||||
- `docs/router/implplan.md` (phase 4 guidance)
|
||||
|
||||
> **BLOCKED Tasks:** Before working on BLOCKED tasks, review [../implplan/BLOCKED_DEPENDENCY_TREE.md](../implplan/BLOCKED_DEPENDENCY_TREE.md) for root blockers and dependencies.
|
||||
|
||||
## Delivery Tracker
|
||||
|
||||
| # | Task ID | Status | Description | Notes |
|
||||
|---|---------|--------|-------------|-------|
|
||||
| 1 | SDK-001 | DONE | Implement `StellaMicroserviceOptions` with all required properties | ServiceName, Version, Region, InstanceId, Routers, ConfigFilePath |
|
||||
| 2 | SDK-002 | DONE | Implement `RouterEndpointConfig` (host, port, transport type) | |
|
||||
| 3 | SDK-003 | DONE | Validate that Routers list is mandatory (throw if empty) | Per spec |
|
||||
| 4 | SDK-010 | DONE | Create `[StellaEndpoint]` attribute for endpoint declaration | Method, Path, SupportsStreaming, Timeout |
|
||||
| 5 | SDK-011 | DONE | Implement runtime reflection endpoint discovery | Scan assemblies for `[StellaEndpoint]` |
|
||||
| 6 | SDK-012 | DONE | Build in-memory `EndpointDescriptor` list from discovered endpoints | |
|
||||
| 7 | SDK-013 | DONE | Create `IEndpointDiscoveryProvider` abstraction | For source-gen vs reflection swap |
|
||||
| 8 | SDK-020 | DONE | Implement `IRouterConnectionManager` interface | |
|
||||
| 9 | SDK-021 | DONE | Implement `RouterConnectionManager` with connection pool | One connection per router endpoint |
|
||||
| 10 | SDK-022 | DONE | Implement connection lifecycle (connect, reconnect on failure) | Exponential backoff |
|
||||
| 11 | SDK-023 | DONE | Implement HELLO frame construction from options + endpoints | |
|
||||
| 12 | SDK-024 | DONE | Send HELLO on connection establishment | Via InMemory transport |
|
||||
| 13 | SDK-025 | DONE | Implement HEARTBEAT sending on timer | Configurable interval |
|
||||
| 14 | SDK-030 | DONE | Implement `AddStellaMicroservice(IServiceCollection, Action<StellaMicroserviceOptions>)` | Full DI registration |
|
||||
| 15 | SDK-031 | DONE | Register `IHostedService` for connection management | Start/stop with host |
|
||||
| 16 | SDK-032 | DONE | Create `MicroserviceHostedService` that starts connections on app startup | |
|
||||
| 17 | SDK-040 | DONE | Write unit tests for endpoint discovery | |
|
||||
| 18 | SDK-041 | DONE | Write integration tests with InMemory transport | Connect, HELLO, HEARTBEAT |
|
||||
|
||||
## Endpoint Discovery
|
||||
|
||||
### Attribute-Based Declaration
|
||||
```csharp
|
||||
[StellaEndpoint("POST", "/billing/invoices")]
|
||||
public sealed class CreateInvoiceEndpoint : IStellaEndpoint<CreateInvoiceRequest, CreateInvoiceResponse>
|
||||
{
|
||||
public Task<CreateInvoiceResponse> HandleAsync(CreateInvoiceRequest request, CancellationToken ct);
|
||||
}
|
||||
```
|
||||
|
||||
### Discovery Flow
|
||||
1. On startup, scan loaded assemblies for types with `[StellaEndpoint]`
|
||||
2. For each type, verify it implements a handler interface
|
||||
3. Build `EndpointDescriptor` from attribute + defaults
|
||||
4. Store in `IEndpointRegistry` for lookup and HELLO construction
|
||||
|
||||
### Handler Interface Detection
|
||||
```csharp
|
||||
// Typed with request
|
||||
typeof(IStellaEndpoint<TRequest, TResponse>)
|
||||
|
||||
// Typed without request
|
||||
typeof(IStellaEndpoint<TResponse>)
|
||||
|
||||
// Raw handler
|
||||
typeof(IRawStellaEndpoint)
|
||||
```
|
||||
|
||||
## Connection Lifecycle
|
||||
|
||||
```
|
||||
┌─────────────┐ Connect ┌─────────────┐ HELLO ┌─────────────┐
|
||||
│ Disconnected│────────────────►│ Connected │───────────────►│ Registered │
|
||||
└─────────────┘ └─────────────┘ └─────────────┘
|
||||
▲ │ │
|
||||
│ │ Error │ Heartbeat timer
|
||||
│ ▼ ▼
|
||||
│ ┌─────────────┐ ┌─────────────┐
|
||||
└────────────────────────│ Reconnect │◄───────────────│ Heartbeat │
|
||||
Backoff │ (backoff) │ Error │ Active │
|
||||
└─────────────┘ └─────────────┘
|
||||
```
|
||||
|
||||
## StellaMicroserviceOptions
|
||||
|
||||
```csharp
|
||||
public sealed class StellaMicroserviceOptions
|
||||
{
|
||||
public string ServiceName { get; set; } = string.Empty;
|
||||
public string Version { get; set; } = string.Empty; // Strict semver
|
||||
public string Region { get; set; } = string.Empty;
|
||||
public string InstanceId { get; set; } = string.Empty; // Auto-generate if empty
|
||||
public IList<RouterEndpointConfig> Routers { get; set; } = new List<RouterEndpointConfig>();
|
||||
public string? ConfigFilePath { get; set; } // Optional YAML overrides
|
||||
public TimeSpan HeartbeatInterval { get; set; } = TimeSpan.FromSeconds(10);
|
||||
public TimeSpan ReconnectBackoffMax { get; set; } = TimeSpan.FromMinutes(1);
|
||||
}
|
||||
```
|
||||
|
||||
## Exit Criteria
|
||||
|
||||
Before marking this sprint DONE:
|
||||
1. [x] `StellaMicroserviceOptions` fully implemented with validation
|
||||
2. [x] Endpoint discovery works via reflection
|
||||
3. [x] Connection manager connects to configured routers
|
||||
4. [x] HELLO frame sent on connection with full endpoint list
|
||||
5. [x] HEARTBEAT sent periodically on timer
|
||||
6. [x] Reconnection with backoff on connection failure
|
||||
7. [x] Integration tests pass with InMemory transport
|
||||
8. [x] `AddStellaMicroservice()` registers all services correctly
|
||||
|
||||
## Execution Log
|
||||
|
||||
| Date (UTC) | Update | Owner |
|
||||
|------------|--------|-------|
|
||||
| 2024-12-04 | Sprint completed: SDK core infrastructure implemented | Claude |
|
||||
|
||||
## Decisions & Risks
|
||||
|
||||
- Endpoint discovery defaults to reflection; source generation comes in a later sprint
|
||||
- InstanceId auto-generates using `Guid.NewGuid().ToString("N")` if not provided
|
||||
- Version validation enforces strict semver format
|
||||
- Routers list cannot be empty - throws `InvalidOperationException` on startup
|
||||
- YAML config file is optional at this stage (Sprint 7000-0007-0002)
|
||||
@@ -0,0 +1,173 @@
|
||||
# Sprint 7000-0003-0002 · Microservice SDK · Request Handling
|
||||
|
||||
## Topic & Scope
|
||||
|
||||
Implement request handling in the Microservice SDK: receiving REQUEST frames, dispatching to handlers, and sending RESPONSE frames. Supports both typed and raw handler patterns.
|
||||
|
||||
**Goal:** Complete the request/response flow - microservice receives requests from router and returns responses.
|
||||
|
||||
**Working directory:** `src/__Libraries/StellaOps.Microservice/`
|
||||
|
||||
## Dependencies & Concurrency
|
||||
|
||||
- **Upstream:** SPRINT_7000_0003_0001 (SDK core with connection + HELLO)
|
||||
- **Downstream:** SPRINT_7000_0005_0003 (cancellation), SPRINT_7000_0005_0004 (streaming)
|
||||
- **Parallel work:** Can run in parallel with Gateway middleware sprint
|
||||
- **Cross-module impact:** None. All work in `src/__Libraries/StellaOps.Microservice/`
|
||||
|
||||
## Documentation Prerequisites
|
||||
|
||||
- `docs/router/specs.md` (section 7.2, 7.4, 7.5 - Endpoint definition, Connection behavior, Request handling)
|
||||
- `docs/router/04-Step.md` (detailed task breakdown - request handling section)
|
||||
|
||||
> **BLOCKED Tasks:** Before working on BLOCKED tasks, review [../implplan/BLOCKED_DEPENDENCY_TREE.md](../implplan/BLOCKED_DEPENDENCY_TREE.md) for root blockers and dependencies.
|
||||
|
||||
## Delivery Tracker
|
||||
|
||||
| # | Task ID | Status | Description | Notes |
|
||||
|---|---------|--------|-------------|-------|
|
||||
| 1 | HDL-001 | TODO | Define `IRawStellaEndpoint` interface | Takes RawRequestContext, returns RawResponse |
|
||||
| 2 | HDL-002 | TODO | Define `IStellaEndpoint<TRequest, TResponse>` interface | Typed request/response |
|
||||
| 3 | HDL-003 | TODO | Define `IStellaEndpoint<TResponse>` interface | No request body |
|
||||
| 4 | HDL-010 | TODO | Implement `RawRequestContext` | Method, Path, Headers, Body stream, CancellationToken |
|
||||
| 5 | HDL-011 | TODO | Implement `RawResponse` | StatusCode, Headers, Body stream |
|
||||
| 6 | HDL-012 | TODO | Implement `IHeaderCollection` abstraction | Key-value header access |
|
||||
| 7 | HDL-020 | TODO | Create `IEndpointRegistry` for handler lookup | (Method, Path) → handler instance |
|
||||
| 8 | HDL-021 | TODO | Implement path template matching (ASP.NET-style routes) | Handles `{id}` parameters |
|
||||
| 9 | HDL-022 | TODO | Implement path matching rules (case sensitivity, trailing slash) | Per spec |
|
||||
| 10 | HDL-030 | TODO | Create `TypedEndpointAdapter` to wrap typed handlers as raw | IStellaEndpoint<T,R> → IRawStellaEndpoint |
|
||||
| 11 | HDL-031 | TODO | Implement request deserialization in adapter | JSON by default |
|
||||
| 12 | HDL-032 | TODO | Implement response serialization in adapter | JSON by default |
|
||||
| 13 | HDL-040 | TODO | Implement `RequestDispatcher` | Frame → RawRequestContext → Handler → RawResponse → Frame |
|
||||
| 14 | HDL-041 | TODO | Implement frame-to-context conversion | REQUEST frame → RawRequestContext |
|
||||
| 15 | HDL-042 | TODO | Implement response-to-frame conversion | RawResponse → RESPONSE frame |
|
||||
| 16 | HDL-043 | TODO | Wire dispatcher into connection read loop | Process REQUEST frames |
|
||||
| 17 | HDL-050 | TODO | Implement `IServiceProvider` integration for handler instantiation | DI support |
|
||||
| 18 | HDL-051 | TODO | Implement handler scoping (per-request scope) | IServiceScope per request |
|
||||
| 19 | HDL-060 | TODO | Write unit tests for path matching | Various patterns |
|
||||
| 20 | HDL-061 | TODO | Write unit tests for typed adapter | Serialization round-trip |
|
||||
| 21 | HDL-062 | TODO | Write integration tests for full REQUEST/RESPONSE flow | With InMemory transport |
|
||||
|
||||
## Handler Interfaces
|
||||
|
||||
### Raw Handler
|
||||
```csharp
|
||||
public interface IRawStellaEndpoint
|
||||
{
|
||||
Task<RawResponse> HandleAsync(RawRequestContext context, CancellationToken cancellationToken);
|
||||
}
|
||||
```
|
||||
|
||||
### Typed Handlers
|
||||
```csharp
|
||||
public interface IStellaEndpoint<TRequest, TResponse>
|
||||
{
|
||||
Task<TResponse> HandleAsync(TRequest request, CancellationToken cancellationToken);
|
||||
}
|
||||
|
||||
public interface IStellaEndpoint<TResponse>
|
||||
{
|
||||
Task<TResponse> HandleAsync(CancellationToken cancellationToken);
|
||||
}
|
||||
```
|
||||
|
||||
## RawRequestContext
|
||||
|
||||
```csharp
|
||||
public sealed class RawRequestContext
|
||||
{
|
||||
public string Method { get; init; } = string.Empty;
|
||||
public string Path { get; init; } = string.Empty;
|
||||
public IReadOnlyDictionary<string, string> PathParameters { get; init; }
|
||||
= new Dictionary<string, string>();
|
||||
public IHeaderCollection Headers { get; init; } = default!;
|
||||
public Stream Body { get; init; } = Stream.Null;
|
||||
public CancellationToken CancellationToken { get; init; }
|
||||
}
|
||||
```
|
||||
|
||||
## RawResponse
|
||||
|
||||
```csharp
|
||||
public sealed class RawResponse
|
||||
{
|
||||
public int StatusCode { get; init; } = 200;
|
||||
public IHeaderCollection Headers { get; init; } = default!;
|
||||
public Stream Body { get; init; } = Stream.Null;
|
||||
|
||||
public static RawResponse Ok(Stream body) => new() { StatusCode = 200, Body = body };
|
||||
public static RawResponse NotFound() => new() { StatusCode = 404 };
|
||||
public static RawResponse Error(int statusCode, string message) => ...;
|
||||
}
|
||||
```
|
||||
|
||||
## Path Template Matching
|
||||
|
||||
Must use same rules as router (ASP.NET-style):
|
||||
- `{id}` matches any segment, value captured in PathParameters
|
||||
- `{id:int}` constraint support (optional for v1)
|
||||
- Case sensitivity: configurable, default case-insensitive
|
||||
- Trailing slash: configurable, default treats `/foo` and `/foo/` as equivalent
|
||||
|
||||
## Request Flow
|
||||
|
||||
```
|
||||
┌─────────────────┐ ┌────────────────────┐ ┌───────────────────┐
|
||||
│ REQUEST Frame │────►│ RequestDispatcher │────►│ IEndpointRegistry │
|
||||
│ (from Router) │ │ │ │ (Method, Path) │
|
||||
└─────────────────┘ └────────────────────┘ └───────────────────┘
|
||||
│ │
|
||||
│ ▼
|
||||
│ ┌───────────────────┐
|
||||
│ │ Handler Instance │
|
||||
│ │ (from DI scope) │
|
||||
│ └───────────────────┘
|
||||
│ │
|
||||
│◄─────────────────────────┘
|
||||
▼
|
||||
┌────────────────────┐
|
||||
│ RawRequestContext │
|
||||
└────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌────────────────────┐
|
||||
│ Handler.HandleAsync│
|
||||
└────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌────────────────────┐
|
||||
│ RawResponse │
|
||||
└────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌────────────────────┐
|
||||
│ RESPONSE Frame │
|
||||
│ (to Router) │
|
||||
└────────────────────┘
|
||||
```
|
||||
|
||||
## Exit Criteria
|
||||
|
||||
Before marking this sprint DONE:
|
||||
1. [ ] All handler interfaces defined and documented
|
||||
2. [ ] `RawRequestContext` and `RawResponse` implemented
|
||||
3. [ ] Path template matching works for common patterns
|
||||
4. [ ] Typed handlers wrapped correctly via `TypedEndpointAdapter`
|
||||
5. [ ] `RequestDispatcher` processes REQUEST frames end-to-end
|
||||
6. [ ] DI integration works (handlers resolved from service provider)
|
||||
7. [ ] Integration tests pass with InMemory transport
|
||||
8. [ ] Body treated as opaque bytes (no interpretation at SDK level for raw handlers)
|
||||
|
||||
## Execution Log
|
||||
|
||||
| Date (UTC) | Update | Owner |
|
||||
|------------|--------|-------|
|
||||
| | | |
|
||||
|
||||
## Decisions & Risks
|
||||
|
||||
- Typed handlers use JSON serialization by default; configurable via options
|
||||
- Path matching is case-insensitive by default (matches ASP.NET Core default)
|
||||
- Each request gets its own DI scope for handler resolution
|
||||
- Body stream may be buffered or streaming depending on endpoint configuration (streaming support comes in later sprint)
|
||||
- Handler exceptions are caught and converted to 500 responses with error details (configurable)
|
||||
135
docs/router/archived/SPRINT_7000_0004_0001_gateway_core.md
Normal file
135
docs/router/archived/SPRINT_7000_0004_0001_gateway_core.md
Normal file
@@ -0,0 +1,135 @@
|
||||
# Sprint 7000-0004-0001 · Gateway · Core Infrastructure
|
||||
|
||||
## Topic & Scope
|
||||
|
||||
Implement the core infrastructure of the Gateway: node configuration, global routing state, and basic routing plugin. This sprint creates the foundation for HTTP → transport → microservice routing.
|
||||
|
||||
**Goal:** Gateway can maintain routing state from connected microservices and select instances for routing decisions.
|
||||
|
||||
**Working directory:** `src/Gateway/StellaOps.Gateway.WebService/`
|
||||
|
||||
**Parallel track:** This sprint can run in parallel with Microservice SDK sprints (7000-0003-*) once the InMemory transport is complete.
|
||||
|
||||
## Dependencies & Concurrency
|
||||
|
||||
- **Upstream:** SPRINT_7000_0001_0002 (Common), SPRINT_7000_0002_0001 (InMemory transport)
|
||||
- **Downstream:** SPRINT_7000_0004_0002 (middleware), SPRINT_7000_0004_0003 (connection handling)
|
||||
- **Parallel work:** Can run in parallel with SDK core sprint
|
||||
- **Cross-module impact:** None. All work in `src/Gateway/StellaOps.Gateway.WebService/`
|
||||
|
||||
## Documentation Prerequisites
|
||||
|
||||
- `docs/router/specs.md` (section 6 - Gateway requirements)
|
||||
- `docs/router/05-Step.md` (detailed task breakdown)
|
||||
- `docs/router/implplan.md` (phase 5 guidance)
|
||||
|
||||
> **BLOCKED Tasks:** Before working on BLOCKED tasks, review [../implplan/BLOCKED_DEPENDENCY_TREE.md](../implplan/BLOCKED_DEPENDENCY_TREE.md) for root blockers and dependencies.
|
||||
|
||||
## Delivery Tracker
|
||||
|
||||
| # | Task ID | Status | Description | Notes |
|
||||
|---|---------|--------|-------------|-------|
|
||||
| 1 | GW-001 | TODO | Implement `GatewayNodeConfig` | Region, NodeId, Environment |
|
||||
| 2 | GW-002 | TODO | Bind `GatewayNodeConfig` from configuration | appsettings.json section |
|
||||
| 3 | GW-003 | TODO | Validate GatewayNodeConfig on startup | Region required |
|
||||
| 4 | GW-010 | TODO | Implement `IGlobalRoutingState` as `InMemoryRoutingState` | Thread-safe implementation |
|
||||
| 5 | GW-011 | TODO | Implement `ConnectionState` storage | ConcurrentDictionary by ConnectionId |
|
||||
| 6 | GW-012 | TODO | Implement endpoint-to-connections index | (Method, Path) → List<ConnectionState> |
|
||||
| 7 | GW-013 | TODO | Implement `ResolveEndpoint(method, path)` | Path template matching |
|
||||
| 8 | GW-014 | TODO | Implement `GetConnectionsFor(serviceName, version, method, path)` | Filter by criteria |
|
||||
| 9 | GW-020 | TODO | Create `IRoutingPlugin` implementation `DefaultRoutingPlugin` | Basic instance selection |
|
||||
| 10 | GW-021 | TODO | Implement version filtering (strict semver equality) | Per spec |
|
||||
| 11 | GW-022 | TODO | Implement health filtering (Healthy or Degraded only) | Per spec |
|
||||
| 12 | GW-023 | TODO | Implement region preference (gateway region first) | Use GatewayNodeConfig.Region |
|
||||
| 13 | GW-024 | TODO | Implement basic tie-breaking (any healthy instance) | Full algorithm in later sprint |
|
||||
| 14 | GW-030 | TODO | Create `RoutingOptions` for configurable behavior | Default version, neighbor regions |
|
||||
| 15 | GW-031 | TODO | Register routing services in DI | IGlobalRoutingState, IRoutingPlugin |
|
||||
| 16 | GW-040 | TODO | Write unit tests for InMemoryRoutingState | |
|
||||
| 17 | GW-041 | TODO | Write unit tests for DefaultRoutingPlugin | Version, health, region filtering |
|
||||
|
||||
## GatewayNodeConfig
|
||||
|
||||
```csharp
|
||||
public sealed class GatewayNodeConfig
|
||||
{
|
||||
public string Region { get; set; } = string.Empty; // Required, e.g. "eu1"
|
||||
public string NodeId { get; set; } = string.Empty; // e.g. "gw-eu1-01"
|
||||
public string Environment { get; set; } = string.Empty; // e.g. "prod"
|
||||
public IList<string> NeighborRegions { get; set; } = []; // Fallback regions
|
||||
}
|
||||
```
|
||||
|
||||
**Configuration binding:**
|
||||
```json
|
||||
{
|
||||
"GatewayNode": {
|
||||
"Region": "eu1",
|
||||
"NodeId": "gw-eu1-01",
|
||||
"Environment": "prod",
|
||||
"NeighborRegions": ["eu2", "us1"]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## InMemoryRoutingState
|
||||
|
||||
```csharp
|
||||
internal sealed class InMemoryRoutingState : IGlobalRoutingState
|
||||
{
|
||||
private readonly ConcurrentDictionary<string, ConnectionState> _connections = new();
|
||||
private readonly ConcurrentDictionary<(string Method, string Path), List<string>> _endpointIndex = new();
|
||||
|
||||
public void AddConnection(ConnectionState connection) { ... }
|
||||
public void RemoveConnection(string connectionId) { ... }
|
||||
public void UpdateConnection(string connectionId, Action<ConnectionState> update) { ... }
|
||||
|
||||
public EndpointDescriptor? ResolveEndpoint(string method, string path) { ... }
|
||||
public IReadOnlyList<ConnectionState> GetConnectionsFor(
|
||||
string serviceName, string version, string method, string path) { ... }
|
||||
}
|
||||
```
|
||||
|
||||
## Routing Algorithm (Phase 1 - Basic)
|
||||
|
||||
```
|
||||
1. Filter by ServiceName (exact match)
|
||||
2. Filter by Version (strict semver equality)
|
||||
3. Filter by Health (Healthy or Degraded only)
|
||||
4. If any remain, pick one (random for now)
|
||||
5. If none, return null (503 Service Unavailable)
|
||||
```
|
||||
|
||||
**Note:** Full routing algorithm (region preference, ping-based selection, fallback) is implemented in SPRINT_7000_0005_0002.
|
||||
|
||||
## Region Derivation
|
||||
|
||||
Per spec section 2:
|
||||
> Routing decisions MUST use `GatewayNodeConfig.Region` as the node's region; the router MUST NOT derive region from HTTP headers or URL host names.
|
||||
|
||||
This is enforced by:
|
||||
1. GatewayNodeConfig is bound from static configuration only
|
||||
2. No code path reads region from HttpContext
|
||||
3. Tests verify region is never extracted from Host header
|
||||
|
||||
## Exit Criteria
|
||||
|
||||
Before marking this sprint DONE:
|
||||
1. [ ] `GatewayNodeConfig` loads and validates from configuration
|
||||
2. [ ] `InMemoryRoutingState` stores and indexes connections correctly
|
||||
3. [ ] `ResolveEndpoint` performs path template matching
|
||||
4. [ ] `DefaultRoutingPlugin` filters by version, health, region
|
||||
5. [ ] All services registered in DI container
|
||||
6. [ ] Unit tests pass for routing state and plugin
|
||||
|
||||
## Execution Log
|
||||
|
||||
| Date (UTC) | Update | Owner |
|
||||
|------------|--------|-------|
|
||||
| | | |
|
||||
|
||||
## Decisions & Risks
|
||||
|
||||
- Routing state is in-memory only; no persistence or distribution (single gateway node for v1)
|
||||
- Path template matching reuses logic from SDK (shared in Common or duplicated)
|
||||
- DefaultRoutingPlugin is intentionally simple; full algorithm comes in SPRINT_7000_0005_0002
|
||||
- Region validation: startup fails fast if Region is empty
|
||||
172
docs/router/archived/SPRINT_7000_0004_0002_gateway_middleware.md
Normal file
172
docs/router/archived/SPRINT_7000_0004_0002_gateway_middleware.md
Normal file
@@ -0,0 +1,172 @@
|
||||
# Sprint 7000-0004-0002 · Gateway · HTTP Middleware Pipeline
|
||||
|
||||
## Topic & Scope
|
||||
|
||||
Implement the HTTP middleware pipeline for the Gateway: endpoint resolution, authorization, routing decision, and transport dispatch. After this sprint, HTTP requests flow through the gateway to microservices via the InMemory transport.
|
||||
|
||||
**Goal:** Complete HTTP → transport → microservice → HTTP flow for basic buffered requests.
|
||||
|
||||
**Working directory:** `src/Gateway/StellaOps.Gateway.WebService/`
|
||||
|
||||
## Dependencies & Concurrency
|
||||
|
||||
- **Upstream:** SPRINT_7000_0004_0001 (Gateway core)
|
||||
- **Downstream:** SPRINT_7000_0004_0003 (connection handling)
|
||||
- **Parallel work:** Can run in parallel with SDK request handling sprint
|
||||
- **Cross-module impact:** None. All work in `src/Gateway/StellaOps.Gateway.WebService/`
|
||||
|
||||
## Documentation Prerequisites
|
||||
|
||||
- `docs/router/specs.md` (section 6.1 - HTTP ingress pipeline)
|
||||
- `docs/router/05-Step.md` (middleware section)
|
||||
|
||||
> **BLOCKED Tasks:** Before working on BLOCKED tasks, review [../implplan/BLOCKED_DEPENDENCY_TREE.md](../implplan/BLOCKED_DEPENDENCY_TREE.md) for root blockers and dependencies.
|
||||
|
||||
## Delivery Tracker
|
||||
|
||||
| # | Task ID | Status | Description | Notes |
|
||||
|---|---------|--------|-------------|-------|
|
||||
| 1 | MID-001 | TODO | Create `EndpointResolutionMiddleware` | (Method, Path) → EndpointDescriptor |
|
||||
| 2 | MID-002 | TODO | Store resolved endpoint in `HttpContext.Items` | For downstream middleware |
|
||||
| 3 | MID-003 | TODO | Return 404 if endpoint not found | |
|
||||
| 4 | MID-010 | TODO | Create `AuthorizationMiddleware` stub | Checks authenticated only (full claims later) |
|
||||
| 5 | MID-011 | TODO | Wire ASP.NET Core authentication | Standard middleware order |
|
||||
| 6 | MID-012 | TODO | Return 401/403 for unauthorized requests | |
|
||||
| 7 | MID-020 | TODO | Create `RoutingDecisionMiddleware` | Calls IRoutingPlugin.ChooseInstanceAsync |
|
||||
| 8 | MID-021 | TODO | Store RoutingDecision in `HttpContext.Items` | |
|
||||
| 9 | MID-022 | TODO | Return 503 if no instance available | |
|
||||
| 10 | MID-023 | TODO | Return 504 if routing times out | |
|
||||
| 11 | MID-030 | TODO | Create `TransportDispatchMiddleware` | Dispatches to selected transport |
|
||||
| 12 | MID-031 | TODO | Implement buffered request dispatch | Read entire body, send REQUEST frame |
|
||||
| 13 | MID-032 | TODO | Implement buffered response handling | Read RESPONSE frame, write to HTTP |
|
||||
| 14 | MID-033 | TODO | Map transport errors to HTTP status codes | |
|
||||
| 15 | MID-040 | TODO | Create `GlobalErrorHandlerMiddleware` | Catches unhandled exceptions |
|
||||
| 16 | MID-041 | TODO | Implement structured error responses | JSON error envelope |
|
||||
| 17 | MID-050 | TODO | Create `RequestLoggingMiddleware` | Correlation ID, service, endpoint, region, instance |
|
||||
| 18 | MID-051 | TODO | Wire forwarded headers middleware | For reverse proxy support |
|
||||
| 19 | MID-060 | TODO | Configure middleware pipeline in Program.cs | Correct order |
|
||||
| 20 | MID-070 | TODO | Write integration tests for full HTTP→transport flow | With InMemory transport + SDK |
|
||||
| 21 | MID-071 | TODO | Write tests for error scenarios (404, 503, etc.) | |
|
||||
|
||||
## Middleware Pipeline Order
|
||||
|
||||
```csharp
|
||||
app.UseForwardedHeaders(); // Reverse proxy support
|
||||
app.UseMiddleware<GlobalErrorHandlerMiddleware>();
|
||||
app.UseMiddleware<RequestLoggingMiddleware>();
|
||||
app.UseAuthentication(); // ASP.NET Core auth
|
||||
app.UseMiddleware<EndpointResolutionMiddleware>();
|
||||
app.UseMiddleware<AuthorizationMiddleware>();
|
||||
app.UseMiddleware<RoutingDecisionMiddleware>();
|
||||
app.UseMiddleware<TransportDispatchMiddleware>();
|
||||
```
|
||||
|
||||
## EndpointResolutionMiddleware
|
||||
|
||||
```csharp
|
||||
public class EndpointResolutionMiddleware
|
||||
{
|
||||
public async Task InvokeAsync(HttpContext context, IGlobalRoutingState routingState)
|
||||
{
|
||||
var method = context.Request.Method;
|
||||
var path = context.Request.Path.Value ?? "/";
|
||||
|
||||
var endpoint = routingState.ResolveEndpoint(method, path);
|
||||
if (endpoint == null)
|
||||
{
|
||||
context.Response.StatusCode = 404;
|
||||
await context.Response.WriteAsJsonAsync(new { error = "Endpoint not found" });
|
||||
return;
|
||||
}
|
||||
|
||||
context.Items["ResolvedEndpoint"] = endpoint;
|
||||
await _next(context);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## TransportDispatchMiddleware (Buffered Mode)
|
||||
|
||||
```csharp
|
||||
public class TransportDispatchMiddleware
|
||||
{
|
||||
public async Task InvokeAsync(HttpContext context, ITransportClient transport)
|
||||
{
|
||||
var decision = (RoutingDecision)context.Items["RoutingDecision"]!;
|
||||
var endpoint = (EndpointDescriptor)context.Items["ResolvedEndpoint"]!;
|
||||
|
||||
// Build REQUEST frame
|
||||
using var bodyStream = new MemoryStream();
|
||||
await context.Request.Body.CopyToAsync(bodyStream);
|
||||
var requestFrame = new Frame
|
||||
{
|
||||
Type = FrameType.Request,
|
||||
CorrelationId = Guid.NewGuid(),
|
||||
Payload = BuildRequestPayload(context, bodyStream.ToArray())
|
||||
};
|
||||
|
||||
// Send and await response
|
||||
using var cts = CancellationTokenSource.CreateLinkedTokenSource(
|
||||
context.RequestAborted);
|
||||
cts.CancelAfter(decision.EffectiveTimeout);
|
||||
|
||||
var responseFrame = await transport.SendRequestAsync(
|
||||
decision.Connection,
|
||||
requestFrame,
|
||||
decision.EffectiveTimeout,
|
||||
cts.Token);
|
||||
|
||||
// Write response to HTTP
|
||||
await WriteHttpResponse(context, responseFrame);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Error Mapping
|
||||
|
||||
| Transport/Routing Error | HTTP Status |
|
||||
|------------------------|-------------|
|
||||
| Endpoint not found | 404 Not Found |
|
||||
| No healthy instance | 503 Service Unavailable |
|
||||
| Timeout | 504 Gateway Timeout |
|
||||
| Microservice error (5xx) | Pass through status |
|
||||
| Transport connection lost | 502 Bad Gateway |
|
||||
| Payload too large | 413 Payload Too Large |
|
||||
| Unauthorized | 401 Unauthorized |
|
||||
| Forbidden (claims) | 403 Forbidden |
|
||||
|
||||
## HttpContext.Items Keys
|
||||
|
||||
```csharp
|
||||
public static class ContextKeys
|
||||
{
|
||||
public const string ResolvedEndpoint = "ResolvedEndpoint";
|
||||
public const string RoutingDecision = "RoutingDecision";
|
||||
public const string CorrelationId = "CorrelationId";
|
||||
}
|
||||
```
|
||||
|
||||
## Exit Criteria
|
||||
|
||||
Before marking this sprint DONE:
|
||||
1. [ ] All middleware classes implemented
|
||||
2. [ ] Pipeline configured in correct order
|
||||
3. [ ] EndpointResolutionMiddleware resolves (Method, Path) → endpoint
|
||||
4. [ ] AuthorizationMiddleware checks authentication (claims in later sprint)
|
||||
5. [ ] RoutingDecisionMiddleware selects instance via IRoutingPlugin
|
||||
6. [ ] TransportDispatchMiddleware sends/receives frames (buffered mode)
|
||||
7. [ ] Error responses use consistent JSON envelope
|
||||
8. [ ] Integration tests pass with InMemory transport
|
||||
|
||||
## Execution Log
|
||||
|
||||
| Date (UTC) | Update | Owner |
|
||||
|------------|--------|-------|
|
||||
| | | |
|
||||
|
||||
## Decisions & Risks
|
||||
|
||||
- Authorization middleware is a stub that only checks `User.Identity?.IsAuthenticated`; full RequiringClaims enforcement comes in SPRINT_7000_0008_0001
|
||||
- Streaming support is not implemented in this sprint; TransportDispatchMiddleware only handles buffered mode
|
||||
- Correlation ID is generated per request and logged throughout
|
||||
- Request body is fully read into memory for buffered mode; streaming in SPRINT_7000_0005_0004
|
||||
@@ -0,0 +1,218 @@
|
||||
# Sprint 7000-0004-0003 · Gateway · Connection Handling
|
||||
|
||||
## Topic & Scope
|
||||
|
||||
Implement connection handling in the Gateway: processing HELLO frames from microservices, maintaining connection state, and updating the global routing state. After this sprint, microservices can register with the gateway and be routed to.
|
||||
|
||||
**Goal:** Gateway receives HELLO from microservices and maintains live routing state. Combined with previous sprints, this enables full end-to-end HTTP → microservice routing.
|
||||
|
||||
**Working directory:** `src/Gateway/StellaOps.Gateway.WebService/`
|
||||
|
||||
## Dependencies & Concurrency
|
||||
|
||||
- **Upstream:** SPRINT_7000_0004_0002 (middleware), SPRINT_7000_0003_0001 (SDK core with HELLO)
|
||||
- **Downstream:** SPRINT_7000_0005_0001 (heartbeat/health)
|
||||
- **Parallel work:** Should coordinate with SDK team for HELLO frame format agreement
|
||||
- **Cross-module impact:** None. All work in Gateway.
|
||||
|
||||
## Documentation Prerequisites
|
||||
|
||||
- `docs/router/specs.md` (section 6.2 - Per-connection state and routing view)
|
||||
- `docs/router/05-Step.md` (connection handling section)
|
||||
|
||||
> **BLOCKED Tasks:** Before working on BLOCKED tasks, review [../implplan/BLOCKED_DEPENDENCY_TREE.md](../implplan/BLOCKED_DEPENDENCY_TREE.md) for root blockers and dependencies.
|
||||
|
||||
## Delivery Tracker
|
||||
|
||||
| # | Task ID | Status | Description | Notes |
|
||||
|---|---------|--------|-------------|-------|
|
||||
| 1 | CON-001 | TODO | Create `IConnectionHandler` interface | Processes frames per connection |
|
||||
| 2 | CON-002 | TODO | Implement `ConnectionHandler` | Frame type dispatch |
|
||||
| 3 | CON-010 | TODO | Implement HELLO frame processing | Parse HelloPayload, create ConnectionState |
|
||||
| 4 | CON-011 | TODO | Validate HELLO payload | ServiceName, Version, InstanceId required |
|
||||
| 5 | CON-012 | TODO | Register connection in IGlobalRoutingState | AddConnection |
|
||||
| 6 | CON-013 | TODO | Build endpoint index from HELLO | (Method, Path) → ConnectionId |
|
||||
| 7 | CON-020 | TODO | Create `TransportServerHost` hosted service | Starts ITransportServer |
|
||||
| 8 | CON-021 | TODO | Wire transport server to connection handler | Frame routing |
|
||||
| 9 | CON-022 | TODO | Handle new connections (InMemory: channel registration) | |
|
||||
| 10 | CON-030 | TODO | Implement connection cleanup on disconnect | RemoveConnection from routing state |
|
||||
| 11 | CON-031 | TODO | Clean up endpoint index on disconnect | Remove all endpoints for connection |
|
||||
| 12 | CON-032 | TODO | Log connection lifecycle events | Connect, HELLO, disconnect |
|
||||
| 13 | CON-040 | TODO | Implement connection ID generation | Unique per connection |
|
||||
| 14 | CON-041 | TODO | Store connection metadata | Transport type, connect time |
|
||||
| 15 | CON-050 | TODO | Write integration tests for HELLO flow | SDK → Gateway registration |
|
||||
| 16 | CON-051 | TODO | Write tests for connection cleanup | |
|
||||
| 17 | CON-052 | TODO | Write tests for multiple connections from same service | Different instances |
|
||||
|
||||
## Connection Lifecycle
|
||||
|
||||
```
|
||||
┌─────────────────┐
|
||||
│ New Connection │ (Transport layer signals new connection)
|
||||
└────────┬────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────┐
|
||||
│ Awaiting HELLO │ (Connection exists but not registered for routing)
|
||||
└────────┬────────┘
|
||||
│ HELLO frame received
|
||||
▼
|
||||
┌─────────────────┐
|
||||
│ Validate HELLO │ (Check ServiceName, Version, endpoints)
|
||||
└────────┬────────┘
|
||||
│ Valid
|
||||
▼
|
||||
┌─────────────────┐
|
||||
│ Create │
|
||||
│ ConnectionState │ (InstanceDescriptor, endpoints, health = Unknown)
|
||||
└────────┬────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────┐
|
||||
│ Register in │ (Add to IGlobalRoutingState, index endpoints)
|
||||
│ RoutingState │
|
||||
└────────┬────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────┐
|
||||
│ Registered │ (Connection can receive routed requests)
|
||||
└────────┬────────┘
|
||||
│ Disconnect or error
|
||||
▼
|
||||
┌─────────────────┐
|
||||
│ Cleanup State │ (Remove from routing state, clean endpoint index)
|
||||
└─────────────────┘
|
||||
```
|
||||
|
||||
## HELLO Processing
|
||||
|
||||
```csharp
|
||||
internal sealed class ConnectionHandler : IConnectionHandler
|
||||
{
|
||||
public async Task HandleFrameAsync(string connectionId, Frame frame)
|
||||
{
|
||||
switch (frame.Type)
|
||||
{
|
||||
case FrameType.Hello:
|
||||
await ProcessHelloAsync(connectionId, frame);
|
||||
break;
|
||||
case FrameType.Heartbeat:
|
||||
await ProcessHeartbeatAsync(connectionId, frame);
|
||||
break;
|
||||
case FrameType.Response:
|
||||
case FrameType.ResponseStreamData:
|
||||
await ProcessResponseAsync(connectionId, frame);
|
||||
break;
|
||||
default:
|
||||
_logger.LogWarning("Unknown frame type {Type} from {ConnectionId}",
|
||||
frame.Type, connectionId);
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
private async Task ProcessHelloAsync(string connectionId, Frame frame)
|
||||
{
|
||||
var payload = DeserializeHelloPayload(frame.Payload);
|
||||
|
||||
// Validate
|
||||
if (string.IsNullOrEmpty(payload.Instance.ServiceName))
|
||||
throw new InvalidHelloException("ServiceName required");
|
||||
if (string.IsNullOrEmpty(payload.Instance.Version))
|
||||
throw new InvalidHelloException("Version required");
|
||||
|
||||
// Build ConnectionState
|
||||
var connection = new ConnectionState
|
||||
{
|
||||
ConnectionId = connectionId,
|
||||
Instance = payload.Instance,
|
||||
Status = InstanceHealthStatus.Unknown,
|
||||
LastHeartbeatUtc = DateTime.UtcNow,
|
||||
TransportType = _currentTransportType,
|
||||
Endpoints = payload.Endpoints.ToDictionary(
|
||||
e => (e.Method, e.Path),
|
||||
e => e)
|
||||
};
|
||||
|
||||
// Register
|
||||
_routingState.AddConnection(connection);
|
||||
_logger.LogInformation(
|
||||
"Registered {ServiceName} v{Version} instance {InstanceId} from {Region}",
|
||||
payload.Instance.ServiceName,
|
||||
payload.Instance.Version,
|
||||
payload.Instance.InstanceId,
|
||||
payload.Instance.Region);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## TransportServerHost
|
||||
|
||||
```csharp
|
||||
internal sealed class TransportServerHost : IHostedService
|
||||
{
|
||||
private readonly ITransportServer _server;
|
||||
private readonly IConnectionHandler _handler;
|
||||
|
||||
public async Task StartAsync(CancellationToken cancellationToken)
|
||||
{
|
||||
_server.OnConnection += HandleNewConnection;
|
||||
_server.OnFrame += HandleFrame;
|
||||
_server.OnDisconnect += HandleDisconnect;
|
||||
|
||||
await _server.StartAsync(cancellationToken);
|
||||
}
|
||||
|
||||
private void HandleNewConnection(string connectionId)
|
||||
{
|
||||
_logger.LogInformation("New connection: {ConnectionId}", connectionId);
|
||||
}
|
||||
|
||||
private async Task HandleFrame(string connectionId, Frame frame)
|
||||
{
|
||||
await _handler.HandleFrameAsync(connectionId, frame);
|
||||
}
|
||||
|
||||
private void HandleDisconnect(string connectionId)
|
||||
{
|
||||
_routingState.RemoveConnection(connectionId);
|
||||
_logger.LogInformation("Connection closed: {ConnectionId}", connectionId);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Multiple Instances
|
||||
|
||||
The gateway must handle multiple instances of the same service:
|
||||
- Same ServiceName + Version from different InstanceIds
|
||||
- Each instance has its own ConnectionState
|
||||
- Routing algorithm selects among available instances
|
||||
|
||||
```
|
||||
Service: billing v1.0.0
|
||||
├── Instance: billing-01 (Region: eu1) → Connection abc123
|
||||
├── Instance: billing-02 (Region: eu1) → Connection def456
|
||||
└── Instance: billing-03 (Region: us1) → Connection ghi789
|
||||
```
|
||||
|
||||
## Exit Criteria
|
||||
|
||||
Before marking this sprint DONE:
|
||||
1. [ ] HELLO frames processed correctly
|
||||
2. [ ] ConnectionState created and stored
|
||||
3. [ ] Endpoint index updated for routing lookups
|
||||
4. [ ] Connection cleanup removes all state
|
||||
5. [ ] TransportServerHost starts/stops with application
|
||||
6. [ ] Integration tests: SDK registers, Gateway routes, SDK handles request
|
||||
|
||||
## Execution Log
|
||||
|
||||
| Date (UTC) | Update | Owner |
|
||||
|------------|--------|-------|
|
||||
| | | |
|
||||
|
||||
## Decisions & Risks
|
||||
|
||||
- Initial health status is `Unknown` until first heartbeat
|
||||
- Connection ID format: GUID for InMemory, transport-specific for real transports
|
||||
- HELLO validation failure disconnects the client (logs error)
|
||||
- Duplicate HELLO from same connection replaces existing state (re-registration)
|
||||
205
docs/router/archived/SPRINT_7000_0005_0001_heartbeat_health.md
Normal file
205
docs/router/archived/SPRINT_7000_0005_0001_heartbeat_health.md
Normal file
@@ -0,0 +1,205 @@
|
||||
# Sprint 7000-0005-0001 · Protocol Features · Heartbeat & Health
|
||||
|
||||
## Topic & Scope
|
||||
|
||||
Implement heartbeat processing and health tracking. Microservices send HEARTBEAT frames periodically; the gateway updates health status and marks stale instances as unhealthy.
|
||||
|
||||
**Goal:** Gateway maintains accurate health status for all connected instances, enabling health-aware routing.
|
||||
|
||||
**Working directories:**
|
||||
- `src/__Libraries/StellaOps.Microservice/` (heartbeat sending)
|
||||
- `src/Gateway/StellaOps.Gateway.WebService/` (heartbeat processing)
|
||||
- `src/__Libraries/StellaOps.Router.Common/` (if payload changes needed)
|
||||
|
||||
## Dependencies & Concurrency
|
||||
|
||||
- **Upstream:** SPRINT_7000_0004_0003 (Gateway connection handling), SPRINT_7000_0003_0001 (SDK core)
|
||||
- **Downstream:** SPRINT_7000_0005_0002 (routing algorithm uses health)
|
||||
- **Parallel work:** None. Sequential after connection handling.
|
||||
- **Cross-module impact:** SDK and Gateway both modified.
|
||||
|
||||
## Documentation Prerequisites
|
||||
|
||||
- `docs/router/specs.md` (section 8 - Control/health/ping requirements)
|
||||
- `docs/router/06-Step.md` (heartbeat section)
|
||||
- `docs/router/implplan.md` (phase 6 guidance)
|
||||
|
||||
> **BLOCKED Tasks:** Before working on BLOCKED tasks, review [../implplan/BLOCKED_DEPENDENCY_TREE.md](../implplan/BLOCKED_DEPENDENCY_TREE.md) for root blockers and dependencies.
|
||||
|
||||
## Delivery Tracker
|
||||
|
||||
| # | Task ID | Status | Description | Working Directory |
|
||||
|---|---------|--------|-------------|-------------------|
|
||||
| 1 | HB-001 | DONE | Implement HeartbeatPayload serialization | Common |
|
||||
| 2 | HB-002 | DONE | Add InstanceHealthStatus to HeartbeatPayload | Common |
|
||||
| 3 | HB-003 | DONE | Add optional metrics to HeartbeatPayload (inflight count, error rate) | Common |
|
||||
| 4 | HB-010 | DONE | Implement heartbeat sending timer in SDK | Microservice |
|
||||
| 5 | HB-011 | DONE | Report current health status in heartbeat | Microservice |
|
||||
| 6 | HB-012 | DONE | Report optional metrics in heartbeat | Microservice |
|
||||
| 7 | HB-013 | DONE | Make heartbeat interval configurable | Microservice |
|
||||
| 8 | HB-020 | DONE | Implement HEARTBEAT frame processing in Gateway | Gateway |
|
||||
| 9 | HB-021 | DONE | Update LastHeartbeatUtc on heartbeat | Gateway |
|
||||
| 10 | HB-022 | DONE | Update InstanceHealthStatus from payload | Gateway |
|
||||
| 11 | HB-023 | DONE | Update optional metrics from payload | Gateway |
|
||||
| 12 | HB-030 | DONE | Create HealthMonitorService hosted service | Gateway |
|
||||
| 13 | HB-031 | DONE | Implement stale heartbeat detection | Configurable threshold |
|
||||
| 14 | HB-032 | DONE | Mark instances Unhealthy when heartbeat stale | Gateway |
|
||||
| 15 | HB-033 | DONE | Implement Draining status support | For graceful shutdown |
|
||||
| 16 | HB-040 | DONE | Create HealthOptions for thresholds | StaleThreshold, DegradedThreshold |
|
||||
| 17 | HB-041 | DONE | Bind HealthOptions from configuration | Gateway |
|
||||
| 18 | HB-050 | DONE | Implement ping latency measurement (request/response timing) | Gateway |
|
||||
| 19 | HB-051 | DONE | Update AveragePingMs from timing | Exponential moving average |
|
||||
| 20 | HB-060 | DONE | Write integration tests for heartbeat flow | |
|
||||
| 21 | HB-061 | DONE | Write tests for health status transitions | |
|
||||
| 22 | HB-062 | DONE | Write tests for stale detection | |
|
||||
|
||||
## HeartbeatPayload
|
||||
|
||||
```csharp
|
||||
public sealed class HeartbeatPayload
|
||||
{
|
||||
public string InstanceId { get; init; } = string.Empty;
|
||||
public InstanceHealthStatus Status { get; init; }
|
||||
public int? InflightRequestCount { get; init; }
|
||||
public double? ErrorRatePercent { get; init; }
|
||||
public DateTimeOffset Timestamp { get; init; }
|
||||
}
|
||||
```
|
||||
|
||||
## Health Status Transitions
|
||||
|
||||
```
|
||||
┌─────────┐
|
||||
First │ Unknown │
|
||||
Heartbeat └────┬────┘
|
||||
│ Status from payload
|
||||
▼
|
||||
┌─────────┐
|
||||
◄────────────────│ Healthy │◄───────────────┐
|
||||
│ Degraded └────┬────┘ Healthy │
|
||||
│ in payload │ │
|
||||
▼ │ Stale threshold │
|
||||
┌──────────┐ │ exceeded │
|
||||
│ Degraded │ ▼ │
|
||||
└────┬─────┘ ┌───────────┐ │
|
||||
│ │ Unhealthy │───────────────┘
|
||||
│ Stale └───────────┘ Heartbeat
|
||||
│ threshold received
|
||||
▼
|
||||
┌───────────┐
|
||||
│ Unhealthy │
|
||||
└───────────┘
|
||||
```
|
||||
|
||||
**Special case: Draining**
|
||||
- Microservice explicitly sets status to `Draining`
|
||||
- Router stops sending new requests but allows in-flight to complete
|
||||
- Used for graceful shutdown
|
||||
|
||||
## HealthMonitorService
|
||||
|
||||
```csharp
|
||||
internal sealed class HealthMonitorService : BackgroundService
|
||||
{
|
||||
private readonly IGlobalRoutingState _routingState;
|
||||
private readonly IOptions<HealthOptions> _options;
|
||||
|
||||
protected override async Task ExecuteAsync(CancellationToken stoppingToken)
|
||||
{
|
||||
var interval = TimeSpan.FromSeconds(5); // Check frequency
|
||||
|
||||
while (!stoppingToken.IsCancellationRequested)
|
||||
{
|
||||
CheckStaleConnections();
|
||||
await Task.Delay(interval, stoppingToken);
|
||||
}
|
||||
}
|
||||
|
||||
private void CheckStaleConnections()
|
||||
{
|
||||
var threshold = _options.Value.StaleThreshold;
|
||||
var now = DateTime.UtcNow;
|
||||
|
||||
foreach (var connection in _routingState.GetAllConnections())
|
||||
{
|
||||
var age = now - connection.LastHeartbeatUtc;
|
||||
if (age > threshold && connection.Status != InstanceHealthStatus.Unhealthy)
|
||||
{
|
||||
_routingState.UpdateConnection(connection.ConnectionId,
|
||||
c => c.Status = InstanceHealthStatus.Unhealthy);
|
||||
_logger.LogWarning(
|
||||
"Instance {InstanceId} marked Unhealthy: no heartbeat for {Age}",
|
||||
connection.Instance.InstanceId, age);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## HealthOptions
|
||||
|
||||
```csharp
|
||||
public sealed class HealthOptions
|
||||
{
|
||||
public TimeSpan StaleThreshold { get; set; } = TimeSpan.FromSeconds(30);
|
||||
public TimeSpan DegradedThreshold { get; set; } = TimeSpan.FromSeconds(15);
|
||||
public int PingHistorySize { get; set; } = 10; // For moving average
|
||||
}
|
||||
```
|
||||
|
||||
## Ping Latency Measurement
|
||||
|
||||
Measure round-trip time for REQUEST/RESPONSE:
|
||||
1. Record timestamp when REQUEST frame sent
|
||||
2. Record timestamp when RESPONSE frame received
|
||||
3. Calculate RTT = response_time - request_time
|
||||
4. Update exponential moving average: `avg = 0.8 * avg + 0.2 * rtt`
|
||||
|
||||
```csharp
|
||||
internal sealed class PingTracker
|
||||
{
|
||||
private readonly ConcurrentDictionary<Guid, long> _pendingRequests = new();
|
||||
private double _averagePingMs;
|
||||
|
||||
public void RecordRequestSent(Guid correlationId)
|
||||
{
|
||||
_pendingRequests[correlationId] = Stopwatch.GetTimestamp();
|
||||
}
|
||||
|
||||
public void RecordResponseReceived(Guid correlationId)
|
||||
{
|
||||
if (_pendingRequests.TryRemove(correlationId, out var startTicks))
|
||||
{
|
||||
var elapsed = Stopwatch.GetElapsedTime(startTicks);
|
||||
var rtt = elapsed.TotalMilliseconds;
|
||||
_averagePingMs = 0.8 * _averagePingMs + 0.2 * rtt;
|
||||
}
|
||||
}
|
||||
|
||||
public double AveragePingMs => _averagePingMs;
|
||||
}
|
||||
```
|
||||
|
||||
## Exit Criteria
|
||||
|
||||
Before marking this sprint DONE:
|
||||
1. [x] SDK sends HEARTBEAT frames on timer
|
||||
2. [x] Gateway processes HEARTBEAT and updates ConnectionState
|
||||
3. [x] HealthMonitorService marks stale instances Unhealthy
|
||||
4. [x] Draining status stops new requests
|
||||
5. [x] Ping latency measured and stored
|
||||
6. [x] Health thresholds configurable
|
||||
7. [x] Integration tests pass
|
||||
|
||||
## Execution Log
|
||||
|
||||
| Date (UTC) | Update | Owner |
|
||||
|------------|--------|-------|
|
||||
| 2025-12-05 | Sprint completed. Implemented heartbeat sending in SDK, health monitoring in Gateway, ping latency tracking. 51 tests passing. | Claude |
|
||||
|
||||
## Decisions & Risks
|
||||
|
||||
- Heartbeat interval default: 10 seconds (configurable)
|
||||
- Stale threshold default: 30 seconds (3 missed heartbeats)
|
||||
- Ping measurement uses REQUEST/RESPONSE timing, not separate PING frames
|
||||
- Health status changes are logged for observability
|
||||
217
docs/router/archived/SPRINT_7000_0005_0002_routing_algorithm.md
Normal file
217
docs/router/archived/SPRINT_7000_0005_0002_routing_algorithm.md
Normal file
@@ -0,0 +1,217 @@
|
||||
# Sprint 7000-0005-0002 · Protocol Features · Full Routing Algorithm
|
||||
|
||||
## Topic & Scope
|
||||
|
||||
Implement the complete routing algorithm as specified: region preference, ping-based selection, heartbeat recency, and fallback logic.
|
||||
|
||||
**Goal:** Routes prefer closest healthy instances with lowest latency, falling back through region tiers when necessary.
|
||||
|
||||
**Working directory:** `src/Gateway/StellaOps.Gateway.WebService/`
|
||||
|
||||
## Dependencies & Concurrency
|
||||
|
||||
- **Upstream:** SPRINT_7000_0005_0001 (heartbeat/health provides the metrics)
|
||||
- **Downstream:** SPRINT_7000_0005_0003 (cancellation), SPRINT_7000_0006_* (real transports)
|
||||
- **Parallel work:** None. Sequential.
|
||||
- **Cross-module impact:** Gateway only.
|
||||
|
||||
## Documentation Prerequisites
|
||||
|
||||
- `docs/router/specs.md` (section 4 - Routing algorithm / instance selection)
|
||||
- `docs/router/06-Step.md` (routing algorithm section)
|
||||
|
||||
> **BLOCKED Tasks:** Before working on BLOCKED tasks, review [../implplan/BLOCKED_DEPENDENCY_TREE.md](../implplan/BLOCKED_DEPENDENCY_TREE.md) for root blockers and dependencies.
|
||||
|
||||
## Delivery Tracker
|
||||
|
||||
| # | Task ID | Status | Description | Notes |
|
||||
|---|---------|--------|-------------|-------|
|
||||
| 1 | RTG-001 | DONE | Implement full filter chain in DefaultRoutingPlugin | |
|
||||
| 2 | RTG-002 | DONE | Filter by ServiceName (exact match) | Via AvailableConnections from context |
|
||||
| 3 | RTG-003 | DONE | Filter by Version (strict semver equality) | FilterByVersion method |
|
||||
| 4 | RTG-004 | DONE | Filter by Health (Healthy or Degraded only) | FilterByHealth method |
|
||||
| 5 | RTG-010 | DONE | Implement region tier logic | SelectByRegionTier method |
|
||||
| 6 | RTG-011 | DONE | Tier 0: Same region as gateway | GatewayNodeConfig.Region |
|
||||
| 7 | RTG-012 | DONE | Tier 1: Configured neighbor regions | NeighborRegions |
|
||||
| 8 | RTG-013 | DONE | Tier 2: All other regions | Fallback |
|
||||
| 9 | RTG-020 | DONE | Implement instance scoring within tier | SelectFromTier method |
|
||||
| 10 | RTG-021 | DONE | Primary sort: lower AveragePingMs | OrderBy AveragePingMs |
|
||||
| 11 | RTG-022 | DONE | Secondary sort: more recent LastHeartbeatUtc | ThenByDescending LastHeartbeatUtc |
|
||||
| 12 | RTG-023 | DONE | Tie-breaker: random or round-robin | Configurable via TieBreakerMode |
|
||||
| 13 | RTG-030 | DONE | Implement fallback decision order | Tier 0 → 1 → 2 |
|
||||
| 14 | RTG-031 | DONE | Fallback 1: Greater ping (latency) | Sorted ascending |
|
||||
| 15 | RTG-032 | DONE | Fallback 2: Greater heartbeat age | Sorted descending |
|
||||
| 16 | RTG-033 | DONE | Fallback 3: Less preferred region tier | Tier cascade |
|
||||
| 17 | RTG-040 | DONE | Create RoutingOptions for algorithm tuning | TieBreakerMode, PingToleranceMs |
|
||||
| 18 | RTG-041 | DONE | Add default version configuration | DefaultVersion property |
|
||||
| 19 | RTG-042 | DONE | Add health status acceptance set | AllowDegradedInstances |
|
||||
| 20 | RTG-050 | DONE | Write unit tests for each filter | 15+ tests |
|
||||
| 21 | RTG-051 | DONE | Write unit tests for region tier logic | Neighbor region tests |
|
||||
| 22 | RTG-052 | DONE | Write unit tests for scoring and tie-breaking | Ping/heartbeat/round-robin tests |
|
||||
| 23 | RTG-053 | DONE | Write integration tests for routing decisions | 55 tests passing |
|
||||
|
||||
## Routing Algorithm
|
||||
|
||||
```
|
||||
Input: (ServiceName, Version, Method, Path)
|
||||
Output: ConnectionState or null
|
||||
|
||||
1. Get all connections from IGlobalRoutingState.GetConnectionsFor(...)
|
||||
|
||||
2. Filter by ServiceName
|
||||
- connections.Where(c => c.Instance.ServiceName == serviceName)
|
||||
|
||||
3. Filter by Version (strict semver equality)
|
||||
- connections.Where(c => c.Instance.Version == version)
|
||||
- If version not specified, use DefaultVersion from config
|
||||
|
||||
4. Filter by Health
|
||||
- connections.Where(c => c.Status in {Healthy, Degraded})
|
||||
- Exclude Unknown, Draining, Unhealthy
|
||||
|
||||
5. Group by Region Tier
|
||||
- Tier 0: c.Instance.Region == GatewayNodeConfig.Region
|
||||
- Tier 1: c.Instance.Region in GatewayNodeConfig.NeighborRegions
|
||||
- Tier 2: All others
|
||||
|
||||
6. For each tier (0, 1, 2), if any candidates exist:
|
||||
a. Sort by AveragePingMs (ascending)
|
||||
b. For ties, sort by LastHeartbeatUtc (descending = more recent first)
|
||||
c. For remaining ties, apply tie-breaker (random or round-robin)
|
||||
d. Return first candidate
|
||||
|
||||
7. If no candidates in any tier, return null (503)
|
||||
```
|
||||
|
||||
## Implementation
|
||||
|
||||
```csharp
|
||||
public class DefaultRoutingPlugin : IRoutingPlugin
|
||||
{
|
||||
public async Task<RoutingDecision?> ChooseInstanceAsync(
|
||||
RoutingContext context, CancellationToken cancellationToken)
|
||||
{
|
||||
var endpoint = context.Endpoint;
|
||||
var gatewayRegion = context.GatewayRegion;
|
||||
|
||||
// Get all matching connections
|
||||
var connections = _routingState.GetConnectionsFor(
|
||||
endpoint.ServiceName,
|
||||
endpoint.Version,
|
||||
endpoint.Method,
|
||||
endpoint.Path);
|
||||
|
||||
// Filter by health
|
||||
var healthy = connections
|
||||
.Where(c => c.Status is InstanceHealthStatus.Healthy
|
||||
or InstanceHealthStatus.Degraded)
|
||||
.ToList();
|
||||
|
||||
if (healthy.Count == 0)
|
||||
return null;
|
||||
|
||||
// Group by region tier
|
||||
var tier0 = healthy.Where(c => c.Instance.Region == gatewayRegion).ToList();
|
||||
var tier1 = healthy.Where(c =>
|
||||
_options.NeighborRegions.Contains(c.Instance.Region)).ToList();
|
||||
var tier2 = healthy.Except(tier0).Except(tier1).ToList();
|
||||
|
||||
// Select from best tier
|
||||
var selected = SelectFromTier(tier0)
|
||||
?? SelectFromTier(tier1)
|
||||
?? SelectFromTier(tier2);
|
||||
|
||||
if (selected == null)
|
||||
return null;
|
||||
|
||||
return new RoutingDecision
|
||||
{
|
||||
Endpoint = endpoint,
|
||||
Connection = selected,
|
||||
TransportType = selected.TransportType,
|
||||
EffectiveTimeout = endpoint.DefaultTimeout
|
||||
};
|
||||
}
|
||||
|
||||
private ConnectionState? SelectFromTier(List<ConnectionState> tier)
|
||||
{
|
||||
if (tier.Count == 0)
|
||||
return null;
|
||||
|
||||
// Sort by ping (asc), then heartbeat (desc)
|
||||
var sorted = tier
|
||||
.OrderBy(c => c.AveragePingMs)
|
||||
.ThenByDescending(c => c.LastHeartbeatUtc)
|
||||
.ToList();
|
||||
|
||||
// Tie-breaker for same ping and heartbeat
|
||||
var best = sorted.First();
|
||||
var tied = sorted.TakeWhile(c =>
|
||||
Math.Abs(c.AveragePingMs - best.AveragePingMs) < 0.1
|
||||
&& c.LastHeartbeatUtc == best.LastHeartbeatUtc).ToList();
|
||||
|
||||
if (tied.Count == 1)
|
||||
return tied[0];
|
||||
|
||||
// Round-robin or random for ties
|
||||
return _options.TieBreaker == TieBreakerMode.Random
|
||||
? tied[Random.Shared.Next(tied.Count)]
|
||||
: tied[_roundRobinCounter++ % tied.Count];
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## RoutingOptions
|
||||
|
||||
```csharp
|
||||
public sealed class RoutingOptions
|
||||
{
|
||||
public Dictionary<string, string> DefaultVersions { get; set; } = new();
|
||||
public HashSet<InstanceHealthStatus> AcceptableStatuses { get; set; }
|
||||
= new() { InstanceHealthStatus.Healthy, InstanceHealthStatus.Degraded };
|
||||
public TieBreakerMode TieBreaker { get; set; } = TieBreakerMode.RoundRobin;
|
||||
}
|
||||
|
||||
public enum TieBreakerMode
|
||||
{
|
||||
Random,
|
||||
RoundRobin
|
||||
}
|
||||
```
|
||||
|
||||
## Spec Compliance Verification
|
||||
|
||||
From specs.md section 4:
|
||||
> * Region:
|
||||
> * Prefer instances whose `Region == GatewayNodeConfig.Region`.
|
||||
> * If none, fall back to configured neighbor regions.
|
||||
> * If none, fall back to all other regions.
|
||||
> * Within a chosen region tier:
|
||||
> * Prefer lower `AveragePingMs`.
|
||||
> * If several are tied, prefer more recent `LastHeartbeatUtc`.
|
||||
> * If still tied, use a balancing strategy (e.g. random or round-robin).
|
||||
|
||||
Implementation must match exactly.
|
||||
|
||||
## Exit Criteria
|
||||
|
||||
Before marking this sprint DONE:
|
||||
1. [x] Full filter chain implemented (service, version, health)
|
||||
2. [x] Region tier logic works (same region → neighbors → others)
|
||||
3. [x] Scoring within tier (ping, heartbeat, tie-breaker)
|
||||
4. [x] RoutingOptions configurable
|
||||
5. [x] All unit tests pass
|
||||
6. [x] Integration tests verify routing decisions
|
||||
|
||||
## Execution Log
|
||||
|
||||
| Date (UTC) | Update | Owner |
|
||||
|------------|--------|-------|
|
||||
| 2025-12-05 | Sprint completed. Full routing algorithm with region tiers, ping/heartbeat scoring, and tie-breaking. 55 tests passing. | Claude |
|
||||
|
||||
## Decisions & Risks
|
||||
|
||||
- Ping tolerance for "ties": 0.1ms difference considered equal
|
||||
- Round-robin counter is per-endpoint to avoid hot instances
|
||||
- DefaultVersion lookup is per-service from configuration
|
||||
- Degraded instances are routed to (may want to prefer Healthy first)
|
||||
230
docs/router/archived/SPRINT_7000_0005_0003_cancellation.md
Normal file
230
docs/router/archived/SPRINT_7000_0005_0003_cancellation.md
Normal file
@@ -0,0 +1,230 @@
|
||||
# Sprint 7000-0005-0003 · Protocol Features · Cancellation Semantics
|
||||
|
||||
## Topic & Scope
|
||||
|
||||
Implement cancellation semantics on both gateway and microservice sides. When HTTP clients disconnect, timeouts occur, or payload limits are breached, CANCEL frames are sent to stop in-flight work.
|
||||
|
||||
**Goal:** Clean cancellation propagation from HTTP client through gateway to microservice handlers.
|
||||
|
||||
**Working directories:**
|
||||
- `src/Gateway/StellaOps.Gateway.WebService/` (send CANCEL)
|
||||
- `src/__Libraries/StellaOps.Microservice/` (receive CANCEL, cancel handler)
|
||||
- `src/__Libraries/StellaOps.Router.Common/` (CancelPayload)
|
||||
|
||||
## Dependencies & Concurrency
|
||||
|
||||
- **Upstream:** SPRINT_7000_0005_0002 (routing algorithm complete)
|
||||
- **Downstream:** SPRINT_7000_0005_0004 (streaming uses cancellation)
|
||||
- **Parallel work:** None. Sequential.
|
||||
- **Cross-module impact:** SDK and Gateway both modified.
|
||||
|
||||
## Documentation Prerequisites
|
||||
|
||||
- `docs/router/specs.md` (sections 7.6, 10 - Cancellation requirements)
|
||||
- `docs/router/07-Step.md` (cancellation section)
|
||||
- `docs/router/implplan.md` (phase 7 guidance)
|
||||
|
||||
> **BLOCKED Tasks:** Before working on BLOCKED tasks, review [../implplan/BLOCKED_DEPENDENCY_TREE.md](../implplan/BLOCKED_DEPENDENCY_TREE.md) for root blockers and dependencies.
|
||||
|
||||
## Delivery Tracker
|
||||
|
||||
| # | Task ID | Status | Description | Working Directory |
|
||||
|---|---------|--------|-------------|-------------------|
|
||||
| 1 | CAN-001 | DONE | Define CancelPayload with Reason code | Common |
|
||||
| 2 | CAN-002 | DONE | Define cancel reason constants | ClientDisconnected, Timeout, PayloadLimitExceeded, Shutdown |
|
||||
| 3 | CAN-010 | DONE | Implement CANCEL frame sending in gateway | Gateway |
|
||||
| 4 | CAN-011 | DONE | Wire HttpContext.RequestAborted to CANCEL | Gateway |
|
||||
| 5 | CAN-012 | DONE | Implement timeout-triggered CANCEL | Gateway |
|
||||
| 6 | CAN-013 | DONE | Implement payload-limit-triggered CANCEL | Gateway |
|
||||
| 7 | CAN-014 | DONE | Implement shutdown-triggered CANCEL for in-flight | Gateway |
|
||||
| 8 | CAN-020 | DONE | Stop forwarding REQUEST_STREAM_DATA after CANCEL | Gateway |
|
||||
| 9 | CAN-021 | DONE | Ignore late RESPONSE frames for cancelled requests | Gateway |
|
||||
| 10 | CAN-022 | DONE | Log cancelled requests with reason | Gateway |
|
||||
| 11 | CAN-030 | DONE | Implement inflight request tracking in SDK | Microservice |
|
||||
| 12 | CAN-031 | DONE | Create ConcurrentDictionary<Guid, CancellationTokenSource> | Microservice |
|
||||
| 13 | CAN-032 | DONE | Add handler task to tracking map | Microservice |
|
||||
| 14 | CAN-033 | DONE | Implement CANCEL frame processing | Microservice |
|
||||
| 15 | CAN-034 | DONE | Call cts.Cancel() on CANCEL frame | Microservice |
|
||||
| 16 | CAN-035 | DONE | Remove from tracking when handler completes | Microservice |
|
||||
| 17 | CAN-040 | DONE | Implement connection-close cancellation | Microservice |
|
||||
| 18 | CAN-041 | DONE | Cancel all inflight on connection loss | Microservice |
|
||||
| 19 | CAN-050 | DONE | Pass CancellationToken to handler interfaces | Microservice |
|
||||
| 20 | CAN-051 | DONE | Document cancellation best practices for handlers | Docs |
|
||||
| 21 | CAN-060 | DONE | Write integration tests: client disconnect → handler cancelled | |
|
||||
| 22 | CAN-061 | DONE | Write integration tests: timeout → handler cancelled | |
|
||||
| 23 | CAN-062 | DONE | Write tests: late response ignored | |
|
||||
|
||||
## CancelPayload
|
||||
|
||||
```csharp
|
||||
public sealed class CancelPayload
|
||||
{
|
||||
public string Reason { get; init; } = string.Empty;
|
||||
}
|
||||
|
||||
public static class CancelReasons
|
||||
{
|
||||
public const string ClientDisconnected = "ClientDisconnected";
|
||||
public const string Timeout = "Timeout";
|
||||
public const string PayloadLimitExceeded = "PayloadLimitExceeded";
|
||||
public const string Shutdown = "Shutdown";
|
||||
}
|
||||
```
|
||||
|
||||
## Gateway-Side: Sending CANCEL
|
||||
|
||||
### On Client Disconnect
|
||||
```csharp
|
||||
// In TransportDispatchMiddleware
|
||||
context.RequestAborted.Register(async () =>
|
||||
{
|
||||
await transport.SendCancelAsync(
|
||||
connection,
|
||||
correlationId,
|
||||
CancelReasons.ClientDisconnected);
|
||||
});
|
||||
```
|
||||
|
||||
### On Timeout
|
||||
```csharp
|
||||
using var cts = CancellationTokenSource.CreateLinkedTokenSource(context.RequestAborted);
|
||||
cts.CancelAfter(decision.EffectiveTimeout);
|
||||
|
||||
try
|
||||
{
|
||||
var response = await transport.SendRequestAsync(..., cts.Token);
|
||||
}
|
||||
catch (OperationCanceledException) when (cts.IsCancellationRequested)
|
||||
{
|
||||
if (!context.RequestAborted.IsCancellationRequested)
|
||||
{
|
||||
// Timeout, not client disconnect
|
||||
await transport.SendCancelAsync(connection, correlationId, CancelReasons.Timeout);
|
||||
context.Response.StatusCode = 504;
|
||||
return;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Late Response Handling
|
||||
```csharp
|
||||
private readonly ConcurrentDictionary<Guid, bool> _cancelledRequests = new();
|
||||
|
||||
public void MarkCancelled(Guid correlationId)
|
||||
{
|
||||
_cancelledRequests[correlationId] = true;
|
||||
}
|
||||
|
||||
public bool IsCancelled(Guid correlationId)
|
||||
{
|
||||
return _cancelledRequests.ContainsKey(correlationId);
|
||||
}
|
||||
|
||||
// When response arrives
|
||||
if (IsCancelled(frame.CorrelationId))
|
||||
{
|
||||
_logger.LogDebug("Ignoring late response for cancelled {CorrelationId}", frame.CorrelationId);
|
||||
return; // Discard
|
||||
}
|
||||
```
|
||||
|
||||
## Microservice-Side: Receiving CANCEL
|
||||
|
||||
### Inflight Tracking
|
||||
```csharp
|
||||
internal sealed class InflightRequestTracker
|
||||
{
|
||||
private readonly ConcurrentDictionary<Guid, InflightRequest> _inflight = new();
|
||||
|
||||
public CancellationToken Track(Guid correlationId, Task handlerTask)
|
||||
{
|
||||
var cts = new CancellationTokenSource();
|
||||
_inflight[correlationId] = new InflightRequest(cts, handlerTask);
|
||||
return cts.Token;
|
||||
}
|
||||
|
||||
public void Cancel(Guid correlationId, string reason)
|
||||
{
|
||||
if (_inflight.TryGetValue(correlationId, out var request))
|
||||
{
|
||||
request.Cts.Cancel();
|
||||
_logger.LogInformation("Cancelled {CorrelationId}: {Reason}", correlationId, reason);
|
||||
}
|
||||
}
|
||||
|
||||
public void Complete(Guid correlationId)
|
||||
{
|
||||
if (_inflight.TryRemove(correlationId, out var request))
|
||||
{
|
||||
request.Cts.Dispose();
|
||||
}
|
||||
}
|
||||
|
||||
public void CancelAll(string reason)
|
||||
{
|
||||
foreach (var kvp in _inflight)
|
||||
{
|
||||
kvp.Value.Cts.Cancel();
|
||||
}
|
||||
_inflight.Clear();
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Connection-Close Handling
|
||||
```csharp
|
||||
// When connection closes unexpectedly
|
||||
_inflightTracker.CancelAll("ConnectionClosed");
|
||||
```
|
||||
|
||||
## Handler Cancellation Guidelines
|
||||
|
||||
Handlers MUST:
|
||||
1. Accept `CancellationToken` parameter
|
||||
2. Pass token to all async I/O operations
|
||||
3. Check `token.IsCancellationRequested` in loops
|
||||
4. Stop work promptly when cancelled
|
||||
|
||||
```csharp
|
||||
public class ProcessDataEndpoint : IStellaEndpoint<DataRequest, DataResponse>
|
||||
{
|
||||
public async Task<DataResponse> HandleAsync(DataRequest request, CancellationToken ct)
|
||||
{
|
||||
// Pass token to I/O
|
||||
var data = await _database.QueryAsync(request.Id, ct);
|
||||
|
||||
// Check in loops
|
||||
foreach (var item in data)
|
||||
{
|
||||
ct.ThrowIfCancellationRequested();
|
||||
await ProcessItemAsync(item, ct);
|
||||
}
|
||||
|
||||
return new DataResponse { ... };
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Exit Criteria
|
||||
|
||||
Before marking this sprint DONE:
|
||||
1. [x] CANCEL frames sent on client disconnect
|
||||
2. [x] CANCEL frames sent on timeout
|
||||
3. [x] SDK tracks inflight requests with CTS
|
||||
4. [x] SDK cancels handlers on CANCEL frame
|
||||
5. [x] Connection close cancels all inflight
|
||||
6. [x] Late responses are ignored/logged
|
||||
7. [x] Integration tests verify cancellation flow
|
||||
|
||||
## Execution Log
|
||||
|
||||
| Date (UTC) | Update | Owner |
|
||||
|------------|--------|-------|
|
||||
| 2025-12-05 | Sprint DONE - CancelReasons defined, InflightRequestTracker implemented, Gateway sends CANCEL on disconnect/timeout, SDK handles CANCEL frames, 67 tests pass | Claude |
|
||||
|
||||
## Decisions & Risks
|
||||
|
||||
- Cancellation is cooperative; handlers must honor the token
|
||||
- CTS disposal happens on completion to avoid leaks
|
||||
- Late response cleanup: entries expire after 60 seconds
|
||||
- Shutdown CANCEL is best-effort (connections may close first)
|
||||
215
docs/router/archived/SPRINT_7000_0005_0004_streaming.md
Normal file
215
docs/router/archived/SPRINT_7000_0005_0004_streaming.md
Normal file
@@ -0,0 +1,215 @@
|
||||
# Sprint 7000-0005-0004 · Protocol Features · Streaming Support
|
||||
|
||||
## Topic & Scope
|
||||
|
||||
Implement streaming request/response support. Large payloads stream through the gateway as `REQUEST_STREAM_DATA` and `RESPONSE_STREAM_DATA` frames rather than being fully buffered.
|
||||
|
||||
**Goal:** Enable large file uploads/downloads without memory exhaustion at gateway.
|
||||
|
||||
**Working directories:**
|
||||
- `src/Gateway/StellaOps.Gateway.WebService/` (streaming dispatch)
|
||||
- `src/__Libraries/StellaOps.Microservice/` (streaming handlers)
|
||||
- `src/__Libraries/StellaOps.Router.Transport.InMemory/` (streaming frames)
|
||||
|
||||
## Dependencies & Concurrency
|
||||
|
||||
- **Upstream:** SPRINT_7000_0005_0003 (cancellation - streaming needs cancel support)
|
||||
- **Downstream:** SPRINT_7000_0005_0005 (payload limits)
|
||||
- **Parallel work:** None. Sequential.
|
||||
- **Cross-module impact:** SDK, Gateway, InMemory transport all modified.
|
||||
|
||||
## Documentation Prerequisites
|
||||
|
||||
- `docs/router/specs.md` (sections 5.4, 6.3, 7.5 - Streaming requirements)
|
||||
- `docs/router/08-Step.md` (streaming section)
|
||||
- `docs/router/implplan.md` (phase 8 guidance)
|
||||
|
||||
> **BLOCKED Tasks:** Before working on BLOCKED tasks, review [../implplan/BLOCKED_DEPENDENCY_TREE.md](../implplan/BLOCKED_DEPENDENCY_TREE.md) for root blockers and dependencies.
|
||||
|
||||
## Delivery Tracker
|
||||
|
||||
| # | Task ID | Status | Description | Working Directory |
|
||||
|---|---------|--------|-------------|-------------------|
|
||||
| 1 | STR-001 | DONE | Add SupportsStreaming flag to EndpointDescriptor | Common |
|
||||
| 2 | STR-002 | DONE | Add streaming attribute support to [StellaEndpoint] | Common |
|
||||
| 3 | STR-010 | DONE | Implement REQUEST_STREAM_DATA frame handling in transport | InMemory |
|
||||
| 4 | STR-011 | DONE | Implement RESPONSE_STREAM_DATA frame handling in transport | InMemory |
|
||||
| 5 | STR-012 | DONE | Implement end-of-stream signaling | InMemory |
|
||||
| 6 | STR-020 | DONE | Implement streaming request dispatch in gateway | Gateway |
|
||||
| 7 | STR-021 | DONE | Pipe HTTP body stream → REQUEST_STREAM_DATA frames | Gateway |
|
||||
| 8 | STR-022 | DONE | Implement chunking for stream data | Configurable chunk size |
|
||||
| 9 | STR-023 | DONE | Honor cancellation during streaming | Gateway |
|
||||
| 10 | STR-030 | DONE | Implement streaming response handling in gateway | Gateway |
|
||||
| 11 | STR-031 | DONE | Pipe RESPONSE_STREAM_DATA frames → HTTP response | Gateway |
|
||||
| 12 | STR-032 | DONE | Set chunked transfer encoding | Gateway |
|
||||
| 13 | STR-040 | DONE | Implement streaming body in RawRequestContext | Microservice |
|
||||
| 14 | STR-041 | DONE | Expose Body as async-readable stream | Microservice |
|
||||
| 15 | STR-042 | DONE | Implement backpressure (slow consumer) | Microservice |
|
||||
| 16 | STR-050 | DONE | Implement streaming response writing | Microservice |
|
||||
| 17 | STR-051 | DONE | Expose WriteBodyAsync for streaming output | Microservice |
|
||||
| 18 | STR-052 | DONE | Chunk output into RESPONSE_STREAM_DATA frames | Microservice |
|
||||
| 19 | STR-060 | DONE | Implement IRawStellaEndpoint streaming pattern | Microservice |
|
||||
| 20 | STR-061 | DONE | Document streaming handler guidelines | Docs |
|
||||
| 21 | STR-070 | DONE | Write integration tests for upload streaming | |
|
||||
| 22 | STR-071 | DONE | Write integration tests for download streaming | |
|
||||
| 23 | STR-072 | DONE | Write tests for cancellation during streaming | |
|
||||
|
||||
## Streaming Frame Protocol
|
||||
|
||||
### Request Streaming
|
||||
```
|
||||
Gateway → Microservice:
|
||||
1. REQUEST frame (headers, method, path, CorrelationId)
|
||||
2. REQUEST_STREAM_DATA frame (chunk 1)
|
||||
3. REQUEST_STREAM_DATA frame (chunk 2)
|
||||
...
|
||||
N. REQUEST_STREAM_DATA frame (final chunk, EndOfStream=true)
|
||||
```
|
||||
|
||||
### Response Streaming
|
||||
```
|
||||
Microservice → Gateway:
|
||||
1. RESPONSE frame (status code, headers, CorrelationId)
|
||||
2. RESPONSE_STREAM_DATA frame (chunk 1)
|
||||
3. RESPONSE_STREAM_DATA frame (chunk 2)
|
||||
...
|
||||
N. RESPONSE_STREAM_DATA frame (final chunk, EndOfStream=true)
|
||||
```
|
||||
|
||||
## StreamDataPayload
|
||||
|
||||
```csharp
|
||||
public sealed class StreamDataPayload
|
||||
{
|
||||
public Guid CorrelationId { get; init; }
|
||||
public byte[] Data { get; init; } = Array.Empty<byte>();
|
||||
public bool EndOfStream { get; init; }
|
||||
public int SequenceNumber { get; init; }
|
||||
}
|
||||
```
|
||||
|
||||
## Gateway Streaming Dispatch
|
||||
|
||||
```csharp
|
||||
// In TransportDispatchMiddleware
|
||||
if (endpoint.SupportsStreaming)
|
||||
{
|
||||
await DispatchStreamingAsync(context, transport, decision, cancellationToken);
|
||||
}
|
||||
else
|
||||
{
|
||||
await DispatchBufferedAsync(context, transport, decision, cancellationToken);
|
||||
}
|
||||
|
||||
private async Task DispatchStreamingAsync(...)
|
||||
{
|
||||
// Send REQUEST header
|
||||
var requestFrame = BuildRequestHeaderFrame(context);
|
||||
await transport.SendFrameAsync(connection, requestFrame, ct);
|
||||
|
||||
// Stream body chunks
|
||||
var buffer = new byte[_options.StreamChunkSize];
|
||||
int bytesRead;
|
||||
int sequence = 0;
|
||||
|
||||
while ((bytesRead = await context.Request.Body.ReadAsync(buffer, ct)) > 0)
|
||||
{
|
||||
var streamFrame = new Frame
|
||||
{
|
||||
Type = FrameType.RequestStreamData,
|
||||
CorrelationId = requestFrame.CorrelationId,
|
||||
Payload = SerializeStreamData(buffer[..bytesRead], sequence++, endOfStream: false)
|
||||
};
|
||||
await transport.SendFrameAsync(connection, streamFrame, ct);
|
||||
}
|
||||
|
||||
// Send end-of-stream
|
||||
var endFrame = new Frame
|
||||
{
|
||||
Type = FrameType.RequestStreamData,
|
||||
CorrelationId = requestFrame.CorrelationId,
|
||||
Payload = SerializeStreamData(Array.Empty<byte>(), sequence, endOfStream: true)
|
||||
};
|
||||
await transport.SendFrameAsync(connection, endFrame, ct);
|
||||
|
||||
// Receive response (streaming or buffered)
|
||||
await ReceiveResponseAsync(context, transport, connection, requestFrame.CorrelationId, ct);
|
||||
}
|
||||
```
|
||||
|
||||
## Microservice Streaming Handler
|
||||
|
||||
```csharp
|
||||
[StellaEndpoint("POST", "/files/upload", SupportsStreaming = true)]
|
||||
public class FileUploadEndpoint : IRawStellaEndpoint
|
||||
{
|
||||
public async Task<RawResponse> HandleAsync(RawRequestContext context, CancellationToken ct)
|
||||
{
|
||||
// Body is a stream that reads from REQUEST_STREAM_DATA frames
|
||||
var tempPath = Path.GetTempFileName();
|
||||
|
||||
await using var fileStream = File.Create(tempPath);
|
||||
await context.Body.CopyToAsync(fileStream, ct);
|
||||
|
||||
return RawResponse.Ok($"Uploaded {fileStream.Length} bytes");
|
||||
}
|
||||
}
|
||||
|
||||
[StellaEndpoint("GET", "/files/{id}/download", SupportsStreaming = true)]
|
||||
public class FileDownloadEndpoint : IRawStellaEndpoint
|
||||
{
|
||||
public async Task<RawResponse> HandleAsync(RawRequestContext context, CancellationToken ct)
|
||||
{
|
||||
var fileId = context.PathParameters["id"];
|
||||
var filePath = _storage.GetPath(fileId);
|
||||
|
||||
// Return streaming response
|
||||
return new RawResponse
|
||||
{
|
||||
StatusCode = 200,
|
||||
Body = File.OpenRead(filePath), // Stream, not buffered
|
||||
Headers = new HeaderCollection
|
||||
{
|
||||
["Content-Type"] = "application/octet-stream"
|
||||
}
|
||||
};
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## StreamingOptions
|
||||
|
||||
```csharp
|
||||
public sealed class StreamingOptions
|
||||
{
|
||||
public int ChunkSize { get; set; } = 64 * 1024; // 64KB default
|
||||
public int MaxConcurrentStreams { get; set; } = 100;
|
||||
public TimeSpan StreamIdleTimeout { get; set; } = TimeSpan.FromMinutes(5);
|
||||
}
|
||||
```
|
||||
|
||||
## Exit Criteria
|
||||
|
||||
Before marking this sprint DONE:
|
||||
1. [x] REQUEST_STREAM_DATA frames implemented in transport
|
||||
2. [x] RESPONSE_STREAM_DATA frames implemented in transport
|
||||
3. [x] Gateway streams request body to microservice
|
||||
4. [x] Gateway streams response body to HTTP client
|
||||
5. [x] SDK exposes streaming Body in RawRequestContext
|
||||
6. [x] SDK can write streaming response
|
||||
7. [x] Cancellation works during streaming
|
||||
8. [x] Integration tests for upload and download streaming
|
||||
|
||||
## Execution Log
|
||||
|
||||
| Date (UTC) | Update | Owner |
|
||||
|------------|--------|-------|
|
||||
| 2025-12-05 | Sprint DONE - StreamDataPayload, StreamingOptions, StreamingRequestBodyStream, StreamingResponseBodyStream, DispatchStreamingAsync in gateway, 80 tests pass | Claude |
|
||||
|
||||
## Decisions & Risks
|
||||
|
||||
- Default chunk size: 64KB (tunable)
|
||||
- End-of-stream is explicit frame, not connection close
|
||||
- Backpressure via channel capacity (bounded channels)
|
||||
- Idle timeout cancels stuck streams
|
||||
- Typed handlers don't support streaming (use IRawStellaEndpoint)
|
||||
231
docs/router/archived/SPRINT_7000_0005_0005_payload_limits.md
Normal file
231
docs/router/archived/SPRINT_7000_0005_0005_payload_limits.md
Normal file
@@ -0,0 +1,231 @@
|
||||
# Sprint 7000-0005-0005 · Protocol Features · Payload Limits
|
||||
|
||||
## Topic & Scope
|
||||
|
||||
Implement payload size limits to protect the gateway from memory exhaustion. Enforce limits per-request, per-connection, and aggregate across all connections.
|
||||
|
||||
**Goal:** Gateway rejects oversized payloads early and cancels streams that exceed limits mid-flight.
|
||||
|
||||
**Working directory:** `src/Gateway/StellaOps.Gateway.WebService/`
|
||||
|
||||
## Dependencies & Concurrency
|
||||
|
||||
- **Upstream:** SPRINT_7000_0005_0004 (streaming - limits apply to streams)
|
||||
- **Downstream:** SPRINT_7000_0006_* (real transports)
|
||||
- **Parallel work:** None. Sequential.
|
||||
- **Cross-module impact:** Gateway only.
|
||||
|
||||
## Documentation Prerequisites
|
||||
|
||||
- `docs/router/specs.md` (section 6.5 - Payload and memory protection)
|
||||
- `docs/router/08-Step.md` (payload limits section)
|
||||
|
||||
> **BLOCKED Tasks:** Before working on BLOCKED tasks, review [../implplan/BLOCKED_DEPENDENCY_TREE.md](../implplan/BLOCKED_DEPENDENCY_TREE.md) for root blockers and dependencies.
|
||||
|
||||
## Delivery Tracker
|
||||
|
||||
| # | Task ID | Status | Description | Notes |
|
||||
|---|---------|--------|-------------|-------|
|
||||
| 1 | LIM-001 | DONE | Implement PayloadLimitsMiddleware | Before dispatch |
|
||||
| 2 | LIM-002 | DONE | Check Content-Length header against MaxRequestBytesPerCall | |
|
||||
| 3 | LIM-003 | DONE | Return 413 for oversized Content-Length | Early rejection |
|
||||
| 4 | LIM-010 | DONE | Implement per-request byte counter | ByteCountingStream |
|
||||
| 5 | LIM-011 | DONE | Track bytes read during streaming | |
|
||||
| 6 | LIM-012 | DONE | Abort when MaxRequestBytesPerCall exceeded mid-stream | |
|
||||
| 7 | LIM-013 | DONE | Send CANCEL frame on limit breach | Via PayloadLimitExceededException |
|
||||
| 8 | LIM-020 | DONE | Implement per-connection byte counter | PayloadTracker |
|
||||
| 9 | LIM-021 | DONE | Track total inflight bytes per connection | |
|
||||
| 10 | LIM-022 | DONE | Throttle/reject when MaxRequestBytesPerConnection exceeded | Returns 429 |
|
||||
| 11 | LIM-030 | DONE | Implement aggregate byte counter | PayloadTracker |
|
||||
| 12 | LIM-031 | DONE | Track total inflight bytes across all connections | |
|
||||
| 13 | LIM-032 | DONE | Throttle/reject when MaxAggregateInflightBytes exceeded | |
|
||||
| 14 | LIM-033 | DONE | Return 503 for aggregate limit | Service overloaded |
|
||||
| 15 | LIM-040 | DONE | Implement ByteCountingStream wrapper | Counts bytes as they flow |
|
||||
| 16 | LIM-041 | DONE | Wire counting stream into dispatch | Via middleware |
|
||||
| 17 | LIM-050 | DONE | Create PayloadLimitOptions | PayloadLimits record |
|
||||
| 18 | LIM-051 | DONE | Bind PayloadLimitOptions from configuration | IOptions<PayloadLimits> |
|
||||
| 19 | LIM-060 | DONE | Log limit breaches with request details | Warning level |
|
||||
| 20 | LIM-061 | DONE | Add metrics for payload tracking | Via IPayloadTracker.CurrentInflightBytes |
|
||||
| 21 | LIM-070 | DONE | Write tests for early rejection (Content-Length) | ByteCountingStreamTests |
|
||||
| 22 | LIM-071 | DONE | Write tests for mid-stream cancellation | |
|
||||
| 23 | LIM-072 | DONE | Write tests for connection limit | PayloadTrackerTests |
|
||||
| 24 | LIM-073 | DONE | Write tests for aggregate limit | PayloadTrackerTests |
|
||||
|
||||
## PayloadLimits
|
||||
|
||||
```csharp
|
||||
public sealed class PayloadLimits
|
||||
{
|
||||
public long MaxRequestBytesPerCall { get; set; } = 10 * 1024 * 1024; // 10 MB
|
||||
public long MaxRequestBytesPerConnection { get; set; } = 100 * 1024 * 1024; // 100 MB
|
||||
public long MaxAggregateInflightBytes { get; set; } = 1024 * 1024 * 1024; // 1 GB
|
||||
}
|
||||
```
|
||||
|
||||
## PayloadLimitsMiddleware
|
||||
|
||||
```csharp
|
||||
public class PayloadLimitsMiddleware
|
||||
{
|
||||
public async Task InvokeAsync(HttpContext context, IPayloadTracker tracker)
|
||||
{
|
||||
// Early rejection for known Content-Length
|
||||
if (context.Request.ContentLength.HasValue)
|
||||
{
|
||||
if (context.Request.ContentLength > _limits.MaxRequestBytesPerCall)
|
||||
{
|
||||
_logger.LogWarning("Request rejected: Content-Length {Length} exceeds limit {Limit}",
|
||||
context.Request.ContentLength, _limits.MaxRequestBytesPerCall);
|
||||
context.Response.StatusCode = 413; // Payload Too Large
|
||||
await context.Response.WriteAsJsonAsync(new
|
||||
{
|
||||
error = "Payload Too Large",
|
||||
maxBytes = _limits.MaxRequestBytesPerCall
|
||||
});
|
||||
return;
|
||||
}
|
||||
}
|
||||
|
||||
// Check aggregate capacity
|
||||
if (!tracker.TryReserve(context.Request.ContentLength ?? 0))
|
||||
{
|
||||
context.Response.StatusCode = 503; // Service Unavailable
|
||||
await context.Response.WriteAsJsonAsync(new
|
||||
{
|
||||
error = "Service Overloaded",
|
||||
message = "Too many concurrent requests"
|
||||
});
|
||||
return;
|
||||
}
|
||||
|
||||
try
|
||||
{
|
||||
await _next(context);
|
||||
}
|
||||
finally
|
||||
{
|
||||
tracker.Release(/* bytes actually used */);
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## IPayloadTracker
|
||||
|
||||
```csharp
|
||||
public interface IPayloadTracker
|
||||
{
|
||||
bool TryReserve(long estimatedBytes);
|
||||
void Release(long actualBytes);
|
||||
long CurrentInflightBytes { get; }
|
||||
bool IsOverloaded { get; }
|
||||
}
|
||||
|
||||
internal sealed class PayloadTracker : IPayloadTracker
|
||||
{
|
||||
private long _totalInflightBytes;
|
||||
private readonly ConcurrentDictionary<string, long> _perConnectionBytes = new();
|
||||
|
||||
public bool TryReserve(long estimatedBytes)
|
||||
{
|
||||
var newTotal = Interlocked.Add(ref _totalInflightBytes, estimatedBytes);
|
||||
if (newTotal > _limits.MaxAggregateInflightBytes)
|
||||
{
|
||||
Interlocked.Add(ref _totalInflightBytes, -estimatedBytes);
|
||||
return false;
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
public void Release(long actualBytes)
|
||||
{
|
||||
Interlocked.Add(ref _totalInflightBytes, -actualBytes);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## ByteCountingStream
|
||||
|
||||
```csharp
|
||||
internal sealed class ByteCountingStream : Stream
|
||||
{
|
||||
private readonly Stream _inner;
|
||||
private readonly long _limit;
|
||||
private readonly Action _onLimitExceeded;
|
||||
private long _bytesRead;
|
||||
|
||||
public override async ValueTask<int> ReadAsync(Memory<byte> buffer, CancellationToken ct)
|
||||
{
|
||||
var read = await _inner.ReadAsync(buffer, ct);
|
||||
_bytesRead += read;
|
||||
|
||||
if (_bytesRead > _limit)
|
||||
{
|
||||
_onLimitExceeded();
|
||||
throw new PayloadLimitExceededException(_bytesRead, _limit);
|
||||
}
|
||||
|
||||
return read;
|
||||
}
|
||||
|
||||
public long BytesRead => _bytesRead;
|
||||
}
|
||||
```
|
||||
|
||||
## Mid-Stream Limit Breach Flow
|
||||
|
||||
```
|
||||
1. Streaming request begins
|
||||
2. Gateway counts bytes as they flow through ByteCountingStream
|
||||
3. When _bytesRead > MaxRequestBytesPerCall:
|
||||
a. Stop reading from HTTP body
|
||||
b. Send CANCEL frame with reason "PayloadLimitExceeded"
|
||||
c. Return 413 to client
|
||||
d. Log the incident with request details
|
||||
```
|
||||
|
||||
## Configuration
|
||||
|
||||
```json
|
||||
{
|
||||
"PayloadLimits": {
|
||||
"MaxRequestBytesPerCall": 10485760,
|
||||
"MaxRequestBytesPerConnection": 104857600,
|
||||
"MaxAggregateInflightBytes": 1073741824
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Error Responses
|
||||
|
||||
| Condition | HTTP Status | Error Message |
|
||||
|-----------|-------------|---------------|
|
||||
| Content-Length exceeds per-call limit | 413 | Payload Too Large |
|
||||
| Streaming exceeds per-call limit | 413 | Payload Too Large |
|
||||
| Per-connection limit exceeded | 429 | Too Many Requests |
|
||||
| Aggregate limit exceeded | 503 | Service Overloaded |
|
||||
|
||||
## Exit Criteria
|
||||
|
||||
Before marking this sprint DONE:
|
||||
1. [x] Early rejection for known oversized Content-Length
|
||||
2. [x] Mid-stream cancellation when limit exceeded
|
||||
3. [x] CANCEL frame sent on limit breach
|
||||
4. [x] Per-connection tracking works
|
||||
5. [x] Aggregate tracking works
|
||||
6. [x] All limit scenarios tested
|
||||
7. [x] Metrics/logging in place
|
||||
|
||||
## Execution Log
|
||||
|
||||
| Date (UTC) | Update | Owner |
|
||||
|------------|--------|-------|
|
||||
| 2025-12-05 | Sprint DONE - PayloadTracker, ByteCountingStream, PayloadLimitsMiddleware, PayloadLimitExceededException, 97 tests pass | Claude |
|
||||
|
||||
## Decisions & Risks
|
||||
|
||||
- Default limits are conservative; tune for your environment
|
||||
- Per-connection limit applies to inflight bytes, not lifetime total
|
||||
- Aggregate limit prevents memory exhaustion but may cause 503s under load
|
||||
- ByteCountingStream adds minimal overhead
|
||||
- Limit breach is logged at Warning level
|
||||
231
docs/router/archived/SPRINT_7000_0006_0001_transport_tcp.md
Normal file
231
docs/router/archived/SPRINT_7000_0006_0001_transport_tcp.md
Normal file
@@ -0,0 +1,231 @@
|
||||
# Sprint 7000-0006-0001 · Real Transports · TCP Plugin
|
||||
|
||||
## Topic & Scope
|
||||
|
||||
Implement the TCP transport plugin. This is the primary production transport with length-prefixed framing for reliable frame delivery.
|
||||
|
||||
**Goal:** Replace InMemory transport with production-grade TCP transport.
|
||||
|
||||
**Working directory:** `src/__Libraries/StellaOps.Router.Transport.Tcp/`
|
||||
|
||||
## Dependencies & Concurrency
|
||||
|
||||
- **Upstream:** SPRINT_7000_0005_0005 (all protocol features proven with InMemory)
|
||||
- **Downstream:** SPRINT_7000_0006_0002 (TLS wraps TCP)
|
||||
- **Parallel work:** None initially; UDP and RabbitMQ can start after TCP basics work
|
||||
- **Cross-module impact:** None. New library only.
|
||||
|
||||
## Documentation Prerequisites
|
||||
|
||||
- `docs/router/specs.md` (section 5 - Transport plugin requirements)
|
||||
- `docs/router/09-Step.md` (TCP transport section)
|
||||
- `docs/router/implplan.md` (phase 9 guidance)
|
||||
|
||||
> **BLOCKED Tasks:** Before working on BLOCKED tasks, review [../implplan/BLOCKED_DEPENDENCY_TREE.md](../implplan/BLOCKED_DEPENDENCY_TREE.md) for root blockers and dependencies.
|
||||
|
||||
## Delivery Tracker
|
||||
|
||||
| # | Task ID | Status | Description | Notes |
|
||||
|---|---------|--------|-------------|-------|
|
||||
| 1 | TCP-001 | DONE | Create `StellaOps.Router.Transport.Tcp` classlib project | Add to solution |
|
||||
| 2 | TCP-002 | DONE | Add project reference to Router.Common | |
|
||||
| 3 | TCP-010 | DONE | Implement `TcpTransportServer` : `ITransportServer` | Gateway side |
|
||||
| 4 | TCP-011 | DONE | Implement TCP listener with configurable bind address/port | |
|
||||
| 5 | TCP-012 | DONE | Implement connection accept loop | One connection per microservice |
|
||||
| 6 | TCP-013 | DONE | Implement connection ID generation | Based on endpoint |
|
||||
| 7 | TCP-020 | DONE | Implement `TcpTransportClient` : `ITransportClient` | Microservice side |
|
||||
| 8 | TCP-021 | DONE | Implement connection establishment | With retry |
|
||||
| 9 | TCP-022 | DONE | Implement reconnection on failure | Exponential backoff |
|
||||
| 10 | TCP-030 | DONE | Implement length-prefixed framing protocol | FrameProtocol class |
|
||||
| 11 | TCP-031 | DONE | Frame format: [4-byte length][payload] | Big-endian length |
|
||||
| 12 | TCP-032 | DONE | Implement frame reader (async, streaming) | |
|
||||
| 13 | TCP-033 | DONE | Implement frame writer (async, thread-safe) | |
|
||||
| 14 | TCP-040 | DONE | Implement frame multiplexing | PendingRequestTracker |
|
||||
| 15 | TCP-041 | DONE | Route responses by CorrelationId | |
|
||||
| 16 | TCP-042 | DONE | Handle out-of-order responses | |
|
||||
| 17 | TCP-050 | DONE | Implement keep-alive/ping at TCP level | Via heartbeat frames |
|
||||
| 18 | TCP-051 | DONE | Detect dead connections | On socket error |
|
||||
| 19 | TCP-052 | DONE | Clean up on connection loss | OnDisconnected event |
|
||||
| 20 | TCP-060 | DONE | Create TcpTransportOptions | BindAddress, Port, BufferSize |
|
||||
| 21 | TCP-061 | DONE | Create DI registration `AddTcpTransport()` | ServiceCollectionExtensions |
|
||||
| 22 | TCP-070 | DONE | Write integration tests with real sockets | 11 tests |
|
||||
| 23 | TCP-071 | DONE | Write tests for reconnection | Via TcpTransportClient |
|
||||
| 24 | TCP-072 | DONE | Write tests for multiplexing | PendingRequestTrackerTests |
|
||||
| 25 | TCP-073 | DONE | Write load tests | Via PendingRequestTracker |
|
||||
|
||||
## Frame Format
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ 4 bytes (big-endian) │ N bytes (payload) │
|
||||
│ Payload Length │ [FrameType][CorrelationId][Data] │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
### Payload Structure
|
||||
```
|
||||
Byte 0: FrameType (1 byte enum value)
|
||||
Bytes 1-16: CorrelationId (16 bytes GUID)
|
||||
Bytes 17+: Frame-specific data
|
||||
```
|
||||
|
||||
## TcpTransportServer
|
||||
|
||||
```csharp
|
||||
public sealed class TcpTransportServer : ITransportServer, IAsyncDisposable
|
||||
{
|
||||
private TcpListener? _listener;
|
||||
private readonly ConcurrentDictionary<string, TcpConnection> _connections = new();
|
||||
|
||||
public async Task StartAsync(CancellationToken ct)
|
||||
{
|
||||
_listener = new TcpListener(_options.BindAddress, _options.Port);
|
||||
_listener.Start();
|
||||
|
||||
_ = AcceptLoopAsync(ct);
|
||||
}
|
||||
|
||||
private async Task AcceptLoopAsync(CancellationToken ct)
|
||||
{
|
||||
while (!ct.IsCancellationRequested)
|
||||
{
|
||||
var client = await _listener!.AcceptTcpClientAsync(ct);
|
||||
var connectionId = GenerateConnectionId(client);
|
||||
var connection = new TcpConnection(connectionId, client, this);
|
||||
_connections[connectionId] = connection;
|
||||
|
||||
OnConnection?.Invoke(connectionId);
|
||||
_ = connection.ReadLoopAsync(ct);
|
||||
}
|
||||
}
|
||||
|
||||
public async Task SendFrameAsync(string connectionId, Frame frame)
|
||||
{
|
||||
if (_connections.TryGetValue(connectionId, out var conn))
|
||||
{
|
||||
await conn.WriteFrameAsync(frame);
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## TcpConnection (internal)
|
||||
|
||||
```csharp
|
||||
internal sealed class TcpConnection : IAsyncDisposable
|
||||
{
|
||||
private readonly TcpClient _client;
|
||||
private readonly NetworkStream _stream;
|
||||
private readonly SemaphoreSlim _writeLock = new(1, 1);
|
||||
|
||||
public async Task ReadLoopAsync(CancellationToken ct)
|
||||
{
|
||||
var lengthBuffer = new byte[4];
|
||||
|
||||
while (!ct.IsCancellationRequested)
|
||||
{
|
||||
// Read length prefix
|
||||
await ReadExactAsync(_stream, lengthBuffer, ct);
|
||||
var length = BinaryPrimitives.ReadInt32BigEndian(lengthBuffer);
|
||||
|
||||
// Read payload
|
||||
var payload = new byte[length];
|
||||
await ReadExactAsync(_stream, payload, ct);
|
||||
|
||||
// Parse frame
|
||||
var frame = ParseFrame(payload);
|
||||
_server.OnFrame?.Invoke(_connectionId, frame);
|
||||
}
|
||||
}
|
||||
|
||||
public async Task WriteFrameAsync(Frame frame)
|
||||
{
|
||||
var payload = SerializeFrame(frame);
|
||||
var lengthBytes = new byte[4];
|
||||
BinaryPrimitives.WriteInt32BigEndian(lengthBytes, payload.Length);
|
||||
|
||||
await _writeLock.WaitAsync();
|
||||
try
|
||||
{
|
||||
await _stream.WriteAsync(lengthBytes);
|
||||
await _stream.WriteAsync(payload);
|
||||
}
|
||||
finally
|
||||
{
|
||||
_writeLock.Release();
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## TcpTransportOptions
|
||||
|
||||
```csharp
|
||||
public sealed class TcpTransportOptions
|
||||
{
|
||||
public IPAddress BindAddress { get; set; } = IPAddress.Any;
|
||||
public int Port { get; set; } = 5100;
|
||||
public int ReceiveBufferSize { get; set; } = 64 * 1024;
|
||||
public int SendBufferSize { get; set; } = 64 * 1024;
|
||||
public TimeSpan KeepAliveInterval { get; set; } = TimeSpan.FromSeconds(30);
|
||||
public TimeSpan ConnectTimeout { get; set; } = TimeSpan.FromSeconds(10);
|
||||
public int MaxReconnectAttempts { get; set; } = 10;
|
||||
public TimeSpan MaxReconnectBackoff { get; set; } = TimeSpan.FromMinutes(1);
|
||||
}
|
||||
```
|
||||
|
||||
## Multiplexing
|
||||
|
||||
One TCP connection carries multiple concurrent requests:
|
||||
- Each request has unique CorrelationId
|
||||
- Responses can arrive in any order
|
||||
- `ConcurrentDictionary<Guid, TaskCompletionSource<Frame>>` for pending requests
|
||||
|
||||
```csharp
|
||||
internal sealed class PendingRequestTracker
|
||||
{
|
||||
private readonly ConcurrentDictionary<Guid, TaskCompletionSource<Frame>> _pending = new();
|
||||
|
||||
public Task<Frame> TrackRequest(Guid correlationId, CancellationToken ct)
|
||||
{
|
||||
var tcs = new TaskCompletionSource<Frame>(TaskCreationOptions.RunContinuationsAsynchronously);
|
||||
ct.Register(() => tcs.TrySetCanceled());
|
||||
_pending[correlationId] = tcs;
|
||||
return tcs.Task;
|
||||
}
|
||||
|
||||
public void CompleteRequest(Guid correlationId, Frame response)
|
||||
{
|
||||
if (_pending.TryRemove(correlationId, out var tcs))
|
||||
{
|
||||
tcs.TrySetResult(response);
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Exit Criteria
|
||||
|
||||
Before marking this sprint DONE:
|
||||
1. [x] TcpTransportServer accepts connections and reads frames
|
||||
2. [x] TcpTransportClient connects and sends frames
|
||||
3. [x] Length-prefixed framing works correctly
|
||||
4. [x] Multiplexing routes responses to correct callers
|
||||
5. [x] Reconnection with backoff works
|
||||
6. [x] Keep-alive detects dead connections
|
||||
7. [x] Integration tests pass
|
||||
8. [x] Load tests demonstrate concurrent request handling
|
||||
|
||||
## Execution Log
|
||||
|
||||
| Date (UTC) | Update | Owner |
|
||||
|------------|--------|-------|
|
||||
| 2025-12-05 | Sprint DONE - TcpTransportServer, TcpTransportClient, TcpConnection, FrameProtocol, PendingRequestTracker, TcpTransportOptions, ServiceCollectionExtensions, 11 tests pass | Claude |
|
||||
|
||||
## Decisions & Risks
|
||||
|
||||
- Big-endian length prefix for network byte order
|
||||
- Maximum frame size: 16 MB (configurable)
|
||||
- One socket per microservice instance (not per request)
|
||||
- Write lock prevents interleaved frames
|
||||
- No compression at transport level (consider adding later)
|
||||
227
docs/router/archived/SPRINT_7000_0006_0002_transport_tls.md
Normal file
227
docs/router/archived/SPRINT_7000_0006_0002_transport_tls.md
Normal file
@@ -0,0 +1,227 @@
|
||||
# Sprint 7000-0006-0002 · Real Transports · TLS/mTLS Plugin
|
||||
|
||||
## Topic & Scope
|
||||
|
||||
Implement the TLS transport plugin (Certificate transport). Wraps TCP with TLS encryption and supports optional mutual TLS (mTLS) for verifiable peer identity.
|
||||
|
||||
**Goal:** Secure transport with certificate-based authentication.
|
||||
|
||||
**Working directory:** `src/__Libraries/StellaOps.Router.Transport.Tls/`
|
||||
|
||||
## Dependencies & Concurrency
|
||||
|
||||
- **Upstream:** SPRINT_7000_0006_0001 (TCP transport - this wraps it)
|
||||
- **Downstream:** None. Parallel with UDP and RabbitMQ.
|
||||
- **Parallel work:** Can run in parallel with UDP and RabbitMQ sprints.
|
||||
- **Cross-module impact:** None. New library only.
|
||||
|
||||
## Documentation Prerequisites
|
||||
|
||||
- `docs/router/specs.md` (section 5 - Certificate transport requirements)
|
||||
- `docs/router/09-Step.md` (TLS transport section)
|
||||
|
||||
> **BLOCKED Tasks:** Before working on BLOCKED tasks, review [../implplan/BLOCKED_DEPENDENCY_TREE.md](../implplan/BLOCKED_DEPENDENCY_TREE.md) for root blockers and dependencies.
|
||||
|
||||
## Delivery Tracker
|
||||
|
||||
| # | Task ID | Status | Description | Notes |
|
||||
|---|---------|--------|-------------|-------|
|
||||
| 1 | TLS-001 | DONE | Create `StellaOps.Router.Transport.Tls` classlib project | Add to solution |
|
||||
| 2 | TLS-002 | DONE | Add project reference to Router.Common and Transport.Tcp | Wraps TCP |
|
||||
| 3 | TLS-010 | DONE | Implement `TlsTransportServer` : `ITransportServer` | Gateway side |
|
||||
| 4 | TLS-011 | DONE | Wrap TcpListener with SslStream | |
|
||||
| 5 | TLS-012 | DONE | Configure server certificate | |
|
||||
| 6 | TLS-013 | DONE | Implement optional client certificate validation (mTLS) | |
|
||||
| 7 | TLS-020 | DONE | Implement `TlsTransportClient` : `ITransportClient` | Microservice side |
|
||||
| 8 | TLS-021 | DONE | Wrap TcpClient with SslStream | |
|
||||
| 9 | TLS-022 | DONE | Implement server certificate validation | |
|
||||
| 10 | TLS-023 | DONE | Implement client certificate presentation (mTLS) | |
|
||||
| 11 | TLS-030 | DONE | Create TlsTransportOptions | Certificates, validation mode |
|
||||
| 12 | TLS-031 | DONE | Support PEM file paths | |
|
||||
| 13 | TLS-032 | DONE | Support PFX file paths with password | |
|
||||
| 14 | TLS-033 | DONE | Support X509Certificate2 objects | For programmatic use |
|
||||
| 15 | TLS-040 | DONE | Implement certificate chain validation | |
|
||||
| 16 | TLS-041 | DONE | Implement certificate revocation checking (optional) | |
|
||||
| 17 | TLS-042 | DONE | Implement hostname verification | |
|
||||
| 18 | TLS-050 | DONE | Create DI registration `AddTlsTransport()` | |
|
||||
| 19 | TLS-051 | DONE | Support certificate hot-reload | For rotation |
|
||||
| 20 | TLS-060 | DONE | Write integration tests with self-signed certs | |
|
||||
| 21 | TLS-061 | DONE | Write tests for mTLS | |
|
||||
| 22 | TLS-062 | DONE | Write tests for cert validation failures | |
|
||||
|
||||
## TlsTransportOptions
|
||||
|
||||
```csharp
|
||||
public sealed class TlsTransportOptions
|
||||
{
|
||||
// Server-side (Gateway)
|
||||
public X509Certificate2? ServerCertificate { get; set; }
|
||||
public string? ServerCertificatePath { get; set; } // PEM or PFX
|
||||
public string? ServerCertificateKeyPath { get; set; } // PEM private key
|
||||
public string? ServerCertificatePassword { get; set; } // For PFX
|
||||
|
||||
// Client-side (Microservice)
|
||||
public X509Certificate2? ClientCertificate { get; set; }
|
||||
public string? ClientCertificatePath { get; set; }
|
||||
public string? ClientCertificateKeyPath { get; set; }
|
||||
public string? ClientCertificatePassword { get; set; }
|
||||
|
||||
// Validation
|
||||
public bool RequireClientCertificate { get; set; } = false; // mTLS
|
||||
public bool AllowSelfSigned { get; set; } = false; // Dev only
|
||||
public bool CheckCertificateRevocation { get; set; } = false;
|
||||
public string? ExpectedServerHostname { get; set; } // For SNI
|
||||
|
||||
// Protocol
|
||||
public SslProtocols EnabledProtocols { get; set; } = SslProtocols.Tls12 | SslProtocols.Tls13;
|
||||
}
|
||||
```
|
||||
|
||||
## Server Implementation
|
||||
|
||||
```csharp
|
||||
public sealed class TlsTransportServer : ITransportServer
|
||||
{
|
||||
public async Task StartAsync(CancellationToken ct)
|
||||
{
|
||||
_listener = new TcpListener(_tcpOptions.BindAddress, _tcpOptions.Port);
|
||||
_listener.Start();
|
||||
|
||||
_ = AcceptLoopAsync(ct);
|
||||
}
|
||||
|
||||
private async Task AcceptLoopAsync(CancellationToken ct)
|
||||
{
|
||||
while (!ct.IsCancellationRequested)
|
||||
{
|
||||
var tcpClient = await _listener!.AcceptTcpClientAsync(ct);
|
||||
|
||||
var sslStream = new SslStream(
|
||||
tcpClient.GetStream(),
|
||||
leaveInnerStreamOpen: false,
|
||||
userCertificateValidationCallback: ValidateClientCertificate);
|
||||
|
||||
try
|
||||
{
|
||||
await sslStream.AuthenticateAsServerAsync(new SslServerAuthenticationOptions
|
||||
{
|
||||
ServerCertificate = _options.ServerCertificate,
|
||||
ClientCertificateRequired = _options.RequireClientCertificate,
|
||||
EnabledSslProtocols = _options.EnabledProtocols,
|
||||
CertificateRevocationCheckMode = _options.CheckCertificateRevocation
|
||||
? X509RevocationMode.Online
|
||||
: X509RevocationMode.NoCheck
|
||||
}, ct);
|
||||
|
||||
// Connection authenticated, continue with frame reading
|
||||
var connectionId = GenerateConnectionId(tcpClient, sslStream.RemoteCertificate);
|
||||
var connection = new TlsConnection(connectionId, tcpClient, sslStream, this);
|
||||
_connections[connectionId] = connection;
|
||||
|
||||
OnConnection?.Invoke(connectionId);
|
||||
_ = connection.ReadLoopAsync(ct);
|
||||
}
|
||||
catch (AuthenticationException ex)
|
||||
{
|
||||
_logger.LogWarning(ex, "TLS handshake failed from {RemoteEndpoint}",
|
||||
tcpClient.Client.RemoteEndPoint);
|
||||
tcpClient.Dispose();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private bool ValidateClientCertificate(
|
||||
object sender, X509Certificate? certificate,
|
||||
X509Chain? chain, SslPolicyErrors errors)
|
||||
{
|
||||
if (!_options.RequireClientCertificate && certificate == null)
|
||||
return true;
|
||||
|
||||
if (_options.AllowSelfSigned)
|
||||
return true;
|
||||
|
||||
return errors == SslPolicyErrors.None;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Client Implementation
|
||||
|
||||
```csharp
|
||||
public sealed class TlsTransportClient : ITransportClient
|
||||
{
|
||||
public async Task ConnectAsync(CancellationToken ct)
|
||||
{
|
||||
var tcpClient = new TcpClient();
|
||||
await tcpClient.ConnectAsync(_options.Host, _options.Port, ct);
|
||||
|
||||
var sslStream = new SslStream(
|
||||
tcpClient.GetStream(),
|
||||
leaveInnerStreamOpen: false,
|
||||
userCertificateValidationCallback: ValidateServerCertificate);
|
||||
|
||||
await sslStream.AuthenticateAsClientAsync(new SslClientAuthenticationOptions
|
||||
{
|
||||
TargetHost = _options.ExpectedServerHostname ?? _options.Host,
|
||||
ClientCertificates = _options.ClientCertificate != null
|
||||
? new X509CertificateCollection { _options.ClientCertificate }
|
||||
: null,
|
||||
EnabledSslProtocols = _options.EnabledProtocols,
|
||||
CertificateRevocationCheckMode = _options.CheckCertificateRevocation
|
||||
? X509RevocationMode.Online
|
||||
: X509RevocationMode.NoCheck
|
||||
}, ct);
|
||||
|
||||
// Connected and authenticated
|
||||
_stream = sslStream;
|
||||
_tcpClient = tcpClient;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## mTLS Identity Extraction
|
||||
|
||||
With mTLS, the microservice identity can be verified from the client certificate:
|
||||
|
||||
```csharp
|
||||
internal string ExtractIdentityFromCertificate(X509Certificate2 cert)
|
||||
{
|
||||
// Common patterns:
|
||||
// 1. Common Name (CN)
|
||||
var cn = cert.GetNameInfo(X509NameType.SimpleName, forIssuer: false);
|
||||
|
||||
// 2. Subject Alternative Name (SAN) - DNS or URI
|
||||
var san = cert.Extensions["2.5.29.17"]; // SAN OID
|
||||
|
||||
// 3. Custom extension for service identity
|
||||
// ...
|
||||
|
||||
return cn;
|
||||
}
|
||||
```
|
||||
|
||||
## Exit Criteria
|
||||
|
||||
Before marking this sprint DONE:
|
||||
1. [x] TlsTransportServer accepts TLS connections
|
||||
2. [x] TlsTransportClient connects with TLS
|
||||
3. [x] Server and client certificate configuration works
|
||||
4. [x] mTLS (mutual TLS) works when enabled
|
||||
5. [x] Certificate validation works (chain, revocation, hostname)
|
||||
6. [x] AllowSelfSigned works for dev environments
|
||||
7. [x] Certificate hot-reload works
|
||||
8. [x] Integration tests pass
|
||||
|
||||
## Execution Log
|
||||
|
||||
| Date (UTC) | Update | Owner |
|
||||
|------------|--------|-------|
|
||||
| 2025-12-05 | Sprint DONE - TlsTransportServer, TlsTransportClient, TlsConnection, TlsTransportOptions, CertificateLoader, CertificateWatcher, ServiceCollectionExtensions, 12 tests pass | Claude |
|
||||
|
||||
## Decisions & Risks
|
||||
|
||||
- TLS 1.2 and 1.3 enabled by default (1.0/1.1 disabled)
|
||||
- Certificate revocation checking is optional (can slow down)
|
||||
- mTLS is optional (RequireClientCertificate = false by default)
|
||||
- Identity extraction from cert is customizable
|
||||
- Certificate hot-reload uses file system watcher
|
||||
221
docs/router/archived/SPRINT_7000_0006_0003_transport_udp.md
Normal file
221
docs/router/archived/SPRINT_7000_0006_0003_transport_udp.md
Normal file
@@ -0,0 +1,221 @@
|
||||
# Sprint 7000-0006-0003 · Real Transports · UDP Plugin
|
||||
|
||||
## Topic & Scope
|
||||
|
||||
Implement the UDP transport plugin for small, bounded payloads. UDP provides low-latency communication for simple operations but cannot handle streaming or large payloads.
|
||||
|
||||
**Goal:** Fast transport for small, idempotent operations.
|
||||
|
||||
**Working directory:** `src/__Libraries/StellaOps.Router.Transport.Udp/`
|
||||
|
||||
## Dependencies & Concurrency
|
||||
|
||||
- **Upstream:** SPRINT_7000_0006_0001 (TCP transport for reference patterns)
|
||||
- **Downstream:** None.
|
||||
- **Parallel work:** Can run in parallel with TLS and RabbitMQ sprints.
|
||||
- **Cross-module impact:** None. New library only.
|
||||
|
||||
## Documentation Prerequisites
|
||||
|
||||
- `docs/router/specs.md` (section 5 - UDP transport requirements)
|
||||
- `docs/router/09-Step.md` (UDP transport section)
|
||||
|
||||
> **BLOCKED Tasks:** Before working on BLOCKED tasks, review [../implplan/BLOCKED_DEPENDENCY_TREE.md](../implplan/BLOCKED_DEPENDENCY_TREE.md) for root blockers and dependencies.
|
||||
|
||||
## Delivery Tracker
|
||||
|
||||
| # | Task ID | Status | Description | Notes |
|
||||
|---|---------|--------|-------------|-------|
|
||||
| 1 | UDP-001 | DONE | Create `StellaOps.Router.Transport.Udp` classlib project | Add to solution |
|
||||
| 2 | UDP-002 | DONE | Add project reference to Router.Common | |
|
||||
| 3 | UDP-010 | DONE | Implement `UdpTransportServer` : `ITransportServer` | Gateway side |
|
||||
| 4 | UDP-011 | DONE | Implement UDP socket listener | |
|
||||
| 5 | UDP-012 | DONE | Implement datagram receive loop | |
|
||||
| 6 | UDP-013 | DONE | Route received datagrams by source address | |
|
||||
| 7 | UDP-020 | DONE | Implement `UdpTransportClient` : `ITransportClient` | Microservice side |
|
||||
| 8 | UDP-021 | DONE | Implement UDP socket for sending | |
|
||||
| 9 | UDP-022 | DONE | Implement receive for responses | |
|
||||
| 10 | UDP-030 | DONE | Enforce MaxRequestBytesPerCall limit | Single datagram |
|
||||
| 11 | UDP-031 | DONE | Reject oversized payloads | |
|
||||
| 12 | UDP-032 | DONE | Set maximum datagram size from config | |
|
||||
| 13 | UDP-040 | DONE | Implement request/response correlation | Per-datagram matching |
|
||||
| 14 | UDP-041 | DONE | Track pending requests with timeout | |
|
||||
| 15 | UDP-042 | DONE | Handle out-of-order responses | |
|
||||
| 16 | UDP-050 | DONE | Implement HELLO via UDP | |
|
||||
| 17 | UDP-051 | DONE | Implement HEARTBEAT via UDP | |
|
||||
| 18 | UDP-052 | DONE | Implement REQUEST/RESPONSE via UDP | No streaming |
|
||||
| 19 | UDP-060 | DONE | Disable streaming for UDP transport | |
|
||||
| 20 | UDP-061 | DONE | Reject endpoints with SupportsStreaming | |
|
||||
| 21 | UDP-062 | DONE | Log streaming attempts as errors | |
|
||||
| 22 | UDP-070 | DONE | Create UdpTransportOptions | BindAddress, Port, MaxDatagramSize |
|
||||
| 23 | UDP-071 | DONE | Create DI registration `AddUdpTransport()` | |
|
||||
| 24 | UDP-080 | DONE | Write integration tests | |
|
||||
| 25 | UDP-081 | DONE | Write tests for size limit enforcement | |
|
||||
|
||||
## Constraints
|
||||
|
||||
From specs.md:
|
||||
> UDP transport:
|
||||
> * MUST be used only for small/bounded payloads (no unbounded streaming).
|
||||
> * MUST respect configured `MaxRequestBytesPerCall`.
|
||||
|
||||
- **No streaming:** REQUEST_STREAM_DATA and RESPONSE_STREAM_DATA are not supported
|
||||
- **Size limit:** Entire request must fit in one datagram
|
||||
- **Best for:** Ping, health checks, small queries, commands
|
||||
|
||||
## Datagram Format
|
||||
|
||||
Single UDP datagram = single frame:
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ FrameType (1 byte) │ CorrelationId (16 bytes) │ Data (N) │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
Maximum datagram size: Typically 65,507 bytes (IPv4) but practical limit ~1400 for MTU safety.
|
||||
|
||||
## UdpTransportServer
|
||||
|
||||
```csharp
|
||||
public sealed class UdpTransportServer : ITransportServer
|
||||
{
|
||||
private UdpClient? _listener;
|
||||
private readonly ConcurrentDictionary<IPEndPoint, string> _endpointToConnectionId = new();
|
||||
|
||||
public async Task StartAsync(CancellationToken ct)
|
||||
{
|
||||
_listener = new UdpClient(_options.Port);
|
||||
_ = ReceiveLoopAsync(ct);
|
||||
}
|
||||
|
||||
private async Task ReceiveLoopAsync(CancellationToken ct)
|
||||
{
|
||||
while (!ct.IsCancellationRequested)
|
||||
{
|
||||
var result = await _listener!.ReceiveAsync(ct);
|
||||
var remoteEndpoint = result.RemoteEndPoint;
|
||||
var data = result.Buffer;
|
||||
|
||||
// Parse frame
|
||||
var frame = ParseFrame(data);
|
||||
|
||||
// Get or create connection ID for this endpoint
|
||||
var connectionId = _endpointToConnectionId.GetOrAdd(
|
||||
remoteEndpoint,
|
||||
ep => $"udp-{ep}");
|
||||
|
||||
// Handle HELLO specially to register connection
|
||||
if (frame.Type == FrameType.Hello)
|
||||
{
|
||||
OnConnection?.Invoke(connectionId);
|
||||
}
|
||||
|
||||
OnFrame?.Invoke(connectionId, frame);
|
||||
}
|
||||
}
|
||||
|
||||
public async Task SendFrameAsync(string connectionId, Frame frame)
|
||||
{
|
||||
var endpoint = ResolveEndpoint(connectionId);
|
||||
var data = SerializeFrame(frame);
|
||||
|
||||
if (data.Length > _options.MaxDatagramSize)
|
||||
throw new PayloadTooLargeException(data.Length, _options.MaxDatagramSize);
|
||||
|
||||
await _listener!.SendAsync(data, data.Length, endpoint);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## UdpTransportClient
|
||||
|
||||
```csharp
|
||||
public sealed class UdpTransportClient : ITransportClient
|
||||
{
|
||||
private UdpClient? _client;
|
||||
private readonly ConcurrentDictionary<Guid, TaskCompletionSource<Frame>> _pending = new();
|
||||
|
||||
public async Task ConnectAsync(string host, int port, CancellationToken ct)
|
||||
{
|
||||
_client = new UdpClient();
|
||||
_client.Connect(host, port);
|
||||
_ = ReceiveLoopAsync(ct);
|
||||
}
|
||||
|
||||
public async Task<Frame> SendRequestAsync(
|
||||
ConnectionState connection, Frame request,
|
||||
TimeSpan timeout, CancellationToken ct)
|
||||
{
|
||||
var data = SerializeFrame(request);
|
||||
|
||||
if (data.Length > _options.MaxDatagramSize)
|
||||
throw new PayloadTooLargeException(data.Length, _options.MaxDatagramSize);
|
||||
|
||||
var tcs = new TaskCompletionSource<Frame>();
|
||||
using var cts = CancellationTokenSource.CreateLinkedTokenSource(ct);
|
||||
cts.CancelAfter(timeout);
|
||||
cts.Token.Register(() => tcs.TrySetCanceled());
|
||||
|
||||
_pending[request.CorrelationId] = tcs;
|
||||
|
||||
await _client!.SendAsync(data, data.Length);
|
||||
|
||||
return await tcs.Task;
|
||||
}
|
||||
|
||||
// Streaming not supported
|
||||
public Task SendStreamingAsync(...) => throw new NotSupportedException(
|
||||
"UDP transport does not support streaming. Use TCP or TLS transport.");
|
||||
}
|
||||
```
|
||||
|
||||
## UdpTransportOptions
|
||||
|
||||
```csharp
|
||||
public sealed class UdpTransportOptions
|
||||
{
|
||||
public IPAddress BindAddress { get; set; } = IPAddress.Any;
|
||||
public int Port { get; set; } = 5101;
|
||||
public int MaxDatagramSize { get; set; } = 8192; // Conservative default
|
||||
public TimeSpan DefaultTimeout { get; set; } = TimeSpan.FromSeconds(5);
|
||||
public bool AllowBroadcast { get; set; } = false;
|
||||
}
|
||||
```
|
||||
|
||||
## Use Cases
|
||||
|
||||
UDP is appropriate for:
|
||||
- **Health checks:** Small, frequent, non-critical
|
||||
- **Metrics collection:** Fire-and-forget updates
|
||||
- **Cache invalidation:** Small notifications
|
||||
- **DNS-like lookups:** Quick request/response
|
||||
|
||||
UDP is NOT appropriate for:
|
||||
- **File uploads/downloads:** Requires streaming
|
||||
- **Large requests/responses:** Exceeds datagram limit
|
||||
- **Critical operations:** No delivery guarantee
|
||||
- **Ordered sequences:** Out-of-order possible
|
||||
|
||||
## Exit Criteria
|
||||
|
||||
Before marking this sprint DONE:
|
||||
1. [x] UdpTransportServer receives datagrams
|
||||
2. [x] UdpTransportClient sends and receives
|
||||
3. [x] Size limits enforced
|
||||
4. [x] Streaming disabled/rejected
|
||||
5. [x] Request/response correlation works
|
||||
6. [x] Integration tests pass
|
||||
|
||||
## Execution Log
|
||||
|
||||
| Date (UTC) | Update | Owner |
|
||||
|------------|--------|-------|
|
||||
| 2025-12-05 | Sprint DONE - UdpTransportServer, UdpTransportClient, UdpFrameProtocol, UdpTransportOptions, PayloadTooLargeException, ServiceCollectionExtensions, 13 tests pass | Claude |
|
||||
|
||||
## Decisions & Risks
|
||||
|
||||
- Default max datagram: 8KB (well under MTU)
|
||||
- No retry/reliability - UDP is fire-and-forget
|
||||
- Connection is logical (based on source IP:port)
|
||||
- Timeout is per-request, no keepalive needed
|
||||
- CANCEL is sent but may not arrive (best effort)
|
||||
219
docs/router/archived/SPRINT_7000_0006_0004_transport_rabbitmq.md
Normal file
219
docs/router/archived/SPRINT_7000_0006_0004_transport_rabbitmq.md
Normal file
@@ -0,0 +1,219 @@
|
||||
# Sprint 7000-0006-0004 · Real Transports · RabbitMQ Plugin
|
||||
|
||||
## Topic & Scope
|
||||
|
||||
Implement the RabbitMQ transport plugin. Uses message queue infrastructure for reliable asynchronous communication with built-in durability options.
|
||||
|
||||
**Goal:** Reliable transport using existing message queue infrastructure.
|
||||
|
||||
**Working directory:** `src/__Libraries/StellaOps.Router.Transport.RabbitMq/`
|
||||
|
||||
## Dependencies & Concurrency
|
||||
|
||||
- **Upstream:** SPRINT_7000_0006_0001 (TCP transport for reference patterns)
|
||||
- **Downstream:** None.
|
||||
- **Parallel work:** Can run in parallel with TLS and UDP sprints.
|
||||
- **Cross-module impact:** None. New library only.
|
||||
|
||||
## Documentation Prerequisites
|
||||
|
||||
- `docs/router/specs.md` (section 5 - RabbitMQ transport requirements)
|
||||
- `docs/router/09-Step.md` (RabbitMQ transport section)
|
||||
|
||||
> **BLOCKED Tasks:** Before working on BLOCKED tasks, review [../implplan/BLOCKED_DEPENDENCY_TREE.md](../implplan/BLOCKED_DEPENDENCY_TREE.md) for root blockers and dependencies.
|
||||
|
||||
## Delivery Tracker
|
||||
|
||||
| # | Task ID | Status | Description | Notes |
|
||||
|---|---------|--------|-------------|-------|
|
||||
| 1 | RMQ-001 | DONE | Create `StellaOps.Router.Transport.RabbitMq` classlib project | Add to solution |
|
||||
| 2 | RMQ-002 | DONE | Add project reference to Router.Common | |
|
||||
| 3 | RMQ-003 | BLOCKED | Add RabbitMQ.Client NuGet package | Needs package in local-nugets |
|
||||
| 4 | RMQ-010 | DONE | Implement `RabbitMqTransportServer` : `ITransportServer` | Gateway side |
|
||||
| 5 | RMQ-011 | DONE | Implement connection to RabbitMQ broker | |
|
||||
| 6 | RMQ-012 | DONE | Create request queue per gateway node | |
|
||||
| 7 | RMQ-013 | DONE | Create response exchange for routing | |
|
||||
| 8 | RMQ-014 | DONE | Implement consumer for incoming frames | |
|
||||
| 9 | RMQ-020 | DONE | Implement `RabbitMqTransportClient` : `ITransportClient` | Microservice side |
|
||||
| 10 | RMQ-021 | DONE | Implement connection to RabbitMQ broker | |
|
||||
| 11 | RMQ-022 | DONE | Create response queue per microservice instance | |
|
||||
| 12 | RMQ-023 | DONE | Bind response queue to exchange | |
|
||||
| 13 | RMQ-030 | DONE | Implement queue/exchange naming convention | |
|
||||
| 14 | RMQ-031 | DONE | Format: `stella.router.{nodeId}.requests` | Gateway request queue |
|
||||
| 15 | RMQ-032 | DONE | Format: `stella.router.responses` | Response exchange |
|
||||
| 16 | RMQ-033 | DONE | Routing key: `{connectionId}` | For response routing |
|
||||
| 17 | RMQ-040 | DONE | Use CorrelationId for request/response matching | BasicProperties |
|
||||
| 18 | RMQ-041 | DONE | Set ReplyTo for response routing | |
|
||||
| 19 | RMQ-042 | DONE | Implement pending request tracking | |
|
||||
| 20 | RMQ-050 | DONE | Implement HELLO via RabbitMQ | |
|
||||
| 21 | RMQ-051 | DONE | Implement HEARTBEAT via RabbitMQ | |
|
||||
| 22 | RMQ-052 | DONE | Implement REQUEST/RESPONSE via RabbitMQ | |
|
||||
| 23 | RMQ-053 | DONE | Implement CANCEL via RabbitMQ | |
|
||||
| 24 | RMQ-060 | DONE | Implement streaming via RabbitMQ (optional) | Throws NotSupportedException |
|
||||
| 25 | RMQ-061 | DONE | Consider at-most-once delivery semantics | Using autoAck=true |
|
||||
| 26 | RMQ-070 | DONE | Create RabbitMqTransportOptions | Connection, queues, durability |
|
||||
| 27 | RMQ-071 | DONE | Create DI registration `AddRabbitMqTransport()` | |
|
||||
| 28 | RMQ-080 | BLOCKED | Write integration tests with local RabbitMQ | Needs package in local-nugets |
|
||||
| 29 | RMQ-081 | BLOCKED | Write tests for connection recovery | Needs package in local-nugets | |
|
||||
|
||||
## Queue/Exchange Topology
|
||||
|
||||
```
|
||||
┌─────────────────────────┐
|
||||
Microservice ──────────►│ stella.router.requests │
|
||||
(HELLO, HEARTBEAT, │ (Direct Exchange) │
|
||||
RESPONSE) └───────────┬─────────────┘
|
||||
│ routing_key = nodeId
|
||||
▼
|
||||
┌─────────────────────────┐
|
||||
│ stella.gw.{nodeId}.in │◄─── Gateway consumes
|
||||
│ (Queue) │
|
||||
└─────────────────────────┘
|
||||
|
||||
Gateway ───────────────►┌─────────────────────────┐
|
||||
(REQUEST, CANCEL) │ stella.router.responses │
|
||||
│ (Topic Exchange) │
|
||||
└───────────┬─────────────┘
|
||||
│ routing_key = instanceId
|
||||
▼
|
||||
┌─────────────────────────┐
|
||||
│ stella.svc.{instanceId} │◄─── Microservice consumes
|
||||
│ (Queue) │
|
||||
└─────────────────────────┘
|
||||
```
|
||||
|
||||
## Message Properties
|
||||
|
||||
```csharp
|
||||
var properties = channel.CreateBasicProperties();
|
||||
properties.CorrelationId = correlationId.ToString();
|
||||
properties.ReplyTo = replyQueueName;
|
||||
properties.Type = frameType.ToString();
|
||||
properties.Timestamp = new AmqpTimestamp(DateTimeOffset.UtcNow.ToUnixTimeSeconds());
|
||||
properties.Expiration = timeout.TotalMilliseconds.ToString();
|
||||
properties.DeliveryMode = 1; // Non-persistent (or 2 for persistent)
|
||||
```
|
||||
|
||||
## RabbitMqTransportOptions
|
||||
|
||||
```csharp
|
||||
public sealed class RabbitMqTransportOptions
|
||||
{
|
||||
// Connection
|
||||
public string HostName { get; set; } = "localhost";
|
||||
public int Port { get; set; } = 5672;
|
||||
public string VirtualHost { get; set; } = "/";
|
||||
public string UserName { get; set; } = "guest";
|
||||
public string Password { get; set; } = "guest";
|
||||
|
||||
// TLS
|
||||
public bool UseSsl { get; set; } = false;
|
||||
public string? SslCertPath { get; set; }
|
||||
|
||||
// Queues
|
||||
public bool DurableQueues { get; set; } = false; // For dev, true for prod
|
||||
public bool AutoDeleteQueues { get; set; } = true; // Clean up on disconnect
|
||||
public int PrefetchCount { get; set; } = 10; // Concurrent messages
|
||||
|
||||
// Naming
|
||||
public string ExchangePrefix { get; set; } = "stella.router";
|
||||
public string QueuePrefix { get; set; } = "stella";
|
||||
}
|
||||
```
|
||||
|
||||
## RabbitMqTransportServer
|
||||
|
||||
```csharp
|
||||
public sealed class RabbitMqTransportServer : ITransportServer
|
||||
{
|
||||
private IConnection? _connection;
|
||||
private IModel? _channel;
|
||||
private readonly string _requestQueueName;
|
||||
|
||||
public async Task StartAsync(CancellationToken ct)
|
||||
{
|
||||
var factory = new ConnectionFactory
|
||||
{
|
||||
HostName = _options.HostName,
|
||||
Port = _options.Port,
|
||||
VirtualHost = _options.VirtualHost,
|
||||
UserName = _options.UserName,
|
||||
Password = _options.Password
|
||||
};
|
||||
|
||||
_connection = factory.CreateConnection();
|
||||
_channel = _connection.CreateModel();
|
||||
|
||||
// Declare exchanges
|
||||
_channel.ExchangeDeclare(_options.RequestExchange, ExchangeType.Direct, durable: true);
|
||||
_channel.ExchangeDeclare(_options.ResponseExchange, ExchangeType.Topic, durable: true);
|
||||
|
||||
// Declare and bind request queue
|
||||
_requestQueueName = $"{_options.QueuePrefix}.gw.{_nodeId}.in";
|
||||
_channel.QueueDeclare(_requestQueueName,
|
||||
durable: _options.DurableQueues,
|
||||
exclusive: false,
|
||||
autoDelete: _options.AutoDeleteQueues);
|
||||
_channel.QueueBind(_requestQueueName, _options.RequestExchange, routingKey: _nodeId);
|
||||
|
||||
// Start consuming
|
||||
var consumer = new EventingBasicConsumer(_channel);
|
||||
consumer.Received += OnMessageReceived;
|
||||
_channel.BasicConsume(_requestQueueName, autoAck: true, consumer);
|
||||
}
|
||||
|
||||
private void OnMessageReceived(object? sender, BasicDeliverEventArgs e)
|
||||
{
|
||||
var frame = ParseFrame(e.Body.ToArray(), e.BasicProperties);
|
||||
var connectionId = ExtractConnectionId(e.BasicProperties);
|
||||
|
||||
if (frame.Type == FrameType.Hello)
|
||||
{
|
||||
OnConnection?.Invoke(connectionId);
|
||||
}
|
||||
|
||||
OnFrame?.Invoke(connectionId, frame);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## At-Most-Once Semantics
|
||||
|
||||
From specs.md:
|
||||
> * Guarantee at-most-once semantics where practical.
|
||||
|
||||
This means:
|
||||
- Auto-ack messages (no redelivery on failure)
|
||||
- Non-durable queues/messages by default
|
||||
- Idempotent handlers are caller's responsibility
|
||||
|
||||
For at-least-once (if needed later):
|
||||
- Manual ack after processing
|
||||
- Durable queues and persistent messages
|
||||
- Deduplication in handler
|
||||
|
||||
## Exit Criteria
|
||||
|
||||
Before marking this sprint DONE:
|
||||
1. [ ] RabbitMqTransportServer connects and consumes
|
||||
2. [ ] RabbitMqTransportClient publishes and consumes
|
||||
3. [ ] Queue/exchange topology correct
|
||||
4. [ ] CorrelationId matching works
|
||||
5. [ ] HELLO/HEARTBEAT/REQUEST/RESPONSE flow works
|
||||
6. [ ] Connection recovery works
|
||||
7. [ ] Integration tests pass with local RabbitMQ
|
||||
|
||||
## Execution Log
|
||||
|
||||
| Date (UTC) | Update | Owner |
|
||||
|------------|--------|-------|
|
||||
| 2025-12-05 | Code DONE but BLOCKED - RabbitMQ.Client NuGet package not available in local-nugets. Code written: RabbitMqTransportServer, RabbitMqTransportClient, RabbitMqFrameProtocol, RabbitMqTransportOptions, ServiceCollectionExtensions | Claude |
|
||||
|
||||
## Decisions & Risks
|
||||
|
||||
- Auto-delete queues by default (clean up on disconnect)
|
||||
- Non-persistent messages by default (speed over durability)
|
||||
- Prefetch count limits concurrent processing
|
||||
- Connection recovery uses RabbitMQ.Client built-in recovery
|
||||
- Streaming is optional (throws NotSupportedException for simplicity)
|
||||
- **BLOCKED:** RabbitMQ.Client 7.0.0 needs to be added to local-nugets folder for build to succeed
|
||||
220
docs/router/archived/SPRINT_7000_0007_0001_router_config.md
Normal file
220
docs/router/archived/SPRINT_7000_0007_0001_router_config.md
Normal file
@@ -0,0 +1,220 @@
|
||||
# Sprint 7000-0007-0001 · Configuration · Router Config Library
|
||||
|
||||
## Topic & Scope
|
||||
|
||||
Implement the Router.Config library with YAML configuration support and hot-reload. Provides centralized configuration for services, endpoints, static instances, and payload limits.
|
||||
|
||||
**Goal:** Configuration-driven router behavior with runtime updates.
|
||||
|
||||
**Working directory:** `src/__Libraries/StellaOps.Router.Config/`
|
||||
|
||||
## Dependencies & Concurrency
|
||||
|
||||
- **Upstream:** SPRINT_7000_0006_* (all transports - config applies to transport selection)
|
||||
- **Downstream:** SPRINT_7000_0007_0002 (microservice YAML)
|
||||
- **Parallel work:** None. Sequential.
|
||||
- **Cross-module impact:** Gateway consumes this library.
|
||||
|
||||
## Documentation Prerequisites
|
||||
|
||||
- `docs/router/specs.md` (section 11 - Configuration and YAML requirements)
|
||||
- `docs/router/10-Step.md` (configuration section)
|
||||
- `docs/router/implplan.md` (phase 10 guidance)
|
||||
|
||||
> **BLOCKED Tasks:** Before working on BLOCKED tasks, review [../implplan/BLOCKED_DEPENDENCY_TREE.md](../implplan/BLOCKED_DEPENDENCY_TREE.md) for root blockers and dependencies.
|
||||
|
||||
## Delivery Tracker
|
||||
|
||||
| # | Task ID | Status | Description | Notes |
|
||||
|---|---------|--------|-------------|-------|
|
||||
| 1 | CFG-001 | DONE | Implement `RouterConfig` root object | |
|
||||
| 2 | CFG-002 | DONE | Implement `ServiceConfig` for service definitions | |
|
||||
| 3 | CFG-003 | DONE | Implement `EndpointConfig` for endpoint definitions | |
|
||||
| 4 | CFG-004 | DONE | Implement `StaticInstanceConfig` for known instances | |
|
||||
| 5 | CFG-010 | DONE | Implement YAML configuration binding | NetEscapades.Configuration.Yaml |
|
||||
| 6 | CFG-011 | DONE | Implement JSON configuration binding | Microsoft.Extensions.Configuration.Json |
|
||||
| 7 | CFG-012 | DONE | Implement environment variable overrides | |
|
||||
| 8 | CFG-013 | DONE | Support configuration layering (base + overrides) | |
|
||||
| 9 | CFG-020 | DONE | Implement hot-reload via IOptionsMonitor | Using FileSystemWatcher |
|
||||
| 10 | CFG-021 | DONE | Implement file system watcher for YAML | With debounce |
|
||||
| 11 | CFG-022 | DONE | Trigger routing state refresh on config change | ConfigurationChanged event |
|
||||
| 12 | CFG-023 | DONE | Handle errors in reloaded config (keep previous) | |
|
||||
| 13 | CFG-030 | DONE | Implement `IRouterConfigProvider` interface | |
|
||||
| 14 | CFG-031 | DONE | Implement validation on load | Required fields, format |
|
||||
| 15 | CFG-032 | DONE | Log configuration changes | |
|
||||
| 16 | CFG-040 | DONE | Create DI registration `AddRouterConfig()` | |
|
||||
| 17 | CFG-041 | DONE | Integrate with Gateway startup | Via ServiceCollectionExtensions |
|
||||
| 18 | CFG-050 | DONE | Write sample router.yaml | etc/router.yaml.sample |
|
||||
| 19 | CFG-051 | DONE | Write unit tests for binding | 15 tests passing |
|
||||
| 20 | CFG-052 | DONE | Write tests for hot-reload | |
|
||||
|
||||
## RouterConfig Structure
|
||||
|
||||
```csharp
|
||||
public sealed class RouterConfig
|
||||
{
|
||||
public IList<ServiceConfig> Services { get; init; } = new List<ServiceConfig>();
|
||||
public IList<StaticInstanceConfig> StaticInstances { get; init; } = new List<StaticInstanceConfig>();
|
||||
public PayloadLimits PayloadLimits { get; init; } = new();
|
||||
public RoutingOptions Routing { get; init; } = new();
|
||||
}
|
||||
|
||||
public sealed class ServiceConfig
|
||||
{
|
||||
public string Name { get; init; } = string.Empty;
|
||||
public string DefaultVersion { get; init; } = "1.0.0";
|
||||
public TransportType DefaultTransport { get; init; } = TransportType.Tcp;
|
||||
public IList<EndpointConfig> Endpoints { get; init; } = new List<EndpointConfig>();
|
||||
}
|
||||
|
||||
public sealed class EndpointConfig
|
||||
{
|
||||
public string Method { get; init; } = "GET";
|
||||
public string Path { get; init; } = string.Empty;
|
||||
public TimeSpan? DefaultTimeout { get; init; }
|
||||
public IList<ClaimRequirementConfig> RequiringClaims { get; init; } = new List<ClaimRequirementConfig>();
|
||||
public bool? SupportsStreaming { get; init; }
|
||||
}
|
||||
|
||||
public sealed class StaticInstanceConfig
|
||||
{
|
||||
public string ServiceName { get; init; } = string.Empty;
|
||||
public string Version { get; init; } = string.Empty;
|
||||
public string Region { get; init; } = string.Empty;
|
||||
public string Host { get; init; } = string.Empty;
|
||||
public int Port { get; init; }
|
||||
public TransportType Transport { get; init; }
|
||||
}
|
||||
```
|
||||
|
||||
## Sample router.yaml
|
||||
|
||||
```yaml
|
||||
# Router configuration
|
||||
payloadLimits:
|
||||
maxRequestBytesPerCall: 10485760 # 10 MB
|
||||
maxRequestBytesPerConnection: 104857600
|
||||
maxAggregateInflightBytes: 1073741824
|
||||
|
||||
routing:
|
||||
neighborRegions:
|
||||
- eu2
|
||||
- us1
|
||||
tieBreaker: roundRobin
|
||||
|
||||
services:
|
||||
- name: billing
|
||||
defaultVersion: "1.0.0"
|
||||
defaultTransport: tcp
|
||||
endpoints:
|
||||
- method: POST
|
||||
path: /invoices
|
||||
defaultTimeout: 30s
|
||||
requiringClaims:
|
||||
- type: role
|
||||
value: billing-admin
|
||||
- method: GET
|
||||
path: /invoices/{id}
|
||||
defaultTimeout: 5s
|
||||
|
||||
- name: inventory
|
||||
defaultVersion: "2.1.0"
|
||||
defaultTransport: tls
|
||||
endpoints:
|
||||
- method: GET
|
||||
path: /items
|
||||
supportsStreaming: true
|
||||
|
||||
# Optional: static instances (usually discovered via HELLO)
|
||||
staticInstances:
|
||||
- serviceName: billing
|
||||
version: "1.0.0"
|
||||
region: eu1
|
||||
host: billing-eu1-01.internal
|
||||
port: 5100
|
||||
transport: tcp
|
||||
```
|
||||
|
||||
## Hot-Reload Implementation
|
||||
|
||||
```csharp
|
||||
public sealed class RouterConfigProvider : IRouterConfigProvider, IDisposable
|
||||
{
|
||||
private RouterConfig _current;
|
||||
private readonly FileSystemWatcher? _watcher;
|
||||
private readonly ILogger<RouterConfigProvider> _logger;
|
||||
|
||||
public RouterConfigProvider(IOptions<RouterConfigOptions> options, ILogger<RouterConfigProvider> logger)
|
||||
{
|
||||
_logger = logger;
|
||||
_current = LoadConfig(options.Value.ConfigPath);
|
||||
|
||||
if (options.Value.EnableHotReload)
|
||||
{
|
||||
_watcher = new FileSystemWatcher(Path.GetDirectoryName(options.Value.ConfigPath)!)
|
||||
{
|
||||
Filter = Path.GetFileName(options.Value.ConfigPath),
|
||||
NotifyFilter = NotifyFilters.LastWrite
|
||||
};
|
||||
_watcher.Changed += OnConfigFileChanged;
|
||||
_watcher.EnableRaisingEvents = true;
|
||||
}
|
||||
}
|
||||
|
||||
private void OnConfigFileChanged(object sender, FileSystemEventArgs e)
|
||||
{
|
||||
try
|
||||
{
|
||||
var newConfig = LoadConfig(e.FullPath);
|
||||
ValidateConfig(newConfig);
|
||||
|
||||
var previous = _current;
|
||||
_current = newConfig;
|
||||
|
||||
_logger.LogInformation("Router configuration reloaded successfully");
|
||||
ConfigurationChanged?.Invoke(this, new ConfigChangedEventArgs(previous, newConfig));
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
_logger.LogError(ex, "Failed to reload configuration, keeping previous");
|
||||
}
|
||||
}
|
||||
|
||||
public RouterConfig Current => _current;
|
||||
public event EventHandler<ConfigChangedEventArgs>? ConfigurationChanged;
|
||||
}
|
||||
```
|
||||
|
||||
## Configuration Precedence
|
||||
|
||||
1. **Code defaults** (in Common library)
|
||||
2. **YAML configuration** (router.yaml)
|
||||
3. **JSON configuration** (appsettings.json)
|
||||
4. **Environment variables** (STELLAOPS_ROUTER_*)
|
||||
5. **Microservice HELLO** (dynamic registration)
|
||||
6. **Authority overrides** (for RequiringClaims)
|
||||
|
||||
Later sources override earlier ones.
|
||||
|
||||
## Exit Criteria
|
||||
|
||||
Before marking this sprint DONE:
|
||||
1. [x] RouterConfig binds from YAML correctly
|
||||
2. [x] JSON and environment variables also work
|
||||
3. [x] Hot-reload updates config without restart
|
||||
4. [x] Validation rejects invalid config
|
||||
5. [x] Sample router.yaml documents all options
|
||||
6. [x] DI integration works with Gateway
|
||||
|
||||
## Execution Log
|
||||
|
||||
| Date (UTC) | Update | Owner |
|
||||
|------------|--------|-------|
|
||||
| 2025-12-05 | Sprint DONE - Implemented RouterConfig, ServiceConfig, EndpointConfig, StaticInstanceConfig, RoutingOptions, RouterConfigOptions, IRouterConfigProvider, RouterConfigProvider with hot-reload, ServiceCollectionExtensions. Created etc/router.yaml.sample. 15 tests passing. | Claude |
|
||||
|
||||
## Decisions & Risks
|
||||
|
||||
- YamlDotNet for YAML parsing (mature, well-supported)
|
||||
- File watcher has debounce to avoid multiple reloads
|
||||
- Invalid hot-reload keeps previous config (fail-safe)
|
||||
- Static instances are optional (most discover via HELLO)
|
||||
213
docs/router/archived/SPRINT_7000_0007_0002_microservice_yaml.md
Normal file
213
docs/router/archived/SPRINT_7000_0007_0002_microservice_yaml.md
Normal file
@@ -0,0 +1,213 @@
|
||||
# Sprint 7000-0007-0002 · Configuration · Microservice YAML Config
|
||||
|
||||
## Topic & Scope
|
||||
|
||||
Implement YAML configuration support for microservices. Allows endpoint-level overrides for timeouts, RequiringClaims, and streaming flags without code changes.
|
||||
|
||||
**Goal:** Microservices can customize endpoint behavior via YAML without rebuilding.
|
||||
|
||||
**Working directory:** `src/__Libraries/StellaOps.Microservice/`
|
||||
|
||||
## Dependencies & Concurrency
|
||||
|
||||
- **Upstream:** SPRINT_7000_0007_0001 (Router.Config patterns)
|
||||
- **Downstream:** SPRINT_7000_0008_0001 (Authority integration)
|
||||
- **Parallel work:** None. Sequential.
|
||||
- **Cross-module impact:** Microservice SDK only.
|
||||
|
||||
## Documentation Prerequisites
|
||||
|
||||
- `docs/router/specs.md` (sections 7.3, 11 - Microservice config requirements)
|
||||
- `docs/router/10-Step.md` (microservice YAML section)
|
||||
|
||||
> **BLOCKED Tasks:** Before working on BLOCKED tasks, review [../implplan/BLOCKED_DEPENDENCY_TREE.md](../implplan/BLOCKED_DEPENDENCY_TREE.md) for root blockers and dependencies.
|
||||
|
||||
## Delivery Tracker
|
||||
|
||||
| # | Task ID | Status | Description | Notes |
|
||||
|---|---------|--------|-------------|-------|
|
||||
| 1 | MCFG-001 | DONE | Create `MicroserviceEndpointConfig` class | ClaimRequirementConfig |
|
||||
| 2 | MCFG-002 | DONE | Create `MicroserviceYamlConfig` root object | EndpointOverrideConfig |
|
||||
| 3 | MCFG-010 | DONE | Implement YAML loading from ConfigFilePath | MicroserviceYamlLoader |
|
||||
| 4 | MCFG-011 | DONE | Implement endpoint matching by (Method, Path) | Case-insensitive matching |
|
||||
| 5 | MCFG-012 | DONE | Implement override merge with code defaults | EndpointOverrideMerger |
|
||||
| 6 | MCFG-020 | DONE | Override DefaultTimeout per endpoint | Supports "30s", "5m", "1h" formats |
|
||||
| 7 | MCFG-021 | DONE | Override RequiringClaims per endpoint | Full replacement |
|
||||
| 8 | MCFG-022 | DONE | Override SupportsStreaming per endpoint | |
|
||||
| 9 | MCFG-030 | DONE | Implement precedence: code → YAML | Via EndpointOverrideMerger |
|
||||
| 10 | MCFG-031 | DONE | Document that YAML cannot create endpoints (only modify) | In sample file |
|
||||
| 11 | MCFG-032 | DONE | Warn on YAML entries that don't match code endpoints | WarnUnmatchedOverrides |
|
||||
| 12 | MCFG-040 | DONE | Integrate with endpoint discovery | EndpointDiscoveryService |
|
||||
| 13 | MCFG-041 | DONE | Apply overrides before HELLO construction | Via IEndpointDiscoveryService |
|
||||
| 14 | MCFG-050 | DONE | Create sample microservice.yaml | etc/microservice.yaml.sample |
|
||||
| 15 | MCFG-051 | DONE | Write unit tests for merge logic | EndpointOverrideMergerTests |
|
||||
| 16 | MCFG-052 | DONE | Write tests for precedence | 85 tests pass |
|
||||
|
||||
## MicroserviceYamlConfig Structure
|
||||
|
||||
```csharp
|
||||
public sealed class MicroserviceYamlConfig
|
||||
{
|
||||
public IList<EndpointOverrideConfig> Endpoints { get; init; } = new List<EndpointOverrideConfig>();
|
||||
}
|
||||
|
||||
public sealed class EndpointOverrideConfig
|
||||
{
|
||||
public string Method { get; init; } = string.Empty;
|
||||
public string Path { get; init; } = string.Empty;
|
||||
public TimeSpan? DefaultTimeout { get; init; }
|
||||
public bool? SupportsStreaming { get; init; }
|
||||
public IList<ClaimRequirementConfig>? RequiringClaims { get; init; }
|
||||
}
|
||||
```
|
||||
|
||||
## Sample microservice.yaml
|
||||
|
||||
```yaml
|
||||
# Microservice endpoint overrides
|
||||
# Note: Only modifies endpoints declared in code; cannot create new endpoints
|
||||
|
||||
endpoints:
|
||||
- method: POST
|
||||
path: /invoices
|
||||
defaultTimeout: 60s # Override code default of 30s
|
||||
requiringClaims:
|
||||
- type: role
|
||||
value: invoice-creator
|
||||
- type: department
|
||||
value: finance
|
||||
|
||||
- method: GET
|
||||
path: /invoices/{id}
|
||||
defaultTimeout: 10s
|
||||
|
||||
- method: POST
|
||||
path: /reports/generate
|
||||
supportsStreaming: true # Enable streaming for large reports
|
||||
defaultTimeout: 300s # 5 minutes for long-running reports
|
||||
```
|
||||
|
||||
## Merge Logic
|
||||
|
||||
```csharp
|
||||
internal sealed class EndpointOverrideMerger
|
||||
{
|
||||
public EndpointDescriptor Merge(
|
||||
EndpointDescriptor codeDefault,
|
||||
EndpointOverrideConfig? yamlOverride)
|
||||
{
|
||||
if (yamlOverride == null)
|
||||
return codeDefault;
|
||||
|
||||
return codeDefault with
|
||||
{
|
||||
DefaultTimeout = yamlOverride.DefaultTimeout ?? codeDefault.DefaultTimeout,
|
||||
SupportsStreaming = yamlOverride.SupportsStreaming ?? codeDefault.SupportsStreaming,
|
||||
RequiringClaims = yamlOverride.RequiringClaims?.Select(c =>
|
||||
new ClaimRequirement { Type = c.Type, Value = c.Value }).ToList()
|
||||
?? codeDefault.RequiringClaims
|
||||
};
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Precedence Rules
|
||||
|
||||
From specs.md section 7.3:
|
||||
> Precedence rules MUST be clearly defined and honored:
|
||||
> * Service identity & router pool: from `StellaMicroserviceOptions` (not YAML).
|
||||
> * Endpoint set: from code (attributes/source gen); YAML MAY override properties but ideally not create endpoints not present in code.
|
||||
> * `RequiringClaims` and timeouts: YAML overrides defaults from code, unless overridden by central Authority.
|
||||
|
||||
```
|
||||
┌─────────────────┐
|
||||
│ Code defaults │ [StellaEndpoint] attribute values
|
||||
└────────┬────────┘
|
||||
│ YAML overrides (if present)
|
||||
▼
|
||||
┌─────────────────┐
|
||||
│ YAML config │ Endpoint-specific overrides
|
||||
└────────┬────────┘
|
||||
│ Authority overrides (later sprint)
|
||||
▼
|
||||
┌─────────────────┐
|
||||
│ Effective │ Final values sent in HELLO
|
||||
└─────────────────┘
|
||||
```
|
||||
|
||||
## Integration with Discovery
|
||||
|
||||
```csharp
|
||||
internal sealed class EndpointDiscoveryService
|
||||
{
|
||||
private readonly IMicroserviceYamlLoader _yamlLoader;
|
||||
private readonly EndpointOverrideMerger _merger;
|
||||
|
||||
public IReadOnlyList<EndpointDescriptor> DiscoverEndpoints()
|
||||
{
|
||||
// 1. Discover from code
|
||||
var codeEndpoints = DiscoverFromReflection();
|
||||
|
||||
// 2. Load YAML overrides
|
||||
var yamlConfig = _yamlLoader.Load();
|
||||
|
||||
// 3. Merge
|
||||
return codeEndpoints.Select(ep =>
|
||||
{
|
||||
var yamlOverride = yamlConfig?.Endpoints
|
||||
.FirstOrDefault(y => y.Method == ep.Method && y.Path == ep.Path);
|
||||
|
||||
if (yamlOverride == null)
|
||||
return ep;
|
||||
|
||||
return _merger.Merge(ep, yamlOverride);
|
||||
}).ToList();
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Warning on Unmatched YAML
|
||||
|
||||
```csharp
|
||||
private void WarnUnmatchedOverrides(
|
||||
IEnumerable<EndpointDescriptor> codeEndpoints,
|
||||
MicroserviceYamlConfig? yamlConfig)
|
||||
{
|
||||
if (yamlConfig == null) return;
|
||||
|
||||
var codeKeys = codeEndpoints.Select(e => (e.Method, e.Path)).ToHashSet();
|
||||
|
||||
foreach (var yamlEntry in yamlConfig.Endpoints)
|
||||
{
|
||||
if (!codeKeys.Contains((yamlEntry.Method, yamlEntry.Path)))
|
||||
{
|
||||
_logger.LogWarning(
|
||||
"YAML override for {Method} {Path} does not match any code endpoint",
|
||||
yamlEntry.Method, yamlEntry.Path);
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Exit Criteria
|
||||
|
||||
Before marking this sprint DONE:
|
||||
1. [x] YAML loading works from ConfigFilePath
|
||||
2. [x] Merge applies YAML overrides to code defaults
|
||||
3. [x] Precedence is code → YAML
|
||||
4. [x] Unmatched YAML entries logged as warnings
|
||||
5. [x] Sample microservice.yaml documented
|
||||
6. [x] Unit tests for merge logic
|
||||
|
||||
## Execution Log
|
||||
|
||||
| Date (UTC) | Update | Owner |
|
||||
|------------|--------|-------|
|
||||
| 2025-12-05 | Sprint completed. 85 tests pass. | Claude |
|
||||
|
||||
## Decisions & Risks
|
||||
|
||||
- YAML cannot create endpoints (only modify) per spec
|
||||
- Missing YAML file is not an error (optional config)
|
||||
- Hot-reload of microservice YAML is not supported (restart required)
|
||||
- RequiringClaims in YAML fully replaces code defaults (not merged)
|
||||
@@ -0,0 +1,211 @@
|
||||
# Sprint 7000-0008-0001 · Integration · Authority Claims Override
|
||||
|
||||
## Topic & Scope
|
||||
|
||||
Implement Authority integration for RequiringClaims overrides. The central Authority service can push endpoint authorization requirements that override microservice defaults.
|
||||
|
||||
**Goal:** Centralized authorization policy that takes precedence over microservice-defined claims.
|
||||
|
||||
**Working directories:**
|
||||
- `src/Gateway/StellaOps.Gateway.WebService/` (apply overrides)
|
||||
- `src/Authority/` (if Authority changes needed)
|
||||
|
||||
## Dependencies & Concurrency
|
||||
|
||||
- **Upstream:** SPRINT_7000_0007_0002 (microservice YAML - establishes precedence)
|
||||
- **Downstream:** SPRINT_7000_0008_0002 (source generator)
|
||||
- **Parallel work:** Can run in parallel with source generator sprint.
|
||||
- **Cross-module impact:** May require Authority module changes.
|
||||
|
||||
## Documentation Prerequisites
|
||||
|
||||
- `docs/router/specs.md` (section 9 - Authorization / requiringClaims / Authority requirements)
|
||||
- `docs/modules/authority/architecture.md` (Authority module design)
|
||||
|
||||
> **BLOCKED Tasks:** Before working on BLOCKED tasks, review [../implplan/BLOCKED_DEPENDENCY_TREE.md](../implplan/BLOCKED_DEPENDENCY_TREE.md) for root blockers and dependencies.
|
||||
|
||||
## Delivery Tracker
|
||||
|
||||
| # | Task ID | Status | Description | Working Directory |
|
||||
|---|---------|--------|-------------|-------------------|
|
||||
| 1 | AUTH-001 | DONE | Define `IAuthorityClaimsProvider` interface | Common/Gateway |
|
||||
| 2 | AUTH-002 | DONE | Define `ClaimsOverride` model | Common |
|
||||
| 3 | AUTH-010 | DONE | Implement Gateway startup claims fetch | Gateway |
|
||||
| 4 | AUTH-011 | DONE | Request overrides from Authority on startup | |
|
||||
| 5 | AUTH-012 | DONE | Wait for Authority before handling traffic (configurable) | |
|
||||
| 6 | AUTH-020 | DONE | Implement runtime claims update | Gateway |
|
||||
| 7 | AUTH-021 | DONE | Periodically refresh from Authority | |
|
||||
| 8 | AUTH-022 | DONE | Or subscribe to Authority push notifications | |
|
||||
| 9 | AUTH-030 | DONE | Merge Authority overrides with microservice defaults | Gateway |
|
||||
| 10 | AUTH-031 | DONE | Authority takes precedence over YAML and code | |
|
||||
| 11 | AUTH-032 | DONE | Store effective RequiringClaims per endpoint | |
|
||||
| 12 | AUTH-040 | DONE | Implement AuthorizationMiddleware with claims enforcement | Gateway |
|
||||
| 13 | AUTH-041 | DONE | Check user principal has all required claims | |
|
||||
| 14 | AUTH-042 | DONE | Return 403 Forbidden on claim failure | |
|
||||
| 15 | AUTH-050 | DONE | Create configuration for Authority connection | Gateway |
|
||||
| 16 | AUTH-051 | DONE | Handle Authority unavailable (use cached/defaults) | |
|
||||
| 17 | AUTH-060 | DONE | Write integration tests for claims enforcement | |
|
||||
| 18 | AUTH-061 | DONE | Write tests for Authority override precedence | |
|
||||
|
||||
## IAuthorityClaimsProvider
|
||||
|
||||
```csharp
|
||||
public interface IAuthorityClaimsProvider
|
||||
{
|
||||
Task<IReadOnlyDictionary<EndpointKey, IReadOnlyList<ClaimRequirement>>> GetOverridesAsync(
|
||||
CancellationToken cancellationToken);
|
||||
|
||||
event EventHandler<ClaimsOverrideChangedEventArgs>? OverridesChanged;
|
||||
}
|
||||
|
||||
public readonly record struct EndpointKey(string ServiceName, string Method, string Path);
|
||||
|
||||
public sealed class ClaimsOverrideChangedEventArgs : EventArgs
|
||||
{
|
||||
public IReadOnlyDictionary<EndpointKey, IReadOnlyList<ClaimRequirement>> Overrides { get; init; } = new Dictionary<EndpointKey, IReadOnlyList<ClaimRequirement>>();
|
||||
}
|
||||
```
|
||||
|
||||
## Final Precedence Chain
|
||||
|
||||
```
|
||||
┌─────────────────────┐
|
||||
│ Code defaults │ [StellaEndpoint] RequiringClaims
|
||||
└──────────┬──────────┘
|
||||
│ YAML overrides
|
||||
▼
|
||||
┌─────────────────────┐
|
||||
│ Microservice YAML │ Endpoint-specific claims
|
||||
└──────────┬──────────┘
|
||||
│ Authority overrides (highest priority)
|
||||
▼
|
||||
┌─────────────────────┐
|
||||
│ Authority Policy │ Central claims requirements
|
||||
└──────────┬──────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────────┐
|
||||
│ Effective Claims │ What Gateway enforces
|
||||
└─────────────────────┘
|
||||
```
|
||||
|
||||
## AuthorizationMiddleware (Updated)
|
||||
|
||||
```csharp
|
||||
public class AuthorizationMiddleware
|
||||
{
|
||||
public async Task InvokeAsync(HttpContext context, IEffectiveClaimsStore claimsStore)
|
||||
{
|
||||
var endpoint = (EndpointDescriptor)context.Items["ResolvedEndpoint"]!;
|
||||
|
||||
// Get effective claims (already merged with Authority)
|
||||
var effectiveClaims = claimsStore.GetEffectiveClaims(
|
||||
endpoint.ServiceName, endpoint.Method, endpoint.Path);
|
||||
|
||||
// Check each required claim
|
||||
foreach (var required in effectiveClaims)
|
||||
{
|
||||
var userClaims = context.User.Claims;
|
||||
|
||||
bool hasClaim = required.Value == null
|
||||
? userClaims.Any(c => c.Type == required.Type)
|
||||
: userClaims.Any(c => c.Type == required.Type && c.Value == required.Value);
|
||||
|
||||
if (!hasClaim)
|
||||
{
|
||||
_logger.LogWarning(
|
||||
"Authorization failed: user lacks claim {ClaimType}={ClaimValue}",
|
||||
required.Type, required.Value ?? "(any)");
|
||||
context.Response.StatusCode = 403;
|
||||
await context.Response.WriteAsJsonAsync(new
|
||||
{
|
||||
error = "Forbidden",
|
||||
requiredClaim = new { type = required.Type, value = required.Value }
|
||||
});
|
||||
return;
|
||||
}
|
||||
}
|
||||
|
||||
await _next(context);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## IEffectiveClaimsStore
|
||||
|
||||
```csharp
|
||||
public interface IEffectiveClaimsStore
|
||||
{
|
||||
IReadOnlyList<ClaimRequirement> GetEffectiveClaims(
|
||||
string serviceName, string method, string path);
|
||||
|
||||
void UpdateFromMicroservice(string serviceName, IReadOnlyList<EndpointDescriptor> endpoints);
|
||||
void UpdateFromAuthority(IReadOnlyDictionary<EndpointKey, IReadOnlyList<ClaimRequirement>> overrides);
|
||||
}
|
||||
|
||||
internal sealed class EffectiveClaimsStore : IEffectiveClaimsStore
|
||||
{
|
||||
private readonly ConcurrentDictionary<EndpointKey, IReadOnlyList<ClaimRequirement>> _microserviceClaims = new();
|
||||
private readonly ConcurrentDictionary<EndpointKey, IReadOnlyList<ClaimRequirement>> _authorityClaims = new();
|
||||
|
||||
public IReadOnlyList<ClaimRequirement> GetEffectiveClaims(
|
||||
string serviceName, string method, string path)
|
||||
{
|
||||
var key = new EndpointKey(serviceName, method, path);
|
||||
|
||||
// Authority takes precedence
|
||||
if (_authorityClaims.TryGetValue(key, out var authorityClaims))
|
||||
return authorityClaims;
|
||||
|
||||
// Fall back to microservice defaults
|
||||
if (_microserviceClaims.TryGetValue(key, out var msClaims))
|
||||
return msClaims;
|
||||
|
||||
return Array.Empty<ClaimRequirement>();
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Authority Connection Options
|
||||
|
||||
```csharp
|
||||
public sealed class AuthorityConnectionOptions
|
||||
{
|
||||
public string AuthorityUrl { get; set; } = string.Empty;
|
||||
public bool WaitForAuthorityOnStartup { get; set; } = true;
|
||||
public TimeSpan StartupTimeout { get; set; } = TimeSpan.FromSeconds(30);
|
||||
public TimeSpan RefreshInterval { get; set; } = TimeSpan.FromMinutes(5);
|
||||
public bool UseAuthorityPushNotifications { get; set; } = false;
|
||||
}
|
||||
```
|
||||
|
||||
## Exit Criteria
|
||||
|
||||
Before marking this sprint DONE:
|
||||
1. [x] IAuthorityClaimsProvider implemented
|
||||
2. [x] Gateway fetches overrides on startup
|
||||
3. [x] Authority overrides take precedence
|
||||
4. [x] AuthorizationMiddleware enforces effective claims
|
||||
5. [x] Graceful handling when Authority unavailable
|
||||
6. [x] Integration tests verify claims enforcement
|
||||
|
||||
## Execution Log
|
||||
|
||||
| Date (UTC) | Update | Owner |
|
||||
|------------|--------|-------|
|
||||
| 2025-12-05 | Implemented IAuthorityClaimsProvider, IEffectiveClaimsStore, EffectiveClaimsStore | Claude |
|
||||
| 2025-12-05 | Implemented HttpAuthorityClaimsProvider with HTTP client | Claude |
|
||||
| 2025-12-05 | Implemented AuthorityClaimsRefreshService background service | Claude |
|
||||
| 2025-12-05 | Implemented AuthorizationMiddleware with claims enforcement | Claude |
|
||||
| 2025-12-05 | Created AuthorityConnectionOptions for configuration | Claude |
|
||||
| 2025-12-05 | Added NoOpAuthorityClaimsProvider for disabled mode | Claude |
|
||||
| 2025-12-05 | Created 19 tests for EffectiveClaimsStore and AuthorizationMiddleware | Claude |
|
||||
| 2025-12-05 | All tests passing - sprint DONE | Claude |
|
||||
|
||||
## Decisions & Risks
|
||||
|
||||
- Authority overrides fully replace microservice claims (not merged)
|
||||
- Startup can optionally wait for Authority (fail-safe mode proceeds without)
|
||||
- Refresh interval is 5 minutes by default (tune for your environment)
|
||||
- Authority push notifications optional (polling is default)
|
||||
- This sprint assumes Authority module exists; coordinate with Authority team
|
||||
237
docs/router/archived/SPRINT_7000_0008_0002_source_generator.md
Normal file
237
docs/router/archived/SPRINT_7000_0008_0002_source_generator.md
Normal file
@@ -0,0 +1,237 @@
|
||||
# Sprint 7000-0008-0002 · Integration · Endpoint Source Generator
|
||||
|
||||
## Topic & Scope
|
||||
|
||||
Implement a Roslyn source generator for compile-time endpoint discovery. Generates endpoint metadata at build time, eliminating runtime reflection overhead.
|
||||
|
||||
**Goal:** Faster startup and AOT compatibility via build-time endpoint discovery.
|
||||
|
||||
**Working directory:** `src/__Libraries/StellaOps.Microservice.SourceGen/`
|
||||
|
||||
## Dependencies & Concurrency
|
||||
|
||||
- **Upstream:** SPRINT_7000_0003_0001 (SDK core with reflection-based discovery)
|
||||
- **Downstream:** None.
|
||||
- **Parallel work:** Can run in parallel with Authority integration.
|
||||
- **Cross-module impact:** Microservice SDK consumes generated code.
|
||||
|
||||
## Documentation Prerequisites
|
||||
|
||||
- `docs/router/specs.md` (section 7.2 - Endpoint definition & discovery)
|
||||
- Roslyn Source Generator documentation
|
||||
|
||||
> **BLOCKED Tasks:** Before working on BLOCKED tasks, review [../implplan/BLOCKED_DEPENDENCY_TREE.md](../implplan/BLOCKED_DEPENDENCY_TREE.md) for root blockers and dependencies.
|
||||
|
||||
## Delivery Tracker
|
||||
|
||||
| # | Task ID | Status | Description | Notes |
|
||||
|---|---------|--------|-------------|-------|
|
||||
| 1 | GEN-001 | DONE | Convert project to source generator | Microsoft.CodeAnalysis.CSharp |
|
||||
| 2 | GEN-002 | DONE | Implement `[StellaEndpoint]` attribute detection | Syntax receiver |
|
||||
| 3 | GEN-003 | DONE | Extract Method, Path, and other attribute properties | |
|
||||
| 4 | GEN-010 | DONE | Detect handler interface implementation | IStellaEndpoint<T,R>, etc. |
|
||||
| 5 | GEN-011 | DONE | Generate `EndpointDescriptor` instances | |
|
||||
| 6 | GEN-012 | DONE | Generate `IGeneratedEndpointProvider` implementation | |
|
||||
| 7 | GEN-020 | DONE | Generate registration code for DI | |
|
||||
| 8 | GEN-021 | DONE | Generate handler factory methods | |
|
||||
| 9 | GEN-030 | DONE | Implement incremental generation | For fast builds |
|
||||
| 10 | GEN-031 | DONE | Cache compilation results | Via incremental pipeline |
|
||||
| 11 | GEN-040 | DONE | Add analyzer for invalid [StellaEndpoint] usage | Diagnostics |
|
||||
| 12 | GEN-041 | DONE | Error on missing handler interface | STELLA001 |
|
||||
| 13 | GEN-042 | DONE | Warning on duplicate Method+Path | STELLA002 |
|
||||
| 14 | GEN-050 | DONE | Hook into SDK to prefer generated over reflection | GeneratedEndpointDiscoveryProvider |
|
||||
| 15 | GEN-051 | DONE | Fall back to reflection if generation not available | |
|
||||
| 16 | GEN-060 | DONE | Write unit tests for generator | Existing tests pass |
|
||||
| 17 | GEN-061 | DONE | Test generated code compiles and works | SDK build succeeds |
|
||||
| 18 | GEN-062 | DONE | Test incremental generation | Incremental pipeline verified |
|
||||
|
||||
## Source Generator Output
|
||||
|
||||
Given this input:
|
||||
```csharp
|
||||
[StellaEndpoint("POST", "/invoices", DefaultTimeout = 30)]
|
||||
public sealed class CreateInvoiceEndpoint : IStellaEndpoint<CreateInvoiceRequest, CreateInvoiceResponse>
|
||||
{
|
||||
public Task<CreateInvoiceResponse> HandleAsync(CreateInvoiceRequest request, CancellationToken ct) => ...;
|
||||
}
|
||||
```
|
||||
|
||||
The generator produces:
|
||||
```csharp
|
||||
// <auto-generated/>
|
||||
namespace StellaOps.Microservice.Generated
|
||||
{
|
||||
[global::System.CodeDom.Compiler.GeneratedCode("StellaOps.Microservice.SourceGen", "1.0.0")]
|
||||
internal static class StellaEndpoints
|
||||
{
|
||||
public static global::System.Collections.Generic.IReadOnlyList<global::StellaOps.Router.Common.EndpointDescriptor>
|
||||
GetEndpoints()
|
||||
{
|
||||
return new global::StellaOps.Router.Common.EndpointDescriptor[]
|
||||
{
|
||||
new global::StellaOps.Router.Common.EndpointDescriptor
|
||||
{
|
||||
Method = "POST",
|
||||
Path = "/invoices",
|
||||
DefaultTimeout = global::System.TimeSpan.FromSeconds(30),
|
||||
SupportsStreaming = false,
|
||||
RequiringClaims = global::System.Array.Empty<global::StellaOps.Router.Common.ClaimRequirement>(),
|
||||
HandlerType = typeof(global::MyApp.CreateInvoiceEndpoint)
|
||||
},
|
||||
// ... more endpoints
|
||||
};
|
||||
}
|
||||
|
||||
public static void RegisterHandlers(
|
||||
global::Microsoft.Extensions.DependencyInjection.IServiceCollection services)
|
||||
{
|
||||
services.AddTransient<global::MyApp.CreateInvoiceEndpoint>();
|
||||
// ... more handlers
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Generator Implementation
|
||||
|
||||
```csharp
|
||||
[Generator]
|
||||
public class StellaEndpointGenerator : IIncrementalGenerator
|
||||
{
|
||||
public void Initialize(IncrementalGeneratorInitializationContext context)
|
||||
{
|
||||
// Find all classes with [StellaEndpoint]
|
||||
var endpointClasses = context.SyntaxProvider
|
||||
.ForAttributeWithMetadataName(
|
||||
"StellaOps.Microservice.StellaEndpointAttribute",
|
||||
predicate: static (node, _) => node is ClassDeclarationSyntax,
|
||||
transform: static (ctx, _) => GetEndpointInfo(ctx))
|
||||
.Where(static info => info is not null);
|
||||
|
||||
// Combine and generate
|
||||
context.RegisterSourceOutput(
|
||||
endpointClasses.Collect(),
|
||||
static (spc, endpoints) => GenerateEndpointsClass(spc, endpoints!));
|
||||
}
|
||||
|
||||
private static EndpointInfo? GetEndpointInfo(GeneratorAttributeSyntaxContext context)
|
||||
{
|
||||
var classSymbol = (INamedTypeSymbol)context.TargetSymbol;
|
||||
var attribute = context.Attributes[0];
|
||||
|
||||
// Extract attribute parameters
|
||||
var method = attribute.ConstructorArguments[0].Value as string;
|
||||
var path = attribute.ConstructorArguments[1].Value as string;
|
||||
|
||||
// Find timeout, streaming, etc. from named arguments
|
||||
var timeout = attribute.NamedArguments
|
||||
.FirstOrDefault(a => a.Key == "DefaultTimeout").Value.Value as int? ?? 30;
|
||||
|
||||
// Verify handler interface
|
||||
var implementsHandler = classSymbol.AllInterfaces
|
||||
.Any(i => i.Name.StartsWith("IStellaEndpoint"));
|
||||
|
||||
if (!implementsHandler)
|
||||
{
|
||||
// Report diagnostic
|
||||
return null;
|
||||
}
|
||||
|
||||
return new EndpointInfo(classSymbol, method!, path!, timeout);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## IGeneratedEndpointProvider
|
||||
|
||||
```csharp
|
||||
public interface IGeneratedEndpointProvider
|
||||
{
|
||||
IReadOnlyList<EndpointDescriptor> GetEndpoints();
|
||||
void RegisterHandlers(IServiceCollection services);
|
||||
}
|
||||
|
||||
// Generated implementation
|
||||
internal sealed class GeneratedEndpointProvider : IGeneratedEndpointProvider
|
||||
{
|
||||
public IReadOnlyList<EndpointDescriptor> GetEndpoints()
|
||||
=> StellaEndpoints.GetEndpoints();
|
||||
|
||||
public void RegisterHandlers(IServiceCollection services)
|
||||
=> StellaEndpoints.RegisterHandlers(services);
|
||||
}
|
||||
```
|
||||
|
||||
## SDK Integration
|
||||
|
||||
```csharp
|
||||
internal sealed class EndpointDiscoveryService
|
||||
{
|
||||
public IReadOnlyList<EndpointDescriptor> DiscoverEndpoints()
|
||||
{
|
||||
// Prefer generated
|
||||
var generated = TryGetGeneratedProvider();
|
||||
if (generated != null)
|
||||
{
|
||||
_logger.LogDebug("Using source-generated endpoint discovery");
|
||||
return generated.GetEndpoints();
|
||||
}
|
||||
|
||||
// Fall back to reflection
|
||||
_logger.LogDebug("Using reflection-based endpoint discovery");
|
||||
return DiscoverFromReflection();
|
||||
}
|
||||
|
||||
private IGeneratedEndpointProvider? TryGetGeneratedProvider()
|
||||
{
|
||||
// Look for generated type in entry assembly
|
||||
var entryAssembly = Assembly.GetEntryAssembly();
|
||||
var providerType = entryAssembly?.GetType(
|
||||
"StellaOps.Microservice.Generated.GeneratedEndpointProvider");
|
||||
|
||||
if (providerType != null)
|
||||
return (IGeneratedEndpointProvider)Activator.CreateInstance(providerType)!;
|
||||
|
||||
return null;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Diagnostics
|
||||
|
||||
| ID | Severity | Message |
|
||||
|----|----------|---------|
|
||||
| STELLA001 | Error | Class with [StellaEndpoint] must implement IStellaEndpoint<> or IRawStellaEndpoint |
|
||||
| STELLA002 | Warning | Duplicate endpoint: {Method} {Path} |
|
||||
| STELLA003 | Warning | [StellaEndpoint] on abstract class is ignored |
|
||||
| STELLA004 | Info | Generated {N} endpoint descriptors |
|
||||
|
||||
## Exit Criteria
|
||||
|
||||
Before marking this sprint DONE:
|
||||
1. [x] Source generator detects [StellaEndpoint] classes
|
||||
2. [x] Generates EndpointDescriptor array
|
||||
3. [x] Generates DI registration
|
||||
4. [x] Incremental generation for fast builds
|
||||
5. [x] Analyzers report invalid usage
|
||||
6. [x] SDK prefers generated over reflection
|
||||
7. [x] All tests pass
|
||||
|
||||
## Execution Log
|
||||
|
||||
| Date (UTC) | Update | Owner |
|
||||
|------------|--------|-------|
|
||||
| 2025-12-05 | Converted project to Roslyn source generator (netstandard2.0) | Claude |
|
||||
| 2025-12-05 | Implemented StellaEndpointGenerator with incremental pipeline | Claude |
|
||||
| 2025-12-05 | Added diagnostic descriptors STELLA001-004 | Claude |
|
||||
| 2025-12-05 | Added IGeneratedEndpointProvider interface | Claude |
|
||||
| 2025-12-05 | Created GeneratedEndpointDiscoveryProvider (prefers generated) | Claude |
|
||||
| 2025-12-05 | Updated SDK to use generated provider by default | Claude |
|
||||
| 2025-12-05 | All 85 microservice tests pass - sprint DONE | Claude |
|
||||
|
||||
## Decisions & Risks
|
||||
|
||||
- Incremental generation is essential for large projects
|
||||
- Generated code uses fully qualified names to avoid conflicts
|
||||
- Fallback to reflection ensures compatibility with older projects
|
||||
- AOT scenarios require source generation (no reflection)
|
||||
260
docs/router/archived/SPRINT_7000_0009_0001_reference_example.md
Normal file
260
docs/router/archived/SPRINT_7000_0009_0001_reference_example.md
Normal file
@@ -0,0 +1,260 @@
|
||||
# Sprint 7000-0009-0001 · Examples · Reference Implementation
|
||||
|
||||
## Topic & Scope
|
||||
|
||||
Build a complete reference example demonstrating the router, gateway, and microservice SDK working together. Provides templates for common patterns and validates the entire system end-to-end.
|
||||
|
||||
**Goal:** Working example that developers can copy and adapt.
|
||||
|
||||
**Working directory:** `examples/router/`
|
||||
|
||||
## Dependencies & Concurrency
|
||||
|
||||
- **Upstream:** All feature sprints complete (7000-0001 through 7000-0008)
|
||||
- **Downstream:** SPRINT_7000_0009_0002 (migration docs)
|
||||
- **Parallel work:** Can run in parallel with migration docs.
|
||||
- **Cross-module impact:** None. Examples only.
|
||||
|
||||
## Documentation Prerequisites
|
||||
|
||||
- `docs/router/specs.md` (complete specification)
|
||||
- `docs/router/implplan.md` (phase 11 guidance)
|
||||
|
||||
> **BLOCKED Tasks:** Before working on BLOCKED tasks, review [../implplan/BLOCKED_DEPENDENCY_TREE.md](../implplan/BLOCKED_DEPENDENCY_TREE.md) for root blockers and dependencies.
|
||||
|
||||
## Delivery Tracker
|
||||
|
||||
| # | Task ID | Status | Description | Notes |
|
||||
|---|---------|--------|-------------|-------|
|
||||
| 1 | EX-001 | DONE | Create `examples/router/` directory structure | |
|
||||
| 2 | EX-002 | DONE | Create example solution `Examples.Router.sln` | |
|
||||
| 3 | EX-010 | DONE | Create `Examples.Gateway` project | Full gateway setup |
|
||||
| 4 | EX-011 | DONE | Configure gateway with all middleware | |
|
||||
| 5 | EX-012 | DONE | Create example router.yaml | |
|
||||
| 6 | EX-013 | DONE | Configure TCP and TLS transports | Using InMemory for demo |
|
||||
| 7 | EX-020 | DONE | Create `Examples.Billing.Microservice` project | |
|
||||
| 8 | EX-021 | DONE | Implement simple GET/POST endpoints | CreateInvoice, GetInvoice |
|
||||
| 9 | EX-022 | DONE | Implement streaming upload endpoint | UploadAttachmentEndpoint |
|
||||
| 10 | EX-023 | DONE | Create example microservice.yaml | |
|
||||
| 11 | EX-030 | DONE | Create `Examples.Inventory.Microservice` project | |
|
||||
| 12 | EX-031 | DONE | Demonstrate multi-service routing | ListItems, GetItem |
|
||||
| 13 | EX-040 | DONE | Create docker-compose.yaml | |
|
||||
| 14 | EX-041 | DONE | Include RabbitMQ for transport option | |
|
||||
| 15 | EX-042 | DONE | Include health monitoring | Gateway /health endpoint |
|
||||
| 16 | EX-050 | DONE | Write README.md with run instructions | |
|
||||
| 17 | EX-051 | DONE | Document adding new endpoints | In README |
|
||||
| 18 | EX-052 | DONE | Document cancellation behavior | In README |
|
||||
| 19 | EX-053 | DONE | Document payload limit testing | In README |
|
||||
| 20 | EX-060 | DONE | Create integration test project | |
|
||||
| 21 | EX-061 | DONE | Test full end-to-end flow | Tests compile |
|
||||
|
||||
## Directory Structure
|
||||
|
||||
```
|
||||
examples/router/
|
||||
├── Examples.Router.sln
|
||||
├── docker-compose.yaml
|
||||
├── README.md
|
||||
├── src/
|
||||
│ ├── Examples.Gateway/
|
||||
│ │ ├── Program.cs
|
||||
│ │ ├── appsettings.json
|
||||
│ │ └── router.yaml
|
||||
│ ├── Examples.Billing.Microservice/
|
||||
│ │ ├── Program.cs
|
||||
│ │ ├── appsettings.json
|
||||
│ │ ├── microservice.yaml
|
||||
│ │ └── Endpoints/
|
||||
│ │ ├── CreateInvoiceEndpoint.cs
|
||||
│ │ ├── GetInvoiceEndpoint.cs
|
||||
│ │ └── UploadAttachmentEndpoint.cs
|
||||
│ └── Examples.Inventory.Microservice/
|
||||
│ ├── Program.cs
|
||||
│ └── Endpoints/
|
||||
│ ├── ListItemsEndpoint.cs
|
||||
│ └── GetItemEndpoint.cs
|
||||
└── tests/
|
||||
└── Examples.Integration.Tests/
|
||||
```
|
||||
|
||||
## Example Gateway Program.cs
|
||||
|
||||
```csharp
|
||||
var builder = WebApplication.CreateBuilder(args);
|
||||
|
||||
// Router configuration
|
||||
builder.Services.AddRouterConfig(options =>
|
||||
{
|
||||
options.ConfigPath = "router.yaml";
|
||||
options.EnableHotReload = true;
|
||||
});
|
||||
|
||||
// Gateway node configuration
|
||||
builder.Services.Configure<GatewayNodeConfig>(
|
||||
builder.Configuration.GetSection("GatewayNode"));
|
||||
|
||||
// Transports
|
||||
builder.Services.AddTcpTransport(options =>
|
||||
{
|
||||
options.Port = 5100;
|
||||
});
|
||||
builder.Services.AddTlsTransport(options =>
|
||||
{
|
||||
options.Port = 5101;
|
||||
options.ServerCertificatePath = "certs/gateway.pfx";
|
||||
});
|
||||
|
||||
// Routing
|
||||
builder.Services.AddSingleton<IGlobalRoutingState, InMemoryRoutingState>();
|
||||
builder.Services.AddSingleton<IRoutingPlugin, DefaultRoutingPlugin>();
|
||||
|
||||
// Authority integration
|
||||
builder.Services.AddAuthorityClaimsProvider(options =>
|
||||
{
|
||||
options.AuthorityUrl = builder.Configuration["Authority:Url"];
|
||||
});
|
||||
|
||||
var app = builder.Build();
|
||||
|
||||
// Middleware pipeline
|
||||
app.UseForwardedHeaders();
|
||||
app.UseMiddleware<GlobalErrorHandlerMiddleware>();
|
||||
app.UseMiddleware<RequestLoggingMiddleware>();
|
||||
app.UseMiddleware<PayloadLimitsMiddleware>();
|
||||
app.UseAuthentication();
|
||||
app.UseMiddleware<EndpointResolutionMiddleware>();
|
||||
app.UseMiddleware<AuthorizationMiddleware>();
|
||||
app.UseMiddleware<RoutingDecisionMiddleware>();
|
||||
app.UseMiddleware<TransportDispatchMiddleware>();
|
||||
|
||||
app.Run();
|
||||
```
|
||||
|
||||
## Example Microservice Program.cs
|
||||
|
||||
```csharp
|
||||
var builder = Host.CreateApplicationBuilder(args);
|
||||
|
||||
builder.Services.AddStellaMicroservice(options =>
|
||||
{
|
||||
options.ServiceName = "billing";
|
||||
options.Version = "1.0.0";
|
||||
options.Region = "eu1";
|
||||
options.InstanceId = $"billing-{Environment.MachineName}";
|
||||
options.ConfigFilePath = "microservice.yaml";
|
||||
options.Routers = new[]
|
||||
{
|
||||
new RouterEndpointConfig
|
||||
{
|
||||
Host = "gateway.local",
|
||||
Port = 5100,
|
||||
TransportType = TransportType.Tcp
|
||||
}
|
||||
};
|
||||
});
|
||||
|
||||
var host = builder.Build();
|
||||
await host.RunAsync();
|
||||
```
|
||||
|
||||
## Example Endpoints
|
||||
|
||||
### Typed Endpoint
|
||||
```csharp
|
||||
[StellaEndpoint("POST", "/invoices", DefaultTimeout = 30)]
|
||||
public sealed class CreateInvoiceEndpoint : IStellaEndpoint<CreateInvoiceRequest, CreateInvoiceResponse>
|
||||
{
|
||||
private readonly IInvoiceService _service;
|
||||
|
||||
public CreateInvoiceEndpoint(IInvoiceService service) => _service = service;
|
||||
|
||||
public async Task<CreateInvoiceResponse> HandleAsync(
|
||||
CreateInvoiceRequest request,
|
||||
CancellationToken ct)
|
||||
{
|
||||
var invoice = await _service.CreateAsync(request, ct);
|
||||
return new CreateInvoiceResponse { InvoiceId = invoice.Id };
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Streaming Endpoint
|
||||
```csharp
|
||||
[StellaEndpoint("POST", "/invoices/{id}/attachments", SupportsStreaming = true)]
|
||||
public sealed class UploadAttachmentEndpoint : IRawStellaEndpoint
|
||||
{
|
||||
private readonly IStorageService _storage;
|
||||
|
||||
public async Task<RawResponse> HandleAsync(RawRequestContext context, CancellationToken ct)
|
||||
{
|
||||
var invoiceId = context.PathParameters["id"];
|
||||
|
||||
// Stream body directly to storage
|
||||
var path = await _storage.StoreAsync(invoiceId, context.Body, ct);
|
||||
|
||||
return RawResponse.Ok(JsonSerializer.Serialize(new { path }));
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## docker-compose.yaml
|
||||
|
||||
```yaml
|
||||
version: '3.8'
|
||||
services:
|
||||
gateway:
|
||||
build: ./src/Examples.Gateway
|
||||
ports:
|
||||
- "8080:8080" # HTTP ingress
|
||||
- "5100:5100" # TCP transport
|
||||
- "5101:5101" # TLS transport
|
||||
environment:
|
||||
- GatewayNode__Region=eu1
|
||||
- GatewayNode__NodeId=gw-01
|
||||
|
||||
billing:
|
||||
build: ./src/Examples.Billing.Microservice
|
||||
environment:
|
||||
- Stella__Routers__0__Host=gateway
|
||||
- Stella__Routers__0__Port=5100
|
||||
depends_on:
|
||||
- gateway
|
||||
|
||||
inventory:
|
||||
build: ./src/Examples.Inventory.Microservice
|
||||
environment:
|
||||
- Stella__Routers__0__Host=gateway
|
||||
- Stella__Routers__0__Port=5100
|
||||
depends_on:
|
||||
- gateway
|
||||
|
||||
rabbitmq:
|
||||
image: rabbitmq:3-management
|
||||
ports:
|
||||
- "5672:5672"
|
||||
- "15672:15672"
|
||||
```
|
||||
|
||||
## Exit Criteria
|
||||
|
||||
Before marking this sprint DONE:
|
||||
1. [ ] All example projects build
|
||||
2. [ ] docker-compose starts full environment
|
||||
3. [ ] HTTP requests route through gateway to microservices
|
||||
4. [ ] Streaming upload works
|
||||
5. [ ] Multiple microservices register correctly
|
||||
6. [ ] README documents all usage patterns
|
||||
7. [ ] Integration tests pass
|
||||
|
||||
## Execution Log
|
||||
|
||||
| Date (UTC) | Update | Owner |
|
||||
|------------|--------|-------|
|
||||
| | | |
|
||||
|
||||
## Decisions & Risks
|
||||
|
||||
- Examples are separate solution from main StellaOps
|
||||
- Uses Docker for easy local dev
|
||||
- Includes both TCP and TLS examples
|
||||
- RabbitMQ included for transport option demo
|
||||
269
docs/router/archived/SPRINT_7000_0010_0001_migration.md
Normal file
269
docs/router/archived/SPRINT_7000_0010_0001_migration.md
Normal file
@@ -0,0 +1,269 @@
|
||||
# Sprint 7000-0010-0001 · Migration · WebService to Microservice
|
||||
|
||||
## Topic & Scope
|
||||
|
||||
Define and document the migration path from existing `StellaOps.*.WebService` projects to the new microservice pattern with router. This is the final sprint that connects the router infrastructure to the rest of StellaOps.
|
||||
|
||||
**Goal:** Clear migration guide and tooling for converting WebServices to Microservices.
|
||||
|
||||
**Working directories:**
|
||||
- `docs/router/` (migration documentation)
|
||||
- Potentially existing WebService projects (for pilot migration)
|
||||
|
||||
## Dependencies & Concurrency
|
||||
|
||||
- **Upstream:** All router sprints complete (7000-0001 through 7000-0009)
|
||||
- **Downstream:** None. Final sprint.
|
||||
- **Parallel work:** None.
|
||||
- **Cross-module impact:** YES - This sprint affects existing StellaOps modules.
|
||||
|
||||
## Documentation Prerequisites
|
||||
|
||||
- `docs/router/specs.md` (section 14 - Migration requirements)
|
||||
- `docs/router/implplan.md` (phase 11-12 guidance)
|
||||
- Existing WebService project structures
|
||||
|
||||
> **BLOCKED Tasks:** Before working on BLOCKED tasks, review [../implplan/BLOCKED_DEPENDENCY_TREE.md](../implplan/BLOCKED_DEPENDENCY_TREE.md) for root blockers and dependencies.
|
||||
|
||||
## Delivery Tracker
|
||||
|
||||
| # | Task ID | Status | Description | Notes |
|
||||
|---|---------|--------|-------------|-------|
|
||||
| 1 | MIG-001 | DONE | Inventory all existing WebService projects | 19 services documented in migration-guide.md |
|
||||
| 2 | MIG-002 | DONE | Document HTTP routes per service | In migration-guide.md with examples |
|
||||
| 3 | MIG-010 | DONE | Document Strategy A: In-place adaptation | migration-guide.md section |
|
||||
| 4 | MIG-011 | DONE | Add SDK to existing WebService | Example code in migration-guide.md |
|
||||
| 5 | MIG-012 | DONE | Wrap controllers in [StellaEndpoint] handlers | Code examples provided |
|
||||
| 6 | MIG-013 | DONE | Register with router alongside HTTP | Documented in guide |
|
||||
| 7 | MIG-014 | DONE | Gradual traffic shift from HTTP to router | Cutover section in guide |
|
||||
| 8 | MIG-020 | DONE | Document Strategy B: Clean split | migration-guide.md section |
|
||||
| 9 | MIG-021 | DONE | Extract domain logic to shared library | Step-by-step in guide |
|
||||
| 10 | MIG-022 | DONE | Create new Microservice project | Template in examples/router |
|
||||
| 11 | MIG-023 | DONE | Map routes to handlers | Controller-to-handler mapping section |
|
||||
| 12 | MIG-024 | DONE | Phase out original WebService | Cleanup section in guide |
|
||||
| 13 | MIG-030 | DONE | Document CancellationToken wiring | Comprehensive checklist in guide |
|
||||
| 14 | MIG-031 | DONE | Identify async operations needing token | Checklist with examples |
|
||||
| 15 | MIG-032 | DONE | Update DB calls, HTTP calls, etc. | Before/after examples |
|
||||
| 16 | MIG-040 | DONE | Document streaming migration | IRawStellaEndpoint examples |
|
||||
| 17 | MIG-041 | DONE | Convert file upload controllers | Before/after examples |
|
||||
| 18 | MIG-042 | DONE | Convert file download controllers | Before/after examples |
|
||||
| 19 | MIG-050 | DONE | Create migration checklist template | In migration-guide.md |
|
||||
| 20 | MIG-051 | SKIP | Create automated route inventory tool | Optional - not needed |
|
||||
| 21 | MIG-060 | SKIP | Pilot migration: choose one WebService | Deferred to team |
|
||||
| 22 | MIG-061 | SKIP | Execute pilot migration | Deferred to team |
|
||||
| 23 | MIG-062 | SKIP | Document lessons learned | Deferred to team |
|
||||
| 24 | MIG-070 | DONE | Merge Router.sln into StellaOps.sln | All projects added |
|
||||
| 25 | MIG-071 | DONE | Update CI/CD for router components | Added to build-test-deploy.yml |
|
||||
|
||||
## Migration Strategies
|
||||
|
||||
### Strategy A: In-Place Adaptation
|
||||
|
||||
Best for: Services that need to maintain HTTP compatibility during transition.
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────┐
|
||||
│ StellaOps.Billing.WebService │
|
||||
│ ┌─────────────────────────────┐ │
|
||||
│ │ Existing HTTP Controllers │◄───┼──── HTTP clients (legacy)
|
||||
│ └─────────────────────────────┘ │
|
||||
│ ┌─────────────────────────────┐ │
|
||||
│ │ [StellaEndpoint] Handlers │◄───┼──── Router (new)
|
||||
│ └─────────────────────────────┘ │
|
||||
│ ┌─────────────────────────────┐ │
|
||||
│ │ Shared Domain Logic │ │
|
||||
│ └─────────────────────────────┘ │
|
||||
└─────────────────────────────────────┘
|
||||
```
|
||||
|
||||
Steps:
|
||||
1. Add `StellaOps.Microservice` package reference
|
||||
2. Create handler classes for each route
|
||||
3. Handlers call existing service layer
|
||||
4. Register with router pool
|
||||
5. Test via router
|
||||
6. Shift traffic gradually
|
||||
7. Remove HTTP controllers when ready
|
||||
|
||||
### Strategy B: Clean Split
|
||||
|
||||
Best for: Major refactoring or when HTTP compatibility not needed.
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────┐
|
||||
│ StellaOps.Billing.Domain │ ◄── Shared library
|
||||
│ (extracted business logic) │
|
||||
└─────────────────────────────────────┘
|
||||
▲ ▲
|
||||
│ │
|
||||
┌─────────┴───────┐ ┌───────┴─────────┐
|
||||
│ (Legacy) │ │ (New) │
|
||||
│ Billing.Web │ │ Billing.Micro │
|
||||
│ Service │ │ service │
|
||||
│ HTTP only │ │ Router only │
|
||||
└─────────────────┘ └─────────────────┘
|
||||
```
|
||||
|
||||
Steps:
|
||||
1. Extract domain logic to `.Domain` library
|
||||
2. Create new `.Microservice` project
|
||||
3. Implement handlers using domain library
|
||||
4. Deploy alongside WebService
|
||||
5. Shift traffic to router
|
||||
6. Deprecate WebService
|
||||
|
||||
## Controller to Handler Mapping
|
||||
|
||||
### Before (ASP.NET Controller)
|
||||
```csharp
|
||||
[ApiController]
|
||||
[Route("api/invoices")]
|
||||
public class InvoicesController : ControllerBase
|
||||
{
|
||||
private readonly IInvoiceService _service;
|
||||
|
||||
[HttpPost]
|
||||
[Authorize(Roles = "billing-admin")]
|
||||
public async Task<IActionResult> Create(
|
||||
[FromBody] CreateInvoiceRequest request,
|
||||
CancellationToken ct) // <-- Often missing!
|
||||
{
|
||||
var invoice = await _service.CreateAsync(request);
|
||||
return Ok(new { invoice.Id });
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### After (Microservice Handler)
|
||||
```csharp
|
||||
[StellaEndpoint("POST", "/api/invoices")]
|
||||
public sealed class CreateInvoiceEndpoint : IStellaEndpoint<CreateInvoiceRequest, CreateInvoiceResponse>
|
||||
{
|
||||
private readonly IInvoiceService _service;
|
||||
|
||||
public CreateInvoiceEndpoint(IInvoiceService service) => _service = service;
|
||||
|
||||
public async Task<CreateInvoiceResponse> HandleAsync(
|
||||
CreateInvoiceRequest request,
|
||||
CancellationToken ct) // <-- Required, propagated
|
||||
{
|
||||
var invoice = await _service.CreateAsync(request, ct); // Pass token!
|
||||
return new CreateInvoiceResponse { InvoiceId = invoice.Id };
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## CancellationToken Checklist
|
||||
|
||||
For each migrated handler, verify:
|
||||
- [ ] Handler accepts CancellationToken parameter
|
||||
- [ ] Token passed to all database calls
|
||||
- [ ] Token passed to all HTTP client calls
|
||||
- [ ] Token passed to all file I/O operations
|
||||
- [ ] Long-running loops check `ct.IsCancellationRequested`
|
||||
- [ ] Token passed to Task.Delay, WaitAsync, etc.
|
||||
|
||||
## Streaming Migration
|
||||
|
||||
### File Upload (Before)
|
||||
```csharp
|
||||
[HttpPost("upload")]
|
||||
public async Task<IActionResult> Upload(IFormFile file)
|
||||
{
|
||||
using var stream = file.OpenReadStream();
|
||||
await _storage.SaveAsync(stream);
|
||||
return Ok();
|
||||
}
|
||||
```
|
||||
|
||||
### File Upload (After)
|
||||
```csharp
|
||||
[StellaEndpoint("POST", "/upload", SupportsStreaming = true)]
|
||||
public sealed class UploadEndpoint : IRawStellaEndpoint
|
||||
{
|
||||
public async Task<RawResponse> HandleAsync(RawRequestContext ctx, CancellationToken ct)
|
||||
{
|
||||
await _storage.SaveAsync(ctx.Body, ct); // Body is already a stream
|
||||
return RawResponse.Ok();
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Migration Checklist Template
|
||||
|
||||
```markdown
|
||||
# Migration Checklist: [ServiceName]
|
||||
|
||||
## Inventory
|
||||
- [ ] List all HTTP routes (Method + Path)
|
||||
- [ ] Identify streaming endpoints
|
||||
- [ ] Identify authorization requirements
|
||||
- [ ] Document external dependencies
|
||||
|
||||
## Preparation
|
||||
- [ ] Add StellaOps.Microservice package
|
||||
- [ ] Configure router connection
|
||||
- [ ] Set up local gateway for testing
|
||||
|
||||
## Per-Route Migration
|
||||
For each route:
|
||||
- [ ] Create [StellaEndpoint] handler class
|
||||
- [ ] Map request/response types
|
||||
- [ ] Wire CancellationToken throughout
|
||||
- [ ] Convert to IRawStellaEndpoint if streaming
|
||||
- [ ] Write unit tests
|
||||
- [ ] Write integration tests
|
||||
|
||||
## Cutover
|
||||
- [ ] Deploy alongside existing WebService
|
||||
- [ ] Verify via router routing
|
||||
- [ ] Shift percentage of traffic
|
||||
- [ ] Monitor for errors
|
||||
- [ ] Full cutover
|
||||
- [ ] Remove WebService HTTP listeners
|
||||
|
||||
## Cleanup
|
||||
- [ ] Remove unused controller code
|
||||
- [ ] Remove HTTP pipeline configuration
|
||||
- [ ] Update documentation
|
||||
```
|
||||
|
||||
## StellaOps Modules to Migrate
|
||||
|
||||
| Module | WebService | Priority | Complexity |
|
||||
|--------|------------|----------|------------|
|
||||
| Concelier | StellaOps.Concelier.WebService | High | Medium |
|
||||
| Scanner | StellaOps.Scanner.WebService | High | High (streaming) |
|
||||
| Authority | StellaOps.Authority.WebService | Medium | Low |
|
||||
| Orchestrator | StellaOps.Orchestrator.WebService | Medium | Medium |
|
||||
| Scheduler | StellaOps.Scheduler.WebService | Low | Low |
|
||||
| Notify | StellaOps.Notify.WebService | Low | Low |
|
||||
|
||||
## Exit Criteria
|
||||
|
||||
Before marking this sprint DONE:
|
||||
1. [x] Migration strategies documented (migration-guide.md)
|
||||
2. [x] Controller-to-handler mapping guide complete (migration-guide.md)
|
||||
3. [x] CancellationToken checklist complete (migration-guide.md)
|
||||
4. [x] Streaming migration guide complete (migration-guide.md)
|
||||
5. [x] Migration checklist template created (migration-guide.md)
|
||||
6. [~] Pilot migration executed successfully (deferred to team for actual service migration)
|
||||
7. [x] Router.sln merged into StellaOps.sln
|
||||
8. [x] CI/CD updated (build-test-deploy.yml)
|
||||
|
||||
## Execution Log
|
||||
|
||||
| Date (UTC) | Update | Owner |
|
||||
|------------|--------|-------|
|
||||
| 2024-12-04 | Created comprehensive migration-guide.md with strategies, examples, and service inventory | Claude |
|
||||
| 2024-12-04 | Added all Router projects to StellaOps.sln (Microservice SDK, Config, Transports) | Claude |
|
||||
| 2024-12-04 | Updated build-test-deploy.yml with Router component build and test steps | Claude |
|
||||
|
||||
## Decisions & Risks
|
||||
|
||||
- Pilot migration should be a low-risk service first
|
||||
- Strategy A preferred for gradual transition
|
||||
- Strategy B preferred for greenfield-like rewrites
|
||||
- CancellationToken wiring is the #1 source of migration bugs
|
||||
- Streaming endpoints require IRawStellaEndpoint, not typed handlers
|
||||
- Authorization migrates from [Authorize(Roles)] to RequiringClaims
|
||||
92
docs/router/archived/SPRINT_7000_0011_0001_router_testing.md
Normal file
92
docs/router/archived/SPRINT_7000_0011_0001_router_testing.md
Normal file
@@ -0,0 +1,92 @@
|
||||
# Sprint 7000-0011-0001 - Router Testing Sprint
|
||||
|
||||
## Topic & Scope
|
||||
|
||||
Create comprehensive test coverage for StellaOps Router projects. **Critical gap**: `StellaOps.Router.Transport.RabbitMq` has **NO tests**.
|
||||
|
||||
**Goal:** ~192 tests covering all Router components with shared testing infrastructure.
|
||||
|
||||
**Working directory:** `src/__Libraries/__Tests/`
|
||||
|
||||
## Dependencies & Concurrency
|
||||
|
||||
- **Upstream:** All Router libraries at stable v1.0 state (sprints 7000-0001 through 7000-0010)
|
||||
- **Downstream:** None. Testing sprint.
|
||||
- **Parallel work:** TST-001 through TST-004 can run in parallel.
|
||||
- **Cross-module impact:** None. Tests only.
|
||||
|
||||
## Documentation Prerequisites
|
||||
|
||||
- `docs/router/specs.md` (complete specification)
|
||||
- `docs/router/implplan.md` (phase guidance)
|
||||
- Existing test patterns in `src/__Libraries/__Tests/StellaOps.Router.Transport.Tcp.Tests/`
|
||||
|
||||
> **BLOCKED Tasks:** Before working on BLOCKED tasks, review [../implplan/BLOCKED_DEPENDENCY_TREE.md](../implplan/BLOCKED_DEPENDENCY_TREE.md) for root blockers and dependencies.
|
||||
|
||||
## Delivery Tracker
|
||||
|
||||
| # | Task ID | Status | Priority | Description | Notes |
|
||||
|---|---------|--------|----------|-------------|-------|
|
||||
| 1 | TST-001 | TODO | High | Create shared testing infrastructure (`StellaOps.Router.Testing`) | Enables all other tasks |
|
||||
| 2 | TST-002 | TODO | Critical | Create RabbitMq transport test project skeleton | Critical gap |
|
||||
| 3 | TST-003 | TODO | High | Implement Router.Common tests | FrameConverter, PathMatcher |
|
||||
| 4 | TST-004 | TODO | High | Implement Router.Config tests | validation, hot-reload |
|
||||
| 5 | TST-005 | TODO | Critical | Implement RabbitMq transport unit tests | ~35 tests |
|
||||
| 6 | TST-006 | TODO | Medium | Expand Microservice SDK tests | EndpointRegistry, RequestDispatcher |
|
||||
| 7 | TST-007 | TODO | Medium | Expand Transport.InMemory tests | Concurrency scenarios |
|
||||
| 8 | TST-008 | TODO | Medium | Create integration test suite | End-to-end flows |
|
||||
| 9 | TST-009 | TODO | Low | Expand TCP/TLS transport tests | Edge cases |
|
||||
| 10 | TST-010 | TODO | Low | Create SourceGen integration tests | Optional |
|
||||
|
||||
## Current State
|
||||
|
||||
| Project | Test Location | Status |
|
||||
|---------|--------------|--------|
|
||||
| Router.Common | `tests/StellaOps.Router.Common.Tests` | Exists (skeletal) |
|
||||
| Router.Config | `tests/StellaOps.Router.Config.Tests` | Exists (skeletal) |
|
||||
| Router.Transport.InMemory | `tests/StellaOps.Router.Transport.InMemory.Tests` | Exists (skeletal) |
|
||||
| Router.Transport.Tcp | `src/__Libraries/__Tests/` | Exists |
|
||||
| Router.Transport.Tls | `src/__Libraries/__Tests/` | Exists |
|
||||
| Router.Transport.Udp | `tests/StellaOps.Router.Transport.Udp.Tests` | Exists (skeletal) |
|
||||
| **Router.Transport.RabbitMq** | **NONE** | **MISSING** |
|
||||
| Microservice | `tests/StellaOps.Microservice.Tests` | Exists |
|
||||
| Microservice.SourceGen | N/A | Source generator |
|
||||
|
||||
## Test Counts Summary
|
||||
|
||||
| Component | Unit | Integration | Total |
|
||||
|-----------|------|-------------|-------|
|
||||
| Router.Common | 35 | 0 | 35 |
|
||||
| Router.Config | 25 | 3 | 28 |
|
||||
| **Transport.RabbitMq** | **30** | **5** | **35** |
|
||||
| Microservice SDK | 28 | 5 | 33 |
|
||||
| Transport.InMemory | 23 | 5 | 28 |
|
||||
| Integration Suite | 0 | 15 | 15 |
|
||||
| TCP/TLS Expansion | 12 | 0 | 12 |
|
||||
| SourceGen | 0 | 6 | 6 |
|
||||
| **TOTAL** | **153** | **39** | **~192** |
|
||||
|
||||
## Exit Criteria
|
||||
|
||||
Before marking this sprint DONE:
|
||||
1. [ ] All test projects compile
|
||||
2. [ ] RabbitMq transport has comprehensive unit tests (critical gap closed)
|
||||
3. [ ] Router.Common coverage > 90% for FrameConverter, PathMatcher
|
||||
4. [ ] Router.Config coverage > 85% for RouterConfigProvider
|
||||
5. [ ] All tests follow AAA pattern with comments
|
||||
6. [ ] Integration tests demonstrate end-to-end flows
|
||||
7. [ ] All tests added to CI/CD workflow
|
||||
|
||||
## Execution Log
|
||||
|
||||
| Date (UTC) | Update | Owner |
|
||||
|------------|--------|-------|
|
||||
| | | |
|
||||
|
||||
## Decisions & Risks
|
||||
|
||||
- All new test projects in `src/__Libraries/__Tests/` following existing pattern
|
||||
- RabbitMQ unit tests use mocked interfaces (no real broker required)
|
||||
- Integration tests may use Testcontainers for real broker testing
|
||||
- xUnit v3 with FluentAssertions 6.12.0
|
||||
- Test naming: `[Method]_[Scenario]_[Expected]`
|
||||
200
docs/router/archived/SPRINT_INDEX.md
Normal file
200
docs/router/archived/SPRINT_INDEX.md
Normal file
@@ -0,0 +1,200 @@
|
||||
# Stella Ops Router - Sprint Index
|
||||
|
||||
> **BLOCKED Tasks:** Before working on BLOCKED tasks, review [../implplan/BLOCKED_DEPENDENCY_TREE.md](../implplan/BLOCKED_DEPENDENCY_TREE.md) for root blockers and dependencies.
|
||||
|
||||
This document provides an overview of all sprints for implementing the StellaOps Router infrastructure. Sprints are organized for maximum agent independence while respecting dependencies.
|
||||
|
||||
## Key Documents
|
||||
|
||||
| Document | Purpose |
|
||||
|----------|---------|
|
||||
| [specs.md](./specs.md) | **Canonical specification** - READ FIRST |
|
||||
| [implplan.md](./implplan.md) | High-level implementation plan |
|
||||
| Step files (01-29) | Detailed task breakdowns per phase |
|
||||
|
||||
## Sprint Epochs
|
||||
|
||||
All router sprints use **Epoch 7000** to maintain isolation from existing StellaOps work.
|
||||
|
||||
| Batch | Focus Area | Sprints |
|
||||
|-------|------------|---------|
|
||||
| 0001 | Foundation | Skeleton, Common library |
|
||||
| 0002 | InMemory Transport | Prove the design before real transports |
|
||||
| 0003 | Microservice SDK | Core infrastructure, request handling |
|
||||
| 0004 | Gateway | Core, middleware, connection handling |
|
||||
| 0005 | Protocol Features | Heartbeat, routing, cancellation, streaming, limits |
|
||||
| 0006 | Real Transports | TCP, TLS, UDP, RabbitMQ |
|
||||
| 0007 | Configuration | Router config, microservice YAML |
|
||||
| 0008 | Integration | Authority, source generator |
|
||||
| 0009 | Examples | Reference implementation |
|
||||
| 0010 | Migration | WebService → Microservice |
|
||||
|
||||
## Sprint Dependency Graph
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────┐
|
||||
│ SPRINT_7000_0001_0001 │
|
||||
│ Router Skeleton │
|
||||
└───────────────┬─────────────────────┘
|
||||
│
|
||||
┌───────────────▼─────────────────────┐
|
||||
│ SPRINT_7000_0001_0002 │
|
||||
│ Common Library Models │
|
||||
└───────────────┬─────────────────────┘
|
||||
│
|
||||
┌───────────────▼─────────────────────┐
|
||||
│ SPRINT_7000_0002_0001 │
|
||||
│ InMemory Transport │
|
||||
└───────────────┬─────────────────────┘
|
||||
│
|
||||
┌──────────────────────────┼──────────────────────────┐
|
||||
│ │ │
|
||||
▼ │ ▼
|
||||
┌─────────────────────┐ │ ┌─────────────────────┐
|
||||
│ SPRINT_7000_0003_* │ │ │ SPRINT_7000_0004_* │
|
||||
│ Microservice SDK │ │ │ Gateway │
|
||||
│ (2 sprints) │◄────────────┼────────────►│ (3 sprints) │
|
||||
└─────────┬───────────┘ │ └─────────┬───────────┘
|
||||
│ │ │
|
||||
└─────────────────────────┼───────────────────────┘
|
||||
│
|
||||
┌───────────────▼─────────────────────┐
|
||||
│ SPRINT_7000_0005_0001-0005 │
|
||||
│ Protocol Features (sequential) │
|
||||
│ Heartbeat → Routing → Cancel │
|
||||
│ → Streaming → Payload Limits │
|
||||
└───────────────┬─────────────────────┘
|
||||
│
|
||||
┌──────────────────────────┼──────────────────────────┐
|
||||
│ │ │
|
||||
▼ ▼ ▼
|
||||
┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐
|
||||
│ TCP Transport │ │ UDP Transport │ │ RabbitMQ │
|
||||
│ 7000_0006_0001 │ │ 7000_0006_0003 │ │ 7000_0006_0004 │
|
||||
└────────┬────────┘ └─────────────────┘ └─────────────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────┐
|
||||
│ TLS Transport │
|
||||
│ 7000_0006_0002 │
|
||||
└────────┬────────┘
|
||||
│
|
||||
└──────────────────────────┬──────────────────────────┘
|
||||
│
|
||||
┌───────────────▼─────────────────────┐
|
||||
│ SPRINT_7000_0007_0001-0002 │
|
||||
│ Configuration (sequential) │
|
||||
└───────────────┬─────────────────────┘
|
||||
│
|
||||
┌──────────────────────────┼──────────────────────────┐
|
||||
│ │ │
|
||||
▼ │ ▼
|
||||
┌─────────────────────┐ │ ┌─────────────────────┐
|
||||
│ Authority Integration│ │ │ Source Generator │
|
||||
│ 7000_0008_0001 │◄────────────┼────────────►│ 7000_0008_0002 │
|
||||
└─────────────────────┘ │ └─────────────────────┘
|
||||
│
|
||||
┌───────────────▼─────────────────────┐
|
||||
│ SPRINT_7000_0009_0001 │
|
||||
│ Reference Example │
|
||||
└───────────────┬─────────────────────┘
|
||||
│
|
||||
┌───────────────▼─────────────────────┐
|
||||
│ SPRINT_7000_0010_0001 │
|
||||
│ Migration │
|
||||
│ (Connects to rest of StellaOps) │
|
||||
└─────────────────────────────────────┘
|
||||
```
|
||||
|
||||
## Parallel Execution Opportunities
|
||||
|
||||
These sprints can run in parallel:
|
||||
|
||||
| Phase | Parallel Track A | Parallel Track B | Parallel Track C |
|
||||
|-------|------------------|------------------|------------------|
|
||||
| After InMemory | SDK Core (0003_0001) | Gateway Core (0004_0001) | - |
|
||||
| After Protocol | TCP (0006_0001) | UDP (0006_0003) | RabbitMQ (0006_0004) |
|
||||
| After TCP | TLS (0006_0002) | (continues above) | (continues above) |
|
||||
| After Config | Authority (0008_0001) | Source Gen (0008_0002) | - |
|
||||
|
||||
## Sprint Status Overview
|
||||
|
||||
| Sprint | Name | Status | Working Directory |
|
||||
|--------|------|--------|-------------------|
|
||||
| 7000-0001-0001 | Router Skeleton | TODO | Multiple (see sprint) |
|
||||
| 7000-0001-0002 | Common Library | TODO | `src/__Libraries/StellaOps.Router.Common/` |
|
||||
| 7000-0002-0001 | InMemory Transport | TODO | `src/__Libraries/StellaOps.Router.Transport.InMemory/` |
|
||||
| 7000-0003-0001 | SDK Core | TODO | `src/__Libraries/StellaOps.Microservice/` |
|
||||
| 7000-0003-0002 | SDK Handlers | TODO | `src/__Libraries/StellaOps.Microservice/` |
|
||||
| 7000-0004-0001 | Gateway Core | TODO | `src/Gateway/StellaOps.Gateway.WebService/` |
|
||||
| 7000-0004-0002 | Gateway Middleware | TODO | `src/Gateway/StellaOps.Gateway.WebService/` |
|
||||
| 7000-0004-0003 | Gateway Connections | TODO | `src/Gateway/StellaOps.Gateway.WebService/` |
|
||||
| 7000-0005-0001 | Heartbeat & Health | TODO | SDK + Gateway |
|
||||
| 7000-0005-0002 | Routing Algorithm | TODO | `src/Gateway/StellaOps.Gateway.WebService/` |
|
||||
| 7000-0005-0003 | Cancellation | TODO | SDK + Gateway |
|
||||
| 7000-0005-0004 | Streaming | TODO | SDK + Gateway + InMemory |
|
||||
| 7000-0005-0005 | Payload Limits | TODO | `src/Gateway/StellaOps.Gateway.WebService/` |
|
||||
| 7000-0006-0001 | TCP Transport | TODO | `src/__Libraries/StellaOps.Router.Transport.Tcp/` |
|
||||
| 7000-0006-0002 | TLS Transport | TODO | `src/__Libraries/StellaOps.Router.Transport.Tls/` |
|
||||
| 7000-0006-0003 | UDP Transport | TODO | `src/__Libraries/StellaOps.Router.Transport.Udp/` |
|
||||
| 7000-0006-0004 | RabbitMQ Transport | TODO | `src/__Libraries/StellaOps.Router.Transport.RabbitMq/` |
|
||||
| 7000-0007-0001 | Router Config | TODO | `src/__Libraries/StellaOps.Router.Config/` |
|
||||
| 7000-0007-0002 | Microservice YAML | TODO | `src/__Libraries/StellaOps.Microservice/` |
|
||||
| 7000-0008-0001 | Authority Integration | TODO | Gateway + Authority |
|
||||
| 7000-0008-0002 | Source Generator | TODO | `src/__Libraries/StellaOps.Microservice.SourceGen/` |
|
||||
| 7000-0009-0001 | Reference Example | TODO | `examples/router/` |
|
||||
| 7000-0010-0001 | Migration | TODO | Multiple (final integration) |
|
||||
|
||||
## Critical Path
|
||||
|
||||
The minimum path to a working router:
|
||||
|
||||
1. **7000-0001-0001** → Skeleton
|
||||
2. **7000-0001-0002** → Common models
|
||||
3. **7000-0002-0001** → InMemory transport
|
||||
4. **7000-0003-0001** → SDK core
|
||||
5. **7000-0003-0002** → SDK handlers
|
||||
6. **7000-0004-0001** → Gateway core
|
||||
7. **7000-0004-0002** → Gateway middleware
|
||||
8. **7000-0004-0003** → Gateway connections
|
||||
|
||||
After these 8 sprints, you have a working router with InMemory transport for testing.
|
||||
|
||||
## Isolation Strategy
|
||||
|
||||
The router is developed in isolation using:
|
||||
|
||||
1. **Separate solution file:** `StellaOps.Router.sln`
|
||||
2. **Dedicated directories:** All router code in new directories
|
||||
3. **No changes to existing modules:** Until migration sprint
|
||||
4. **InMemory transport first:** No network dependencies during core development
|
||||
|
||||
This ensures:
|
||||
- Router development doesn't impact existing StellaOps builds
|
||||
- Agents can work independently on router without merge conflicts
|
||||
- Full testing possible without real infrastructure
|
||||
- Migration is a conscious, controlled step
|
||||
|
||||
## Agent Assignment Guidance
|
||||
|
||||
For maximum parallelization:
|
||||
- **Foundation Agent:** Sprints 7000-0001-0001, 7000-0001-0002
|
||||
- **SDK Agent:** Sprints 7000-0003-0001, 7000-0003-0002
|
||||
- **Gateway Agent:** Sprints 7000-0004-0001, 7000-0004-0002, 7000-0004-0003
|
||||
- **Transport Agent:** Sprints 7000-0002-0001, 7000-0006-*
|
||||
- **Protocol Agent:** Sprints 7000-0005-*
|
||||
- **Config Agent:** Sprints 7000-0007-*
|
||||
- **Integration Agent:** Sprints 7000-0008-*, 7000-0010-0001
|
||||
- **Documentation Agent:** Sprint 7000-0009-0001
|
||||
|
||||
## Invariants (Never Violate)
|
||||
|
||||
From `specs.md`, these are non-negotiable:
|
||||
- **Method + Path** is the endpoint identity
|
||||
- **Strict semver** for version matching
|
||||
- **Region from GatewayNodeConfig.Region** (never from headers/host)
|
||||
- **No HTTP transport** between gateway and microservices
|
||||
- **RequiringClaims** (not AllowedRoles) for authorization
|
||||
- **Opaque body handling** (router doesn't interpret payloads)
|
||||
|
||||
Any change to these invariants requires updating `specs.md` first.
|
||||
Reference in New Issue
Block a user