git.stella-ops.org/docs/router/07-Step.md

For this step you’re wiring request cancellation end‑to‑end in the InMemory setup:

Client / gateway gives up → gateway sends CANCEL → microservice cancels handler

No need to mix in streaming or payload limits yet; just enforce cancellation for timeouts and client disconnects.

---

0. Preconditions

Have in place:

• FrameType.Cancel in StellaOps.Router.Common.FrameType.
• ITransportClient.SendCancelAsync(ConnectionState, Guid, string?) in Common.
• Minimal InMemory path from HTTP → gateway → microservice (HELLO + REQUEST/RESPONSE) working.

If FrameType.Cancel or SendCancelAsync aren’t there yet, add them first.

---

1. Common: cancel payload (optional, but useful)

If you want reasons attached, add a DTO in Common:

public sealed class CancelPayload
{
    public string Reason { get; init; } = string.Empty; // eg: "ClientDisconnected", "Timeout"
}

You’ll serialize this into Frame.Payload when sending a CANCEL. If you don’t care about reasons yet, you can skip the payload and just use the correlation id.

No behavior in Common, just the shape.

---

2. Gateway: trigger CANCEL on client abort and timeout

2.1 Extend TransportDispatchMiddleware

You already:

• Generate a correlationId.
• Build a FrameType.Request.
• Call ITransportClient.SendRequestAsync(...) and await it.

Now:

1. Create a linked CTS that combines:

   • HttpContext.RequestAborted
   • The endpoint timeout

2. Register a callback on RequestAborted that sends a CANCEL with the same correlationId.

3. On OperationCanceledException where the HTTP token is not canceled (pure timeout), send a CANCEL once and return 504.

Sketch:

public async Task Invoke(HttpContext context, ITransportClient transportClient)
{
    var decision = (RoutingDecision)context.Items[RouterHttpContextKeys.RoutingDecision]!;
    var correlationId = Guid.NewGuid();

    // build requestFrame as before

    var timeout = decision.EffectiveTimeout;
    using var linkedCts = CancellationTokenSource.CreateLinkedTokenSource(context.RequestAborted);
    linkedCts.CancelAfter(timeout);

    // fire-and-forget cancel on client disconnect
    context.RequestAborted.Register(() =>
    {
        _ = transportClient.SendCancelAsync(
            decision.Connection, correlationId, "ClientDisconnected");
    });

    Frame responseFrame;
    try
    {
        responseFrame = await transportClient.SendRequestAsync(
            decision.Connection,
            requestFrame,
            timeout,
            linkedCts.Token);
    }
    catch (OperationCanceledException) when (!context.RequestAborted.IsCancellationRequested)
    {
        // internal timeout
        await transportClient.SendCancelAsync(
            decision.Connection, correlationId, "Timeout");

        context.Response.StatusCode = StatusCodes.Status504GatewayTimeout;
        await context.Response.WriteAsync("Upstream timeout");
        return;
    }

    // existing response mapping goes here
}

Key points:

• The gateway sends CANCEL as soon as:

  • The client disconnects (RequestAborted).
  • Or the internal timeout triggers (catch branch).

• We do not need any global correlation registry on the gateway side; the middleware has the correlationId and Connection.

---

3. InMemory transport: propagate CANCEL to microservice

3.1 Implement SendCancelAsync in InMemoryTransportClient (gateway side)

In your gateway InMemory implementation:

public Task SendCancelAsync(ConnectionState connection, Guid correlationId, string? reason = null)
{
    var payload = reason is null
        ? Array.Empty<byte>()
        : SerializeCancelPayload(new CancelPayload { Reason = reason });

    var frame = new Frame
    {
        Type = FrameType.Cancel,
        CorrelationId = correlationId,
        Payload = payload
    };

    return _hub.SendFromGatewayAsync(connection.ConnectionId, frame, CancellationToken.None);
}

_hub.SendFromGatewayAsync must route the frame to the microservice’s receive loop for that connection.

3.2 Hub routing

Ensure your IInMemoryRouterHub implementation:

• When SendFromGatewayAsync(connectionId, cancelFrame, ct) is called:

  • Enqueues that frame onto the microservice’s incoming channel (GetFramesForMicroserviceAsync stream).

No extra logic; just treat CANCEL like REQUEST/HELLO in terms of delivery.

---

4. Microservice: track in-flight requests

Now microservice needs to know which request to cancel when a CANCEL arrives.

4.1 In-flight registry

In the microservice connection class (the one doing the receive loop):

private readonly ConcurrentDictionary<Guid, RequestExecution> _inflight =
    new();

private sealed class RequestExecution
{
    public CancellationTokenSource Cts { get; init; } = default!;
    public Task ExecutionTask { get; init; } = default!;
}

When a Request frame arrives:

• Create a CancellationTokenSource.
• Start the handler using that token.
• Store both in _inflight.

Example pattern in ReceiveLoopAsync:

private async Task ReceiveLoopAsync(CancellationToken ct)
{
    await foreach (var frame in _routerClient.GetIncomingFramesAsync(ct))
    {
        switch (frame.Type)
        {
            case FrameType.Request:
                HandleRequest(frame);
                break;

            case FrameType.Cancel:
                HandleCancel(frame);
                break;

            // other frame types...
        }
    }
}

private void HandleRequest(Frame frame)
{
    var cts = new CancellationTokenSource();
    var linkedCts = CancellationTokenSource.CreateLinkedTokenSource(cts.Token); // later link to global shutdown if needed

    var exec = new RequestExecution
    {
        Cts = cts,
        ExecutionTask = HandleRequestCoreAsync(frame, linkedCts.Token)
    };

    _inflight[frame.CorrelationId] = exec;

    _ = exec.ExecutionTask.ContinueWith(_ =>
    {
        _inflight.TryRemove(frame.CorrelationId, out _);
        cts.Dispose();
        linkedCts.Dispose();
    }, TaskScheduler.Default);
}

4.2 Wire CancellationToken into dispatcher

HandleRequestCoreAsync should:

• Deserialize the request payload.

• Build a RawRequestContext with CancellationToken = token.

• Pass that token through to:

  • IRawStellaEndpoint.HandleAsync(context) (via the context).
  • Or typed handler adapter (IStellaEndpoint<,> / IStellaEndpoint<TResponse>), passing it explicitly.

Example pattern:

private async Task HandleRequestCoreAsync(Frame frame, CancellationToken ct)
{
    var req = DeserializeRequestPayload(frame.Payload);

    if (!_catalog.TryGetHandler(req.Method, req.Path, out var registration))
    {
        var notFound = BuildNotFoundResponse(frame.CorrelationId);
        await _routerClient.SendFrameAsync(notFound, ct);
        return;
    }

    using var bodyStream = new MemoryStream(req.Body); // minimal case

    var ctx = new RawRequestContext
    {
        Method = req.Method,
        Path = req.Path,
        Headers = req.Headers,
        Body = bodyStream,
        CancellationToken = ct
    };

    var handler = (IRawStellaEndpoint)_serviceProvider.GetRequiredService(registration.HandlerType);

    var response = await handler.HandleAsync(ctx);

    var respFrame = BuildResponseFrame(frame.CorrelationId, response);
    await _routerClient.SendFrameAsync(respFrame, ct);
}

Now each handler sees a token that will be canceled when a CANCEL frame arrives.

4.3 Handle CANCEL frames

When a Cancel frame arrives:

private void HandleCancel(Frame frame)
{
    if (_inflight.TryGetValue(frame.CorrelationId, out var exec))
    {
        exec.Cts.Cancel();
    }
    // Ignore if not found (e.g. already completed)
}

If you care about the reason, deserialize CancelPayload and log it; not required for behavior.

---

5. Handler guidance (for your Microservice docs)

In Stella Ops Router – Microservice.md, add simple rules devs must follow:

• Any long‑running or IO-heavy code in endpoints MUST:

  • Accept a CancellationToken (for typed endpoints).
  • Or use RawRequestContext.CancellationToken for raw endpoints.

• Always pass the token into:

  • DB calls.
  • File I/O and stream operations.
  • HTTP/gRPC calls to other services.

• Do not swallow OperationCanceledException unless there is a good reason; normally let it bubble or treat it as a normal cancellation.

Concrete example for devs:

[StellaEndpoint("POST", "/billing/slow-operation")]
public sealed class SlowEndpoint : IRawStellaEndpoint
{
    public async Task<RawResponse> HandleAsync(RawRequestContext ctx)
    {
        // Correct: observe token
        await Task.Delay(TimeSpan.FromMinutes(5), ctx.CancellationToken);

        return new RawResponse { StatusCode = 204 };
    }
}

---

6. Tests

6.1 Client abort → CANCEL

Test outline:

• Setup:

  • Gateway + microservice wired via InMemory hub.

  • Microservice endpoint that:

    • Waits on Task.Delay(TimeSpan.FromMinutes(5), ctx.CancellationToken).

• Test:

  1. Start HTTP request to /slow.

  2. After sending request, cancel the client’s HttpClient token or close the connection.

  3. Assert:

     • Gateway’s InMemory transport sent a FrameType.Cancel.
     • Microservice’s handler is canceled (e.g. no longer running after a short time).
     • No response (or partial) is written; HTTP side will produce whatever your test harness expects when client aborts.

6.2 Gateway timeout → CANCEL

• Configure endpoint timeout small (e.g. 100 ms).

• Have endpoint sleep for 5 seconds with the token.

• Assert:

  • Gateway returns 504.
  • Cancel frame was sent.
  • Handler is canceled (task completes early).

These tests lock in the semantics so later additions (real transports, streaming) don’t regress cancellation.

---

7. Done criteria for “Add cancellation semantics (with InMemory)”

You can mark step 7 as complete when:

• For every routed request, the gateway knows its correlationId and connection.

• On client disconnect:

  • Gateway sends a FrameType.Cancel with that correlationId.

• On internal timeout:

  • Gateway sends a FrameType.Cancel and returns 504 to the client.

• InMemory hub delivers CANCEL frames to the microservice.

• Microservice:

  • Tracks in‑flight requests by correlationId.
  • Cancels the proper CancellationTokenSource when CANCEL arrives.
  • Passes the token into handlers via RawRequestContext and typed adapters.

• At least one automated test proves:

  • Cancellation propagates from gateway to microservice and stops the handler.

Once this is done, you’ll be in good shape to add streaming & payload-limits on top, because the cancel path is already wired end‑to‑end.