git.stella-ops.org/docs/router/SPRINT_7000_0005_0004_streaming.md

Sprint 7000-0005-0004 · Protocol Features · Streaming Support

Topic & Scope

Implement streaming request/response support. Large payloads stream through the gateway as REQUEST_STREAM_DATA and RESPONSE_STREAM_DATA frames rather than being fully buffered.

Goal: Enable large file uploads/downloads without memory exhaustion at gateway.

Working directories:

• src/Gateway/StellaOps.Gateway.WebService/ (streaming dispatch)
• src/__Libraries/StellaOps.Microservice/ (streaming handlers)
• src/__Libraries/StellaOps.Router.Transport.InMemory/ (streaming frames)

Dependencies & Concurrency

• Upstream: SPRINT_7000_0005_0003 (cancellation - streaming needs cancel support)
• Downstream: SPRINT_7000_0005_0005 (payload limits)
• Parallel work: None. Sequential.
• Cross-module impact: SDK, Gateway, InMemory transport all modified.

Documentation Prerequisites

• docs/router/specs.md (sections 5.4, 6.3, 7.5 - Streaming requirements)
• docs/router/08-Step.md (streaming section)
• docs/router/implplan.md (phase 8 guidance)

BLOCKED Tasks: Before working on BLOCKED tasks, review ../implplan/BLOCKED_DEPENDENCY_TREE.md for root blockers and dependencies.

Delivery Tracker

#	Task ID	Status	Description	Working Directory
1	STR-001	DONE	Add SupportsStreaming flag to EndpointDescriptor	Common
2	STR-002	DONE	Add streaming attribute support to [StellaEndpoint]	Common
3	STR-010	DONE	Implement REQUEST_STREAM_DATA frame handling in transport	InMemory
4	STR-011	DONE	Implement RESPONSE_STREAM_DATA frame handling in transport	InMemory
5	STR-012	DONE	Implement end-of-stream signaling	InMemory
6	STR-020	DONE	Implement streaming request dispatch in gateway	Gateway
7	STR-021	DONE	Pipe HTTP body stream → REQUEST_STREAM_DATA frames	Gateway
8	STR-022	DONE	Implement chunking for stream data	Configurable chunk size
9	STR-023	DONE	Honor cancellation during streaming	Gateway
10	STR-030	DONE	Implement streaming response handling in gateway	Gateway
11	STR-031	DONE	Pipe RESPONSE_STREAM_DATA frames → HTTP response	Gateway
12	STR-032	DONE	Set chunked transfer encoding	Gateway
13	STR-040	DONE	Implement streaming body in RawRequestContext	Microservice
14	STR-041	DONE	Expose Body as async-readable stream	Microservice
15	STR-042	DONE	Implement backpressure (slow consumer)	Microservice
16	STR-050	DONE	Implement streaming response writing	Microservice
17	STR-051	DONE	Expose WriteBodyAsync for streaming output	Microservice
18	STR-052	DONE	Chunk output into RESPONSE_STREAM_DATA frames	Microservice
19	STR-060	DONE	Implement IRawStellaEndpoint streaming pattern	Microservice
20	STR-061	DONE	Document streaming handler guidelines	Docs
21	STR-070	DONE	Write integration tests for upload streaming
22	STR-071	DONE	Write integration tests for download streaming
23	STR-072	DONE	Write tests for cancellation during streaming

Streaming Frame Protocol

Request Streaming

Gateway → Microservice:
1. REQUEST frame (headers, method, path, CorrelationId)
2. REQUEST_STREAM_DATA frame (chunk 1)
3. REQUEST_STREAM_DATA frame (chunk 2)
...
N. REQUEST_STREAM_DATA frame (final chunk, EndOfStream=true)

Response Streaming

Microservice → Gateway:
1. RESPONSE frame (status code, headers, CorrelationId)
2. RESPONSE_STREAM_DATA frame (chunk 1)
3. RESPONSE_STREAM_DATA frame (chunk 2)
...
N. RESPONSE_STREAM_DATA frame (final chunk, EndOfStream=true)

StreamDataPayload

public sealed class StreamDataPayload
{
    public Guid CorrelationId { get; init; }
    public byte[] Data { get; init; } = Array.Empty<byte>();
    public bool EndOfStream { get; init; }
    public int SequenceNumber { get; init; }
}

Gateway Streaming Dispatch

// In TransportDispatchMiddleware
if (endpoint.SupportsStreaming)
{
    await DispatchStreamingAsync(context, transport, decision, cancellationToken);
}
else
{
    await DispatchBufferedAsync(context, transport, decision, cancellationToken);
}

private async Task DispatchStreamingAsync(...)
{
    // Send REQUEST header
    var requestFrame = BuildRequestHeaderFrame(context);
    await transport.SendFrameAsync(connection, requestFrame, ct);

    // Stream body chunks
    var buffer = new byte[_options.StreamChunkSize];
    int bytesRead;
    int sequence = 0;

    while ((bytesRead = await context.Request.Body.ReadAsync(buffer, ct)) > 0)
    {
        var streamFrame = new Frame
        {
            Type = FrameType.RequestStreamData,
            CorrelationId = requestFrame.CorrelationId,
            Payload = SerializeStreamData(buffer[..bytesRead], sequence++, endOfStream: false)
        };
        await transport.SendFrameAsync(connection, streamFrame, ct);
    }

    // Send end-of-stream
    var endFrame = new Frame
    {
        Type = FrameType.RequestStreamData,
        CorrelationId = requestFrame.CorrelationId,
        Payload = SerializeStreamData(Array.Empty<byte>(), sequence, endOfStream: true)
    };
    await transport.SendFrameAsync(connection, endFrame, ct);

    // Receive response (streaming or buffered)
    await ReceiveResponseAsync(context, transport, connection, requestFrame.CorrelationId, ct);
}

Microservice Streaming Handler

[StellaEndpoint("POST", "/files/upload", SupportsStreaming = true)]
public class FileUploadEndpoint : IRawStellaEndpoint
{
    public async Task<RawResponse> HandleAsync(RawRequestContext context, CancellationToken ct)
    {
        // Body is a stream that reads from REQUEST_STREAM_DATA frames
        var tempPath = Path.GetTempFileName();

        await using var fileStream = File.Create(tempPath);
        await context.Body.CopyToAsync(fileStream, ct);

        return RawResponse.Ok($"Uploaded {fileStream.Length} bytes");
    }
}

[StellaEndpoint("GET", "/files/{id}/download", SupportsStreaming = true)]
public class FileDownloadEndpoint : IRawStellaEndpoint
{
    public async Task<RawResponse> HandleAsync(RawRequestContext context, CancellationToken ct)
    {
        var fileId = context.PathParameters["id"];
        var filePath = _storage.GetPath(fileId);

        // Return streaming response
        return new RawResponse
        {
            StatusCode = 200,
            Body = File.OpenRead(filePath), // Stream, not buffered
            Headers = new HeaderCollection
            {
                ["Content-Type"] = "application/octet-stream"
            }
        };
    }
}

StreamingOptions

public sealed class StreamingOptions
{
    public int ChunkSize { get; set; } = 64 * 1024; // 64KB default
    public int MaxConcurrentStreams { get; set; } = 100;
    public TimeSpan StreamIdleTimeout { get; set; } = TimeSpan.FromMinutes(5);
}

Exit Criteria

Before marking this sprint DONE:

1. REQUEST_STREAM_DATA frames implemented in transport
2. RESPONSE_STREAM_DATA frames implemented in transport
3. Gateway streams request body to microservice
4. Gateway streams response body to HTTP client
5. SDK exposes streaming Body in RawRequestContext
6. SDK can write streaming response
7. Cancellation works during streaming
8. Integration tests for upload and download streaming

Execution Log

Date (UTC)	Update	Owner
2025-12-05	Sprint DONE - StreamDataPayload, StreamingOptions, StreamingRequestBodyStream, StreamingResponseBodyStream, DispatchStreamingAsync in gateway, 80 tests pass	Claude

Decisions & Risks

• Default chunk size: 64KB (tunable)
• End-of-stream is explicit frame, not connection close
• Backpressure via channel capacity (bounded channels)
• Idle timeout cancels stuck streams
• Typed handlers don't support streaming (use IRawStellaEndpoint)