stella-ops.org/git.stella-ops.org
1
0
Fork 0
Code Issues Pull Requests Actions 2 Releases Wiki Activity

Update docs, sprint plans, and compose configuration

Browse Source 
Add 12 new sprint files (Integrations, Graph, JobEngine, FE, Router,
AdvisoryAI), archive completed scheduler UI sprint, update module
architecture docs (router, graph, jobengine, web, integrations),
and add Gitea entrypoint script for local dev.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
master
2026-04-06 08:53:50 +03:00
parent 8e823792a3
commit 50abd2137f
36 changed files with 1723 additions and 69 deletions
Download Patch File Download Diff File Expand all files Collapse all files

35
docs/modules/router/messaging-valkey-transport.md
View File

@@ -3,7 +3,7 @@
## Status
- **Implemented** in Sprint 8100.0011.0003.
- Core components: Gateway DI wiring, GatewayHostedService integration, GatewayTransportClient dispatch.
- Last updated: 2025-12-24 (UTC).
- Last updated: 2026-04-05 (UTC).
## Purpose
Enable Gateway ↔ microservice Router traffic over an offline-friendly, Redis-compatible transport (Valkey) by using the existing **Messaging** transport layer:
@@ -27,20 +27,28 @@ This supports environments where direct TCP/TLS microservice connections are und
## High-Level Flow
1) Microservice connects via messaging transport:
- publishes a HELLO message to the gateway request queue
- publishes a slim `HELLO` message with instance identity to the gateway control queue
2) Gateway processes HELLO:
- registers instance + endpoints into routing state
3) Gateway routes an HTTP request to a microservice:
- registers the connection identity and requests endpoint metadata replay when needed
3) Microservice answers the replay request:
- publishes an `EndpointsUpdate` frame with endpoints, schemas, and OpenAPI metadata
4) Gateway applies the metadata replay:
- updates routing state, effective claims, and aggregated OpenAPI
5) Gateway routes an HTTP request to a microservice:
- publishes a REQUEST message to the service request queue
4) Microservice handles request:
6) Microservice handles request:
- executes handler (or ASP.NET bridge) and publishes a RESPONSE message
5) Gateway returns response to the client.
7) Gateway returns response to the client.
Messaging-specific recovery behavior:
- Startup resync: the gateway sends `ResyncRequest` immediately after a slim `HELLO`.
- Administrative resync: `POST /api/v1/gateway/administration/router/resync` can request replay for one connection or the whole messaging fleet.
- Gateway-state miss: if a heartbeat arrives for an unknown messaging connection, the gateway seeds minimal state from the heartbeat identity and requests replay instead of waiting for a reconnect.
## Queue Topology (Conceptual)
The Messaging transport uses a small set of queues (names are configurable):
- **Gateway request queue**: receives HELLO / HEARTBEAT / REQUEST frames from services
- **Gateway response queue**: receives RESPONSE frames from services
- **Per-service request queues**: gateway publishes REQUEST frames targeted to a service
- **Gateway control queue**: receives service-to-gateway HELLO / HEARTBEAT / ENDPOINTS_UPDATE / RESPONSE frames
- **Per-service incoming queues**: gateway publishes REQUEST / CANCEL / RESYNC_REQUEST frames targeted to a service
- **Dead letter queues** (optional): for messages that exceed retries/leases
## Configuration
@@ -87,6 +95,8 @@ if (bootstrapOptions.Transports.Messaging.Enabled)
- **At-least-once** delivery: message queues and leases imply retries are possible; handlers should be idempotent where feasible.
- **Lease timeouts**: must be tuned to max handler execution time; long-running tasks should respond with 202 + job id rather than blocking.
- **Determinism**: message ordering may vary; Router must not depend on arrival order for correctness (only for freshness/telemetry).
- **Push-first with recovery fallback**: Valkey Pub/Sub notifications wake consumers immediately when possible. If notifications silently stop, the queue layer still wakes via timeout fallback, connection-restored hooks, and randomized proactive re-subscription so requests and resync control frames do not wedge forever.
- **Queue fallback cost**: every wake can perform `XAUTOCLAIM` plus `XREADGROUP` checks before sleeping again. That traffic is expected resilience overhead, but it is materially smaller than replaying the full endpoint catalog on every heartbeat interval.
## Security Notes
- Messaging transport is internal. External identity must still be enforced at the Gateway.
@@ -97,11 +107,12 @@ if (bootstrapOptions.Transports.Messaging.Enabled)
### Completed (Sprint 8100.0011.0003)
1. ✅ Wire Messaging transport into Gateway:
- start/stop `MessagingTransportServer` in `GatewayHostedService`
- subscribe to `OnHelloReceived`, `OnHeartbeatReceived`, `OnResponseReceived`, `OnConnectionClosed` events
- reuse routing state updates and claims store updates
- start/stop `MessagingTransportServer` in `GatewayHostedService`
- subscribe to `OnHelloReceived`, `OnHeartbeatReceived`, `OnEndpointsUpdated`, `OnResponseReceived`, `OnConnectionClosed` events
- reuse routing state updates and claims store updates
2. ✅ Extend Gateway transport client to support `TransportType.Messaging` for dispatch.
3. ✅ Add config options (`GatewayMessagingTransportOptions`) and DI mappings.
4. ✅ Switch messaging registration from periodic full HELLO replay to explicit `ResyncRequest` / `EndpointsUpdate` control frames.
### Remaining Work
1. Add deployment examples (compose/helm) for Valkey transport.