stella-ops.org/git.stella-ops.org
1
0
Fork 0
Code Issues Pull Requests Actions 2 Releases Wiki Activity
Files
4bdc298ec1fc821ee0d9dda6d2d4643dcedf6b35
git.stella-ops.org/docs/features/unchecked/orchestrator/quota-governance-and-circuit-breakers.md

3.3 KiB
Raw Blame History

Quota Governance and Circuit Breakers

Module

Orchestrator

Status

IMPLEMENTED

Description

Job scheduling exists but dedicated quota governance services and circuit breaker automation were not found as separate implementations. May be embedded in scheduler logic.

What's Implemented

  • Quota (src/Orchestrator/StellaOps.Orchestrator/StellaOps.Orchestrator.Core/Domain/Quota.cs) - quota entity with limits and allocation
  • QuotaEndpoints (src/Orchestrator/StellaOps.Orchestrator/StellaOps.Orchestrator.WebService/Endpoints/QuotaEndpoints.cs) - REST API for quota queries and adjustments
  • QuotaContracts (src/Orchestrator/StellaOps.Orchestrator/StellaOps.Orchestrator.WebService/Contracts/QuotaContracts.cs) - API contracts for quota operations
  • Throttle (src/Orchestrator/StellaOps.Orchestrator/StellaOps.Orchestrator.Core/Domain/Throttle.cs) - throttle configuration for rate limiting
  • AdaptiveRateLimiter (src/Orchestrator/StellaOps.Orchestrator/StellaOps.Orchestrator.Core/RateLimiting/AdaptiveRateLimiter.cs) - adaptive rate limiting based on system load
  • ConcurrencyLimiter (src/Orchestrator/StellaOps.Orchestrator/StellaOps.Orchestrator.Core/RateLimiting/ConcurrencyLimiter.cs) - limits concurrent job execution
  • BackpressureHandler (src/Orchestrator/StellaOps.Orchestrator/StellaOps.Orchestrator.Core/RateLimiting/BackpressureHandler.cs) - backpressure signaling
  • LoadShedder (src/Orchestrator/StellaOps.Orchestrator/StellaOps.Orchestrator.Core/Scale/LoadShedder.cs) - load shedding under saturation
  • PostgresQuotaRepository (src/Orchestrator/StellaOps.Orchestrator/StellaOps.Orchestrator.Infrastructure/Postgres/PostgresQuotaRepository.cs) - Postgres-backed quota storage
  • PostgresThrottleRepository (src/Orchestrator/StellaOps.Orchestrator/StellaOps.Orchestrator.Infrastructure/Postgres/PostgresThrottleRepository.cs) - Postgres-backed throttle storage

What's Missing

  • Dedicated quota governance service: No standalone QuotaGovernanceService enforcing cross-tenant quota allocation, burst capacity, and fair scheduling across tenants
  • Circuit breaker automation: No automated circuit breaker that opens when a downstream service (e.g., scanner, attestor) fails repeatedly, preventing cascade failures across orchestrator jobs
  • Quota allocation policies: No configurable policies for quota allocation (e.g., proportional allocation, priority-based allocation, reserved capacity)
  • Circuit breaker dashboard: No UI showing circuit breaker states for each downstream service
  • Quota usage alerts: No alerting when tenants approach their quota limits via Notifier integration
  • Circuit breaker state persistence: No persistent storage for circuit breaker state across orchestrator restarts

Implementation Plan

  • Create QuotaGovernanceService enforcing cross-tenant allocation policies
  • Implement circuit breaker pattern for downstream services (scanner, attestor, policy engine)
  • Add configurable quota allocation policies (proportional, priority-based)
  • Add circuit breaker state persistence in PostgreSQL
  • Build circuit breaker dashboard UI component
  • Add quota usage alerting via Notifier integration
  • Add tests for quota governance, circuit breaker state transitions, and allocation policies

Related Documentation

  • Source: See feature catalog