Files
git.stella-ops.org/docs/modules/orchestrator
StellaOps Bot e923880694
Some checks failed
AOC Guard CI / aoc-guard (push) Has been cancelled
AOC Guard CI / aoc-verify (push) Has been cancelled
Concelier Attestation Tests / attestation-tests (push) Has been cancelled
Docs CI / lint-and-preview (push) Has been cancelled
Export Center CI / export-ci (push) Has been cancelled
Mirror Thin Bundle Sign & Verify / mirror-sign (push) Has been cancelled
feat: Add DigestUpsertRequest and LockEntity models
- Introduced DigestUpsertRequest for handling digest upsert requests with properties like ChannelId, Recipient, DigestKey, Events, and CollectUntil.
- Created LockEntity to represent a lightweight distributed lock entry with properties such as Id, TenantId, Resource, Owner, ExpiresAt, and CreatedAt.

feat: Implement ILockRepository interface and LockRepository class

- Defined ILockRepository interface with methods for acquiring and releasing locks.
- Implemented LockRepository class with methods to try acquiring a lock and releasing it, using SQL for upsert operations.

feat: Add SurfaceManifestPointer record for manifest pointers

- Introduced SurfaceManifestPointer to represent a minimal pointer to a Surface.FS manifest associated with an image digest.

feat: Create PolicySimulationInputLock and related validation logic

- Added PolicySimulationInputLock record to describe policy simulation inputs and expected digests.
- Implemented validation logic for policy simulation inputs, including checks for digest drift and shadow mode requirements.

test: Add unit tests for ReplayVerificationService and ReplayVerifier

- Created ReplayVerificationServiceTests to validate the behavior of the ReplayVerificationService under various scenarios.
- Developed ReplayVerifierTests to ensure the correctness of the ReplayVerifier logic.

test: Implement PolicySimulationInputLockValidatorTests

- Added tests for PolicySimulationInputLockValidator to verify the validation logic against expected inputs and conditions.

chore: Add cosign key example and signing scripts

- Included a placeholder cosign key example for development purposes.
- Added a script for signing Signals artifacts using cosign with support for both v2 and v3.

chore: Create script for uploading evidence to the evidence locker

- Developed a script to upload evidence to the evidence locker, ensuring required environment variables are set.
2025-12-03 07:51:50 +02:00
..
2025-11-21 06:56:36 +00:00

StellaOps Source & Job Orchestrator

The Orchestrator schedules, observes, and recovers ingestion and analysis jobs across the StellaOps platform.

Latest updates (2025-11-30)

  • OpenAPI discovery published at /.well-known/openapi with openapi/orchestrator.json; includes pagination/idempotency/error-envelope examples and version headers.
  • Legacy job detail/summary endpoints now emit Deprecation + Link headers pointing to the stable replacements.
  • Job leasing flows through the Task Runner bridge: allocations carry idempotency keys, lease durations, and retry hints; workers acknowledge via claim/ack and emit heartbeats.
  • Event envelopes remain interim pending ORCH-SVC-37-101; include provenance (tenant/project, job type, correlationId, task runner id) in all notifier events.
  • Authority orch:quota / orch:backfill scopes require reason/ticket audit fields; include them in runbooks and dashboard overrides.

Responsibilities

  • Track job state, throughput, and errors for Concelier, Excititor, Scheduler, and export pipelines.
  • Expose dashboards and APIs for throttling, replays, and failover.
  • Enforce rate-limits, concurrency and dependency chains across queues.
  • Stream structured events and audit logs for incident response.
  • Provide Task Runner bridge semantics (claim/ack, heartbeats, progress, artifacts, backfills) for Go/Python SDKs.

Key components

  • Orchestrator WebService (control plane).
  • Queue adapters (Redis/NATS) and job ledger.
  • Console dashboard module and CLI integration for operators.

Integrations & dependencies

  • Authority for authN/Z on operational actions.
  • Telemetry stack for job metrics and alerts.
  • Scheduler/Concelier/Excititor workers for job lifecycle.
  • Offline Kit for state export/import during air-gap refreshes.

Operational notes

  • Job recovery runbooks and dashboard JSON as described in Epic 9.
  • Rate-limit and lease reconfiguration guidelines; keep lease defaults aligned across runners and SDKs (Go/Python).
  • Log streaming: SSE/WS endpoints carry correlationId + tenant/project; buffer size and retention must be documented in runbooks.
  • When using orch:quota / orch:backfill scopes, capture reason/ticket fields in runbooks and audit checklists.

Epic alignment

  • Epic 9: Source & Job Orchestrator Dashboard.
  • ORCH stories in ../../TASKS.md.