stella-ops.org/git.stella-ops.org
1
0
Fork 0
Code Issues Pull Requests Actions 2 Releases Wiki Activity
Files
cf5b72974f923126944f4dcd47e85864ebb31a3b
git.stella-ops.org/docs/features/unchecked/concelier/ingestion-telemetry-and-orchestration.md

2.1 KiB
Raw Blame History

Ingestion Telemetry and Orchestration

Module

Concelier

Status

IMPLEMENTED

Description

Telemetry instrumentation for ingestion pipeline with OpenTelemetry metrics and orchestration registry for connector management.

Implementation Details

  • Modules: src/Concelier/__Libraries/StellaOps.Concelier.Core/Orchestration/, src/Concelier/StellaOps.Concelier.WebService/Diagnostics/
  • Key Classes:
    • ConnectorWorker (src/Concelier/__Libraries/StellaOps.Concelier.Core/Orchestration/ConnectorWorker.cs) - orchestrates connector ingestion cycles with telemetry hooks
    • ConnectorWorkerFactory (src/Concelier/__Libraries/StellaOps.Concelier.Core/Orchestration/ConnectorWorker.cs) - factory for creating connector worker instances
    • ConnectorRegistrationService (src/Concelier/__Libraries/StellaOps.Concelier.Core/Orchestration/ConnectorRegistrationService.cs) - connector discovery and registration
    • ConnectorMetadata (src/Concelier/__Libraries/StellaOps.Concelier.Core/Orchestration/ConnectorMetadata.cs) - metadata model for registered connectors
    • IngestionMetrics (src/Concelier/StellaOps.Concelier.WebService/Diagnostics/IngestionMetrics.cs) - OpenTelemetry metrics for ingestion operations
  • Interfaces: IConnectorWorker, IConnectorWorkerFactory, IConnectorRegistrationService
  • Registration: OrchestrationServiceCollectionExtensions (src/Concelier/__Libraries/StellaOps.Concelier.Core/Orchestration/OrchestrationServiceCollectionExtensions.cs)
  • Source: Feature matrix scan

E2E Test Plan

  • Start the Concelier web service and verify all registered connectors appear in ConnectorRegistrationService
  • Trigger a connector ingestion cycle and verify IngestionMetrics emits advisory count, duration, and error metrics
  • Verify ConnectorWorkerFactory creates workers with correct connector metadata
  • Verify orchestration: trigger multiple connectors and verify they execute according to their configured schedules
  • Verify OpenTelemetry export: confirm ingestion metrics are visible in the configured OTel collector