stella-ops.org/git.stella-ops.org
1
0
Fork 0
Code Issues Pull Requests Actions 2 Releases Wiki Activity
Files
main
git.stella-ops.org/docs/modules/integrations/architecture.md

8.9 KiB
Raw Permalink Blame History

Integrations Architecture

Technical architecture for the central integration catalog and connector hub.

Overview

The Integrations module provides a unified service for managing connections to external tools (SCM providers, container registries, CI systems). It uses a plugin-based architecture where each external provider is implemented as a connector plugin, enabling new integrations to be added without modifying core logic. All integration state is persisted in PostgreSQL with tenant-scoped isolation.

Design Principles

  1. Plugin extensibility - New providers are added as plugins, not core code changes
  2. Credential indirection - Secrets are referenced via AuthRef URIs pointing to external vaults; the integration catalog never stores raw credentials
  3. Tenant isolation - Every integration record is scoped to a tenant; queries are filtered at the data layer
  4. Health-first - Every integration has a health check contract; unhealthy integrations surface alerts

Components

Integrations/
├── StellaOps.Integrations.WebService/       # HTTP API (Minimal API)
├── StellaOps.Integrations.Core/             # Business logic and plugin orchestration
├── StellaOps.Integrations.Contracts/        # Shared DTOs and interfaces
├── StellaOps.Integrations.Persistence/      # PostgreSQL via IntegrationDbContext
├── Plugins/
│   ├── StellaOps.Integrations.GitHubApp/    # GitHub App connector
│   ├── StellaOps.Integrations.GitLab/       # GitLab connector
│   ├── StellaOps.Integrations.Harbor/       # Harbor registry connector
│   └── StellaOps.Integrations.InMemory/     # In-memory connector for testing
└── __Tests/
    └── StellaOps.Integrations.Tests/        # Unit and integration tests

Data Flow

[External Tool] <── health check ── [Integrations WebService]
                                           │
[Client Module] ── GET /integrations ──>   │
                                           │
[Admin/CLI] ── POST /integrations ──>      │ ── [IntegrationDbContext] ── [PostgreSQL]
                                           │
[Plugin Loader] ── discover plugins ──>    │ ── [Plugin Framework]
  1. Registration: An administrator or automated onboarding flow creates an integration record via POST /integrations, providing the provider type, configuration JSON, and an AuthRef URI for credentials.
  2. Discovery: The PluginLoader scans configured paths for available connector plugins and registers their capabilities in the plugin metadata table.
  3. Health checking: A background timer invokes each integration's health check endpoint (provider-specific) and persists the result. Failures update the integration status and emit alerts.
  4. Consumption: Downstream modules (Scanner, Orchestrator, Signals) query GET /integrations/{type} to retrieve connection details for a specific provider type, filtered by tenant.

Database Schema

Database: stellaops_integrations (PostgreSQL)

Table Purpose
integrations Primary catalog of registered integrations (id, tenant_id, type, name, config_json, auth_ref_uri, status, created_at, updated_at)
plugin_metadata Discovered plugin descriptors (plugin_id, type, version, capabilities, path)
health_checks Health check history (integration_id, checked_at, status, latency_ms, error_message)
audit_logs Audit trail for integration CRUD operations (actor, action, integration_id, timestamp, details)

Endpoints

Integration CRUD (/api/v1/integrations)

  • GET / - List integrations for current tenant (optional ?type= filter)
  • GET /{id} - Get integration by ID
  • POST / - Register a new integration (body: type, name, config, authRefUri)
  • PUT /{id} - Update integration configuration
  • DELETE /{id} - Remove integration (soft delete with audit)

Health (/api/v1/integrations)

  • POST /{id}/health-check - Trigger an on-demand health check for a specific integration
  • GET /{id}/health - Retrieve latest health status and history

Type-filtered queries

  • GET /by-type/{type} - List integrations of a specific provider type (e.g., github, harbor)

Plugin discovery

  • POST /plugins/discover - Trigger plugin discovery scan and register new connectors

Plugin Architecture

Each connector plugin implements a standard interface:

public interface IIntegrationPlugin
{
    string ProviderType { get; }
    Task<IntegrationHealthResult> CheckHealthAsync(IntegrationConfig config, CancellationToken ct);
    Task<bool> ValidateConfigAsync(JsonDocument config, CancellationToken ct);
}

Built-in plugins:

  • GitHubApp - GitHub App installation authentication, repository listing, webhook setup
  • GitLab - Personal/project access token authentication, project discovery
  • Harbor - Robot account authentication, project and repository enumeration
  • InMemory - Deterministic test double for integration tests and offline development

Provider endpoint contracts

  • GitHub App - Operators provide either the GitHub Cloud root (https://github.com), a GitHub Enterprise Server root, or an explicit /api/v3 base. The connector normalizes the endpoint to a single API root and probes relative app / rate_limit paths so GitHub Enterprise onboarding never falls back to origin-root /app.
  • Harbor - Operators provide the Harbor base URL. Stella Ops probes the provider-specific /api/v2.0/health route for connection tests and health checks.

Security Considerations

  • AuthRef URI credential model: Credentials are stored in an external vault (e.g., HashiCorp Vault, Azure Key Vault). The integration catalog stores only the URI reference (authref://vault/path/to/secret), never the raw secret.
  • Tenant scoping: All database queries include a mandatory tenant filter enforced at the DbContext level via query filters.
  • Audit logging: Every create, update, and delete operation is recorded in the audit log with actor identity, timestamp, and change details.
  • Plugin sandboxing: Connector plugins run within the Plugin Framework's trust boundary; untrusted plugins are process-isolated.

Observability

  • Metrics: integration_health_status{type,tenant}, integration_health_latency_ms, integration_crud_total{action}
  • Logs: Structured logs with integrationId, tenantId, providerType, action
  • Traces: Spans for health checks, plugin discovery, and CRUD operations

Performance Characteristics

  • Health checks run on a configurable interval (default: 5 minutes) per integration
  • Plugin discovery is triggered on startup and on-demand; results are cached
  • Integration queries use indexed tenant_id + type columns for fast filtering

IDE Extensions (VS Code, JetBrains)

The Integrations module also owns the IDE extension plugins, located under src/Integrations/__Extensions/. These are non-.NET projects that provide developer-facing tooling consuming the same Orchestrator/Router APIs as other integrations.

VS Code Extension (__Extensions/vscode-stella-ops/)

  • Technology: TypeScript, VS Code Extension API
  • Build: npm run compile (TypeScript compilation)
  • Features: Tree views for releases and environments, CodeLens annotations for stella.yaml, command palette integration, status bar widget
  • Manifest: package.json (extension manifest, commands, views, configuration)

JetBrains Plugin (__Extensions/jetbrains-stella-ops/)

  • Technology: Kotlin, IntelliJ Platform SDK
  • Build: Gradle (./gradlew build)
  • Features: Tool windows (Releases/Environments/Deployments tabs), YAML annotator, action menus, status bar widget
  • Entry point: StellaOpsPlugin.kt

Design Principles (Extensions)

  1. Thin client - Extensions contain no business logic; all state and decisions live in backend services
  2. Consistent experience - Both plugins expose equivalent functionality despite different technology stacks
  3. Non-blocking - All API calls are asynchronous; the IDE remains responsive during network operations
  4. Offline-tolerant - Graceful degradation when the Stella Ops backend is unreachable

Data Flow (Extensions)

[Developer IDE] --> [Extension/Plugin]
                         |
                         +-- GET /api/v1/releases/* --------> [Orchestrator API]
                         +-- GET /api/v1/environments/* ----> [Orchestrator API]
                         +-- POST /api/v1/promotions/* -----> [Orchestrator API]
                         +-- POST /oauth/token -------------> [Authority]

Authentication uses OAuth tokens obtained from the Authority service, stored in the IDE's secure credential store (VS Code SecretStorage, JetBrains PasswordSafe).

References