166 lines
8.9 KiB
Markdown
166 lines
8.9 KiB
Markdown
# Integrations Architecture
|
|
|
|
> Technical architecture for the central integration catalog and connector hub.
|
|
|
|
## Overview
|
|
|
|
The Integrations module provides a unified service for managing connections to external tools (SCM providers, container registries, CI systems). It uses a plugin-based architecture where each external provider is implemented as a connector plugin, enabling new integrations to be added without modifying core logic. All integration state is persisted in PostgreSQL with tenant-scoped isolation.
|
|
|
|
## Design Principles
|
|
|
|
1. **Plugin extensibility** - New providers are added as plugins, not core code changes
|
|
2. **Credential indirection** - Secrets are referenced via AuthRef URIs pointing to external vaults; the integration catalog never stores raw credentials
|
|
3. **Tenant isolation** - Every integration record is scoped to a tenant; queries are filtered at the data layer
|
|
4. **Health-first** - Every integration has a health check contract; unhealthy integrations surface alerts
|
|
|
|
## Components
|
|
|
|
```
|
|
Integrations/
|
|
├── StellaOps.Integrations.WebService/ # HTTP API (Minimal API)
|
|
├── StellaOps.Integrations.Core/ # Business logic and plugin orchestration
|
|
├── StellaOps.Integrations.Contracts/ # Shared DTOs and interfaces
|
|
├── StellaOps.Integrations.Persistence/ # PostgreSQL via IntegrationDbContext
|
|
├── Plugins/
|
|
│ ├── StellaOps.Integrations.GitHubApp/ # GitHub App connector
|
|
│ ├── StellaOps.Integrations.GitLab/ # GitLab connector
|
|
│ ├── StellaOps.Integrations.Harbor/ # Harbor registry connector
|
|
│ └── StellaOps.Integrations.InMemory/ # In-memory connector for testing
|
|
└── __Tests/
|
|
└── StellaOps.Integrations.Tests/ # Unit and integration tests
|
|
```
|
|
|
|
## Data Flow
|
|
|
|
```
|
|
[External Tool] <── health check ── [Integrations WebService]
|
|
│
|
|
[Client Module] ── GET /integrations ──> │
|
|
│
|
|
[Admin/CLI] ── POST /integrations ──> │ ── [IntegrationDbContext] ── [PostgreSQL]
|
|
│
|
|
[Plugin Loader] ── discover plugins ──> │ ── [Plugin Framework]
|
|
```
|
|
|
|
1. **Registration:** An administrator or automated onboarding flow creates an integration record via `POST /integrations`, providing the provider type, configuration JSON, and an AuthRef URI for credentials.
|
|
2. **Discovery:** The `PluginLoader` scans configured paths for available connector plugins and registers their capabilities in the plugin metadata table.
|
|
3. **Health checking:** A background timer invokes each integration's health check endpoint (provider-specific) and persists the result. Failures update the integration status and emit alerts.
|
|
4. **Consumption:** Downstream modules (Scanner, Orchestrator, Signals) query `GET /integrations/{type}` to retrieve connection details for a specific provider type, filtered by tenant.
|
|
|
|
## Database Schema
|
|
|
|
Database: `stellaops_integrations` (PostgreSQL)
|
|
|
|
| Table | Purpose |
|
|
|-------|---------|
|
|
| `integrations` | Primary catalog of registered integrations (id, tenant_id, type, name, config_json, auth_ref_uri, status, created_at, updated_at) |
|
|
| `plugin_metadata` | Discovered plugin descriptors (plugin_id, type, version, capabilities, path) |
|
|
| `health_checks` | Health check history (integration_id, checked_at, status, latency_ms, error_message) |
|
|
| `audit_logs` | Audit trail for integration CRUD operations (actor, action, integration_id, timestamp, details) |
|
|
|
|
## Endpoints
|
|
|
|
### Integration CRUD (`/api/v1/integrations`)
|
|
- `GET /` - List integrations for current tenant (optional `?type=` filter)
|
|
- `GET /{id}` - Get integration by ID
|
|
- `POST /` - Register a new integration (body: type, name, config, authRefUri)
|
|
- `PUT /{id}` - Update integration configuration
|
|
- `DELETE /{id}` - Remove integration (soft delete with audit)
|
|
|
|
### Health (`/api/v1/integrations`)
|
|
- `POST /{id}/health-check` - Trigger an on-demand health check for a specific integration
|
|
- `GET /{id}/health` - Retrieve latest health status and history
|
|
|
|
### Type-filtered queries
|
|
- `GET /by-type/{type}` - List integrations of a specific provider type (e.g., `github`, `harbor`)
|
|
|
|
### Plugin discovery
|
|
- `POST /plugins/discover` - Trigger plugin discovery scan and register new connectors
|
|
|
|
## Plugin Architecture
|
|
|
|
Each connector plugin implements a standard interface:
|
|
|
|
```csharp
|
|
public interface IIntegrationPlugin
|
|
{
|
|
string ProviderType { get; }
|
|
Task<IntegrationHealthResult> CheckHealthAsync(IntegrationConfig config, CancellationToken ct);
|
|
Task<bool> ValidateConfigAsync(JsonDocument config, CancellationToken ct);
|
|
}
|
|
```
|
|
|
|
**Built-in plugins:**
|
|
- **GitHubApp** - GitHub App installation authentication, repository listing, webhook setup
|
|
- **GitLab** - Personal/project access token authentication, project discovery
|
|
- **Harbor** - Robot account authentication, project and repository enumeration
|
|
- **InMemory** - Deterministic test double for integration tests and offline development
|
|
|
|
### Provider endpoint contracts
|
|
|
|
- **GitHub App** - Operators provide either the GitHub Cloud root (`https://github.com`), a GitHub Enterprise Server root, or an explicit `/api/v3` base. The connector normalizes the endpoint to a single API root and probes relative `app` / `rate_limit` paths so GitHub Enterprise onboarding never falls back to origin-root `/app`.
|
|
- **Harbor** - Operators provide the Harbor base URL. Stella Ops probes the provider-specific `/api/v2.0/health` route for connection tests and health checks.
|
|
|
|
## Security Considerations
|
|
|
|
- **AuthRef URI credential model:** Credentials are stored in an external vault (e.g., HashiCorp Vault, Azure Key Vault). The integration catalog stores only the URI reference (`authref://vault/path/to/secret`), never the raw secret.
|
|
- **Tenant scoping:** All database queries include a mandatory tenant filter enforced at the `DbContext` level via query filters.
|
|
- **Audit logging:** Every create, update, and delete operation is recorded in the audit log with actor identity, timestamp, and change details.
|
|
- **Plugin sandboxing:** Connector plugins run within the Plugin Framework's trust boundary; untrusted plugins are process-isolated.
|
|
|
|
## Observability
|
|
|
|
- **Metrics:** `integration_health_status{type,tenant}`, `integration_health_latency_ms`, `integration_crud_total{action}`
|
|
- **Logs:** Structured logs with `integrationId`, `tenantId`, `providerType`, `action`
|
|
- **Traces:** Spans for health checks, plugin discovery, and CRUD operations
|
|
|
|
## Performance Characteristics
|
|
|
|
- Health checks run on a configurable interval (default: 5 minutes) per integration
|
|
- Plugin discovery is triggered on startup and on-demand; results are cached
|
|
- Integration queries use indexed tenant_id + type columns for fast filtering
|
|
|
|
## IDE Extensions (VS Code, JetBrains)
|
|
|
|
The Integrations module also owns the IDE extension plugins, located under `src/Integrations/__Extensions/`. These are non-.NET projects that provide developer-facing tooling consuming the same Orchestrator/Router APIs as other integrations.
|
|
|
|
### VS Code Extension (`__Extensions/vscode-stella-ops/`)
|
|
|
|
- **Technology:** TypeScript, VS Code Extension API
|
|
- **Build:** `npm run compile` (TypeScript compilation)
|
|
- **Features:** Tree views for releases and environments, CodeLens annotations for `stella.yaml`, command palette integration, status bar widget
|
|
- **Manifest:** `package.json` (extension manifest, commands, views, configuration)
|
|
|
|
### JetBrains Plugin (`__Extensions/jetbrains-stella-ops/`)
|
|
|
|
- **Technology:** Kotlin, IntelliJ Platform SDK
|
|
- **Build:** Gradle (`./gradlew build`)
|
|
- **Features:** Tool windows (Releases/Environments/Deployments tabs), YAML annotator, action menus, status bar widget
|
|
- **Entry point:** `StellaOpsPlugin.kt`
|
|
|
|
### Design Principles (Extensions)
|
|
|
|
1. **Thin client** - Extensions contain no business logic; all state and decisions live in backend services
|
|
2. **Consistent experience** - Both plugins expose equivalent functionality despite different technology stacks
|
|
3. **Non-blocking** - All API calls are asynchronous; the IDE remains responsive during network operations
|
|
4. **Offline-tolerant** - Graceful degradation when the Stella Ops backend is unreachable
|
|
|
|
### Data Flow (Extensions)
|
|
|
|
```
|
|
[Developer IDE] --> [Extension/Plugin]
|
|
|
|
|
+-- GET /api/v1/releases/* --------> [Orchestrator API]
|
|
+-- GET /api/v1/environments/* ----> [Orchestrator API]
|
|
+-- POST /api/v1/promotions/* -----> [Orchestrator API]
|
|
+-- POST /oauth/token -------------> [Authority]
|
|
```
|
|
|
|
Authentication uses OAuth tokens obtained from the Authority service, stored in the IDE's secure credential store (VS Code `SecretStorage`, JetBrains `PasswordSafe`).
|
|
|
|
## References
|
|
|
|
- [Module README](./README.md)
|
|
- [Plugin Framework Architecture](../plugin/architecture.md)
|
|
- [Scanner Architecture](../scanner/architecture.md)
|