git.stella-ops.org/docs/architecture/integrations.md

Integrations Architecture

Overview

The Integrations module provides a unified catalog for external service connections including SCM providers (GitHub, GitLab, Bitbucket), container registries (Harbor, ECR, GCR, ACR), CI systems, and runtime hosts. It implements a plugin-based architecture for extensibility while maintaining consistent security and observability patterns.

Architecture Diagram

┌─────────────────────────────────────────────────────────────────────────────┐
│                         Integrations Module                                  │
├─────────────────────────────────────────────────────────────────────────────┤
│                                                                              │
│  ┌──────────────────────┐    ┌──────────────────────┐                       │
│  │   WebService Host    │    │   Plugin Loader      │                       │
│  │   (ASP.NET Core)     │────│   (DI Registration)  │                       │
│  └──────────┬───────────┘    └──────────┬───────────┘                       │
│             │                           │                                    │
│  ┌──────────▼───────────────────────────▼───────────┐                       │
│  │              Integration Catalog                  │                       │
│  │  - Registration CRUD                              │                       │
│  │  - Health Polling                                 │                       │
│  │  - Test Connection                                │                       │
│  └──────────┬───────────────────────────────────────┘                       │
│             │                                                                │
│  ┌──────────▼───────────────────────────────────────┐                       │
│  │              Plugin Contracts                     │                       │
│  │  - IIntegrationConnectorPlugin                    │                       │
│  │  - IScmAnnotationClient                           │                       │
│  │  - IRegistryConnector                             │                       │
│  └──────────────────────────────────────────────────┘                       │
│             │                                                                │
│  ┌──────────▼───────────────────────────────────────┐                       │
│  │              Provider Plugins                     │                       │
│  │  ┌─────────┐ ┌─────────┐ ┌─────────┐ ┌─────────┐ │                       │
│  │  │ GitHub  │ │ GitLab  │ │ Harbor  │ │  ECR    │ │                       │
│  │  │  App    │ │         │ │         │ │         │ │                       │
│  │  └─────────┘ └─────────┘ └─────────┘ └─────────┘ │                       │
│  │  ┌─────────┐ ┌─────────┐ ┌─────────┐            │                       │
│  │  │  GCR    │ │  ACR    │ │InMemory │            │                       │
│  │  │         │ │         │ │ (test)  │            │                       │
│  │  └─────────┘ └─────────┘ └─────────┘            │                       │
│  └──────────────────────────────────────────────────┘                       │
│                                                                              │
└─────────────────────────────────────────────────────────────────────────────┘

Core Components

Integration Catalog

The central registry for all external service connections:

• Registration: Store connection configuration with encrypted credentials
• Health Monitoring: Periodic health checks with status tracking
• Test Connection: On-demand connectivity verification
• Lifecycle Events: Emit events for Scheduler/Signals integration

Plugin System

Extensible plugin architecture for provider support:

public interface IIntegrationConnectorPlugin : IAvailabilityPlugin
{
    IntegrationType Type { get; }
    IntegrationProvider Provider { get; }
    Task<TestConnectionResult> TestConnectionAsync(IntegrationConfig config, CancellationToken ct);
    Task<HealthCheckResult> CheckHealthAsync(IntegrationConfig config, CancellationToken ct);
}

SCM Annotation Client

Interface for PR/MR comments and status checks:

public interface IScmAnnotationClient
{
    Task<ScmOperationResult<ScmCommentResponse>> PostCommentAsync(
        ScmCommentRequest request, CancellationToken ct);
    
    Task<ScmOperationResult<ScmStatusResponse>> PostStatusAsync(
        ScmStatusRequest request, CancellationToken ct);
    
    Task<ScmOperationResult<ScmCheckRunResponse>> CreateCheckRunAsync(
        ScmCheckRunRequest request, CancellationToken ct);
}

SCM Annotation Architecture

Comment and Status Flow

┌────────────┐     ┌─────────────┐     ┌────────────────┐     ┌──────────┐
│  Scanner   │────▶│ Integrations│────▶│ SCM Annotation │────▶│ GitHub/  │
│  Service   │     │   Service   │     │    Client      │     │ GitLab   │
└────────────┘     └─────────────┘     └────────────────┘     └──────────┘
      │                                        │
      │         ┌─────────────────┐            │
      └────────▶│ Annotation      │◀───────────┘
                │ Payload Builder │
                └─────────────────┘

Supported Operations

Operation	GitHub	GitLab
PR/MR Comment	Issue comment / Review comment	MR Note / Discussion
Commit Status	Commit status API	Commit status API
Check Run	Checks API with annotations	Pipeline status (emulated)
Inline Annotation	Check run annotation	MR discussion on line

Payload Models

Comment Request

public record ScmCommentRequest
{
    public required string Owner { get; init; }
    public required string Repository { get; init; }
    public required int PullRequestNumber { get; init; }
    public required string Body { get; init; }
    public string? CommentId { get; init; }  // For updates
    public bool UpdateExisting { get; init; } = true;
}

Status Request

public record ScmStatusRequest
{
    public required string Owner { get; init; }
    public required string Repository { get; init; }
    public required string CommitSha { get; init; }
    public required ScmStatusState State { get; init; }
    public required string Context { get; init; }
    public string? Description { get; init; }
    public string? TargetUrl { get; init; }
}

public enum ScmStatusState
{
    Pending,
    Success,
    Failure,
    Error
}

Check Run Request

public record ScmCheckRunRequest
{
    public required string Owner { get; init; }
    public required string Repository { get; init; }
    public required string HeadSha { get; init; }
    public required string Name { get; init; }
    public string? Status { get; init; }  // queued, in_progress, completed
    public string? Conclusion { get; init; }  // success, failure, neutral, etc.
    public string? Summary { get; init; }
    public string? Text { get; init; }
    public IReadOnlyList<ScmCheckRunAnnotation>? Annotations { get; init; }
}

public record ScmCheckRunAnnotation
{
    public required string Path { get; init; }
    public required int StartLine { get; init; }
    public required int EndLine { get; init; }
    public required string AnnotationLevel { get; init; }  // notice, warning, failure
    public required string Message { get; init; }
    public string? Title { get; init; }
}

Provider Implementations

GitHub App Plugin

• Uses GitHub App authentication (installation tokens)
• Supports: PR comments, commit status, check runs with annotations
• Handles rate limiting with exponential backoff
• Maps StellaOps severity to GitHub annotation levels

GitLab Plugin

• Uses Personal Access Token or CI Job Token
• Supports: MR notes, discussions, commit status
• Emulates check runs via pipeline status + MR discussions
• Handles project path encoding for API calls

Security

Credential Management

• All credentials stored as AuthRef URIs
• Resolved at runtime through Authority
• No plaintext secrets in configuration
• Audit trail for credential access

Token Scopes

Provider	Required Scopes
GitHub App	checks:write, pull_requests:write, statuses:write
GitLab	api, read_repository, write_repository

Error Handling

Offline-Safe Operations

All SCM operations return ScmOperationResult<T>:

public record ScmOperationResult<T>
{
    public bool Success { get; init; }
    public T? Result { get; init; }
    public string? ErrorMessage { get; init; }
    public bool IsTransient { get; init; }  // Retry-able
    public bool SkippedOffline { get; init; }
}

Retry Policy

• Transient errors (rate limit, network): Retry with exponential backoff
• Permanent errors (auth, not found): Fail immediately
• Offline mode: Skip with warning, log payload for manual posting

Observability

Metrics

Metric	Type	Labels
integrations_health_check_total	Counter	provider, status
integrations_test_connection_duration_seconds	Histogram	provider
scm_annotation_total	Counter	provider, operation, status
scm_annotation_duration_seconds	Histogram	provider, operation

Structured Logging

All operations log with:

• integrationId: Registration ID
• provider: GitHub, GitLab, etc.
• operation: comment, status, check_run
• prNumber / commitSha: Target reference

Related Documentation

• CI/CD Gate Flow
• Authority Architecture
• Scanner Architecture