git.stella-ops.org/docs/architecture/integrations.md

Integrations Architecture

Overview

The Integrations module provides a unified catalog for external service connections including SCM providers (GitHub, GitLab, Bitbucket), container registries (Harbor, ECR, GCR, ACR), CI systems, and runtime hosts. It implements a plugin-based architecture for extensibility while maintaining consistent security and observability patterns.

Architecture Diagram

┌─────────────────────────────────────────────────────────────────────────────┐
│                         Integrations Module                                  │
├─────────────────────────────────────────────────────────────────────────────┤
│                                                                              │
│  ┌──────────────────────┐    ┌──────────────────────┐                       │
│  │   WebService Host    │    │   Plugin Loader      │                       │
│  │   (ASP.NET Core)     │────│   (DI Registration)  │                       │
│  └──────────┬───────────┘    └──────────┬───────────┘                       │
│             │                           │                                    │
│  ┌──────────▼───────────────────────────▼───────────┐                       │
│  │              Integration Catalog                  │                       │
│  │  - Registration CRUD                              │                       │
│  │  - Health Polling                                 │                       │
│  │  - Test Connection                                │                       │
│  └──────────┬───────────────────────────────────────┘                       │
│             │                                                                │
│  ┌──────────▼───────────────────────────────────────┐                       │
│  │              Plugin Contracts                     │                       │
│  │  - IIntegrationConnectorPlugin                    │                       │
│  │  - IScmAnnotationClient                           │                       │
│  │  - IRegistryConnector                             │                       │
│  └──────────────────────────────────────────────────┘                       │
│             │                                                                │
│  ┌──────────▼───────────────────────────────────────┐                       │
│  │              Provider Plugins                     │                       │
│  │  ┌─────────┐ ┌─────────┐ ┌─────────┐ ┌─────────┐ │                       │
│  │  │ GitHub  │ │ GitLab  │ │ Harbor  │ │  ECR    │ │                       │
│  │  │  App    │ │         │ │         │ │         │ │                       │
│  │  └─────────┘ └─────────┘ └─────────┘ └─────────┘ │                       │
│  │  ┌─────────┐ ┌─────────┐ ┌─────────┐            │                       │
│  │  │  GCR    │ │  ACR    │ │InMemory │            │                       │
│  │  │         │ │         │ │ (test)  │            │                       │
│  │  └─────────┘ └─────────┘ └─────────┘            │                       │
│  └──────────────────────────────────────────────────┘                       │
│                                                                              │
└─────────────────────────────────────────────────────────────────────────────┘

Core Components

Integration Catalog

The central registry for all external service connections:

• Registration: Store connection configuration with encrypted credentials
• Health Monitoring: Periodic health checks with status tracking
• Test Connection: On-demand connectivity verification
• Lifecycle Events: Emit events for Scheduler/Signals integration

Plugin System

Extensible plugin architecture for provider support:

public interface IIntegrationConnectorPlugin : IAvailabilityPlugin
{
    IntegrationType Type { get; }
    IntegrationProvider Provider { get; }
    Task<TestConnectionResult> TestConnectionAsync(IntegrationConfig config, CancellationToken ct);
    Task<HealthCheckResult> CheckHealthAsync(IntegrationConfig config, CancellationToken ct);
}

SCM Annotation Client

Interface for PR/MR comments and status checks:

public interface IScmAnnotationClient
{
    Task<ScmOperationResult<ScmCommentResponse>> PostCommentAsync(
        ScmCommentRequest request, CancellationToken ct);
    
    Task<ScmOperationResult<ScmStatusResponse>> PostStatusAsync(
        ScmStatusRequest request, CancellationToken ct);
    
    Task<ScmOperationResult<ScmCheckRunResponse>> CreateCheckRunAsync(
        ScmCheckRunRequest request, CancellationToken ct);
}

SCM Annotation Architecture

Comment and Status Flow

┌────────────┐     ┌─────────────┐     ┌────────────────┐     ┌──────────┐
│  Scanner   │────▶│ Integrations│────▶│ SCM Annotation │────▶│ GitHub/  │
│  Service   │     │   Service   │     │    Client      │     │ GitLab   │
└────────────┘     └─────────────┘     └────────────────┘     └──────────┘
      │                                        │
      │         ┌─────────────────┐            │
      └────────▶│ Annotation      │◀───────────┘
                │ Payload Builder │
                └─────────────────┘

Supported Operations

Operation	GitHub	GitLab
PR/MR Comment	Issue comment / Review comment	MR Note / Discussion
Commit Status	Commit status API	Commit status API
Check Run	Checks API with annotations	Pipeline status (emulated)
Inline Annotation	Check run annotation	MR discussion on line

Payload Models

Comment Request

public record ScmCommentRequest
{
    public required string Owner { get; init; }
    public required string Repository { get; init; }
    public required int PullRequestNumber { get; init; }
    public required string Body { get; init; }
    public string? CommentId { get; init; }  // For updates
    public bool UpdateExisting { get; init; } = true;
}

Status Request

public record ScmStatusRequest
{
    public required string Owner { get; init; }
    public required string Repository { get; init; }
    public required string CommitSha { get; init; }
    public required ScmStatusState State { get; init; }
    public required string Context { get; init; }
    public string? Description { get; init; }
    public string? TargetUrl { get; init; }
}

public enum ScmStatusState
{
    Pending,
    Success,
    Failure,
    Error
}

Check Run Request

public record ScmCheckRunRequest
{
    public required string Owner { get; init; }
    public required string Repository { get; init; }
    public required string HeadSha { get; init; }
    public required string Name { get; init; }
    public string? Status { get; init; }  // queued, in_progress, completed
    public string? Conclusion { get; init; }  // success, failure, neutral, etc.
    public string? Summary { get; init; }
    public string? Text { get; init; }
    public IReadOnlyList<ScmCheckRunAnnotation>? Annotations { get; init; }
}

public record ScmCheckRunAnnotation
{
    public required string Path { get; init; }
    public required int StartLine { get; init; }
    public required int EndLine { get; init; }
    public required string AnnotationLevel { get; init; }  // notice, warning, failure
    public required string Message { get; init; }
    public string? Title { get; init; }
}

Provider Implementations

GitHub App Plugin

• Uses GitHub App authentication (installation tokens)
• Supports: PR comments, commit status, check runs with annotations
• Handles rate limiting with exponential backoff
• Maps StellaOps severity to GitHub annotation levels

GitLab Plugin

• Uses Personal Access Token or CI Job Token
• Supports: MR notes, discussions, commit status
• Emulates check runs via pipeline status + MR discussions
• Handles project path encoding for API calls

Security

Credential Management

• All credentials stored as AuthRef URIs
• Resolved at runtime through Authority
• No plaintext secrets in configuration
• Audit trail for credential access

Token Scopes

Provider	Required Scopes
GitHub App	checks:write, pull_requests:write, statuses:write
GitLab	api, read_repository, write_repository

Error Handling

Offline-Safe Operations

All SCM operations return ScmOperationResult<T>:

public record ScmOperationResult<T>
{
    public bool Success { get; init; }
    public T? Result { get; init; }
    public string? ErrorMessage { get; init; }
    public bool IsTransient { get; init; }  // Retry-able
    public bool SkippedOffline { get; init; }
}

Retry Policy

• Transient errors (rate limit, network): Retry with exponential backoff
• Permanent errors (auth, not found): Fail immediately
• Offline mode: Skip with warning, log payload for manual posting

Observability

Metrics

Metric	Type	Labels
integrations_health_check_total	Counter	provider, status
integrations_test_connection_duration_seconds	Histogram	provider
scm_annotation_total	Counter	provider, operation, status
scm_annotation_duration_seconds	Histogram	provider, operation

Structured Logging

All operations log with:

• integrationId: Registration ID
• provider: GitHub, GitLab, etc.
• operation: comment, status, check_run
• prNumber / commitSha: Target reference

Current Catalog Contract

The live Integration Catalog contract is served by the Integrations WebService and is the source of truth for provider discovery and resource discovery.

Provider Metadata

• GET /api/v1/integrations/providers returns ProviderInfo[] with name, type, provider, isTestOnly, supportsDiscovery, and supportedResourceTypes.
• Test-only providers are hidden by default. GET /api/v1/integrations/providers?includeTestOnly=true exposes providers such as InMemory for explicit test/dev workflows.
• Built-in provider coverage now includes Harbor, Docker Registry, GitLab Container Registry, GitHub App, Gitea, GitLab Server, GitLab CI, Jenkins, Nexus, Vault, Consul, eBPF Agent, the S3Compatible object-storage provider, feed mirror providers (StellaOpsMirror, NvdMirror, OsvMirror), and the hidden test-only InMemory plugin.

Discovery

• POST /api/v1/integrations/{id}/discover accepts:

{
  "resourceType": "repositories",
  "filter": {
    "namePattern": "team/*"
  }
}

• Successful responses return DiscoverIntegrationResponse with the normalized resourceType, the ordered supportedResourceTypes, and the discovered resources.
• Unsupported discovery requests return 400 Bad Request with the supported resource types for that provider.
• Missing or cross-tenant integration IDs return 404 Not Found.

Discovery-Capable Providers

• OCI registries: repositories, tags
• SCM: projects, repositories
• CI/CD: jobs, pipelines
• Feed mirror, object-storage, and secrets/runtime providers currently expose health/test flows only

Credential Resolution

• Integration secrets are supplied as AuthRef URIs such as authref://vault/gitlab#access-token
• The runtime resolver is Vault-backed; there is no product-path stub resolver in the shipped service
• Registry connectors accept bearer tokens and username:password or username:token secrets for Basic auth-compatible registries

Related Documentation

• CI/CD Gate Flow
• Authority Architecture
• Scanner Architecture

AI Code Guard Standalone Run (Sprint 20260208_040)

This sprint adds deterministic standalone execution for AI Code Guard checks in the Integrations WebService.

API Surface

• Endpoint: POST /api/v1/integrations/ai-code-guard/run
• Mapped in: src/Integrations/StellaOps.Integrations.WebService/IntegrationEndpoints.cs
• Service contract: IAiCodeGuardRunService in src/Integrations/StellaOps.Integrations.WebService/AiCodeGuard/AiCodeGuardRunService.cs

The endpoint executes the equivalent of stella guard run behavior through an offline-safe API surface inside the Integrations module.

YAML-Driven Configuration

Configuration is parsed by AiCodeGuardPipelineConfigLoader:

• secrets / enableSecretsScan
• attribution / enableAttributionCheck
• license / enableLicenseHygiene
• maxFindings
• allowedSpdxLicenses / licenseAllowList
• customSecretPatterns / secretPatterns

The loader is deterministic and rejects unsupported keys or invalid values with explicit FormatException errors.

Scanning Behavior

AiCodeGuardRunService adds deterministic checks for:

• Secrets (built-in + optional custom regex patterns)
• Attribution markers
• SPDX license presence / allow-list validation

Output ordering is stable:

1. Severity descending
2. Path ordinal
3. Line number
4. Rule ID
5. Finding ID

Contracts

New contracts in src/Integrations/__Libraries/StellaOps.Integrations.Contracts/AiCodeGuardRunContracts.cs:

• AiCodeGuardRunRequest
• AiCodeGuardSourceFile
• AiCodeGuardRunConfiguration
• AiCodeGuardRunResponse

Test Evidence

Validated in src/Integrations/__Tests/StellaOps.Integrations.Tests/AiCodeGuardRunServiceTests.cs:

• Deterministic repeated output
• YAML configuration application and max-finding truncation
• Invalid YAML validation failure

Execution command:

• dotnet test src/Integrations/__Tests/StellaOps.Integrations.Tests/StellaOps.Integrations.Tests.csproj -p:BuildProjectReferences=false --no-restore

Result on 2026-02-08: passed (37/37).