261 lines
12 KiB
Markdown
261 lines
12 KiB
Markdown
# Integrations Architecture
|
|
|
|
## Overview
|
|
|
|
The Integrations module provides a unified catalog for external service connections including SCM providers (GitHub, GitLab, Bitbucket), container registries (Harbor, ECR, GCR, ACR), CI systems, and runtime hosts. It implements a plugin-based architecture for extensibility while maintaining consistent security and observability patterns.
|
|
|
|
## Architecture Diagram
|
|
|
|
```
|
|
┌─────────────────────────────────────────────────────────────────────────────┐
|
|
│ Integrations Module │
|
|
├─────────────────────────────────────────────────────────────────────────────┤
|
|
│ │
|
|
│ ┌──────────────────────┐ ┌──────────────────────┐ │
|
|
│ │ WebService Host │ │ Plugin Loader │ │
|
|
│ │ (ASP.NET Core) │────│ (DI Registration) │ │
|
|
│ └──────────┬───────────┘ └──────────┬───────────┘ │
|
|
│ │ │ │
|
|
│ ┌──────────▼───────────────────────────▼───────────┐ │
|
|
│ │ Integration Catalog │ │
|
|
│ │ - Registration CRUD │ │
|
|
│ │ - Health Polling │ │
|
|
│ │ - Test Connection │ │
|
|
│ └──────────┬───────────────────────────────────────┘ │
|
|
│ │ │
|
|
│ ┌──────────▼───────────────────────────────────────┐ │
|
|
│ │ Plugin Contracts │ │
|
|
│ │ - IIntegrationConnectorPlugin │ │
|
|
│ │ - IScmAnnotationClient │ │
|
|
│ │ - IRegistryConnector │ │
|
|
│ └──────────────────────────────────────────────────┘ │
|
|
│ │ │
|
|
│ ┌──────────▼───────────────────────────────────────┐ │
|
|
│ │ Provider Plugins │ │
|
|
│ │ ┌─────────┐ ┌─────────┐ ┌─────────┐ ┌─────────┐ │ │
|
|
│ │ │ GitHub │ │ GitLab │ │ Harbor │ │ ECR │ │ │
|
|
│ │ │ App │ │ │ │ │ │ │ │ │
|
|
│ │ └─────────┘ └─────────┘ └─────────┘ └─────────┘ │ │
|
|
│ │ ┌─────────┐ ┌─────────┐ ┌─────────┐ │ │
|
|
│ │ │ GCR │ │ ACR │ │InMemory │ │ │
|
|
│ │ │ │ │ │ │ (test) │ │ │
|
|
│ │ └─────────┘ └─────────┘ └─────────┘ │ │
|
|
│ └──────────────────────────────────────────────────┘ │
|
|
│ │
|
|
└─────────────────────────────────────────────────────────────────────────────┘
|
|
```
|
|
|
|
## Core Components
|
|
|
|
### Integration Catalog
|
|
|
|
The central registry for all external service connections:
|
|
|
|
- **Registration**: Store connection configuration with encrypted credentials
|
|
- **Health Monitoring**: Periodic health checks with status tracking
|
|
- **Test Connection**: On-demand connectivity verification
|
|
- **Lifecycle Events**: Emit events for Scheduler/Signals integration
|
|
|
|
### Plugin System
|
|
|
|
Extensible plugin architecture for provider support:
|
|
|
|
```csharp
|
|
public interface IIntegrationConnectorPlugin : IAvailabilityPlugin
|
|
{
|
|
IntegrationType Type { get; }
|
|
IntegrationProvider Provider { get; }
|
|
Task<TestConnectionResult> TestConnectionAsync(IntegrationConfig config, CancellationToken ct);
|
|
Task<HealthCheckResult> CheckHealthAsync(IntegrationConfig config, CancellationToken ct);
|
|
}
|
|
```
|
|
|
|
### SCM Annotation Client
|
|
|
|
Interface for PR/MR comments and status checks:
|
|
|
|
```csharp
|
|
public interface IScmAnnotationClient
|
|
{
|
|
Task<ScmOperationResult<ScmCommentResponse>> PostCommentAsync(
|
|
ScmCommentRequest request, CancellationToken ct);
|
|
|
|
Task<ScmOperationResult<ScmStatusResponse>> PostStatusAsync(
|
|
ScmStatusRequest request, CancellationToken ct);
|
|
|
|
Task<ScmOperationResult<ScmCheckRunResponse>> CreateCheckRunAsync(
|
|
ScmCheckRunRequest request, CancellationToken ct);
|
|
}
|
|
```
|
|
|
|
## SCM Annotation Architecture
|
|
|
|
### Comment and Status Flow
|
|
|
|
```
|
|
┌────────────┐ ┌─────────────┐ ┌────────────────┐ ┌──────────┐
|
|
│ Scanner │────▶│ Integrations│────▶│ SCM Annotation │────▶│ GitHub/ │
|
|
│ Service │ │ Service │ │ Client │ │ GitLab │
|
|
└────────────┘ └─────────────┘ └────────────────┘ └──────────┘
|
|
│ │
|
|
│ ┌─────────────────┐ │
|
|
└────────▶│ Annotation │◀───────────┘
|
|
│ Payload Builder │
|
|
└─────────────────┘
|
|
```
|
|
|
|
### Supported Operations
|
|
|
|
| Operation | GitHub | GitLab |
|
|
|-----------|--------|--------|
|
|
| PR/MR Comment | Issue comment / Review comment | MR Note / Discussion |
|
|
| Commit Status | Commit status API | Commit status API |
|
|
| Check Run | Checks API with annotations | Pipeline status (emulated) |
|
|
| Inline Annotation | Check run annotation | MR discussion on line |
|
|
|
|
### Payload Models
|
|
|
|
#### Comment Request
|
|
|
|
```csharp
|
|
public record ScmCommentRequest
|
|
{
|
|
public required string Owner { get; init; }
|
|
public required string Repository { get; init; }
|
|
public required int PullRequestNumber { get; init; }
|
|
public required string Body { get; init; }
|
|
public string? CommentId { get; init; } // For updates
|
|
public bool UpdateExisting { get; init; } = true;
|
|
}
|
|
```
|
|
|
|
#### Status Request
|
|
|
|
```csharp
|
|
public record ScmStatusRequest
|
|
{
|
|
public required string Owner { get; init; }
|
|
public required string Repository { get; init; }
|
|
public required string CommitSha { get; init; }
|
|
public required ScmStatusState State { get; init; }
|
|
public required string Context { get; init; }
|
|
public string? Description { get; init; }
|
|
public string? TargetUrl { get; init; }
|
|
}
|
|
|
|
public enum ScmStatusState
|
|
{
|
|
Pending,
|
|
Success,
|
|
Failure,
|
|
Error
|
|
}
|
|
```
|
|
|
|
#### Check Run Request
|
|
|
|
```csharp
|
|
public record ScmCheckRunRequest
|
|
{
|
|
public required string Owner { get; init; }
|
|
public required string Repository { get; init; }
|
|
public required string HeadSha { get; init; }
|
|
public required string Name { get; init; }
|
|
public string? Status { get; init; } // queued, in_progress, completed
|
|
public string? Conclusion { get; init; } // success, failure, neutral, etc.
|
|
public string? Summary { get; init; }
|
|
public string? Text { get; init; }
|
|
public IReadOnlyList<ScmCheckRunAnnotation>? Annotations { get; init; }
|
|
}
|
|
|
|
public record ScmCheckRunAnnotation
|
|
{
|
|
public required string Path { get; init; }
|
|
public required int StartLine { get; init; }
|
|
public required int EndLine { get; init; }
|
|
public required string AnnotationLevel { get; init; } // notice, warning, failure
|
|
public required string Message { get; init; }
|
|
public string? Title { get; init; }
|
|
}
|
|
```
|
|
|
|
## Provider Implementations
|
|
|
|
### GitHub App Plugin
|
|
|
|
- Uses GitHub App authentication (installation tokens)
|
|
- Supports: PR comments, commit status, check runs with annotations
|
|
- Handles rate limiting with exponential backoff
|
|
- Maps StellaOps severity to GitHub annotation levels
|
|
|
|
### GitLab Plugin
|
|
|
|
- Uses Personal Access Token or CI Job Token
|
|
- Supports: MR notes, discussions, commit status
|
|
- Emulates check runs via pipeline status + MR discussions
|
|
- Handles project path encoding for API calls
|
|
|
|
## Security
|
|
|
|
### Credential Management
|
|
|
|
- All credentials stored as AuthRef URIs
|
|
- Resolved at runtime through Authority
|
|
- No plaintext secrets in configuration
|
|
- Audit trail for credential access
|
|
|
|
### Token Scopes
|
|
|
|
| Provider | Required Scopes |
|
|
|----------|----------------|
|
|
| GitHub App | `checks:write`, `pull_requests:write`, `statuses:write` |
|
|
| GitLab | `api`, `read_repository`, `write_repository` |
|
|
|
|
## Error Handling
|
|
|
|
### Offline-Safe Operations
|
|
|
|
All SCM operations return `ScmOperationResult<T>`:
|
|
|
|
```csharp
|
|
public record ScmOperationResult<T>
|
|
{
|
|
public bool Success { get; init; }
|
|
public T? Result { get; init; }
|
|
public string? ErrorMessage { get; init; }
|
|
public bool IsTransient { get; init; } // Retry-able
|
|
public bool SkippedOffline { get; init; }
|
|
}
|
|
```
|
|
|
|
### Retry Policy
|
|
|
|
- Transient errors (rate limit, network): Retry with exponential backoff
|
|
- Permanent errors (auth, not found): Fail immediately
|
|
- Offline mode: Skip with warning, log payload for manual posting
|
|
|
|
## Observability
|
|
|
|
### Metrics
|
|
|
|
| Metric | Type | Labels |
|
|
|--------|------|--------|
|
|
| `integrations_health_check_total` | Counter | `provider`, `status` |
|
|
| `integrations_test_connection_duration_seconds` | Histogram | `provider` |
|
|
| `scm_annotation_total` | Counter | `provider`, `operation`, `status` |
|
|
| `scm_annotation_duration_seconds` | Histogram | `provider`, `operation` |
|
|
|
|
### Structured Logging
|
|
|
|
All operations log with:
|
|
- `integrationId`: Registration ID
|
|
- `provider`: GitHub, GitLab, etc.
|
|
- `operation`: comment, status, check_run
|
|
- `prNumber` / `commitSha`: Target reference
|
|
|
|
## Related Documentation
|
|
|
|
- [CI/CD Gate Flow](../../flows/10-cicd-gate-flow.md)
|
|
- [Authority Architecture](../authority/architecture.md)
|
|
- [Scanner Architecture](../scanner/architecture.md)
|