# SPRINT_1229_002_BE: SBOM Sources Trigger Service ## Executive Summary This sprint implements the **trigger service** that dispatches scans based on source configurations. It handles scheduled (cron) triggers, webhook handlers (Zastava registry, Git), manual triggers, and retry logic. **Working Directory:** `src/Scanner/`, `src/Scheduler/` **Module:** BE (Backend) **Dependencies:** SPRINT_1229_001_BE (Sources Foundation) --- ## Architecture ``` ┌─────────────────────────────────────────────────────────────────────────────┐ │ Source Trigger Service │ │ │ │ ┌───────────────────────────────────────────────────────────────────────┐ │ │ │ Trigger Dispatcher │ │ │ │ │ │ │ │ ┌─────────┐ ┌─────────┐ ┌─────────┐ ┌─────────┐ │ │ │ │ │Schedule │ │ Webhook │ │ Manual │ │ Retry │ │ │ │ │ │ (Cron) │ │ Handler │ │ Trigger │ │ Handler │ │ │ │ │ └────┬────┘ └────┬────┘ └────┬────┘ └────┬────┘ │ │ │ │ │ │ │ │ │ │ │ │ └──────────────┴──────────────┴──────────────┘ │ │ │ │ │ │ │ │ │ ┌─────────▼─────────┐ │ │ │ │ │ Source Context │ │ │ │ │ │ Resolver │ │ │ │ │ └─────────┬─────────┘ │ │ │ │ │ │ │ │ └──────────────────────────────┼────────────────────────────────────────┘ │ │ │ │ │ ┌──────────────────────────────▼────────────────────────────────────────┐ │ │ │ Source Type Handlers │ │ │ │ │ │ │ │ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────────┐ │ │ │ │ │ Zastava │ │ Docker │ │ CLI │ │ Git │ │ │ │ │ │ Handler │ │ Handler │ │ Handler │ │ Handler │ │ │ │ │ └──────┬──────┘ └──────┬──────┘ └──────┬──────┘ └────────┬────────┘ │ │ │ │ │ │ │ │ │ │ │ └─────────┼───────────────┼───────────────┼──────────────────┼──────────┘ │ │ │ │ │ │ │ └────────────┼───────────────┼───────────────┼──────────────────┼────────────┘ │ │ │ │ ▼ ▼ ▼ ▼ ┌───────────────────────────────────────────────────────────────────┐ │ Scanner Job Queue │ │ │ │ ScanJob { imageRef, sourceId, correlationId, metadata } │ │ │ └───────────────────────────────────────────────────────────────────┘ ``` --- ## Component Design ### 1. Trigger Dispatcher Centralized coordinator for all trigger types: ```csharp public interface ISourceTriggerDispatcher { /// /// Dispatch a trigger for a source, creating scan jobs as appropriate. /// Task DispatchAsync( Guid sourceId, SbomSourceRunTrigger trigger, string? triggerDetails = null, CancellationToken ct = default); /// /// Process scheduled sources that are due. /// Called by scheduler worker. /// Task ProcessScheduledSourcesAsync(CancellationToken ct); } ``` ### 2. Source Type Handlers Each source type has a dedicated handler: ```csharp public interface ISourceTypeHandler { SbomSourceType SourceType { get; } /// /// Discover items to scan based on source configuration. /// Task> DiscoverTargetsAsync( SbomSource source, TriggerContext context, CancellationToken ct); /// /// Validate source configuration. /// ValidationResult ValidateConfiguration(JsonDocument configuration); /// /// Test source connection with credentials. /// Task TestConnectionAsync( SbomSource source, CancellationToken ct); } public record ScanTarget( string Reference, // Image ref, repo URL, etc. string? Digest, // Optional pinned digest Dictionary Metadata ); public record TriggerContext( SbomSourceRunTrigger Trigger, string? TriggerDetails, string CorrelationId, JsonDocument? WebhookPayload ); ``` --- ## Webhook Handlers ### Zastava Registry Webhook **Endpoint:** `POST /api/v1/webhooks/zastava/{sourceId}` **Supported Registry Types:** - Docker Hub - Harbor - Quay - AWS ECR - Google GCR - Azure ACR - GitHub Container Registry - Generic (configurable payload mapping) ```csharp public class ZastavaWebhookHandler { /// /// Handle registry push webhook. /// public async Task HandleAsync( Guid sourceId, HttpRequest request, CancellationToken ct) { // 1. Verify webhook signature var source = await _sourceRepo.GetAsync(sourceId, ct); if (!VerifySignature(request, source.WebhookSecretRef)) return WebhookResult.Unauthorized(); // 2. Parse payload based on registry type var config = source.GetConfiguration(); var payload = await ParsePayload(request, config.RegistryType); // 3. Check filters (repo patterns, tag patterns) if (!MatchesFilters(payload, config.Filters)) return WebhookResult.Skipped("Does not match filters"); // 4. Dispatch trigger var run = await _dispatcher.DispatchAsync( sourceId, SbomSourceRunTrigger.Webhook, $"push:{payload.Repository}:{payload.Tag}"); return WebhookResult.Accepted(run.RunId); } } ``` **Payload Normalization:** ```csharp public record RegistryPushPayload( string Repository, string Tag, string? Digest, string? PushedBy, DateTimeOffset Timestamp, Dictionary RawHeaders ); public interface IRegistryPayloadParser { RegistryPushPayload Parse(HttpRequest request); } // Implementations: // - DockerHubPayloadParser // - HarborPayloadParser // - QuayPayloadParser // - EcrPayloadParser // - GcrPayloadParser // - AcrPayloadParser // - GhcrPayloadParser // - GenericPayloadParser (JSONPath-based configuration) ``` ### Git Webhook **Endpoint:** `POST /api/v1/webhooks/git/{sourceId}` **Supported Providers:** - GitHub - GitLab - Bitbucket - Azure DevOps - Gitea ```csharp public class GitWebhookHandler { public async Task HandleAsync( Guid sourceId, HttpRequest request, CancellationToken ct) { var source = await _sourceRepo.GetAsync(sourceId, ct); var config = source.GetConfiguration(); // 1. Verify webhook signature if (!VerifySignature(request, config.Provider, source.WebhookSecretRef)) return WebhookResult.Unauthorized(); // 2. Parse event type var eventType = DetectEventType(request, config.Provider); // 3. Check if event matches triggers if (!ShouldTrigger(eventType, config.Triggers)) return WebhookResult.Skipped("Event type not configured for trigger"); // 4. Parse payload var payload = await ParsePayload(request, config.Provider, eventType); // 5. Check branch/tag filters if (!MatchesBranchFilters(payload, config.Branches)) return WebhookResult.Skipped("Branch does not match filters"); // 6. Dispatch var run = await _dispatcher.DispatchAsync( sourceId, SbomSourceRunTrigger.Webhook, $"{eventType}:{payload.Ref}@{payload.CommitSha}"); return WebhookResult.Accepted(run.RunId); } } ``` --- ## Scheduled Trigger Integration ### Scheduler Job Type Register a new job type with the Scheduler service: ```csharp public class SourceSchedulerJob : IScheduledJob { public string JobType => "sbom-source-scheduled"; public async Task ExecuteAsync(JobContext context, CancellationToken ct) { var sourceId = Guid.Parse(context.Payload["sourceId"]); await _dispatcher.DispatchAsync( sourceId, SbomSourceRunTrigger.Scheduled, context.Payload["cronExpression"]); } } ``` ### Schedule Registration When a source with cron schedule is created/updated: ```csharp public async Task RegisterScheduleAsync(SbomSource source) { if (string.IsNullOrEmpty(source.CronSchedule)) return; await _schedulerClient.UpsertScheduleAsync(new ScheduleRequest { ScheduleId = $"sbom-source-{source.SourceId}", JobType = "sbom-source-scheduled", Cron = source.CronSchedule, Timezone = source.CronTimezone ?? "UTC", Payload = new Dictionary { ["sourceId"] = source.SourceId.ToString(), ["cronExpression"] = source.CronSchedule }, Enabled = source.Status == SbomSourceStatus.Active && !source.Paused }); } ``` --- ## Source Type Handler Implementations ### Zastava Handler ```csharp public class ZastavaSourceHandler : ISourceTypeHandler { public SbomSourceType SourceType => SbomSourceType.Zastava; public async Task> DiscoverTargetsAsync( SbomSource source, TriggerContext context, CancellationToken ct) { // For webhook triggers, target is in the payload if (context.Trigger == SbomSourceRunTrigger.Webhook && context.WebhookPayload != null) { var payload = ParseWebhookPayload(context.WebhookPayload); return [new ScanTarget( $"{payload.Repository}:{payload.Tag}", payload.Digest, new() { ["pushedBy"] = payload.PushedBy ?? "unknown" } )]; } // For scheduled/manual, discover from registry var config = source.GetConfiguration(); var credentials = await _credentialStore.GetAsync(source.AuthRef!); var client = _registryClientFactory.Create(config.RegistryType, config.RegistryUrl, credentials); var targets = new List(); foreach (var repoPattern in config.Filters.Repositories) { var repos = await client.ListRepositoriesAsync(repoPattern, ct); foreach (var repo in repos) { var tags = await client.ListTagsAsync(repo, config.Filters.Tags, ct); foreach (var tag in tags) { if (ShouldExclude(repo, tag, config.Filters)) continue; targets.Add(new ScanTarget($"{repo}:{tag}", null, new())); } } } return targets; } public async Task TestConnectionAsync( SbomSource source, CancellationToken ct) { var config = source.GetConfiguration(); var credentials = await _credentialStore.GetAsync(source.AuthRef!); try { var client = _registryClientFactory.Create( config.RegistryType, config.RegistryUrl, credentials); await client.PingAsync(ct); return ConnectionTestResult.Success(); } catch (Exception ex) { return ConnectionTestResult.Failure(ex.Message); } } } ``` ### Docker Handler ```csharp public class DockerSourceHandler : ISourceTypeHandler { public SbomSourceType SourceType => SbomSourceType.Docker; public async Task> DiscoverTargetsAsync( SbomSource source, TriggerContext context, CancellationToken ct) { var config = source.GetConfiguration(); var targets = new List(); foreach (var imageSpec in config.Images) { if (imageSpec.TagPatterns?.Any() == true) { // Discover matching tags var credentials = await _credentialStore.GetAsync(source.AuthRef!); var client = _registryClientFactory.Create(config.RegistryUrl, credentials); var (repo, _) = ParseImageReference(imageSpec.Reference); var tags = await client.ListTagsAsync(repo, imageSpec.TagPatterns, ct); foreach (var tag in tags) { targets.Add(new ScanTarget($"{repo}:{tag}", null, new())); } } else { // Scan specific reference targets.Add(new ScanTarget(imageSpec.Reference, null, new())); } } return targets; } } ``` ### Git Handler ```csharp public class GitSourceHandler : ISourceTypeHandler { public SbomSourceType SourceType => SbomSourceType.Git; public async Task> DiscoverTargetsAsync( SbomSource source, TriggerContext context, CancellationToken ct) { var config = source.GetConfiguration(); // For webhook triggers, use the payload if (context.Trigger == SbomSourceRunTrigger.Webhook && context.WebhookPayload != null) { var payload = ParseGitPayload(context.WebhookPayload, config.Provider); return [new ScanTarget( config.RepositoryUrl, null, new() { ["ref"] = payload.Ref, ["commitSha"] = payload.CommitSha, ["branch"] = payload.Branch ?? "", ["tag"] = payload.Tag ?? "" } )]; } // For scheduled/manual, scan default branch or configured branches var credentials = await _credentialStore.GetAsync(source.AuthRef!); var gitClient = _gitClientFactory.Create(config.Provider, credentials); var branches = await gitClient.ListBranchesAsync(config.RepositoryUrl, ct); var matchingBranches = branches .Where(b => MatchesBranchPattern(b, config.Branches.Include)) .Where(b => !MatchesBranchPattern(b, config.Branches.Exclude ?? [])) .ToList(); return matchingBranches.Select(b => new ScanTarget( config.RepositoryUrl, null, new() { ["branch"] = b, ["ref"] = $"refs/heads/{b}" } )).ToList(); } } ``` ### CLI Handler ```csharp public class CliSourceHandler : ISourceTypeHandler { public SbomSourceType SourceType => SbomSourceType.Cli; public Task> DiscoverTargetsAsync( SbomSource source, TriggerContext context, CancellationToken ct) { // CLI sources don't "discover" targets - they receive submissions // This handler validates incoming submissions against source config return Task.FromResult>([]); } /// /// Validate an incoming CLI submission against source configuration. /// public ValidationResult ValidateSubmission( SbomSource source, CliSubmissionRequest submission) { var config = source.GetConfiguration(); var errors = new List(); // Check tool allowlist if (!config.AllowedTools.Contains(submission.Tool)) errors.Add($"Tool '{submission.Tool}' not in allowed list"); // Check CI system if (config.AllowedCiSystems?.Any() == true && !config.AllowedCiSystems.Contains(submission.CiSystem ?? "")) errors.Add($"CI system '{submission.CiSystem}' not allowed"); // Check format if (!config.Validation.AllowedFormats.Contains(submission.Format)) errors.Add($"Format '{submission.Format}' not allowed"); // Check size if (submission.SbomSizeBytes > config.Validation.MaxSbomSizeBytes) errors.Add($"SBOM size {submission.SbomSizeBytes} exceeds max {config.Validation.MaxSbomSizeBytes}"); // Check attribution requirements if (config.Attribution.RequireBuildId && string.IsNullOrEmpty(submission.BuildId)) errors.Add("Build ID is required"); if (config.Attribution.RequireRepository && string.IsNullOrEmpty(submission.Repository)) errors.Add("Repository is required"); if (config.Attribution.RequireCommitSha && string.IsNullOrEmpty(submission.CommitSha)) errors.Add("Commit SHA is required"); return errors.Any() ? ValidationResult.Failure(errors) : ValidationResult.Success(); } } ``` --- ## Task Breakdown ### T1: Trigger Dispatcher Service (TODO) **Files to Create:** - `src/Scanner/__Libraries/StellaOps.Scanner.Sources/Triggers/ISourceTriggerDispatcher.cs` - `src/Scanner/__Libraries/StellaOps.Scanner.Sources/Triggers/SourceTriggerDispatcher.cs` - `src/Scanner/__Libraries/StellaOps.Scanner.Sources/Triggers/TriggerContext.cs` ### T2: Source Type Handler Interface & Base (TODO) **Files to Create:** - `src/Scanner/__Libraries/StellaOps.Scanner.Sources/Handlers/ISourceTypeHandler.cs` - `src/Scanner/__Libraries/StellaOps.Scanner.Sources/Handlers/SourceTypeHandlerBase.cs` - `src/Scanner/__Libraries/StellaOps.Scanner.Sources/Handlers/ScanTarget.cs` ### T3: Zastava Handler Implementation (TODO) **Files to Create:** - `src/Scanner/__Libraries/StellaOps.Scanner.Sources/Handlers/Zastava/ZastavaSourceHandler.cs` - `src/Scanner/__Libraries/StellaOps.Scanner.Sources/Handlers/Zastava/RegistryPayloadParsers.cs` - `src/Scanner/__Libraries/StellaOps.Scanner.Sources/Handlers/Zastava/IRegistryClient.cs` ### T4: Docker Handler Implementation (TODO) **Files to Create:** - `src/Scanner/__Libraries/StellaOps.Scanner.Sources/Handlers/Docker/DockerSourceHandler.cs` - `src/Scanner/__Libraries/StellaOps.Scanner.Sources/Handlers/Docker/ImageDiscovery.cs` ### T5: Git Handler Implementation (TODO) **Files to Create:** - `src/Scanner/__Libraries/StellaOps.Scanner.Sources/Handlers/Git/GitSourceHandler.cs` - `src/Scanner/__Libraries/StellaOps.Scanner.Sources/Handlers/Git/GitPayloadParsers.cs` - `src/Scanner/__Libraries/StellaOps.Scanner.Sources/Handlers/Git/IGitClient.cs` ### T6: CLI Handler Implementation (TODO) **Files to Create:** - `src/Scanner/__Libraries/StellaOps.Scanner.Sources/Handlers/Cli/CliSourceHandler.cs` - `src/Scanner/__Libraries/StellaOps.Scanner.Sources/Handlers/Cli/CliSubmissionValidator.cs` ### T7: Webhook Endpoints (TODO) **Files to Create:** - `src/Scanner/StellaOps.Scanner.WebService/Endpoints/WebhookEndpoints.cs` - `src/Scanner/StellaOps.Scanner.WebService/Webhooks/WebhookSignatureValidator.cs` ### T8: Scheduler Integration (TODO) **Files to Create:** - `src/Scanner/__Libraries/StellaOps.Scanner.Sources/Scheduling/SourceSchedulerJob.cs` - `src/Scanner/__Libraries/StellaOps.Scanner.Sources/Scheduling/ScheduleRegistration.cs` ### T9: Retry Handler (TODO) **Files to Create:** - `src/Scanner/__Libraries/StellaOps.Scanner.Sources/Triggers/RetryHandler.cs` - `src/Scanner/__Libraries/StellaOps.Scanner.Sources/Triggers/RetryPolicy.cs` ### T10: Unit & Integration Tests (TODO) **Files to Create:** - `src/Scanner/__Tests/StellaOps.Scanner.Sources.Tests/Triggers/SourceTriggerDispatcherTests.cs` - `src/Scanner/__Tests/StellaOps.Scanner.Sources.Tests/Handlers/*Tests.cs` - `src/Scanner/__Tests/StellaOps.Scanner.Sources.Tests/Webhooks/WebhookEndpointsTests.cs` --- ## Delivery Tracker | Task | Status | Notes | |------|--------|-------| | T1: Trigger Dispatcher | TODO | | | T2: Handler Interface | TODO | | | T3: Zastava Handler | TODO | | | T4: Docker Handler | TODO | | | T5: Git Handler | TODO | | | T6: CLI Handler | TODO | | | T7: Webhook Endpoints | TODO | | | T8: Scheduler Integration | TODO | | | T9: Retry Handler | TODO | | | T10: Tests | TODO | | --- ## Next Sprint **SPRINT_1229_003_FE_sbom-sources-ui** - Frontend Sources Manager: - Sources list page with status indicators - Add/Edit source wizard per type - Connection test UI - Source detail page with run history