diff --git a/docs/aoc/guard-library.md b/docs/aoc/guard-library.md index 9993ddee9..26606f962 100644 --- a/docs/aoc/guard-library.md +++ b/docs/aoc/guard-library.md @@ -73,6 +73,8 @@ Key points: - Register the guard singleton before wiring repositories or worker services. - Use `AocGuardEndpointFilter` to protect Minimal API endpoints. The `payloadSelector` can yield multiple payloads (e.g. batch ingestion) and the filter will validate each one. +- Prefer the `RequireAocGuard` extension when wiring endpoints; it wraps `AddEndpointFilter` + and handles single-payload scenarios without additional boilerplate. - Wrap guard exceptions with `AocHttpResults.Problem` to ensure clients receive machine-readables codes (`ERR_AOC_00x`). ## Worker / repository usage diff --git a/docs/implplan/SPRINT_180_experience_sdks.md b/docs/implplan/SPRINT_180_experience_sdks.md index 3bc0275bd..23ae7e193 100644 --- a/docs/implplan/SPRINT_180_experience_sdks.md +++ b/docs/implplan/SPRINT_180_experience_sdks.md @@ -223,6 +223,7 @@ WEB-AIAI-31-001 `API routing` | TODO | Route `/advisory/ai/*` endpoints through WEB-AIAI-31-002 `Batch orchestration` | TODO | Provide batching job handlers and streaming responses for CLI automation with retry/backoff. Dependencies: WEB-AIAI-31-001. | BE-Base Platform Guild (src/Web/StellaOps.Web/TASKS.md) WEB-AIAI-31-003 `Telemetry & audit` | TODO | Emit metrics/logs (latency, guardrail blocks, validation failures) and forward anonymized prompt hashes to analytics. Dependencies: WEB-AIAI-31-002. | BE-Base Platform Guild, Observability Guild (src/Web/StellaOps.Web/TASKS.md) WEB-AOC-19-001 `Shared AOC guard primitives` | DOING (2025-10-26) | Provide `AOCForbiddenKeys`, guard middleware/interceptor hooks, and error types (`AOCError`, `AOCViolationCode`) for ingestion services. Publish sample usage + analyzer to ensure guard registered. | BE-Base Platform Guild (src/Web/StellaOps.Web/TASKS.md) +> 2025-11-06: Added the `RequireAocGuard` endpoint extension, wired Concelier advisory ingestion through the shared filter, refreshed docs, and introduced extension tests. WEB-AOC-19-002 `Provenance & signature helpers` | TODO | Ship `ProvenanceBuilder`, checksum utilities, and signature verification helper integrated with guard logging. Cover DSSE/CMS formats with unit tests. Dependencies: WEB-AOC-19-001. | BE-Base Platform Guild (src/Web/StellaOps.Web/TASKS.md) WEB-AOC-19-003 `Analyzer + test fixtures` | TODO | Author Roslyn analyzer preventing ingestion modules from writing forbidden keys without guard, and provide shared test fixtures for guard validation used by Concelier/Excititor service tests. Dependencies: WEB-AOC-19-002. | QA Guild, BE-Base Platform Guild (src/Web/StellaOps.Web/TASKS.md) WEB-CONSOLE-23-001 `Global posture endpoints` | TODO | Provide consolidated `/console/dashboard` and `/console/filters` APIs returning tenant-scoped aggregates (findings by severity, VEX override counts, advisory deltas, run health, policy change log). Enforce AOC labelling, deterministic ordering, and cursor-based pagination for drill-down hints. | BE-Base Platform Guild, Product Analytics Guild (src/Web/StellaOps.Web/TASKS.md) diff --git a/docs/migration/no-merge.md b/docs/migration/no-merge.md index 44b36f224..12b57cfd8 100644 --- a/docs/migration/no-merge.md +++ b/docs/migration/no-merge.md @@ -30,12 +30,12 @@ Do not proceed to Phase 1 until all prerequisites are checked or explicitly wa | Toggle | Default | Purpose | Notes | | --- | --- | --- | --- | -| `concelier:features:noMergeEnabled` | `false` | Master switch to disable legacy Merge job scheduling/execution. | Applies to WebService + Worker; gate `AdvisoryMergeService` DI registration. | +| `concelier:features:noMergeEnabled` | `true` | Master switch to disable legacy Merge job scheduling/execution. | Applies to WebService + Worker; gate `AdvisoryMergeService` DI registration. | | `concelier:features:lnmShadowWrites` | `true` | Enables dual-write of linksets while Merge remains active. | Keep enabled throughout Phase 0–1 to validate parity. | | `concelier:jobs:merge:allowlist` | `[]` | Explicit allowlist for Merge jobs when noMergeEnabled is `false`. | Set to empty during Phase 2+ to prevent accidental restarts. | | `policy:overlays:requireLinksetEvidence` | `false` | Policy engine safety net to require linkset-backed findings. | Flip to `true` only after cutover (Phase 2). | -> 2025-11-05: WebService honours `concelier:features:noMergeEnabled` by skipping Merge DI registration and removing the `merge:reconcile` job definition (MERGE-LNM-21-002). +> 2025-11-06: WebService now defaults `concelier:features:noMergeEnabled` to `true`, skipping Merge DI registration and removing the `merge:reconcile` job unless operators set the flag to `false` and allowlist the job (MERGE-LNM-21-002). > > 2025-11-06: Analyzer `CONCELIER0002` ships with Concelier hosts to block new references to `AdvisoryMergeService` / `AddMergeModule`. Suppressions must be paired with an explicit migration note. > 2025-11-06: Analyzer coverage validated via unit tests catching object creation, field declarations, `typeof`, and DI extension invocations; merge assemblies remain exempt for legacy cleanup helpers. diff --git a/src/Aoc/__Libraries/StellaOps.Aoc.AspNetCore/Routing/AocGuardEndpointFilterExtensions.cs b/src/Aoc/__Libraries/StellaOps.Aoc.AspNetCore/Routing/AocGuardEndpointFilterExtensions.cs new file mode 100644 index 000000000..86d7921bb --- /dev/null +++ b/src/Aoc/__Libraries/StellaOps.Aoc.AspNetCore/Routing/AocGuardEndpointFilterExtensions.cs @@ -0,0 +1,60 @@ +using System; +using System.Collections.Generic; +using System.Text.Json; +using Microsoft.AspNetCore.Builder; + +namespace StellaOps.Aoc.AspNetCore.Routing; + +public static class AocGuardEndpointFilterExtensions +{ + public static RouteHandlerBuilder RequireAocGuard( + this RouteHandlerBuilder builder, + Func> payloadSelector, + JsonSerializerOptions? serializerOptions = null, + AocGuardOptions? guardOptions = null) + { + if (builder is null) + { + throw new ArgumentNullException(nameof(builder)); + } + + if (payloadSelector is null) + { + throw new ArgumentNullException(nameof(payloadSelector)); + } + + builder.Add(endpointBuilder => + { + endpointBuilder.FilterFactories.Add((routeContext, next) => + { + var filter = new AocGuardEndpointFilter(payloadSelector, serializerOptions, guardOptions); + return invocationContext => filter.InvokeAsync(invocationContext, next); + }); + }); + return builder; + } + + public static RouteHandlerBuilder RequireAocGuard( + this RouteHandlerBuilder builder, + Func payloadSelector, + JsonSerializerOptions? serializerOptions = null, + AocGuardOptions? guardOptions = null) + { + if (payloadSelector is null) + { + throw new ArgumentNullException(nameof(payloadSelector)); + } + + return AocGuardEndpointFilterExtensions.RequireAocGuard( + builder, + request => + { + var payload = payloadSelector(request); + return payload is null + ? Array.Empty() + : new object?[] { payload }; + }, + serializerOptions, + guardOptions); + } +} diff --git a/src/Aoc/__Tests/StellaOps.Aoc.AspNetCore.Tests/AocGuardEndpointFilterExtensionsTests.cs b/src/Aoc/__Tests/StellaOps.Aoc.AspNetCore.Tests/AocGuardEndpointFilterExtensionsTests.cs new file mode 100644 index 000000000..b6a061a26 --- /dev/null +++ b/src/Aoc/__Tests/StellaOps.Aoc.AspNetCore.Tests/AocGuardEndpointFilterExtensionsTests.cs @@ -0,0 +1,52 @@ +using System; +using System.Text.Json; +using Microsoft.AspNetCore.Builder; +using Microsoft.AspNetCore.Http; +using Microsoft.Extensions.DependencyInjection; +using StellaOps.Aoc.AspNetCore.Routing; + +namespace StellaOps.Aoc.AspNetCore.Tests; + +public sealed class AocGuardEndpointFilterExtensionsTests +{ + [Fact] + public void RequireAocGuard_ReturnsBuilderInstance() + { + var builder = WebApplication.CreateBuilder(); + builder.Services.AddAocGuard(); + using var app = builder.Build(); + + var route = app.MapPost("/guard", (GuardPayload _) => TypedResults.Ok()); + + var result = route.RequireAocGuard(_ => Array.Empty()); + + Assert.Same(route, result); + } + + [Fact] + public void RequireAocGuard_WithNullBuilder_Throws() + { + RouteHandlerBuilder? builder = null; + + Assert.Throws(() => + AocGuardEndpointFilterExtensions.RequireAocGuard( + builder!, + _ => Array.Empty())); + } + + [Fact] + public void RequireAocGuard_WithObjectSelector_UsesOverload() + { + var builder = WebApplication.CreateBuilder(); + builder.Services.AddAocGuard(); + using var app = builder.Build(); + + var route = app.MapPost("/guard-object", (GuardPayload _) => TypedResults.Ok()); + + var result = route.RequireAocGuard(_ => new GuardPayload(JsonDocument.Parse("{}").RootElement)); + + Assert.Same(route, result); + } + + private sealed record GuardPayload(JsonElement Payload); +} diff --git a/src/Concelier/StellaOps.Concelier.WebService/Extensions/JobRegistrationExtensions.cs b/src/Concelier/StellaOps.Concelier.WebService/Extensions/JobRegistrationExtensions.cs index 271e152d8..f6823294e 100644 --- a/src/Concelier/StellaOps.Concelier.WebService/Extensions/JobRegistrationExtensions.cs +++ b/src/Concelier/StellaOps.Concelier.WebService/Extensions/JobRegistrationExtensions.cs @@ -1,100 +1,134 @@ -using System; -using System.Collections.Generic; -using System.Globalization; -using Microsoft.Extensions.DependencyInjection; -using Microsoft.Extensions.Options; -using StellaOps.Concelier.Core.Jobs; -using StellaOps.Concelier.Merge.Jobs; - -namespace StellaOps.Concelier.WebService.Extensions; - -internal static class JobRegistrationExtensions -{ - private sealed record BuiltInJob( - string Kind, - string JobType, - string AssemblyName, - TimeSpan Timeout, - TimeSpan LeaseDuration, - string? CronExpression = null); - - private static readonly IReadOnlyList BuiltInJobs = new List - { - new("source:redhat:fetch", "StellaOps.Concelier.Connector.Distro.RedHat.RedHatFetchJob", "StellaOps.Concelier.Connector.Distro.RedHat", TimeSpan.FromMinutes(12), TimeSpan.FromMinutes(6), "0,15,30,45 * * * *"), - new("source:redhat:parse", "StellaOps.Concelier.Connector.Distro.RedHat.RedHatParseJob", "StellaOps.Concelier.Connector.Distro.RedHat", TimeSpan.FromMinutes(15), TimeSpan.FromMinutes(6), "5,20,35,50 * * * *"), - new("source:redhat:map", "StellaOps.Concelier.Connector.Distro.RedHat.RedHatMapJob", "StellaOps.Concelier.Connector.Distro.RedHat", TimeSpan.FromMinutes(20), TimeSpan.FromMinutes(6), "10,25,40,55 * * * *"), - - new("source:cert-in:fetch", "StellaOps.Concelier.Connector.CertIn.CertInFetchJob", "StellaOps.Concelier.Connector.CertIn", TimeSpan.FromMinutes(15), TimeSpan.FromMinutes(5)), - new("source:cert-in:parse", "StellaOps.Concelier.Connector.CertIn.CertInParseJob", "StellaOps.Concelier.Connector.CertIn", TimeSpan.FromMinutes(15), TimeSpan.FromMinutes(5)), - new("source:cert-in:map", "StellaOps.Concelier.Connector.CertIn.CertInMapJob", "StellaOps.Concelier.Connector.CertIn", TimeSpan.FromMinutes(15), TimeSpan.FromMinutes(5)), - - new("source:cert-fr:fetch", "StellaOps.Concelier.Connector.CertFr.CertFrFetchJob", "StellaOps.Concelier.Connector.CertFr", TimeSpan.FromMinutes(15), TimeSpan.FromMinutes(5)), - new("source:cert-fr:parse", "StellaOps.Concelier.Connector.CertFr.CertFrParseJob", "StellaOps.Concelier.Connector.CertFr", TimeSpan.FromMinutes(15), TimeSpan.FromMinutes(5)), - new("source:cert-fr:map", "StellaOps.Concelier.Connector.CertFr.CertFrMapJob", "StellaOps.Concelier.Connector.CertFr", TimeSpan.FromMinutes(15), TimeSpan.FromMinutes(5)), - - new("source:jvn:fetch", "StellaOps.Concelier.Connector.Jvn.JvnFetchJob", "StellaOps.Concelier.Connector.Jvn", TimeSpan.FromMinutes(15), TimeSpan.FromMinutes(5)), - new("source:jvn:parse", "StellaOps.Concelier.Connector.Jvn.JvnParseJob", "StellaOps.Concelier.Connector.Jvn", TimeSpan.FromMinutes(15), TimeSpan.FromMinutes(5)), - new("source:jvn:map", "StellaOps.Concelier.Connector.Jvn.JvnMapJob", "StellaOps.Concelier.Connector.Jvn", TimeSpan.FromMinutes(15), TimeSpan.FromMinutes(5)), - - new("source:ics-kaspersky:fetch", "StellaOps.Concelier.Connector.Ics.Kaspersky.KasperskyFetchJob", "StellaOps.Concelier.Connector.Ics.Kaspersky", TimeSpan.FromMinutes(15), TimeSpan.FromMinutes(5)), - new("source:ics-kaspersky:parse", "StellaOps.Concelier.Connector.Ics.Kaspersky.KasperskyParseJob", "StellaOps.Concelier.Connector.Ics.Kaspersky", TimeSpan.FromMinutes(15), TimeSpan.FromMinutes(5)), - new("source:ics-kaspersky:map", "StellaOps.Concelier.Connector.Ics.Kaspersky.KasperskyMapJob", "StellaOps.Concelier.Connector.Ics.Kaspersky", TimeSpan.FromMinutes(15), TimeSpan.FromMinutes(5)), - - new("source:osv:fetch", "StellaOps.Concelier.Connector.Osv.OsvFetchJob", "StellaOps.Concelier.Connector.Osv", TimeSpan.FromMinutes(15), TimeSpan.FromMinutes(5)), - new("source:osv:parse", "StellaOps.Concelier.Connector.Osv.OsvParseJob", "StellaOps.Concelier.Connector.Osv", TimeSpan.FromMinutes(15), TimeSpan.FromMinutes(5)), - new("source:osv:map", "StellaOps.Concelier.Connector.Osv.OsvMapJob", "StellaOps.Concelier.Connector.Osv", TimeSpan.FromMinutes(15), TimeSpan.FromMinutes(5)), - - new("source:vmware:fetch", "StellaOps.Concelier.Connector.Vndr.Vmware.VmwareFetchJob", "StellaOps.Concelier.Connector.Vndr.Vmware", TimeSpan.FromMinutes(15), TimeSpan.FromMinutes(5)), - new("source:vmware:parse", "StellaOps.Concelier.Connector.Vndr.Vmware.VmwareParseJob", "StellaOps.Concelier.Connector.Vndr.Vmware", TimeSpan.FromMinutes(15), TimeSpan.FromMinutes(5)), - new("source:vmware:map", "StellaOps.Concelier.Connector.Vndr.Vmware.VmwareMapJob", "StellaOps.Concelier.Connector.Vndr.Vmware", TimeSpan.FromMinutes(15), TimeSpan.FromMinutes(5)), - - new("source:vndr-oracle:fetch", "StellaOps.Concelier.Connector.Vndr.Oracle.OracleFetchJob", "StellaOps.Concelier.Connector.Vndr.Oracle", TimeSpan.FromMinutes(15), TimeSpan.FromMinutes(5)), - new("source:vndr-oracle:parse", "StellaOps.Concelier.Connector.Vndr.Oracle.OracleParseJob", "StellaOps.Concelier.Connector.Vndr.Oracle", TimeSpan.FromMinutes(15), TimeSpan.FromMinutes(5)), - new("source:vndr-oracle:map", "StellaOps.Concelier.Connector.Vndr.Oracle.OracleMapJob", "StellaOps.Concelier.Connector.Vndr.Oracle", TimeSpan.FromMinutes(15), TimeSpan.FromMinutes(5)), - +using System; +using System.Collections.Generic; +using Microsoft.Extensions.Configuration; +using Microsoft.Extensions.DependencyInjection; +using Microsoft.Extensions.Options; +using StellaOps.Concelier.Core.Jobs; + +namespace StellaOps.Concelier.WebService.Extensions; + +internal static class JobRegistrationExtensions +{ + private sealed record BuiltInJob( + string Kind, + string JobType, + string AssemblyName, + TimeSpan Timeout, + TimeSpan LeaseDuration, + string? CronExpression = null); + + private static readonly IReadOnlyList BaseBuiltInJobs = new List + { + new("source:redhat:fetch", "StellaOps.Concelier.Connector.Distro.RedHat.RedHatFetchJob", "StellaOps.Concelier.Connector.Distro.RedHat", TimeSpan.FromMinutes(12), TimeSpan.FromMinutes(6), "0,15,30,45 * * * *"), + new("source:redhat:parse", "StellaOps.Concelier.Connector.Distro.RedHat.RedHatParseJob", "StellaOps.Concelier.Connector.Distro.RedHat", TimeSpan.FromMinutes(15), TimeSpan.FromMinutes(6), "5,20,35,50 * * * *"), + new("source:redhat:map", "StellaOps.Concelier.Connector.Distro.RedHat.RedHatMapJob", "StellaOps.Concelier.Connector.Distro.RedHat", TimeSpan.FromMinutes(20), TimeSpan.FromMinutes(6), "10,25,40,55 * * * *"), + + new("source:cert-in:fetch", "StellaOps.Concelier.Connector.CertIn.CertInFetchJob", "StellaOps.Concelier.Connector.CertIn", TimeSpan.FromMinutes(15), TimeSpan.FromMinutes(5)), + new("source:cert-in:parse", "StellaOps.Concelier.Connector.CertIn.CertInParseJob", "StellaOps.Concelier.Connector.CertIn", TimeSpan.FromMinutes(15), TimeSpan.FromMinutes(5)), + new("source:cert-in:map", "StellaOps.Concelier.Connector.CertIn.CertInMapJob", "StellaOps.Concelier.Connector.CertIn", TimeSpan.FromMinutes(15), TimeSpan.FromMinutes(5)), + + new("source:cert-fr:fetch", "StellaOps.Concelier.Connector.CertFr.CertFrFetchJob", "StellaOps.Concelier.Connector.CertFr", TimeSpan.FromMinutes(15), TimeSpan.FromMinutes(5)), + new("source:cert-fr:parse", "StellaOps.Concelier.Connector.CertFr.CertFrParseJob", "StellaOps.Concelier.Connector.CertFr", TimeSpan.FromMinutes(15), TimeSpan.FromMinutes(5)), + new("source:cert-fr:map", "StellaOps.Concelier.Connector.CertFr.CertFrMapJob", "StellaOps.Concelier.Connector.CertFr", TimeSpan.FromMinutes(15), TimeSpan.FromMinutes(5)), + + new("source:jvn:fetch", "StellaOps.Concelier.Connector.Jvn.JvnFetchJob", "StellaOps.Concelier.Connector.Jvn", TimeSpan.FromMinutes(15), TimeSpan.FromMinutes(5)), + new("source:jvn:parse", "StellaOps.Concelier.Connector.Jvn.JvnParseJob", "StellaOps.Concelier.Connector.Jvn", TimeSpan.FromMinutes(15), TimeSpan.FromMinutes(5)), + new("source:jvn:map", "StellaOps.Concelier.Connector.Jvn.JvnMapJob", "StellaOps.Concelier.Connector.Jvn", TimeSpan.FromMinutes(15), TimeSpan.FromMinutes(5)), + + new("source:ics-kaspersky:fetch", "StellaOps.Concelier.Connector.Ics.Kaspersky.KasperskyFetchJob", "StellaOps.Concelier.Connector.Ics.Kaspersky", TimeSpan.FromMinutes(15), TimeSpan.FromMinutes(5)), + new("source:ics-kaspersky:parse", "StellaOps.Concelier.Connector.Ics.Kaspersky.KasperskyParseJob", "StellaOps.Concelier.Connector.Ics.Kaspersky", TimeSpan.FromMinutes(15), TimeSpan.FromMinutes(5)), + new("source:ics-kaspersky:map", "StellaOps.Concelier.Connector.Ics.Kaspersky.KasperskyMapJob", "StellaOps.Concelier.Connector.Ics.Kaspersky", TimeSpan.FromMinutes(15), TimeSpan.FromMinutes(5)), + + new("source:osv:fetch", "StellaOps.Concelier.Connector.Osv.OsvFetchJob", "StellaOps.Concelier.Connector.Osv", TimeSpan.FromMinutes(15), TimeSpan.FromMinutes(5)), + new("source:osv:parse", "StellaOps.Concelier.Connector.Osv.OsvParseJob", "StellaOps.Concelier.Connector.Osv", TimeSpan.FromMinutes(15), TimeSpan.FromMinutes(5)), + new("source:osv:map", "StellaOps.Concelier.Connector.Osv.OsvMapJob", "StellaOps.Concelier.Connector.Osv", TimeSpan.FromMinutes(15), TimeSpan.FromMinutes(5)), + + new("source:vmware:fetch", "StellaOps.Concelier.Connector.Vndr.Vmware.VmwareFetchJob", "StellaOps.Concelier.Connector.Vndr.Vmware", TimeSpan.FromMinutes(15), TimeSpan.FromMinutes(5)), + new("source:vmware:parse", "StellaOps.Concelier.Connector.Vndr.Vmware.VmwareParseJob", "StellaOps.Concelier.Connector.Vndr.Vmware", TimeSpan.FromMinutes(15), TimeSpan.FromMinutes(5)), + new("source:vmware:map", "StellaOps.Concelier.Connector.Vndr.Vmware.VmwareMapJob", "StellaOps.Concelier.Connector.Vndr.Vmware", TimeSpan.FromMinutes(15), TimeSpan.FromMinutes(5)), + + new("source:vndr-oracle:fetch", "StellaOps.Concelier.Connector.Vndr.Oracle.OracleFetchJob", "StellaOps.Concelier.Connector.Vndr.Oracle", TimeSpan.FromMinutes(15), TimeSpan.FromMinutes(5)), + new("source:vndr-oracle:parse", "StellaOps.Concelier.Connector.Vndr.Oracle.OracleParseJob", "StellaOps.Concelier.Connector.Vndr.Oracle", TimeSpan.FromMinutes(15), TimeSpan.FromMinutes(5)), + new("source:vndr-oracle:map", "StellaOps.Concelier.Connector.Vndr.Oracle.OracleMapJob", "StellaOps.Concelier.Connector.Vndr.Oracle", TimeSpan.FromMinutes(15), TimeSpan.FromMinutes(5)), + new("export:json", "StellaOps.Concelier.Exporter.Json.JsonExportJob", "StellaOps.Concelier.Exporter.Json", TimeSpan.FromMinutes(10), TimeSpan.FromMinutes(5)), - new("export:trivy-db", "StellaOps.Concelier.Exporter.TrivyDb.TrivyDbExportJob", "StellaOps.Concelier.Exporter.TrivyDb", TimeSpan.FromMinutes(20), TimeSpan.FromMinutes(10)), -#pragma warning disable CS0618, CONCELIER0001 // Legacy merge job remains available until MERGE-LNM-21-002 completes. - new("merge:reconcile", "StellaOps.Concelier.Merge.Jobs.MergeReconcileJob", "StellaOps.Concelier.Merge", TimeSpan.FromMinutes(15), TimeSpan.FromMinutes(5)) -#pragma warning restore CS0618, CONCELIER0001 + new("export:trivy-db", "StellaOps.Concelier.Exporter.TrivyDb.TrivyDbExportJob", "StellaOps.Concelier.Exporter.TrivyDb", TimeSpan.FromMinutes(20), TimeSpan.FromMinutes(10)) }; - - public static IServiceCollection AddBuiltInConcelierJobs(this IServiceCollection services) - { - ArgumentNullException.ThrowIfNull(services); - - services.PostConfigure(options => - { - foreach (var registration in BuiltInJobs) - { - if (options.Definitions.ContainsKey(registration.Kind)) - { - continue; - } - - var jobType = Type.GetType( - $"{registration.JobType}, {registration.AssemblyName}", - throwOnError: false, - ignoreCase: false); - - if (jobType is null) - { - continue; - } - - var timeout = registration.Timeout > TimeSpan.Zero ? registration.Timeout : options.DefaultTimeout; - var lease = registration.LeaseDuration > TimeSpan.Zero ? registration.LeaseDuration : options.DefaultLeaseDuration; - - options.Definitions[registration.Kind] = new JobDefinition( - registration.Kind, - jobType, - timeout, - lease, - registration.CronExpression, - Enabled: true); - } - }); - - return services; - } -} + + private static readonly BuiltInJob MergeReconcileBuiltInJob = new( + "merge:reconcile", + "StellaOps.Concelier.Merge.Jobs.MergeReconcileJob", + "StellaOps.Concelier.Merge", + TimeSpan.FromMinutes(15), + TimeSpan.FromMinutes(5)); + + public static IServiceCollection AddBuiltInConcelierJobs(this IServiceCollection services) + { + ArgumentNullException.ThrowIfNull(services); + + services.AddOptions() + .Configure((options, configuration) => + { + foreach (var registration in BaseBuiltInJobs) + { + AddJobIfMissing(options, registration); + } + + ConfigureMergeJob(options, configuration); + }); + + return services; + } + + private static void AddJobIfMissing(JobSchedulerOptions options, BuiltInJob registration) + { + if (options.Definitions.ContainsKey(registration.Kind)) + { + return; + } + + var jobType = Type.GetType( + $"{registration.JobType}, {registration.AssemblyName}", + throwOnError: false, + ignoreCase: false); + + if (jobType is null) + { + return; + } + + var timeout = registration.Timeout > TimeSpan.Zero ? registration.Timeout : options.DefaultTimeout; + var lease = registration.LeaseDuration > TimeSpan.Zero ? registration.LeaseDuration : options.DefaultLeaseDuration; + + options.Definitions[registration.Kind] = new JobDefinition( + registration.Kind, + jobType, + timeout, + lease, + registration.CronExpression, + Enabled: true); + } + + private static void ConfigureMergeJob(JobSchedulerOptions options, IConfiguration configuration) + { + var noMergeEnabled = configuration.GetValue("concelier:features:noMergeEnabled", true); + if (noMergeEnabled) + { + options.Definitions.Remove(MergeReconcileBuiltInJob.Kind); + return; + } + + var allowlist = configuration.GetSection("concelier:jobs:merge:allowlist").Get(); + if (allowlist is { Length: > 0 }) + { + var allowlistSet = new HashSet(allowlist, StringComparer.OrdinalIgnoreCase); + if (!allowlistSet.Contains(MergeReconcileBuiltInJob.Kind)) + { + options.Definitions.Remove(MergeReconcileBuiltInJob.Kind); + return; + } + } + + AddJobIfMissing(options, MergeReconcileBuiltInJob); + } +} diff --git a/src/Concelier/StellaOps.Concelier.WebService/Program.cs b/src/Concelier/StellaOps.Concelier.WebService/Program.cs index 68f94a12d..6821564d3 100644 --- a/src/Concelier/StellaOps.Concelier.WebService/Program.cs +++ b/src/Concelier/StellaOps.Concelier.WebService/Program.cs @@ -15,7 +15,7 @@ using System.Diagnostics; using System.Text.Json; using System.Text.Json.Serialization; using Microsoft.Extensions.Logging; -using Microsoft.Extensions.Options; +using Microsoft.Extensions.Options; using MongoDB.Bson; using MongoDB.Driver; using StellaOps.Concelier.Core.Events; @@ -40,6 +40,7 @@ using StellaOps.Auth.Abstractions; using StellaOps.Auth.Client; using StellaOps.Auth.ServerIntegration; using StellaOps.Aoc; +using StellaOps.Aoc.AspNetCore.Routing; using StellaOps.Aoc.AspNetCore.Results; using StellaOps.Concelier.WebService.Contracts; using StellaOps.Concelier.Core.Aoc; @@ -427,6 +428,41 @@ var advisoryIngestEndpoint = app.MapPost("/ingest/advisory", async ( return MapAocGuardException(context, guardException); } }); + +var advisoryIngestGuardOptions = AocGuardOptions.Default with +{ + RequireTenant = false, + RequiredTopLevelFields = AocGuardOptions.Default.RequiredTopLevelFields.Remove("tenant") +}; + +advisoryIngestEndpoint.RequireAocGuard(request => +{ + if (request?.Source is null || request.Upstream is null || request.Content is null || request.Identifiers is null) + { + return Array.Empty(); + } + + var linkset = request.Linkset ?? new AdvisoryLinksetRequest( + Array.Empty(), + Array.Empty(), + Array.Empty(), + Array.Empty(), + Array.Empty(), + new Dictionary(StringComparer.Ordinal)); + + var payload = new + { + tenant = "guard-tenant", + source = request.Source, + upstream = request.Upstream, + content = request.Content, + identifiers = request.Identifiers, + linkset + }; + + return new object?[] { payload }; +}, guardOptions: advisoryIngestGuardOptions); + if (authorityConfigured) { advisoryIngestEndpoint.RequireAuthorization(AdvisoryIngestPolicyName); diff --git a/src/Concelier/StellaOps.Concelier.WebService/Properties/AssemblyInfo.cs b/src/Concelier/StellaOps.Concelier.WebService/Properties/AssemblyInfo.cs new file mode 100644 index 000000000..91f2514b1 --- /dev/null +++ b/src/Concelier/StellaOps.Concelier.WebService/Properties/AssemblyInfo.cs @@ -0,0 +1,3 @@ +using System.Runtime.CompilerServices; + +[assembly: InternalsVisibleTo("StellaOps.Concelier.WebService.Tests")] diff --git a/src/Concelier/__Libraries/StellaOps.Concelier.Merge/MergeServiceCollectionExtensions.cs b/src/Concelier/__Libraries/StellaOps.Concelier.Merge/MergeServiceCollectionExtensions.cs index f28a80e7c..1f9add9c0 100644 --- a/src/Concelier/__Libraries/StellaOps.Concelier.Merge/MergeServiceCollectionExtensions.cs +++ b/src/Concelier/__Libraries/StellaOps.Concelier.Merge/MergeServiceCollectionExtensions.cs @@ -5,21 +5,21 @@ using Microsoft.Extensions.DependencyInjection.Extensions; using Microsoft.Extensions.Logging; using StellaOps.Concelier.Core; using StellaOps.Concelier.Merge.Jobs; -using StellaOps.Concelier.Merge.Options; -using StellaOps.Concelier.Merge.Services; - +using StellaOps.Concelier.Merge.Options; +using StellaOps.Concelier.Merge.Services; + namespace StellaOps.Concelier.Merge; [Obsolete("Legacy merge module is deprecated; prefer Link-Not-Merge linkset pipelines. Track MERGE-LNM-21-002 and set concelier:features:noMergeEnabled=true to disable registration.", DiagnosticId = "CONCELIER0001", UrlFormat = "https://stella-ops.org/docs/migration/no-merge")] public static class MergeServiceCollectionExtensions -{ +{ public static IServiceCollection AddMergeModule(this IServiceCollection services, IConfiguration configuration) { ArgumentNullException.ThrowIfNull(services); ArgumentNullException.ThrowIfNull(configuration); var noMergeEnabled = configuration.GetValue("concelier:features:noMergeEnabled"); - if (noMergeEnabled is true) + if (noMergeEnabled) { return services; } @@ -28,20 +28,20 @@ public static class MergeServiceCollectionExtensions services.TryAddSingleton(); services.TryAddSingleton(); services.TryAddSingleton(sp => - { - var options = configuration.GetSection("concelier:merge:precedence").Get(); - return options is null ? new AffectedPackagePrecedenceResolver() : new AffectedPackagePrecedenceResolver(options); - }); - - services.TryAddSingleton(sp => - { - var resolver = sp.GetRequiredService(); - var options = configuration.GetSection("concelier:merge:precedence").Get(); - var timeProvider = sp.GetRequiredService(); - var logger = sp.GetRequiredService>(); - return new AdvisoryPrecedenceMerger(resolver, options, timeProvider, logger); - }); - + { + var options = configuration.GetSection("concelier:merge:precedence").Get(); + return options is null ? new AffectedPackagePrecedenceResolver() : new AffectedPackagePrecedenceResolver(options); + }); + + services.TryAddSingleton(sp => + { + var resolver = sp.GetRequiredService(); + var options = configuration.GetSection("concelier:merge:precedence").Get(); + var timeProvider = sp.GetRequiredService(); + var logger = sp.GetRequiredService>(); + return new AdvisoryPrecedenceMerger(resolver, options, timeProvider, logger); + }); + #pragma warning disable CS0618 // Legacy merge services are marked obsolete. services.TryAddSingleton(); services.TryAddSingleton(); diff --git a/src/Concelier/__Libraries/StellaOps.Concelier.Merge/TASKS.md b/src/Concelier/__Libraries/StellaOps.Concelier.Merge/TASKS.md index 7aa0b3ba7..589bc785a 100644 --- a/src/Concelier/__Libraries/StellaOps.Concelier.Merge/TASKS.md +++ b/src/Concelier/__Libraries/StellaOps.Concelier.Merge/TASKS.md @@ -10,6 +10,6 @@ | Task | Owner(s) | Depends on | Notes | |---|---|---|---| |MERGE-LNM-21-001 Migration plan authoring|BE-Merge, Architecture Guild|CONCELIER-LNM-21-101|**DONE (2025-11-03)** – Authored `docs/migration/no-merge.md` with rollout phases, backfill/validation checklists, rollback guidance, and ownership matrix for the Link-Not-Merge cutover.| -|MERGE-LNM-21-002 Merge service deprecation|BE-Merge|MERGE-LNM-21-001|**DOING (2025-11-03)** – Auditing service registrations, DI bindings, and tests consuming `AdvisoryMergeService`; drafting deprecation plan and analyzer scope prior to code removal.
2025-11-05 14:42Z: Implementing `concelier:features:noMergeEnabled` gate, merge job allowlist checks, `[Obsolete]` markings, and analyzer scaffolding to steer consumers toward linkset APIs.
2025-11-06 16:10Z: Introduced Roslyn analyzer (`CONCELIER0002`) referenced by Concelier WebService + tests, documented suppression guidance, and updated migration playbook.
2025-11-06 23:45Z: Analyzer enforcement merged; DI removal + feature-flag default change remain. Analyzer tests compile locally but restore blocked offline (`Microsoft.Bcl.AsyncInterfaces >= 8.0` absent) — capture follow-up once nuget mirror updated.| +|MERGE-LNM-21-002 Merge service deprecation|BE-Merge|MERGE-LNM-21-001|**DONE (2025-11-06)** – Audited service registrations, gated legacy bindings, and delivered analyzer coverage ahead of removal.
2025-11-05 14:42Z: Implemented `concelier:features:noMergeEnabled` gate, merge job allowlist checks, `[Obsolete]` markings, and analyzer scaffolding to steer consumers toward linkset APIs.
2025-11-06 16:10Z: Introduced Roslyn analyzer (`CONCELIER0002`) referenced by Concelier WebService + tests, documented suppression guidance, and updated migration playbook.
2025-11-06 23:58Z: Defaulted `concelier:features:noMergeEnabled` to `true`, removed the built-in `merge:reconcile` job unless explicitly allowlisted, refreshed WebService tests/docs, and verified analyzer suites restore against local feeds.| > 2025-11-03: Catalogued call sites (WebService Program `AddMergeModule`, built-in job registration `merge:reconcile`, `MergeReconcileJob`) and confirmed unit tests are the only direct `MergeAsync` callers; next step is to define analyzer + replacement observability coverage. |MERGE-LNM-21-003 Determinism/test updates|QA Guild, BE-Merge|MERGE-LNM-21-002|Replace merge determinism suites with observation/linkset regression tests verifying no data mutation and conflicts remain visible.| diff --git a/src/Concelier/__Tests/StellaOps.Concelier.WebService.Tests/WebServiceEndpointsTests.cs b/src/Concelier/__Tests/StellaOps.Concelier.WebService.Tests/WebServiceEndpointsTests.cs index cfe0dbb52..afdde007c 100644 --- a/src/Concelier/__Tests/StellaOps.Concelier.WebService.Tests/WebServiceEndpointsTests.cs +++ b/src/Concelier/__Tests/StellaOps.Concelier.WebService.Tests/WebServiceEndpointsTests.cs @@ -1,907 +1,903 @@ -using System; -using System.Collections.Generic; -using System.Diagnostics.Metrics; -using System.Globalization; -using System.IdentityModel.Tokens.Jwt; -using System.IO; -using System.Linq; -using System.Net; -using System.Net.Http.Json; -using System.Net.Http.Headers; -using System.Security.Claims; -using System.Text; -using System.Text.Json; -using Microsoft.AspNetCore.Builder; -using Microsoft.AspNetCore.Hosting; -using Microsoft.AspNetCore.Mvc.Testing; -using Microsoft.AspNetCore.TestHost; -using Microsoft.AspNetCore.Authentication.JwtBearer; -using Microsoft.Extensions.Configuration; -using Microsoft.Extensions.DependencyInjection; -using Microsoft.Extensions.Logging; -using Microsoft.Extensions.Options; -using Mongo2Go; -using MongoDB.Bson; -using MongoDB.Driver; -using StellaOps.Concelier.Core.Events; -using StellaOps.Concelier.Core.Jobs; -using StellaOps.Concelier.Models; -using StellaOps.Concelier.Merge.Services; -using StellaOps.Concelier.Storage.Mongo; -using StellaOps.Concelier.Storage.Mongo.Observations; -using StellaOps.Concelier.WebService.Jobs; -using StellaOps.Concelier.WebService.Options; -using StellaOps.Concelier.WebService.Contracts; -using Xunit.Sdk; -using StellaOps.Auth.Abstractions; -using StellaOps.Auth.Client; -using Xunit; -using Microsoft.IdentityModel.Protocols; -using Microsoft.IdentityModel.Protocols.OpenIdConnect; -using Microsoft.IdentityModel.Tokens; - -namespace StellaOps.Concelier.WebService.Tests; - -public sealed class WebServiceEndpointsTests : IAsyncLifetime -{ - private const string TestAuthorityIssuer = "https://authority.example"; - private const string TestAuthorityAudience = "api://concelier"; - private const string TestSigningSecret = "0123456789ABCDEF0123456789ABCDEF"; - private static readonly SymmetricSecurityKey TestSigningKey = new(Encoding.UTF8.GetBytes(TestSigningSecret)); - - private MongoDbRunner _runner = null!; - private ConcelierApplicationFactory _factory = null!; - - public Task InitializeAsync() - { - _runner = MongoDbRunner.Start(singleNodeReplSet: true); - _factory = new ConcelierApplicationFactory(_runner.ConnectionString); - return Task.CompletedTask; - } - - public Task DisposeAsync() - { - _factory.Dispose(); - _runner.Dispose(); - return Task.CompletedTask; - } - - [Fact] - public async Task HealthAndReadyEndpointsRespond() - { - using var client = _factory.CreateClient(); - - var healthResponse = await client.GetAsync("/health"); - if (!healthResponse.IsSuccessStatusCode) - { - var body = await healthResponse.Content.ReadAsStringAsync(); - throw new Xunit.Sdk.XunitException($"/health failed: {(int)healthResponse.StatusCode} {body}"); - } - - var readyResponse = await client.GetAsync("/ready"); - if (!readyResponse.IsSuccessStatusCode) - { - var body = await readyResponse.Content.ReadAsStringAsync(); - throw new Xunit.Sdk.XunitException($"/ready failed: {(int)readyResponse.StatusCode} {body}"); - } - - var healthPayload = await healthResponse.Content.ReadFromJsonAsync(); - Assert.NotNull(healthPayload); - Assert.Equal("healthy", healthPayload!.Status); - Assert.Equal("mongo", healthPayload.Storage.Driver); - - var readyPayload = await readyResponse.Content.ReadFromJsonAsync(); - Assert.NotNull(readyPayload); - Assert.Equal("ready", readyPayload!.Status); - Assert.Equal("ready", readyPayload.Mongo.Status); - } - - [Fact] - public async Task ObservationsEndpoint_ReturnsTenantScopedResults() - { - await SeedObservationDocumentsAsync(BuildSampleObservationDocuments()); - - using var client = _factory.CreateClient(); - - var response = await client.GetAsync("/concelier/observations?tenant=tenant-a&alias=CVE-2025-0001"); - if (!response.IsSuccessStatusCode) - { - var body = await response.Content.ReadAsStringAsync(); - throw new XunitException($"/concelier/observations failed: {(int)response.StatusCode} {body}"); - } - - using var document = await response.Content.ReadFromJsonAsync(); - Assert.NotNull(document); - var root = document!.RootElement; - var observations = root.GetProperty("observations").EnumerateArray().ToArray(); - Assert.Equal(2, observations.Length); - Assert.Equal("tenant-a:ghsa:beta:1", observations[0].GetProperty("observationId").GetString()); - Assert.Equal("tenant-a:nvd:alpha:1", observations[1].GetProperty("observationId").GetString()); - - var linkset = root.GetProperty("linkset"); - Assert.Equal(new[] { "cve-2025-0001", "ghsa-2025-xyz" }, linkset.GetProperty("aliases").EnumerateArray().Select(x => x.GetString()).ToArray()); - Assert.Equal(new[] { "pkg:npm/demo@1.0.0", "pkg:npm/demo@1.1.0" }, linkset.GetProperty("purls").EnumerateArray().Select(x => x.GetString()).ToArray()); - Assert.Equal(new[] { "cpe:/a:vendor:product:1.0", "cpe:/a:vendor:product:1.1" }, linkset.GetProperty("cpes").EnumerateArray().Select(x => x.GetString()).ToArray()); - - var references = linkset.GetProperty("references").EnumerateArray().ToArray(); - Assert.Equal(2, references.Length); - Assert.Equal("advisory", references[0].GetProperty("type").GetString()); - Assert.Equal("https://example.test/advisory-1", references[0].GetProperty("url").GetString()); - Assert.Equal("patch", references[1].GetProperty("type").GetString()); - - Assert.False(root.GetProperty("hasMore").GetBoolean()); - Assert.True(root.GetProperty("nextCursor").ValueKind == JsonValueKind.Null); - } - - [Fact] - public async Task ObservationsEndpoint_AppliesObservationIdFilter() - { - await SeedObservationDocumentsAsync(BuildSampleObservationDocuments()); - - using var client = _factory.CreateClient(); - var observationId = Uri.EscapeDataString("tenant-a:ghsa:beta:1"); - var response = await client.GetAsync($"/concelier/observations?tenant=tenant-a&observationId={observationId}&cpe=cpe:/a:vendor:product:1.1"); - if (!response.IsSuccessStatusCode) - { - var body = await response.Content.ReadAsStringAsync(); - throw new XunitException($"/concelier/observations filter failed: {(int)response.StatusCode} {body}"); - } - - using var document = await response.Content.ReadFromJsonAsync(); - Assert.NotNull(document); - var root = document!.RootElement; - var observations = root.GetProperty("observations").EnumerateArray().ToArray(); - Assert.Single(observations); - Assert.Equal("tenant-a:ghsa:beta:1", observations[0].GetProperty("observationId").GetString()); - Assert.Equal(new[] { "pkg:npm/demo@1.1.0" }, observations[0].GetProperty("linkset").GetProperty("purls").EnumerateArray().Select(x => x.GetString()).ToArray()); - Assert.Equal(new[] { "cpe:/a:vendor:product:1.1" }, observations[0].GetProperty("linkset").GetProperty("cpes").EnumerateArray().Select(x => x.GetString()).ToArray()); - - Assert.False(root.GetProperty("hasMore").GetBoolean()); - Assert.True(root.GetProperty("nextCursor").ValueKind == JsonValueKind.Null); - } - - [Fact] - public async Task ObservationsEndpoint_SupportsPagination() - { - await SeedObservationDocumentsAsync(BuildSampleObservationDocuments()); - - using var client = _factory.CreateClient(); - - var firstResponse = await client.GetAsync("/concelier/observations?tenant=tenant-a&limit=1"); - firstResponse.EnsureSuccessStatusCode(); - using var firstDocument = await firstResponse.Content.ReadFromJsonAsync(); - Assert.NotNull(firstDocument); - var firstRoot = firstDocument!.RootElement; - var firstObservations = firstRoot.GetProperty("observations").EnumerateArray().ToArray(); - Assert.Single(firstObservations); - var nextCursor = firstRoot.GetProperty("nextCursor").GetString(); - Assert.True(firstRoot.GetProperty("hasMore").GetBoolean()); - Assert.False(string.IsNullOrWhiteSpace(nextCursor)); - - var secondResponse = await client.GetAsync($"/concelier/observations?tenant=tenant-a&limit=2&cursor={Uri.EscapeDataString(nextCursor!)}"); - secondResponse.EnsureSuccessStatusCode(); - using var secondDocument = await secondResponse.Content.ReadFromJsonAsync(); - Assert.NotNull(secondDocument); - var secondRoot = secondDocument!.RootElement; - var secondObservations = secondRoot.GetProperty("observations").EnumerateArray().ToArray(); - Assert.Single(secondObservations); - Assert.False(secondRoot.GetProperty("hasMore").GetBoolean()); - Assert.True(secondRoot.GetProperty("nextCursor").ValueKind == JsonValueKind.Null); - Assert.Equal("tenant-a:nvd:alpha:1", secondObservations[0].GetProperty("observationId").GetString()); - } - - [Fact] - public async Task ObservationsEndpoint_ReturnsBadRequestWhenTenantMissing() - { - using var client = _factory.CreateClient(); - var response = await client.GetAsync("/concelier/observations"); - var body = await response.Content.ReadAsStringAsync(); - Assert.True(response.StatusCode == HttpStatusCode.BadRequest, $"Expected 400 but got {(int)response.StatusCode}: {body}"); - } - - [Fact] - public async Task AdvisoryIngestEndpoint_PersistsDocumentAndSupportsReadback() - { - using var client = _factory.CreateClient(); - client.DefaultRequestHeaders.Add("X-Stella-Tenant", "tenant-ingest"); - - var ingestRequest = BuildAdvisoryIngestRequest( - contentHash: "sha256:abc123", - upstreamId: "GHSA-INGEST-0001"); - - var ingestResponse = await client.PostAsJsonAsync("/ingest/advisory", ingestRequest); - Assert.Equal(HttpStatusCode.Created, ingestResponse.StatusCode); - - var ingestPayload = await ingestResponse.Content.ReadFromJsonAsync(); - Assert.NotNull(ingestPayload); - Assert.True(ingestPayload!.Inserted); - Assert.False(string.IsNullOrWhiteSpace(ingestPayload.Id)); - Assert.Equal("tenant-ingest", ingestPayload.Tenant); - Assert.Equal("sha256:abc123", ingestPayload.ContentHash); - Assert.NotNull(ingestResponse.Headers.Location); - var locationValue = ingestResponse.Headers.Location!.ToString(); - Assert.False(string.IsNullOrWhiteSpace(locationValue)); +using System; +using System.Collections.Generic; +using System.Diagnostics.Metrics; +using System.Globalization; +using System.IdentityModel.Tokens.Jwt; +using System.IO; +using System.Linq; +using System.Net; +using System.Net.Http.Json; +using System.Net.Http.Headers; +using System.Security.Claims; +using System.Text; +using System.Text.Json; +using Microsoft.AspNetCore.Builder; +using Microsoft.AspNetCore.Hosting; +using Microsoft.AspNetCore.Mvc.Testing; +using Microsoft.AspNetCore.TestHost; +using Microsoft.AspNetCore.Authentication.JwtBearer; +using Microsoft.Extensions.Configuration; +using Microsoft.Extensions.DependencyInjection; +using Microsoft.Extensions.Logging; +using Microsoft.Extensions.Options; +using Mongo2Go; +using MongoDB.Bson; +using MongoDB.Driver; +using StellaOps.Concelier.Core.Events; +using StellaOps.Concelier.Core.Jobs; +using StellaOps.Concelier.Models; +using StellaOps.Concelier.Merge.Services; +using StellaOps.Concelier.Storage.Mongo; +using StellaOps.Concelier.Storage.Mongo.Observations; +using StellaOps.Concelier.WebService.Jobs; +using StellaOps.Concelier.WebService.Options; +using StellaOps.Concelier.WebService.Contracts; +using Xunit.Sdk; +using StellaOps.Auth.Abstractions; +using StellaOps.Auth.Client; +using Xunit; +using Microsoft.IdentityModel.Protocols; +using Microsoft.IdentityModel.Protocols.OpenIdConnect; +using StellaOps.Concelier.WebService.Diagnostics; +using Microsoft.IdentityModel.Tokens; + +namespace StellaOps.Concelier.WebService.Tests; + +public sealed class WebServiceEndpointsTests : IAsyncLifetime +{ + private const string TestAuthorityIssuer = "https://authority.example"; + private const string TestAuthorityAudience = "api://concelier"; + private const string TestSigningSecret = "0123456789ABCDEF0123456789ABCDEF"; + private static readonly SymmetricSecurityKey TestSigningKey = new(Encoding.UTF8.GetBytes(TestSigningSecret)); + + private MongoDbRunner _runner = null!; + private ConcelierApplicationFactory _factory = null!; + + public Task InitializeAsync() + { + _runner = MongoDbRunner.Start(singleNodeReplSet: true); + _factory = new ConcelierApplicationFactory(_runner.ConnectionString); + return Task.CompletedTask; + } + + public Task DisposeAsync() + { + _factory.Dispose(); + _runner.Dispose(); + return Task.CompletedTask; + } + + [Fact] + public async Task HealthAndReadyEndpointsRespond() + { + using var client = _factory.CreateClient(); + + var healthResponse = await client.GetAsync("/health"); + if (!healthResponse.IsSuccessStatusCode) + { + var body = await healthResponse.Content.ReadAsStringAsync(); + throw new Xunit.Sdk.XunitException($"/health failed: {(int)healthResponse.StatusCode} {body}"); + } + + var readyResponse = await client.GetAsync("/ready"); + if (!readyResponse.IsSuccessStatusCode) + { + var body = await readyResponse.Content.ReadAsStringAsync(); + throw new Xunit.Sdk.XunitException($"/ready failed: {(int)readyResponse.StatusCode} {body}"); + } + + var healthPayload = await healthResponse.Content.ReadFromJsonAsync(); + Assert.NotNull(healthPayload); + Assert.Equal("healthy", healthPayload!.Status); + Assert.Equal("mongo", healthPayload.Storage.Driver); + + var readyPayload = await readyResponse.Content.ReadFromJsonAsync(); + Assert.NotNull(readyPayload); + Assert.Equal("ready", readyPayload!.Status); + Assert.Equal("ready", readyPayload.Mongo.Status); + } + + [Fact] + public async Task ObservationsEndpoint_ReturnsTenantScopedResults() + { + await SeedObservationDocumentsAsync(BuildSampleObservationDocuments()); + + using var client = _factory.CreateClient(); + + var response = await client.GetAsync("/concelier/observations?tenant=tenant-a&alias=CVE-2025-0001"); + if (!response.IsSuccessStatusCode) + { + var body = await response.Content.ReadAsStringAsync(); + throw new XunitException($"/concelier/observations failed: {(int)response.StatusCode} {body}"); + } + + using var document = await response.Content.ReadFromJsonAsync(); + Assert.NotNull(document); + var root = document!.RootElement; + var observations = root.GetProperty("observations").EnumerateArray().ToArray(); + Assert.Equal(2, observations.Length); + Assert.Equal("tenant-a:ghsa:beta:1", observations[0].GetProperty("observationId").GetString()); + Assert.Equal("tenant-a:nvd:alpha:1", observations[1].GetProperty("observationId").GetString()); + + var linkset = root.GetProperty("linkset"); + Assert.Equal(new[] { "cve-2025-0001", "ghsa-2025-xyz" }, linkset.GetProperty("aliases").EnumerateArray().Select(x => x.GetString()).ToArray()); + Assert.Equal(new[] { "pkg:npm/demo@1.0.0", "pkg:npm/demo@1.1.0" }, linkset.GetProperty("purls").EnumerateArray().Select(x => x.GetString()).ToArray()); + Assert.Equal(new[] { "cpe:/a:vendor:product:1.0", "cpe:/a:vendor:product:1.1" }, linkset.GetProperty("cpes").EnumerateArray().Select(x => x.GetString()).ToArray()); + + var references = linkset.GetProperty("references").EnumerateArray().ToArray(); + Assert.Equal(2, references.Length); + Assert.Equal("advisory", references[0].GetProperty("type").GetString()); + Assert.Equal("https://example.test/advisory-1", references[0].GetProperty("url").GetString()); + Assert.Equal("patch", references[1].GetProperty("type").GetString()); + + Assert.False(root.GetProperty("hasMore").GetBoolean()); + Assert.True(root.GetProperty("nextCursor").ValueKind == JsonValueKind.Null); + } + + [Fact] + public async Task ObservationsEndpoint_AppliesObservationIdFilter() + { + await SeedObservationDocumentsAsync(BuildSampleObservationDocuments()); + + using var client = _factory.CreateClient(); + var observationId = Uri.EscapeDataString("tenant-a:ghsa:beta:1"); + var response = await client.GetAsync($"/concelier/observations?tenant=tenant-a&observationId={observationId}&cpe=cpe:/a:vendor:product:1.1"); + if (!response.IsSuccessStatusCode) + { + var body = await response.Content.ReadAsStringAsync(); + throw new XunitException($"/concelier/observations filter failed: {(int)response.StatusCode} {body}"); + } + + using var document = await response.Content.ReadFromJsonAsync(); + Assert.NotNull(document); + var root = document!.RootElement; + var observations = root.GetProperty("observations").EnumerateArray().ToArray(); + Assert.Single(observations); + Assert.Equal("tenant-a:ghsa:beta:1", observations[0].GetProperty("observationId").GetString()); + Assert.Equal(new[] { "pkg:npm/demo@1.1.0" }, observations[0].GetProperty("linkset").GetProperty("purls").EnumerateArray().Select(x => x.GetString()).ToArray()); + Assert.Equal(new[] { "cpe:/a:vendor:product:1.1" }, observations[0].GetProperty("linkset").GetProperty("cpes").EnumerateArray().Select(x => x.GetString()).ToArray()); + + Assert.False(root.GetProperty("hasMore").GetBoolean()); + Assert.True(root.GetProperty("nextCursor").ValueKind == JsonValueKind.Null); + } + + [Fact] + public async Task ObservationsEndpoint_SupportsPagination() + { + await SeedObservationDocumentsAsync(BuildSampleObservationDocuments()); + + using var client = _factory.CreateClient(); + + var firstResponse = await client.GetAsync("/concelier/observations?tenant=tenant-a&limit=1"); + firstResponse.EnsureSuccessStatusCode(); + using var firstDocument = await firstResponse.Content.ReadFromJsonAsync(); + Assert.NotNull(firstDocument); + var firstRoot = firstDocument!.RootElement; + var firstObservations = firstRoot.GetProperty("observations").EnumerateArray().ToArray(); + Assert.Single(firstObservations); + var nextCursor = firstRoot.GetProperty("nextCursor").GetString(); + Assert.True(firstRoot.GetProperty("hasMore").GetBoolean()); + Assert.False(string.IsNullOrWhiteSpace(nextCursor)); + + var secondResponse = await client.GetAsync($"/concelier/observations?tenant=tenant-a&limit=2&cursor={Uri.EscapeDataString(nextCursor!)}"); + secondResponse.EnsureSuccessStatusCode(); + using var secondDocument = await secondResponse.Content.ReadFromJsonAsync(); + Assert.NotNull(secondDocument); + var secondRoot = secondDocument!.RootElement; + var secondObservations = secondRoot.GetProperty("observations").EnumerateArray().ToArray(); + Assert.Single(secondObservations); + Assert.False(secondRoot.GetProperty("hasMore").GetBoolean()); + Assert.True(secondRoot.GetProperty("nextCursor").ValueKind == JsonValueKind.Null); + Assert.Equal("tenant-a:nvd:alpha:1", secondObservations[0].GetProperty("observationId").GetString()); + } + + [Fact] + public async Task ObservationsEndpoint_ReturnsBadRequestWhenTenantMissing() + { + using var client = _factory.CreateClient(); + var response = await client.GetAsync("/concelier/observations"); + var body = await response.Content.ReadAsStringAsync(); + Assert.True(response.StatusCode == HttpStatusCode.BadRequest, $"Expected 400 but got {(int)response.StatusCode}: {body}"); + } + + [Fact] + public async Task AdvisoryIngestEndpoint_PersistsDocumentAndSupportsReadback() + { + using var client = _factory.CreateClient(); + client.DefaultRequestHeaders.Add("X-Stella-Tenant", "tenant-ingest"); + + var ingestRequest = BuildAdvisoryIngestRequest( + contentHash: "sha256:abc123", + upstreamId: "GHSA-INGEST-0001"); + + var ingestResponse = await client.PostAsJsonAsync("/ingest/advisory", ingestRequest); + Assert.Equal(HttpStatusCode.Created, ingestResponse.StatusCode); + + var ingestPayload = await ingestResponse.Content.ReadFromJsonAsync(); + Assert.NotNull(ingestPayload); + Assert.True(ingestPayload!.Inserted); + Assert.False(string.IsNullOrWhiteSpace(ingestPayload.Id)); + Assert.Equal("tenant-ingest", ingestPayload.Tenant); + Assert.Equal("sha256:abc123", ingestPayload.ContentHash); + Assert.NotNull(ingestResponse.Headers.Location); + var locationValue = ingestResponse.Headers.Location!.ToString(); + Assert.False(string.IsNullOrWhiteSpace(locationValue)); var lastSlashIndex = locationValue.LastIndexOf('/'); - var idSegment = lastSlashIndex >= 0 - ? locationValue[(lastSlashIndex + 1)..] - : locationValue; - var decodedSegment = Uri.UnescapeDataString(idSegment); - Assert.Equal(ingestPayload.Id, decodedSegment); - - var duplicateResponse = await client.PostAsJsonAsync("/ingest/advisory", BuildAdvisoryIngestRequest( - contentHash: "sha256:abc123", - upstreamId: "GHSA-INGEST-0001")); - Assert.Equal(HttpStatusCode.OK, duplicateResponse.StatusCode); - var duplicatePayload = await duplicateResponse.Content.ReadFromJsonAsync(); - Assert.NotNull(duplicatePayload); - Assert.False(duplicatePayload!.Inserted); - - using (var getRequest = new HttpRequestMessage(HttpMethod.Get, $"/advisories/raw/{ingestPayload.Id}")) - { - getRequest.Headers.Add("X-Stella-Tenant", "tenant-ingest"); - var getResponse = await client.SendAsync(getRequest); - getResponse.EnsureSuccessStatusCode(); - - var record = await getResponse.Content.ReadFromJsonAsync(); - Assert.NotNull(record); - Assert.Equal(ingestPayload.Id, record!.Id); - Assert.Equal("tenant-ingest", record.Tenant); - Assert.Equal("sha256:abc123", record.Document.Upstream.ContentHash); - } - - using (var listRequest = new HttpRequestMessage(HttpMethod.Get, "/advisories/raw?limit=10")) - { - listRequest.Headers.Add("X-Stella-Tenant", "tenant-ingest"); - var listResponse = await client.SendAsync(listRequest); - listResponse.EnsureSuccessStatusCode(); - - var listPayload = await listResponse.Content.ReadFromJsonAsync(); - Assert.NotNull(listPayload); - var record = Assert.Single(listPayload!.Records); - Assert.Equal(ingestPayload.Id, record.Id); - } - } - - [Fact] - public async Task AocVerifyEndpoint_ReturnsSummaryForTenant() - { - using var client = _factory.CreateClient(); - client.DefaultRequestHeaders.Add("X-Stella-Tenant", "tenant-verify"); - - await client.PostAsJsonAsync("/ingest/advisory", BuildAdvisoryIngestRequest( - contentHash: "sha256:verify-1", - upstreamId: "GHSA-VERIFY-001")); - - var verifyResponse = await client.PostAsJsonAsync("/aoc/verify", new AocVerifyRequest(null, null, null, null, null)); - verifyResponse.EnsureSuccessStatusCode(); - - var verifyPayload = await verifyResponse.Content.ReadFromJsonAsync(); - Assert.NotNull(verifyPayload); - Assert.Equal("tenant-verify", verifyPayload!.Tenant); - Assert.True(verifyPayload.Checked.Advisories >= 1); - Assert.Equal(0, verifyPayload.Checked.Vex); - Assert.True(verifyPayload.Metrics.IngestionWriteTotal >= verifyPayload.Checked.Advisories); - Assert.Empty(verifyPayload.Violations); - Assert.False(verifyPayload.Truncated); - } - - [Fact] - public async Task AocVerifyEndpoint_ReturnsViolationsForGuardFailures() - { - await SeedAdvisoryRawDocumentsAsync( - CreateAdvisoryRawDocument( - tenant: "tenant-verify-violations", - vendor: "osv", - upstreamId: "GHSA-VERIFY-ERR", - contentHash: string.Empty, - raw: new BsonDocument - { - { "id", "GHSA-VERIFY-ERR" } - })); - - using var client = _factory.CreateClient(); - client.DefaultRequestHeaders.Add("X-Stella-Tenant", "tenant-verify-violations"); - - var verifyResponse = await client.PostAsJsonAsync("/aoc/verify", new AocVerifyRequest(null, null, null, null, null)); - verifyResponse.EnsureSuccessStatusCode(); - - var verifyPayload = await verifyResponse.Content.ReadFromJsonAsync(); - Assert.NotNull(verifyPayload); - Assert.Equal("tenant-verify-violations", verifyPayload!.Tenant); - Assert.True(verifyPayload.Checked.Advisories >= 1); - var violation = Assert.Single(verifyPayload.Violations); - Assert.Equal("ERR_AOC_001", violation.Code); - Assert.True(violation.Count >= 1); - Assert.NotEmpty(violation.Examples); - } - - [Fact] - public async Task AdvisoryRawListEndpoint_SupportsCursorPagination() - { - using var client = _factory.CreateClient(); - client.DefaultRequestHeaders.Add("X-Stella-Tenant", "tenant-list"); - - await client.PostAsJsonAsync("/ingest/advisory", BuildAdvisoryIngestRequest("sha256:list-1", "GHSA-LIST-001")); - await client.PostAsJsonAsync("/ingest/advisory", BuildAdvisoryIngestRequest("sha256:list-2", "GHSA-LIST-002")); - await client.PostAsJsonAsync("/ingest/advisory", BuildAdvisoryIngestRequest("sha256:list-3", "GHSA-LIST-003")); - - using var firstRequest = new HttpRequestMessage(HttpMethod.Get, "/advisories/raw?limit=2"); - firstRequest.Headers.Add("X-Stella-Tenant", "tenant-list"); - var firstResponse = await client.SendAsync(firstRequest); - firstResponse.EnsureSuccessStatusCode(); - - var firstPage = await firstResponse.Content.ReadFromJsonAsync(); - Assert.NotNull(firstPage); - Assert.Equal(2, firstPage!.Records.Count); - Assert.True(firstPage.HasMore); - Assert.False(string.IsNullOrWhiteSpace(firstPage.NextCursor)); - - using var secondRequest = new HttpRequestMessage(HttpMethod.Get, $"/advisories/raw?cursor={Uri.EscapeDataString(firstPage.NextCursor!)}"); - secondRequest.Headers.Add("X-Stella-Tenant", "tenant-list"); - var secondResponse = await client.SendAsync(secondRequest); - secondResponse.EnsureSuccessStatusCode(); - - var secondPage = await secondResponse.Content.ReadFromJsonAsync(); - Assert.NotNull(secondPage); - Assert.Single(secondPage!.Records); - Assert.False(secondPage.HasMore); - Assert.Null(secondPage.NextCursor); - - var firstIds = firstPage.Records.Select(record => record.Id).ToArray(); - var secondIds = secondPage.Records.Select(record => record.Id).ToArray(); - Assert.Empty(firstIds.Intersect(secondIds)); - } - - [Fact] - public async Task AdvisoryIngestEndpoint_EmitsMetricsWithExpectedTags() - { - var measurements = await CaptureMetricsAsync( - IngestionMetrics.MeterName, - "ingestion_write_total", - async () => - { - using var client = _factory.CreateClient(); - client.DefaultRequestHeaders.Add("X-Stella-Tenant", "tenant-metrics"); - - await client.PostAsJsonAsync("/ingest/advisory", BuildAdvisoryIngestRequest("sha256:metric-1", "GHSA-METRIC-001")); - await client.PostAsJsonAsync("/ingest/advisory", BuildAdvisoryIngestRequest("sha256:metric-1", "GHSA-METRIC-001")); - }); - - Assert.Equal(2, measurements.Count); - - var inserted = measurements.FirstOrDefault(measurement => - string.Equals(GetTagValue(measurement, "tenant"), "tenant-metrics", StringComparison.Ordinal) && - string.Equals(GetTagValue(measurement, "result"), "inserted", StringComparison.Ordinal)); - Assert.NotNull(inserted); - Assert.Equal(1, inserted!.Value); - Assert.Equal("osv", GetTagValue(inserted, "source")); - - var duplicate = measurements.FirstOrDefault(measurement => - string.Equals(GetTagValue(measurement, "tenant"), "tenant-metrics", StringComparison.Ordinal) && - string.Equals(GetTagValue(measurement, "result"), "duplicate", StringComparison.Ordinal)); - Assert.NotNull(duplicate); - Assert.Equal(1, duplicate!.Value); - Assert.Equal("osv", GetTagValue(duplicate, "source")); - } - - [Fact] - public async Task AocVerifyEndpoint_EmitsVerificationMetric() - { - var measurements = await CaptureMetricsAsync( - IngestionMetrics.MeterName, - "verify_runs_total", - async () => - { - using var client = _factory.CreateClient(); - client.DefaultRequestHeaders.Add("X-Stella-Tenant", "tenant-verify-metrics"); - - await client.PostAsJsonAsync("/ingest/advisory", BuildAdvisoryIngestRequest("sha256:verify-metric", "GHSA-VERIFY-METRIC")); - var verifyResponse = await client.PostAsJsonAsync("/aoc/verify", new AocVerifyRequest(null, null, null, null, null)); - verifyResponse.EnsureSuccessStatusCode(); - }); - - var measurement = Assert.Single(measurements); - Assert.Equal("tenant-verify-metrics", GetTagValue(measurement, "tenant")); - Assert.Equal("ok", GetTagValue(measurement, "result")); - Assert.Equal(1, measurement.Value); - } - - [Fact] - public async Task AdvisoryIngestEndpoint_RejectsCrossTenantWhenAuthenticated() - { - var environment = new Dictionary - { - ["CONCELIER_AUTHORITY__ENABLED"] = "true", - ["CONCELIER_AUTHORITY__ALLOWANONYMOUSFALLBACK"] = "false", - ["CONCELIER_AUTHORITY__ISSUER"] = TestAuthorityIssuer, - ["CONCELIER_AUTHORITY__REQUIREHTTPSMETADATA"] = "false", - ["CONCELIER_AUTHORITY__AUDIENCES__0"] = TestAuthorityAudience, - ["CONCELIER_AUTHORITY__CLIENTID"] = "webservice-tests", - ["CONCELIER_AUTHORITY__CLIENTSECRET"] = "unused", - }; - - using var factory = new ConcelierApplicationFactory( - _runner.ConnectionString, - authority => - { - authority.Enabled = true; - authority.AllowAnonymousFallback = false; - authority.Issuer = TestAuthorityIssuer; - authority.RequireHttpsMetadata = false; - authority.Audiences.Clear(); - authority.Audiences.Add(TestAuthorityAudience); - authority.ClientId = "webservice-tests"; - authority.ClientSecret = "unused"; - }, - environment); - - using var client = factory.CreateClient(); - var token = CreateTestToken("tenant-auth", StellaOpsScopes.AdvisoryIngest); - client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer", token); - client.DefaultRequestHeaders.Add("X-Stella-Tenant", "tenant-auth"); - - var ingestResponse = await client.PostAsJsonAsync("/ingest/advisory", BuildAdvisoryIngestRequest("sha256:auth-1", "GHSA-AUTH-001")); - Assert.Equal(HttpStatusCode.Created, ingestResponse.StatusCode); - - client.DefaultRequestHeaders.Remove("X-Stella-Tenant"); - client.DefaultRequestHeaders.Add("X-Stella-Tenant", "tenant-other"); - - var crossTenantResponse = await client.PostAsJsonAsync("/ingest/advisory", BuildAdvisoryIngestRequest("sha256:auth-2", "GHSA-AUTH-002")); - Assert.Equal(HttpStatusCode.Forbidden, crossTenantResponse.StatusCode); - } - - [Fact] - public async Task AdvisoryIngestEndpoint_ReturnsGuardViolationWhenContentHashMissing() - { - using var client = _factory.CreateClient(); - client.DefaultRequestHeaders.Add("X-Stella-Tenant", "tenant-violation"); - - var invalidRequest = BuildAdvisoryIngestRequest(contentHash: string.Empty, upstreamId: "GHSA-INVALID-1"); - var response = await client.PostAsJsonAsync("/ingest/advisory", invalidRequest); - - Assert.Equal(HttpStatusCode.UnprocessableEntity, response.StatusCode); - var problemJson = await response.Content.ReadAsStringAsync(); - using var document = JsonDocument.Parse(problemJson); - var root = document.RootElement; - Assert.Equal("Aggregation-Only Contract violation", root.GetProperty("title").GetString()); - Assert.Equal(422, root.GetProperty("status").GetInt32()); - Assert.True(root.TryGetProperty("violations", out var violations), "Problem response missing violations payload."); - Assert.True(root.TryGetProperty("code", out var codeElement), "Problem response missing code payload."); - Assert.Equal("ERR_AOC_004", codeElement.GetString()); - var violation = Assert.Single(violations.EnumerateArray()); - Assert.Equal("ERR_AOC_004", violation.GetProperty("code").GetString()); - } - - [Fact] - public async Task JobsEndpointsReturnExpectedStatuses() - { - using var client = _factory.CreateClient(); - - var definitions = await client.GetAsync("/jobs/definitions"); - if (!definitions.IsSuccessStatusCode) - { - var body = await definitions.Content.ReadAsStringAsync(); - throw new Xunit.Sdk.XunitException($"/jobs/definitions failed: {(int)definitions.StatusCode} {body}"); - } - - var trigger = await client.PostAsync("/jobs/unknown", new StringContent("{}", System.Text.Encoding.UTF8, "application/json")); - if (trigger.StatusCode != HttpStatusCode.NotFound) - { - var payload = await trigger.Content.ReadAsStringAsync(); - throw new Xunit.Sdk.XunitException($"/jobs/unknown expected 404, got {(int)trigger.StatusCode}: {payload}"); - } - var problem = await trigger.Content.ReadFromJsonAsync(); - Assert.NotNull(problem); - Assert.Equal("https://stellaops.org/problems/not-found", problem!.Type); - Assert.Equal(404, problem.Status); - } - - [Fact] - public async Task JobRunEndpointReturnsProblemWhenNotFound() - { - using var client = _factory.CreateClient(); - var response = await client.GetAsync($"/jobs/{Guid.NewGuid()}"); - if (response.StatusCode != HttpStatusCode.NotFound) - { - var body = await response.Content.ReadAsStringAsync(); - throw new Xunit.Sdk.XunitException($"/jobs/{{id}} expected 404, got {(int)response.StatusCode}: {body}"); - } - var problem = await response.Content.ReadFromJsonAsync(); - Assert.NotNull(problem); - Assert.Equal("https://stellaops.org/problems/not-found", problem!.Type); - } - - [Fact] - public async Task JobTriggerMapsCoordinatorOutcomes() - { - var handler = _factory.Services.GetRequiredService(); - using var client = _factory.CreateClient(); - - handler.NextResult = JobTriggerResult.AlreadyRunning("busy"); - var conflict = await client.PostAsync("/jobs/test", JsonContent.Create(new JobTriggerRequest())); - if (conflict.StatusCode != HttpStatusCode.Conflict) - { - var payload = await conflict.Content.ReadAsStringAsync(); - throw new Xunit.Sdk.XunitException($"Conflict path expected 409, got {(int)conflict.StatusCode}: {payload}"); - } - var conflictProblem = await conflict.Content.ReadFromJsonAsync(); - Assert.NotNull(conflictProblem); - Assert.Equal("https://stellaops.org/problems/conflict", conflictProblem!.Type); - - handler.NextResult = JobTriggerResult.Accepted(new JobRunSnapshot(Guid.NewGuid(), "demo", JobRunStatus.Pending, DateTimeOffset.UtcNow, null, null, "api", null, null, null, null, new Dictionary())); - var accepted = await client.PostAsync("/jobs/test", JsonContent.Create(new JobTriggerRequest())); - if (accepted.StatusCode != HttpStatusCode.Accepted) - { - var payload = await accepted.Content.ReadAsStringAsync(); - throw new Xunit.Sdk.XunitException($"Accepted path expected 202, got {(int)accepted.StatusCode}: {payload}"); - } - Assert.NotNull(accepted.Headers.Location); - var acceptedPayload = await accepted.Content.ReadFromJsonAsync(); - Assert.NotNull(acceptedPayload); - - handler.NextResult = JobTriggerResult.Failed(new JobRunSnapshot(Guid.NewGuid(), "demo", JobRunStatus.Failed, DateTimeOffset.UtcNow, null, DateTimeOffset.UtcNow, "api", null, "err", null, null, new Dictionary()), "boom"); - var failed = await client.PostAsync("/jobs/test", JsonContent.Create(new JobTriggerRequest())); - if (failed.StatusCode != HttpStatusCode.InternalServerError) - { - var payload = await failed.Content.ReadAsStringAsync(); - throw new Xunit.Sdk.XunitException($"Failed path expected 500, got {(int)failed.StatusCode}: {payload}"); - } - var failureProblem = await failed.Content.ReadFromJsonAsync(); - Assert.NotNull(failureProblem); - Assert.Equal("https://stellaops.org/problems/job-failure", failureProblem!.Type); - } - - [Fact] - public async Task JobsEndpointsExposeJobData() - { - var handler = _factory.Services.GetRequiredService(); - var now = DateTimeOffset.UtcNow; - var run = new JobRunSnapshot( - Guid.NewGuid(), - "demo", - JobRunStatus.Succeeded, - now, - now, - now.AddSeconds(2), - "api", - "hash", - null, - TimeSpan.FromMinutes(5), - TimeSpan.FromMinutes(1), - new Dictionary { ["key"] = "value" }); - - handler.Definitions = new[] - { - new JobDefinition("demo", typeof(DemoJob), TimeSpan.FromMinutes(5), TimeSpan.FromMinutes(1), "*/5 * * * *", true) - }; - handler.LastRuns["demo"] = run; - handler.RecentRuns = new[] { run }; - handler.ActiveRuns = Array.Empty(); - handler.Runs[run.RunId] = run; - - try - { - using var client = _factory.CreateClient(); - - var definitions = await client.GetFromJsonAsync>("/jobs/definitions"); - Assert.NotNull(definitions); - Assert.Single(definitions!); - Assert.Equal("demo", definitions![0].Kind); - Assert.NotNull(definitions[0].LastRun); - Assert.Equal(run.RunId, definitions[0].LastRun!.RunId); - - var runPayload = await client.GetFromJsonAsync($"/jobs/{run.RunId}"); - Assert.NotNull(runPayload); - Assert.Equal(run.RunId, runPayload!.RunId); - Assert.Equal("Succeeded", runPayload.Status); - - var runs = await client.GetFromJsonAsync>("/jobs?kind=demo&limit=5"); - Assert.NotNull(runs); - Assert.Single(runs!); - Assert.Equal(run.RunId, runs![0].RunId); - - var runsByDefinition = await client.GetFromJsonAsync>("/jobs/definitions/demo/runs"); - Assert.NotNull(runsByDefinition); - Assert.Single(runsByDefinition!); - - var active = await client.GetFromJsonAsync>("/jobs/active"); - Assert.NotNull(active); - Assert.Empty(active!); - } - finally - { - handler.Definitions = Array.Empty(); - handler.RecentRuns = Array.Empty(); - handler.ActiveRuns = Array.Empty(); - handler.Runs.Clear(); - handler.LastRuns.Clear(); - } - } - - [Fact] - public async Task AdvisoryReplayEndpointReturnsLatestStatement() - { - var vulnerabilityKey = "CVE-2025-9000"; - var advisory = new Advisory( - advisoryKey: vulnerabilityKey, - title: "Replay Test", - summary: "Example summary", - language: "en", - published: DateTimeOffset.Parse("2025-01-01T00:00:00Z", CultureInfo.InvariantCulture), - modified: DateTimeOffset.Parse("2025-01-02T00:00:00Z", CultureInfo.InvariantCulture), - severity: "medium", - exploitKnown: false, - aliases: new[] { vulnerabilityKey }, - references: Array.Empty(), - affectedPackages: Array.Empty(), - cvssMetrics: Array.Empty(), - provenance: Array.Empty()); - - var statementId = Guid.NewGuid(); - using (var scope = _factory.Services.CreateScope()) - { - var eventLog = scope.ServiceProvider.GetRequiredService(); - var appendRequest = new AdvisoryEventAppendRequest(new[] - { - new AdvisoryStatementInput( - vulnerabilityKey, - advisory, - advisory.Modified ?? advisory.Published ?? DateTimeOffset.UtcNow, - Array.Empty(), - StatementId: statementId, - AdvisoryKey: advisory.AdvisoryKey) - }); - - await eventLog.AppendAsync(appendRequest, CancellationToken.None); - } - - using var client = _factory.CreateClient(); - var response = await client.GetAsync($"/concelier/advisories/{vulnerabilityKey}/replay"); - - Assert.Equal(HttpStatusCode.OK, response.StatusCode); - var payload = await response.Content.ReadFromJsonAsync(); - Assert.NotNull(payload); - Assert.Equal(vulnerabilityKey, payload!.VulnerabilityKey, ignoreCase: true); - var statement = Assert.Single(payload.Statements); - Assert.Equal(statementId, statement.StatementId); - Assert.Equal(advisory.AdvisoryKey, statement.Advisory.AdvisoryKey); - Assert.False(string.IsNullOrWhiteSpace(statement.StatementHash)); - Assert.True(payload.Conflicts is null || payload.Conflicts!.Count == 0); - } - - [Fact] - public async Task AdvisoryReplayEndpointReturnsConflictExplainer() - { - var vulnerabilityKey = "CVE-2025-9100"; - var statementId = Guid.NewGuid(); - var conflictId = Guid.NewGuid(); - var recordedAt = DateTimeOffset.Parse("2025-02-01T00:00:00Z", CultureInfo.InvariantCulture); - - using (var scope = _factory.Services.CreateScope()) - { - var eventLog = scope.ServiceProvider.GetRequiredService(); - var advisory = new Advisory( - advisoryKey: vulnerabilityKey, - title: "Base advisory", - summary: "Baseline summary", - language: "en", - published: recordedAt.AddDays(-1), - modified: recordedAt, - severity: "critical", - exploitKnown: false, - aliases: new[] { vulnerabilityKey }, - references: Array.Empty(), - affectedPackages: Array.Empty(), - cvssMetrics: Array.Empty(), - provenance: Array.Empty()); - - var statementInput = new AdvisoryStatementInput( - vulnerabilityKey, - advisory, - recordedAt, - Array.Empty(), - StatementId: statementId, - AdvisoryKey: advisory.AdvisoryKey); - - await eventLog.AppendAsync(new AdvisoryEventAppendRequest(new[] { statementInput }), CancellationToken.None); - - var explainer = new MergeConflictExplainerPayload( - Type: "severity", - Reason: "mismatch", - PrimarySources: new[] { "vendor" }, - PrimaryRank: 1, - SuppressedSources: new[] { "nvd" }, - SuppressedRank: 5, - PrimaryValue: "CRITICAL", - SuppressedValue: "MEDIUM"); - - using var conflictDoc = JsonDocument.Parse(explainer.ToCanonicalJson()); - var conflictInput = new AdvisoryConflictInput( - vulnerabilityKey, - conflictDoc, - recordedAt, - new[] { statementId }, - ConflictId: conflictId); - - await eventLog.AppendAsync(new AdvisoryEventAppendRequest(Array.Empty(), new[] { conflictInput }), CancellationToken.None); - } - - using var client = _factory.CreateClient(); - var response = await client.GetAsync($"/concelier/advisories/{vulnerabilityKey}/replay"); - Assert.Equal(HttpStatusCode.OK, response.StatusCode); - var payload = await response.Content.ReadFromJsonAsync(); - Assert.NotNull(payload); - var conflict = Assert.Single(payload!.Conflicts); - Assert.Equal(conflictId, conflict.ConflictId); - Assert.Equal("severity", conflict.Explainer.Type); - Assert.Equal("mismatch", conflict.Explainer.Reason); - Assert.Equal("CRITICAL", conflict.Explainer.PrimaryValue); - Assert.Equal("MEDIUM", conflict.Explainer.SuppressedValue); - Assert.Equal(conflict.Explainer.ComputeHashHex(), conflict.ConflictHash); - } - - [Fact] - public async Task MirrorEndpointsServeConfiguredArtifacts() - { - using var temp = new TempDirectory(); - var exportId = "20251019T120000Z"; - var exportRoot = Path.Combine(temp.Path, exportId); - var mirrorRoot = Path.Combine(exportRoot, "mirror"); - var domainRoot = Path.Combine(mirrorRoot, "primary"); - Directory.CreateDirectory(domainRoot); - - await File.WriteAllTextAsync( - Path.Combine(mirrorRoot, "index.json"), - """{"schemaVersion":1,"domains":[]}"""); - await File.WriteAllTextAsync( - Path.Combine(domainRoot, "manifest.json"), - """{"domainId":"primary"}"""); - await File.WriteAllTextAsync( - Path.Combine(domainRoot, "bundle.json"), - """{"advisories":[]}"""); - await File.WriteAllTextAsync( - Path.Combine(domainRoot, "bundle.json.jws"), - "test-signature"); - - var environment = new Dictionary - { - ["CONCELIER_MIRROR__ENABLED"] = "true", - ["CONCELIER_MIRROR__EXPORTROOT"] = temp.Path, - ["CONCELIER_MIRROR__ACTIVEEXPORTID"] = exportId, - ["CONCELIER_MIRROR__DOMAINS__0__ID"] = "primary", - ["CONCELIER_MIRROR__DOMAINS__0__DISPLAYNAME"] = "Primary", - ["CONCELIER_MIRROR__DOMAINS__0__REQUIREAUTHENTICATION"] = "false", - ["CONCELIER_MIRROR__DOMAINS__0__MAXDOWNLOADREQUESTSPERHOUR"] = "5", - ["CONCELIER_MIRROR__MAXINDEXREQUESTSPERHOUR"] = "5" - }; - - using var factory = new ConcelierApplicationFactory(_runner.ConnectionString, environmentOverrides: environment); - using var client = factory.CreateClient(); - - var indexResponse = await client.GetAsync("/concelier/exports/index.json"); - Assert.Equal(HttpStatusCode.OK, indexResponse.StatusCode); - var indexContent = await indexResponse.Content.ReadAsStringAsync(); - Assert.Contains(@"""schemaVersion"":1", indexContent, StringComparison.Ordinal); - - var manifestResponse = await client.GetAsync("/concelier/exports/mirror/primary/manifest.json"); - Assert.Equal(HttpStatusCode.OK, manifestResponse.StatusCode); - var manifestContent = await manifestResponse.Content.ReadAsStringAsync(); - Assert.Contains(@"""domainId"":""primary""", manifestContent, StringComparison.Ordinal); - - var bundleResponse = await client.GetAsync("/concelier/exports/mirror/primary/bundle.json.jws"); - Assert.Equal(HttpStatusCode.OK, bundleResponse.StatusCode); - var signatureContent = await bundleResponse.Content.ReadAsStringAsync(); - Assert.Equal("test-signature", signatureContent); - } - - [Fact] - public async Task MirrorEndpointsEnforceAuthenticationForProtectedDomains() - { - using var temp = new TempDirectory(); - var exportId = "20251019T120000Z"; - var secureRoot = Path.Combine(temp.Path, exportId, "mirror", "secure"); - Directory.CreateDirectory(secureRoot); - - await File.WriteAllTextAsync( - Path.Combine(temp.Path, exportId, "mirror", "index.json"), - """{"schemaVersion":1,"domains":[]}"""); - await File.WriteAllTextAsync( - Path.Combine(secureRoot, "manifest.json"), - """{"domainId":"secure"}"""); - - var environment = new Dictionary - { - ["CONCELIER_MIRROR__ENABLED"] = "true", - ["CONCELIER_MIRROR__EXPORTROOT"] = temp.Path, - ["CONCELIER_MIRROR__ACTIVEEXPORTID"] = exportId, - ["CONCELIER_MIRROR__DOMAINS__0__ID"] = "secure", - ["CONCELIER_MIRROR__DOMAINS__0__REQUIREAUTHENTICATION"] = "true", - ["CONCELIER_MIRROR__DOMAINS__0__MAXDOWNLOADREQUESTSPERHOUR"] = "5", - ["CONCELIER_AUTHORITY__ENABLED"] = "true", - ["CONCELIER_AUTHORITY__ALLOWANONYMOUSFALLBACK"] = "false", - ["CONCELIER_AUTHORITY__ISSUER"] = "https://authority.example", - ["CONCELIER_AUTHORITY__REQUIREHTTPSMETADATA"] = "false", - ["CONCELIER_AUTHORITY__AUDIENCES__0"] = "api://concelier", - ["CONCELIER_AUTHORITY__REQUIREDSCOPES__0"] = StellaOpsScopes.ConcelierJobsTrigger, - ["CONCELIER_AUTHORITY__CLIENTID"] = "concelier-jobs", - ["CONCELIER_AUTHORITY__CLIENTSECRET"] = "secret", - ["CONCELIER_AUTHORITY__CLIENTSCOPES__0"] = StellaOpsScopes.ConcelierJobsTrigger - }; - - using var factory = new ConcelierApplicationFactory( - _runner.ConnectionString, - authority => - { - authority.Enabled = true; - authority.AllowAnonymousFallback = false; - authority.Issuer = "https://authority.example"; - authority.RequireHttpsMetadata = false; - authority.Audiences.Clear(); - authority.Audiences.Add("api://concelier"); - authority.RequiredScopes.Clear(); - authority.RequiredScopes.Add(StellaOpsScopes.ConcelierJobsTrigger); - authority.ClientId = "concelier-jobs"; - authority.ClientSecret = "secret"; - }, - environment); - - using var client = factory.CreateClient(); - var response = await client.GetAsync("/concelier/exports/mirror/secure/manifest.json"); - Assert.Equal(HttpStatusCode.Unauthorized, response.StatusCode); - var authHeader = Assert.Single(response.Headers.WwwAuthenticate); - Assert.Equal("Bearer", authHeader.Scheme); - } - - [Fact] - public async Task MirrorEndpointsRespectRateLimits() - { - using var temp = new TempDirectory(); - var exportId = "20251019T130000Z"; - var exportRoot = Path.Combine(temp.Path, exportId); - var mirrorRoot = Path.Combine(exportRoot, "mirror"); - Directory.CreateDirectory(mirrorRoot); - - await File.WriteAllTextAsync( - Path.Combine(mirrorRoot, "index.json"), - """{\"schemaVersion\":1,\"domains\":[]}""" - ); - - var environment = new Dictionary - { - ["CONCELIER_MIRROR__ENABLED"] = "true", - ["CONCELIER_MIRROR__EXPORTROOT"] = temp.Path, - ["CONCELIER_MIRROR__ACTIVEEXPORTID"] = exportId, - ["CONCELIER_MIRROR__MAXINDEXREQUESTSPERHOUR"] = "1", - ["CONCELIER_MIRROR__DOMAINS__0__ID"] = "primary", - ["CONCELIER_MIRROR__DOMAINS__0__REQUIREAUTHENTICATION"] = "false", - ["CONCELIER_MIRROR__DOMAINS__0__MAXDOWNLOADREQUESTSPERHOUR"] = "1" - }; - - using var factory = new ConcelierApplicationFactory(_runner.ConnectionString, environmentOverrides: environment); - using var client = factory.CreateClient(); - - var okResponse = await client.GetAsync("/concelier/exports/index.json"); - Assert.Equal(HttpStatusCode.OK, okResponse.StatusCode); - - var limitedResponse = await client.GetAsync("/concelier/exports/index.json"); - Assert.Equal((HttpStatusCode)429, limitedResponse.StatusCode); - Assert.NotNull(limitedResponse.Headers.RetryAfter); + var idSegment = lastSlashIndex >= 0 + ? locationValue[(lastSlashIndex + 1)..] + : locationValue; + var decodedSegment = Uri.UnescapeDataString(idSegment); + Assert.Equal(ingestPayload.Id, decodedSegment); + + var duplicateResponse = await client.PostAsJsonAsync("/ingest/advisory", BuildAdvisoryIngestRequest( + contentHash: "sha256:abc123", + upstreamId: "GHSA-INGEST-0001")); + Assert.Equal(HttpStatusCode.OK, duplicateResponse.StatusCode); + var duplicatePayload = await duplicateResponse.Content.ReadFromJsonAsync(); + Assert.NotNull(duplicatePayload); + Assert.False(duplicatePayload!.Inserted); + + using (var getRequest = new HttpRequestMessage(HttpMethod.Get, $"/advisories/raw/{ingestPayload.Id}")) + { + getRequest.Headers.Add("X-Stella-Tenant", "tenant-ingest"); + var getResponse = await client.SendAsync(getRequest); + getResponse.EnsureSuccessStatusCode(); + + var record = await getResponse.Content.ReadFromJsonAsync(); + Assert.NotNull(record); + Assert.Equal(ingestPayload.Id, record!.Id); + Assert.Equal("tenant-ingest", record.Tenant); + Assert.Equal("sha256:abc123", record.Document.Upstream.ContentHash); + } + + using (var listRequest = new HttpRequestMessage(HttpMethod.Get, "/advisories/raw?limit=10")) + { + listRequest.Headers.Add("X-Stella-Tenant", "tenant-ingest"); + var listResponse = await client.SendAsync(listRequest); + listResponse.EnsureSuccessStatusCode(); + + var listPayload = await listResponse.Content.ReadFromJsonAsync(); + Assert.NotNull(listPayload); + var record = Assert.Single(listPayload!.Records); + Assert.Equal(ingestPayload.Id, record.Id); + } + } + + [Fact] + public async Task AocVerifyEndpoint_ReturnsSummaryForTenant() + { + using var client = _factory.CreateClient(); + client.DefaultRequestHeaders.Add("X-Stella-Tenant", "tenant-verify"); + + await client.PostAsJsonAsync("/ingest/advisory", BuildAdvisoryIngestRequest( + contentHash: "sha256:verify-1", + upstreamId: "GHSA-VERIFY-001")); + + var verifyResponse = await client.PostAsJsonAsync("/aoc/verify", new AocVerifyRequest(null, null, null, null, null)); + verifyResponse.EnsureSuccessStatusCode(); + + var verifyPayload = await verifyResponse.Content.ReadFromJsonAsync(); + Assert.NotNull(verifyPayload); + Assert.Equal("tenant-verify", verifyPayload!.Tenant); + Assert.True(verifyPayload.Checked.Advisories >= 1); + Assert.Equal(0, verifyPayload.Checked.Vex); + Assert.True(verifyPayload.Metrics.IngestionWriteTotal >= verifyPayload.Checked.Advisories); + Assert.Empty(verifyPayload.Violations); + Assert.False(verifyPayload.Truncated); + } + + [Fact] + public async Task AocVerifyEndpoint_ReturnsViolationsForGuardFailures() + { + await SeedAdvisoryRawDocumentsAsync( + CreateAdvisoryRawDocument( + tenant: "tenant-verify-violations", + vendor: "osv", + upstreamId: "GHSA-VERIFY-ERR", + contentHash: string.Empty, + raw: new BsonDocument + { + { "id", "GHSA-VERIFY-ERR" } + })); + + using var client = _factory.CreateClient(); + client.DefaultRequestHeaders.Add("X-Stella-Tenant", "tenant-verify-violations"); + + var verifyResponse = await client.PostAsJsonAsync("/aoc/verify", new AocVerifyRequest(null, null, null, null, null)); + verifyResponse.EnsureSuccessStatusCode(); + + var verifyPayload = await verifyResponse.Content.ReadFromJsonAsync(); + Assert.NotNull(verifyPayload); + Assert.Equal("tenant-verify-violations", verifyPayload!.Tenant); + Assert.True(verifyPayload.Checked.Advisories >= 1); + var violation = Assert.Single(verifyPayload.Violations); + Assert.Equal("ERR_AOC_001", violation.Code); + Assert.True(violation.Count >= 1); + Assert.NotEmpty(violation.Examples); + } + + [Fact] + public async Task AdvisoryRawListEndpoint_SupportsCursorPagination() + { + using var client = _factory.CreateClient(); + client.DefaultRequestHeaders.Add("X-Stella-Tenant", "tenant-list"); + + await client.PostAsJsonAsync("/ingest/advisory", BuildAdvisoryIngestRequest("sha256:list-1", "GHSA-LIST-001")); + await client.PostAsJsonAsync("/ingest/advisory", BuildAdvisoryIngestRequest("sha256:list-2", "GHSA-LIST-002")); + await client.PostAsJsonAsync("/ingest/advisory", BuildAdvisoryIngestRequest("sha256:list-3", "GHSA-LIST-003")); + + using var firstRequest = new HttpRequestMessage(HttpMethod.Get, "/advisories/raw?limit=2"); + firstRequest.Headers.Add("X-Stella-Tenant", "tenant-list"); + var firstResponse = await client.SendAsync(firstRequest); + firstResponse.EnsureSuccessStatusCode(); + + var firstPage = await firstResponse.Content.ReadFromJsonAsync(); + Assert.NotNull(firstPage); + Assert.Equal(2, firstPage!.Records.Count); + Assert.True(firstPage.HasMore); + Assert.False(string.IsNullOrWhiteSpace(firstPage.NextCursor)); + + using var secondRequest = new HttpRequestMessage(HttpMethod.Get, $"/advisories/raw?cursor={Uri.EscapeDataString(firstPage.NextCursor!)}"); + secondRequest.Headers.Add("X-Stella-Tenant", "tenant-list"); + var secondResponse = await client.SendAsync(secondRequest); + secondResponse.EnsureSuccessStatusCode(); + + var secondPage = await secondResponse.Content.ReadFromJsonAsync(); + Assert.NotNull(secondPage); + Assert.Single(secondPage!.Records); + Assert.False(secondPage.HasMore); + Assert.Null(secondPage.NextCursor); + + var firstIds = firstPage.Records.Select(record => record.Id).ToArray(); + var secondIds = secondPage.Records.Select(record => record.Id).ToArray(); + Assert.Empty(firstIds.Intersect(secondIds)); + } + + [Fact] + public async Task AdvisoryIngestEndpoint_EmitsMetricsWithExpectedTags() + { + var measurements = await CaptureMetricsAsync( + IngestionMetrics.MeterName, + "ingestion_write_total", + async () => + { + using var client = _factory.CreateClient(); + client.DefaultRequestHeaders.Add("X-Stella-Tenant", "tenant-metrics"); + + await client.PostAsJsonAsync("/ingest/advisory", BuildAdvisoryIngestRequest("sha256:metric-1", "GHSA-METRIC-001")); + await client.PostAsJsonAsync("/ingest/advisory", BuildAdvisoryIngestRequest("sha256:metric-1", "GHSA-METRIC-001")); + }); + + Assert.Equal(2, measurements.Count); + + var inserted = measurements.FirstOrDefault(measurement => + string.Equals(GetTagValue(measurement, "tenant"), "tenant-metrics", StringComparison.Ordinal) && + string.Equals(GetTagValue(measurement, "result"), "inserted", StringComparison.Ordinal)); + Assert.NotNull(inserted); + Assert.Equal(1, inserted!.Value); + Assert.Equal("osv", GetTagValue(inserted, "source")); + + var duplicate = measurements.FirstOrDefault(measurement => + string.Equals(GetTagValue(measurement, "tenant"), "tenant-metrics", StringComparison.Ordinal) && + string.Equals(GetTagValue(measurement, "result"), "duplicate", StringComparison.Ordinal)); + Assert.NotNull(duplicate); + Assert.Equal(1, duplicate!.Value); + Assert.Equal("osv", GetTagValue(duplicate, "source")); + } + + [Fact] + public async Task AocVerifyEndpoint_EmitsVerificationMetric() + { + var measurements = await CaptureMetricsAsync( + IngestionMetrics.MeterName, + "verify_runs_total", + async () => + { + using var client = _factory.CreateClient(); + client.DefaultRequestHeaders.Add("X-Stella-Tenant", "tenant-verify-metrics"); + + await client.PostAsJsonAsync("/ingest/advisory", BuildAdvisoryIngestRequest("sha256:verify-metric", "GHSA-VERIFY-METRIC")); + var verifyResponse = await client.PostAsJsonAsync("/aoc/verify", new AocVerifyRequest(null, null, null, null, null)); + verifyResponse.EnsureSuccessStatusCode(); + }); + + var measurement = Assert.Single(measurements); + Assert.Equal("tenant-verify-metrics", GetTagValue(measurement, "tenant")); + Assert.Equal("ok", GetTagValue(measurement, "result")); + Assert.Equal(1, measurement.Value); + } + + [Fact] + public async Task AdvisoryIngestEndpoint_RejectsCrossTenantWhenAuthenticated() + { + var environment = new Dictionary + { + ["CONCELIER_AUTHORITY__ENABLED"] = "true", + ["CONCELIER_AUTHORITY__ALLOWANONYMOUSFALLBACK"] = "false", + ["CONCELIER_AUTHORITY__ISSUER"] = TestAuthorityIssuer, + ["CONCELIER_AUTHORITY__REQUIREHTTPSMETADATA"] = "false", + ["CONCELIER_AUTHORITY__AUDIENCES__0"] = TestAuthorityAudience, + ["CONCELIER_AUTHORITY__CLIENTID"] = "webservice-tests", + ["CONCELIER_AUTHORITY__CLIENTSECRET"] = "unused", + }; + + using var factory = new ConcelierApplicationFactory( + _runner.ConnectionString, + authority => + { + authority.Enabled = true; + authority.AllowAnonymousFallback = false; + authority.Issuer = TestAuthorityIssuer; + authority.RequireHttpsMetadata = false; + authority.Audiences.Clear(); + authority.Audiences.Add(TestAuthorityAudience); + authority.ClientId = "webservice-tests"; + authority.ClientSecret = "unused"; + }, + environment); + + using var client = factory.CreateClient(); + var token = CreateTestToken("tenant-auth", StellaOpsScopes.AdvisoryIngest); + client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer", token); + client.DefaultRequestHeaders.Add("X-Stella-Tenant", "tenant-auth"); + + var ingestResponse = await client.PostAsJsonAsync("/ingest/advisory", BuildAdvisoryIngestRequest("sha256:auth-1", "GHSA-AUTH-001")); + Assert.Equal(HttpStatusCode.Created, ingestResponse.StatusCode); + + client.DefaultRequestHeaders.Remove("X-Stella-Tenant"); + client.DefaultRequestHeaders.Add("X-Stella-Tenant", "tenant-other"); + + var crossTenantResponse = await client.PostAsJsonAsync("/ingest/advisory", BuildAdvisoryIngestRequest("sha256:auth-2", "GHSA-AUTH-002")); + Assert.Equal(HttpStatusCode.Forbidden, crossTenantResponse.StatusCode); + } + + [Fact] + public async Task AdvisoryIngestEndpoint_ReturnsGuardViolationWhenContentHashMissing() + { + using var client = _factory.CreateClient(); + client.DefaultRequestHeaders.Add("X-Stella-Tenant", "tenant-violation"); + + var invalidRequest = BuildAdvisoryIngestRequest(contentHash: string.Empty, upstreamId: "GHSA-INVALID-1"); + var response = await client.PostAsJsonAsync("/ingest/advisory", invalidRequest); + + Assert.Equal(HttpStatusCode.UnprocessableEntity, response.StatusCode); + var problemJson = await response.Content.ReadAsStringAsync(); + using var document = JsonDocument.Parse(problemJson); + var root = document.RootElement; + Assert.Equal("Aggregation-Only Contract violation", root.GetProperty("title").GetString()); + Assert.Equal(422, root.GetProperty("status").GetInt32()); + Assert.True(root.TryGetProperty("violations", out var violations), "Problem response missing violations payload."); + Assert.True(root.TryGetProperty("code", out var codeElement), "Problem response missing code payload."); + Assert.Equal("ERR_AOC_004", codeElement.GetString()); + var violation = Assert.Single(violations.EnumerateArray()); + Assert.Equal("ERR_AOC_004", violation.GetProperty("code").GetString()); + } + + [Fact] + public async Task JobsEndpointsReturnExpectedStatuses() + { + using var client = _factory.CreateClient(); + + var definitions = await client.GetAsync("/jobs/definitions"); + if (!definitions.IsSuccessStatusCode) + { + var body = await definitions.Content.ReadAsStringAsync(); + throw new Xunit.Sdk.XunitException($"/jobs/definitions failed: {(int)definitions.StatusCode} {body}"); + } + + var trigger = await client.PostAsync("/jobs/unknown", new StringContent("{}", System.Text.Encoding.UTF8, "application/json")); + if (trigger.StatusCode != HttpStatusCode.NotFound) + { + var payload = await trigger.Content.ReadAsStringAsync(); + throw new Xunit.Sdk.XunitException($"/jobs/unknown expected 404, got {(int)trigger.StatusCode}: {payload}"); + } + var problem = await trigger.Content.ReadFromJsonAsync(); + Assert.NotNull(problem); + Assert.Equal("https://stellaops.org/problems/not-found", problem!.Type); + Assert.Equal(404, problem.Status); + } + + [Fact] + public async Task JobRunEndpointReturnsProblemWhenNotFound() + { + using var client = _factory.CreateClient(); + var response = await client.GetAsync($"/jobs/{Guid.NewGuid()}"); + if (response.StatusCode != HttpStatusCode.NotFound) + { + var body = await response.Content.ReadAsStringAsync(); + throw new Xunit.Sdk.XunitException($"/jobs/{{id}} expected 404, got {(int)response.StatusCode}: {body}"); + } + var problem = await response.Content.ReadFromJsonAsync(); + Assert.NotNull(problem); + Assert.Equal("https://stellaops.org/problems/not-found", problem!.Type); + } + + [Fact] + public async Task JobTriggerMapsCoordinatorOutcomes() + { + var handler = _factory.Services.GetRequiredService(); + using var client = _factory.CreateClient(); + + handler.NextResult = JobTriggerResult.AlreadyRunning("busy"); + var conflict = await client.PostAsync("/jobs/test", JsonContent.Create(new JobTriggerRequest())); + if (conflict.StatusCode != HttpStatusCode.Conflict) + { + var payload = await conflict.Content.ReadAsStringAsync(); + throw new Xunit.Sdk.XunitException($"Conflict path expected 409, got {(int)conflict.StatusCode}: {payload}"); + } + var conflictProblem = await conflict.Content.ReadFromJsonAsync(); + Assert.NotNull(conflictProblem); + Assert.Equal("https://stellaops.org/problems/conflict", conflictProblem!.Type); + + handler.NextResult = JobTriggerResult.Accepted(new JobRunSnapshot(Guid.NewGuid(), "demo", JobRunStatus.Pending, DateTimeOffset.UtcNow, null, null, "api", null, null, null, null, new Dictionary())); + var accepted = await client.PostAsync("/jobs/test", JsonContent.Create(new JobTriggerRequest())); + if (accepted.StatusCode != HttpStatusCode.Accepted) + { + var payload = await accepted.Content.ReadAsStringAsync(); + throw new Xunit.Sdk.XunitException($"Accepted path expected 202, got {(int)accepted.StatusCode}: {payload}"); + } + Assert.NotNull(accepted.Headers.Location); + var acceptedPayload = await accepted.Content.ReadFromJsonAsync(); + Assert.NotNull(acceptedPayload); + + handler.NextResult = JobTriggerResult.Failed(new JobRunSnapshot(Guid.NewGuid(), "demo", JobRunStatus.Failed, DateTimeOffset.UtcNow, null, DateTimeOffset.UtcNow, "api", null, "err", null, null, new Dictionary()), "boom"); + var failed = await client.PostAsync("/jobs/test", JsonContent.Create(new JobTriggerRequest())); + if (failed.StatusCode != HttpStatusCode.InternalServerError) + { + var payload = await failed.Content.ReadAsStringAsync(); + throw new Xunit.Sdk.XunitException($"Failed path expected 500, got {(int)failed.StatusCode}: {payload}"); + } + var failureProblem = await failed.Content.ReadFromJsonAsync(); + Assert.NotNull(failureProblem); + Assert.Equal("https://stellaops.org/problems/job-failure", failureProblem!.Type); + } + + [Fact] + public async Task JobsEndpointsExposeJobData() + { + var handler = _factory.Services.GetRequiredService(); + var now = DateTimeOffset.UtcNow; + var run = new JobRunSnapshot( + Guid.NewGuid(), + "demo", + JobRunStatus.Succeeded, + now, + now, + now.AddSeconds(2), + "api", + "hash", + null, + TimeSpan.FromMinutes(5), + TimeSpan.FromMinutes(1), + new Dictionary { ["key"] = "value" }); + + handler.Definitions = new[] + { + new JobDefinition("demo", typeof(DemoJob), TimeSpan.FromMinutes(5), TimeSpan.FromMinutes(1), "*/5 * * * *", true) + }; + handler.LastRuns["demo"] = run; + handler.RecentRuns = new[] { run }; + handler.ActiveRuns = Array.Empty(); + handler.Runs[run.RunId] = run; + + try + { + using var client = _factory.CreateClient(); + + var definitions = await client.GetFromJsonAsync>("/jobs/definitions"); + Assert.NotNull(definitions); + Assert.Single(definitions!); + Assert.Equal("demo", definitions![0].Kind); + Assert.NotNull(definitions[0].LastRun); + Assert.Equal(run.RunId, definitions[0].LastRun!.RunId); + + var runPayload = await client.GetFromJsonAsync($"/jobs/{run.RunId}"); + Assert.NotNull(runPayload); + Assert.Equal(run.RunId, runPayload!.RunId); + Assert.Equal("Succeeded", runPayload.Status); + + var runs = await client.GetFromJsonAsync>("/jobs?kind=demo&limit=5"); + Assert.NotNull(runs); + Assert.Single(runs!); + Assert.Equal(run.RunId, runs![0].RunId); + + var runsByDefinition = await client.GetFromJsonAsync>("/jobs/definitions/demo/runs"); + Assert.NotNull(runsByDefinition); + Assert.Single(runsByDefinition!); + + var active = await client.GetFromJsonAsync>("/jobs/active"); + Assert.NotNull(active); + Assert.Empty(active!); + } + finally + { + handler.Definitions = Array.Empty(); + handler.RecentRuns = Array.Empty(); + handler.ActiveRuns = Array.Empty(); + handler.Runs.Clear(); + handler.LastRuns.Clear(); + } + } + + [Fact] + public async Task AdvisoryReplayEndpointReturnsLatestStatement() + { + var vulnerabilityKey = "CVE-2025-9000"; + var advisory = new Advisory( + advisoryKey: vulnerabilityKey, + title: "Replay Test", + summary: "Example summary", + language: "en", + published: DateTimeOffset.Parse("2025-01-01T00:00:00Z", CultureInfo.InvariantCulture), + modified: DateTimeOffset.Parse("2025-01-02T00:00:00Z", CultureInfo.InvariantCulture), + severity: "medium", + exploitKnown: false, + aliases: new[] { vulnerabilityKey }, + references: Array.Empty(), + affectedPackages: Array.Empty(), + cvssMetrics: Array.Empty(), + provenance: Array.Empty()); + + var statementId = Guid.NewGuid(); + using (var scope = _factory.Services.CreateScope()) + { + var eventLog = scope.ServiceProvider.GetRequiredService(); + var appendRequest = new AdvisoryEventAppendRequest(new[] + { + new AdvisoryStatementInput( + vulnerabilityKey, + advisory, + advisory.Modified ?? advisory.Published ?? DateTimeOffset.UtcNow, + Array.Empty(), + StatementId: statementId, + AdvisoryKey: advisory.AdvisoryKey) + }); + + await eventLog.AppendAsync(appendRequest, CancellationToken.None); + } + + using var client = _factory.CreateClient(); + var response = await client.GetAsync($"/concelier/advisories/{vulnerabilityKey}/replay"); + + Assert.Equal(HttpStatusCode.OK, response.StatusCode); + var payload = await response.Content.ReadFromJsonAsync(); + Assert.NotNull(payload); + Assert.Equal(vulnerabilityKey, payload!.VulnerabilityKey, ignoreCase: true); + var statement = Assert.Single(payload.Statements); + Assert.Equal(statementId, statement.StatementId); + Assert.Equal(advisory.AdvisoryKey, statement.Advisory.AdvisoryKey); + Assert.False(string.IsNullOrWhiteSpace(statement.StatementHash)); + Assert.True(payload.Conflicts is null || payload.Conflicts!.Count == 0); + } + + [Fact] + public async Task AdvisoryReplayEndpointReturnsConflictExplainer() + { + var vulnerabilityKey = "CVE-2025-9100"; + var statementId = Guid.NewGuid(); + var conflictId = Guid.NewGuid(); + var recordedAt = DateTimeOffset.Parse("2025-02-01T00:00:00Z", CultureInfo.InvariantCulture); + + using (var scope = _factory.Services.CreateScope()) + { + var eventLog = scope.ServiceProvider.GetRequiredService(); + var advisory = new Advisory( + advisoryKey: vulnerabilityKey, + title: "Base advisory", + summary: "Baseline summary", + language: "en", + published: recordedAt.AddDays(-1), + modified: recordedAt, + severity: "critical", + exploitKnown: false, + aliases: new[] { vulnerabilityKey }, + references: Array.Empty(), + affectedPackages: Array.Empty(), + cvssMetrics: Array.Empty(), + provenance: Array.Empty()); + + var statementInput = new AdvisoryStatementInput( + vulnerabilityKey, + advisory, + recordedAt, + Array.Empty(), + StatementId: statementId, + AdvisoryKey: advisory.AdvisoryKey); + + await eventLog.AppendAsync(new AdvisoryEventAppendRequest(new[] { statementInput }), CancellationToken.None); + + var explainer = new MergeConflictExplainerPayload( + Type: "severity", + Reason: "mismatch", + PrimarySources: new[] { "vendor" }, + PrimaryRank: 1, + SuppressedSources: new[] { "nvd" }, + SuppressedRank: 5, + PrimaryValue: "CRITICAL", + SuppressedValue: "MEDIUM"); + + using var conflictDoc = JsonDocument.Parse(explainer.ToCanonicalJson()); + var conflictInput = new AdvisoryConflictInput( + vulnerabilityKey, + conflictDoc, + recordedAt, + new[] { statementId }, + ConflictId: conflictId); + + await eventLog.AppendAsync(new AdvisoryEventAppendRequest(Array.Empty(), new[] { conflictInput }), CancellationToken.None); + } + + using var client = _factory.CreateClient(); + var response = await client.GetAsync($"/concelier/advisories/{vulnerabilityKey}/replay"); + Assert.Equal(HttpStatusCode.OK, response.StatusCode); + var payload = await response.Content.ReadFromJsonAsync(); + Assert.NotNull(payload); + var conflict = Assert.Single(payload!.Conflicts); + Assert.Equal(conflictId, conflict.ConflictId); + Assert.Equal("severity", conflict.Explainer.Type); + Assert.Equal("mismatch", conflict.Explainer.Reason); + Assert.Equal("CRITICAL", conflict.Explainer.PrimaryValue); + Assert.Equal("MEDIUM", conflict.Explainer.SuppressedValue); + Assert.Equal(conflict.Explainer.ComputeHashHex(), conflict.ConflictHash); + } + + [Fact] + public async Task MirrorEndpointsServeConfiguredArtifacts() + { + using var temp = new TempDirectory(); + var exportId = "20251019T120000Z"; + var exportRoot = Path.Combine(temp.Path, exportId); + var mirrorRoot = Path.Combine(exportRoot, "mirror"); + var domainRoot = Path.Combine(mirrorRoot, "primary"); + Directory.CreateDirectory(domainRoot); + + await File.WriteAllTextAsync( + Path.Combine(mirrorRoot, "index.json"), + """{"schemaVersion":1,"domains":[]}"""); + await File.WriteAllTextAsync( + Path.Combine(domainRoot, "manifest.json"), + """{"domainId":"primary"}"""); + await File.WriteAllTextAsync( + Path.Combine(domainRoot, "bundle.json"), + """{"advisories":[]}"""); + await File.WriteAllTextAsync( + Path.Combine(domainRoot, "bundle.json.jws"), + "test-signature"); + + var environment = new Dictionary + { + ["CONCELIER_MIRROR__ENABLED"] = "true", + ["CONCELIER_MIRROR__EXPORTROOT"] = temp.Path, + ["CONCELIER_MIRROR__ACTIVEEXPORTID"] = exportId, + ["CONCELIER_MIRROR__DOMAINS__0__ID"] = "primary", + ["CONCELIER_MIRROR__DOMAINS__0__DISPLAYNAME"] = "Primary", + ["CONCELIER_MIRROR__DOMAINS__0__REQUIREAUTHENTICATION"] = "false", + ["CONCELIER_MIRROR__DOMAINS__0__MAXDOWNLOADREQUESTSPERHOUR"] = "5", + ["CONCELIER_MIRROR__MAXINDEXREQUESTSPERHOUR"] = "5" + }; + + using var factory = new ConcelierApplicationFactory(_runner.ConnectionString, environmentOverrides: environment); + using var client = factory.CreateClient(); + + var indexResponse = await client.GetAsync("/concelier/exports/index.json"); + Assert.Equal(HttpStatusCode.OK, indexResponse.StatusCode); + var indexContent = await indexResponse.Content.ReadAsStringAsync(); + Assert.Contains(@"""schemaVersion"":1", indexContent, StringComparison.Ordinal); + + var manifestResponse = await client.GetAsync("/concelier/exports/mirror/primary/manifest.json"); + Assert.Equal(HttpStatusCode.OK, manifestResponse.StatusCode); + var manifestContent = await manifestResponse.Content.ReadAsStringAsync(); + Assert.Contains(@"""domainId"":""primary""", manifestContent, StringComparison.Ordinal); + + var bundleResponse = await client.GetAsync("/concelier/exports/mirror/primary/bundle.json.jws"); + Assert.Equal(HttpStatusCode.OK, bundleResponse.StatusCode); + var signatureContent = await bundleResponse.Content.ReadAsStringAsync(); + Assert.Equal("test-signature", signatureContent); + } + + [Fact] + public async Task MirrorEndpointsEnforceAuthenticationForProtectedDomains() + { + using var temp = new TempDirectory(); + var exportId = "20251019T120000Z"; + var secureRoot = Path.Combine(temp.Path, exportId, "mirror", "secure"); + Directory.CreateDirectory(secureRoot); + + await File.WriteAllTextAsync( + Path.Combine(temp.Path, exportId, "mirror", "index.json"), + """{"schemaVersion":1,"domains":[]}"""); + await File.WriteAllTextAsync( + Path.Combine(secureRoot, "manifest.json"), + """{"domainId":"secure"}"""); + + var environment = new Dictionary + { + ["CONCELIER_MIRROR__ENABLED"] = "true", + ["CONCELIER_MIRROR__EXPORTROOT"] = temp.Path, + ["CONCELIER_MIRROR__ACTIVEEXPORTID"] = exportId, + ["CONCELIER_MIRROR__DOMAINS__0__ID"] = "secure", + ["CONCELIER_MIRROR__DOMAINS__0__REQUIREAUTHENTICATION"] = "true", + ["CONCELIER_MIRROR__DOMAINS__0__MAXDOWNLOADREQUESTSPERHOUR"] = "5", + ["CONCELIER_AUTHORITY__ENABLED"] = "true", + ["CONCELIER_AUTHORITY__ALLOWANONYMOUSFALLBACK"] = "false", + ["CONCELIER_AUTHORITY__ISSUER"] = "https://authority.example", + ["CONCELIER_AUTHORITY__REQUIREHTTPSMETADATA"] = "false", + ["CONCELIER_AUTHORITY__AUDIENCES__0"] = "api://concelier", + ["CONCELIER_AUTHORITY__REQUIREDSCOPES__0"] = StellaOpsScopes.ConcelierJobsTrigger, + ["CONCELIER_AUTHORITY__CLIENTID"] = "concelier-jobs", + ["CONCELIER_AUTHORITY__CLIENTSECRET"] = "secret", + ["CONCELIER_AUTHORITY__CLIENTSCOPES__0"] = StellaOpsScopes.ConcelierJobsTrigger + }; + + using var factory = new ConcelierApplicationFactory( + _runner.ConnectionString, + authority => + { + authority.Enabled = true; + authority.AllowAnonymousFallback = false; + authority.Issuer = "https://authority.example"; + authority.RequireHttpsMetadata = false; + authority.Audiences.Clear(); + authority.Audiences.Add("api://concelier"); + authority.RequiredScopes.Clear(); + authority.RequiredScopes.Add(StellaOpsScopes.ConcelierJobsTrigger); + authority.ClientId = "concelier-jobs"; + authority.ClientSecret = "secret"; + }, + environment); + + using var client = factory.CreateClient(); + var response = await client.GetAsync("/concelier/exports/mirror/secure/manifest.json"); + Assert.Equal(HttpStatusCode.Unauthorized, response.StatusCode); + var authHeader = Assert.Single(response.Headers.WwwAuthenticate); + Assert.Equal("Bearer", authHeader.Scheme); + } + + [Fact] + public async Task MirrorEndpointsRespectRateLimits() + { + using var temp = new TempDirectory(); + var exportId = "20251019T130000Z"; + var exportRoot = Path.Combine(temp.Path, exportId); + var mirrorRoot = Path.Combine(exportRoot, "mirror"); + Directory.CreateDirectory(mirrorRoot); + + await File.WriteAllTextAsync( + Path.Combine(mirrorRoot, "index.json"), + """{\"schemaVersion\":1,\"domains\":[]}""" + ); + + var environment = new Dictionary + { + ["CONCELIER_MIRROR__ENABLED"] = "true", + ["CONCELIER_MIRROR__EXPORTROOT"] = temp.Path, + ["CONCELIER_MIRROR__ACTIVEEXPORTID"] = exportId, + ["CONCELIER_MIRROR__MAXINDEXREQUESTSPERHOUR"] = "1", + ["CONCELIER_MIRROR__DOMAINS__0__ID"] = "primary", + ["CONCELIER_MIRROR__DOMAINS__0__REQUIREAUTHENTICATION"] = "false", + ["CONCELIER_MIRROR__DOMAINS__0__MAXDOWNLOADREQUESTSPERHOUR"] = "1" + }; + + using var factory = new ConcelierApplicationFactory(_runner.ConnectionString, environmentOverrides: environment); + using var client = factory.CreateClient(); + + var okResponse = await client.GetAsync("/concelier/exports/index.json"); + Assert.Equal(HttpStatusCode.OK, okResponse.StatusCode); + + var limitedResponse = await client.GetAsync("/concelier/exports/index.json"); + Assert.Equal((HttpStatusCode)429, limitedResponse.StatusCode); + Assert.NotNull(limitedResponse.Headers.RetryAfter); Assert.True(limitedResponse.Headers.RetryAfter!.Delta.HasValue); Assert.True(limitedResponse.Headers.RetryAfter!.Delta!.Value.TotalSeconds > 0); } [Fact] - public void MergeModuleDisabledWhenFeatureFlagEnabled() + public void MergeModuleDisabledByDefault() { - var environment = new Dictionary - { - ["CONCELIER_FEATURES__NOMERGEENABLED"] = "true" - }; - using var factory = new ConcelierApplicationFactory( _runner.ConnectionString, authorityConfigure: null, - environmentOverrides: environment); + environmentOverrides: null); using var scope = factory.Services.CreateScope(); var provider = scope.ServiceProvider; @@ -913,11 +909,59 @@ public sealed class WebServiceEndpointsTests : IAsyncLifetime Assert.DoesNotContain("merge:reconcile", schedulerOptions.Definitions.Keys); } + [Fact] + public void MergeModuleReenabledWhenFeatureFlagCleared() + { + var environment = new Dictionary + { + ["CONCELIER_FEATURES__NOMERGEENABLED"] = "false" + }; + + using var factory = new ConcelierApplicationFactory( + _runner.ConnectionString, + authorityConfigure: null, + environmentOverrides: environment); + using var scope = factory.Services.CreateScope(); + var provider = scope.ServiceProvider; + +#pragma warning disable CS0618, CONCELIER0001, CONCELIER0002 // Checking deprecated service registration state. + Assert.NotNull(provider.GetService()); +#pragma warning restore CS0618, CONCELIER0001, CONCELIER0002 + + var schedulerOptions = provider.GetRequiredService>().Value; + Assert.Contains("merge:reconcile", schedulerOptions.Definitions.Keys); + } + + [Fact] + public void MergeJobRemovedWhenAllowlistExcludes() + { + var environment = new Dictionary + { + ["CONCELIER_FEATURES__NOMERGEENABLED"] = "false", + ["CONCELIER_FEATURES__MERGEJOBALLOWLIST__0"] = "export:json" + }; + + using var factory = new ConcelierApplicationFactory( + _runner.ConnectionString, + authorityConfigure: null, + environmentOverrides: environment); + using var scope = factory.Services.CreateScope(); + var provider = scope.ServiceProvider; + +#pragma warning disable CS0618, CONCELIER0001, CONCELIER0002 // Checking deprecated service registration state. + Assert.NotNull(provider.GetService()); +#pragma warning restore CS0618, CONCELIER0001, CONCELIER0002 + + var schedulerOptions = provider.GetRequiredService>().Value; + Assert.DoesNotContain("merge:reconcile", schedulerOptions.Definitions.Keys); + } + [Fact] public void MergeJobRemainsWhenAllowlisted() { var environment = new Dictionary { + ["CONCELIER_FEATURES__NOMERGEENABLED"] = "false", ["CONCELIER_FEATURES__MERGEJOBALLOWLIST__0"] = "merge:reconcile" }; @@ -941,248 +985,248 @@ public sealed class WebServiceEndpointsTests : IAsyncLifetime public async Task JobsEndpointsAllowBypassWhenAuthorityEnabled() { var environment = new Dictionary - { - ["CONCELIER_AUTHORITY__ENABLED"] = "true", - ["CONCELIER_AUTHORITY__ALLOWANONYMOUSFALLBACK"] = "false", - ["CONCELIER_AUTHORITY__ISSUER"] = "https://authority.example", - ["CONCELIER_AUTHORITY__REQUIREHTTPSMETADATA"] = "false", - ["CONCELIER_AUTHORITY__AUDIENCES__0"] = "api://concelier", - ["CONCELIER_AUTHORITY__REQUIREDSCOPES__0"] = StellaOpsScopes.ConcelierJobsTrigger, - ["CONCELIER_AUTHORITY__BYPASSNETWORKS__0"] = "127.0.0.1/32", - ["CONCELIER_AUTHORITY__BYPASSNETWORKS__1"] = "::1/128", - ["CONCELIER_AUTHORITY__CLIENTID"] = "concelier-jobs", - ["CONCELIER_AUTHORITY__CLIENTSECRET"] = "test-secret", - ["CONCELIER_AUTHORITY__CLIENTSCOPES__0"] = StellaOpsScopes.ConcelierJobsTrigger, - }; - - using var factory = new ConcelierApplicationFactory( - _runner.ConnectionString, - authority => - { - authority.Enabled = true; - authority.AllowAnonymousFallback = false; - authority.Issuer = "https://authority.example"; - authority.RequireHttpsMetadata = false; - authority.Audiences.Clear(); - authority.Audiences.Add("api://concelier"); - authority.RequiredScopes.Clear(); - authority.RequiredScopes.Add(StellaOpsScopes.ConcelierJobsTrigger); - authority.BypassNetworks.Clear(); - authority.BypassNetworks.Add("127.0.0.1/32"); - authority.BypassNetworks.Add("::1/128"); - authority.ClientId = "concelier-jobs"; - authority.ClientSecret = "test-secret"; - }, - environment); - - var handler = factory.Services.GetRequiredService(); - handler.Definitions = new[] { new JobDefinition("demo", typeof(DemoJob), TimeSpan.FromMinutes(5), TimeSpan.FromMinutes(1), null, true) }; - - using var client = factory.CreateClient(); - client.DefaultRequestHeaders.Add("X-Test-RemoteAddr", "127.0.0.1"); - var response = await client.GetAsync("/jobs/definitions"); - - Assert.Equal(HttpStatusCode.OK, response.StatusCode); - - var auditLogs = factory.LoggerProvider.Snapshot("Concelier.Authorization.Audit"); - var bypassLog = Assert.Single(auditLogs, entry => entry.TryGetState("Bypass", out var state) && state is bool flag && flag); - Assert.True(bypassLog.TryGetState("RemoteAddress", out var remoteObj) && string.Equals(remoteObj?.ToString(), "127.0.0.1", StringComparison.Ordinal)); - Assert.True(bypassLog.TryGetState("StatusCode", out var statusObj) && Convert.ToInt32(statusObj) == (int)HttpStatusCode.OK); - } - - [Fact] - public async Task JobsEndpointsRequireAuthWhenFallbackDisabled() - { - var enforcementEnvironment = new Dictionary - { - ["CONCELIER_AUTHORITY__ENABLED"] = "true", - ["CONCELIER_AUTHORITY__ALLOWANONYMOUSFALLBACK"] = "false", - ["CONCELIER_AUTHORITY__ISSUER"] = "https://authority.example", - ["CONCELIER_AUTHORITY__REQUIREHTTPSMETADATA"] = "false", - ["CONCELIER_AUTHORITY__AUDIENCES__0"] = "api://concelier", - ["CONCELIER_AUTHORITY__REQUIREDSCOPES__0"] = StellaOpsScopes.ConcelierJobsTrigger, - ["CONCELIER_AUTHORITY__CLIENTID"] = "concelier-jobs", - ["CONCELIER_AUTHORITY__CLIENTSECRET"] = "test-secret", - ["CONCELIER_AUTHORITY__CLIENTSCOPES__0"] = StellaOpsScopes.ConcelierJobsTrigger, - }; - - using var factory = new ConcelierApplicationFactory( - _runner.ConnectionString, - authority => - { - authority.Enabled = true; - authority.AllowAnonymousFallback = false; - authority.Issuer = "https://authority.example"; - authority.RequireHttpsMetadata = false; - authority.Audiences.Clear(); - authority.Audiences.Add("api://concelier"); - authority.RequiredScopes.Clear(); - authority.RequiredScopes.Add(StellaOpsScopes.ConcelierJobsTrigger); - authority.BypassNetworks.Clear(); - authority.ClientId = "concelier-jobs"; - authority.ClientSecret = "test-secret"; - }, - enforcementEnvironment); - - var resolved = factory.Services.GetRequiredService>().Value; - Assert.False(resolved.Authority.AllowAnonymousFallback); - - using var client = factory.CreateClient(); - client.DefaultRequestHeaders.Add("X-Test-RemoteAddr", "127.0.0.1"); - var response = await client.GetAsync("/jobs/definitions"); - - Assert.Equal(HttpStatusCode.Unauthorized, response.StatusCode); - - var auditLogs = factory.LoggerProvider.Snapshot("Concelier.Authorization.Audit"); - var enforcementLog = Assert.Single(auditLogs); - Assert.True(enforcementLog.TryGetState("BypassAllowed", out var bypassAllowedObj) && bypassAllowedObj is bool bypassAllowed && bypassAllowed == false); - Assert.True(enforcementLog.TryGetState("HasPrincipal", out var principalObj) && principalObj is bool hasPrincipal && hasPrincipal == false); - } - - [Fact] - public void AuthorityClientResilienceOptionsAreBound() - { - var environment = new Dictionary - { - ["CONCELIER_AUTHORITY__ENABLED"] = "true", - ["CONCELIER_AUTHORITY__ISSUER"] = "https://authority.example", - ["CONCELIER_AUTHORITY__REQUIREHTTPSMETADATA"] = "false", - ["CONCELIER_AUTHORITY__AUDIENCES__0"] = "api://concelier", - ["CONCELIER_AUTHORITY__REQUIREDSCOPES__0"] = StellaOpsScopes.ConcelierJobsTrigger, - ["CONCELIER_AUTHORITY__CLIENTSCOPES__0"] = StellaOpsScopes.ConcelierJobsTrigger, - ["CONCELIER_AUTHORITY__BACKCHANNELTIMEOUTSECONDS"] = "45", - ["CONCELIER_AUTHORITY__RESILIENCE__ENABLERETRIES"] = "true", - ["CONCELIER_AUTHORITY__RESILIENCE__RETRYDELAYS__0"] = "00:00:02", - ["CONCELIER_AUTHORITY__RESILIENCE__RETRYDELAYS__1"] = "00:00:04", - ["CONCELIER_AUTHORITY__RESILIENCE__ALLOWOFFLINECACHEFALLBACK"] = "false", - ["CONCELIER_AUTHORITY__RESILIENCE__OFFLINECACHETOLERANCE"] = "00:02:30" - }; - - using var factory = new ConcelierApplicationFactory( - _runner.ConnectionString, - authority => - { - authority.Enabled = true; - authority.Issuer = "https://authority.example"; - authority.RequireHttpsMetadata = false; - authority.Audiences.Clear(); - authority.Audiences.Add("api://concelier"); - authority.RequiredScopes.Clear(); - authority.RequiredScopes.Add(StellaOpsScopes.ConcelierJobsTrigger); - authority.ClientScopes.Clear(); - authority.ClientScopes.Add(StellaOpsScopes.ConcelierJobsTrigger); - authority.BackchannelTimeoutSeconds = 45; - }, - environment); - - var monitor = factory.Services.GetRequiredService>(); - var options = monitor.CurrentValue; - - Assert.Equal("https://authority.example", options.Authority); - Assert.Equal(TimeSpan.FromSeconds(45), options.HttpTimeout); - Assert.Equal(new[] { StellaOpsScopes.ConcelierJobsTrigger }, options.NormalizedScopes); - Assert.Equal(new[] { TimeSpan.FromSeconds(2), TimeSpan.FromSeconds(4) }, options.NormalizedRetryDelays); - Assert.False(options.AllowOfflineCacheFallback); - Assert.Equal(TimeSpan.FromSeconds(150), options.OfflineCacheTolerance); - } - - private async Task SeedObservationDocumentsAsync(IEnumerable documents) - { - var client = new MongoClient(_runner.ConnectionString); - var database = client.GetDatabase(MongoStorageDefaults.DefaultDatabaseName); - var collection = database.GetCollection(MongoStorageDefaults.Collections.AdvisoryObservations); - - try - { - await database.DropCollectionAsync(MongoStorageDefaults.Collections.AdvisoryObservations); - } - catch (MongoCommandException ex) when (ex.CodeName == "NamespaceNotFound" || ex.Message.Contains("ns not found", StringComparison.OrdinalIgnoreCase)) - { - // Collection does not exist yet; ignore. - } - - var snapshot = documents?.ToArray() ?? Array.Empty(); - if (snapshot.Length == 0) - { - return; - } - - await collection.InsertManyAsync(snapshot); - } - - private static AdvisoryObservationDocument[] BuildSampleObservationDocuments() - { - return new[] - { - CreateObservationDocument( - id: "tenant-a:nvd:alpha:1", - tenant: "tenant-a", - createdAt: new DateTime(2025, 1, 5, 0, 0, 0, DateTimeKind.Utc), - aliases: new[] { "cve-2025-0001" }, - purls: new[] { "pkg:npm/demo@1.0.0" }, - cpes: new[] { "cpe:/a:vendor:product:1.0" }, - references: new[] { ("advisory", "https://example.test/advisory-1") }), - CreateObservationDocument( - id: "tenant-a:ghsa:beta:1", - tenant: "tenant-a", - createdAt: new DateTime(2025, 1, 6, 0, 0, 0, DateTimeKind.Utc), - aliases: new[] { "ghsa-2025-xyz", "cve-2025-0001" }, - purls: new[] { "pkg:npm/demo@1.1.0" }, - cpes: new[] { "cpe:/a:vendor:product:1.1" }, - references: new[] { ("patch", "https://example.test/patch-1") }), - CreateObservationDocument( - id: "tenant-b:nvd:alpha:1", - tenant: "tenant-b", - createdAt: new DateTime(2025, 1, 7, 0, 0, 0, DateTimeKind.Utc), - aliases: new[] { "cve-2025-0001" }, - purls: new[] { "pkg:npm/demo@2.0.0" }, - cpes: new[] { "cpe:/a:vendor:product:2.0" }, - references: new[] { ("advisory", "https://example.test/advisory-2") }) - }; - } - - private static AdvisoryObservationDocument CreateObservationDocument( - string id, - string tenant, - DateTime createdAt, - IEnumerable? aliases = null, - IEnumerable? purls = null, - IEnumerable? cpes = null, - IEnumerable<(string Type, string Url)>? references = null) - { - return new AdvisoryObservationDocument - { - Id = id, - Tenant = tenant.ToLowerInvariant(), - CreatedAt = createdAt, - Source = new AdvisoryObservationSourceDocument - { - Vendor = "nvd", - Stream = "feed", - Api = "https://example.test/api" - }, - Upstream = new AdvisoryObservationUpstreamDocument - { - UpstreamId = id, - DocumentVersion = null, - FetchedAt = createdAt, - ReceivedAt = createdAt, - ContentHash = $"sha256:{id}", - Signature = new AdvisoryObservationSignatureDocument - { - Present = false - }, - Metadata = new Dictionary(StringComparer.Ordinal) - }, - Content = new AdvisoryObservationContentDocument - { - Format = "csaf", - SpecVersion = "2.0", - Raw = BsonDocument.Parse("""{"observation":"%ID%"}""".Replace("%ID%", id)), - Metadata = new Dictionary(StringComparer.Ordinal) - }, - Linkset = new AdvisoryObservationLinksetDocument - { + { + ["CONCELIER_AUTHORITY__ENABLED"] = "true", + ["CONCELIER_AUTHORITY__ALLOWANONYMOUSFALLBACK"] = "false", + ["CONCELIER_AUTHORITY__ISSUER"] = "https://authority.example", + ["CONCELIER_AUTHORITY__REQUIREHTTPSMETADATA"] = "false", + ["CONCELIER_AUTHORITY__AUDIENCES__0"] = "api://concelier", + ["CONCELIER_AUTHORITY__REQUIREDSCOPES__0"] = StellaOpsScopes.ConcelierJobsTrigger, + ["CONCELIER_AUTHORITY__BYPASSNETWORKS__0"] = "127.0.0.1/32", + ["CONCELIER_AUTHORITY__BYPASSNETWORKS__1"] = "::1/128", + ["CONCELIER_AUTHORITY__CLIENTID"] = "concelier-jobs", + ["CONCELIER_AUTHORITY__CLIENTSECRET"] = "test-secret", + ["CONCELIER_AUTHORITY__CLIENTSCOPES__0"] = StellaOpsScopes.ConcelierJobsTrigger, + }; + + using var factory = new ConcelierApplicationFactory( + _runner.ConnectionString, + authority => + { + authority.Enabled = true; + authority.AllowAnonymousFallback = false; + authority.Issuer = "https://authority.example"; + authority.RequireHttpsMetadata = false; + authority.Audiences.Clear(); + authority.Audiences.Add("api://concelier"); + authority.RequiredScopes.Clear(); + authority.RequiredScopes.Add(StellaOpsScopes.ConcelierJobsTrigger); + authority.BypassNetworks.Clear(); + authority.BypassNetworks.Add("127.0.0.1/32"); + authority.BypassNetworks.Add("::1/128"); + authority.ClientId = "concelier-jobs"; + authority.ClientSecret = "test-secret"; + }, + environment); + + var handler = factory.Services.GetRequiredService(); + handler.Definitions = new[] { new JobDefinition("demo", typeof(DemoJob), TimeSpan.FromMinutes(5), TimeSpan.FromMinutes(1), null, true) }; + + using var client = factory.CreateClient(); + client.DefaultRequestHeaders.Add("X-Test-RemoteAddr", "127.0.0.1"); + var response = await client.GetAsync("/jobs/definitions"); + + Assert.Equal(HttpStatusCode.OK, response.StatusCode); + + var auditLogs = factory.LoggerProvider.Snapshot("Concelier.Authorization.Audit"); + var bypassLog = Assert.Single(auditLogs, entry => entry.TryGetState("Bypass", out var state) && state is bool flag && flag); + Assert.True(bypassLog.TryGetState("RemoteAddress", out var remoteObj) && string.Equals(remoteObj?.ToString(), "127.0.0.1", StringComparison.Ordinal)); + Assert.True(bypassLog.TryGetState("StatusCode", out var statusObj) && Convert.ToInt32(statusObj) == (int)HttpStatusCode.OK); + } + + [Fact] + public async Task JobsEndpointsRequireAuthWhenFallbackDisabled() + { + var enforcementEnvironment = new Dictionary + { + ["CONCELIER_AUTHORITY__ENABLED"] = "true", + ["CONCELIER_AUTHORITY__ALLOWANONYMOUSFALLBACK"] = "false", + ["CONCELIER_AUTHORITY__ISSUER"] = "https://authority.example", + ["CONCELIER_AUTHORITY__REQUIREHTTPSMETADATA"] = "false", + ["CONCELIER_AUTHORITY__AUDIENCES__0"] = "api://concelier", + ["CONCELIER_AUTHORITY__REQUIREDSCOPES__0"] = StellaOpsScopes.ConcelierJobsTrigger, + ["CONCELIER_AUTHORITY__CLIENTID"] = "concelier-jobs", + ["CONCELIER_AUTHORITY__CLIENTSECRET"] = "test-secret", + ["CONCELIER_AUTHORITY__CLIENTSCOPES__0"] = StellaOpsScopes.ConcelierJobsTrigger, + }; + + using var factory = new ConcelierApplicationFactory( + _runner.ConnectionString, + authority => + { + authority.Enabled = true; + authority.AllowAnonymousFallback = false; + authority.Issuer = "https://authority.example"; + authority.RequireHttpsMetadata = false; + authority.Audiences.Clear(); + authority.Audiences.Add("api://concelier"); + authority.RequiredScopes.Clear(); + authority.RequiredScopes.Add(StellaOpsScopes.ConcelierJobsTrigger); + authority.BypassNetworks.Clear(); + authority.ClientId = "concelier-jobs"; + authority.ClientSecret = "test-secret"; + }, + enforcementEnvironment); + + var resolved = factory.Services.GetRequiredService>().Value; + Assert.False(resolved.Authority.AllowAnonymousFallback); + + using var client = factory.CreateClient(); + client.DefaultRequestHeaders.Add("X-Test-RemoteAddr", "127.0.0.1"); + var response = await client.GetAsync("/jobs/definitions"); + + Assert.Equal(HttpStatusCode.Unauthorized, response.StatusCode); + + var auditLogs = factory.LoggerProvider.Snapshot("Concelier.Authorization.Audit"); + var enforcementLog = Assert.Single(auditLogs); + Assert.True(enforcementLog.TryGetState("BypassAllowed", out var bypassAllowedObj) && bypassAllowedObj is bool bypassAllowed && bypassAllowed == false); + Assert.True(enforcementLog.TryGetState("HasPrincipal", out var principalObj) && principalObj is bool hasPrincipal && hasPrincipal == false); + } + + [Fact] + public void AuthorityClientResilienceOptionsAreBound() + { + var environment = new Dictionary + { + ["CONCELIER_AUTHORITY__ENABLED"] = "true", + ["CONCELIER_AUTHORITY__ISSUER"] = "https://authority.example", + ["CONCELIER_AUTHORITY__REQUIREHTTPSMETADATA"] = "false", + ["CONCELIER_AUTHORITY__AUDIENCES__0"] = "api://concelier", + ["CONCELIER_AUTHORITY__REQUIREDSCOPES__0"] = StellaOpsScopes.ConcelierJobsTrigger, + ["CONCELIER_AUTHORITY__CLIENTSCOPES__0"] = StellaOpsScopes.ConcelierJobsTrigger, + ["CONCELIER_AUTHORITY__BACKCHANNELTIMEOUTSECONDS"] = "45", + ["CONCELIER_AUTHORITY__RESILIENCE__ENABLERETRIES"] = "true", + ["CONCELIER_AUTHORITY__RESILIENCE__RETRYDELAYS__0"] = "00:00:02", + ["CONCELIER_AUTHORITY__RESILIENCE__RETRYDELAYS__1"] = "00:00:04", + ["CONCELIER_AUTHORITY__RESILIENCE__ALLOWOFFLINECACHEFALLBACK"] = "false", + ["CONCELIER_AUTHORITY__RESILIENCE__OFFLINECACHETOLERANCE"] = "00:02:30" + }; + + using var factory = new ConcelierApplicationFactory( + _runner.ConnectionString, + authority => + { + authority.Enabled = true; + authority.Issuer = "https://authority.example"; + authority.RequireHttpsMetadata = false; + authority.Audiences.Clear(); + authority.Audiences.Add("api://concelier"); + authority.RequiredScopes.Clear(); + authority.RequiredScopes.Add(StellaOpsScopes.ConcelierJobsTrigger); + authority.ClientScopes.Clear(); + authority.ClientScopes.Add(StellaOpsScopes.ConcelierJobsTrigger); + authority.BackchannelTimeoutSeconds = 45; + }, + environment); + + var monitor = factory.Services.GetRequiredService>(); + var options = monitor.CurrentValue; + + Assert.Equal("https://authority.example", options.Authority); + Assert.Equal(TimeSpan.FromSeconds(45), options.HttpTimeout); + Assert.Equal(new[] { StellaOpsScopes.ConcelierJobsTrigger }, options.NormalizedScopes); + Assert.Equal(new[] { TimeSpan.FromSeconds(2), TimeSpan.FromSeconds(4) }, options.NormalizedRetryDelays); + Assert.False(options.AllowOfflineCacheFallback); + Assert.Equal(TimeSpan.FromSeconds(150), options.OfflineCacheTolerance); + } + + private async Task SeedObservationDocumentsAsync(IEnumerable documents) + { + var client = new MongoClient(_runner.ConnectionString); + var database = client.GetDatabase(MongoStorageDefaults.DefaultDatabaseName); + var collection = database.GetCollection(MongoStorageDefaults.Collections.AdvisoryObservations); + + try + { + await database.DropCollectionAsync(MongoStorageDefaults.Collections.AdvisoryObservations); + } + catch (MongoCommandException ex) when (ex.CodeName == "NamespaceNotFound" || ex.Message.Contains("ns not found", StringComparison.OrdinalIgnoreCase)) + { + // Collection does not exist yet; ignore. + } + + var snapshot = documents?.ToArray() ?? Array.Empty(); + if (snapshot.Length == 0) + { + return; + } + + await collection.InsertManyAsync(snapshot); + } + + private static AdvisoryObservationDocument[] BuildSampleObservationDocuments() + { + return new[] + { + CreateObservationDocument( + id: "tenant-a:nvd:alpha:1", + tenant: "tenant-a", + createdAt: new DateTime(2025, 1, 5, 0, 0, 0, DateTimeKind.Utc), + aliases: new[] { "cve-2025-0001" }, + purls: new[] { "pkg:npm/demo@1.0.0" }, + cpes: new[] { "cpe:/a:vendor:product:1.0" }, + references: new[] { ("advisory", "https://example.test/advisory-1") }), + CreateObservationDocument( + id: "tenant-a:ghsa:beta:1", + tenant: "tenant-a", + createdAt: new DateTime(2025, 1, 6, 0, 0, 0, DateTimeKind.Utc), + aliases: new[] { "ghsa-2025-xyz", "cve-2025-0001" }, + purls: new[] { "pkg:npm/demo@1.1.0" }, + cpes: new[] { "cpe:/a:vendor:product:1.1" }, + references: new[] { ("patch", "https://example.test/patch-1") }), + CreateObservationDocument( + id: "tenant-b:nvd:alpha:1", + tenant: "tenant-b", + createdAt: new DateTime(2025, 1, 7, 0, 0, 0, DateTimeKind.Utc), + aliases: new[] { "cve-2025-0001" }, + purls: new[] { "pkg:npm/demo@2.0.0" }, + cpes: new[] { "cpe:/a:vendor:product:2.0" }, + references: new[] { ("advisory", "https://example.test/advisory-2") }) + }; + } + + private static AdvisoryObservationDocument CreateObservationDocument( + string id, + string tenant, + DateTime createdAt, + IEnumerable? aliases = null, + IEnumerable? purls = null, + IEnumerable? cpes = null, + IEnumerable<(string Type, string Url)>? references = null) + { + return new AdvisoryObservationDocument + { + Id = id, + Tenant = tenant.ToLowerInvariant(), + CreatedAt = createdAt, + Source = new AdvisoryObservationSourceDocument + { + Vendor = "nvd", + Stream = "feed", + Api = "https://example.test/api" + }, + Upstream = new AdvisoryObservationUpstreamDocument + { + UpstreamId = id, + DocumentVersion = null, + FetchedAt = createdAt, + ReceivedAt = createdAt, + ContentHash = $"sha256:{id}", + Signature = new AdvisoryObservationSignatureDocument + { + Present = false + }, + Metadata = new Dictionary(StringComparer.Ordinal) + }, + Content = new AdvisoryObservationContentDocument + { + Format = "csaf", + SpecVersion = "2.0", + Raw = BsonDocument.Parse("""{"observation":"%ID%"}""".Replace("%ID%", id)), + Metadata = new Dictionary(StringComparer.Ordinal) + }, + Linkset = new AdvisoryObservationLinksetDocument + { Aliases = aliases?.Where(value => value is not null).ToList(), Purls = purls?.Where(value => value is not null).ToList(), Cpes = cpes?.Where(value => value is not null).ToList(), @@ -1196,641 +1240,641 @@ public sealed class WebServiceEndpointsTests : IAsyncLifetime }) .ToList() }, - Attributes = new Dictionary(StringComparer.Ordinal) - }; - } - - private sealed record ReplayResponse( - string VulnerabilityKey, - DateTimeOffset? AsOf, - List Statements, - List? Conflicts); - - private sealed record ReplayStatement( - Guid StatementId, - string VulnerabilityKey, - string AdvisoryKey, - Advisory Advisory, - string StatementHash, - DateTimeOffset AsOf, - DateTimeOffset RecordedAt, - IReadOnlyList InputDocumentIds); - - private sealed record ReplayConflict( - Guid ConflictId, - string VulnerabilityKey, - IReadOnlyList StatementIds, - string ConflictHash, - DateTimeOffset AsOf, - DateTimeOffset RecordedAt, - string Details, - MergeConflictExplainerPayload Explainer); - - private sealed class ConcelierApplicationFactory : WebApplicationFactory - { - private readonly string _connectionString; - private readonly string? _previousDsn; - private readonly string? _previousDriver; - private readonly string? _previousTimeout; - private readonly string? _previousTelemetryEnabled; - private readonly string? _previousTelemetryLogging; - private readonly string? _previousTelemetryTracing; - private readonly string? _previousTelemetryMetrics; - private readonly Action? _authorityConfigure; - private readonly IDictionary _additionalPreviousEnvironment = new Dictionary(StringComparer.OrdinalIgnoreCase); - public CollectingLoggerProvider LoggerProvider { get; } = new(); - - public ConcelierApplicationFactory( - string connectionString, - Action? authorityConfigure = null, - IDictionary? environmentOverrides = null) - { - _connectionString = connectionString; - _authorityConfigure = authorityConfigure; - _previousDsn = Environment.GetEnvironmentVariable("CONCELIER_STORAGE__DSN"); - _previousDriver = Environment.GetEnvironmentVariable("CONCELIER_STORAGE__DRIVER"); - _previousTimeout = Environment.GetEnvironmentVariable("CONCELIER_STORAGE__COMMANDTIMEOUTSECONDS"); - _previousTelemetryEnabled = Environment.GetEnvironmentVariable("CONCELIER_TELEMETRY__ENABLED"); - _previousTelemetryLogging = Environment.GetEnvironmentVariable("CONCELIER_TELEMETRY__ENABLELOGGING"); - _previousTelemetryTracing = Environment.GetEnvironmentVariable("CONCELIER_TELEMETRY__ENABLETRACING"); - _previousTelemetryMetrics = Environment.GetEnvironmentVariable("CONCELIER_TELEMETRY__ENABLEMETRICS"); - Environment.SetEnvironmentVariable("CONCELIER_STORAGE__DSN", connectionString); - Environment.SetEnvironmentVariable("CONCELIER_STORAGE__DRIVER", "mongo"); - Environment.SetEnvironmentVariable("CONCELIER_STORAGE__COMMANDTIMEOUTSECONDS", "30"); - Environment.SetEnvironmentVariable("CONCELIER_TELEMETRY__ENABLED", "false"); - Environment.SetEnvironmentVariable("CONCELIER_TELEMETRY__ENABLELOGGING", "false"); - Environment.SetEnvironmentVariable("CONCELIER_TELEMETRY__ENABLETRACING", "false"); - Environment.SetEnvironmentVariable("CONCELIER_TELEMETRY__ENABLEMETRICS", "false"); - if (environmentOverrides is not null) - { - foreach (var kvp in environmentOverrides) - { - var previous = Environment.GetEnvironmentVariable(kvp.Key); - _additionalPreviousEnvironment[kvp.Key] = previous; - Environment.SetEnvironmentVariable(kvp.Key, kvp.Value); - } - } - } - - protected override void ConfigureWebHost(IWebHostBuilder builder) - { - builder.ConfigureAppConfiguration((context, configurationBuilder) => - { - var settings = new Dictionary - { - ["Plugins:Directory"] = Path.Combine(context.HostingEnvironment.ContentRootPath, "StellaOps.Concelier.PluginBinaries"), - }; - - configurationBuilder.AddInMemoryCollection(settings!); - }); - - builder.ConfigureLogging(logging => - { - logging.AddProvider(LoggerProvider); - }); - - builder.ConfigureServices(services => - { - services.AddSingleton(); - services.AddSingleton(sp => sp.GetRequiredService()); - services.PostConfigure(options => - { - options.Storage.Driver = "mongo"; - options.Storage.Dsn = _connectionString; - options.Storage.CommandTimeoutSeconds = 30; - options.Plugins.Directory ??= Path.Combine(AppContext.BaseDirectory, "StellaOps.Concelier.PluginBinaries"); - options.Telemetry.Enabled = false; - options.Telemetry.EnableLogging = false; - options.Telemetry.EnableTracing = false; - options.Telemetry.EnableMetrics = false; - options.Authority ??= new ConcelierOptions.AuthorityOptions(); - _authorityConfigure?.Invoke(options.Authority); - }); - }); - - builder.ConfigureTestServices(services => - { - services.AddSingleton(); - services.PostConfigure(StellaOpsAuthenticationDefaults.AuthenticationScheme, options => - { - options.RequireHttpsMetadata = false; - options.TokenValidationParameters = new TokenValidationParameters - { - ValidateIssuerSigningKey = true, - IssuerSigningKey = TestSigningKey, - ValidateIssuer = false, - ValidateAudience = false, - ValidateLifetime = false, - NameClaimType = ClaimTypes.Name, - RoleClaimType = ClaimTypes.Role, - ClockSkew = TimeSpan.Zero - }; - var issuer = string.IsNullOrWhiteSpace(options.Authority) ? TestAuthorityIssuer : options.Authority; - options.ConfigurationManager = new StaticConfigurationManager(new OpenIdConnectConfiguration - { - Issuer = issuer - }); - }); - }); - } - - protected override void Dispose(bool disposing) - { - base.Dispose(disposing); - Environment.SetEnvironmentVariable("CONCELIER_STORAGE__DSN", _previousDsn); - Environment.SetEnvironmentVariable("CONCELIER_STORAGE__DRIVER", _previousDriver); - Environment.SetEnvironmentVariable("CONCELIER_STORAGE__COMMANDTIMEOUTSECONDS", _previousTimeout); - Environment.SetEnvironmentVariable("CONCELIER_TELEMETRY__ENABLED", _previousTelemetryEnabled); - Environment.SetEnvironmentVariable("CONCELIER_TELEMETRY__ENABLELOGGING", _previousTelemetryLogging); - Environment.SetEnvironmentVariable("CONCELIER_TELEMETRY__ENABLETRACING", _previousTelemetryTracing); - Environment.SetEnvironmentVariable("CONCELIER_TELEMETRY__ENABLEMETRICS", _previousTelemetryMetrics); - foreach (var kvp in _additionalPreviousEnvironment) - { - Environment.SetEnvironmentVariable(kvp.Key, kvp.Value); - } - - LoggerProvider.Dispose(); - } - - private sealed class RemoteIpStartupFilter : IStartupFilter - { - public Action Configure(Action next) - { - return app => - { - app.Use(async (context, nextMiddleware) => - { - if (context.Request.Headers.TryGetValue("X-Test-RemoteAddr", out var values) - && values.Count > 0 - && IPAddress.TryParse(values[0], out var remote)) - { - context.Connection.RemoteIpAddress = remote; - } - - await nextMiddleware(); - }); - - next(app); - }; - } - } - - public sealed record LogEntry( - string LoggerName, - LogLevel Level, - EventId EventId, - string? Message, - Exception? Exception, - IReadOnlyList> State) - { - public bool TryGetState(string name, out object? value) - { - foreach (var kvp in State) - { - if (string.Equals(kvp.Key, name, StringComparison.Ordinal)) - { - value = kvp.Value; - return true; - } - } - - value = null; - return false; - } - } - - public sealed class CollectingLoggerProvider : ILoggerProvider - { - private readonly object syncRoot = new(); - private readonly List entries = new(); - private bool disposed; - - public ILogger CreateLogger(string categoryName) => new CollectingLogger(categoryName, this); - - public IReadOnlyList Snapshot(string loggerName) - { - lock (syncRoot) - { - return entries - .Where(entry => string.Equals(entry.LoggerName, loggerName, StringComparison.Ordinal)) - .ToArray(); - } - } - - public void Dispose() - { - disposed = true; - lock (syncRoot) - { - entries.Clear(); - } - } - - private void Append(LogEntry entry) - { - if (disposed) - { - return; - } - - lock (syncRoot) - { - entries.Add(entry); - } - } - - private sealed class CollectingLogger : ILogger - { - private readonly string categoryName; - private readonly CollectingLoggerProvider provider; - - public CollectingLogger(string categoryName, CollectingLoggerProvider provider) - { - this.categoryName = categoryName; - this.provider = provider; - } - - public IDisposable? BeginScope(TState state) where TState : notnull => NullScope.Instance; - - public bool IsEnabled(LogLevel logLevel) => true; - - public void Log(LogLevel logLevel, EventId eventId, TState state, Exception? exception, Func formatter) - { - if (formatter is null) - { - throw new ArgumentNullException(nameof(formatter)); - } - - var message = formatter(state, exception); - var kvps = ExtractState(state); - var entry = new LogEntry(categoryName, logLevel, eventId, message, exception, kvps); - provider.Append(entry); - } - - private static IReadOnlyList> ExtractState(TState state) - { - if (state is IReadOnlyList> list) - { - return list; - } - - if (state is IEnumerable> enumerable) - { - return enumerable.ToArray(); - } - - if (state is null) - { - return Array.Empty>(); - } - - return new[] { new KeyValuePair("State", state) }; - } - } - - private sealed class NullScope : IDisposable - { - public static readonly NullScope Instance = new(); - public void Dispose() - { - } - } - } - } - - private sealed class TempDirectory : IDisposable - { - public string Path { get; } - - public TempDirectory() - { - Path = System.IO.Path.Combine(System.IO.Path.GetTempPath(), "concelier-mirror-" + Guid.NewGuid().ToString("N", CultureInfo.InvariantCulture)); - Directory.CreateDirectory(Path); - } - - public void Dispose() - { - try - { - if (Directory.Exists(Path)) - { - Directory.Delete(Path, recursive: true); - } - } - catch - { - // best effort cleanup - } - } - } - - private sealed record HealthPayload(string Status, DateTimeOffset StartedAt, double UptimeSeconds, StoragePayload Storage, TelemetryPayload Telemetry); - - private sealed record StoragePayload(string Driver, bool Completed, DateTimeOffset? CompletedAt, double? DurationMs); - - private sealed record TelemetryPayload(bool Enabled, bool Tracing, bool Metrics, bool Logging); - - private sealed record ReadyPayload(string Status, DateTimeOffset StartedAt, double UptimeSeconds, ReadyMongoPayload Mongo); - - private sealed record ReadyMongoPayload(string Status, double? LatencyMs, DateTimeOffset? CheckedAt, string? Error); - - private sealed record JobDefinitionPayload(string Kind, bool Enabled, string? CronExpression, TimeSpan Timeout, TimeSpan LeaseDuration, JobRunPayload? LastRun); - - private sealed record JobRunPayload(Guid RunId, string Kind, string Status, string Trigger, DateTimeOffset CreatedAt, DateTimeOffset? StartedAt, DateTimeOffset? CompletedAt, string? Error, TimeSpan? Duration, Dictionary Parameters); - - private sealed record ProblemDocument(string? Type, string? Title, int? Status, string? Detail, string? Instance); - - private async Task SeedAdvisoryRawDocumentsAsync(params BsonDocument[] documents) - { - var client = new MongoClient(_runner.ConnectionString); - var database = client.GetDatabase(MongoStorageDefaults.DefaultDatabaseName); - var collection = database.GetCollection(MongoStorageDefaults.Collections.AdvisoryRaw); - await collection.DeleteManyAsync(FilterDefinition.Empty); - if (documents.Length > 0) - { - await collection.InsertManyAsync(documents); - } - } - - private static BsonDocument CreateAdvisoryRawDocument( - string tenant, - string vendor, - string upstreamId, - string contentHash, - BsonDocument? raw = null, - string? supersedes = null) - { - var now = DateTime.UtcNow; - return new BsonDocument - { - { "_id", BuildRawDocumentId(vendor, upstreamId, contentHash) }, - { "tenant", tenant }, - { - "source", - new BsonDocument - { - { "vendor", vendor }, - { "connector", "test-connector" }, - { "version", "1.0.0" } - } - }, - { - "upstream", - new BsonDocument - { - { "upstream_id", upstreamId }, - { "document_version", "1" }, - { "retrieved_at", now }, - { "content_hash", contentHash }, - { "signature", new BsonDocument { { "present", false } } }, - { "provenance", new BsonDocument { { "api", "https://example.test" } } } - } - }, - { - "content", - new BsonDocument - { - { "format", "osv" }, - { "raw", raw ?? new BsonDocument("id", upstreamId) } - } - }, - { - "identifiers", - new BsonDocument - { - { "aliases", new BsonArray(new[] { upstreamId }) }, - { "primary", upstreamId } - } - }, - { - "linkset", - new BsonDocument - { - { "aliases", new BsonArray() }, - { "purls", new BsonArray() }, - { "cpes", new BsonArray() }, - { "references", new BsonArray() }, - { "reconciled_from", new BsonArray() }, - { "notes", new BsonDocument() } - } - }, - { "supersedes", supersedes is null ? BsonNull.Value : supersedes }, - { "ingested_at", now }, - { "created_at", now } - }; - } - - private static string BuildRawDocumentId(string vendor, string upstreamId, string contentHash) - { - static string Sanitize(string value) - { - if (string.IsNullOrWhiteSpace(value)) - { - return "unknown"; - } - - var buffer = new char[value.Length]; - var index = 0; - foreach (var ch in value.Trim().ToLowerInvariant()) - { - buffer[index++] = char.IsLetterOrDigit(ch) ? ch : '-'; - } - - var sanitized = new string(buffer, 0, index).Trim('-'); - return string.IsNullOrEmpty(sanitized) ? "unknown" : sanitized; - } - - var vendorSegment = Sanitize(vendor); - var upstreamSegment = Sanitize(upstreamId); - var hashSegment = Sanitize(contentHash.Replace(":", "-")); - return $"advisory_raw:{vendorSegment}:{upstreamSegment}:{hashSegment}"; - } - - private static AdvisoryIngestRequest BuildAdvisoryIngestRequest(string contentHash, string upstreamId) - { - var raw = CreateJsonElement($@"{{""id"":""{upstreamId}"",""modified"":""{DateTime.UtcNow:O}""}}"); - var references = new[] - { - new AdvisoryLinksetReferenceRequest("advisory", $"https://example.test/advisories/{upstreamId}", null) - }; - - return new AdvisoryIngestRequest( - new AdvisorySourceRequest("osv", "osv-connector", "1.0.0", "feed"), - new AdvisoryUpstreamRequest( - upstreamId, - "2025-01-01T00:00:00Z", - DateTimeOffset.UtcNow, - contentHash, - new AdvisorySignatureRequest(false, null, null, null, null, null), - new Dictionary { ["http.method"] = "GET" }), - new AdvisoryContentRequest("osv", "1.3.0", raw, null), - new AdvisoryIdentifiersRequest( - upstreamId, - new[] { upstreamId, $"{upstreamId}-ALIAS" }), - new AdvisoryLinksetRequest( - new[] { upstreamId }, - new[] { "pkg:npm/demo@1.0.0" }, - Array.Empty(), - references, - Array.Empty(), - new Dictionary { ["note"] = "ingest-test" })); - } - - private static JsonElement CreateJsonElement(string json) - { - using var document = JsonDocument.Parse(json); - return document.RootElement.Clone(); - } - - private static async Task> CaptureMetricsAsync(string meterName, string instrumentName, Func action) - { - var measurements = new List(); - var listener = new MeterListener(); - - listener.InstrumentPublished += (instrument, currentListener) => - { - if (string.Equals(instrument.Meter.Name, meterName, StringComparison.Ordinal) && - string.Equals(instrument.Name, instrumentName, StringComparison.Ordinal)) - { - currentListener.EnableMeasurementEvents(instrument); - } - }; - - listener.SetMeasurementEventCallback((instrument, measurement, tags, state) => - { - var tagDictionary = new Dictionary(StringComparer.Ordinal); - foreach (var tag in tags) - { - tagDictionary[tag.Key] = tag.Value; - } - - measurements.Add(new MetricMeasurement(instrument.Name, measurement, tagDictionary)); - }); - - listener.Start(); - try - { - await action().ConfigureAwait(false); - } - finally - { - listener.Dispose(); - } - - return measurements; - } - - private static string? GetTagValue(MetricMeasurement measurement, string tag) - { - if (measurement.Tags.TryGetValue(tag, out var value)) - { - return value?.ToString(); - } - - return null; - } - - private static string CreateTestToken(string tenant, params string[] scopes) - { - var normalizedTenant = string.IsNullOrWhiteSpace(tenant) ? "default" : tenant.Trim().ToLowerInvariant(); - var scopeSet = scopes is { Length: > 0 } - ? scopes - .Select(StellaOpsScopes.Normalize) - .Where(static scope => !string.IsNullOrEmpty(scope)) - .Select(static scope => scope!) - .Distinct(StringComparer.Ordinal) - .ToArray() - : Array.Empty(); - - var claims = new List - { - new Claim(StellaOpsClaimTypes.Subject, "test-user"), - new Claim(StellaOpsClaimTypes.Tenant, normalizedTenant), - new Claim(StellaOpsClaimTypes.Scope, string.Join(' ', scopeSet)) - }; - - foreach (var scope in scopeSet) - { - claims.Add(new Claim(StellaOpsClaimTypes.ScopeItem, scope)); - } - - var credentials = new SigningCredentials(TestSigningKey, SecurityAlgorithms.HmacSha256); - var now = DateTime.UtcNow; - var token = new JwtSecurityToken( - issuer: TestAuthorityIssuer, - audience: TestAuthorityAudience, - claims: claims, - notBefore: now.AddMinutes(-5), - expires: now.AddMinutes(30), - signingCredentials: credentials); - - return new JwtSecurityTokenHandler().WriteToken(token); - } - - private sealed record MetricMeasurement(string Instrument, long Value, IReadOnlyDictionary Tags); - - private sealed class DemoJob : IJob - { - public Task ExecuteAsync(JobExecutionContext context, CancellationToken cancellationToken) => Task.CompletedTask; - } - - private sealed class StubJobCoordinator : IJobCoordinator - { - public JobTriggerResult NextResult { get; set; } = JobTriggerResult.NotFound("not set"); - - public IReadOnlyList Definitions { get; set; } = Array.Empty(); - - public IReadOnlyList RecentRuns { get; set; } = Array.Empty(); - - public IReadOnlyList ActiveRuns { get; set; } = Array.Empty(); - - public Dictionary Runs { get; } = new(); - - public Dictionary LastRuns { get; } = new(StringComparer.Ordinal); - - public Task TriggerAsync(string kind, IReadOnlyDictionary? parameters, string trigger, CancellationToken cancellationToken) - => Task.FromResult(NextResult); - - public Task> GetDefinitionsAsync(CancellationToken cancellationToken) - => Task.FromResult(Definitions); - - public Task> GetRecentRunsAsync(string? kind, int limit, CancellationToken cancellationToken) - { - IEnumerable query = RecentRuns; - if (!string.IsNullOrWhiteSpace(kind)) - { - query = query.Where(run => string.Equals(run.Kind, kind, StringComparison.Ordinal)); - } - - return Task.FromResult>(query.Take(limit).ToArray()); - } - - public Task> GetActiveRunsAsync(CancellationToken cancellationToken) - => Task.FromResult(ActiveRuns); - - public Task GetRunAsync(Guid runId, CancellationToken cancellationToken) - => Task.FromResult(Runs.TryGetValue(runId, out var run) ? run : null); - - public Task GetLastRunAsync(string kind, CancellationToken cancellationToken) - => Task.FromResult(LastRuns.TryGetValue(kind, out var run) ? run : null); - - public Task> GetLastRunsAsync(IEnumerable kinds, CancellationToken cancellationToken) - { - var map = new Dictionary(StringComparer.Ordinal); - foreach (var kind in kinds) - { - if (kind is null) - { - continue; - } - - if (LastRuns.TryGetValue(kind, out var run) && run is not null) - { - map[kind] = run; - } - } - - return Task.FromResult>(map); - } - } -} + Attributes = new Dictionary(StringComparer.Ordinal) + }; + } + + private sealed record ReplayResponse( + string VulnerabilityKey, + DateTimeOffset? AsOf, + List Statements, + List? Conflicts); + + private sealed record ReplayStatement( + Guid StatementId, + string VulnerabilityKey, + string AdvisoryKey, + Advisory Advisory, + string StatementHash, + DateTimeOffset AsOf, + DateTimeOffset RecordedAt, + IReadOnlyList InputDocumentIds); + + private sealed record ReplayConflict( + Guid ConflictId, + string VulnerabilityKey, + IReadOnlyList StatementIds, + string ConflictHash, + DateTimeOffset AsOf, + DateTimeOffset RecordedAt, + string Details, + MergeConflictExplainerPayload Explainer); + + private sealed class ConcelierApplicationFactory : WebApplicationFactory + { + private readonly string _connectionString; + private readonly string? _previousDsn; + private readonly string? _previousDriver; + private readonly string? _previousTimeout; + private readonly string? _previousTelemetryEnabled; + private readonly string? _previousTelemetryLogging; + private readonly string? _previousTelemetryTracing; + private readonly string? _previousTelemetryMetrics; + private readonly Action? _authorityConfigure; + private readonly IDictionary _additionalPreviousEnvironment = new Dictionary(StringComparer.OrdinalIgnoreCase); + public CollectingLoggerProvider LoggerProvider { get; } = new(); + + public ConcelierApplicationFactory( + string connectionString, + Action? authorityConfigure = null, + IDictionary? environmentOverrides = null) + { + _connectionString = connectionString; + _authorityConfigure = authorityConfigure; + _previousDsn = Environment.GetEnvironmentVariable("CONCELIER_STORAGE__DSN"); + _previousDriver = Environment.GetEnvironmentVariable("CONCELIER_STORAGE__DRIVER"); + _previousTimeout = Environment.GetEnvironmentVariable("CONCELIER_STORAGE__COMMANDTIMEOUTSECONDS"); + _previousTelemetryEnabled = Environment.GetEnvironmentVariable("CONCELIER_TELEMETRY__ENABLED"); + _previousTelemetryLogging = Environment.GetEnvironmentVariable("CONCELIER_TELEMETRY__ENABLELOGGING"); + _previousTelemetryTracing = Environment.GetEnvironmentVariable("CONCELIER_TELEMETRY__ENABLETRACING"); + _previousTelemetryMetrics = Environment.GetEnvironmentVariable("CONCELIER_TELEMETRY__ENABLEMETRICS"); + Environment.SetEnvironmentVariable("CONCELIER_STORAGE__DSN", connectionString); + Environment.SetEnvironmentVariable("CONCELIER_STORAGE__DRIVER", "mongo"); + Environment.SetEnvironmentVariable("CONCELIER_STORAGE__COMMANDTIMEOUTSECONDS", "30"); + Environment.SetEnvironmentVariable("CONCELIER_TELEMETRY__ENABLED", "false"); + Environment.SetEnvironmentVariable("CONCELIER_TELEMETRY__ENABLELOGGING", "false"); + Environment.SetEnvironmentVariable("CONCELIER_TELEMETRY__ENABLETRACING", "false"); + Environment.SetEnvironmentVariable("CONCELIER_TELEMETRY__ENABLEMETRICS", "false"); + if (environmentOverrides is not null) + { + foreach (var kvp in environmentOverrides) + { + var previous = Environment.GetEnvironmentVariable(kvp.Key); + _additionalPreviousEnvironment[kvp.Key] = previous; + Environment.SetEnvironmentVariable(kvp.Key, kvp.Value); + } + } + } + + protected override void ConfigureWebHost(IWebHostBuilder builder) + { + builder.ConfigureAppConfiguration((context, configurationBuilder) => + { + var settings = new Dictionary + { + ["Plugins:Directory"] = Path.Combine(context.HostingEnvironment.ContentRootPath, "StellaOps.Concelier.PluginBinaries"), + }; + + configurationBuilder.AddInMemoryCollection(settings!); + }); + + builder.ConfigureLogging(logging => + { + logging.AddProvider(LoggerProvider); + }); + + builder.ConfigureServices(services => + { + services.AddSingleton(); + services.AddSingleton(sp => sp.GetRequiredService()); + services.PostConfigure(options => + { + options.Storage.Driver = "mongo"; + options.Storage.Dsn = _connectionString; + options.Storage.CommandTimeoutSeconds = 30; + options.Plugins.Directory ??= Path.Combine(AppContext.BaseDirectory, "StellaOps.Concelier.PluginBinaries"); + options.Telemetry.Enabled = false; + options.Telemetry.EnableLogging = false; + options.Telemetry.EnableTracing = false; + options.Telemetry.EnableMetrics = false; + options.Authority ??= new ConcelierOptions.AuthorityOptions(); + _authorityConfigure?.Invoke(options.Authority); + }); + }); + + builder.ConfigureTestServices(services => + { + services.AddSingleton(); + services.PostConfigure(StellaOpsAuthenticationDefaults.AuthenticationScheme, options => + { + options.RequireHttpsMetadata = false; + options.TokenValidationParameters = new TokenValidationParameters + { + ValidateIssuerSigningKey = true, + IssuerSigningKey = TestSigningKey, + ValidateIssuer = false, + ValidateAudience = false, + ValidateLifetime = false, + NameClaimType = ClaimTypes.Name, + RoleClaimType = ClaimTypes.Role, + ClockSkew = TimeSpan.Zero + }; + var issuer = string.IsNullOrWhiteSpace(options.Authority) ? TestAuthorityIssuer : options.Authority; + options.ConfigurationManager = new StaticConfigurationManager(new OpenIdConnectConfiguration + { + Issuer = issuer + }); + }); + }); + } + + protected override void Dispose(bool disposing) + { + base.Dispose(disposing); + Environment.SetEnvironmentVariable("CONCELIER_STORAGE__DSN", _previousDsn); + Environment.SetEnvironmentVariable("CONCELIER_STORAGE__DRIVER", _previousDriver); + Environment.SetEnvironmentVariable("CONCELIER_STORAGE__COMMANDTIMEOUTSECONDS", _previousTimeout); + Environment.SetEnvironmentVariable("CONCELIER_TELEMETRY__ENABLED", _previousTelemetryEnabled); + Environment.SetEnvironmentVariable("CONCELIER_TELEMETRY__ENABLELOGGING", _previousTelemetryLogging); + Environment.SetEnvironmentVariable("CONCELIER_TELEMETRY__ENABLETRACING", _previousTelemetryTracing); + Environment.SetEnvironmentVariable("CONCELIER_TELEMETRY__ENABLEMETRICS", _previousTelemetryMetrics); + foreach (var kvp in _additionalPreviousEnvironment) + { + Environment.SetEnvironmentVariable(kvp.Key, kvp.Value); + } + + LoggerProvider.Dispose(); + } + + private sealed class RemoteIpStartupFilter : IStartupFilter + { + public Action Configure(Action next) + { + return app => + { + app.Use(async (context, nextMiddleware) => + { + if (context.Request.Headers.TryGetValue("X-Test-RemoteAddr", out var values) + && values.Count > 0 + && IPAddress.TryParse(values[0], out var remote)) + { + context.Connection.RemoteIpAddress = remote; + } + + await nextMiddleware(); + }); + + next(app); + }; + } + } + + public sealed record LogEntry( + string LoggerName, + LogLevel Level, + EventId EventId, + string? Message, + Exception? Exception, + IReadOnlyList> State) + { + public bool TryGetState(string name, out object? value) + { + foreach (var kvp in State) + { + if (string.Equals(kvp.Key, name, StringComparison.Ordinal)) + { + value = kvp.Value; + return true; + } + } + + value = null; + return false; + } + } + + public sealed class CollectingLoggerProvider : ILoggerProvider + { + private readonly object syncRoot = new(); + private readonly List entries = new(); + private bool disposed; + + public ILogger CreateLogger(string categoryName) => new CollectingLogger(categoryName, this); + + public IReadOnlyList Snapshot(string loggerName) + { + lock (syncRoot) + { + return entries + .Where(entry => string.Equals(entry.LoggerName, loggerName, StringComparison.Ordinal)) + .ToArray(); + } + } + + public void Dispose() + { + disposed = true; + lock (syncRoot) + { + entries.Clear(); + } + } + + private void Append(LogEntry entry) + { + if (disposed) + { + return; + } + + lock (syncRoot) + { + entries.Add(entry); + } + } + + private sealed class CollectingLogger : ILogger + { + private readonly string categoryName; + private readonly CollectingLoggerProvider provider; + + public CollectingLogger(string categoryName, CollectingLoggerProvider provider) + { + this.categoryName = categoryName; + this.provider = provider; + } + + public IDisposable? BeginScope(TState state) where TState : notnull => NullScope.Instance; + + public bool IsEnabled(LogLevel logLevel) => true; + + public void Log(LogLevel logLevel, EventId eventId, TState state, Exception? exception, Func formatter) + { + if (formatter is null) + { + throw new ArgumentNullException(nameof(formatter)); + } + + var message = formatter(state, exception); + var kvps = ExtractState(state); + var entry = new LogEntry(categoryName, logLevel, eventId, message, exception, kvps); + provider.Append(entry); + } + + private static IReadOnlyList> ExtractState(TState state) + { + if (state is IReadOnlyList> list) + { + return list; + } + + if (state is IEnumerable> enumerable) + { + return enumerable.ToArray(); + } + + if (state is null) + { + return Array.Empty>(); + } + + return new[] { new KeyValuePair("State", state) }; + } + } + + private sealed class NullScope : IDisposable + { + public static readonly NullScope Instance = new(); + public void Dispose() + { + } + } + } + } + + private sealed class TempDirectory : IDisposable + { + public string Path { get; } + + public TempDirectory() + { + Path = System.IO.Path.Combine(System.IO.Path.GetTempPath(), "concelier-mirror-" + Guid.NewGuid().ToString("N", CultureInfo.InvariantCulture)); + Directory.CreateDirectory(Path); + } + + public void Dispose() + { + try + { + if (Directory.Exists(Path)) + { + Directory.Delete(Path, recursive: true); + } + } + catch + { + // best effort cleanup + } + } + } + + private sealed record HealthPayload(string Status, DateTimeOffset StartedAt, double UptimeSeconds, StoragePayload Storage, TelemetryPayload Telemetry); + + private sealed record StoragePayload(string Driver, bool Completed, DateTimeOffset? CompletedAt, double? DurationMs); + + private sealed record TelemetryPayload(bool Enabled, bool Tracing, bool Metrics, bool Logging); + + private sealed record ReadyPayload(string Status, DateTimeOffset StartedAt, double UptimeSeconds, ReadyMongoPayload Mongo); + + private sealed record ReadyMongoPayload(string Status, double? LatencyMs, DateTimeOffset? CheckedAt, string? Error); + + private sealed record JobDefinitionPayload(string Kind, bool Enabled, string? CronExpression, TimeSpan Timeout, TimeSpan LeaseDuration, JobRunPayload? LastRun); + + private sealed record JobRunPayload(Guid RunId, string Kind, string Status, string Trigger, DateTimeOffset CreatedAt, DateTimeOffset? StartedAt, DateTimeOffset? CompletedAt, string? Error, TimeSpan? Duration, Dictionary Parameters); + + private sealed record ProblemDocument(string? Type, string? Title, int? Status, string? Detail, string? Instance); + + private async Task SeedAdvisoryRawDocumentsAsync(params BsonDocument[] documents) + { + var client = new MongoClient(_runner.ConnectionString); + var database = client.GetDatabase(MongoStorageDefaults.DefaultDatabaseName); + var collection = database.GetCollection(MongoStorageDefaults.Collections.AdvisoryRaw); + await collection.DeleteManyAsync(FilterDefinition.Empty); + if (documents.Length > 0) + { + await collection.InsertManyAsync(documents); + } + } + + private static BsonDocument CreateAdvisoryRawDocument( + string tenant, + string vendor, + string upstreamId, + string contentHash, + BsonDocument? raw = null, + string? supersedes = null) + { + var now = DateTime.UtcNow; + return new BsonDocument + { + { "_id", BuildRawDocumentId(vendor, upstreamId, contentHash) }, + { "tenant", tenant }, + { + "source", + new BsonDocument + { + { "vendor", vendor }, + { "connector", "test-connector" }, + { "version", "1.0.0" } + } + }, + { + "upstream", + new BsonDocument + { + { "upstream_id", upstreamId }, + { "document_version", "1" }, + { "retrieved_at", now }, + { "content_hash", contentHash }, + { "signature", new BsonDocument { { "present", false } } }, + { "provenance", new BsonDocument { { "api", "https://example.test" } } } + } + }, + { + "content", + new BsonDocument + { + { "format", "osv" }, + { "raw", raw ?? new BsonDocument("id", upstreamId) } + } + }, + { + "identifiers", + new BsonDocument + { + { "aliases", new BsonArray(new[] { upstreamId }) }, + { "primary", upstreamId } + } + }, + { + "linkset", + new BsonDocument + { + { "aliases", new BsonArray() }, + { "purls", new BsonArray() }, + { "cpes", new BsonArray() }, + { "references", new BsonArray() }, + { "reconciled_from", new BsonArray() }, + { "notes", new BsonDocument() } + } + }, + { "supersedes", supersedes is null ? BsonNull.Value : supersedes }, + { "ingested_at", now }, + { "created_at", now } + }; + } + + private static string BuildRawDocumentId(string vendor, string upstreamId, string contentHash) + { + static string Sanitize(string value) + { + if (string.IsNullOrWhiteSpace(value)) + { + return "unknown"; + } + + var buffer = new char[value.Length]; + var index = 0; + foreach (var ch in value.Trim().ToLowerInvariant()) + { + buffer[index++] = char.IsLetterOrDigit(ch) ? ch : '-'; + } + + var sanitized = new string(buffer, 0, index).Trim('-'); + return string.IsNullOrEmpty(sanitized) ? "unknown" : sanitized; + } + + var vendorSegment = Sanitize(vendor); + var upstreamSegment = Sanitize(upstreamId); + var hashSegment = Sanitize(contentHash.Replace(":", "-")); + return $"advisory_raw:{vendorSegment}:{upstreamSegment}:{hashSegment}"; + } + + private static AdvisoryIngestRequest BuildAdvisoryIngestRequest(string contentHash, string upstreamId) + { + var raw = CreateJsonElement($@"{{""id"":""{upstreamId}"",""modified"":""{DateTime.UtcNow:O}""}}"); + var references = new[] + { + new AdvisoryLinksetReferenceRequest("advisory", $"https://example.test/advisories/{upstreamId}", null) + }; + + return new AdvisoryIngestRequest( + new AdvisorySourceRequest("osv", "osv-connector", "1.0.0", "feed"), + new AdvisoryUpstreamRequest( + upstreamId, + "2025-01-01T00:00:00Z", + DateTimeOffset.UtcNow, + contentHash, + new AdvisorySignatureRequest(false, null, null, null, null, null), + new Dictionary { ["http.method"] = "GET" }), + new AdvisoryContentRequest("osv", "1.3.0", raw, null), + new AdvisoryIdentifiersRequest( + upstreamId, + new[] { upstreamId, $"{upstreamId}-ALIAS" }), + new AdvisoryLinksetRequest( + new[] { upstreamId }, + new[] { "pkg:npm/demo@1.0.0" }, + Array.Empty(), + references, + Array.Empty(), + new Dictionary { ["note"] = "ingest-test" })); + } + + private static JsonElement CreateJsonElement(string json) + { + using var document = JsonDocument.Parse(json); + return document.RootElement.Clone(); + } + + private static async Task> CaptureMetricsAsync(string meterName, string instrumentName, Func action) + { + var measurements = new List(); + var listener = new MeterListener(); + + listener.InstrumentPublished += (instrument, currentListener) => + { + if (string.Equals(instrument.Meter.Name, meterName, StringComparison.Ordinal) && + string.Equals(instrument.Name, instrumentName, StringComparison.Ordinal)) + { + currentListener.EnableMeasurementEvents(instrument); + } + }; + + listener.SetMeasurementEventCallback((instrument, measurement, tags, state) => + { + var tagDictionary = new Dictionary(StringComparer.Ordinal); + foreach (var tag in tags) + { + tagDictionary[tag.Key] = tag.Value; + } + + measurements.Add(new MetricMeasurement(instrument.Name, measurement, tagDictionary)); + }); + + listener.Start(); + try + { + await action().ConfigureAwait(false); + } + finally + { + listener.Dispose(); + } + + return measurements; + } + + private static string? GetTagValue(MetricMeasurement measurement, string tag) + { + if (measurement.Tags.TryGetValue(tag, out var value)) + { + return value?.ToString(); + } + + return null; + } + + private static string CreateTestToken(string tenant, params string[] scopes) + { + var normalizedTenant = string.IsNullOrWhiteSpace(tenant) ? "default" : tenant.Trim().ToLowerInvariant(); + var scopeSet = scopes is { Length: > 0 } + ? scopes + .Select(StellaOpsScopes.Normalize) + .Where(static scope => !string.IsNullOrEmpty(scope)) + .Select(static scope => scope!) + .Distinct(StringComparer.Ordinal) + .ToArray() + : Array.Empty(); + + var claims = new List + { + new Claim(StellaOpsClaimTypes.Subject, "test-user"), + new Claim(StellaOpsClaimTypes.Tenant, normalizedTenant), + new Claim(StellaOpsClaimTypes.Scope, string.Join(' ', scopeSet)) + }; + + foreach (var scope in scopeSet) + { + claims.Add(new Claim(StellaOpsClaimTypes.ScopeItem, scope)); + } + + var credentials = new SigningCredentials(TestSigningKey, SecurityAlgorithms.HmacSha256); + var now = DateTime.UtcNow; + var token = new JwtSecurityToken( + issuer: TestAuthorityIssuer, + audience: TestAuthorityAudience, + claims: claims, + notBefore: now.AddMinutes(-5), + expires: now.AddMinutes(30), + signingCredentials: credentials); + + return new JwtSecurityTokenHandler().WriteToken(token); + } + + private sealed record MetricMeasurement(string Instrument, long Value, IReadOnlyDictionary Tags); + + private sealed class DemoJob : IJob + { + public Task ExecuteAsync(JobExecutionContext context, CancellationToken cancellationToken) => Task.CompletedTask; + } + + private sealed class StubJobCoordinator : IJobCoordinator + { + public JobTriggerResult NextResult { get; set; } = JobTriggerResult.NotFound("not set"); + + public IReadOnlyList Definitions { get; set; } = Array.Empty(); + + public IReadOnlyList RecentRuns { get; set; } = Array.Empty(); + + public IReadOnlyList ActiveRuns { get; set; } = Array.Empty(); + + public Dictionary Runs { get; } = new(); + + public Dictionary LastRuns { get; } = new(StringComparer.Ordinal); + + public Task TriggerAsync(string kind, IReadOnlyDictionary? parameters, string trigger, CancellationToken cancellationToken) + => Task.FromResult(NextResult); + + public Task> GetDefinitionsAsync(CancellationToken cancellationToken) + => Task.FromResult(Definitions); + + public Task> GetRecentRunsAsync(string? kind, int limit, CancellationToken cancellationToken) + { + IEnumerable query = RecentRuns; + if (!string.IsNullOrWhiteSpace(kind)) + { + query = query.Where(run => string.Equals(run.Kind, kind, StringComparison.Ordinal)); + } + + return Task.FromResult>(query.Take(limit).ToArray()); + } + + public Task> GetActiveRunsAsync(CancellationToken cancellationToken) + => Task.FromResult(ActiveRuns); + + public Task GetRunAsync(Guid runId, CancellationToken cancellationToken) + => Task.FromResult(Runs.TryGetValue(runId, out var run) ? run : null); + + public Task GetLastRunAsync(string kind, CancellationToken cancellationToken) + => Task.FromResult(LastRuns.TryGetValue(kind, out var run) ? run : null); + + public Task> GetLastRunsAsync(IEnumerable kinds, CancellationToken cancellationToken) + { + var map = new Dictionary(StringComparer.Ordinal); + foreach (var kind in kinds) + { + if (kind is null) + { + continue; + } + + if (LastRuns.TryGetValue(kind, out var run) && run is not null) + { + map[kind] = run; + } + } + + return Task.FromResult>(map); + } + } +} diff --git a/src/Web/StellaOps.Web/TASKS.md b/src/Web/StellaOps.Web/TASKS.md index 65d07952d..5df14a912 100644 --- a/src/Web/StellaOps.Web/TASKS.md +++ b/src/Web/StellaOps.Web/TASKS.md @@ -5,6 +5,7 @@ > 2025-10-26: Introduced `StellaOps.Aoc` library with forbidden key list, guard result/options, and baseline write guard + tests. Middleware/analyzer wiring still pending. > 2025-10-30: Added `StellaOps.Aoc.AspNetCore` helpers (`AddAocGuard`, `AocHttpResults`) and switched Concelier WebService to the shared problem-details mapper; analyzer wiring remains pending. > 2025-10-30: Published `docs/aoc/guard-library.md` covering registration patterns, endpoint filters, and error mapping for ingestion services. +> 2025-11-06: Added `RequireAocGuard` route helper, wired Concelier advisory ingestion endpoint to the shared filter, refreshed docs, and introduced extension tests. | WEB-AOC-19-002 `Provenance & signature helpers` | TODO | BE-Base Platform Guild | WEB-AOC-19-001 | Ship `ProvenanceBuilder`, checksum utilities, and signature verification helper integrated with guard logging. Cover DSSE/CMS formats with unit tests. | | WEB-AOC-19-003 `Analyzer + test fixtures` | TODO | QA Guild, BE-Base Platform Guild | WEB-AOC-19-001 | Author Roslyn analyzer preventing ingestion modules from writing forbidden keys without guard, and provide shared test fixtures for guard validation used by Concelier/Excititor service tests. | > Docs alignment (2025-10-26): Analyzer expectations detailed in `docs/ingestion/aggregation-only-contract.md` §3/5; CI integration tracked via DEVOPS-AOC-19-001.