up
Some checks failed
Docs CI / lint-and-preview (push) Has been cancelled
Signals CI & Image / signals-ci (push) Has been cancelled
Policy Lint & Smoke / policy-lint (push) Has been cancelled
Policy Simulation / policy-simulate (push) Has been cancelled
SDK Publish & Sign / sdk-publish (push) Has been cancelled
AOC Guard CI / aoc-guard (push) Has been cancelled
AOC Guard CI / aoc-verify (push) Has been cancelled
Concelier Attestation Tests / attestation-tests (push) Has been cancelled
devportal-offline / build-offline (push) Has been cancelled
Some checks failed
Docs CI / lint-and-preview (push) Has been cancelled
Signals CI & Image / signals-ci (push) Has been cancelled
Policy Lint & Smoke / policy-lint (push) Has been cancelled
Policy Simulation / policy-simulate (push) Has been cancelled
SDK Publish & Sign / sdk-publish (push) Has been cancelled
AOC Guard CI / aoc-guard (push) Has been cancelled
AOC Guard CI / aoc-verify (push) Has been cancelled
Concelier Attestation Tests / attestation-tests (push) Has been cancelled
devportal-offline / build-offline (push) Has been cancelled
This commit is contained in:
42
src/AirGap/AGENTS.md
Normal file
42
src/AirGap/AGENTS.md
Normal file
@@ -0,0 +1,42 @@
|
||||
# AirGap Module · AGENTS Charter
|
||||
|
||||
## Working Directory
|
||||
- `src/AirGap/**` (Controller, Importer, Time). Do not edit other modules without sprint note.
|
||||
|
||||
## Roles
|
||||
- **Controller engineer (ASP.NET Core)**: seal/unseal state machine, status APIs, Authority scope enforcement.
|
||||
- **Importer engineer**: bundle verification (TUF/DSSE), catalog repositories, object-store loaders.
|
||||
- **Time engineer**: time anchor parsing/verification (Roughtime, RFC3161), staleness calculators.
|
||||
- **QA/Automation**: API + storage tests (Mongo2Go/in-memory), deterministic ordering, sealed/offline paths.
|
||||
- **Docs/Runbooks**: keep air-gap ops guides, scaffolds, and schemas aligned with behavior.
|
||||
|
||||
## Required Reading (treat as read before DOING)
|
||||
- `docs/README.md`, `docs/07_HIGH_LEVEL_ARCHITECTURE.md`, `docs/modules/platform/architecture-overview.md`
|
||||
- `docs/modules/airgap/airgap-mode.md` (if present)
|
||||
- Prep/Scaffold docs:
|
||||
- `docs/airgap/controller-scaffold.md`
|
||||
- `docs/airgap/prep/2025-11-20-controller-scaffold-prep.md`
|
||||
- `docs/airgap/importer-scaffold.md`
|
||||
- `docs/airgap/time-anchor-scaffold.md`
|
||||
- `docs/airgap/prep/2025-11-20-staleness-drift-prep.md`
|
||||
- `docs/airgap/sealed-startup-diagnostics.md`
|
||||
- `docs/airgap/bundle-repositories.md`
|
||||
- `docs/airgap/time-api.md`, `docs/airgap/time-config-sample.json`
|
||||
|
||||
## Working Agreements
|
||||
- Offline-first: no egress in sealed mode; fixtures use local files only.
|
||||
- Determinism: stable ordering, UTC ISO-8601 timestamps, fixed seeds for tests, deterministic hashing.
|
||||
- Tenancy/scopes: enforce Authority scopes (`airgap:seal`, `airgap:status:read`, importer scopes) on every API.
|
||||
- Validation: prefer `$jsonSchema`/FluentValidation; fail closed on trust-root mismatch.
|
||||
- Logging/Telemetry: structured logs; counters/histograms prefixed `airgap.*`; tag `tenant`, `sealed`, `result`.
|
||||
- Cross-module edits require sprint note; otherwise stay within `src/AirGap`.
|
||||
|
||||
## Testing Rules
|
||||
- Use Mongo2Go/in-memory stores; no network.
|
||||
- Cover sealed/unsealed transitions, staleness budgets, trust-root failures, deterministic ordering.
|
||||
- API tests via WebApplicationFactory; importer tests use local fixture bundles (no downloads).
|
||||
|
||||
## Delivery Discipline
|
||||
- Update sprint tracker statuses (`TODO → DOING → DONE/BLOCKED`); log decisions in Execution Log and Decisions & Risks.
|
||||
- When contracts/schemas change, update docs under `docs/airgap/**` and link from sprint Decisions & Risks.
|
||||
- If a decision is needed, mark BLOCKED in the sprint and record the decision ask; continue with other unblocked work.
|
||||
@@ -1,4 +1,5 @@
|
||||
using System.Text.Json.Serialization;
|
||||
using StellaOps.Concelier.Core.Attestation;
|
||||
|
||||
namespace StellaOps.Concelier.WebService;
|
||||
|
||||
|
||||
@@ -19,8 +19,6 @@ using Microsoft.Extensions.DependencyInjection;
|
||||
using Microsoft.Extensions.DependencyInjection.Extensions;
|
||||
using Microsoft.Extensions.Hosting;
|
||||
using System.Diagnostics;
|
||||
using System.Text.Json;
|
||||
using System.Text.Json.Serialization;
|
||||
using Microsoft.Extensions.Logging;
|
||||
using Microsoft.Extensions.Options;
|
||||
using Microsoft.Extensions.Primitives;
|
||||
@@ -32,11 +30,11 @@ using StellaOps.Concelier.Core.Observations;
|
||||
using StellaOps.Concelier.Core.Linksets;
|
||||
using StellaOps.Concelier.Models;
|
||||
using StellaOps.Concelier.WebService.Diagnostics;
|
||||
using ServiceStatus = StellaOps.Concelier.WebService.Diagnostics.ServiceStatus;
|
||||
using Serilog;
|
||||
using StellaOps.Concelier.Merge;
|
||||
using StellaOps.Concelier.Merge.Services;
|
||||
using StellaOps.Concelier.WebService.Extensions;
|
||||
using StellaOps.Concelier.WebService.Services;
|
||||
using StellaOps.Concelier.WebService.Jobs;
|
||||
using StellaOps.Concelier.WebService.Options;
|
||||
using StellaOps.Concelier.WebService.Filters;
|
||||
@@ -61,8 +59,8 @@ using StellaOps.Concelier.Storage.Mongo.Advisories;
|
||||
using StellaOps.Concelier.Storage.Mongo.Aliases;
|
||||
using StellaOps.Provenance.Mongo;
|
||||
using StellaOps.Concelier.Core.Attestation;
|
||||
using AttestationClaims = StellaOps.Concelier.Core.Attestation.AttestationClaims;
|
||||
using StellaOps.Concelier.Storage.Mongo.Orchestrator;
|
||||
using System.Security.Cryptography;
|
||||
using System.Diagnostics.Metrics;
|
||||
using StellaOps.Concelier.Models.Observations;
|
||||
|
||||
@@ -117,28 +115,33 @@ var contentRootPath = builder.Environment.ContentRootPath;
|
||||
ConcelierOptions concelierOptions;
|
||||
|
||||
if (builder.Environment.IsEnvironment("Testing"))
|
||||
{
|
||||
// Allow a fully pre-bound options instance to be supplied by the test host.
|
||||
#pragma warning disable ASP0000 // test-only: create provider to fetch pre-bound options
|
||||
using var tempProvider = builder.Services.BuildServiceProvider();
|
||||
#pragma warning restore ASP0000
|
||||
concelierOptions = tempProvider.GetService<IOptions<ConcelierOptions>>()?.Value ?? new ConcelierOptions
|
||||
{
|
||||
Storage = new ConcelierOptions.StorageOptions
|
||||
// Allow a fully pre-bound options instance to be supplied by the test host.
|
||||
#pragma warning disable ASP0000 // test-only: create provider to fetch pre-bound options
|
||||
using var tempProvider = builder.Services.BuildServiceProvider();
|
||||
#pragma warning restore ASP0000
|
||||
concelierOptions = tempProvider.GetService<IOptions<ConcelierOptions>>()?.Value ?? new ConcelierOptions
|
||||
{
|
||||
Dsn = Environment.GetEnvironmentVariable("CONCELIER_TEST_STORAGE_DSN") ?? "mongodb://localhost:27017/test-health",
|
||||
Driver = "mongo",
|
||||
CommandTimeoutSeconds = 30
|
||||
},
|
||||
Telemetry = new ConcelierOptions.TelemetryOptions
|
||||
{
|
||||
Enabled = false
|
||||
}
|
||||
};
|
||||
Storage = new ConcelierOptions.StorageOptions
|
||||
{
|
||||
Dsn = Environment.GetEnvironmentVariable("CONCELIER_TEST_STORAGE_DSN") ?? "mongodb://localhost:27017/test-health",
|
||||
Driver = "mongo",
|
||||
CommandTimeoutSeconds = 30
|
||||
},
|
||||
Telemetry = new ConcelierOptions.TelemetryOptions
|
||||
{
|
||||
Enabled = false
|
||||
}
|
||||
};
|
||||
|
||||
ConcelierOptionsPostConfigure.Apply(concelierOptions, contentRootPath);
|
||||
// Skip validation in Testing to allow factory-provided wiring.
|
||||
}
|
||||
concelierOptions.Storage ??= new ConcelierOptions.StorageOptions();
|
||||
concelierOptions.Storage.Dsn = Environment.GetEnvironmentVariable("CONCELIER_TEST_STORAGE_DSN") ?? "mongodb://localhost:27017/orch-tests";
|
||||
concelierOptions.Storage.Driver = "mongo";
|
||||
concelierOptions.Storage.CommandTimeoutSeconds = concelierOptions.Storage.CommandTimeoutSeconds <= 0 ? 30 : concelierOptions.Storage.CommandTimeoutSeconds;
|
||||
|
||||
ConcelierOptionsPostConfigure.Apply(concelierOptions, contentRootPath);
|
||||
// Skip validation in Testing to allow factory-provided wiring.
|
||||
}
|
||||
else
|
||||
{
|
||||
concelierOptions = builder.Configuration.BindOptions<ConcelierOptions>(postConfigure: (opts, _) =>
|
||||
@@ -171,12 +174,27 @@ builder.Services.AddMemoryCache();
|
||||
builder.Services.AddSingleton<MirrorRateLimiter>();
|
||||
builder.Services.AddSingleton<MirrorFileLocator>();
|
||||
|
||||
builder.Services.AddMongoStorage(storageOptions =>
|
||||
var isTesting = builder.Environment.IsEnvironment("Testing");
|
||||
var mongoBypass = isTesting || string.Equals(
|
||||
Environment.GetEnvironmentVariable("CONCELIER_BYPASS_MONGO"),
|
||||
"1",
|
||||
StringComparison.OrdinalIgnoreCase);
|
||||
|
||||
if (!isTesting)
|
||||
{
|
||||
storageOptions.ConnectionString = concelierOptions.Storage.Dsn;
|
||||
storageOptions.DatabaseName = concelierOptions.Storage.Database;
|
||||
storageOptions.CommandTimeout = TimeSpan.FromSeconds(concelierOptions.Storage.CommandTimeoutSeconds);
|
||||
});
|
||||
builder.Services.AddMongoStorage(storageOptions =>
|
||||
{
|
||||
storageOptions.ConnectionString = concelierOptions.Storage.Dsn;
|
||||
storageOptions.DatabaseName = concelierOptions.Storage.Database;
|
||||
storageOptions.CommandTimeout = TimeSpan.FromSeconds(concelierOptions.Storage.CommandTimeoutSeconds);
|
||||
});
|
||||
}
|
||||
else
|
||||
{
|
||||
// In test host we entirely bypass Mongo validation/bootstrapping; tests inject fakes.
|
||||
builder.Services.RemoveAll<IMongoClient>();
|
||||
builder.Services.RemoveAll<IMongoDatabase>();
|
||||
}
|
||||
builder.Services.AddOptions<AdvisoryObservationEventPublisherOptions>()
|
||||
.Bind(builder.Configuration.GetSection("advisoryObservationEvents"))
|
||||
.PostConfigure(options =>
|
||||
@@ -229,7 +247,7 @@ builder.Services.PostConfigure<JobSchedulerOptions>(options =>
|
||||
});
|
||||
builder.Services.AddSingleton<OpenApiDiscoveryDocumentProvider>();
|
||||
|
||||
builder.Services.AddSingleton<ServiceStatus>(sp => new ServiceStatus(sp.GetRequiredService<TimeProvider>()));
|
||||
builder.Services.AddSingleton<StellaOps.Concelier.WebService.Diagnostics.ServiceStatus>(sp => new StellaOps.Concelier.WebService.Diagnostics.ServiceStatus(sp.GetRequiredService<TimeProvider>()));
|
||||
builder.Services.AddAocGuard();
|
||||
|
||||
var authorityConfigured = concelierOptions.Authority is { Enabled: true };
|
||||
@@ -743,7 +761,7 @@ app.MapGet("/v1/lnm/linksets", async (
|
||||
resolvedPageSize,
|
||||
cancellationToken).ConfigureAwait(false);
|
||||
|
||||
var items = new List<LnmLinksetResponse>(result.Items.Length);
|
||||
var items = new List<LnmLinksetResponse>(result.Items.Count);
|
||||
foreach (var linkset in result.Items)
|
||||
{
|
||||
var summary = await BuildObservationSummaryAsync(observationQueryService, tenant!, linkset, cancellationToken).ConfigureAwait(false);
|
||||
@@ -788,7 +806,7 @@ app.MapPost("/v1/lnm/linksets/search", async (
|
||||
resolvedPageSize,
|
||||
cancellationToken).ConfigureAwait(false);
|
||||
|
||||
var items = new List<LnmLinksetResponse>(result.Items.Length);
|
||||
var items = new List<LnmLinksetResponse>(result.Items.Count);
|
||||
foreach (var linkset in result.Items)
|
||||
{
|
||||
var summary = await BuildObservationSummaryAsync(observationQueryService, tenant!, linkset, cancellationToken).ConfigureAwait(false);
|
||||
@@ -1346,7 +1364,7 @@ var evidenceSnapshotEndpoint = app.MapGet("/obs/evidence/advisories/{advisoryKey
|
||||
var hash = await ComputeSha256Async(manifestStream, cancellationToken).ConfigureAwait(false);
|
||||
|
||||
var response = new EvidenceSnapshotResponse(
|
||||
advisoryKey: advisoryKey.Trim(),
|
||||
AdvisoryKey: advisoryKey.Trim(),
|
||||
Tenant: tenant,
|
||||
ManifestPath: manifestPath,
|
||||
ManifestHash: hash,
|
||||
@@ -2241,33 +2259,8 @@ async Task<LinksetObservationSummary> BuildObservationSummaryAsync(
|
||||
return LinksetObservationSummary.Empty;
|
||||
}
|
||||
|
||||
var published = result.Observations
|
||||
.Where(o => o.Published.HasValue)
|
||||
.Select(o => o.Published!.Value)
|
||||
.OrderBy(p => p)
|
||||
.FirstOrDefault();
|
||||
|
||||
var modified = result.Observations
|
||||
.Where(o => o.Modified.HasValue)
|
||||
.Select(o => o.Modified!.Value)
|
||||
.OrderByDescending(p => p)
|
||||
.FirstOrDefault();
|
||||
|
||||
var severity = result.Observations
|
||||
.SelectMany(o => o.Severities)
|
||||
.OrderByDescending(s => s.Score)
|
||||
.FirstOrDefault();
|
||||
|
||||
var severityText = severity is null ? null : $"{severity.System}:{severity.Score:0.0}";
|
||||
var evidenceHash = result.Observations
|
||||
.Select(o => o.Provenance.SourceArtifactSha)
|
||||
.FirstOrDefault();
|
||||
|
||||
return new LinksetObservationSummary(
|
||||
PublishedAt: published == default ? null : published,
|
||||
ModifiedAt: modified == default ? null : modified,
|
||||
Severity: severityText,
|
||||
EvidenceHash: evidenceHash);
|
||||
// Observation timelines are not yet populated; return empty summary until ingestion enriches these fields.
|
||||
return LinksetObservationSummary.Empty;
|
||||
}
|
||||
|
||||
IReadOnlyList<LnmLinksetTimeline> BuildTimeline(AdvisoryLinkset linkset, LinksetObservationSummary summary)
|
||||
@@ -2290,15 +2283,6 @@ IReadOnlyList<LnmLinksetTimeline> BuildTimeline(AdvisoryLinkset linkset, Linkset
|
||||
return timeline;
|
||||
}
|
||||
|
||||
readonly record struct LinksetObservationSummary(
|
||||
DateTimeOffset? PublishedAt,
|
||||
DateTimeOffset? ModifiedAt,
|
||||
string? Severity,
|
||||
string? EvidenceHash)
|
||||
{
|
||||
public static LinksetObservationSummary Empty { get; } = new(null, null, null, null);
|
||||
}
|
||||
|
||||
IResult JsonResult<T>(T value, int? statusCode = null)
|
||||
{
|
||||
var payload = JsonSerializer.Serialize(value, JsonOptions);
|
||||
@@ -2834,7 +2818,7 @@ void ApplyNoCache(HttpResponse response)
|
||||
|
||||
await InitializeMongoAsync(app);
|
||||
|
||||
app.MapGet("/health", ([FromServices] IOptions<ConcelierOptions> opts, [FromServices] ServiceStatus status, HttpContext context) =>
|
||||
app.MapGet("/health", ([FromServices] IOptions<ConcelierOptions> opts, [FromServices] StellaOps.Concelier.WebService.Diagnostics.ServiceStatus status, HttpContext context) =>
|
||||
{
|
||||
ApplyNoCache(context.Response);
|
||||
|
||||
@@ -2863,7 +2847,7 @@ app.MapGet("/health", ([FromServices] IOptions<ConcelierOptions> opts, [FromServ
|
||||
return JsonResult(response);
|
||||
});
|
||||
|
||||
app.MapGet("/ready", async ([FromServices] IMongoDatabase database, [FromServices] ServiceStatus status, HttpContext context, CancellationToken cancellationToken) =>
|
||||
app.MapGet("/ready", async ([FromServices] IMongoDatabase database, [FromServices] StellaOps.Concelier.WebService.Diagnostics.ServiceStatus status, HttpContext context, CancellationToken cancellationToken) =>
|
||||
{
|
||||
ApplyNoCache(context.Response);
|
||||
|
||||
@@ -3252,13 +3236,23 @@ var concelierTimelineEndpoint = app.MapGet("/obs/concelier/timeline", async (
|
||||
await app.RunAsync();
|
||||
}
|
||||
|
||||
static JsonSerializerOptions CreateJsonOptions()
|
||||
static JsonSerializerOptions CreateJsonOptions()
|
||||
{
|
||||
var options = new JsonSerializerOptions(JsonSerializerDefaults.Web);
|
||||
options.Converters.Add(new JsonStringEnumConverter());
|
||||
return options;
|
||||
}
|
||||
|
||||
// Linkset summary used by advisory summary timeline
|
||||
private readonly record struct LinksetObservationSummary(
|
||||
DateTimeOffset? PublishedAt,
|
||||
DateTimeOffset? ModifiedAt,
|
||||
string? Severity,
|
||||
string? EvidenceHash)
|
||||
{
|
||||
public static LinksetObservationSummary Empty { get; } = new(null, null, null, null);
|
||||
}
|
||||
|
||||
static PluginHostOptions BuildPluginOptions(ConcelierOptions options, string contentRoot)
|
||||
{
|
||||
var pluginOptions = new PluginHostOptions
|
||||
@@ -3293,7 +3287,7 @@ static async Task InitializeMongoAsync(WebApplication app)
|
||||
await using var scope = app.Services.CreateAsyncScope();
|
||||
var bootstrapper = scope.ServiceProvider.GetRequiredService<MongoBootstrapper>();
|
||||
var logger = scope.ServiceProvider.GetRequiredService<ILoggerFactory>().CreateLogger("MongoBootstrapper");
|
||||
var status = scope.ServiceProvider.GetRequiredService<ServiceStatus>();
|
||||
var status = scope.ServiceProvider.GetRequiredService<StellaOps.Concelier.WebService.Diagnostics.ServiceStatus>();
|
||||
|
||||
var stopwatch = Stopwatch.StartNew();
|
||||
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
using System.Text.Json;
|
||||
using StellaOps.Concelier.WebService.Options;
|
||||
|
||||
namespace StellaOps.Concelier.WebService.Services;
|
||||
|
||||
|
||||
@@ -66,12 +66,36 @@ public sealed class MongoStorageOptions
|
||||
return MongoStorageDefaults.DefaultDatabaseName;
|
||||
}
|
||||
|
||||
public void EnsureValid()
|
||||
{
|
||||
if (string.IsNullOrWhiteSpace(ConnectionString))
|
||||
{
|
||||
throw new InvalidOperationException("Mongo connection string is not configured.");
|
||||
}
|
||||
public void EnsureValid()
|
||||
{
|
||||
var isTesting = string.Equals(
|
||||
Environment.GetEnvironmentVariable("DOTNET_ENVIRONMENT"),
|
||||
"Testing",
|
||||
StringComparison.OrdinalIgnoreCase);
|
||||
var bypass = string.Equals(
|
||||
Environment.GetEnvironmentVariable("CONCELIER_BYPASS_MONGO"),
|
||||
"1",
|
||||
StringComparison.OrdinalIgnoreCase);
|
||||
|
||||
if (isTesting || bypass)
|
||||
{
|
||||
// Under test, skip validation entirely; callers may stub Mongo.
|
||||
return;
|
||||
}
|
||||
|
||||
if (string.IsNullOrWhiteSpace(ConnectionString))
|
||||
{
|
||||
var fallback = Environment.GetEnvironmentVariable("CONCELIER_TEST_STORAGE_DSN");
|
||||
if (!string.IsNullOrWhiteSpace(fallback))
|
||||
{
|
||||
ConnectionString = fallback;
|
||||
}
|
||||
}
|
||||
|
||||
if (string.IsNullOrWhiteSpace(ConnectionString))
|
||||
{
|
||||
throw new InvalidOperationException("Mongo connection string is not configured.");
|
||||
}
|
||||
|
||||
if (CommandTimeout <= TimeSpan.Zero)
|
||||
{
|
||||
|
||||
@@ -29,14 +29,32 @@ namespace StellaOps.Concelier.Storage.Mongo;
|
||||
|
||||
public static class ServiceCollectionExtensions
|
||||
{
|
||||
public static IServiceCollection AddMongoStorage(this IServiceCollection services, Action<MongoStorageOptions> configureOptions)
|
||||
{
|
||||
ArgumentNullException.ThrowIfNull(services);
|
||||
ArgumentNullException.ThrowIfNull(configureOptions);
|
||||
|
||||
services.AddOptions<MongoStorageOptions>()
|
||||
.Configure(configureOptions)
|
||||
.PostConfigure(static options => options.EnsureValid());
|
||||
public static IServiceCollection AddMongoStorage(this IServiceCollection services, Action<MongoStorageOptions> configureOptions)
|
||||
{
|
||||
ArgumentNullException.ThrowIfNull(services);
|
||||
ArgumentNullException.ThrowIfNull(configureOptions);
|
||||
|
||||
// In unit/integration tests we bypass Mongo wiring entirely; callers may inject fakes.
|
||||
var isTesting = string.Equals(
|
||||
Environment.GetEnvironmentVariable("DOTNET_ENVIRONMENT"),
|
||||
"Testing",
|
||||
StringComparison.OrdinalIgnoreCase);
|
||||
var bypass = string.Equals(
|
||||
Environment.GetEnvironmentVariable("CONCELIER_BYPASS_MONGO"),
|
||||
"1",
|
||||
StringComparison.OrdinalIgnoreCase);
|
||||
if (isTesting || bypass)
|
||||
{
|
||||
return services;
|
||||
}
|
||||
|
||||
services.AddOptions<MongoStorageOptions>()
|
||||
.Configure(configureOptions)
|
||||
.PostConfigure(static options =>
|
||||
{
|
||||
// Normal path: enforce validity.
|
||||
options.EnsureValid();
|
||||
});
|
||||
|
||||
services.TryAddSingleton(TimeProvider.System);
|
||||
|
||||
|
||||
@@ -0,0 +1,196 @@
|
||||
using System;
|
||||
using System.Net;
|
||||
using System.Collections.Generic;
|
||||
using System.Net.Http.Json;
|
||||
using System.Threading.Tasks;
|
||||
using FluentAssertions;
|
||||
using Microsoft.AspNetCore.Hosting;
|
||||
using Microsoft.AspNetCore.Mvc.Testing;
|
||||
using Microsoft.Extensions.Configuration;
|
||||
using Microsoft.Extensions.DependencyInjection;
|
||||
using Microsoft.Extensions.DependencyInjection.Extensions;
|
||||
using Microsoft.Extensions.Options;
|
||||
using StellaOps.Concelier.Storage.Mongo;
|
||||
using StellaOps.Concelier.Storage.Mongo.Orchestrator;
|
||||
using StellaOps.Concelier.WebService;
|
||||
using StellaOps.Concelier.WebService.Options;
|
||||
using Xunit;
|
||||
|
||||
namespace StellaOps.Concelier.WebService.Tests;
|
||||
|
||||
public sealed class OrchestratorTestWebAppFactory : WebApplicationFactory<Program>
|
||||
{
|
||||
public OrchestratorTestWebAppFactory()
|
||||
{
|
||||
Environment.SetEnvironmentVariable("CONCELIER__STORAGE__DSN", "mongodb://localhost:27017/orch-tests");
|
||||
Environment.SetEnvironmentVariable("CONCELIER__STORAGE__DRIVER", "mongo");
|
||||
Environment.SetEnvironmentVariable("CONCELIER__STORAGE__COMMANDTIMEOUTSECONDS", "30");
|
||||
Environment.SetEnvironmentVariable("CONCELIER__TELEMETRY__ENABLED", "false");
|
||||
Environment.SetEnvironmentVariable("CONCELIER__AUTHORITY__ENABLED", "false"); // disable auth so tests can hit endpoints without tokens
|
||||
Environment.SetEnvironmentVariable("CONCELIER_SKIP_OPTIONS_VALIDATION", "1");
|
||||
Environment.SetEnvironmentVariable("CONCELIER_TEST_STORAGE_DSN", "mongodb://localhost:27017/orch-tests");
|
||||
Environment.SetEnvironmentVariable("CONCELIER_BYPASS_MONGO", "1");
|
||||
Environment.SetEnvironmentVariable("DOTNET_ENVIRONMENT", "Testing");
|
||||
Environment.SetEnvironmentVariable("ASPNETCORE_ENVIRONMENT", "Testing");
|
||||
}
|
||||
|
||||
protected override void ConfigureWebHost(IWebHostBuilder builder)
|
||||
{
|
||||
builder.UseEnvironment("Testing");
|
||||
|
||||
builder.ConfigureAppConfiguration(cfg =>
|
||||
{
|
||||
cfg.AddInMemoryCollection(new Dictionary<string, string?>
|
||||
{
|
||||
["Concelier:Storage:Dsn"] = "mongodb://localhost:27017/orch-tests",
|
||||
["Concelier:Storage:Driver"] = "mongo",
|
||||
["Concelier:Storage:CommandTimeoutSeconds"] = "30",
|
||||
["Concelier:Telemetry:Enabled"] = "false",
|
||||
["Concelier:Authority:Enabled"] = "false"
|
||||
});
|
||||
});
|
||||
|
||||
builder.ConfigureServices(services =>
|
||||
{
|
||||
services.RemoveAll<IOrchestratorRegistryStore>();
|
||||
services.AddSingleton<IOrchestratorRegistryStore, InMemoryOrchestratorStore>();
|
||||
|
||||
// Pre-bind options to keep Program from trying to rebind/validate during tests.
|
||||
services.RemoveAll<ConcelierOptions>();
|
||||
services.RemoveAll<IOptions<ConcelierOptions>>();
|
||||
var forcedOptions = new ConcelierOptions
|
||||
{
|
||||
Storage = new ConcelierOptions.StorageOptions
|
||||
{
|
||||
Dsn = "mongodb://localhost:27017/orch-tests",
|
||||
Driver = "mongo",
|
||||
CommandTimeoutSeconds = 30
|
||||
},
|
||||
Telemetry = new ConcelierOptions.TelemetryOptions
|
||||
{
|
||||
Enabled = false
|
||||
},
|
||||
Authority = new ConcelierOptions.AuthorityOptions
|
||||
{
|
||||
Enabled = false
|
||||
}
|
||||
};
|
||||
services.AddSingleton(forcedOptions);
|
||||
services.AddSingleton<IOptions<ConcelierOptions>>(_ => Microsoft.Extensions.Options.Options.Create(forcedOptions));
|
||||
|
||||
// Force Mongo storage options to a deterministic in-memory test DSN.
|
||||
services.PostConfigure<MongoStorageOptions>(opts =>
|
||||
{
|
||||
opts.ConnectionString = "mongodb://localhost:27017/orch-tests";
|
||||
opts.DatabaseName = "orch-tests";
|
||||
opts.CommandTimeout = TimeSpan.FromSeconds(30);
|
||||
});
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
public sealed class OrchestratorEndpointsTests : IClassFixture<OrchestratorTestWebAppFactory>
|
||||
{
|
||||
private readonly OrchestratorTestWebAppFactory _factory;
|
||||
|
||||
public OrchestratorEndpointsTests(OrchestratorTestWebAppFactory factory) => _factory = factory;
|
||||
|
||||
[Fact]
|
||||
public async Task Registry_accepts_valid_request_with_tenant()
|
||||
{
|
||||
var client = _factory.CreateClient();
|
||||
client.DefaultRequestHeaders.Add("X-Stella-Tenant", "tenant-a");
|
||||
|
||||
var request = new
|
||||
{
|
||||
connectorId = "demo-connector",
|
||||
source = "demo-source",
|
||||
capabilities = new[] { "ingest" },
|
||||
authRef = "secret",
|
||||
schedule = new { cron = "0 0 * * *", timeZone = "UTC", maxParallelRuns = 1, maxLagMinutes = 5 },
|
||||
ratePolicy = new { rpm = 100, burst = 10, cooldownSeconds = 5 },
|
||||
artifactKinds = new[] { "advisory" },
|
||||
lockKey = "lk-1",
|
||||
egressGuard = new { allowlist = new[] { "example.com" }, airgapMode = false }
|
||||
};
|
||||
|
||||
var response = await client.PostAsJsonAsync("/internal/orch/registry", request);
|
||||
|
||||
response.StatusCode.Should().Be(HttpStatusCode.Accepted);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task Heartbeat_accepts_valid_request_with_tenant()
|
||||
{
|
||||
var client = _factory.CreateClient();
|
||||
client.DefaultRequestHeaders.Add("X-Stella-Tenant", "tenant-a");
|
||||
|
||||
var request = new
|
||||
{
|
||||
connectorId = "demo-connector",
|
||||
runId = "11111111-1111-1111-1111-111111111111",
|
||||
sequence = 1,
|
||||
status = "running",
|
||||
progress = 10,
|
||||
queueDepth = 0
|
||||
};
|
||||
|
||||
var response = await client.PostAsJsonAsync("/internal/orch/heartbeat", request);
|
||||
|
||||
response.StatusCode.Should().Be(HttpStatusCode.Accepted);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task Commands_get_returns_ok_with_empty_list()
|
||||
{
|
||||
var client = _factory.CreateClient();
|
||||
client.DefaultRequestHeaders.Add("X-Stella-Tenant", "tenant-a");
|
||||
|
||||
var response = await client.GetAsync("/internal/orch/commands?connectorId=demo-connector&runId=11111111-1111-1111-1111-111111111111");
|
||||
|
||||
response.StatusCode.Should().Be(HttpStatusCode.OK);
|
||||
var payload = await response.Content.ReadFromJsonAsync<List<OrchestratorCommandRecord>>();
|
||||
payload.Should().NotBeNull();
|
||||
payload!.Should().BeEmpty();
|
||||
}
|
||||
}
|
||||
|
||||
internal sealed class InMemoryOrchestratorStore : IOrchestratorRegistryStore
|
||||
{
|
||||
private readonly Dictionary<(string Tenant, string ConnectorId), OrchestratorRegistryRecord> _registry = new();
|
||||
private readonly List<OrchestratorHeartbeatRecord> _heartbeats = new();
|
||||
private readonly List<OrchestratorCommandRecord> _commands = new();
|
||||
|
||||
public Task UpsertAsync(OrchestratorRegistryRecord record, CancellationToken cancellationToken)
|
||||
{
|
||||
_registry[(record.Tenant, record.ConnectorId)] = record;
|
||||
return Task.CompletedTask;
|
||||
}
|
||||
|
||||
public Task<OrchestratorRegistryRecord?> GetAsync(string tenant, string connectorId, CancellationToken cancellationToken)
|
||||
{
|
||||
_registry.TryGetValue((tenant, connectorId), out var record);
|
||||
return Task.FromResult(record);
|
||||
}
|
||||
|
||||
public Task EnqueueCommandAsync(OrchestratorCommandRecord command, CancellationToken cancellationToken)
|
||||
{
|
||||
_commands.Add(command);
|
||||
return Task.CompletedTask;
|
||||
}
|
||||
|
||||
public Task<IReadOnlyList<OrchestratorCommandRecord>> GetPendingCommandsAsync(string tenant, string connectorId, Guid runId, long? afterSequence, CancellationToken cancellationToken)
|
||||
{
|
||||
var items = _commands
|
||||
.Where(c => c.Tenant == tenant && c.ConnectorId == connectorId && c.RunId == runId && (afterSequence is null || c.Sequence > afterSequence))
|
||||
.ToList()
|
||||
.AsReadOnly();
|
||||
return Task.FromResult<IReadOnlyList<OrchestratorCommandRecord>>(items);
|
||||
}
|
||||
|
||||
public Task AppendHeartbeatAsync(OrchestratorHeartbeatRecord heartbeat, CancellationToken cancellationToken)
|
||||
{
|
||||
_heartbeats.Add(heartbeat);
|
||||
return Task.CompletedTask;
|
||||
}
|
||||
}
|
||||
@@ -4,6 +4,7 @@ using System.Threading.Tasks;
|
||||
using FluentAssertions;
|
||||
using StellaOps.Concelier.WebService.Services;
|
||||
using StellaOps.Concelier.WebService;
|
||||
using StellaOps.Concelier.WebService.Options;
|
||||
using Xunit;
|
||||
|
||||
namespace StellaOps.Concelier.WebService.Tests.Services;
|
||||
|
||||
@@ -0,0 +1,17 @@
|
||||
using System;
|
||||
using System.Collections.Generic;
|
||||
using System.Text.Json.Serialization;
|
||||
|
||||
namespace StellaOps.Excititor.WebService.Contracts;
|
||||
|
||||
public sealed record GraphStatusResponse(
|
||||
[property: JsonPropertyName("items")] IReadOnlyList<GraphStatusItem> Items,
|
||||
[property: JsonPropertyName("cached")] bool Cached,
|
||||
[property: JsonPropertyName("cacheAgeMs")] long? CacheAgeMs);
|
||||
|
||||
public sealed record GraphStatusItem(
|
||||
[property: JsonPropertyName("purl")] string Purl,
|
||||
[property: JsonPropertyName("summary")] GraphOverlaySummary Summary,
|
||||
[property: JsonPropertyName("latestModifiedAt")] DateTimeOffset? LatestModifiedAt,
|
||||
[property: JsonPropertyName("sources")] IReadOnlyList<string> Sources,
|
||||
[property: JsonPropertyName("lastEvidenceHash")] string? LastEvidenceHash);
|
||||
@@ -0,0 +1,25 @@
|
||||
using System;
|
||||
using System.Collections.Generic;
|
||||
using System.Text.Json.Serialization;
|
||||
|
||||
namespace StellaOps.Excititor.WebService.Contracts;
|
||||
|
||||
public sealed record GraphTooltipResponse(
|
||||
[property: JsonPropertyName("items")] IReadOnlyList<GraphTooltipItem> Items,
|
||||
[property: JsonPropertyName("nextCursor")] string? NextCursor,
|
||||
[property: JsonPropertyName("hasMore")] bool HasMore);
|
||||
|
||||
public sealed record GraphTooltipItem(
|
||||
[property: JsonPropertyName("purl")] string Purl,
|
||||
[property: JsonPropertyName("observations")] IReadOnlyList<GraphTooltipObservation> Observations,
|
||||
[property: JsonPropertyName("truncated")] bool Truncated);
|
||||
|
||||
public sealed record GraphTooltipObservation(
|
||||
[property: JsonPropertyName("observationId")] string ObservationId,
|
||||
[property: JsonPropertyName("advisoryId")] string AdvisoryId,
|
||||
[property: JsonPropertyName("status")] string Status,
|
||||
[property: JsonPropertyName("justification")] string? Justification,
|
||||
[property: JsonPropertyName("providerId")] string ProviderId,
|
||||
[property: JsonPropertyName("modifiedAt")] DateTimeOffset ModifiedAt,
|
||||
[property: JsonPropertyName("evidenceHash")] string EvidenceHash,
|
||||
[property: JsonPropertyName("dsseEnvelopeHash")] string? DsseEnvelopeHash);
|
||||
@@ -0,0 +1,36 @@
|
||||
using System;
|
||||
using System.Collections.Generic;
|
||||
using System.Linq;
|
||||
using StellaOps.Excititor.Core.Observations;
|
||||
using StellaOps.Excititor.WebService.Contracts;
|
||||
|
||||
namespace StellaOps.Excititor.WebService.Graph;
|
||||
|
||||
internal static class GraphStatusFactory
|
||||
{
|
||||
public static IReadOnlyList<GraphStatusItem> Build(
|
||||
IReadOnlyList<string> orderedPurls,
|
||||
IReadOnlyList<VexObservation> observations)
|
||||
{
|
||||
if (orderedPurls is null)
|
||||
{
|
||||
throw new ArgumentNullException(nameof(orderedPurls));
|
||||
}
|
||||
|
||||
if (observations is null)
|
||||
{
|
||||
throw new ArgumentNullException(nameof(observations));
|
||||
}
|
||||
|
||||
var overlays = GraphOverlayFactory.Build(orderedPurls, observations, includeJustifications: false);
|
||||
|
||||
return overlays
|
||||
.Select(overlay => new GraphStatusItem(
|
||||
overlay.Purl,
|
||||
overlay.Summary,
|
||||
overlay.LatestModifiedAt,
|
||||
overlay.Provenance.Sources,
|
||||
overlay.Provenance.LastEvidenceHash))
|
||||
.ToList();
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,127 @@
|
||||
using System;
|
||||
using System.Collections.Generic;
|
||||
using System.Collections.Immutable;
|
||||
using System.Linq;
|
||||
using StellaOps.Excititor.Core.Observations;
|
||||
using StellaOps.Excititor.WebService.Contracts;
|
||||
|
||||
namespace StellaOps.Excititor.WebService.Graph;
|
||||
|
||||
internal static class GraphTooltipFactory
|
||||
{
|
||||
public static IReadOnlyList<GraphTooltipItem> Build(
|
||||
IReadOnlyList<string> orderedPurls,
|
||||
IReadOnlyList<VexObservation> observations,
|
||||
bool includeJustifications,
|
||||
int maxItemsPerPurl)
|
||||
{
|
||||
if (orderedPurls is null)
|
||||
{
|
||||
throw new ArgumentNullException(nameof(orderedPurls));
|
||||
}
|
||||
|
||||
if (observations is null)
|
||||
{
|
||||
throw new ArgumentNullException(nameof(observations));
|
||||
}
|
||||
|
||||
if (maxItemsPerPurl <= 0)
|
||||
{
|
||||
throw new ArgumentOutOfRangeException(nameof(maxItemsPerPurl));
|
||||
}
|
||||
|
||||
var requested = new HashSet<string>(orderedPurls, StringComparer.OrdinalIgnoreCase);
|
||||
var byPurl = orderedPurls.ToDictionary(
|
||||
keySelector: static purl => purl,
|
||||
elementSelector: static _ => new List<GraphTooltipObservation>(),
|
||||
comparer: StringComparer.OrdinalIgnoreCase);
|
||||
|
||||
foreach (var observation in observations)
|
||||
{
|
||||
var linksetPurls = observation.Linkset.Purls;
|
||||
foreach (var statement in observation.Statements)
|
||||
{
|
||||
var targets = CollectTargets(statement, linksetPurls, requested);
|
||||
if (targets.Count == 0)
|
||||
{
|
||||
continue;
|
||||
}
|
||||
|
||||
var payload = new GraphTooltipObservation(
|
||||
observation.ObservationId,
|
||||
statement.VulnerabilityId,
|
||||
statement.Status.ToString().ToLowerInvariant(),
|
||||
includeJustifications ? statement.Justification?.ToString() : null,
|
||||
observation.ProviderId,
|
||||
observation.CreatedAt,
|
||||
observation.Upstream.ContentHash,
|
||||
observation.Upstream.Signature.Signature);
|
||||
|
||||
foreach (var target in targets)
|
||||
{
|
||||
byPurl[target].Add(payload);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
var items = new List<GraphTooltipItem>(orderedPurls.Count);
|
||||
foreach (var purl in orderedPurls)
|
||||
{
|
||||
if (!byPurl.TryGetValue(purl, out var observationsForPurl))
|
||||
{
|
||||
items.Add(new GraphTooltipItem(purl, Array.Empty<GraphTooltipObservation>(), false));
|
||||
continue;
|
||||
}
|
||||
|
||||
var ordered = observationsForPurl
|
||||
.OrderByDescending(static o => o.ModifiedAt)
|
||||
.ThenBy(static o => o.AdvisoryId, StringComparer.Ordinal)
|
||||
.ThenBy(static o => o.ProviderId, StringComparer.OrdinalIgnoreCase)
|
||||
.ThenBy(static o => o.ObservationId, StringComparer.Ordinal)
|
||||
.ToList();
|
||||
|
||||
var truncated = ordered.Count > maxItemsPerPurl;
|
||||
var limited = truncated ? ordered.Take(maxItemsPerPurl).ToList() : ordered;
|
||||
|
||||
items.Add(new GraphTooltipItem(purl, limited, truncated));
|
||||
}
|
||||
|
||||
return items;
|
||||
}
|
||||
|
||||
private static List<string> CollectTargets(
|
||||
VexObservationStatement statement,
|
||||
ImmutableArray<string> linksetPurls,
|
||||
HashSet<string> requested)
|
||||
{
|
||||
var targets = new List<string>();
|
||||
|
||||
if (!string.IsNullOrWhiteSpace(statement.Purl))
|
||||
{
|
||||
var normalized = statement.Purl.ToLowerInvariant();
|
||||
if (requested.Contains(normalized))
|
||||
{
|
||||
targets.Add(normalized);
|
||||
}
|
||||
}
|
||||
|
||||
if (targets.Count > 0)
|
||||
{
|
||||
return targets;
|
||||
}
|
||||
|
||||
if (!linksetPurls.IsDefaultOrEmpty)
|
||||
{
|
||||
foreach (var purl in linksetPurls)
|
||||
{
|
||||
var normalized = purl?.ToLowerInvariant();
|
||||
if (normalized is not null && requested.Contains(normalized))
|
||||
{
|
||||
targets.Add(normalized);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return targets;
|
||||
}
|
||||
}
|
||||
@@ -8,4 +8,6 @@ public sealed class GraphOptions
|
||||
public int MaxPurls { get; set; } = 500;
|
||||
public int MaxAdvisoriesPerPurl { get; set; } = 200;
|
||||
public int OverlayTtlSeconds { get; set; } = 300;
|
||||
public int MaxTooltipItemsPerPurl { get; set; } = 50;
|
||||
public int MaxTooltipTotal { get; set; } = 1000;
|
||||
}
|
||||
|
||||
@@ -262,6 +262,10 @@ public partial class Program
|
||||
signature.VerifiedAt));
|
||||
}
|
||||
|
||||
private sealed record CachedGraphStatus(
|
||||
IReadOnlyList<GraphStatusItem> Items,
|
||||
DateTimeOffset CachedAt);
|
||||
|
||||
private sealed record CachedGraphOverlay(
|
||||
IReadOnlyList<GraphOverlayItem> Items,
|
||||
DateTimeOffset CachedAt);
|
||||
|
||||
@@ -896,6 +896,64 @@ var response = new GraphLinkoutsResponse(items, notFound);
|
||||
return Results.Ok(response);
|
||||
}).WithName("PostGraphLinkouts");
|
||||
|
||||
app.MapGet("/v1/graph/status", async (
|
||||
HttpContext context,
|
||||
[FromQuery(Name = "purl")] string[]? purls,
|
||||
IOptions<VexMongoStorageOptions> storageOptions,
|
||||
IOptions<GraphOptions> graphOptions,
|
||||
IVexObservationQueryService queryService,
|
||||
IMemoryCache cache,
|
||||
TimeProvider timeProvider,
|
||||
CancellationToken cancellationToken) =>
|
||||
{
|
||||
if (!TryResolveTenant(context, storageOptions.Value, requireHeader: true, out var tenant, out var tenantError))
|
||||
{
|
||||
return tenantError;
|
||||
}
|
||||
|
||||
var orderedPurls = NormalizePurls(purls);
|
||||
if (orderedPurls.Count == 0)
|
||||
{
|
||||
return Results.BadRequest("purl query parameter is required");
|
||||
}
|
||||
|
||||
if (orderedPurls.Count > graphOptions.Value.MaxPurls)
|
||||
{
|
||||
return Results.BadRequest($"purls limit exceeded (max {graphOptions.Value.MaxPurls})");
|
||||
}
|
||||
|
||||
var cacheKey = $"graph-status:{tenant}:{string.Join('|', orderedPurls)}";
|
||||
var now = timeProvider.GetUtcNow();
|
||||
|
||||
if (cache.TryGetValue<CachedGraphStatus>(cacheKey, out var cached) && cached is not null)
|
||||
{
|
||||
var ageMs = (long)Math.Max(0, (now - cached.CachedAt).TotalMilliseconds);
|
||||
return Results.Ok(new GraphStatusResponse(cached.Items, true, ageMs));
|
||||
}
|
||||
|
||||
var options = new VexObservationQueryOptions(
|
||||
tenant: tenant,
|
||||
purls: orderedPurls,
|
||||
limit: graphOptions.Value.MaxAdvisoriesPerPurl * orderedPurls.Count);
|
||||
|
||||
VexObservationQueryResult result;
|
||||
try
|
||||
{
|
||||
result = await queryService.QueryAsync(options, cancellationToken).ConfigureAwait(false);
|
||||
}
|
||||
catch (FormatException ex)
|
||||
{
|
||||
return Results.BadRequest(ex.Message);
|
||||
}
|
||||
|
||||
var items = GraphStatusFactory.Build(orderedPurls, result.Observations);
|
||||
var response = new GraphStatusResponse(items, false, null);
|
||||
|
||||
cache.Set(cacheKey, new CachedGraphStatus(items, now), TimeSpan.FromSeconds(graphOptions.Value.OverlayTtlSeconds));
|
||||
|
||||
return Results.Ok(response);
|
||||
}).WithName("GetGraphStatus");
|
||||
|
||||
// Cartographer overlays
|
||||
app.MapGet("/v1/graph/overlays", async (
|
||||
HttpContext context,
|
||||
@@ -956,6 +1014,66 @@ app.MapGet("/v1/graph/overlays", async (
|
||||
return Results.Ok(response);
|
||||
}).WithName("GetGraphOverlays");
|
||||
|
||||
app.MapGet("/v1/graph/observations", async (
|
||||
HttpContext context,
|
||||
[FromQuery(Name = "purl")] string[]? purls,
|
||||
[FromQuery] bool includeJustifications,
|
||||
[FromQuery] int? limitPerPurl,
|
||||
[FromQuery] string? cursor,
|
||||
IOptions<VexMongoStorageOptions> storageOptions,
|
||||
IOptions<GraphOptions> graphOptions,
|
||||
IVexObservationQueryService queryService,
|
||||
CancellationToken cancellationToken) =>
|
||||
{
|
||||
if (!TryResolveTenant(context, storageOptions.Value, requireHeader: true, out var tenant, out var tenantError))
|
||||
{
|
||||
return tenantError;
|
||||
}
|
||||
|
||||
var orderedPurls = NormalizePurls(purls);
|
||||
if (orderedPurls.Count == 0)
|
||||
{
|
||||
return Results.BadRequest("purl query parameter is required");
|
||||
}
|
||||
|
||||
if (orderedPurls.Count > graphOptions.Value.MaxPurls)
|
||||
{
|
||||
return Results.BadRequest($"purls limit exceeded (max {graphOptions.Value.MaxPurls})");
|
||||
}
|
||||
|
||||
var perPurlLimit = limitPerPurl.GetValueOrDefault(graphOptions.Value.MaxTooltipItemsPerPurl);
|
||||
if (perPurlLimit <= 0)
|
||||
{
|
||||
return Results.BadRequest("limitPerPurl must be greater than zero when provided.");
|
||||
}
|
||||
|
||||
var effectivePerPurlLimit = Math.Min(perPurlLimit, graphOptions.Value.MaxAdvisoriesPerPurl);
|
||||
var totalLimit = Math.Min(
|
||||
Math.Max(1, effectivePerPurlLimit * orderedPurls.Count),
|
||||
graphOptions.Value.MaxTooltipTotal);
|
||||
|
||||
var options = new VexObservationQueryOptions(
|
||||
tenant: tenant,
|
||||
purls: orderedPurls,
|
||||
limit: totalLimit,
|
||||
cursor: cursor);
|
||||
|
||||
VexObservationQueryResult result;
|
||||
try
|
||||
{
|
||||
result = await queryService.QueryAsync(options, cancellationToken).ConfigureAwait(false);
|
||||
}
|
||||
catch (FormatException ex)
|
||||
{
|
||||
return Results.BadRequest(ex.Message);
|
||||
}
|
||||
|
||||
var items = GraphTooltipFactory.Build(orderedPurls, result.Observations, includeJustifications, effectivePerPurlLimit);
|
||||
var response = new GraphTooltipResponse(items, result.NextCursor, result.HasMore);
|
||||
|
||||
return Results.Ok(response);
|
||||
}).WithName("GetGraphObservations");
|
||||
|
||||
app.MapPost("/ingest/vex", async (
|
||||
HttpContext context,
|
||||
VexIngestRequest request,
|
||||
|
||||
@@ -0,0 +1,115 @@
|
||||
using System.Threading;
|
||||
using System.Threading.Tasks;
|
||||
using MongoDB.Bson;
|
||||
using MongoDB.Driver;
|
||||
|
||||
namespace StellaOps.Excititor.Storage.Mongo.Migrations;
|
||||
|
||||
/// <summary>
|
||||
/// Adds a $jsonSchema validator to the raw VEX collection to enforce aggregation-only
|
||||
/// shape (immutable content hash + provenance fields).
|
||||
/// ValidationAction=warn keeps rollout safe while surfacing violations.
|
||||
/// </summary>
|
||||
internal sealed class VexRawSchemaMigration : IVexMongoMigration
|
||||
{
|
||||
public string Id => "20251125-vex-raw-json-schema";
|
||||
|
||||
public async ValueTask ExecuteAsync(IMongoDatabase database, CancellationToken cancellationToken)
|
||||
{
|
||||
ArgumentNullException.ThrowIfNull(database);
|
||||
|
||||
var exists = await CollectionExistsAsync(database, VexMongoCollectionNames.Raw, cancellationToken)
|
||||
.ConfigureAwait(false);
|
||||
|
||||
var validator = BuildValidator();
|
||||
|
||||
if (!exists)
|
||||
{
|
||||
await database.CreateCollectionAsync(
|
||||
VexMongoCollectionNames.Raw,
|
||||
new CreateCollectionOptions
|
||||
{
|
||||
Validator = validator,
|
||||
ValidationAction = DocumentValidationAction.Warn,
|
||||
ValidationLevel = DocumentValidationLevel.Moderate,
|
||||
},
|
||||
cancellationToken).ConfigureAwait(false);
|
||||
return;
|
||||
}
|
||||
|
||||
var command = new BsonDocument
|
||||
{
|
||||
{ "collMod", VexMongoCollectionNames.Raw },
|
||||
{ "validator", validator },
|
||||
{ "validationAction", "warn" },
|
||||
{ "validationLevel", "moderate" },
|
||||
};
|
||||
|
||||
await database.RunCommandAsync<BsonDocument>(command, cancellationToken: cancellationToken)
|
||||
.ConfigureAwait(false);
|
||||
}
|
||||
|
||||
private static async Task<bool> CollectionExistsAsync(
|
||||
IMongoDatabase database,
|
||||
string name,
|
||||
CancellationToken cancellationToken)
|
||||
{
|
||||
using var cursor = await database.ListCollectionNamesAsync(
|
||||
new ListCollectionNamesOptions { Filter = new BsonDocument("name", name) },
|
||||
cancellationToken).ConfigureAwait(false);
|
||||
return await cursor.AnyAsync(cancellationToken).ConfigureAwait(false);
|
||||
}
|
||||
|
||||
private static BsonDocument BuildValidator()
|
||||
{
|
||||
var properties = new BsonDocument
|
||||
{
|
||||
{ "_id", new BsonDocument { { "bsonType", "string" }, { "description", "digest" } } },
|
||||
{ "providerId", new BsonDocument { { "bsonType", "string" }, { "minLength", 1 } } },
|
||||
{ "format", new BsonDocument
|
||||
{
|
||||
{ "bsonType", "string" },
|
||||
{ "enum", new BsonArray { "csaf", "cyclonedx", "openvex" } }
|
||||
}
|
||||
},
|
||||
{ "sourceUri", new BsonDocument { { "bsonType", "string" }, { "minLength", 1 } } },
|
||||
{ "retrievedAt", new BsonDocument { { "bsonType", "date" } } },
|
||||
{ "digest", new BsonDocument { { "bsonType", "string" }, { "minLength", 32 } } },
|
||||
{ "content", new BsonDocument
|
||||
{
|
||||
{ "bsonType", new BsonArray { "binData", "string" } }
|
||||
}
|
||||
},
|
||||
{ "gridFsObjectId", new BsonDocument
|
||||
{
|
||||
{ "bsonType", new BsonArray { "objectId", "null", "string" } }
|
||||
}
|
||||
},
|
||||
{ "metadata", new BsonDocument
|
||||
{
|
||||
{ "bsonType", "object" },
|
||||
{ "additionalProperties", true },
|
||||
{ "patternProperties", new BsonDocument
|
||||
{
|
||||
{ ".*", new BsonDocument { { "bsonType", "string" } } }
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
return new BsonDocument
|
||||
{
|
||||
{
|
||||
"$jsonSchema",
|
||||
new BsonDocument
|
||||
{
|
||||
{ "bsonType", "object" },
|
||||
{ "required", new BsonArray { "_id", "providerId", "format", "sourceUri", "retrievedAt", "digest" } },
|
||||
{ "properties", properties },
|
||||
{ "additionalProperties", true }
|
||||
}
|
||||
}
|
||||
};
|
||||
}
|
||||
}
|
||||
@@ -61,6 +61,7 @@ public static class VexMongoServiceCollectionExtensions
|
||||
services.AddScoped<VexStatementBackfillService>();
|
||||
services.AddScoped<IVexObservationLookup, MongoVexObservationLookup>();
|
||||
services.AddSingleton<IVexMongoMigration, VexInitialIndexMigration>();
|
||||
services.AddSingleton<IVexMongoMigration, VexRawSchemaMigration>();
|
||||
services.AddSingleton<IVexMongoMigration, VexConsensusSignalsMigration>();
|
||||
services.AddSingleton<IVexMongoMigration, VexConsensusHoldMigration>();
|
||||
services.AddSingleton<IVexMongoMigration, VexObservationCollectionsMigration>();
|
||||
|
||||
@@ -0,0 +1,90 @@
|
||||
using System;
|
||||
using System.Collections.Immutable;
|
||||
using StellaOps.Excititor.Core;
|
||||
using StellaOps.Excititor.Core.Observations;
|
||||
using StellaOps.Excititor.WebService.Graph;
|
||||
using Xunit;
|
||||
|
||||
namespace StellaOps.Excititor.WebService.Tests;
|
||||
|
||||
public sealed class GraphStatusFactoryTests
|
||||
{
|
||||
[Fact]
|
||||
public void Build_ProjectsOverlaySummariesAndProvenance()
|
||||
{
|
||||
var now = DateTimeOffset.UtcNow;
|
||||
var observations = new[]
|
||||
{
|
||||
CreateObservation(
|
||||
providerId: "ubuntu",
|
||||
createdAt: now,
|
||||
purls: new[] { "pkg:rpm/redhat/openssl@1.1.1" },
|
||||
statements: new[]
|
||||
{
|
||||
new VexObservationStatement(
|
||||
vulnerabilityId: "CVE-2025-1001",
|
||||
productKey: "pkg:rpm/redhat/openssl@1.1.1",
|
||||
status: VexClaimStatus.NotAffected,
|
||||
lastObserved: now,
|
||||
justification: VexJustification.ComponentNotPresent,
|
||||
purl: "pkg:rpm/redhat/openssl@1.1.1")
|
||||
},
|
||||
contentHash: "hash-new"),
|
||||
CreateObservation(
|
||||
providerId: "oracle",
|
||||
createdAt: now.AddMinutes(-2),
|
||||
purls: new[] { "pkg:rpm/redhat/openssl@1.1.1" },
|
||||
statements: Array.Empty<VexObservationStatement>(),
|
||||
contentHash: "hash-old")
|
||||
};
|
||||
|
||||
var items = GraphStatusFactory.Build(
|
||||
orderedPurls: new[] { "pkg:rpm/redhat/openssl@1.1.1" },
|
||||
observations: observations);
|
||||
|
||||
var item = Assert.Single(items);
|
||||
Assert.Equal("pkg:rpm/redhat/openssl@1.1.1", item.Purl);
|
||||
Assert.Equal(0, item.Summary.Open);
|
||||
Assert.Equal(1, item.Summary.NotAffected);
|
||||
Assert.Equal(0, item.Summary.UnderInvestigation);
|
||||
Assert.Equal(1, item.Summary.NoStatement);
|
||||
Assert.Equal(now, item.LatestModifiedAt);
|
||||
Assert.Equal("hash-new", item.LastEvidenceHash);
|
||||
Assert.Equal(new[] { "oracle", "ubuntu" }, item.Sources);
|
||||
}
|
||||
|
||||
private static VexObservation CreateObservation(
|
||||
string providerId,
|
||||
DateTimeOffset createdAt,
|
||||
string[] purls,
|
||||
VexObservationStatement[] statements,
|
||||
string contentHash)
|
||||
{
|
||||
return new VexObservation(
|
||||
observationId: $"obs-{providerId}-{createdAt.ToUnixTimeMilliseconds()}",
|
||||
tenant: "tenant-a",
|
||||
providerId: providerId,
|
||||
streamId: "csaf",
|
||||
upstream: new VexObservationUpstream(
|
||||
upstreamId: Guid.NewGuid().ToString("N"),
|
||||
documentVersion: "1",
|
||||
fetchedAt: createdAt,
|
||||
receivedAt: createdAt,
|
||||
contentHash: contentHash,
|
||||
signature: new VexObservationSignature(present: true, format: "sig", keyId: null, signature: null)),
|
||||
statements: statements.ToImmutableArray(),
|
||||
content: new VexObservationContent(
|
||||
format: "csaf",
|
||||
specVersion: "1",
|
||||
raw: System.Text.Json.Nodes.JsonValue.Create("raw")!,
|
||||
metadata: ImmutableDictionary<string, string>.Empty),
|
||||
linkset: new VexObservationLinkset(
|
||||
aliases: Array.Empty<string>(),
|
||||
purls: purls,
|
||||
cpes: Array.Empty<string>(),
|
||||
references: Array.Empty<VexObservationReference>()),
|
||||
createdAt: createdAt,
|
||||
supersedes: ImmutableArray<string>.Empty,
|
||||
attributes: ImmutableDictionary<string, string>.Empty);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,137 @@
|
||||
using System;
|
||||
using System.Collections.Immutable;
|
||||
using System.Linq;
|
||||
using StellaOps.Excititor.Core;
|
||||
using StellaOps.Excititor.Core.Observations;
|
||||
using StellaOps.Excititor.WebService.Graph;
|
||||
using Xunit;
|
||||
|
||||
namespace StellaOps.Excititor.WebService.Tests;
|
||||
|
||||
public sealed class GraphTooltipFactoryTests
|
||||
{
|
||||
[Fact]
|
||||
public void Build_OrdersByNewestAndTruncatesPerPurl()
|
||||
{
|
||||
var now = DateTimeOffset.UtcNow;
|
||||
var observations = new[]
|
||||
{
|
||||
CreateObservation(
|
||||
providerId: "ubuntu",
|
||||
createdAt: now,
|
||||
purls: new[] { "pkg:rpm/openssl@1.1.1" },
|
||||
statements: new[]
|
||||
{
|
||||
new VexObservationStatement(
|
||||
vulnerabilityId: "CVE-2025-1000",
|
||||
productKey: "pkg:rpm/openssl@1.1.1",
|
||||
status: VexClaimStatus.NotAffected,
|
||||
lastObserved: now,
|
||||
justification: VexJustification.ComponentNotPresent,
|
||||
purl: "pkg:rpm/openssl@1.1.1")
|
||||
},
|
||||
contentHash: "hash-ubuntu"),
|
||||
CreateObservation(
|
||||
providerId: "redhat",
|
||||
createdAt: now.AddMinutes(-1),
|
||||
purls: new[] { "pkg:rpm/openssl@1.1.1" },
|
||||
statements: new[]
|
||||
{
|
||||
new VexObservationStatement(
|
||||
vulnerabilityId: "CVE-2025-1000",
|
||||
productKey: "pkg:rpm/openssl@1.1.1",
|
||||
status: VexClaimStatus.UnderInvestigation,
|
||||
lastObserved: now.AddMinutes(-1),
|
||||
justification: null,
|
||||
purl: "pkg:rpm/openssl@1.1.1")
|
||||
},
|
||||
contentHash: "hash-redhat")
|
||||
};
|
||||
|
||||
var items = GraphTooltipFactory.Build(
|
||||
orderedPurls: new[] { "pkg:rpm/openssl@1.1.1" },
|
||||
observations: observations,
|
||||
includeJustifications: true,
|
||||
maxItemsPerPurl: 1);
|
||||
|
||||
var item = Assert.Single(items);
|
||||
Assert.True(item.Truncated);
|
||||
var obs = Assert.Single(item.Observations);
|
||||
Assert.Equal("CVE-2025-1000", obs.AdvisoryId);
|
||||
Assert.Equal("notaffected", obs.Status);
|
||||
Assert.Equal("ComponentNotPresent", obs.Justification);
|
||||
Assert.Equal("ubuntu", obs.ProviderId);
|
||||
Assert.Equal("hash-ubuntu", obs.EvidenceHash);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Build_UsesLinksetPurlsWhenStatementMissing()
|
||||
{
|
||||
var now = DateTimeOffset.UtcNow;
|
||||
var observations = new[]
|
||||
{
|
||||
CreateObservation(
|
||||
providerId: "oracle",
|
||||
createdAt: now,
|
||||
purls: new[] { "pkg:rpm/httpd@2.4.0" },
|
||||
statements: new[]
|
||||
{
|
||||
new VexObservationStatement(
|
||||
vulnerabilityId: "CVE-2025-2000",
|
||||
productKey: "pkg:rpm/httpd@2.4.0",
|
||||
status: VexClaimStatus.Affected,
|
||||
lastObserved: now,
|
||||
justification: null,
|
||||
purl: null)
|
||||
},
|
||||
contentHash: "hash-oracle")
|
||||
};
|
||||
|
||||
var items = GraphTooltipFactory.Build(
|
||||
orderedPurls: new[] { "pkg:rpm/httpd@2.4.0" },
|
||||
observations: observations,
|
||||
includeJustifications: false,
|
||||
maxItemsPerPurl: 5);
|
||||
|
||||
var item = Assert.Single(items);
|
||||
var observation = Assert.Single(item.Observations);
|
||||
Assert.Null(observation.Justification);
|
||||
Assert.Equal("oracle", observation.ProviderId);
|
||||
Assert.Equal("CVE-2025-2000", observation.AdvisoryId);
|
||||
}
|
||||
|
||||
private static VexObservation CreateObservation(
|
||||
string providerId,
|
||||
DateTimeOffset createdAt,
|
||||
string[] purls,
|
||||
VexObservationStatement[] statements,
|
||||
string contentHash)
|
||||
{
|
||||
return new VexObservation(
|
||||
observationId: $"obs-{providerId}-{createdAt.ToUnixTimeMilliseconds()}",
|
||||
tenant: "tenant-a",
|
||||
providerId: providerId,
|
||||
streamId: "csaf",
|
||||
upstream: new VexObservationUpstream(
|
||||
upstreamId: Guid.NewGuid().ToString("N"),
|
||||
documentVersion: "1",
|
||||
fetchedAt: createdAt,
|
||||
receivedAt: createdAt,
|
||||
contentHash: contentHash,
|
||||
signature: new VexObservationSignature(present: true, format: "sig", keyId: null, signature: null)),
|
||||
statements: statements.ToImmutableArray(),
|
||||
content: new VexObservationContent(
|
||||
format: "csaf",
|
||||
specVersion: "1",
|
||||
raw: System.Text.Json.Nodes.JsonValue.Create("raw")!,
|
||||
metadata: ImmutableDictionary<string, string>.Empty),
|
||||
linkset: new VexObservationLinkset(
|
||||
aliases: Array.Empty<string>(),
|
||||
purls: purls,
|
||||
cpes: Array.Empty<string>(),
|
||||
references: Array.Empty<VexObservationReference>()),
|
||||
createdAt: createdAt,
|
||||
supersedes: ImmutableArray<string>.Empty,
|
||||
attributes: ImmutableDictionary<string, string>.Empty);
|
||||
}
|
||||
}
|
||||
@@ -37,6 +37,8 @@
|
||||
<Compile Include="TestServiceOverrides.cs" />
|
||||
<Compile Include="TestWebApplicationFactory.cs" />
|
||||
<Compile Include="GraphOverlayFactoryTests.cs" />
|
||||
<Compile Include="GraphStatusFactoryTests.cs" />
|
||||
<Compile Include="GraphTooltipFactoryTests.cs" />
|
||||
<Compile Include="AttestationVerifyEndpointTests.cs" />
|
||||
</ItemGroup>
|
||||
</Project>
|
||||
|
||||
@@ -9,4 +9,10 @@ namespace StellaOps.RiskEngine.Core.Services;
|
||||
public interface IRiskScoreResultStore
|
||||
{
|
||||
Task SaveAsync(RiskScoreResult result, CancellationToken cancellationToken);
|
||||
|
||||
/// <summary>
|
||||
/// Attempts to read a previously persisted result by job identifier.
|
||||
/// Implementations must be deterministic and side-effect free.
|
||||
/// </summary>
|
||||
bool TryGet(Guid jobId, out RiskScoreResult result);
|
||||
}
|
||||
|
||||
@@ -45,6 +45,17 @@ public sealed class RiskScoreQueue
|
||||
return channel.Writer.WriteAsync(job, cancellationToken);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Enqueues a request and returns the assigned job id for later retrieval.
|
||||
/// </summary>
|
||||
public async ValueTask<Guid> EnqueueWithIdAsync(ScoreRequest request, CancellationToken cancellationToken)
|
||||
{
|
||||
ArgumentNullException.ThrowIfNull(request);
|
||||
var job = new RiskScoreJob(Guid.NewGuid(), request);
|
||||
await channel.Writer.WriteAsync(job, cancellationToken).ConfigureAwait(false);
|
||||
return job.JobId;
|
||||
}
|
||||
|
||||
public ValueTask<RiskScoreJob> DequeueAsync(CancellationToken cancellationToken) =>
|
||||
channel.Reader.ReadAsync(cancellationToken);
|
||||
|
||||
|
||||
@@ -0,0 +1,119 @@
|
||||
using System.Net;
|
||||
using System.Net.Http.Json;
|
||||
using Microsoft.AspNetCore.Mvc.Testing;
|
||||
using StellaOps.RiskEngine.Core.Contracts;
|
||||
using StellaOps.RiskEngine.Core.Providers;
|
||||
using Xunit;
|
||||
|
||||
namespace StellaOps.RiskEngine.Tests;
|
||||
|
||||
public class RiskEngineApiTests : IClassFixture<WebApplicationFactory<Program>>
|
||||
{
|
||||
private readonly WebApplicationFactory<Program> factory;
|
||||
|
||||
public RiskEngineApiTests(WebApplicationFactory<Program> factory)
|
||||
{
|
||||
this.factory = factory.WithWebHostBuilder(_ => { });
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task Providers_ListsDefaultTransforms()
|
||||
{
|
||||
var client = factory.CreateClient();
|
||||
var ct = TestContext.Current.CancellationToken;
|
||||
|
||||
var response = await client.GetAsync("/risk-scores/providers", ct);
|
||||
response.EnsureSuccessStatusCode();
|
||||
|
||||
var payload = await response.Content.ReadFromJsonAsync<ProvidersResponse>(cancellationToken: ct);
|
||||
Assert.NotNull(payload);
|
||||
Assert.Contains(DefaultTransformsProvider.ProviderName, payload!.Providers);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task Job_SubmitAndRetrieve_PersistsResult()
|
||||
{
|
||||
var client = factory.CreateClient();
|
||||
var ct = TestContext.Current.CancellationToken;
|
||||
|
||||
var request = new ScoreRequest(DefaultTransformsProvider.ProviderName, "asset-1", new Dictionary<string, double>
|
||||
{
|
||||
["signal"] = 0.5
|
||||
});
|
||||
|
||||
var submit = await client.PostAsJsonAsync("/risk-scores/jobs", request, ct);
|
||||
Assert.Equal(HttpStatusCode.Accepted, submit.StatusCode);
|
||||
|
||||
var accepted = await submit.Content.ReadFromJsonAsync<JobAccepted>(cancellationToken: ct);
|
||||
Assert.NotNull(accepted);
|
||||
Assert.True(accepted!.Result.Success);
|
||||
|
||||
var fetched = await client.GetFromJsonAsync<RiskScoreResult>($"/risk-scores/jobs/{accepted.JobId}", ct);
|
||||
Assert.NotNull(fetched);
|
||||
Assert.Equal(accepted.JobId, fetched!.JobId);
|
||||
Assert.True(fetched.Success);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task Simulations_ReturnsBatch()
|
||||
{
|
||||
var client = factory.CreateClient();
|
||||
var ct = TestContext.Current.CancellationToken;
|
||||
|
||||
var requests = new[]
|
||||
{
|
||||
new ScoreRequest(DefaultTransformsProvider.ProviderName, "asset-a", new Dictionary<string, double>{{"a", 0.2}}),
|
||||
new ScoreRequest(DefaultTransformsProvider.ProviderName, "asset-b", new Dictionary<string, double>{{"b", 0.8}})
|
||||
};
|
||||
|
||||
var response = await client.PostAsJsonAsync("/risk-scores/simulations", requests, ct);
|
||||
response.EnsureSuccessStatusCode();
|
||||
|
||||
var payload = await response.Content.ReadFromJsonAsync<SimulationResponse>(cancellationToken: ct);
|
||||
Assert.NotNull(payload);
|
||||
Assert.Equal(2, payload!.Results.Count);
|
||||
Assert.All(payload.Results, r => Assert.True(r.Success));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task Simulations_Summary_ReturnsAggregatesAndTopMovers()
|
||||
{
|
||||
var client = factory.CreateClient();
|
||||
var ct = TestContext.Current.CancellationToken;
|
||||
|
||||
var requests = new[]
|
||||
{
|
||||
new ScoreRequest(DefaultTransformsProvider.ProviderName, "asset-high", new Dictionary<string, double>{{"s1", 1.0}}),
|
||||
new ScoreRequest(DefaultTransformsProvider.ProviderName, "asset-mid", new Dictionary<string, double>{{"s1", 0.5}}),
|
||||
new ScoreRequest(DefaultTransformsProvider.ProviderName, "asset-low", new Dictionary<string, double>{{"s1", 0.2}})
|
||||
};
|
||||
|
||||
var response = await client.PostAsJsonAsync("/risk-scores/simulations/summary", requests, ct);
|
||||
response.EnsureSuccessStatusCode();
|
||||
|
||||
var payload = await response.Content.ReadFromJsonAsync<SimulationSummaryResponse>(cancellationToken: ct);
|
||||
Assert.NotNull(payload);
|
||||
Assert.Equal(3, payload!.Results.Count);
|
||||
Assert.All(payload.Results, r => Assert.True(r.Success));
|
||||
|
||||
Assert.Equal(0.566667, Math.Round(payload.Summary.AverageScore, 6));
|
||||
Assert.Equal(0.2, payload.Summary.MinScore);
|
||||
Assert.Equal(1.0, payload.Summary.MaxScore);
|
||||
|
||||
Assert.Equal(3, payload.Summary.TopMovers.Count);
|
||||
Assert.Collection(payload.Summary.TopMovers,
|
||||
first =>
|
||||
{
|
||||
Assert.Equal("asset-high", first.Subject);
|
||||
Assert.Equal(1.0, first.Score);
|
||||
},
|
||||
second => Assert.Equal("asset-mid", second.Subject),
|
||||
third => Assert.Equal("asset-low", third.Subject));
|
||||
}
|
||||
|
||||
private sealed record ProvidersResponse(IReadOnlyList<string> Providers);
|
||||
private sealed record JobAccepted(Guid JobId, RiskScoreResult Result);
|
||||
private sealed record SimulationResponse(IReadOnlyList<RiskScoreResult> Results);
|
||||
private sealed record SimulationSummaryDto(double AverageScore, double MinScore, double MaxScore, IReadOnlyList<RiskScoreResult> TopMovers);
|
||||
private sealed record SimulationSummaryResponse(SimulationSummaryDto Summary, IReadOnlyList<RiskScoreResult> Results);
|
||||
}
|
||||
@@ -58,17 +58,10 @@
|
||||
|
||||
|
||||
|
||||
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.14.1"/>
|
||||
|
||||
|
||||
|
||||
|
||||
<PackageReference Include="xunit.v3" Version="3.0.0"/>
|
||||
|
||||
|
||||
|
||||
|
||||
<PackageReference Include="xunit.runner.visualstudio" Version="3.1.3"/>
|
||||
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.14.1"/>
|
||||
<PackageReference Include="Microsoft.AspNetCore.Mvc.Testing" Version="10.0.0-rc.2.25502.107"/>
|
||||
<PackageReference Include="xunit.v3" Version="3.0.0"/>
|
||||
<PackageReference Include="xunit.runner.visualstudio" Version="3.1.3"/>
|
||||
|
||||
|
||||
|
||||
@@ -116,12 +109,9 @@
|
||||
|
||||
|
||||
|
||||
<ProjectReference Include="..\StellaOps.RiskEngine.Core\StellaOps.RiskEngine.Core.csproj"/>
|
||||
|
||||
|
||||
|
||||
|
||||
<ProjectReference Include="..\StellaOps.RiskEngine.Infrastructure\StellaOps.RiskEngine.Infrastructure.csproj"/>
|
||||
<ProjectReference Include="..\StellaOps.RiskEngine.Core\StellaOps.RiskEngine.Core.csproj"/>
|
||||
<ProjectReference Include="..\StellaOps.RiskEngine.Infrastructure\StellaOps.RiskEngine.Infrastructure.csproj"/>
|
||||
<ProjectReference Include="..\StellaOps.RiskEngine.WebService\StellaOps.RiskEngine.WebService.csproj"/>
|
||||
|
||||
|
||||
|
||||
|
||||
@@ -1,3 +1,5 @@
|
||||
using Microsoft.AspNetCore.Mvc;
|
||||
using System.Linq;
|
||||
using StellaOps.RiskEngine.Core.Contracts;
|
||||
using StellaOps.RiskEngine.Core.Providers;
|
||||
using StellaOps.RiskEngine.Core.Services;
|
||||
@@ -32,51 +34,92 @@ app.MapGet("/risk-scores/providers", (IRiskScoreProviderRegistry registry) =>
|
||||
|
||||
app.MapPost("/risk-scores/jobs", async (
|
||||
ScoreRequest request,
|
||||
RiskScoreQueue queue,
|
||||
IRiskScoreProviderRegistry registry,
|
||||
IRiskScoreResultStore store,
|
||||
TimeProvider timeProvider,
|
||||
[FromServices] RiskScoreQueue queue,
|
||||
[FromServices] IRiskScoreProviderRegistry registry,
|
||||
[FromServices] IRiskScoreResultStore store,
|
||||
CancellationToken ct) =>
|
||||
{
|
||||
var job = new RiskScoreJob(Guid.NewGuid(), request);
|
||||
await queue.EnqueueAsync(job.Request, ct).ConfigureAwait(false);
|
||||
var worker = new RiskScoreWorker(queue, registry, store, timeProvider);
|
||||
var normalized = new ScoreRequest(
|
||||
request.Provider,
|
||||
request.Subject,
|
||||
request.Signals ?? new Dictionary<string, double>());
|
||||
|
||||
var jobId = await queue.EnqueueWithIdAsync(normalized, ct).ConfigureAwait(false);
|
||||
var worker = new RiskScoreWorker(queue, registry, store, TimeProvider.System);
|
||||
var result = await worker.ProcessNextAsync(ct).ConfigureAwait(false);
|
||||
return Results.Accepted($"/risk-scores/jobs/{job.JobId}", new { jobId = job.JobId, result });
|
||||
return Results.Accepted($"/risk-scores/jobs/{jobId}", new { jobId, result });
|
||||
});
|
||||
|
||||
app.MapGet("/risk-scores/jobs/{jobId:guid}", (Guid jobId, InMemoryRiskScoreResultStore store) =>
|
||||
app.MapGet("/risk-scores/jobs/{jobId:guid}", (
|
||||
Guid jobId,
|
||||
[FromServices] IRiskScoreResultStore store) =>
|
||||
store.TryGet(jobId, out var result)
|
||||
? Results.Ok(result)
|
||||
: Results.NotFound());
|
||||
|
||||
app.MapPost("/risk-scores/simulations", async (
|
||||
IReadOnlyCollection<ScoreRequest> requests,
|
||||
IRiskScoreProviderRegistry registry,
|
||||
TimeProvider timeProvider,
|
||||
[FromServices] IRiskScoreProviderRegistry registry,
|
||||
CancellationToken ct) =>
|
||||
{
|
||||
var results = await EvaluateAsync(requests, registry, ct).ConfigureAwait(false);
|
||||
return Results.Ok(new { results });
|
||||
});
|
||||
|
||||
app.MapPost("/risk-scores/simulations/summary", async (
|
||||
IReadOnlyCollection<ScoreRequest> requests,
|
||||
[FromServices] IRiskScoreProviderRegistry registry,
|
||||
CancellationToken ct) =>
|
||||
{
|
||||
var results = await EvaluateAsync(requests, registry, ct).ConfigureAwait(false);
|
||||
|
||||
var scores = results.Select(r => r.Score).ToArray();
|
||||
var summary = new
|
||||
{
|
||||
averageScore = scores.Length == 0 ? 0d : scores.Average(),
|
||||
minScore = scores.Length == 0 ? 0d : scores.Min(),
|
||||
maxScore = scores.Length == 0 ? 0d : scores.Max(),
|
||||
topMovers = results
|
||||
.OrderByDescending(r => r.Score)
|
||||
.ThenBy(r => r.Subject, StringComparer.Ordinal)
|
||||
.Take(3)
|
||||
.ToArray()
|
||||
};
|
||||
|
||||
return Results.Ok(new { summary, results });
|
||||
});
|
||||
|
||||
app.Run();
|
||||
|
||||
static async Task<List<RiskScoreResult>> EvaluateAsync(
|
||||
IReadOnlyCollection<ScoreRequest> requests,
|
||||
IRiskScoreProviderRegistry registry,
|
||||
CancellationToken ct)
|
||||
{
|
||||
var results = new List<RiskScoreResult>(requests.Count);
|
||||
foreach (var req in requests)
|
||||
{
|
||||
if (!registry.TryGet(req.Provider, out var provider))
|
||||
var normalized = new ScoreRequest(
|
||||
req.Provider,
|
||||
req.Subject,
|
||||
req.Signals ?? new Dictionary<string, double>());
|
||||
|
||||
if (!registry.TryGet(normalized.Provider, out var provider))
|
||||
{
|
||||
results.Add(new RiskScoreResult(Guid.NewGuid(), req.Provider, req.Subject, 0d, false, "Provider not registered", req.Signals, timeProvider.GetUtcNow()));
|
||||
results.Add(new RiskScoreResult(Guid.NewGuid(), normalized.Provider, normalized.Subject, 0d, false, "Provider not registered", normalized.Signals, TimeProvider.System.GetUtcNow()));
|
||||
continue;
|
||||
}
|
||||
|
||||
try
|
||||
{
|
||||
var score = await provider.ScoreAsync(req, ct).ConfigureAwait(false);
|
||||
results.Add(new RiskScoreResult(Guid.NewGuid(), req.Provider, req.Subject, score, true, null, req.Signals, timeProvider.GetUtcNow()));
|
||||
var score = await provider.ScoreAsync(normalized, ct).ConfigureAwait(false);
|
||||
results.Add(new RiskScoreResult(Guid.NewGuid(), normalized.Provider, normalized.Subject, score, true, null, normalized.Signals, TimeProvider.System.GetUtcNow()));
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
results.Add(new RiskScoreResult(Guid.NewGuid(), req.Provider, req.Subject, 0d, false, ex.Message, req.Signals, timeProvider.GetUtcNow()));
|
||||
results.Add(new RiskScoreResult(Guid.NewGuid(), normalized.Provider, normalized.Subject, 0d, false, ex.Message, normalized.Signals, TimeProvider.System.GetUtcNow()));
|
||||
}
|
||||
}
|
||||
|
||||
return Results.Ok(new { results });
|
||||
});
|
||||
|
||||
app.Run();
|
||||
return results;
|
||||
}
|
||||
|
||||
@@ -50,6 +50,13 @@ internal static class SampleData
|
||||
{
|
||||
new AdvisoryRef("https://example.com/advisory/0001", "Upstream advisory")
|
||||
},
|
||||
Rationale: new PolicyRationale("rat-0001", "High severity RCE with known exploit; fix available"),
|
||||
Paths: new[] { "/src/app/Program.cs", "/src/lib/utils/net.cs" },
|
||||
Evidence: new[]
|
||||
{
|
||||
new EvidenceRef("sbom", "sbom-0001", "Inventory evidence"),
|
||||
new EvidenceRef("vex", "vex-0001", "Vendor statement")
|
||||
},
|
||||
FirstSeen: DateTimeOffset.Parse("2025-01-01T00:00:00Z"),
|
||||
LastSeen: DateTimeOffset.Parse("2025-11-01T00:00:00Z"),
|
||||
PolicyVersion: summaries[0].PolicyVersion,
|
||||
@@ -70,6 +77,12 @@ internal static class SampleData
|
||||
new PackageAffect("pkg:npm/foo", new[] { "4.5.6" })
|
||||
},
|
||||
AdvisoryRefs: Array.Empty<AdvisoryRef>(),
|
||||
Rationale: new PolicyRationale("rat-0002", "Medium severity; no exploit observed; fix unavailable"),
|
||||
Paths: new[] { "/app/node_modules/foo/index.js" },
|
||||
Evidence: new[]
|
||||
{
|
||||
new EvidenceRef("sbom", "sbom-0002", "Inventory evidence")
|
||||
},
|
||||
FirstSeen: DateTimeOffset.Parse("2024-06-10T00:00:00Z"),
|
||||
LastSeen: DateTimeOffset.Parse("2025-08-15T00:00:00Z"),
|
||||
PolicyVersion: summaries[1].PolicyVersion,
|
||||
|
||||
@@ -24,6 +24,9 @@ public sealed record VulnDetail(
|
||||
string Summary,
|
||||
IReadOnlyList<PackageAffect> AffectedPackages,
|
||||
IReadOnlyList<AdvisoryRef> AdvisoryRefs,
|
||||
PolicyRationale Rationale,
|
||||
IReadOnlyList<string> Paths,
|
||||
IReadOnlyList<EvidenceRef> Evidence,
|
||||
DateTimeOffset FirstSeen,
|
||||
DateTimeOffset LastSeen,
|
||||
string PolicyVersion,
|
||||
@@ -34,6 +37,10 @@ public sealed record PackageAffect(string Purl, IReadOnlyList<string> Versions);
|
||||
|
||||
public sealed record AdvisoryRef(string Url, string Title);
|
||||
|
||||
public sealed record EvidenceRef(string Kind, string Reference, string? Title = null);
|
||||
|
||||
public sealed record EvidenceProvenance(string LedgerEntryId, string EvidenceBundleId);
|
||||
|
||||
public sealed record PolicyRationale(string Id, string Summary);
|
||||
|
||||
public sealed record VulnListResponse(IReadOnlyList<VulnSummary> Items, string? NextPageToken);
|
||||
|
||||
@@ -1,28 +1,36 @@
|
||||
using System.Collections.Generic;
|
||||
using Swashbuckle.AspNetCore.SwaggerGen;
|
||||
using System.Globalization;
|
||||
using Microsoft.AspNetCore.Mvc;
|
||||
using Microsoft.AspNetCore.Builder;
|
||||
using Microsoft.AspNetCore.OpenApi;
|
||||
using Microsoft.Extensions.DependencyInjection;
|
||||
using Microsoft.OpenApi.Models;
|
||||
using StellaOps.VulnExplorer.Api.Data;
|
||||
using StellaOps.VulnExplorer.Api.Models;
|
||||
|
||||
var builder = WebApplication.CreateBuilder(args);
|
||||
builder.Services.AddEndpointsApiExplorer();
|
||||
builder.Services.AddOpenApi();
|
||||
builder.Services.AddSwaggerGen(options =>
|
||||
{
|
||||
options.SwaggerDoc("v1", new OpenApiInfo
|
||||
{
|
||||
Title = "StellaOps Vuln Explorer API",
|
||||
Version = "v1",
|
||||
Description = "Deterministic vulnerability listing/detail endpoints"
|
||||
});
|
||||
});
|
||||
|
||||
var app = builder.Build();
|
||||
|
||||
if (app.Environment.IsDevelopment())
|
||||
{
|
||||
app.MapOpenApi();
|
||||
}
|
||||
app.UseSwagger();
|
||||
app.UseSwaggerUI();
|
||||
|
||||
app.UseHttpsRedirection();
|
||||
|
||||
app.MapGet("/v1/vulns", (
|
||||
HttpRequest request,
|
||||
[AsParameters] VulnFilter filter) =>
|
||||
app.MapGet("/v1/vulns", ([AsParameters] VulnFilter filter) =>
|
||||
{
|
||||
var tenant = request.Headers["x-stella-tenant"].ToString();
|
||||
if (string.IsNullOrWhiteSpace(tenant))
|
||||
if (string.IsNullOrWhiteSpace(filter.Tenant))
|
||||
{
|
||||
return Results.BadRequest(new { error = "x-stella-tenant required" });
|
||||
}
|
||||
@@ -40,9 +48,8 @@ app.MapGet("/v1/vulns", (
|
||||
})
|
||||
.WithOpenApi();
|
||||
|
||||
app.MapGet("/v1/vulns/{id}", (HttpRequest request, string id) =>
|
||||
app.MapGet("/v1/vulns/{id}", ([FromHeader(Name = "x-stella-tenant")] string? tenant, string id) =>
|
||||
{
|
||||
var tenant = request.Headers["x-stella-tenant"].ToString();
|
||||
if (string.IsNullOrWhiteSpace(tenant))
|
||||
{
|
||||
return Results.BadRequest(new { error = "x-stella-tenant required" });
|
||||
@@ -63,19 +70,19 @@ static IReadOnlyList<VulnSummary> ApplyFilter(IReadOnlyList<VulnSummary> source,
|
||||
{
|
||||
IEnumerable<VulnSummary> query = source;
|
||||
|
||||
if (filter.Cve?.Count > 0)
|
||||
if (filter.Cve is { Length: > 0 })
|
||||
{
|
||||
var set = filter.Cve.ToHashSet(StringComparer.OrdinalIgnoreCase);
|
||||
query = query.Where(v => v.CveIds.Any(set.Contains));
|
||||
}
|
||||
|
||||
if (filter.Purl?.Count > 0)
|
||||
if (filter.Purl is { Length: > 0 })
|
||||
{
|
||||
var set = filter.Purl.ToHashSet(StringComparer.OrdinalIgnoreCase);
|
||||
query = query.Where(v => v.Purls.Any(set.Contains));
|
||||
}
|
||||
|
||||
if (filter.Severity?.Count > 0)
|
||||
if (filter.Severity is { Length: > 0 })
|
||||
{
|
||||
var set = filter.Severity.ToHashSet(StringComparer.OrdinalIgnoreCase);
|
||||
query = query.Where(v => set.Contains(v.Severity));
|
||||
@@ -100,13 +107,13 @@ static IReadOnlyList<VulnSummary> ApplyFilter(IReadOnlyList<VulnSummary> source,
|
||||
}
|
||||
|
||||
public record VulnFilter(
|
||||
[FromHeader(Name = "x-stella-tenant")] string Tenant,
|
||||
[FromHeader(Name = "x-stella-tenant")] string? Tenant,
|
||||
[FromQuery(Name = "policyVersion")] string? PolicyVersion,
|
||||
[FromQuery(Name = "pageSize")] int? PageSize,
|
||||
[FromQuery(Name = "pageToken")] string? PageToken,
|
||||
[FromQuery(Name = "cve")] IReadOnlyList<string>? Cve,
|
||||
[FromQuery(Name = "purl")] IReadOnlyList<string>? Purl,
|
||||
[FromQuery(Name = "severity")] IReadOnlyList<string>? Severity,
|
||||
[FromQuery(Name = "cve")] string[]? Cve,
|
||||
[FromQuery(Name = "purl")] string[]? Purl,
|
||||
[FromQuery(Name = "severity")] string[]? Severity,
|
||||
[FromQuery(Name = "exploitability")] string? Exploitability,
|
||||
[FromQuery(Name = "fixAvailable")] bool? FixAvailable);
|
||||
|
||||
|
||||
@@ -10,5 +10,6 @@
|
||||
</PropertyGroup>
|
||||
<ItemGroup>
|
||||
<PackageReference Include="Microsoft.AspNetCore.OpenApi" Version="8.0.8" />
|
||||
<PackageReference Include="Swashbuckle.AspNetCore" Version="6.6.2" />
|
||||
</ItemGroup>
|
||||
</Project>
|
||||
|
||||
7
src/VulnExplorer/StellaOps.VulnExplorer.Api/TASKS.md
Normal file
7
src/VulnExplorer/StellaOps.VulnExplorer.Api/TASKS.md
Normal file
@@ -0,0 +1,7 @@
|
||||
# Vuln Explorer API Tasks (Sprint 0129-0001-0001)
|
||||
|
||||
| Task ID | Status | Notes |
|
||||
| --- | --- | --- |
|
||||
| VULN-API-29-001 | DONE (2025-11-25) | OpenAPI v1 draft published at `docs/modules/vuln-explorer/openapi/vuln-explorer.v1.yaml` with tenant header, filters, deterministic paging. |
|
||||
| VULN-API-29-002 | DONE (2025-11-25) | Implemented `/v1/vulns` list + `/v1/vulns/{id}` detail with deterministic paging/filtering, sample data, Swagger UI; tests green (`tests/TestResults/vuln-explorer/api.trx`). |
|
||||
| VULN-API-29-003 | DONE (2025-11-25) | Detail endpoint now returns rationale, paths, evidence references; covered by Vuln Explorer API integration tests. |
|
||||
Reference in New Issue
Block a user