up
Some checks failed
Docs CI / lint-and-preview (push) Has been cancelled
Signals CI & Image / signals-ci (push) Has been cancelled
Policy Lint & Smoke / policy-lint (push) Has been cancelled
Policy Simulation / policy-simulate (push) Has been cancelled
SDK Publish & Sign / sdk-publish (push) Has been cancelled
AOC Guard CI / aoc-guard (push) Has been cancelled
AOC Guard CI / aoc-verify (push) Has been cancelled
Concelier Attestation Tests / attestation-tests (push) Has been cancelled
devportal-offline / build-offline (push) Has been cancelled

This commit is contained in:
StellaOps Bot
2025-11-25 22:09:44 +02:00
parent 6bee1fdcf5
commit 9f6e6f7fb3
116 changed files with 4495 additions and 730 deletions

42
src/AirGap/AGENTS.md Normal file
View File

@@ -0,0 +1,42 @@
# AirGap Module · AGENTS Charter
## Working Directory
- `src/AirGap/**` (Controller, Importer, Time). Do not edit other modules without sprint note.
## Roles
- **Controller engineer (ASP.NET Core)**: seal/unseal state machine, status APIs, Authority scope enforcement.
- **Importer engineer**: bundle verification (TUF/DSSE), catalog repositories, object-store loaders.
- **Time engineer**: time anchor parsing/verification (Roughtime, RFC3161), staleness calculators.
- **QA/Automation**: API + storage tests (Mongo2Go/in-memory), deterministic ordering, sealed/offline paths.
- **Docs/Runbooks**: keep air-gap ops guides, scaffolds, and schemas aligned with behavior.
## Required Reading (treat as read before DOING)
- `docs/README.md`, `docs/07_HIGH_LEVEL_ARCHITECTURE.md`, `docs/modules/platform/architecture-overview.md`
- `docs/modules/airgap/airgap-mode.md` (if present)
- Prep/Scaffold docs:
- `docs/airgap/controller-scaffold.md`
- `docs/airgap/prep/2025-11-20-controller-scaffold-prep.md`
- `docs/airgap/importer-scaffold.md`
- `docs/airgap/time-anchor-scaffold.md`
- `docs/airgap/prep/2025-11-20-staleness-drift-prep.md`
- `docs/airgap/sealed-startup-diagnostics.md`
- `docs/airgap/bundle-repositories.md`
- `docs/airgap/time-api.md`, `docs/airgap/time-config-sample.json`
## Working Agreements
- Offline-first: no egress in sealed mode; fixtures use local files only.
- Determinism: stable ordering, UTC ISO-8601 timestamps, fixed seeds for tests, deterministic hashing.
- Tenancy/scopes: enforce Authority scopes (`airgap:seal`, `airgap:status:read`, importer scopes) on every API.
- Validation: prefer `$jsonSchema`/FluentValidation; fail closed on trust-root mismatch.
- Logging/Telemetry: structured logs; counters/histograms prefixed `airgap.*`; tag `tenant`, `sealed`, `result`.
- Cross-module edits require sprint note; otherwise stay within `src/AirGap`.
## Testing Rules
- Use Mongo2Go/in-memory stores; no network.
- Cover sealed/unsealed transitions, staleness budgets, trust-root failures, deterministic ordering.
- API tests via WebApplicationFactory; importer tests use local fixture bundles (no downloads).
## Delivery Discipline
- Update sprint tracker statuses (`TODO → DOING → DONE/BLOCKED`); log decisions in Execution Log and Decisions & Risks.
- When contracts/schemas change, update docs under `docs/airgap/**` and link from sprint Decisions & Risks.
- If a decision is needed, mark BLOCKED in the sprint and record the decision ask; continue with other unblocked work.

View File

@@ -1,4 +1,5 @@
using System.Text.Json.Serialization;
using StellaOps.Concelier.Core.Attestation;
namespace StellaOps.Concelier.WebService;

View File

@@ -19,8 +19,6 @@ using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.DependencyInjection.Extensions;
using Microsoft.Extensions.Hosting;
using System.Diagnostics;
using System.Text.Json;
using System.Text.Json.Serialization;
using Microsoft.Extensions.Logging;
using Microsoft.Extensions.Options;
using Microsoft.Extensions.Primitives;
@@ -32,11 +30,11 @@ using StellaOps.Concelier.Core.Observations;
using StellaOps.Concelier.Core.Linksets;
using StellaOps.Concelier.Models;
using StellaOps.Concelier.WebService.Diagnostics;
using ServiceStatus = StellaOps.Concelier.WebService.Diagnostics.ServiceStatus;
using Serilog;
using StellaOps.Concelier.Merge;
using StellaOps.Concelier.Merge.Services;
using StellaOps.Concelier.WebService.Extensions;
using StellaOps.Concelier.WebService.Services;
using StellaOps.Concelier.WebService.Jobs;
using StellaOps.Concelier.WebService.Options;
using StellaOps.Concelier.WebService.Filters;
@@ -61,8 +59,8 @@ using StellaOps.Concelier.Storage.Mongo.Advisories;
using StellaOps.Concelier.Storage.Mongo.Aliases;
using StellaOps.Provenance.Mongo;
using StellaOps.Concelier.Core.Attestation;
using AttestationClaims = StellaOps.Concelier.Core.Attestation.AttestationClaims;
using StellaOps.Concelier.Storage.Mongo.Orchestrator;
using System.Security.Cryptography;
using System.Diagnostics.Metrics;
using StellaOps.Concelier.Models.Observations;
@@ -117,28 +115,33 @@ var contentRootPath = builder.Environment.ContentRootPath;
ConcelierOptions concelierOptions;
if (builder.Environment.IsEnvironment("Testing"))
{
// Allow a fully pre-bound options instance to be supplied by the test host.
#pragma warning disable ASP0000 // test-only: create provider to fetch pre-bound options
using var tempProvider = builder.Services.BuildServiceProvider();
#pragma warning restore ASP0000
concelierOptions = tempProvider.GetService<IOptions<ConcelierOptions>>()?.Value ?? new ConcelierOptions
{
Storage = new ConcelierOptions.StorageOptions
// Allow a fully pre-bound options instance to be supplied by the test host.
#pragma warning disable ASP0000 // test-only: create provider to fetch pre-bound options
using var tempProvider = builder.Services.BuildServiceProvider();
#pragma warning restore ASP0000
concelierOptions = tempProvider.GetService<IOptions<ConcelierOptions>>()?.Value ?? new ConcelierOptions
{
Dsn = Environment.GetEnvironmentVariable("CONCELIER_TEST_STORAGE_DSN") ?? "mongodb://localhost:27017/test-health",
Driver = "mongo",
CommandTimeoutSeconds = 30
},
Telemetry = new ConcelierOptions.TelemetryOptions
{
Enabled = false
}
};
Storage = new ConcelierOptions.StorageOptions
{
Dsn = Environment.GetEnvironmentVariable("CONCELIER_TEST_STORAGE_DSN") ?? "mongodb://localhost:27017/test-health",
Driver = "mongo",
CommandTimeoutSeconds = 30
},
Telemetry = new ConcelierOptions.TelemetryOptions
{
Enabled = false
}
};
ConcelierOptionsPostConfigure.Apply(concelierOptions, contentRootPath);
// Skip validation in Testing to allow factory-provided wiring.
}
concelierOptions.Storage ??= new ConcelierOptions.StorageOptions();
concelierOptions.Storage.Dsn = Environment.GetEnvironmentVariable("CONCELIER_TEST_STORAGE_DSN") ?? "mongodb://localhost:27017/orch-tests";
concelierOptions.Storage.Driver = "mongo";
concelierOptions.Storage.CommandTimeoutSeconds = concelierOptions.Storage.CommandTimeoutSeconds <= 0 ? 30 : concelierOptions.Storage.CommandTimeoutSeconds;
ConcelierOptionsPostConfigure.Apply(concelierOptions, contentRootPath);
// Skip validation in Testing to allow factory-provided wiring.
}
else
{
concelierOptions = builder.Configuration.BindOptions<ConcelierOptions>(postConfigure: (opts, _) =>
@@ -171,12 +174,27 @@ builder.Services.AddMemoryCache();
builder.Services.AddSingleton<MirrorRateLimiter>();
builder.Services.AddSingleton<MirrorFileLocator>();
builder.Services.AddMongoStorage(storageOptions =>
var isTesting = builder.Environment.IsEnvironment("Testing");
var mongoBypass = isTesting || string.Equals(
Environment.GetEnvironmentVariable("CONCELIER_BYPASS_MONGO"),
"1",
StringComparison.OrdinalIgnoreCase);
if (!isTesting)
{
storageOptions.ConnectionString = concelierOptions.Storage.Dsn;
storageOptions.DatabaseName = concelierOptions.Storage.Database;
storageOptions.CommandTimeout = TimeSpan.FromSeconds(concelierOptions.Storage.CommandTimeoutSeconds);
});
builder.Services.AddMongoStorage(storageOptions =>
{
storageOptions.ConnectionString = concelierOptions.Storage.Dsn;
storageOptions.DatabaseName = concelierOptions.Storage.Database;
storageOptions.CommandTimeout = TimeSpan.FromSeconds(concelierOptions.Storage.CommandTimeoutSeconds);
});
}
else
{
// In test host we entirely bypass Mongo validation/bootstrapping; tests inject fakes.
builder.Services.RemoveAll<IMongoClient>();
builder.Services.RemoveAll<IMongoDatabase>();
}
builder.Services.AddOptions<AdvisoryObservationEventPublisherOptions>()
.Bind(builder.Configuration.GetSection("advisoryObservationEvents"))
.PostConfigure(options =>
@@ -229,7 +247,7 @@ builder.Services.PostConfigure<JobSchedulerOptions>(options =>
});
builder.Services.AddSingleton<OpenApiDiscoveryDocumentProvider>();
builder.Services.AddSingleton<ServiceStatus>(sp => new ServiceStatus(sp.GetRequiredService<TimeProvider>()));
builder.Services.AddSingleton<StellaOps.Concelier.WebService.Diagnostics.ServiceStatus>(sp => new StellaOps.Concelier.WebService.Diagnostics.ServiceStatus(sp.GetRequiredService<TimeProvider>()));
builder.Services.AddAocGuard();
var authorityConfigured = concelierOptions.Authority is { Enabled: true };
@@ -743,7 +761,7 @@ app.MapGet("/v1/lnm/linksets", async (
resolvedPageSize,
cancellationToken).ConfigureAwait(false);
var items = new List<LnmLinksetResponse>(result.Items.Length);
var items = new List<LnmLinksetResponse>(result.Items.Count);
foreach (var linkset in result.Items)
{
var summary = await BuildObservationSummaryAsync(observationQueryService, tenant!, linkset, cancellationToken).ConfigureAwait(false);
@@ -788,7 +806,7 @@ app.MapPost("/v1/lnm/linksets/search", async (
resolvedPageSize,
cancellationToken).ConfigureAwait(false);
var items = new List<LnmLinksetResponse>(result.Items.Length);
var items = new List<LnmLinksetResponse>(result.Items.Count);
foreach (var linkset in result.Items)
{
var summary = await BuildObservationSummaryAsync(observationQueryService, tenant!, linkset, cancellationToken).ConfigureAwait(false);
@@ -1346,7 +1364,7 @@ var evidenceSnapshotEndpoint = app.MapGet("/obs/evidence/advisories/{advisoryKey
var hash = await ComputeSha256Async(manifestStream, cancellationToken).ConfigureAwait(false);
var response = new EvidenceSnapshotResponse(
advisoryKey: advisoryKey.Trim(),
AdvisoryKey: advisoryKey.Trim(),
Tenant: tenant,
ManifestPath: manifestPath,
ManifestHash: hash,
@@ -2241,33 +2259,8 @@ async Task<LinksetObservationSummary> BuildObservationSummaryAsync(
return LinksetObservationSummary.Empty;
}
var published = result.Observations
.Where(o => o.Published.HasValue)
.Select(o => o.Published!.Value)
.OrderBy(p => p)
.FirstOrDefault();
var modified = result.Observations
.Where(o => o.Modified.HasValue)
.Select(o => o.Modified!.Value)
.OrderByDescending(p => p)
.FirstOrDefault();
var severity = result.Observations
.SelectMany(o => o.Severities)
.OrderByDescending(s => s.Score)
.FirstOrDefault();
var severityText = severity is null ? null : $"{severity.System}:{severity.Score:0.0}";
var evidenceHash = result.Observations
.Select(o => o.Provenance.SourceArtifactSha)
.FirstOrDefault();
return new LinksetObservationSummary(
PublishedAt: published == default ? null : published,
ModifiedAt: modified == default ? null : modified,
Severity: severityText,
EvidenceHash: evidenceHash);
// Observation timelines are not yet populated; return empty summary until ingestion enriches these fields.
return LinksetObservationSummary.Empty;
}
IReadOnlyList<LnmLinksetTimeline> BuildTimeline(AdvisoryLinkset linkset, LinksetObservationSummary summary)
@@ -2290,15 +2283,6 @@ IReadOnlyList<LnmLinksetTimeline> BuildTimeline(AdvisoryLinkset linkset, Linkset
return timeline;
}
readonly record struct LinksetObservationSummary(
DateTimeOffset? PublishedAt,
DateTimeOffset? ModifiedAt,
string? Severity,
string? EvidenceHash)
{
public static LinksetObservationSummary Empty { get; } = new(null, null, null, null);
}
IResult JsonResult<T>(T value, int? statusCode = null)
{
var payload = JsonSerializer.Serialize(value, JsonOptions);
@@ -2834,7 +2818,7 @@ void ApplyNoCache(HttpResponse response)
await InitializeMongoAsync(app);
app.MapGet("/health", ([FromServices] IOptions<ConcelierOptions> opts, [FromServices] ServiceStatus status, HttpContext context) =>
app.MapGet("/health", ([FromServices] IOptions<ConcelierOptions> opts, [FromServices] StellaOps.Concelier.WebService.Diagnostics.ServiceStatus status, HttpContext context) =>
{
ApplyNoCache(context.Response);
@@ -2863,7 +2847,7 @@ app.MapGet("/health", ([FromServices] IOptions<ConcelierOptions> opts, [FromServ
return JsonResult(response);
});
app.MapGet("/ready", async ([FromServices] IMongoDatabase database, [FromServices] ServiceStatus status, HttpContext context, CancellationToken cancellationToken) =>
app.MapGet("/ready", async ([FromServices] IMongoDatabase database, [FromServices] StellaOps.Concelier.WebService.Diagnostics.ServiceStatus status, HttpContext context, CancellationToken cancellationToken) =>
{
ApplyNoCache(context.Response);
@@ -3252,13 +3236,23 @@ var concelierTimelineEndpoint = app.MapGet("/obs/concelier/timeline", async (
await app.RunAsync();
}
static JsonSerializerOptions CreateJsonOptions()
static JsonSerializerOptions CreateJsonOptions()
{
var options = new JsonSerializerOptions(JsonSerializerDefaults.Web);
options.Converters.Add(new JsonStringEnumConverter());
return options;
}
// Linkset summary used by advisory summary timeline
private readonly record struct LinksetObservationSummary(
DateTimeOffset? PublishedAt,
DateTimeOffset? ModifiedAt,
string? Severity,
string? EvidenceHash)
{
public static LinksetObservationSummary Empty { get; } = new(null, null, null, null);
}
static PluginHostOptions BuildPluginOptions(ConcelierOptions options, string contentRoot)
{
var pluginOptions = new PluginHostOptions
@@ -3293,7 +3287,7 @@ static async Task InitializeMongoAsync(WebApplication app)
await using var scope = app.Services.CreateAsyncScope();
var bootstrapper = scope.ServiceProvider.GetRequiredService<MongoBootstrapper>();
var logger = scope.ServiceProvider.GetRequiredService<ILoggerFactory>().CreateLogger("MongoBootstrapper");
var status = scope.ServiceProvider.GetRequiredService<ServiceStatus>();
var status = scope.ServiceProvider.GetRequiredService<StellaOps.Concelier.WebService.Diagnostics.ServiceStatus>();
var stopwatch = Stopwatch.StartNew();

View File

@@ -1,4 +1,5 @@
using System.Text.Json;
using StellaOps.Concelier.WebService.Options;
namespace StellaOps.Concelier.WebService.Services;

View File

@@ -66,12 +66,36 @@ public sealed class MongoStorageOptions
return MongoStorageDefaults.DefaultDatabaseName;
}
public void EnsureValid()
{
if (string.IsNullOrWhiteSpace(ConnectionString))
{
throw new InvalidOperationException("Mongo connection string is not configured.");
}
public void EnsureValid()
{
var isTesting = string.Equals(
Environment.GetEnvironmentVariable("DOTNET_ENVIRONMENT"),
"Testing",
StringComparison.OrdinalIgnoreCase);
var bypass = string.Equals(
Environment.GetEnvironmentVariable("CONCELIER_BYPASS_MONGO"),
"1",
StringComparison.OrdinalIgnoreCase);
if (isTesting || bypass)
{
// Under test, skip validation entirely; callers may stub Mongo.
return;
}
if (string.IsNullOrWhiteSpace(ConnectionString))
{
var fallback = Environment.GetEnvironmentVariable("CONCELIER_TEST_STORAGE_DSN");
if (!string.IsNullOrWhiteSpace(fallback))
{
ConnectionString = fallback;
}
}
if (string.IsNullOrWhiteSpace(ConnectionString))
{
throw new InvalidOperationException("Mongo connection string is not configured.");
}
if (CommandTimeout <= TimeSpan.Zero)
{

View File

@@ -29,14 +29,32 @@ namespace StellaOps.Concelier.Storage.Mongo;
public static class ServiceCollectionExtensions
{
public static IServiceCollection AddMongoStorage(this IServiceCollection services, Action<MongoStorageOptions> configureOptions)
{
ArgumentNullException.ThrowIfNull(services);
ArgumentNullException.ThrowIfNull(configureOptions);
services.AddOptions<MongoStorageOptions>()
.Configure(configureOptions)
.PostConfigure(static options => options.EnsureValid());
public static IServiceCollection AddMongoStorage(this IServiceCollection services, Action<MongoStorageOptions> configureOptions)
{
ArgumentNullException.ThrowIfNull(services);
ArgumentNullException.ThrowIfNull(configureOptions);
// In unit/integration tests we bypass Mongo wiring entirely; callers may inject fakes.
var isTesting = string.Equals(
Environment.GetEnvironmentVariable("DOTNET_ENVIRONMENT"),
"Testing",
StringComparison.OrdinalIgnoreCase);
var bypass = string.Equals(
Environment.GetEnvironmentVariable("CONCELIER_BYPASS_MONGO"),
"1",
StringComparison.OrdinalIgnoreCase);
if (isTesting || bypass)
{
return services;
}
services.AddOptions<MongoStorageOptions>()
.Configure(configureOptions)
.PostConfigure(static options =>
{
// Normal path: enforce validity.
options.EnsureValid();
});
services.TryAddSingleton(TimeProvider.System);

View File

@@ -0,0 +1,196 @@
using System;
using System.Net;
using System.Collections.Generic;
using System.Net.Http.Json;
using System.Threading.Tasks;
using FluentAssertions;
using Microsoft.AspNetCore.Hosting;
using Microsoft.AspNetCore.Mvc.Testing;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.DependencyInjection.Extensions;
using Microsoft.Extensions.Options;
using StellaOps.Concelier.Storage.Mongo;
using StellaOps.Concelier.Storage.Mongo.Orchestrator;
using StellaOps.Concelier.WebService;
using StellaOps.Concelier.WebService.Options;
using Xunit;
namespace StellaOps.Concelier.WebService.Tests;
public sealed class OrchestratorTestWebAppFactory : WebApplicationFactory<Program>
{
public OrchestratorTestWebAppFactory()
{
Environment.SetEnvironmentVariable("CONCELIER__STORAGE__DSN", "mongodb://localhost:27017/orch-tests");
Environment.SetEnvironmentVariable("CONCELIER__STORAGE__DRIVER", "mongo");
Environment.SetEnvironmentVariable("CONCELIER__STORAGE__COMMANDTIMEOUTSECONDS", "30");
Environment.SetEnvironmentVariable("CONCELIER__TELEMETRY__ENABLED", "false");
Environment.SetEnvironmentVariable("CONCELIER__AUTHORITY__ENABLED", "false"); // disable auth so tests can hit endpoints without tokens
Environment.SetEnvironmentVariable("CONCELIER_SKIP_OPTIONS_VALIDATION", "1");
Environment.SetEnvironmentVariable("CONCELIER_TEST_STORAGE_DSN", "mongodb://localhost:27017/orch-tests");
Environment.SetEnvironmentVariable("CONCELIER_BYPASS_MONGO", "1");
Environment.SetEnvironmentVariable("DOTNET_ENVIRONMENT", "Testing");
Environment.SetEnvironmentVariable("ASPNETCORE_ENVIRONMENT", "Testing");
}
protected override void ConfigureWebHost(IWebHostBuilder builder)
{
builder.UseEnvironment("Testing");
builder.ConfigureAppConfiguration(cfg =>
{
cfg.AddInMemoryCollection(new Dictionary<string, string?>
{
["Concelier:Storage:Dsn"] = "mongodb://localhost:27017/orch-tests",
["Concelier:Storage:Driver"] = "mongo",
["Concelier:Storage:CommandTimeoutSeconds"] = "30",
["Concelier:Telemetry:Enabled"] = "false",
["Concelier:Authority:Enabled"] = "false"
});
});
builder.ConfigureServices(services =>
{
services.RemoveAll<IOrchestratorRegistryStore>();
services.AddSingleton<IOrchestratorRegistryStore, InMemoryOrchestratorStore>();
// Pre-bind options to keep Program from trying to rebind/validate during tests.
services.RemoveAll<ConcelierOptions>();
services.RemoveAll<IOptions<ConcelierOptions>>();
var forcedOptions = new ConcelierOptions
{
Storage = new ConcelierOptions.StorageOptions
{
Dsn = "mongodb://localhost:27017/orch-tests",
Driver = "mongo",
CommandTimeoutSeconds = 30
},
Telemetry = new ConcelierOptions.TelemetryOptions
{
Enabled = false
},
Authority = new ConcelierOptions.AuthorityOptions
{
Enabled = false
}
};
services.AddSingleton(forcedOptions);
services.AddSingleton<IOptions<ConcelierOptions>>(_ => Microsoft.Extensions.Options.Options.Create(forcedOptions));
// Force Mongo storage options to a deterministic in-memory test DSN.
services.PostConfigure<MongoStorageOptions>(opts =>
{
opts.ConnectionString = "mongodb://localhost:27017/orch-tests";
opts.DatabaseName = "orch-tests";
opts.CommandTimeout = TimeSpan.FromSeconds(30);
});
});
}
}
public sealed class OrchestratorEndpointsTests : IClassFixture<OrchestratorTestWebAppFactory>
{
private readonly OrchestratorTestWebAppFactory _factory;
public OrchestratorEndpointsTests(OrchestratorTestWebAppFactory factory) => _factory = factory;
[Fact]
public async Task Registry_accepts_valid_request_with_tenant()
{
var client = _factory.CreateClient();
client.DefaultRequestHeaders.Add("X-Stella-Tenant", "tenant-a");
var request = new
{
connectorId = "demo-connector",
source = "demo-source",
capabilities = new[] { "ingest" },
authRef = "secret",
schedule = new { cron = "0 0 * * *", timeZone = "UTC", maxParallelRuns = 1, maxLagMinutes = 5 },
ratePolicy = new { rpm = 100, burst = 10, cooldownSeconds = 5 },
artifactKinds = new[] { "advisory" },
lockKey = "lk-1",
egressGuard = new { allowlist = new[] { "example.com" }, airgapMode = false }
};
var response = await client.PostAsJsonAsync("/internal/orch/registry", request);
response.StatusCode.Should().Be(HttpStatusCode.Accepted);
}
[Fact]
public async Task Heartbeat_accepts_valid_request_with_tenant()
{
var client = _factory.CreateClient();
client.DefaultRequestHeaders.Add("X-Stella-Tenant", "tenant-a");
var request = new
{
connectorId = "demo-connector",
runId = "11111111-1111-1111-1111-111111111111",
sequence = 1,
status = "running",
progress = 10,
queueDepth = 0
};
var response = await client.PostAsJsonAsync("/internal/orch/heartbeat", request);
response.StatusCode.Should().Be(HttpStatusCode.Accepted);
}
[Fact]
public async Task Commands_get_returns_ok_with_empty_list()
{
var client = _factory.CreateClient();
client.DefaultRequestHeaders.Add("X-Stella-Tenant", "tenant-a");
var response = await client.GetAsync("/internal/orch/commands?connectorId=demo-connector&runId=11111111-1111-1111-1111-111111111111");
response.StatusCode.Should().Be(HttpStatusCode.OK);
var payload = await response.Content.ReadFromJsonAsync<List<OrchestratorCommandRecord>>();
payload.Should().NotBeNull();
payload!.Should().BeEmpty();
}
}
internal sealed class InMemoryOrchestratorStore : IOrchestratorRegistryStore
{
private readonly Dictionary<(string Tenant, string ConnectorId), OrchestratorRegistryRecord> _registry = new();
private readonly List<OrchestratorHeartbeatRecord> _heartbeats = new();
private readonly List<OrchestratorCommandRecord> _commands = new();
public Task UpsertAsync(OrchestratorRegistryRecord record, CancellationToken cancellationToken)
{
_registry[(record.Tenant, record.ConnectorId)] = record;
return Task.CompletedTask;
}
public Task<OrchestratorRegistryRecord?> GetAsync(string tenant, string connectorId, CancellationToken cancellationToken)
{
_registry.TryGetValue((tenant, connectorId), out var record);
return Task.FromResult(record);
}
public Task EnqueueCommandAsync(OrchestratorCommandRecord command, CancellationToken cancellationToken)
{
_commands.Add(command);
return Task.CompletedTask;
}
public Task<IReadOnlyList<OrchestratorCommandRecord>> GetPendingCommandsAsync(string tenant, string connectorId, Guid runId, long? afterSequence, CancellationToken cancellationToken)
{
var items = _commands
.Where(c => c.Tenant == tenant && c.ConnectorId == connectorId && c.RunId == runId && (afterSequence is null || c.Sequence > afterSequence))
.ToList()
.AsReadOnly();
return Task.FromResult<IReadOnlyList<OrchestratorCommandRecord>>(items);
}
public Task AppendHeartbeatAsync(OrchestratorHeartbeatRecord heartbeat, CancellationToken cancellationToken)
{
_heartbeats.Add(heartbeat);
return Task.CompletedTask;
}
}

View File

@@ -4,6 +4,7 @@ using System.Threading.Tasks;
using FluentAssertions;
using StellaOps.Concelier.WebService.Services;
using StellaOps.Concelier.WebService;
using StellaOps.Concelier.WebService.Options;
using Xunit;
namespace StellaOps.Concelier.WebService.Tests.Services;

View File

@@ -0,0 +1,17 @@
using System;
using System.Collections.Generic;
using System.Text.Json.Serialization;
namespace StellaOps.Excititor.WebService.Contracts;
public sealed record GraphStatusResponse(
[property: JsonPropertyName("items")] IReadOnlyList<GraphStatusItem> Items,
[property: JsonPropertyName("cached")] bool Cached,
[property: JsonPropertyName("cacheAgeMs")] long? CacheAgeMs);
public sealed record GraphStatusItem(
[property: JsonPropertyName("purl")] string Purl,
[property: JsonPropertyName("summary")] GraphOverlaySummary Summary,
[property: JsonPropertyName("latestModifiedAt")] DateTimeOffset? LatestModifiedAt,
[property: JsonPropertyName("sources")] IReadOnlyList<string> Sources,
[property: JsonPropertyName("lastEvidenceHash")] string? LastEvidenceHash);

View File

@@ -0,0 +1,25 @@
using System;
using System.Collections.Generic;
using System.Text.Json.Serialization;
namespace StellaOps.Excititor.WebService.Contracts;
public sealed record GraphTooltipResponse(
[property: JsonPropertyName("items")] IReadOnlyList<GraphTooltipItem> Items,
[property: JsonPropertyName("nextCursor")] string? NextCursor,
[property: JsonPropertyName("hasMore")] bool HasMore);
public sealed record GraphTooltipItem(
[property: JsonPropertyName("purl")] string Purl,
[property: JsonPropertyName("observations")] IReadOnlyList<GraphTooltipObservation> Observations,
[property: JsonPropertyName("truncated")] bool Truncated);
public sealed record GraphTooltipObservation(
[property: JsonPropertyName("observationId")] string ObservationId,
[property: JsonPropertyName("advisoryId")] string AdvisoryId,
[property: JsonPropertyName("status")] string Status,
[property: JsonPropertyName("justification")] string? Justification,
[property: JsonPropertyName("providerId")] string ProviderId,
[property: JsonPropertyName("modifiedAt")] DateTimeOffset ModifiedAt,
[property: JsonPropertyName("evidenceHash")] string EvidenceHash,
[property: JsonPropertyName("dsseEnvelopeHash")] string? DsseEnvelopeHash);

View File

@@ -0,0 +1,36 @@
using System;
using System.Collections.Generic;
using System.Linq;
using StellaOps.Excititor.Core.Observations;
using StellaOps.Excititor.WebService.Contracts;
namespace StellaOps.Excititor.WebService.Graph;
internal static class GraphStatusFactory
{
public static IReadOnlyList<GraphStatusItem> Build(
IReadOnlyList<string> orderedPurls,
IReadOnlyList<VexObservation> observations)
{
if (orderedPurls is null)
{
throw new ArgumentNullException(nameof(orderedPurls));
}
if (observations is null)
{
throw new ArgumentNullException(nameof(observations));
}
var overlays = GraphOverlayFactory.Build(orderedPurls, observations, includeJustifications: false);
return overlays
.Select(overlay => new GraphStatusItem(
overlay.Purl,
overlay.Summary,
overlay.LatestModifiedAt,
overlay.Provenance.Sources,
overlay.Provenance.LastEvidenceHash))
.ToList();
}
}

View File

@@ -0,0 +1,127 @@
using System;
using System.Collections.Generic;
using System.Collections.Immutable;
using System.Linq;
using StellaOps.Excititor.Core.Observations;
using StellaOps.Excititor.WebService.Contracts;
namespace StellaOps.Excititor.WebService.Graph;
internal static class GraphTooltipFactory
{
public static IReadOnlyList<GraphTooltipItem> Build(
IReadOnlyList<string> orderedPurls,
IReadOnlyList<VexObservation> observations,
bool includeJustifications,
int maxItemsPerPurl)
{
if (orderedPurls is null)
{
throw new ArgumentNullException(nameof(orderedPurls));
}
if (observations is null)
{
throw new ArgumentNullException(nameof(observations));
}
if (maxItemsPerPurl <= 0)
{
throw new ArgumentOutOfRangeException(nameof(maxItemsPerPurl));
}
var requested = new HashSet<string>(orderedPurls, StringComparer.OrdinalIgnoreCase);
var byPurl = orderedPurls.ToDictionary(
keySelector: static purl => purl,
elementSelector: static _ => new List<GraphTooltipObservation>(),
comparer: StringComparer.OrdinalIgnoreCase);
foreach (var observation in observations)
{
var linksetPurls = observation.Linkset.Purls;
foreach (var statement in observation.Statements)
{
var targets = CollectTargets(statement, linksetPurls, requested);
if (targets.Count == 0)
{
continue;
}
var payload = new GraphTooltipObservation(
observation.ObservationId,
statement.VulnerabilityId,
statement.Status.ToString().ToLowerInvariant(),
includeJustifications ? statement.Justification?.ToString() : null,
observation.ProviderId,
observation.CreatedAt,
observation.Upstream.ContentHash,
observation.Upstream.Signature.Signature);
foreach (var target in targets)
{
byPurl[target].Add(payload);
}
}
}
var items = new List<GraphTooltipItem>(orderedPurls.Count);
foreach (var purl in orderedPurls)
{
if (!byPurl.TryGetValue(purl, out var observationsForPurl))
{
items.Add(new GraphTooltipItem(purl, Array.Empty<GraphTooltipObservation>(), false));
continue;
}
var ordered = observationsForPurl
.OrderByDescending(static o => o.ModifiedAt)
.ThenBy(static o => o.AdvisoryId, StringComparer.Ordinal)
.ThenBy(static o => o.ProviderId, StringComparer.OrdinalIgnoreCase)
.ThenBy(static o => o.ObservationId, StringComparer.Ordinal)
.ToList();
var truncated = ordered.Count > maxItemsPerPurl;
var limited = truncated ? ordered.Take(maxItemsPerPurl).ToList() : ordered;
items.Add(new GraphTooltipItem(purl, limited, truncated));
}
return items;
}
private static List<string> CollectTargets(
VexObservationStatement statement,
ImmutableArray<string> linksetPurls,
HashSet<string> requested)
{
var targets = new List<string>();
if (!string.IsNullOrWhiteSpace(statement.Purl))
{
var normalized = statement.Purl.ToLowerInvariant();
if (requested.Contains(normalized))
{
targets.Add(normalized);
}
}
if (targets.Count > 0)
{
return targets;
}
if (!linksetPurls.IsDefaultOrEmpty)
{
foreach (var purl in linksetPurls)
{
var normalized = purl?.ToLowerInvariant();
if (normalized is not null && requested.Contains(normalized))
{
targets.Add(normalized);
}
}
}
return targets;
}
}

View File

@@ -8,4 +8,6 @@ public sealed class GraphOptions
public int MaxPurls { get; set; } = 500;
public int MaxAdvisoriesPerPurl { get; set; } = 200;
public int OverlayTtlSeconds { get; set; } = 300;
public int MaxTooltipItemsPerPurl { get; set; } = 50;
public int MaxTooltipTotal { get; set; } = 1000;
}

View File

@@ -262,6 +262,10 @@ public partial class Program
signature.VerifiedAt));
}
private sealed record CachedGraphStatus(
IReadOnlyList<GraphStatusItem> Items,
DateTimeOffset CachedAt);
private sealed record CachedGraphOverlay(
IReadOnlyList<GraphOverlayItem> Items,
DateTimeOffset CachedAt);

View File

@@ -896,6 +896,64 @@ var response = new GraphLinkoutsResponse(items, notFound);
return Results.Ok(response);
}).WithName("PostGraphLinkouts");
app.MapGet("/v1/graph/status", async (
HttpContext context,
[FromQuery(Name = "purl")] string[]? purls,
IOptions<VexMongoStorageOptions> storageOptions,
IOptions<GraphOptions> graphOptions,
IVexObservationQueryService queryService,
IMemoryCache cache,
TimeProvider timeProvider,
CancellationToken cancellationToken) =>
{
if (!TryResolveTenant(context, storageOptions.Value, requireHeader: true, out var tenant, out var tenantError))
{
return tenantError;
}
var orderedPurls = NormalizePurls(purls);
if (orderedPurls.Count == 0)
{
return Results.BadRequest("purl query parameter is required");
}
if (orderedPurls.Count > graphOptions.Value.MaxPurls)
{
return Results.BadRequest($"purls limit exceeded (max {graphOptions.Value.MaxPurls})");
}
var cacheKey = $"graph-status:{tenant}:{string.Join('|', orderedPurls)}";
var now = timeProvider.GetUtcNow();
if (cache.TryGetValue<CachedGraphStatus>(cacheKey, out var cached) && cached is not null)
{
var ageMs = (long)Math.Max(0, (now - cached.CachedAt).TotalMilliseconds);
return Results.Ok(new GraphStatusResponse(cached.Items, true, ageMs));
}
var options = new VexObservationQueryOptions(
tenant: tenant,
purls: orderedPurls,
limit: graphOptions.Value.MaxAdvisoriesPerPurl * orderedPurls.Count);
VexObservationQueryResult result;
try
{
result = await queryService.QueryAsync(options, cancellationToken).ConfigureAwait(false);
}
catch (FormatException ex)
{
return Results.BadRequest(ex.Message);
}
var items = GraphStatusFactory.Build(orderedPurls, result.Observations);
var response = new GraphStatusResponse(items, false, null);
cache.Set(cacheKey, new CachedGraphStatus(items, now), TimeSpan.FromSeconds(graphOptions.Value.OverlayTtlSeconds));
return Results.Ok(response);
}).WithName("GetGraphStatus");
// Cartographer overlays
app.MapGet("/v1/graph/overlays", async (
HttpContext context,
@@ -956,6 +1014,66 @@ app.MapGet("/v1/graph/overlays", async (
return Results.Ok(response);
}).WithName("GetGraphOverlays");
app.MapGet("/v1/graph/observations", async (
HttpContext context,
[FromQuery(Name = "purl")] string[]? purls,
[FromQuery] bool includeJustifications,
[FromQuery] int? limitPerPurl,
[FromQuery] string? cursor,
IOptions<VexMongoStorageOptions> storageOptions,
IOptions<GraphOptions> graphOptions,
IVexObservationQueryService queryService,
CancellationToken cancellationToken) =>
{
if (!TryResolveTenant(context, storageOptions.Value, requireHeader: true, out var tenant, out var tenantError))
{
return tenantError;
}
var orderedPurls = NormalizePurls(purls);
if (orderedPurls.Count == 0)
{
return Results.BadRequest("purl query parameter is required");
}
if (orderedPurls.Count > graphOptions.Value.MaxPurls)
{
return Results.BadRequest($"purls limit exceeded (max {graphOptions.Value.MaxPurls})");
}
var perPurlLimit = limitPerPurl.GetValueOrDefault(graphOptions.Value.MaxTooltipItemsPerPurl);
if (perPurlLimit <= 0)
{
return Results.BadRequest("limitPerPurl must be greater than zero when provided.");
}
var effectivePerPurlLimit = Math.Min(perPurlLimit, graphOptions.Value.MaxAdvisoriesPerPurl);
var totalLimit = Math.Min(
Math.Max(1, effectivePerPurlLimit * orderedPurls.Count),
graphOptions.Value.MaxTooltipTotal);
var options = new VexObservationQueryOptions(
tenant: tenant,
purls: orderedPurls,
limit: totalLimit,
cursor: cursor);
VexObservationQueryResult result;
try
{
result = await queryService.QueryAsync(options, cancellationToken).ConfigureAwait(false);
}
catch (FormatException ex)
{
return Results.BadRequest(ex.Message);
}
var items = GraphTooltipFactory.Build(orderedPurls, result.Observations, includeJustifications, effectivePerPurlLimit);
var response = new GraphTooltipResponse(items, result.NextCursor, result.HasMore);
return Results.Ok(response);
}).WithName("GetGraphObservations");
app.MapPost("/ingest/vex", async (
HttpContext context,
VexIngestRequest request,

View File

@@ -0,0 +1,115 @@
using System.Threading;
using System.Threading.Tasks;
using MongoDB.Bson;
using MongoDB.Driver;
namespace StellaOps.Excititor.Storage.Mongo.Migrations;
/// <summary>
/// Adds a $jsonSchema validator to the raw VEX collection to enforce aggregation-only
/// shape (immutable content hash + provenance fields).
/// ValidationAction=warn keeps rollout safe while surfacing violations.
/// </summary>
internal sealed class VexRawSchemaMigration : IVexMongoMigration
{
public string Id => "20251125-vex-raw-json-schema";
public async ValueTask ExecuteAsync(IMongoDatabase database, CancellationToken cancellationToken)
{
ArgumentNullException.ThrowIfNull(database);
var exists = await CollectionExistsAsync(database, VexMongoCollectionNames.Raw, cancellationToken)
.ConfigureAwait(false);
var validator = BuildValidator();
if (!exists)
{
await database.CreateCollectionAsync(
VexMongoCollectionNames.Raw,
new CreateCollectionOptions
{
Validator = validator,
ValidationAction = DocumentValidationAction.Warn,
ValidationLevel = DocumentValidationLevel.Moderate,
},
cancellationToken).ConfigureAwait(false);
return;
}
var command = new BsonDocument
{
{ "collMod", VexMongoCollectionNames.Raw },
{ "validator", validator },
{ "validationAction", "warn" },
{ "validationLevel", "moderate" },
};
await database.RunCommandAsync<BsonDocument>(command, cancellationToken: cancellationToken)
.ConfigureAwait(false);
}
private static async Task<bool> CollectionExistsAsync(
IMongoDatabase database,
string name,
CancellationToken cancellationToken)
{
using var cursor = await database.ListCollectionNamesAsync(
new ListCollectionNamesOptions { Filter = new BsonDocument("name", name) },
cancellationToken).ConfigureAwait(false);
return await cursor.AnyAsync(cancellationToken).ConfigureAwait(false);
}
private static BsonDocument BuildValidator()
{
var properties = new BsonDocument
{
{ "_id", new BsonDocument { { "bsonType", "string" }, { "description", "digest" } } },
{ "providerId", new BsonDocument { { "bsonType", "string" }, { "minLength", 1 } } },
{ "format", new BsonDocument
{
{ "bsonType", "string" },
{ "enum", new BsonArray { "csaf", "cyclonedx", "openvex" } }
}
},
{ "sourceUri", new BsonDocument { { "bsonType", "string" }, { "minLength", 1 } } },
{ "retrievedAt", new BsonDocument { { "bsonType", "date" } } },
{ "digest", new BsonDocument { { "bsonType", "string" }, { "minLength", 32 } } },
{ "content", new BsonDocument
{
{ "bsonType", new BsonArray { "binData", "string" } }
}
},
{ "gridFsObjectId", new BsonDocument
{
{ "bsonType", new BsonArray { "objectId", "null", "string" } }
}
},
{ "metadata", new BsonDocument
{
{ "bsonType", "object" },
{ "additionalProperties", true },
{ "patternProperties", new BsonDocument
{
{ ".*", new BsonDocument { { "bsonType", "string" } } }
}
}
}
}
};
return new BsonDocument
{
{
"$jsonSchema",
new BsonDocument
{
{ "bsonType", "object" },
{ "required", new BsonArray { "_id", "providerId", "format", "sourceUri", "retrievedAt", "digest" } },
{ "properties", properties },
{ "additionalProperties", true }
}
}
};
}
}

View File

@@ -61,6 +61,7 @@ public static class VexMongoServiceCollectionExtensions
services.AddScoped<VexStatementBackfillService>();
services.AddScoped<IVexObservationLookup, MongoVexObservationLookup>();
services.AddSingleton<IVexMongoMigration, VexInitialIndexMigration>();
services.AddSingleton<IVexMongoMigration, VexRawSchemaMigration>();
services.AddSingleton<IVexMongoMigration, VexConsensusSignalsMigration>();
services.AddSingleton<IVexMongoMigration, VexConsensusHoldMigration>();
services.AddSingleton<IVexMongoMigration, VexObservationCollectionsMigration>();

View File

@@ -0,0 +1,90 @@
using System;
using System.Collections.Immutable;
using StellaOps.Excititor.Core;
using StellaOps.Excititor.Core.Observations;
using StellaOps.Excititor.WebService.Graph;
using Xunit;
namespace StellaOps.Excititor.WebService.Tests;
public sealed class GraphStatusFactoryTests
{
[Fact]
public void Build_ProjectsOverlaySummariesAndProvenance()
{
var now = DateTimeOffset.UtcNow;
var observations = new[]
{
CreateObservation(
providerId: "ubuntu",
createdAt: now,
purls: new[] { "pkg:rpm/redhat/openssl@1.1.1" },
statements: new[]
{
new VexObservationStatement(
vulnerabilityId: "CVE-2025-1001",
productKey: "pkg:rpm/redhat/openssl@1.1.1",
status: VexClaimStatus.NotAffected,
lastObserved: now,
justification: VexJustification.ComponentNotPresent,
purl: "pkg:rpm/redhat/openssl@1.1.1")
},
contentHash: "hash-new"),
CreateObservation(
providerId: "oracle",
createdAt: now.AddMinutes(-2),
purls: new[] { "pkg:rpm/redhat/openssl@1.1.1" },
statements: Array.Empty<VexObservationStatement>(),
contentHash: "hash-old")
};
var items = GraphStatusFactory.Build(
orderedPurls: new[] { "pkg:rpm/redhat/openssl@1.1.1" },
observations: observations);
var item = Assert.Single(items);
Assert.Equal("pkg:rpm/redhat/openssl@1.1.1", item.Purl);
Assert.Equal(0, item.Summary.Open);
Assert.Equal(1, item.Summary.NotAffected);
Assert.Equal(0, item.Summary.UnderInvestigation);
Assert.Equal(1, item.Summary.NoStatement);
Assert.Equal(now, item.LatestModifiedAt);
Assert.Equal("hash-new", item.LastEvidenceHash);
Assert.Equal(new[] { "oracle", "ubuntu" }, item.Sources);
}
private static VexObservation CreateObservation(
string providerId,
DateTimeOffset createdAt,
string[] purls,
VexObservationStatement[] statements,
string contentHash)
{
return new VexObservation(
observationId: $"obs-{providerId}-{createdAt.ToUnixTimeMilliseconds()}",
tenant: "tenant-a",
providerId: providerId,
streamId: "csaf",
upstream: new VexObservationUpstream(
upstreamId: Guid.NewGuid().ToString("N"),
documentVersion: "1",
fetchedAt: createdAt,
receivedAt: createdAt,
contentHash: contentHash,
signature: new VexObservationSignature(present: true, format: "sig", keyId: null, signature: null)),
statements: statements.ToImmutableArray(),
content: new VexObservationContent(
format: "csaf",
specVersion: "1",
raw: System.Text.Json.Nodes.JsonValue.Create("raw")!,
metadata: ImmutableDictionary<string, string>.Empty),
linkset: new VexObservationLinkset(
aliases: Array.Empty<string>(),
purls: purls,
cpes: Array.Empty<string>(),
references: Array.Empty<VexObservationReference>()),
createdAt: createdAt,
supersedes: ImmutableArray<string>.Empty,
attributes: ImmutableDictionary<string, string>.Empty);
}
}

View File

@@ -0,0 +1,137 @@
using System;
using System.Collections.Immutable;
using System.Linq;
using StellaOps.Excititor.Core;
using StellaOps.Excititor.Core.Observations;
using StellaOps.Excititor.WebService.Graph;
using Xunit;
namespace StellaOps.Excititor.WebService.Tests;
public sealed class GraphTooltipFactoryTests
{
[Fact]
public void Build_OrdersByNewestAndTruncatesPerPurl()
{
var now = DateTimeOffset.UtcNow;
var observations = new[]
{
CreateObservation(
providerId: "ubuntu",
createdAt: now,
purls: new[] { "pkg:rpm/openssl@1.1.1" },
statements: new[]
{
new VexObservationStatement(
vulnerabilityId: "CVE-2025-1000",
productKey: "pkg:rpm/openssl@1.1.1",
status: VexClaimStatus.NotAffected,
lastObserved: now,
justification: VexJustification.ComponentNotPresent,
purl: "pkg:rpm/openssl@1.1.1")
},
contentHash: "hash-ubuntu"),
CreateObservation(
providerId: "redhat",
createdAt: now.AddMinutes(-1),
purls: new[] { "pkg:rpm/openssl@1.1.1" },
statements: new[]
{
new VexObservationStatement(
vulnerabilityId: "CVE-2025-1000",
productKey: "pkg:rpm/openssl@1.1.1",
status: VexClaimStatus.UnderInvestigation,
lastObserved: now.AddMinutes(-1),
justification: null,
purl: "pkg:rpm/openssl@1.1.1")
},
contentHash: "hash-redhat")
};
var items = GraphTooltipFactory.Build(
orderedPurls: new[] { "pkg:rpm/openssl@1.1.1" },
observations: observations,
includeJustifications: true,
maxItemsPerPurl: 1);
var item = Assert.Single(items);
Assert.True(item.Truncated);
var obs = Assert.Single(item.Observations);
Assert.Equal("CVE-2025-1000", obs.AdvisoryId);
Assert.Equal("notaffected", obs.Status);
Assert.Equal("ComponentNotPresent", obs.Justification);
Assert.Equal("ubuntu", obs.ProviderId);
Assert.Equal("hash-ubuntu", obs.EvidenceHash);
}
[Fact]
public void Build_UsesLinksetPurlsWhenStatementMissing()
{
var now = DateTimeOffset.UtcNow;
var observations = new[]
{
CreateObservation(
providerId: "oracle",
createdAt: now,
purls: new[] { "pkg:rpm/httpd@2.4.0" },
statements: new[]
{
new VexObservationStatement(
vulnerabilityId: "CVE-2025-2000",
productKey: "pkg:rpm/httpd@2.4.0",
status: VexClaimStatus.Affected,
lastObserved: now,
justification: null,
purl: null)
},
contentHash: "hash-oracle")
};
var items = GraphTooltipFactory.Build(
orderedPurls: new[] { "pkg:rpm/httpd@2.4.0" },
observations: observations,
includeJustifications: false,
maxItemsPerPurl: 5);
var item = Assert.Single(items);
var observation = Assert.Single(item.Observations);
Assert.Null(observation.Justification);
Assert.Equal("oracle", observation.ProviderId);
Assert.Equal("CVE-2025-2000", observation.AdvisoryId);
}
private static VexObservation CreateObservation(
string providerId,
DateTimeOffset createdAt,
string[] purls,
VexObservationStatement[] statements,
string contentHash)
{
return new VexObservation(
observationId: $"obs-{providerId}-{createdAt.ToUnixTimeMilliseconds()}",
tenant: "tenant-a",
providerId: providerId,
streamId: "csaf",
upstream: new VexObservationUpstream(
upstreamId: Guid.NewGuid().ToString("N"),
documentVersion: "1",
fetchedAt: createdAt,
receivedAt: createdAt,
contentHash: contentHash,
signature: new VexObservationSignature(present: true, format: "sig", keyId: null, signature: null)),
statements: statements.ToImmutableArray(),
content: new VexObservationContent(
format: "csaf",
specVersion: "1",
raw: System.Text.Json.Nodes.JsonValue.Create("raw")!,
metadata: ImmutableDictionary<string, string>.Empty),
linkset: new VexObservationLinkset(
aliases: Array.Empty<string>(),
purls: purls,
cpes: Array.Empty<string>(),
references: Array.Empty<VexObservationReference>()),
createdAt: createdAt,
supersedes: ImmutableArray<string>.Empty,
attributes: ImmutableDictionary<string, string>.Empty);
}
}

View File

@@ -37,6 +37,8 @@
<Compile Include="TestServiceOverrides.cs" />
<Compile Include="TestWebApplicationFactory.cs" />
<Compile Include="GraphOverlayFactoryTests.cs" />
<Compile Include="GraphStatusFactoryTests.cs" />
<Compile Include="GraphTooltipFactoryTests.cs" />
<Compile Include="AttestationVerifyEndpointTests.cs" />
</ItemGroup>
</Project>

View File

@@ -9,4 +9,10 @@ namespace StellaOps.RiskEngine.Core.Services;
public interface IRiskScoreResultStore
{
Task SaveAsync(RiskScoreResult result, CancellationToken cancellationToken);
/// <summary>
/// Attempts to read a previously persisted result by job identifier.
/// Implementations must be deterministic and side-effect free.
/// </summary>
bool TryGet(Guid jobId, out RiskScoreResult result);
}

View File

@@ -45,6 +45,17 @@ public sealed class RiskScoreQueue
return channel.Writer.WriteAsync(job, cancellationToken);
}
/// <summary>
/// Enqueues a request and returns the assigned job id for later retrieval.
/// </summary>
public async ValueTask<Guid> EnqueueWithIdAsync(ScoreRequest request, CancellationToken cancellationToken)
{
ArgumentNullException.ThrowIfNull(request);
var job = new RiskScoreJob(Guid.NewGuid(), request);
await channel.Writer.WriteAsync(job, cancellationToken).ConfigureAwait(false);
return job.JobId;
}
public ValueTask<RiskScoreJob> DequeueAsync(CancellationToken cancellationToken) =>
channel.Reader.ReadAsync(cancellationToken);

View File

@@ -0,0 +1,119 @@
using System.Net;
using System.Net.Http.Json;
using Microsoft.AspNetCore.Mvc.Testing;
using StellaOps.RiskEngine.Core.Contracts;
using StellaOps.RiskEngine.Core.Providers;
using Xunit;
namespace StellaOps.RiskEngine.Tests;
public class RiskEngineApiTests : IClassFixture<WebApplicationFactory<Program>>
{
private readonly WebApplicationFactory<Program> factory;
public RiskEngineApiTests(WebApplicationFactory<Program> factory)
{
this.factory = factory.WithWebHostBuilder(_ => { });
}
[Fact]
public async Task Providers_ListsDefaultTransforms()
{
var client = factory.CreateClient();
var ct = TestContext.Current.CancellationToken;
var response = await client.GetAsync("/risk-scores/providers", ct);
response.EnsureSuccessStatusCode();
var payload = await response.Content.ReadFromJsonAsync<ProvidersResponse>(cancellationToken: ct);
Assert.NotNull(payload);
Assert.Contains(DefaultTransformsProvider.ProviderName, payload!.Providers);
}
[Fact]
public async Task Job_SubmitAndRetrieve_PersistsResult()
{
var client = factory.CreateClient();
var ct = TestContext.Current.CancellationToken;
var request = new ScoreRequest(DefaultTransformsProvider.ProviderName, "asset-1", new Dictionary<string, double>
{
["signal"] = 0.5
});
var submit = await client.PostAsJsonAsync("/risk-scores/jobs", request, ct);
Assert.Equal(HttpStatusCode.Accepted, submit.StatusCode);
var accepted = await submit.Content.ReadFromJsonAsync<JobAccepted>(cancellationToken: ct);
Assert.NotNull(accepted);
Assert.True(accepted!.Result.Success);
var fetched = await client.GetFromJsonAsync<RiskScoreResult>($"/risk-scores/jobs/{accepted.JobId}", ct);
Assert.NotNull(fetched);
Assert.Equal(accepted.JobId, fetched!.JobId);
Assert.True(fetched.Success);
}
[Fact]
public async Task Simulations_ReturnsBatch()
{
var client = factory.CreateClient();
var ct = TestContext.Current.CancellationToken;
var requests = new[]
{
new ScoreRequest(DefaultTransformsProvider.ProviderName, "asset-a", new Dictionary<string, double>{{"a", 0.2}}),
new ScoreRequest(DefaultTransformsProvider.ProviderName, "asset-b", new Dictionary<string, double>{{"b", 0.8}})
};
var response = await client.PostAsJsonAsync("/risk-scores/simulations", requests, ct);
response.EnsureSuccessStatusCode();
var payload = await response.Content.ReadFromJsonAsync<SimulationResponse>(cancellationToken: ct);
Assert.NotNull(payload);
Assert.Equal(2, payload!.Results.Count);
Assert.All(payload.Results, r => Assert.True(r.Success));
}
[Fact]
public async Task Simulations_Summary_ReturnsAggregatesAndTopMovers()
{
var client = factory.CreateClient();
var ct = TestContext.Current.CancellationToken;
var requests = new[]
{
new ScoreRequest(DefaultTransformsProvider.ProviderName, "asset-high", new Dictionary<string, double>{{"s1", 1.0}}),
new ScoreRequest(DefaultTransformsProvider.ProviderName, "asset-mid", new Dictionary<string, double>{{"s1", 0.5}}),
new ScoreRequest(DefaultTransformsProvider.ProviderName, "asset-low", new Dictionary<string, double>{{"s1", 0.2}})
};
var response = await client.PostAsJsonAsync("/risk-scores/simulations/summary", requests, ct);
response.EnsureSuccessStatusCode();
var payload = await response.Content.ReadFromJsonAsync<SimulationSummaryResponse>(cancellationToken: ct);
Assert.NotNull(payload);
Assert.Equal(3, payload!.Results.Count);
Assert.All(payload.Results, r => Assert.True(r.Success));
Assert.Equal(0.566667, Math.Round(payload.Summary.AverageScore, 6));
Assert.Equal(0.2, payload.Summary.MinScore);
Assert.Equal(1.0, payload.Summary.MaxScore);
Assert.Equal(3, payload.Summary.TopMovers.Count);
Assert.Collection(payload.Summary.TopMovers,
first =>
{
Assert.Equal("asset-high", first.Subject);
Assert.Equal(1.0, first.Score);
},
second => Assert.Equal("asset-mid", second.Subject),
third => Assert.Equal("asset-low", third.Subject));
}
private sealed record ProvidersResponse(IReadOnlyList<string> Providers);
private sealed record JobAccepted(Guid JobId, RiskScoreResult Result);
private sealed record SimulationResponse(IReadOnlyList<RiskScoreResult> Results);
private sealed record SimulationSummaryDto(double AverageScore, double MinScore, double MaxScore, IReadOnlyList<RiskScoreResult> TopMovers);
private sealed record SimulationSummaryResponse(SimulationSummaryDto Summary, IReadOnlyList<RiskScoreResult> Results);
}

View File

@@ -58,17 +58,10 @@
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.14.1"/>
<PackageReference Include="xunit.v3" Version="3.0.0"/>
<PackageReference Include="xunit.runner.visualstudio" Version="3.1.3"/>
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.14.1"/>
<PackageReference Include="Microsoft.AspNetCore.Mvc.Testing" Version="10.0.0-rc.2.25502.107"/>
<PackageReference Include="xunit.v3" Version="3.0.0"/>
<PackageReference Include="xunit.runner.visualstudio" Version="3.1.3"/>
@@ -116,12 +109,9 @@
<ProjectReference Include="..\StellaOps.RiskEngine.Core\StellaOps.RiskEngine.Core.csproj"/>
<ProjectReference Include="..\StellaOps.RiskEngine.Infrastructure\StellaOps.RiskEngine.Infrastructure.csproj"/>
<ProjectReference Include="..\StellaOps.RiskEngine.Core\StellaOps.RiskEngine.Core.csproj"/>
<ProjectReference Include="..\StellaOps.RiskEngine.Infrastructure\StellaOps.RiskEngine.Infrastructure.csproj"/>
<ProjectReference Include="..\StellaOps.RiskEngine.WebService\StellaOps.RiskEngine.WebService.csproj"/>

View File

@@ -1,3 +1,5 @@
using Microsoft.AspNetCore.Mvc;
using System.Linq;
using StellaOps.RiskEngine.Core.Contracts;
using StellaOps.RiskEngine.Core.Providers;
using StellaOps.RiskEngine.Core.Services;
@@ -32,51 +34,92 @@ app.MapGet("/risk-scores/providers", (IRiskScoreProviderRegistry registry) =>
app.MapPost("/risk-scores/jobs", async (
ScoreRequest request,
RiskScoreQueue queue,
IRiskScoreProviderRegistry registry,
IRiskScoreResultStore store,
TimeProvider timeProvider,
[FromServices] RiskScoreQueue queue,
[FromServices] IRiskScoreProviderRegistry registry,
[FromServices] IRiskScoreResultStore store,
CancellationToken ct) =>
{
var job = new RiskScoreJob(Guid.NewGuid(), request);
await queue.EnqueueAsync(job.Request, ct).ConfigureAwait(false);
var worker = new RiskScoreWorker(queue, registry, store, timeProvider);
var normalized = new ScoreRequest(
request.Provider,
request.Subject,
request.Signals ?? new Dictionary<string, double>());
var jobId = await queue.EnqueueWithIdAsync(normalized, ct).ConfigureAwait(false);
var worker = new RiskScoreWorker(queue, registry, store, TimeProvider.System);
var result = await worker.ProcessNextAsync(ct).ConfigureAwait(false);
return Results.Accepted($"/risk-scores/jobs/{job.JobId}", new { jobId = job.JobId, result });
return Results.Accepted($"/risk-scores/jobs/{jobId}", new { jobId, result });
});
app.MapGet("/risk-scores/jobs/{jobId:guid}", (Guid jobId, InMemoryRiskScoreResultStore store) =>
app.MapGet("/risk-scores/jobs/{jobId:guid}", (
Guid jobId,
[FromServices] IRiskScoreResultStore store) =>
store.TryGet(jobId, out var result)
? Results.Ok(result)
: Results.NotFound());
app.MapPost("/risk-scores/simulations", async (
IReadOnlyCollection<ScoreRequest> requests,
IRiskScoreProviderRegistry registry,
TimeProvider timeProvider,
[FromServices] IRiskScoreProviderRegistry registry,
CancellationToken ct) =>
{
var results = await EvaluateAsync(requests, registry, ct).ConfigureAwait(false);
return Results.Ok(new { results });
});
app.MapPost("/risk-scores/simulations/summary", async (
IReadOnlyCollection<ScoreRequest> requests,
[FromServices] IRiskScoreProviderRegistry registry,
CancellationToken ct) =>
{
var results = await EvaluateAsync(requests, registry, ct).ConfigureAwait(false);
var scores = results.Select(r => r.Score).ToArray();
var summary = new
{
averageScore = scores.Length == 0 ? 0d : scores.Average(),
minScore = scores.Length == 0 ? 0d : scores.Min(),
maxScore = scores.Length == 0 ? 0d : scores.Max(),
topMovers = results
.OrderByDescending(r => r.Score)
.ThenBy(r => r.Subject, StringComparer.Ordinal)
.Take(3)
.ToArray()
};
return Results.Ok(new { summary, results });
});
app.Run();
static async Task<List<RiskScoreResult>> EvaluateAsync(
IReadOnlyCollection<ScoreRequest> requests,
IRiskScoreProviderRegistry registry,
CancellationToken ct)
{
var results = new List<RiskScoreResult>(requests.Count);
foreach (var req in requests)
{
if (!registry.TryGet(req.Provider, out var provider))
var normalized = new ScoreRequest(
req.Provider,
req.Subject,
req.Signals ?? new Dictionary<string, double>());
if (!registry.TryGet(normalized.Provider, out var provider))
{
results.Add(new RiskScoreResult(Guid.NewGuid(), req.Provider, req.Subject, 0d, false, "Provider not registered", req.Signals, timeProvider.GetUtcNow()));
results.Add(new RiskScoreResult(Guid.NewGuid(), normalized.Provider, normalized.Subject, 0d, false, "Provider not registered", normalized.Signals, TimeProvider.System.GetUtcNow()));
continue;
}
try
{
var score = await provider.ScoreAsync(req, ct).ConfigureAwait(false);
results.Add(new RiskScoreResult(Guid.NewGuid(), req.Provider, req.Subject, score, true, null, req.Signals, timeProvider.GetUtcNow()));
var score = await provider.ScoreAsync(normalized, ct).ConfigureAwait(false);
results.Add(new RiskScoreResult(Guid.NewGuid(), normalized.Provider, normalized.Subject, score, true, null, normalized.Signals, TimeProvider.System.GetUtcNow()));
}
catch (Exception ex)
{
results.Add(new RiskScoreResult(Guid.NewGuid(), req.Provider, req.Subject, 0d, false, ex.Message, req.Signals, timeProvider.GetUtcNow()));
results.Add(new RiskScoreResult(Guid.NewGuid(), normalized.Provider, normalized.Subject, 0d, false, ex.Message, normalized.Signals, TimeProvider.System.GetUtcNow()));
}
}
return Results.Ok(new { results });
});
app.Run();
return results;
}

View File

@@ -50,6 +50,13 @@ internal static class SampleData
{
new AdvisoryRef("https://example.com/advisory/0001", "Upstream advisory")
},
Rationale: new PolicyRationale("rat-0001", "High severity RCE with known exploit; fix available"),
Paths: new[] { "/src/app/Program.cs", "/src/lib/utils/net.cs" },
Evidence: new[]
{
new EvidenceRef("sbom", "sbom-0001", "Inventory evidence"),
new EvidenceRef("vex", "vex-0001", "Vendor statement")
},
FirstSeen: DateTimeOffset.Parse("2025-01-01T00:00:00Z"),
LastSeen: DateTimeOffset.Parse("2025-11-01T00:00:00Z"),
PolicyVersion: summaries[0].PolicyVersion,
@@ -70,6 +77,12 @@ internal static class SampleData
new PackageAffect("pkg:npm/foo", new[] { "4.5.6" })
},
AdvisoryRefs: Array.Empty<AdvisoryRef>(),
Rationale: new PolicyRationale("rat-0002", "Medium severity; no exploit observed; fix unavailable"),
Paths: new[] { "/app/node_modules/foo/index.js" },
Evidence: new[]
{
new EvidenceRef("sbom", "sbom-0002", "Inventory evidence")
},
FirstSeen: DateTimeOffset.Parse("2024-06-10T00:00:00Z"),
LastSeen: DateTimeOffset.Parse("2025-08-15T00:00:00Z"),
PolicyVersion: summaries[1].PolicyVersion,

View File

@@ -24,6 +24,9 @@ public sealed record VulnDetail(
string Summary,
IReadOnlyList<PackageAffect> AffectedPackages,
IReadOnlyList<AdvisoryRef> AdvisoryRefs,
PolicyRationale Rationale,
IReadOnlyList<string> Paths,
IReadOnlyList<EvidenceRef> Evidence,
DateTimeOffset FirstSeen,
DateTimeOffset LastSeen,
string PolicyVersion,
@@ -34,6 +37,10 @@ public sealed record PackageAffect(string Purl, IReadOnlyList<string> Versions);
public sealed record AdvisoryRef(string Url, string Title);
public sealed record EvidenceRef(string Kind, string Reference, string? Title = null);
public sealed record EvidenceProvenance(string LedgerEntryId, string EvidenceBundleId);
public sealed record PolicyRationale(string Id, string Summary);
public sealed record VulnListResponse(IReadOnlyList<VulnSummary> Items, string? NextPageToken);

View File

@@ -1,28 +1,36 @@
using System.Collections.Generic;
using Swashbuckle.AspNetCore.SwaggerGen;
using System.Globalization;
using Microsoft.AspNetCore.Mvc;
using Microsoft.AspNetCore.Builder;
using Microsoft.AspNetCore.OpenApi;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.OpenApi.Models;
using StellaOps.VulnExplorer.Api.Data;
using StellaOps.VulnExplorer.Api.Models;
var builder = WebApplication.CreateBuilder(args);
builder.Services.AddEndpointsApiExplorer();
builder.Services.AddOpenApi();
builder.Services.AddSwaggerGen(options =>
{
options.SwaggerDoc("v1", new OpenApiInfo
{
Title = "StellaOps Vuln Explorer API",
Version = "v1",
Description = "Deterministic vulnerability listing/detail endpoints"
});
});
var app = builder.Build();
if (app.Environment.IsDevelopment())
{
app.MapOpenApi();
}
app.UseSwagger();
app.UseSwaggerUI();
app.UseHttpsRedirection();
app.MapGet("/v1/vulns", (
HttpRequest request,
[AsParameters] VulnFilter filter) =>
app.MapGet("/v1/vulns", ([AsParameters] VulnFilter filter) =>
{
var tenant = request.Headers["x-stella-tenant"].ToString();
if (string.IsNullOrWhiteSpace(tenant))
if (string.IsNullOrWhiteSpace(filter.Tenant))
{
return Results.BadRequest(new { error = "x-stella-tenant required" });
}
@@ -40,9 +48,8 @@ app.MapGet("/v1/vulns", (
})
.WithOpenApi();
app.MapGet("/v1/vulns/{id}", (HttpRequest request, string id) =>
app.MapGet("/v1/vulns/{id}", ([FromHeader(Name = "x-stella-tenant")] string? tenant, string id) =>
{
var tenant = request.Headers["x-stella-tenant"].ToString();
if (string.IsNullOrWhiteSpace(tenant))
{
return Results.BadRequest(new { error = "x-stella-tenant required" });
@@ -63,19 +70,19 @@ static IReadOnlyList<VulnSummary> ApplyFilter(IReadOnlyList<VulnSummary> source,
{
IEnumerable<VulnSummary> query = source;
if (filter.Cve?.Count > 0)
if (filter.Cve is { Length: > 0 })
{
var set = filter.Cve.ToHashSet(StringComparer.OrdinalIgnoreCase);
query = query.Where(v => v.CveIds.Any(set.Contains));
}
if (filter.Purl?.Count > 0)
if (filter.Purl is { Length: > 0 })
{
var set = filter.Purl.ToHashSet(StringComparer.OrdinalIgnoreCase);
query = query.Where(v => v.Purls.Any(set.Contains));
}
if (filter.Severity?.Count > 0)
if (filter.Severity is { Length: > 0 })
{
var set = filter.Severity.ToHashSet(StringComparer.OrdinalIgnoreCase);
query = query.Where(v => set.Contains(v.Severity));
@@ -100,13 +107,13 @@ static IReadOnlyList<VulnSummary> ApplyFilter(IReadOnlyList<VulnSummary> source,
}
public record VulnFilter(
[FromHeader(Name = "x-stella-tenant")] string Tenant,
[FromHeader(Name = "x-stella-tenant")] string? Tenant,
[FromQuery(Name = "policyVersion")] string? PolicyVersion,
[FromQuery(Name = "pageSize")] int? PageSize,
[FromQuery(Name = "pageToken")] string? PageToken,
[FromQuery(Name = "cve")] IReadOnlyList<string>? Cve,
[FromQuery(Name = "purl")] IReadOnlyList<string>? Purl,
[FromQuery(Name = "severity")] IReadOnlyList<string>? Severity,
[FromQuery(Name = "cve")] string[]? Cve,
[FromQuery(Name = "purl")] string[]? Purl,
[FromQuery(Name = "severity")] string[]? Severity,
[FromQuery(Name = "exploitability")] string? Exploitability,
[FromQuery(Name = "fixAvailable")] bool? FixAvailable);

View File

@@ -10,5 +10,6 @@
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Microsoft.AspNetCore.OpenApi" Version="8.0.8" />
<PackageReference Include="Swashbuckle.AspNetCore" Version="6.6.2" />
</ItemGroup>
</Project>

View File

@@ -0,0 +1,7 @@
# Vuln Explorer API Tasks (Sprint 0129-0001-0001)
| Task ID | Status | Notes |
| --- | --- | --- |
| VULN-API-29-001 | DONE (2025-11-25) | OpenAPI v1 draft published at `docs/modules/vuln-explorer/openapi/vuln-explorer.v1.yaml` with tenant header, filters, deterministic paging. |
| VULN-API-29-002 | DONE (2025-11-25) | Implemented `/v1/vulns` list + `/v1/vulns/{id}` detail with deterministic paging/filtering, sample data, Swagger UI; tests green (`tests/TestResults/vuln-explorer/api.trx`). |
| VULN-API-29-003 | DONE (2025-11-25) | Detail endpoint now returns rationale, paths, evidence references; covered by Vuln Explorer API integration tests. |