work
Some checks failed
AOC Guard CI / aoc-guard (push) Has been cancelled
AOC Guard CI / aoc-verify (push) Has been cancelled
Concelier Attestation Tests / attestation-tests (push) Has been cancelled
Docs CI / lint-and-preview (push) Has been cancelled

This commit is contained in:
StellaOps Bot
2025-11-25 08:01:23 +02:00
parent d92973d6fd
commit 6bee1fdcf5
207 changed files with 12816 additions and 2295 deletions

44
src/Excititor/AGENTS.md Normal file
View File

@@ -0,0 +1,44 @@
# Excititor · AGENTS Charter (Air-Gap & Trust Connectors)
## Module Scope & Working Directory
- Working directory: `src/Excititor/**` (WebService, Worker, __Libraries, __Tests, connectors, scripts). No cross-module edits unless explicitly noted in sprint Decisions & Risks.
- Mission (current sprint): air-gap parity for evidence chunks, trust connector wiring, and attestation verification aligned to Evidence Locker contract.
## Roles
- **Backend engineer (ASP.NET Core / Mongo):** chunk ingestion/export, attestation verifier, trust connector.
- **Air-Gap/Platform engineer:** sealed-mode switches, offline bundles, deterministic cache/path handling.
- **QA automation:** WebApplicationFactory + Mongo2Go tests for chunk APIs, attestations, and trust connector; deterministic ordering/hashes.
- **Docs/Schema steward:** keep chunk API, attestation plan, and trust connector docs in sync with behavior; update schemas and samples.
## Required Reading (treat as read before DOING)
- `docs/README.md`
- `docs/07_HIGH_LEVEL_ARCHITECTURE.md`
- `docs/modules/platform/architecture-overview.md`
- `docs/modules/excititor/architecture.md`
- `docs/modules/excititor/attestation-plan.md`
- `docs/modules/excititor/operations/chunk-api-user-guide.md`
- `docs/modules/excititor/schemas/vex-chunk-api.yaml`
- `docs/modules/evidence-locker/attestation-contract.md`
## Working Agreements
- Determinism: canonical JSON ordering; stable pagination; UTC ISO-8601 timestamps; sort chunk edges deterministically.
- Offline-first: default sealed-mode must not reach external networks; connectors obey allowlist; feature flags default safe.
- Attestation: DSSE/Envelope per contract; always include tenant/source identifiers; validation fixtures required.
- Tenant safety: enforce tenant headers/guards on every API; no cross-tenant leakage.
- Logging/metrics: structured logs; meters under `StellaOps.Excititor.*`; tag `tenant`, `source`, `result`.
- Cross-module edits: require sprint note; otherwise, stay within Excititor working dir.
## Testing Rules
- Use Mongo2Go/in-memory fixtures; avoid network.
- API tests in `StellaOps.Excititor.WebService.Tests`; worker/connectors in `StellaOps.Excititor.Worker.Tests`; shared fixtures in `__Tests`.
- Tests must assert determinism (ordering/hashes), tenant enforcement, and sealed-mode behavior.
## Delivery Discipline
- Update sprint tracker status (`TODO → DOING → DONE/BLOCKED`) for each task; mirror changes in Execution Log and Decisions & Risks.
- When changing contracts (API/attestation schemas), update docs and samples and link from sprint Decisions & Risks.
- If a decision is needed, mark the task BLOCKED and record the decision ask—do not pause work.
## Tooling/Env Notes
- .NET 10 with preview features enabled; Mongo driver ≥ 3.x.
- Signing/verifier hooks rely on Evidence Locker contract fixtures under `docs/modules/evidence-locker/`.
- Sealed-mode tests should run with `EXCITITOR_SEALED=1` (env var) to enforce offline code paths.

View File

@@ -21,6 +21,9 @@ public sealed class AirgapImportRequest
[JsonPropertyName("publisher")]
public string? Publisher { get; init; }
[JsonPropertyName("tenantId")]
public string? TenantId { get; init; }
[JsonPropertyName("payloadHash")]
public string? PayloadHash { get; init; }

View File

@@ -0,0 +1,25 @@
using System.Collections.Generic;
namespace StellaOps.Excititor.WebService.Options;
internal sealed class AirgapOptions
{
public const string SectionName = "Excititor:Airgap";
/// <summary>
/// Enables sealed-mode enforcement for air-gapped imports.
/// When true, external payload URLs are rejected and publisher allowlist is applied.
/// </summary>
public bool SealedMode { get; set; } = false;
/// <summary>
/// When true, imports must originate from mirror/offline sources (no HTTP/HTTPS URLs).
/// </summary>
public bool MirrorOnly { get; set; } = true;
/// <summary>
/// Optional allowlist of publishers that may submit bundles while sealed mode is enabled.
/// Empty list means allow all.
/// </summary>
public List<string> TrustedPublishers { get; } = new();
}

View File

@@ -2,7 +2,6 @@ using System;
using System.Collections.Generic;
using System.Linq;
using System.Collections.Immutable;
using System.Globalization;
using System.Diagnostics;
using System.Reflection;
using System.Text;
@@ -54,8 +53,10 @@ services.AddCycloneDxNormalizer();
services.AddOpenVexNormalizer();
services.AddSingleton<IVexSignatureVerifier, NoopVexSignatureVerifier>();
// TODO: replace NoopVexSignatureVerifier with hardened verifier once portable bundle signatures are finalized.
services.Configure<AirgapOptions>(configuration.GetSection(AirgapOptions.SectionName));
services.AddSingleton<AirgapImportValidator>();
services.AddSingleton<AirgapSignerTrustService>();
services.AddSingleton<AirgapModeEnforcer>();
services.AddSingleton<ConsoleTelemetry>();
services.AddMemoryCache();
services.AddScoped<IVexIngestOrchestrator, VexIngestOrchestrator>();
@@ -185,7 +186,7 @@ app.MapGet("/openapi/excititor.json", () =>
get = new
{
summary = "Service status (aggregation-only metadata)",
responses = new
responses = new Dictionary<string, object>
{
["200"] = new
{
@@ -219,7 +220,7 @@ app.MapGet("/openapi/excititor.json", () =>
get = new
{
summary = "Health check",
responses = new
responses = new Dictionary<string, object>
{
["200"] = new
{
@@ -254,7 +255,7 @@ app.MapGet("/openapi/excititor.json", () =>
new { name = "cursor", @in = "query", schema = new { type = "string" }, required = false, description = "Numeric cursor or Last-Event-ID" },
new { name = "limit", @in = "query", schema = new { type = "integer", minimum = 1, maximum = 100 }, required = false }
},
responses = new
responses = new Dictionary<string, object>
{
["200"] = new
{
@@ -331,7 +332,7 @@ app.MapGet("/openapi/excititor.json", () =>
}
}
},
responses = new
responses = new Dictionary<string, object>
{
["200"] = new { description = "Accepted" },
["400"] = new
@@ -448,16 +449,47 @@ app.MapGet("/openapi/excititor.json", () =>
app.MapPost("/airgap/v1/vex/import", async (
[FromServices] AirgapImportValidator validator,
[FromServices] AirgapSignerTrustService trustService,
[FromServices] AirgapModeEnforcer modeEnforcer,
[FromServices] IAirgapImportStore store,
[FromServices] ILoggerFactory loggerFactory,
[FromServices] TimeProvider timeProvider,
[FromBody] AirgapImportRequest request,
CancellationToken cancellationToken) =>
{
var logger = loggerFactory.CreateLogger("AirgapImport");
var nowUtc = timeProvider.GetUtcNow();
var tenantId = string.IsNullOrWhiteSpace(request.TenantId)
? "default"
: request.TenantId!.Trim().ToLowerInvariant();
var stalenessSeconds = request.SignedAt is null
? (int?)null
: (int)Math.Round((nowUtc - request.SignedAt.Value).TotalSeconds);
var timeline = new List<AirgapTimelineEntry>();
void RecordEvent(string eventType, string? code = null, string? message = null)
{
var entry = new AirgapTimelineEntry
{
EventType = eventType,
CreatedAt = nowUtc,
TenantId = tenantId,
BundleId = request.BundleId ?? string.Empty,
MirrorGeneration = request.MirrorGeneration ?? string.Empty,
StalenessSeconds = stalenessSeconds,
ErrorCode = code,
Message = message
};
timeline.Add(entry);
logger.LogInformation("Airgap timeline event {EventType} bundle={BundleId} gen={Gen} tenant={Tenant} code={Code}", eventType, entry.BundleId, entry.MirrorGeneration, tenantId, code);
}
RecordEvent("airgap.import.started");
var errors = validator.Validate(request, nowUtc);
if (errors.Count > 0)
{
var first = errors[0];
RecordEvent("airgap.import.failed", first.Code, first.Message);
return Results.BadRequest(new
{
error = new
@@ -468,8 +500,22 @@ app.MapPost("/airgap/v1/vex/import", async (
});
}
if (!modeEnforcer.Validate(request, out var sealedCode, out var sealedMessage))
{
RecordEvent("airgap.import.failed", sealedCode, sealedMessage);
return Results.Json(new
{
error = new
{
code = sealedCode,
message = sealedMessage
}
}, statusCode: StatusCodes.Status403Forbidden);
}
if (!trustService.Validate(request, out var trustCode, out var trustMessage))
{
RecordEvent("airgap.import.failed", trustCode, trustMessage);
return Results.Json(new
{
error = new
@@ -480,9 +526,16 @@ app.MapPost("/airgap/v1/vex/import", async (
}, statusCode: StatusCodes.Status403Forbidden);
}
var manifestPath = $"mirror/{request.BundleId}/{request.MirrorGeneration}/manifest.json";
var evidenceLockerPath = $"evidence/{request.BundleId}/{request.MirrorGeneration}/bundle.ndjson";
var manifestHash = ComputeSha256($"{request.BundleId}:{request.MirrorGeneration}:{request.PayloadHash}");
RecordEvent("airgap.import.completed");
var record = new AirgapImportRecord
{
Id = $"{request.BundleId}:{request.MirrorGeneration}",
TenantId = tenantId,
BundleId = request.BundleId!,
MirrorGeneration = request.MirrorGeneration!,
SignedAt = request.SignedAt!.Value,
@@ -491,7 +544,11 @@ app.MapPost("/airgap/v1/vex/import", async (
PayloadUrl = request.PayloadUrl,
Signature = request.Signature!,
TransparencyLog = request.TransparencyLog,
ImportedAt = nowUtc
ImportedAt = nowUtc,
PortableManifestPath = manifestPath,
PortableManifestHash = manifestHash,
EvidenceLockerPath = evidenceLockerPath,
Timeline = timeline
};
try
@@ -500,6 +557,7 @@ app.MapPost("/airgap/v1/vex/import", async (
}
catch (DuplicateAirgapImportException dup)
{
RecordEvent("airgap.import.failed", "AIRGAP_IMPORT_DUPLICATE", dup.Message);
return Results.Conflict(new
{
error = new
@@ -513,10 +571,20 @@ app.MapPost("/airgap/v1/vex/import", async (
return Results.Accepted($"/airgap/v1/vex/import/{request.BundleId}", new
{
bundleId = request.BundleId,
generation = request.MirrorGeneration
generation = request.MirrorGeneration,
manifest = manifestPath,
evidence = evidenceLockerPath,
manifestSha256 = manifestHash
});
});
static string ComputeSha256(string value)
{
var bytes = Encoding.UTF8.GetBytes(value);
var hash = System.Security.Cryptography.SHA256.HashData(bytes);
return "sha256:" + Convert.ToHexString(hash).ToLowerInvariant();
}
app.MapPost("/v1/attestations/verify", async (
[FromServices] IVexAttestationClient attestationClient,
[FromBody] AttestationVerifyRequest request,
@@ -1548,6 +1616,15 @@ app.MapGet("/v1/vex/linksets", async (HttpContext _, CancellationToken __) =>
app.Run();
internal sealed record ExcititorTimelineEvent(
string Type,
string Tenant,
string Source,
int Count,
int Errors,
string? TraceId,
string OccurredAt);
public partial class Program;
internal sealed record StatusResponse(DateTimeOffset UtcNow, string MongoBucket, int InlineThreshold, string[] ArtifactStores);

View File

@@ -0,0 +1,65 @@
using System;
using System.Linq;
using Microsoft.Extensions.Logging;
using Microsoft.Extensions.Options;
using StellaOps.Excititor.WebService.Contracts;
using StellaOps.Excititor.WebService.Options;
namespace StellaOps.Excititor.WebService.Services;
internal sealed class AirgapModeEnforcer
{
private readonly AirgapOptions _options;
private readonly ILogger<AirgapModeEnforcer> _logger;
public AirgapModeEnforcer(IOptions<AirgapOptions> options, ILogger<AirgapModeEnforcer> logger)
{
_options = options.Value;
_logger = logger;
}
public bool Validate(AirgapImportRequest request, out string? errorCode, out string? message)
{
errorCode = null;
message = null;
if (!_options.SealedMode)
{
return true;
}
if (_options.MirrorOnly && !string.IsNullOrWhiteSpace(request.PayloadUrl) && LooksLikeExternal(request.PayloadUrl))
{
errorCode = "AIRGAP_EGRESS_BLOCKED";
message = "Sealed mode forbids external payload URLs; stage bundle via mirror/portable media.";
_logger.LogWarning("Blocked airgap import because payloadUrl points to external location: {Url}", request.PayloadUrl);
return false;
}
if (_options.TrustedPublishers.Count > 0 && !string.IsNullOrWhiteSpace(request.Publisher))
{
var allowed = _options.TrustedPublishers.Any(p => string.Equals(p, request.Publisher, StringComparison.OrdinalIgnoreCase));
if (!allowed)
{
errorCode = "AIRGAP_SOURCE_UNTRUSTED";
message = $"Publisher '{request.Publisher}' is not allowlisted for sealed-mode imports.";
_logger.LogWarning("Blocked airgap import because publisher {Publisher} is not allowlisted.", request.Publisher);
return false;
}
}
return true;
}
private static bool LooksLikeExternal(string url)
{
if (string.IsNullOrWhiteSpace(url))
{
return false;
}
return url.StartsWith("http://", StringComparison.OrdinalIgnoreCase)
|| url.StartsWith("https://", StringComparison.OrdinalIgnoreCase)
|| url.StartsWith("ftp://", StringComparison.OrdinalIgnoreCase);
}
}

View File

@@ -0,0 +1,27 @@
using System;
using MongoDB.Bson.Serialization.Attributes;
namespace StellaOps.Excititor.Storage.Mongo;
[BsonIgnoreExtraElements]
public sealed class AirgapTimelineEntry
{
public string EventType { get; set; } = string.Empty;
public DateTimeOffset CreatedAt { get; set; } = DateTimeOffset.UtcNow;
public string TenantId { get; set; } = "default";
public string BundleId { get; set; } = string.Empty;
public string MirrorGeneration { get; set; } = string.Empty;
public int? StalenessSeconds { get; set; }
= null;
public string? ErrorCode { get; set; }
= null;
public string? Message { get; set; }
= null;
}

View File

@@ -316,6 +316,8 @@ public sealed class AirgapImportRecord
[BsonId]
public string Id { get; set; } = default!;
public string TenantId { get; set; } = "default";
public string BundleId { get; set; } = default!;
public string MirrorGeneration { get; set; } = default!;
@@ -333,6 +335,14 @@ public sealed class AirgapImportRecord
public string? TransparencyLog { get; set; } = null;
public DateTimeOffset ImportedAt { get; set; } = DateTimeOffset.UtcNow;
public string PortableManifestPath { get; set; } = string.Empty;
public string PortableManifestHash { get; set; } = string.Empty;
public string EvidenceLockerPath { get; set; } = string.Empty;
public List<AirgapTimelineEntry> Timeline { get; set; } = new();
}
[BsonIgnoreExtraElements]

View File

@@ -31,8 +31,15 @@ public sealed class AirgapImportValidatorTests
[Fact]
public void Validate_InvalidHash_ReturnsError()
{
var req = Valid();
req.PayloadHash = "not-a-hash";
var req = new AirgapImportRequest
{
BundleId = "bundle-123",
MirrorGeneration = "5",
Publisher = "stellaops",
PayloadHash = "not-a-hash",
Signature = Convert.ToBase64String(new byte[] { 5, 6, 7 }),
SignedAt = _now
};
var result = _validator.Validate(req, _now);
@@ -42,8 +49,15 @@ public sealed class AirgapImportValidatorTests
[Fact]
public void Validate_InvalidSignature_ReturnsError()
{
var req = Valid();
req.Signature = "???";
var req = new AirgapImportRequest
{
BundleId = "bundle-123",
MirrorGeneration = "5",
Publisher = "stellaops",
PayloadHash = "sha256:" + new string('b', 64),
Signature = "???",
SignedAt = _now
};
var result = _validator.Validate(req, _now);
@@ -53,8 +67,15 @@ public sealed class AirgapImportValidatorTests
[Fact]
public void Validate_MirrorGenerationNonNumeric_ReturnsError()
{
var req = Valid();
req.MirrorGeneration = "abc";
var req = new AirgapImportRequest
{
BundleId = "bundle-123",
MirrorGeneration = "abc",
Publisher = "stellaops",
PayloadHash = "sha256:" + new string('b', 64),
Signature = Convert.ToBase64String(new byte[] { 5, 6, 7 }),
SignedAt = _now
};
var result = _validator.Validate(req, _now);
@@ -64,8 +85,15 @@ public sealed class AirgapImportValidatorTests
[Fact]
public void Validate_SignedAtTooOld_ReturnsError()
{
var req = Valid();
req.SignedAt = _now.AddSeconds(-10);
var req = new AirgapImportRequest
{
BundleId = "bundle-123",
MirrorGeneration = "5",
Publisher = "stellaops",
PayloadHash = "sha256:" + new string('b', 64),
Signature = Convert.ToBase64String(new byte[] { 5, 6, 7 }),
SignedAt = _now.AddSeconds(-10)
};
var result = _validator.Validate(req, _now);

View File

@@ -0,0 +1,44 @@
using Microsoft.Extensions.Logging.Abstractions;
using Microsoft.Extensions.Options;
using StellaOps.Excititor.WebService.Contracts;
using StellaOps.Excititor.WebService.Options;
using StellaOps.Excititor.WebService.Services;
using Xunit;
namespace StellaOps.Excititor.WebService.Tests;
public class AirgapModeEnforcerTests
{
[Fact]
public void Validate_Allows_WhenNotSealed()
{
var enforcer = new AirgapModeEnforcer(Microsoft.Extensions.Options.Options.Create(new AirgapOptions { SealedMode = false }), NullLogger<AirgapModeEnforcer>.Instance);
var ok = enforcer.Validate(new AirgapImportRequest { PayloadUrl = "https://example.com" }, out var code, out var message);
Assert.True(ok);
Assert.Null(code);
Assert.Null(message);
}
[Fact]
public void Validate_Blocks_ExternalUrl_WhenSealed()
{
var enforcer = new AirgapModeEnforcer(Microsoft.Extensions.Options.Options.Create(new AirgapOptions { SealedMode = true, MirrorOnly = true }), NullLogger<AirgapModeEnforcer>.Instance);
var ok = enforcer.Validate(new AirgapImportRequest { PayloadUrl = "https://example.com" }, out var code, out var message);
Assert.False(ok);
Assert.Equal("AIRGAP_EGRESS_BLOCKED", code);
Assert.NotNull(message);
}
[Fact]
public void Validate_Blocks_Untrusted_Publisher_WhenAllowlistSet()
{
var enforcer = new AirgapModeEnforcer(Microsoft.Extensions.Options.Options.Create(new AirgapOptions { SealedMode = true, TrustedPublishers = { "mirror-a" } }), NullLogger<AirgapModeEnforcer>.Instance);
var ok = enforcer.Validate(new AirgapImportRequest { Publisher = "mirror-b" }, out var code, out var message);
Assert.False(ok);
Assert.Equal("AIRGAP_SOURCE_UNTRUSTED", code);
Assert.NotNull(message);
}
}

View File

@@ -29,6 +29,8 @@
<ItemGroup>
<Compile Remove="**/*.cs" />
<Compile Include="AirgapImportEndpointTests.cs" />
<Compile Include="AirgapImportValidatorTests.cs" />
<Compile Include="AirgapModeEnforcerTests.cs" />
<Compile Include="EvidenceTelemetryTests.cs" />
<Compile Include="DevRuntimeEnvironmentStub.cs" />
<Compile Include="TestAuthentication.cs" />