feat: Initialize Zastava Webhook service with TLS and Authority authentication

- Added Program.cs to set up the web application with Serilog for logging, health check endpoints, and a placeholder admission endpoint. - Configured Kestrel server to use TLS 1.3 and handle client certificates appropriately. - Created StellaOps.Zastava.Webhook.csproj with necessary dependencies including Serilog and Polly. - Documented tasks in TASKS.md for the Zastava Webhook project, outlining current work and exit criteria for each task.
2025-10-19 18:36:22 +03:00
parent 7e2fa0a42a
commit 5ce40d2eeb
966 changed files with 91038 additions and 1850 deletions
--- a/docs/scanner-core-contracts.md
+++ b/docs/scanner-core-contracts.md
@@ -2,18 +2,58 @@

 The **Scanner Core** library provides shared contracts, observability helpers, and security utilities consumed by `Scanner.WebService`, `Scanner.Worker`, analyzers, and tooling. These primitives guarantee deterministic identifiers, timestamps, and log context for all scanning flows.

-## DTOs
+## Canonical DTOs

 - `ScanJob` & `ScanJobStatus` – canonical job metadata (image reference/digest, tenant, correlation ID, timestamps, failure details). Constructors normalise timestamps to UTC microsecond precision and canonicalise image digests. Round-trips with `JsonSerializerDefaults.Web` using `ScannerJsonOptions`.
 - `ScanProgressEvent` & `ScanStage`/`ScanProgressEventKind` – stage-level progress surface for queue/stream consumers. Includes deterministic sequence numbers, optional progress percentage, attributes, and attached `ScannerError`.
 - `ScannerError` & `ScannerErrorCode` – shared error taxonomy spanning queue, analyzers, storage, exporters, and signing. Carries severity, retryability, structured details, and microsecond-precision timestamps.
 - `ScanJobId` – strongly-typed identifier rendered as `Guid` (lowercase `N` format) with deterministic parsing.

+### Canonical JSON samples
+
+The golden fixtures consumed by `ScannerCoreContractsTests` document the wire shape shared with downstream services. They live under `src/StellaOps.Scanner.Core.Tests/Fixtures/` and a representative extract is shown below.
+
+```json
+{
+  "id": "8f4cc9c582454b9d9b4f5ae049631b7d",
+  "status": "running",
+  "imageReference": "registry.example.com/stellaops/scanner:1.2.3",
+  "imageDigest": "sha256:abcdef",
+  "createdAt": "2025-10-18T14:30:15.123456+00:00",
+  "updatedAt": "2025-10-18T14:30:20.123456+00:00",
+  "correlationId": "scan-analyzeoperatingsystem-8f4cc9c582454b9d9b4f5ae049631b7d",
+  "tenantId": "tenant-a",
+  "metadata": {
+    "requestId": "req-1234",
+    "source": "ci"
+  },
+  "failure": {
+    "code": "analyzerFailure",
+    "severity": "error",
+    "message": "Analyzer failed to parse layer",
+    "timestamp": "2025-10-18T14:30:15.123456+00:00",
+    "retryable": false,
+    "stage": "AnalyzeOperatingSystem",
+    "component": "os-analyzer",
+    "details": {
+      "layerDigest": "sha256:deadbeef",
+      "attempt": "1"
+    }
+  }
+}
+```
+
+Progress events follow the same conventions (`jobId`, `stage`, `kind`, `timestamp`, `attributes`, optional embedded `ScannerError`). The fixtures are verified via deterministic JSON comparison in every CI run.
+
 ## Deterministic helpers

 - `ScannerIdentifiers` – derives `ScanJobId`, correlation IDs, and SHA-256 hashes from normalised inputs (image reference/digest, tenant, salt). Ensures case-insensitive stability and reproducible metric keys.
 - `ScannerTimestamps` – trims to microsecond precision, provides ISO-8601 (`yyyy-MM-ddTHH:mm:ss.ffffffZ`) rendering, and parsing helpers.
 - `ScannerJsonOptions` – standard JSON options (web defaults, camel-case enums) shared by services/tests.
+- `ScanAnalysisStore` & `ScanAnalysisKeys` – shared in-memory analysis cache flowing through Worker stages. OS analyzers populate
+  `analysis.os.packages` (raw output), `analysis.os.fragments` (per-analyzer component fragments), and merge into
+  `analysis.layers.fragments` so emit/diff stages can compose SBOMs and diffs without knowledge of individual analyzer
+  implementations.

 ## Observability primitives

@@ -22,9 +62,74 @@ The **Scanner Core** library provides shared contracts, observability helpers, a
 - `ScannerCorrelationContext` & `ScannerCorrelationContextAccessor` – ambient correlation propagation via `AsyncLocal` for log scopes, metrics, and diagnostics.
 - `ScannerLogExtensions` – `ILogger` scopes for jobs/progress events with automatic correlation context push, minimal allocations, and consistent structured fields.

+### Observability overhead validation
+
+A micro-benchmark executed on 2025-10-19 (4 vCPU runner, .NET 10.0.100-rc.1) measured the average scope cost across 1 000 000 iterations:
+
+| Scope | Mean (µs/call) |
+|-------|----------------|
+| `BeginScanScope` (logger attached) | 0.80 |
+| `BeginScanScope` (noop logger) | 0.31 |
+| `BeginProgressScope` | 0.57 |
+
+To reproduce, run `dotnet test src/StellaOps.Scanner.Core.Tests -c Release` (see `ScannerLogExtensionsPerformanceTests`) or copy the snippet below into a throwaway `dotnet run` console project and execute it with `dotnet run -c Release`:
+
+```csharp
+using System.Collections.Generic;
+using System.Diagnostics;
+using Microsoft.Extensions.Logging;
+using StellaOps.Scanner.Core.Contracts;
+using StellaOps.Scanner.Core.Observability;
+using StellaOps.Scanner.Core.Utility;
+
+var factory = LoggerFactory.Create(builder => builder.AddFilter(static _ => true));
+var logger = factory.CreateLogger("bench");
+
+var jobId = ScannerIdentifiers.CreateJobId("registry.example.com/stellaops/scanner:1.2.3", "sha256:abcdef", "tenant-a", "benchmark");
+var correlationId = ScannerIdentifiers.CreateCorrelationId(jobId, nameof(ScanStage.AnalyzeOperatingSystem));
+var now = ScannerTimestamps.Normalize(new DateTimeOffset(2025, 10, 19, 12, 0, 0, TimeSpan.Zero));
+
+var job = new ScanJob(jobId, ScanJobStatus.Running, "registry.example.com/stellaops/scanner:1.2.3", "sha256:abcdef", now, now, correlationId, "tenant-a", new Dictionary<string, string>(StringComparer.Ordinal) { ["requestId"] = "req-bench" });
+var progress = new ScanProgressEvent(jobId, ScanStage.AnalyzeOperatingSystem, ScanProgressEventKind.Progress, 42, now, 10.5, "benchmark", new Dictionary<string, string>(StringComparer.Ordinal) { ["sample"] = "true" });
+
+Console.WriteLine("Scanner Core Observability micro-bench (1,000,000 iterations)");
+Report("BeginScanScope (logger)", Measure(static ctx => ctx.Logger.BeginScanScope(ctx.Job, ctx.Stage, ctx.Component), new ScopeContext(logger, job, nameof(ScanStage.AnalyzeOperatingSystem), "os-analyzer")));
+Report("BeginScanScope (no logger)", Measure(static ctx => ScannerLogExtensions.BeginScanScope(null, ctx.Job, ctx.Stage, ctx.Component), new ScopeContext(logger, job, nameof(ScanStage.AnalyzeOperatingSystem), "os-analyzer")));
+Report("BeginProgressScope", Measure(static ctx => ctx.Logger.BeginProgressScope(ctx.Progress!, ctx.Component), new ScopeContext(logger, job, nameof(ScanStage.AnalyzeOperatingSystem), "os-analyzer", progress)));
+
+static double Measure(Func<ScopeContext, IDisposable> factory, ScopeContext context)
+{
+    const int iterations = 1_000_000;
+    for (var i = 0; i < 10_000; i++)
+    {
+        using var scope = factory(context);
+    }
+
+    GC.Collect();
+    GC.WaitForPendingFinalizers();
+    GC.Collect();
+
+    var sw = Stopwatch.StartNew();
+    for (var i = 0; i < iterations; i++)
+    {
+        using var scope = factory(context);
+    }
+
+    sw.Stop();
+    return sw.Elapsed.TotalSeconds * 1_000_000 / iterations;
+}
+
+static void Report(string label, double microseconds)
+    => Console.WriteLine($"{label,-28}: {microseconds:F3} µs");
+
+readonly record struct ScopeContext(ILogger Logger, ScanJob Job, string? Stage, string? Component, ScanProgressEvent? Progress = null);
+```
+
+Both guardrails enforce the ≤ 5 µs acceptance target for SP9-G1.
+
 ## Security utilities

- `AuthorityTokenSource` – caches short-lived OpToks per audience+scope using deterministic keys and refresh skew (default 30 s). Integrates with `StellaOps.Auth.Client`.
+- `AuthorityTokenSource` – caches short-lived OpToks per audience+scope using deterministic keys and refresh skew (default 30 s). Integrates with `StellaOps.Auth.Client`.
 - `DpopProofValidator` – validates DPoP proofs (alg allowlist, `htm`/`htu`, nonce, replay window, signature) backed by pluggable `IDpopReplayCache`. Ships with `InMemoryDpopReplayCache` for restart-only deployments.
 - `RestartOnlyPluginGuard` – enforces restart-time plug-in registration (deterministic path normalisation; throws if new plug-ins added post-seal).
 - `ServiceCollectionExtensions.AddScannerAuthorityCore` – DI helper wiring Authority client, OpTok source, DPoP validation, replay cache, and plug-in guard.
@@ -33,10 +138,10 @@ The **Scanner Core** library provides shared contracts, observability helpers, a

 Unit tests (`StellaOps.Scanner.Core.Tests`) assert:

- DTO JSON round-trips are stable and deterministic.
+- DTO JSON round-trips are stable and deterministic (`ScannerCoreContractsTests` + golden fixtures).
 - Identifier/hash helpers ignore case and emit lowercase hex.
 - Timestamp normalisation retains UTC semantics.
- Log scopes push/pop correlation context predictably.
+- Log scopes push/pop correlation context predictably while staying under the 5 µs envelope.
 - Authority token caching honours refresh skew and invalidation.
 - DPoP validator accepts valid proofs, rejects nonce mismatch/replay, and enforces signature validation.
 - Restart-only plug-in guard blocks runtime additions post-seal.