Add new features and tests for AirGap and Time modules
Some checks failed
Docs CI / lint-and-preview (push) Has been cancelled

- Introduced `SbomService` tasks documentation.
- Updated `StellaOps.sln` to include new projects: `StellaOps.AirGap.Time` and `StellaOps.AirGap.Importer`.
- Added unit tests for `BundleImportPlanner`, `DsseVerifier`, `ImportValidator`, and other components in the `StellaOps.AirGap.Importer.Tests` namespace.
- Implemented `InMemoryBundleRepositories` for testing bundle catalog and item repositories.
- Created `MerkleRootCalculator`, `RootRotationPolicy`, and `TufMetadataValidator` tests.
- Developed `StalenessCalculator` and `TimeAnchorLoader` tests in the `StellaOps.AirGap.Time.Tests` namespace.
- Added `fetch-sbomservice-deps.sh` script for offline dependency fetching.
This commit is contained in:
master
2025-11-20 23:29:54 +02:00
parent 65b1599229
commit 79b8e53441
182 changed files with 6660 additions and 1242 deletions

View File

@@ -608,12 +608,14 @@ excititor:
### 9.1 WebService endpoints
With storage configured, the WebService exposes the following ingress and diagnostic APIs:
With storage configured, the WebService exposes the following ingress and diagnostic APIs (deterministic ordering, offline-friendly):
* `GET /excititor/status` returns the active storage configuration and registered artifact stores.
* `GET /excititor/health` simple liveness probe.
* `POST /excititor/statements` accepts normalized VEX statements and persists them via `IVexClaimStore`; use this for migrations/backfills.
* `GET /excititor/statements/{vulnId}/{productKey}?since=` returns the immutable statement log for a vulnerability/product pair.
* `POST /vex/evidence/chunks` submits aggregation-only chunks (OpenAPI: `schemas/vex-chunk-api.yaml`); responds with deterministic `chunk_digest` and queue id. Telemetry published under meter `StellaOps.Excititor.Chunks` (see Operations).
* `POST /v1/attestations/verify` verifies Evidence Locker attestations for exports/chunks using `IVexAttestationVerifier`; returns `{ valid, diagnostics }` (deterministic key order). Aligns with Evidence Locker contract v1.
* `POST /excititor/resolve` requires `vex.read` scope; accepts up to 256 `(vulnId, productKey)` pairs via `productKeys` or `purls` and returns deterministic consensus results, decision telemetry, and a signed envelope (`artifact` digest, optional signer signature, optional attestation metadata + DSSE envelope). Returns **409 Conflict** when the requested `policyRevisionId` mismatches the active snapshot.
Run the ingestion endpoint once after applying migration `20251019-consensus-signals-statements` to repopulate historical statements with the new severity/KEV/EPSS signal fields.

View File

@@ -0,0 +1,43 @@
# Excititor Attestation Plan (Sprint 110)
## Goals
- Align Excititor chunk API and attestation envelopes with Evidence Locker contract.
- Provide offline-ready chunk submission/attestation flow for VEX evidence.
## Chunk API shape (`/vex/evidence/chunks`)
- POST body (NDJSON, deterministic order by `chunk_id`):
```json
{
"chunk_id": "uuid",
"tenant": "acme",
"source": "ghsa",
"schema": "stellaops.vex.chunk.v1",
"items": [ {"advisory_id":"GHSA-123","status":"affected","purl":"pkg:npm/foo@1.0.0"} ],
"provenance": {"fetched_at":"2025-11-20T00:00:00Z","artifact_sha":"abc"}
}
```
- At submission, Excititor returns `chunk_digest` (sha256 of canonical JSON) and queue id.
## Attestation envelope
- Subject: `chunk_digest` from above.
- Predicates attached:
- `stellaops.vex.chunk.meta.v1` (tenant, source, schema version, item count).
- `stellaops.vex.chunk.integrity.v1` (sha256 per item block, canonical order).
- Optional `stellaops.transparency.v1` (Rekor UUID/logIndex) when online.
- Envelope format: DSSE using Evidence Locker provider registry; signing profile mirrors Evidence Locker bundle profile for tenant.
## DSSE bundling rules
- Deterministic JSON (sorted keys) before hashing.
- Canonical NDJSON for chunk payload; no gzip inside envelope.
- Attach verification report alongside attestation as `chunk-verify.json` (hashes + signature check results).
## Sample payloads
- `docs/samples/excititor/chunk-sample.ndjson`
- `docs/samples/excititor/chunk-attestation-sample.json`
## Integration points
- Evidence Locker contract v1 (see `docs/modules/evidence-locker/attestation-contract.md`).
- Concelier LNM schemas (observations remain aggregation-only; attestation is evidence, not merge).
## Ownership
- Excititor Guild (primary); Evidence Locker Guild reviewer.

View File

@@ -0,0 +1,36 @@
# Connector signer metadata (v1.0.0)
**Scope.** Defines the canonical, offline-friendly metadata for Excititor connectors that validate signed feeds (MSRC CSAF, Oracle OVAL, Ubuntu OVAL, StellaOps mirror OpenVEX). The file is consumed by WebService/Worker composition roots and by Offline Kits to pin trust material deterministically.
**Location & format.**
- Schema: `docs/modules/excititor/schemas/connector-signer-metadata.schema.json` (JSON Schema 202012).
- Sample: `docs/samples/excititor/connector-signer-metadata-sample.json` (aligns with schema).
- Expected production artifact: NDJSON or JSON stamped per release; store in offline kits alongside connector bundles.
## Required fields (summary)
- `schemaVersion` — must be `1.0.0`.
- `generatedAt` — ISO-8601 UTC timestamp for the metadata file.
- `connectors[]` — one entry per connector:
- `connectorId` — stable slug, e.g., `excititor-msrc-csaf`.
- `provider { name, slug }` — human label and slug.
- `issuerTier``tier-0`, `tier-1`, `tier-2`, or `untrusted` (aligns with trust weighting).
- `signers[]` — one per signing path; each has `usage` (`csaf|oval|openvex|bulk-meta|attestation`) and `fingerprints[]` (algorithm + format + value). Optional `keyLocator` and `certificateChain` for offline key retrieval.
- `bundle` — reference to the sealed bundle containing the feed/signing material (`kind`: `oci-referrer|oci-tag|file|tuf`, plus `uri`, optional `digest`, `publishedAt`).
- Optional `validFrom`, `validTo`, `revoked`, `notes` for rollover and incident handling.
## Rollover / migration guidance
1) **Author the metadata** using the schema and place the JSON next to connector bundles in the offline kit (`out/connectors/<provider>/signer-metadata.json`).
2) **Validate** with `dotnet tool run njsonschema validate connector-signer-metadata.schema.json connector-signer-metadata.json` (or `ajv validate`).
3) **Wire connector code** to load the file on startup (Worker + WebService) and pin signers per `connectorId`; reject feeds whose fingerprints are absent or marked `revoked=true` or out of `validFrom/To` range.
- Connectors look for `STELLAOPS_CONNECTOR_SIGNER_METADATA_PATH` (absolute/relative) and enrich provenance metadata automatically when present.
4) **Rollover keys** by appending a new `signers` entry and setting a future `validFrom`; keep the previous signer until all mirrors have caught up. Use `issuerTier` downgrades to quarantine while keeping history.
5) **Mirror references**: store the referenced bundles/keys under OCI tags or TUF targets already shipped in the offline kit so no live network is required.
6) **Record decisions** in sprint Decisions & Risks when changing trust tiers or fingerpints; update this doc if formats change.
## Sample entries (non-production)
See `docs/samples/excititor/connector-signer-metadata-sample.json` for MSRC, Oracle, Ubuntu, and StellaOps example entries. These fingerprints are illustrative only; replace with real values before shipping.
## Consumer expectations
- Deterministic: sort connectors alphabetically before persistence; avoid clock-based defaults.
- Offline-first: all `keyLocator`/`bundle.uri` values must resolve inside the air-gap kit (OCI/TUF/file).
- Observability: emit a structured warning when metadata is missing or stale (>7 days) and fail closed for missing signers.

View File

@@ -0,0 +1,24 @@
# Using the Chunk API
Endpoint: `POST /vex/evidence/chunks`
- Content-Type: `application/x-ndjson`
- See schema: `docs/modules/excititor/schemas/vex-chunk-api.yaml`
Response: `202 Accepted`
```json
{ "chunk_digest": "sha256:…", "queue_id": "uuid" }
```
Operational notes
- Deterministic hashing: server recomputes `chunk_digest` from canonical JSON; mismatches return 400.
- Limits: default 500 items, max 2000 (aligned with Program.cs guard).
- Telemetry: metrics under `StellaOps.Excititor.Chunks` (see chunk-telemetry.md).
- Headers: correlation/trace headers echoed (`X-Stella-TraceId`, `X-Stella-CorrelationId`).
Example curl
```bash
curl -X POST https://excitor.local/vex/evidence/chunks \
-H "Authorization: Bearer <token>" \
-H "Content-Type: application/x-ndjson" \
--data-binary @docs/samples/excititor/chunk-sample.ndjson
```

View File

@@ -0,0 +1,26 @@
# Excititor Chunk Telemetry (Sprint 110)
## Metrics (Meter: `StellaOps.Excititor.Chunks`)
- `vex_chunks_ingested_total` (counter) — tags: `tenant`, `source`, `status` (`accepted|rejected`), `reason` (nullable for accepted). Increments per chunk submitted.
- `vex_chunks_item_count` (histogram, unit=items) — records item count per chunk.
- `vex_chunks_payload_bytes` (histogram, unit=bytes) — measured from NDJSON payload length.
- `vex_chunks_latency_ms` (histogram) — end-to-end ingestion latency per request.
## Logs
- `vex.chunk.ingest.accepted` — includes `chunk_id`, `tenant`, `source`, `item_count`, `chunk_digest`.
- `vex.chunk.ingest.rejected` — includes `chunk_id`, `tenant`, `source`, `reason`, validation errors (summarized).
## Wiring steps
1. Register `ChunkTelemetry` as singleton with shared `Meter` instance.
2. In `/vex/evidence/chunks` handler, compute `chunk_digest` deterministically from canonical JSON and emit counters/histograms via `ChunkTelemetry`.
3. Log using structured templates above; avoid request bodies in logs.
4. Expose metrics via default ASP.NET metrics export (Prometheus/OpenTelemetry) already configured in WebService.
## Determinism & offline posture
- Do not include host-specific paths or timestamps in metric dimensions.
- Histogram buckets: use standard OTEL defaults; no runtime-generated buckets.
- Keep meter name stable; adding new instruments requires version note in sprint Decisions & Risks.
## Ownership
- Implementer: Excititor Observability Guild
- Reviewers: Evidence Locker Guild (for parity with attestation metrics)

View File

@@ -0,0 +1,125 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://stellaops.dev/schemas/excititor/connector-signer-metadata.schema.json",
"title": "Excititor Connector Signer Metadata",
"type": "object",
"additionalProperties": false,
"required": ["schemaVersion", "generatedAt", "connectors"],
"properties": {
"schemaVersion": {
"type": "string",
"pattern": "^1\\.0\\.0$"
},
"generatedAt": {
"type": "string",
"format": "date-time"
},
"connectors": {
"type": "array",
"minItems": 1,
"items": {
"$ref": "#/$defs/connector"
}
}
},
"$defs": {
"connector": {
"type": "object",
"additionalProperties": false,
"required": [
"connectorId",
"provider",
"issuerTier",
"signers"
],
"properties": {
"connectorId": {
"type": "string",
"pattern": "^[a-z0-9:-\\.]+$"
},
"provider": {
"type": "object",
"additionalProperties": false,
"required": ["name", "slug"],
"properties": {
"name": { "type": "string", "minLength": 3 },
"slug": { "type": "string", "pattern": "^[a-z0-9-]+$" }
}
},
"issuerTier": {
"type": "string",
"enum": ["tier-0", "tier-1", "tier-2", "untrusted"]
},
"signers": {
"type": "array",
"minItems": 1,
"items": { "$ref": "#/$defs/signer" }
},
"bundle": { "$ref": "#/$defs/bundleRef" },
"validFrom": { "type": "string", "format": "date" },
"validTo": { "type": "string", "format": "date" },
"revoked": { "type": "boolean", "default": false },
"notes": { "type": "string", "maxLength": 2000 }
}
},
"signer": {
"type": "object",
"additionalProperties": false,
"required": ["usage", "fingerprints"],
"properties": {
"usage": {
"type": "string",
"enum": ["csaf", "oval", "openvex", "bulk-meta", "attestation"]
},
"fingerprints": {
"type": "array",
"minItems": 1,
"items": { "$ref": "#/$defs/fingerprint" }
},
"keyLocator": {
"type": "string",
"description": "Path or URL (mirror/OCI/TUF) where the signing key or certificate chain can be retrieved in offline kits."
},
"certificateChain": {
"type": "array",
"items": { "type": "string" },
"description": "Optional PEM-encoded certificates for x509/cosign keys."
}
}
},
"fingerprint": {
"type": "object",
"additionalProperties": false,
"required": ["alg", "value"],
"properties": {
"alg": {
"type": "string",
"enum": ["sha256", "sha512", "sha1"]
},
"format": {
"type": "string",
"enum": ["pgp", "x509-spki", "x509-ski", "cosign", "pem"]
},
"value": {
"type": "string",
"minLength": 16,
"maxLength": 128
}
}
},
"bundleRef": {
"type": "object",
"additionalProperties": false,
"required": ["kind", "uri"],
"properties": {
"kind": {
"type": "string",
"enum": ["oci-referrer", "oci-tag", "file", "tuf"]
},
"uri": { "type": "string", "minLength": 8 },
"digest": { "type": "string", "minLength": 32 },
"publishedAt": { "type": "string", "format": "date-time" }
}
}
}
}

View File

@@ -0,0 +1,82 @@
openapi: 3.1.0
info:
title: StellaOps Excititor Chunk API
version: "0.1.0"
description: |
Frozen for Sprint 110 (EXCITITOR-AIAI-31-002). Aligns with Evidence Locker attestation contract v1.
servers:
- url: https://excitor.local
paths:
/vex/evidence/chunks:
post:
summary: Submit VEX evidence chunk (aggregation-only)
requestBody:
required: true
content:
application/x-ndjson:
schema:
$ref: '#/components/schemas/VexChunk'
responses:
'202':
description: Accepted for processing
content:
application/json:
schema:
type: object
required: [chunk_digest, queue_id]
properties:
chunk_digest:
type: string
description: sha256 of canonical chunk JSON
queue_id:
type: string
description: Background job identifier
'400':
description: Validation error
components:
schemas:
VexChunk:
type: object
required: [chunk_id, tenant, source, schema, items, provenance]
properties:
chunk_id:
type: string
format: uuid
tenant:
type: string
source:
type: string
description: feed id (e.g., ghsa, nvd)
schema:
type: string
enum: [stellaops.vex.chunk.v1]
items:
type: array
items:
type: object
required: [advisory_id, status, purl]
properties:
advisory_id:
type: string
status:
type: string
enum: [affected, unaffected, under_investigation, fixed, unknown]
purl:
type: string
justification:
type: string
last_observed:
type: string
format: date-time
provenance:
type: object
required: [fetched_at, artifact_sha]
properties:
fetched_at:
type: string
format: date-time
artifact_sha:
type: string
signature:
type: object
nullable: true