Add new features and tests for AirGap and Time modules
Some checks failed
Docs CI / lint-and-preview (push) Has been cancelled
Some checks failed
Docs CI / lint-and-preview (push) Has been cancelled
- Introduced `SbomService` tasks documentation. - Updated `StellaOps.sln` to include new projects: `StellaOps.AirGap.Time` and `StellaOps.AirGap.Importer`. - Added unit tests for `BundleImportPlanner`, `DsseVerifier`, `ImportValidator`, and other components in the `StellaOps.AirGap.Importer.Tests` namespace. - Implemented `InMemoryBundleRepositories` for testing bundle catalog and item repositories. - Created `MerkleRootCalculator`, `RootRotationPolicy`, and `TufMetadataValidator` tests. - Developed `StalenessCalculator` and `TimeAnchorLoader` tests in the `StellaOps.AirGap.Time.Tests` namespace. - Added `fetch-sbomservice-deps.sh` script for offline dependency fetching.
This commit is contained in:
@@ -608,12 +608,14 @@ excititor:
|
||||
|
||||
### 9.1 WebService endpoints
|
||||
|
||||
With storage configured, the WebService exposes the following ingress and diagnostic APIs:
|
||||
With storage configured, the WebService exposes the following ingress and diagnostic APIs (deterministic ordering, offline-friendly):
|
||||
|
||||
* `GET /excititor/status` – returns the active storage configuration and registered artifact stores.
|
||||
* `GET /excititor/health` – simple liveness probe.
|
||||
* `POST /excititor/statements` – accepts normalized VEX statements and persists them via `IVexClaimStore`; use this for migrations/backfills.
|
||||
* `GET /excititor/statements/{vulnId}/{productKey}?since=` – returns the immutable statement log for a vulnerability/product pair.
|
||||
* `POST /vex/evidence/chunks` – submits aggregation-only chunks (OpenAPI: `schemas/vex-chunk-api.yaml`); responds with deterministic `chunk_digest` and queue id. Telemetry published under meter `StellaOps.Excititor.Chunks` (see Operations).
|
||||
* `POST /v1/attestations/verify` – verifies Evidence Locker attestations for exports/chunks using `IVexAttestationVerifier`; returns `{ valid, diagnostics }` (deterministic key order). Aligns with Evidence Locker contract v1.
|
||||
* `POST /excititor/resolve` – requires `vex.read` scope; accepts up to 256 `(vulnId, productKey)` pairs via `productKeys` or `purls` and returns deterministic consensus results, decision telemetry, and a signed envelope (`artifact` digest, optional signer signature, optional attestation metadata + DSSE envelope). Returns **409 Conflict** when the requested `policyRevisionId` mismatches the active snapshot.
|
||||
|
||||
Run the ingestion endpoint once after applying migration `20251019-consensus-signals-statements` to repopulate historical statements with the new severity/KEV/EPSS signal fields.
|
||||
|
||||
43
docs/modules/excititor/attestation-plan.md
Normal file
43
docs/modules/excititor/attestation-plan.md
Normal file
@@ -0,0 +1,43 @@
|
||||
# Excititor Attestation Plan (Sprint 110)
|
||||
|
||||
## Goals
|
||||
- Align Excititor chunk API and attestation envelopes with Evidence Locker contract.
|
||||
- Provide offline-ready chunk submission/attestation flow for VEX evidence.
|
||||
|
||||
## Chunk API shape (`/vex/evidence/chunks`)
|
||||
- POST body (NDJSON, deterministic order by `chunk_id`):
|
||||
```json
|
||||
{
|
||||
"chunk_id": "uuid",
|
||||
"tenant": "acme",
|
||||
"source": "ghsa",
|
||||
"schema": "stellaops.vex.chunk.v1",
|
||||
"items": [ {"advisory_id":"GHSA-123","status":"affected","purl":"pkg:npm/foo@1.0.0"} ],
|
||||
"provenance": {"fetched_at":"2025-11-20T00:00:00Z","artifact_sha":"abc"}
|
||||
}
|
||||
```
|
||||
- At submission, Excititor returns `chunk_digest` (sha256 of canonical JSON) and queue id.
|
||||
|
||||
## Attestation envelope
|
||||
- Subject: `chunk_digest` from above.
|
||||
- Predicates attached:
|
||||
- `stellaops.vex.chunk.meta.v1` (tenant, source, schema version, item count).
|
||||
- `stellaops.vex.chunk.integrity.v1` (sha256 per item block, canonical order).
|
||||
- Optional `stellaops.transparency.v1` (Rekor UUID/logIndex) when online.
|
||||
- Envelope format: DSSE using Evidence Locker provider registry; signing profile mirrors Evidence Locker bundle profile for tenant.
|
||||
|
||||
## DSSE bundling rules
|
||||
- Deterministic JSON (sorted keys) before hashing.
|
||||
- Canonical NDJSON for chunk payload; no gzip inside envelope.
|
||||
- Attach verification report alongside attestation as `chunk-verify.json` (hashes + signature check results).
|
||||
|
||||
## Sample payloads
|
||||
- `docs/samples/excititor/chunk-sample.ndjson`
|
||||
- `docs/samples/excititor/chunk-attestation-sample.json`
|
||||
|
||||
## Integration points
|
||||
- Evidence Locker contract v1 (see `docs/modules/evidence-locker/attestation-contract.md`).
|
||||
- Concelier LNM schemas (observations remain aggregation-only; attestation is evidence, not merge).
|
||||
|
||||
## Ownership
|
||||
- Excititor Guild (primary); Evidence Locker Guild reviewer.
|
||||
@@ -0,0 +1,36 @@
|
||||
# Connector signer metadata (v1.0.0)
|
||||
|
||||
**Scope.** Defines the canonical, offline-friendly metadata for Excititor connectors that validate signed feeds (MSRC CSAF, Oracle OVAL, Ubuntu OVAL, StellaOps mirror OpenVEX). The file is consumed by WebService/Worker composition roots and by Offline Kits to pin trust material deterministically.
|
||||
|
||||
**Location & format.**
|
||||
- Schema: `docs/modules/excititor/schemas/connector-signer-metadata.schema.json` (JSON Schema 2020‑12).
|
||||
- Sample: `docs/samples/excititor/connector-signer-metadata-sample.json` (aligns with schema).
|
||||
- Expected production artifact: NDJSON or JSON stamped per release; store in offline kits alongside connector bundles.
|
||||
|
||||
## Required fields (summary)
|
||||
- `schemaVersion` — must be `1.0.0`.
|
||||
- `generatedAt` — ISO-8601 UTC timestamp for the metadata file.
|
||||
- `connectors[]` — one entry per connector:
|
||||
- `connectorId` — stable slug, e.g., `excititor-msrc-csaf`.
|
||||
- `provider { name, slug }` — human label and slug.
|
||||
- `issuerTier` — `tier-0`, `tier-1`, `tier-2`, or `untrusted` (aligns with trust weighting).
|
||||
- `signers[]` — one per signing path; each has `usage` (`csaf|oval|openvex|bulk-meta|attestation`) and `fingerprints[]` (algorithm + format + value). Optional `keyLocator` and `certificateChain` for offline key retrieval.
|
||||
- `bundle` — reference to the sealed bundle containing the feed/signing material (`kind`: `oci-referrer|oci-tag|file|tuf`, plus `uri`, optional `digest`, `publishedAt`).
|
||||
- Optional `validFrom`, `validTo`, `revoked`, `notes` for rollover and incident handling.
|
||||
|
||||
## Rollover / migration guidance
|
||||
1) **Author the metadata** using the schema and place the JSON next to connector bundles in the offline kit (`out/connectors/<provider>/signer-metadata.json`).
|
||||
2) **Validate** with `dotnet tool run njsonschema validate connector-signer-metadata.schema.json connector-signer-metadata.json` (or `ajv validate`).
|
||||
3) **Wire connector code** to load the file on startup (Worker + WebService) and pin signers per `connectorId`; reject feeds whose fingerprints are absent or marked `revoked=true` or out of `validFrom/To` range.
|
||||
- Connectors look for `STELLAOPS_CONNECTOR_SIGNER_METADATA_PATH` (absolute/relative) and enrich provenance metadata automatically when present.
|
||||
4) **Rollover keys** by appending a new `signers` entry and setting a future `validFrom`; keep the previous signer until all mirrors have caught up. Use `issuerTier` downgrades to quarantine while keeping history.
|
||||
5) **Mirror references**: store the referenced bundles/keys under OCI tags or TUF targets already shipped in the offline kit so no live network is required.
|
||||
6) **Record decisions** in sprint Decisions & Risks when changing trust tiers or fingerpints; update this doc if formats change.
|
||||
|
||||
## Sample entries (non-production)
|
||||
See `docs/samples/excititor/connector-signer-metadata-sample.json` for MSRC, Oracle, Ubuntu, and StellaOps example entries. These fingerprints are illustrative only; replace with real values before shipping.
|
||||
|
||||
## Consumer expectations
|
||||
- Deterministic: sort connectors alphabetically before persistence; avoid clock-based defaults.
|
||||
- Offline-first: all `keyLocator`/`bundle.uri` values must resolve inside the air-gap kit (OCI/TUF/file).
|
||||
- Observability: emit a structured warning when metadata is missing or stale (>7 days) and fail closed for missing signers.
|
||||
24
docs/modules/excititor/operations/chunk-api-user-guide.md
Normal file
24
docs/modules/excititor/operations/chunk-api-user-guide.md
Normal file
@@ -0,0 +1,24 @@
|
||||
# Using the Chunk API
|
||||
|
||||
Endpoint: `POST /vex/evidence/chunks`
|
||||
- Content-Type: `application/x-ndjson`
|
||||
- See schema: `docs/modules/excititor/schemas/vex-chunk-api.yaml`
|
||||
|
||||
Response: `202 Accepted`
|
||||
```json
|
||||
{ "chunk_digest": "sha256:…", "queue_id": "uuid" }
|
||||
```
|
||||
|
||||
Operational notes
|
||||
- Deterministic hashing: server recomputes `chunk_digest` from canonical JSON; mismatches return 400.
|
||||
- Limits: default 500 items, max 2000 (aligned with Program.cs guard).
|
||||
- Telemetry: metrics under `StellaOps.Excititor.Chunks` (see chunk-telemetry.md).
|
||||
- Headers: correlation/trace headers echoed (`X-Stella-TraceId`, `X-Stella-CorrelationId`).
|
||||
|
||||
Example curl
|
||||
```bash
|
||||
curl -X POST https://excitor.local/vex/evidence/chunks \
|
||||
-H "Authorization: Bearer <token>" \
|
||||
-H "Content-Type: application/x-ndjson" \
|
||||
--data-binary @docs/samples/excititor/chunk-sample.ndjson
|
||||
```
|
||||
26
docs/modules/excititor/operations/chunk-telemetry.md
Normal file
26
docs/modules/excititor/operations/chunk-telemetry.md
Normal file
@@ -0,0 +1,26 @@
|
||||
# Excititor Chunk Telemetry (Sprint 110)
|
||||
|
||||
## Metrics (Meter: `StellaOps.Excititor.Chunks`)
|
||||
- `vex_chunks_ingested_total` (counter) — tags: `tenant`, `source`, `status` (`accepted|rejected`), `reason` (nullable for accepted). Increments per chunk submitted.
|
||||
- `vex_chunks_item_count` (histogram, unit=items) — records item count per chunk.
|
||||
- `vex_chunks_payload_bytes` (histogram, unit=bytes) — measured from NDJSON payload length.
|
||||
- `vex_chunks_latency_ms` (histogram) — end-to-end ingestion latency per request.
|
||||
|
||||
## Logs
|
||||
- `vex.chunk.ingest.accepted` — includes `chunk_id`, `tenant`, `source`, `item_count`, `chunk_digest`.
|
||||
- `vex.chunk.ingest.rejected` — includes `chunk_id`, `tenant`, `source`, `reason`, validation errors (summarized).
|
||||
|
||||
## Wiring steps
|
||||
1. Register `ChunkTelemetry` as singleton with shared `Meter` instance.
|
||||
2. In `/vex/evidence/chunks` handler, compute `chunk_digest` deterministically from canonical JSON and emit counters/histograms via `ChunkTelemetry`.
|
||||
3. Log using structured templates above; avoid request bodies in logs.
|
||||
4. Expose metrics via default ASP.NET metrics export (Prometheus/OpenTelemetry) already configured in WebService.
|
||||
|
||||
## Determinism & offline posture
|
||||
- Do not include host-specific paths or timestamps in metric dimensions.
|
||||
- Histogram buckets: use standard OTEL defaults; no runtime-generated buckets.
|
||||
- Keep meter name stable; adding new instruments requires version note in sprint Decisions & Risks.
|
||||
|
||||
## Ownership
|
||||
- Implementer: Excititor Observability Guild
|
||||
- Reviewers: Evidence Locker Guild (for parity with attestation metrics)
|
||||
@@ -0,0 +1,125 @@
|
||||
{
|
||||
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
||||
"$id": "https://stellaops.dev/schemas/excititor/connector-signer-metadata.schema.json",
|
||||
"title": "Excititor Connector Signer Metadata",
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": ["schemaVersion", "generatedAt", "connectors"],
|
||||
"properties": {
|
||||
"schemaVersion": {
|
||||
"type": "string",
|
||||
"pattern": "^1\\.0\\.0$"
|
||||
},
|
||||
"generatedAt": {
|
||||
"type": "string",
|
||||
"format": "date-time"
|
||||
},
|
||||
"connectors": {
|
||||
"type": "array",
|
||||
"minItems": 1,
|
||||
"items": {
|
||||
"$ref": "#/$defs/connector"
|
||||
}
|
||||
}
|
||||
},
|
||||
"$defs": {
|
||||
"connector": {
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": [
|
||||
"connectorId",
|
||||
"provider",
|
||||
"issuerTier",
|
||||
"signers"
|
||||
],
|
||||
"properties": {
|
||||
"connectorId": {
|
||||
"type": "string",
|
||||
"pattern": "^[a-z0-9:-\\.]+$"
|
||||
},
|
||||
"provider": {
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": ["name", "slug"],
|
||||
"properties": {
|
||||
"name": { "type": "string", "minLength": 3 },
|
||||
"slug": { "type": "string", "pattern": "^[a-z0-9-]+$" }
|
||||
}
|
||||
},
|
||||
"issuerTier": {
|
||||
"type": "string",
|
||||
"enum": ["tier-0", "tier-1", "tier-2", "untrusted"]
|
||||
},
|
||||
"signers": {
|
||||
"type": "array",
|
||||
"minItems": 1,
|
||||
"items": { "$ref": "#/$defs/signer" }
|
||||
},
|
||||
"bundle": { "$ref": "#/$defs/bundleRef" },
|
||||
"validFrom": { "type": "string", "format": "date" },
|
||||
"validTo": { "type": "string", "format": "date" },
|
||||
"revoked": { "type": "boolean", "default": false },
|
||||
"notes": { "type": "string", "maxLength": 2000 }
|
||||
}
|
||||
},
|
||||
"signer": {
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": ["usage", "fingerprints"],
|
||||
"properties": {
|
||||
"usage": {
|
||||
"type": "string",
|
||||
"enum": ["csaf", "oval", "openvex", "bulk-meta", "attestation"]
|
||||
},
|
||||
"fingerprints": {
|
||||
"type": "array",
|
||||
"minItems": 1,
|
||||
"items": { "$ref": "#/$defs/fingerprint" }
|
||||
},
|
||||
"keyLocator": {
|
||||
"type": "string",
|
||||
"description": "Path or URL (mirror/OCI/TUF) where the signing key or certificate chain can be retrieved in offline kits."
|
||||
},
|
||||
"certificateChain": {
|
||||
"type": "array",
|
||||
"items": { "type": "string" },
|
||||
"description": "Optional PEM-encoded certificates for x509/cosign keys."
|
||||
}
|
||||
}
|
||||
},
|
||||
"fingerprint": {
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": ["alg", "value"],
|
||||
"properties": {
|
||||
"alg": {
|
||||
"type": "string",
|
||||
"enum": ["sha256", "sha512", "sha1"]
|
||||
},
|
||||
"format": {
|
||||
"type": "string",
|
||||
"enum": ["pgp", "x509-spki", "x509-ski", "cosign", "pem"]
|
||||
},
|
||||
"value": {
|
||||
"type": "string",
|
||||
"minLength": 16,
|
||||
"maxLength": 128
|
||||
}
|
||||
}
|
||||
},
|
||||
"bundleRef": {
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"required": ["kind", "uri"],
|
||||
"properties": {
|
||||
"kind": {
|
||||
"type": "string",
|
||||
"enum": ["oci-referrer", "oci-tag", "file", "tuf"]
|
||||
},
|
||||
"uri": { "type": "string", "minLength": 8 },
|
||||
"digest": { "type": "string", "minLength": 32 },
|
||||
"publishedAt": { "type": "string", "format": "date-time" }
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
82
docs/modules/excititor/schemas/vex-chunk-api.yaml
Normal file
82
docs/modules/excititor/schemas/vex-chunk-api.yaml
Normal file
@@ -0,0 +1,82 @@
|
||||
openapi: 3.1.0
|
||||
info:
|
||||
title: StellaOps Excititor Chunk API
|
||||
version: "0.1.0"
|
||||
description: |
|
||||
Frozen for Sprint 110 (EXCITITOR-AIAI-31-002). Aligns with Evidence Locker attestation contract v1.
|
||||
servers:
|
||||
- url: https://excitor.local
|
||||
paths:
|
||||
/vex/evidence/chunks:
|
||||
post:
|
||||
summary: Submit VEX evidence chunk (aggregation-only)
|
||||
requestBody:
|
||||
required: true
|
||||
content:
|
||||
application/x-ndjson:
|
||||
schema:
|
||||
$ref: '#/components/schemas/VexChunk'
|
||||
responses:
|
||||
'202':
|
||||
description: Accepted for processing
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
type: object
|
||||
required: [chunk_digest, queue_id]
|
||||
properties:
|
||||
chunk_digest:
|
||||
type: string
|
||||
description: sha256 of canonical chunk JSON
|
||||
queue_id:
|
||||
type: string
|
||||
description: Background job identifier
|
||||
'400':
|
||||
description: Validation error
|
||||
components:
|
||||
schemas:
|
||||
VexChunk:
|
||||
type: object
|
||||
required: [chunk_id, tenant, source, schema, items, provenance]
|
||||
properties:
|
||||
chunk_id:
|
||||
type: string
|
||||
format: uuid
|
||||
tenant:
|
||||
type: string
|
||||
source:
|
||||
type: string
|
||||
description: feed id (e.g., ghsa, nvd)
|
||||
schema:
|
||||
type: string
|
||||
enum: [stellaops.vex.chunk.v1]
|
||||
items:
|
||||
type: array
|
||||
items:
|
||||
type: object
|
||||
required: [advisory_id, status, purl]
|
||||
properties:
|
||||
advisory_id:
|
||||
type: string
|
||||
status:
|
||||
type: string
|
||||
enum: [affected, unaffected, under_investigation, fixed, unknown]
|
||||
purl:
|
||||
type: string
|
||||
justification:
|
||||
type: string
|
||||
last_observed:
|
||||
type: string
|
||||
format: date-time
|
||||
provenance:
|
||||
type: object
|
||||
required: [fetched_at, artifact_sha]
|
||||
properties:
|
||||
fetched_at:
|
||||
type: string
|
||||
format: date-time
|
||||
artifact_sha:
|
||||
type: string
|
||||
signature:
|
||||
type: object
|
||||
nullable: true
|
||||
Reference in New Issue
Block a user