feat: Initialize Zastava Webhook service with TLS and Authority authentication

- Added Program.cs to set up the web application with Serilog for logging, health check endpoints, and a placeholder admission endpoint.
- Configured Kestrel server to use TLS 1.3 and handle client certificates appropriately.
- Created StellaOps.Zastava.Webhook.csproj with necessary dependencies including Serilog and Polly.
- Documented tasks in TASKS.md for the Zastava Webhook project, outlining current work and exit criteria for each task.
This commit is contained in:
master
2025-10-19 18:36:22 +03:00
parent 2062da7a8b
commit d099a90f9b
966 changed files with 91038 additions and 1850 deletions

View File

@@ -149,14 +149,114 @@ Client then generates SBOM **only** for the `missing` layers and reposts `/sc
---
### 2.3 Policy Endpoints
### 2.3 Policy Endpoints *(preview feature flag: `scanner.features.enablePolicyPreview`)*
| Method | Path | Purpose |
| ------ | ------------------ | ------------------------------------ |
| `GET` | `/policy/export` | Download live YAML ruleset |
| `POST` | `/policy/import` | Upload YAML or Rego; replaces active |
| `POST` | `/policy/validate` | Lint only; returns 400 on error |
| `GET` | `/policy/history` | Paginated change log (audit trail) |
All policy APIs require **`scanner.reports`** scope (or anonymous access while auth is disabled).
**Fetch schema**
```
GET /api/v1/policy/schema
Authorization: Bearer <token>
Accept: application/schema+json
```
Returns the embedded `policy-schema@1` JSON schema used by the binder.
**Run diagnostics**
```
POST /api/v1/policy/diagnostics
Content-Type: application/json
Authorization: Bearer <token>
```
```json
{
"policy": {
"format": "yaml",
"actor": "cli",
"description": "dev override",
"content": "version: \"1.0\"\nrules:\n - name: Quiet Dev\n environments: [dev]\n action:\n type: ignore\n justification: dev waiver\n"
}
}
```
**Response 200**:
```json
{
"success": false,
"version": "1.0",
"ruleCount": 1,
"errorCount": 0,
"warningCount": 1,
"generatedAt": "2025-10-19T03:25:14.112Z",
"issues": [
{ "code": "policy.rule.quiet.missing_vex", "message": "Quiet flag ignored: rule must specify requireVex justifications.", "severity": "Warning", "path": "$.rules[0]" }
],
"recommendations": [
"Review policy warnings and ensure intentional overrides are documented."
]
}
```
`success` is `false` when blocking issues remain; recommendations aggregate YAML ignore rules, VEX include/exclude hints, and vendor precedence guidance.
**Preview impact**
```
POST /api/v1/policy/preview
Authorization: Bearer <token>
Content-Type: application/json
```
```json
{
"imageDigest": "sha256:abc123",
"findings": [
{ "id": "finding-1", "severity": "Critical", "source": "NVD" }
],
"policy": {
"format": "yaml",
"content": "version: \"1.0\"\nrules:\n - name: Block Critical\n severity: [Critical]\n action: block\n"
}
}
```
**Response 200**:
```json
{
"success": true,
"policyDigest": "9c5e...",
"revisionId": "preview",
"changed": 1,
"diffs": [
{
"findingId": "finding-1",
"baseline": {"findingId": "finding-1", "status": "Pass"},
"projected": {
"findingId": "finding-1",
"status": "Blocked",
"ruleName": "Block Critical",
"ruleAction": "Block",
"score": 5.0,
"configVersion": "1.0",
"inputs": {"severityWeight": 5.0}
},
"changed": true
}
],
"issues": []
}
```
- Provide `policy` to preview staged changes; omit it to compare against the active snapshot.
- Baseline verdicts are optional; when omitted, the API synthesises pass baselines before computing diffs.
- Quieted verdicts include `quietedBy` and `quiet` flags; score inputs now surface reachability/vendor trust weights (`reachability.*`, `trustWeight.*`).
**OpenAPI**: the full API document (including these endpoints) is exposed at `/openapi/v1.json` and can be fetched for tooling or contract regeneration.
### 2.4 Scanner Queue a Scan Job *(SP9 milestone)*
@@ -238,6 +338,96 @@ Accept: application/json
Statuses: `Pending`, `Running`, `Succeeded`, `Failed`, `Cancelled`.
### 2.6 Scanner Stream Progress (SSE / JSONL)
```
GET /api/v1/scans/{scanId}/events?format=sse|jsonl
Authorization: Bearer <token with scanner.scans.read>
Accept: text/event-stream
```
When `format` is omitted the endpoint emits **Server-Sent Events** (SSE). Specify `format=jsonl` to receive newline-delimited JSON (`application/x-ndjson`). Response headers include `Cache-Control: no-store` and `X-Accel-Buffering: no` so intermediaries avoid buffering the stream.
**SSE frame** (default):
```
id: 1
event: pending
data: {"scanId":"2f6c17f9b3f548e2a28b9c412f4d63f8","sequence":1,"state":"Pending","message":"queued","timestamp":"2025-10-19T03:12:45.118Z","correlationId":"2f6c17f9b3f548e2a28b9c412f4d63f8:0001","data":{"force":false,"meta.pipeline":"github"}}
```
**JSONL frame** (`format=jsonl`):
```json
{"scanId":"2f6c17f9b3f548e2a28b9c412f4d63f8","sequence":1,"state":"Pending","message":"queued","timestamp":"2025-10-19T03:12:45.118Z","correlationId":"2f6c17f9b3f548e2a28b9c412f4d63f8:0001","data":{"force":false,"meta.pipeline":"github"}}
```
- `sequence` is monotonic starting at `1`.
- `correlationId` is deterministic (`{scanId}:{sequence:0000}`) unless a custom identifier is supplied by the publisher.
- `timestamp` is ISO8601 UTC with millisecond precision, ensuring deterministic ordering for consumers.
- The stream completes when the client disconnects or the coordinator stops publishing events.
### 2.7 Scanner Assemble Report (Signed Envelope)
```
POST /api/v1/reports
Authorization: Bearer <token with scanner.reports>
Content-Type: application/json
```
Request body mirrors policy preview inputs (image digest plus findings). The service evaluates the active policy snapshot, assembles a verdict, and signs the canonical report payload.
**Response 200**:
```json
{
"report": {
"reportId": "report-3def5f362aa475ef14b6",
"imageDigest": "sha256:deadbeef",
"verdict": "blocked",
"policy": { "revisionId": "rev-1", "digest": "27d2ec2b34feedc304fc564d252ecee1c8fa14ea581a5ff5c1ea8963313d5c8d" },
"summary": { "total": 1, "blocked": 1, "warned": 0, "ignored": 0, "quieted": 0 },
"verdicts": [
{
"findingId": "finding-1",
"status": "Blocked",
"ruleName": "Block Critical",
"ruleAction": "Block",
"score": 40.5,
"configVersion": "1.0",
"inputs": {
"reachabilityWeight": 0.45,
"baseScore": 40.5,
"severityWeight": 90,
"trustWeight": 1,
"trustWeight.NVD": 1,
"reachability.runtime": 0.45
},
"quiet": false,
"sourceTrust": "NVD",
"reachability": "runtime"
}
],
"issues": []
},
"dsse": {
"payloadType": "application/vnd.stellaops.report+json",
"payload": "<base64 canonical report>",
"signatures": [
{
"keyId": "scanner-report-signing",
"algorithm": "hs256",
"signature": "<base64 signature>"
}
]
}
}
```
- The `report` object omits null fields and is deterministic (ISO timestamps, sorted keys).
- `dsse` follows the DSSE (Dead Simple Signing Envelope) shape; `payload` is the canonical UTF-8 JSON and `signatures[0].signature` is the base64 HMAC/Ed25519 value depending on configuration.
- A runnable sample envelope is available at `samples/api/reports/report-sample.dsse.json` for tooling tests or signature verification.
**Response 404** `application/problem+json` payload with type `https://stellaops.org/problems/not-found` when the scan identifier is unknown.
> **Tip**  poll `Location` from the submission call until `status` transitions away from `Pending`/`Running`.
@@ -332,6 +522,7 @@ See `docs/dev/32_AUTH_CLIENT_GUIDE.md` for recommended profiles (online vs. air-
| `stellaops-cli auth revoke export` | Export the Authority revocation bundle | `--output <directory>` (defaults to CWD) | Writes `revocation-bundle.json`, `.json.jws`, and `.json.sha256`; verifies the digest locally and includes key metadata in the log summary. |
| `stellaops-cli auth revoke verify` | Validate a revocation bundle offline | `--bundle <path>` `--signature <path>` `--key <path>`<br>`--verbose` | Verifies detached JWS signatures, reports the computed SHA-256, and can fall back to cached JWKS when `--key` is omitted. |
| `stellaops-cli config show` | Display resolved configuration | — | Masks secret values; helpful for airgapped installs |
| `stellaops-cli runtime policy test` | Ask Scanner.WebService for runtime verdicts (Webhook parity) | `--image/-i <digest>` (repeatable, comma/space lists supported)<br>`--file/-f <path>`<br>`--namespace/--ns <name>`<br>`--label/-l key=value` (repeatable)<br>`--json` | Posts to `POST /api/v1/scanner/policy/runtime`, deduplicates image digests, and prints TTL + per-image verdict/signed/SBOM status. Accepts newline/whitespace-delimited stdin when piped; `--json` emits the raw response without additional logging. |
When running on an interactive terminal without explicit override flags, the CLI uses Spectre.Console prompts to let you choose per-run ORAS/offline bundle behaviour.

View File

@@ -86,16 +86,152 @@ Only enabled when `MONGO_URI` is supplied (for longterm audit).
Schema detail for **policy_versions**:
```jsonc
{
"_id": "6619e90b8c5e1f76",
"yaml": "version: 1.0\nrules:\n - …",
"rego": null, // filled when Rego uploaded
"authorId": "u_1021",
"created": "2025-07-14T08:15:04Z",
"comment": "Imported via API"
}
```
Samples live under `samples/api/scheduler/` (e.g., `schedule.json`, `run.json`, `impact-set.json`, `audit.json`) and mirror the canonical serializer output shown below.
```jsonc
{
"_id": "6619e90b8c5e1f76",
"yaml": "version: 1.0\nrules:\n - …",
"rego": null, // filled when Rego uploaded
"authorId": "u_1021",
"created": "2025-07-14T08:15:04Z",
"comment": "Imported via API"
}
```
###3.1Scheduler Sprints 16 Artifacts
**Collections.** `schedules`, `runs`, `impact_snapshots`, `audit` (modulelocal). All documents reuse the canonical JSON emitted by `StellaOps.Scheduler.Models` so agents and fixtures remain deterministic.
####3.1.1Schedule (`schedules`)
```jsonc
{
"_id": "sch_20251018a",
"tenantId": "tenant-alpha",
"name": "Nightly Prod",
"enabled": true,
"cronExpression": "0 2 * * *",
"timezone": "UTC",
"mode": "analysis-only",
"selection": {
"scope": "by-namespace",
"namespaces": ["team-a", "team-b"],
"repositories": ["app/service-api"],
"includeTags": ["canary", "prod"],
"labels": [{"key": "env", "values": ["prod", "staging"]}],
"resolvesTags": true
},
"onlyIf": {"lastReportOlderThanDays": 7, "policyRevision": "policy@42"},
"notify": {"onNewFindings": true, "minSeverity": "high", "includeKev": true},
"limits": {"maxJobs": 1000, "ratePerSecond": 25, "parallelism": 4},
"subscribers": ["notify.ops"],
"createdAt": "2025-10-18T22:00:00Z",
"createdBy": "svc_scheduler",
"updatedAt": "2025-10-18T22:00:00Z",
"updatedBy": "svc_scheduler"
}
```
*Constraints*: arrays are alphabetically sorted; `selection.tenantId` is optional but when present must match `tenantId`. Cron expressions are validated for newline/length, timezones are validated via `TimeZoneInfo`.
####3.1.2Run (`runs`)
```jsonc
{
"_id": "run_20251018_0001",
"tenantId": "tenant-alpha",
"scheduleId": "sch_20251018a",
"trigger": "feedser",
"state": "running",
"stats": {
"candidates": 1280,
"deduped": 910,
"queued": 624,
"completed": 310,
"deltas": 42,
"newCriticals": 7,
"newHigh": 11,
"newMedium": 18,
"newLow": 6
},
"reason": {"feedserExportId": "exp-20251018-03"},
"createdAt": "2025-10-18T22:03:14Z",
"startedAt": "2025-10-18T22:03:20Z",
"finishedAt": null,
"error": null,
"deltas": [
{
"imageDigest": "sha256:a1b2c3",
"newFindings": 3,
"newCriticals": 1,
"newHigh": 1,
"newMedium": 1,
"newLow": 0,
"kevHits": ["CVE-2025-0002"],
"topFindings": [
{
"purl": "pkg:rpm/openssl@3.0.12-5.el9",
"vulnerabilityId": "CVE-2025-0002",
"severity": "critical",
"link": "https://ui.internal/scans/sha256:a1b2c3"
}
],
"attestation": {"uuid": "rekor-314", "verified": true},
"detectedAt": "2025-10-18T22:03:21Z"
}
]
}
```
Counters are clamped to ≥0, timestamps are converted to UTC, and delta arrays are sorted (critical → info severity precedence, then vulnerability id). Missing `deltas` implies "no change" snapshots.
####3.1.3Impact Snapshot (`impact_snapshots`)
```jsonc
{
"selector": {
"scope": "all-images",
"tenantId": "tenant-alpha"
},
"images": [
{
"imageDigest": "sha256:f1e2d3",
"registry": "registry.internal",
"repository": "app/api",
"namespaces": ["team-a"],
"tags": ["prod"],
"usedByEntrypoint": true,
"labels": {"env": "prod"}
}
],
"usageOnly": true,
"generatedAt": "2025-10-18T22:02:58Z",
"total": 412,
"snapshotId": "impact-20251018-1"
}
```
Images are deduplicated and sorted by digest. Label keys are normalised to lowercase to avoid casesensitive duplicates during reconciliation. `snapshotId` enables run planners to compare subsequent snapshots for drift.
####3.1.4Audit (`audit`)
```jsonc
{
"_id": "audit_169754",
"tenantId": "tenant-alpha",
"category": "scheduler",
"action": "pause",
"occurredAt": "2025-10-18T22:10:00Z",
"actor": {"actorId": "user_admin", "displayName": "Cluster Admin", "kind": "user"},
"scheduleId": "sch_20251018a",
"correlationId": "corr-123",
"metadata": {"details": "schedule paused", "reason": "maintenance"},
"message": "Paused via API"
}
```
Metadata keys are lowercased, firstwriter wins (duplicates with different casing are ignored), and optional IDs (`scheduleId`, `runId`) are trimmed when empty. Use the canonical serializer when emitting events so audit digests remain reproducible.
---
@@ -133,18 +269,53 @@ await new PolicyValidationCli().RunAsync(new PolicyValidationCliOptions
});
```
###4.1Rego Variant (Advanced  TODO)
*Accepted but stored asis in `rego` field.*
Evaluated via internal **OPA** sidecar once feature graduates from TODO list.
---
##5SLSA Attestation Schema 
Planned for Q12026 (kept here for early plugin authors).
```jsonc
###4.1Rego Variant (Advanced  TODO)
*Accepted but stored asis in `rego` field.*
Evaluated via internal **OPA** sidecar once feature graduates from TODO list.
###4.2Policy Scoring Config (JSON)
*Schema id.* `https://schemas.stella-ops.org/policy/policy-scoring-schema@1.json`
*Source.* `src/StellaOps.Policy/Schemas/policy-scoring-schema@1.json` (embedded in `StellaOps.Policy`), default fixture at `src/StellaOps.Policy/Schemas/policy-scoring-default.json`.
```jsonc
{
"version": "1.0",
"severityWeights": {"Critical": 90, "High": 75, "Unknown": 60, "...": 0},
"quietPenalty": 45,
"warnPenalty": 15,
"ignorePenalty": 35,
"trustOverrides": {"vendor": 1.0, "distro": 0.85},
"reachabilityBuckets": {"entrypoint": 1.0, "direct": 0.85, "runtime": 0.45, "unknown": 0.5},
"unknownConfidence": {
"initial": 0.8,
"decayPerDay": 0.05,
"floor": 0.2,
"bands": [
{"name": "high", "min": 0.65},
{"name": "medium", "min": 0.35},
{"name": "low", "min": 0.0}
]
}
}
```
Validation occurs alongside policy binding (`PolicyScoringConfigBinder`), producing deterministic digests via `PolicyScoringConfigDigest`. Bands are ordered descending by `min` so consumers can resolve confidence tiers deterministically. Reachability buckets are case-insensitive keys (`entrypoint`, `direct`, `indirect`, `runtime`, `unreachable`, `unknown`) with numeric multipliers (default ≤1.0).
**Runtime usage**
- `trustOverrides` are matched against `finding.tags` (`trust:<key>`) first, then `finding.source`/`finding.vendor`; missing keys default to `1.0`.
- `reachabilityBuckets` consume `finding.tags` with prefix `reachability:` (fallback `usage:` or `unknown`). Missing buckets fall back to `unknown` weight when present, otherwise `1.0`.
- Policy verdicts expose scoring inputs (`severityWeight`, `trustWeight`, `reachabilityWeight`, `baseScore`, penalties) plus unknown-state metadata (`unknownConfidence`, `unknownAgeDays`, `confidenceBand`) for auditability. See `samples/policy/policy-preview-unknown.json` for an end-to-end preview payload.
- Unknown confidence derives from `unknown-age-days:` (preferred) or `unknown-since:` + `observed-at:` tags; with no hints the engine keeps `initial` confidence. Values decay by `decayPerDay` down to `floor`, then resolve to the first matching `bands[].name`.
---
##5SLSA Attestation Schema 
Planned for Q12026 (kept here for early plugin authors).
```jsonc
{
"id": "prov_0291",
"imageDigest": "sha256:e2b9…",
@@ -164,8 +335,70 @@ Planned for Q12026 (kept here for early plugin authors).
{"uri": "git+https://git…", "digest": {"sha1": "f6a1…"}}
],
"rekorLogIndex": 99817 // entry in local Rekor mirror
}
```
}
```
---
##6NotifyFoundations (Rule·Channel·Event)
*Sprint 15 target* canonically describe the Notify data shapes that UI, workers, and storage consume. JSON Schemas live under `docs/notify/schemas/` and deterministic fixtures under `docs/notify/samples/`.
| Artifact | Schema | Sample |
|----------|--------|--------|
| **Rule** (catalogued routing logic) | `docs/notify/schemas/notify-rule@1.json` | `docs/notify/samples/notify-rule@1.sample.json` |
| **Channel** (delivery endpoint definition) | `docs/notify/schemas/notify-channel@1.json` | `docs/notify/samples/notify-channel@1.sample.json` |
| **Template** (rendering payload) | `docs/notify/schemas/notify-template@1.json` | `docs/notify/samples/notify-template@1.sample.json` |
| **Event envelope** (Notify ingest surface) | `docs/notify/schemas/notify-event@1.json` | `docs/notify/samples/notify-event@1.sample.json` |
###6.1Rule highlights (`notify-rule@1`)
* Keys are lowercased camelCase. `schemaVersion` (`notify.rule@1`), `ruleId`, `tenantId`, `name`, `match`, `actions`, `createdAt`, and `updatedAt` are mandatory.
* `match.eventKinds`, `match.verdicts`, and other array selectors are presorted and casenormalized (e.g. `scanner.report.ready`).
* `actions[].throttle` serialises as ISO8601 duration (`PT5M`), mirroring worker backoff guardrails.
* `vex` gates let operators exclude accepted/notaffected justifications; omit the block to inherit default behaviour.
* Use `StellaOps.Notify.Models.NotifySchemaMigration.UpgradeRule(JsonNode)` when deserialising legacy payloads that might lack `schemaVersion` or retain older revisions.
* Soft deletions persist `deletedAt` in Mongo (and disable the rule); repository queries automatically filter them.
###6.2Channel highlights (`notify-channel@1`)
* `schemaVersion` is pinned to `notify.channel@1` and must accompany persisted documents.
* `type` matches plugin identifiers (`slack`, `teams`, `email`, `webhook`, `custom`).
* `config.secretRef` stores an external secret handle (Authority, Vault, K8s). Notify never persists raw credentials.
* Optional `config.limits.timeout` uses ISO8601 durations identical to rule throttles; concurrency/RPM defaults apply when absent.
* `StellaOps.Notify.Models.NotifySchemaMigration.UpgradeChannel(JsonNode)` backfills the schema version when older documents omit it.
* Channels share the same soft-delete marker (`deletedAt`) so operators can restore prior configuration without purging history.
###6.3Event envelope (`notify-event@1`)
* Aligns with the platform event contract—`eventId` UUID, RFC3339 `ts`, tenant isolation enforced.
* Enumerated `kind` covers the initial Notify surface (`scanner.report.ready`, `scheduler.rescan.delta`, `zastava.admission`, etc.).
* `scope.labels`/`scope.attributes` and top-level `attributes` mirror the metadata dictionaries workers surface for templating and audits.
* Notify workers use the same migration helper to wrap event payloads before template rendering, so schema additions remain additive.
###6.4Template highlights (`notify-template@1`)
* Carries the presentation key (`channelType`, `key`, `locale`) and the raw template body; `schemaVersion` is fixed to `notify.template@1`.
* `renderMode` enumerates supported engines (`markdown`, `html`, `adaptiveCard`, `plainText`, `json`) aligning with `NotifyTemplateRenderMode`.
* `format` signals downstream connector expectations (`slack`, `teams`, `email`, `webhook`, `json`).
* Upgrade legacy definitions with `NotifySchemaMigration.UpgradeTemplate(JsonNode)` to auto-apply the new schema version and ordering.
* Templates also record soft deletes via `deletedAt`; UI/API skip them by default while retaining revision history.
**Validation loop:**
```bash
# Validate Notify schemas and samples (matches Docs CI)
for schema in docs/notify/schemas/*.json; do
npx ajv compile -c ajv-formats -s "$schema"
done
for sample in docs/notify/samples/*.sample.json; do
schema="docs/notify/schemas/$(basename "${sample%.sample.json}").json"
npx ajv validate -c ajv-formats -s "$schema" -d "$sample"
done
```
Integration tests can embed the sample fixtures to guarantee deterministic serialisation from the `StellaOps.Notify.Models` DTOs introduced in Sprint15.
---

View File

@@ -302,6 +302,18 @@ authority:
auth: { type: "mtls" }
senderConstraint: "mtls"
scopes: [ "signer.sign" ]
- clientId: notify-web-dev
grantTypes: [ "client_credentials" ]
audiences: [ "notify.dev" ]
auth: { type: "client_secret", secretFile: "/secrets/notify-web-dev.secret" }
senderConstraint: "dpop"
scopes: [ "notify.read", "notify.admin" ]
- clientId: notify-web
grantTypes: [ "client_credentials" ]
audiences: [ "notify" ]
auth: { type: "client_secret", secretFile: "/secrets/notify-web.secret" }
senderConstraint: "dpop"
scopes: [ "notify.read", "notify.admin" ]
```
---

View File

@@ -89,7 +89,7 @@ src/
### 2.6 Runtime (Zastava helper)
* `runtime policy test --images <digest,...> [--ns <name> --labels k=v,...]` — ask backend `/policy/runtime` like the webhook would.
* `runtime policy test --image/-i <digest> [--file <path> --ns <name> --label key=value --json]` — ask backend `/policy/runtime` like the webhook would (accepts multiple `--image`, comma/space lists, or stdin pipelines).
### 2.7 Offline kit

View File

@@ -297,6 +297,12 @@ s3://stellaops/
* **Concelier/Excititor**: raw docs keep **last N windows**; canonical stores permanent.
* **Attestor**: `entries` permanent; `dedupe` TTL 2448h.
### 7.5 Mongo server baseline
* **Minimum supported server:** MongoDB **4.2+**. Driver 3.5.0 removes compatibility shims for 4.0; upstream has already announced 4.0 support will be dropped in upcoming C# driver releases. citeturn1open1
* **Deploy images:** Compose/Helm defaults stay on `mongo:7.x`. For air-gapped installs, refresh Offline Kit bundles so the packaged `mongod` matches ≥4.2.
* **Upgrade guard:** During rollout, verify replica sets reach FCV `4.2` or above before swapping binaries; automation should hard-stop if FCV is <4.2.
---
## 8) Observability & SLOs (operations)

View File

@@ -112,10 +112,10 @@ disposition: kept|replaced|superseded
correlation: { replaces?: sha256, replacedBy?: sha256 }
```
**`vex.claims`** (normalized rows; dedupe on providerId+vulnId+productKey+docDigest)
**`vex.statements`** (immutable normalized rows; append-only event log)
```
_id
_id: ObjectId
providerId
vulnId
productKey
@@ -127,9 +127,16 @@ lastObserved
docDigest
provenance { uri, line?, pointer?, signatureState }
evidence[] { key, value, locator }
signals? {
severity? { scheme, score?, label?, vector? }
kev?: bool
epss?: double
}
insertedAt
indices:
- {vulnId:1, productKey:1}
- {providerId:1, lastObserved:-1}
- {providerId:1, insertedAt:-1}
- {docDigest:1}
- {status:1}
- text index (optional) on evidence.value for debugging
```
@@ -146,6 +153,11 @@ sources[]: [
]
policyRevisionId
evaluatedAt
signals? {
severity? { scheme, score?, label?, vector? }
kev?: bool
epss?: double
}
consensusDigest // same as _id
indices:
- {vulnId:1, productKey:1}
@@ -175,6 +187,7 @@ ttl, hits
**`vex.migrations`**
* ordered migrations applied at bootstrap to ensure indexes.
* `20251019-consensus-signals-statements` introduces the statements log indexes and the `policyRevisionId + evaluatedAt` lookup for consensus — rerun consensus writers once to hydrate newly persisted signals.
### 3.2 Indexing strategy
@@ -339,6 +352,10 @@ excititor:
platform: 0.7
hub: 0.5
attestation: 0.6
ceiling: 1.25
scoring:
alpha: 0.25
beta: 0.5
providerOverrides:
redhat: 1.0
suse: 0.95
@@ -367,6 +384,20 @@ excititor:
signaturePolicy: { type: cosign, cosignKeylessRoots: [ "sigstore-root" ] }
```
### 9.1 WebService endpoints
With storage configured, the WebService exposes the following ingress and diagnostic APIs:
* `GET /excititor/status` returns the active storage configuration and registered artifact stores.
* `GET /excititor/health` simple liveness probe.
* `POST /excititor/statements` accepts normalized VEX statements and persists them via `IVexClaimStore`; use this for migrations/backfills.
* `GET /excititor/statements/{vulnId}/{productKey}?since=` returns the immutable statement log for a vulnerability/product pair.
Run the ingestion endpoint once after applying migration `20251019-consensus-signals-statements` to repopulate historical statements with the new severity/KEV/EPSS signal fields.
* `weights.ceiling` raises the deterministic clamp applied to provider tiers/overrides (range 1.05.0). Values outside the range are clamped with warnings so operators can spot typos.
* `scoring.alpha` / `scoring.beta` configure KEV/EPSS boosts for the Phase1 → Phase2 scoring pipeline. Defaults (0.25, 0.5) preserve prior behaviour; negative or excessively large values fall back with diagnostics.
---
## 10) Security model

View File

@@ -0,0 +1,138 @@
# architecture_excititor_mirrors.md — Excititor Mirror Distribution
> **Status:** Draft (Sprint 7). Complements `docs/ARCHITECTURE_EXCITITOR.md` by describing the mirror export surface exposed by `Excititor.WebService` and the configuration hooks used by operators and downstream mirrors.
---
## 0) Purpose
Excititor publishes canonical VEX consensus data. Operators (or StellaOps-managed mirrors) need a deterministic way to sync those exports into downstream environments. Mirror distribution provides:
* A declarative map of export bundles (`json`, `jsonl`, `openvex`, `csaf`) reachable via signed HTTP endpoints under `/excititor/mirror`.
* Thin quota/authentication controls on top of the existing export cache so mirrors cannot starve the web service.
* Stable payload shapes that downstream automation can monitor (index → fetch updates → download artifact → verify signature).
Mirror endpoints are intentionally **read-only**. Write paths (export generation, attestation, cache) remain the responsibility of the export pipeline.
---
## 1) Configuration model
The web service reads mirror configuration from `Excititor:Mirror` (YAML/JSON/appsettings). Each domain groups a set of exports that share rate limits and authentication rules.
```yaml
Excititor:
Mirror:
Domains:
- id: primary
displayName: Primary Mirror
requireAuthentication: false
maxIndexRequestsPerHour: 600
maxDownloadRequestsPerHour: 1200
exports:
- key: consensus
format: json
filters:
vulnId: CVE-2025-0001
productKey: pkg:test/demo
sort:
createdAt: false # descending
limit: 1000
- key: consensus-openvex
format: openvex
filters:
vulnId: CVE-2025-0001
```
### Field reference
| Field | Required | Description |
| --- | --- | --- |
| `id` | ✅ | Stable identifier. Appears in URLs (`/excititor/mirror/domains/{id}`) and download filenames. |
| `displayName` | | Human-friendly label surfaced in the `/domains` listing. Falls back to `id`. |
| `requireAuthentication` | | When `true` the service enforces that the caller is authenticated (Authority token). |
| `maxIndexRequestsPerHour` | | Per-domain quota for index endpoints. `0`/negative disables the guard. |
| `maxDownloadRequestsPerHour` | | Per-domain quota for artifact downloads. |
| `exports` | ✅ | Collection of export projections. |
Export-level fields:
| Field | Required | Description |
| --- | --- | --- |
| `key` | ✅ | Unique key within the domain. Used in URLs (`/exports/{key}`) and filenames. |
| `format` | ✅ | One of `json`, `jsonl`, `openvex`, `csaf`. Maps to `VexExportFormat`. |
| `filters` | | Key/value pairs executed via `VexQueryFilter`. Keys must match export data source columns (e.g., `vulnId`, `productKey`). |
| `sort` | | Key/boolean map (false = descending). |
| `limit`, `offset`, `view` | | Optional query bounds passed through to the export query. |
⚠️ **Misconfiguration:** invalid formats or missing keys cause exports to be flagged with `status` in the index response; they are not exposed downstream.
---
## 2) HTTP surface
Routes are grouped under `/excititor/mirror`.
| Method | Path | Description |
| --- | --- | --- |
| `GET` | `/domains` | Returns configured domains with quota metadata. |
| `GET` | `/domains/{domainId}` | Domain detail (auth/quota + export keys). `404` for unknown domains. |
| `GET` | `/domains/{domainId}/index` | Lists exports with exportId, query signature, format, artifact digest, attestation metadata, and size. Applies index quota. |
| `GET` | `/domains/{domainId}/exports/{exportKey}` | Returns manifest metadata (single export). `404` if unknown/missing. |
| `GET` | `/domains/{domainId}/exports/{exportKey}/download` | Streams export content from the artifact store. Applies download quota. |
Responses are serialized via `VexCanonicalJsonSerializer` ensuring stable ordering. Download responses include a content-disposition header naming the file `<domain>-<export>.<ext>`.
### Error handling
* `401` authentication required (`requireAuthentication=true`).
* `404` domain/export not found or manifest not persisted.
* `429` per-domain quota exceeded (`Retry-After` header set in seconds).
* `503` export misconfiguration (invalid format/query).
---
## 3) Rate limiting
`MirrorRateLimiter` implements a simple rolling 1-hour window using `IMemoryCache`. Each domain has two quotas:
* `index` scope → `maxIndexRequestsPerHour`
* `download` scope → `maxDownloadRequestsPerHour`
`0` or negative limits disable enforcement. Quotas are best-effort (per-instance). For HA deployments, configure sticky routing at the ingress or replace the limiter with a distributed implementation.
---
## 4) Interaction with export pipeline
Mirror endpoints consume manifests produced by the export engine (`MongoVexExportStore`). They do **not** trigger new exports. Operators must configure connectors/exporters to keep targeted exports fresh (see `EXCITITOR-EXPORT-01-005/006/007`).
Recommended workflow:
1. Define export plans at the export layer (JSON/OpenVEX/CSAF).
2. Configure mirror domains mapping to those plans.
3. Downstream mirror automation:
* `GET /domains/{id}/index`
* Compare `exportId` / `consensusRevision`
* `GET /download` when new
* Verify digest + attestation
When the export team lands deterministic mirror bundles (Sprint 7 tasks 01-005/006/007), these configurations can be generated automatically.
---
## 5) Operational guidance
* Track quota utilisation via HTTP 429 metrics (configure structured logging or OTEL counters when rate limiting triggers).
* Mirror domains can be deployed per tenant (e.g., `tenant-a`, `tenant-b`) with different auth requirements.
* Ensure the underlying artifact stores (`FileSystem`, `S3`, offline bundle) retain artefacts long enough for mirrors to sync.
* For air-gapped mirrors, combine mirror endpoints with the Offline Kit (see `docs/24_OFFLINE_KIT.md`).
---
## 6) Future alignment
* Replace manual export definitions with generated mirror bundle manifests once `EXCITITOR-EXPORT-01-007` ships.
* Extend `/index` payload with quiet-provenance when `EXCITITOR-EXPORT-01-006` adds that metadata.
* Integrate domain manifests with DevOps mirror profiles (`DEVOPS-MIRROR-08-001`) so helm/compose overlays can enable or disable domains declaratively.

View File

@@ -36,6 +36,25 @@ src/
**Dependencies**: Authority (OpToks; DPoP/mTLS), MongoDB, Redis/NATS (bus), HTTP egress to Slack/Teams/Webhooks, SMTP relay for Email.
> **Configuration.** Notify.WebService bootstraps from `notify.yaml` (see `etc/notify.yaml.sample`). Use `storage.driver: mongo` with a production connection string; the optional `memory` driver exists only for tests. Authority settings follow the platform defaults—when running locally without Authority, set `authority.enabled: false` and supply `developmentSigningKey` so JWTs can be validated offline.
> **Plug-ins.** All channel connectors are packaged under `<baseDirectory>/plugins/notify`. The ordered load list must start with Slack/Teams before Email/Webhook so chat-first actions are registered deterministically for Offline Kit bundles:
>
> ```yaml
> plugins:
> baseDirectory: "/var/opt/stellaops"
> directory: "plugins/notify"
> orderedPlugins:
> - StellaOps.Notify.Connectors.Slack
> - StellaOps.Notify.Connectors.Teams
> - StellaOps.Notify.Connectors.Email
> - StellaOps.Notify.Connectors.Webhook
> ```
>
> The Offline Kit job simply copies the `plugins/notify` tree into the air-gapped bundle; the ordered list keeps connector manifests stable across environments.
> **Authority clients.** Register two OAuth clients in StellaOps Authority: `notify-web-dev` (audience `notify.dev`) for development and `notify-web` (audience `notify`) for staging/production. Both require `notify.read` and `notify.admin` scopes and use DPoP-bound client credentials (`client_secret` in the samples). Reference entries live in `etc/authority.yaml.sample`, with placeholder secrets under `etc/secrets/notify-web*.secret.example`.
---
## 2) Responsibilities
@@ -81,10 +100,32 @@ Notify subscribes to the **internal event bus** (produced by services, escaped J
* `scanner.report.ready`:
```json
{ "verdict":"fail|warn|pass",
"delta": { "newCritical":1, "newHigh":2, "kev":["CVE-2025-..."] },
"topFindings":[{"purl":"pkg:rpm/openssl","vulnId":"CVE-2025-...","severity":"critical"}],
"links":{"ui":"https://ui/...","rekor":"https://rekor/..."} }
{
"reportId": "report-3def...",
"verdict": "fail",
"summary": {"total": 12, "blocked": 2, "warned": 3, "ignored": 5, "quieted": 2},
"delta": {"newCritical": 1, "kev": ["CVE-2025-..."]},
"links": {"ui": "https://ui/.../reports/report-3def...", "rekor": "https://rekor/..."},
"dsse": { "...": "..." },
"report": { "...": "..." }
}
```
Payload embeds both the canonical report document and the DSSE envelope so connectors, Notify, and UI tooling can reuse the signed bytes without re-serialising.
* `scanner.scan.completed`:
```json
{
"reportId": "report-3def...",
"digest": "sha256:...",
"verdict": "fail",
"summary": {"total": 12, "blocked": 2, "warned": 3, "ignored": 5, "quieted": 2},
"delta": {"newCritical": 1, "kev": ["CVE-2025-..."]},
"policy": {"revisionId": "rev-42", "digest": "27d2..."},
"findings": [{"id": "finding-1", "severity": "Critical", "cve": "CVE-2025-...", "reachability": "runtime"}],
"dsse": { "...": "..." }
}
```
* `zastava.admission`:
@@ -195,6 +236,8 @@ public interface INotifyConnector {
## 7) Data model (Mongo)
Canonical JSON Schemas for rules/channels/events live in `docs/notify/schemas/`. Sample payloads intended for tests/UI mock responses are captured in `docs/notify/samples/`.
**Database**: `notify`
* `rules`
@@ -240,6 +283,14 @@ public interface INotifyConnector {
Base path: `/api/v1/notify` (Authority OpToks; scopes: `notify.admin` for write, `notify.read` for view).
*All* REST calls require the tenant header `X-StellaOps-Tenant` (matches the canonical `tenantId` stored in Mongo). Payloads are normalised via `NotifySchemaMigration` before persistence to guarantee schema version pinning.
Authentication today is stubbed with Bearer tokens (`Authorization: Bearer <token>`). When Authority wiring lands, this will switch to OpTok validation + scope enforcement, but the header contract will remain the same.
Service configuration exposes `notify:auth:*` keys (issuer, audience, signing key, scope names) so operators can wire the Authority JWKS or (in dev) a symmetric test key. `notify:storage:*` keys cover Mongo URI/database/collection overrides. Both sets are required for the new API surface.
Internal tooling can hit `/internal/notify/<entity>/normalize` to upgrade legacy JSON and return canonical output used in the docs fixtures.
* **Channels**
* `POST /channels` | `GET /channels` | `GET /channels/{id}` | `PATCH /channels/{id}` | `DELETE /channels/{id}`
@@ -253,14 +304,18 @@ Base path: `/api/v1/notify` (Authority OpToks; scopes: `notify.admin` for write,
* **Deliveries**
* `GET /deliveries?tenant=...&since=...` → list
* `POST /deliveries` → ingest worker delivery state (idempotent via `deliveryId`).
* `GET /deliveries?since=...&status=...&limit=...` → list (most recent first)
* `GET /deliveries/{id}` → detail (redacted body + metadata)
* `POST /deliveries/{id}/retry` → force retry (admin)
* `POST /deliveries/{id}/retry` → force retry (admin, future sprint)
* **Admin**
* `GET /stats` (per tenant counts, last hour/day)
* `GET /healthz|readyz` (liveness)
* `POST /locks/acquire` | `POST /locks/release` worker coordination primitives (short TTL).
* `POST /digests` | `GET /digests/{actionKey}` | `DELETE /digests/{actionKey}` manage open digest windows.
* `POST /audit` | `GET /audit?since=&limit=` append/query structured audit trail entries.
**Ingestion**: workers do **not** expose public ingestion; they **subscribe** to the internal bus. (Optional `/events/test` for integration testing, adminonly.)

View File

@@ -190,6 +190,12 @@ When `scanner.events.enabled = true`, the WebService serialises the signed repor
* **rpm**: `/var/lib/rpm/Packages` (via librpm or parser)
* Record `name`, `version` (epoch/revision), `arch`, source package where present, and **declared file lists**.
> **Data flow note:** Each OS analyzer now writes its canonical output into the shared `ScanAnalysisStore` under
> `analysis.os.packages` (raw results), `analysis.os.fragments` (per-analyzer layer fragments), and contributes to
> `analysis.layers.fragments` (the aggregated view consumed by emit/diff pipelines). Helpers in
> `ScanAnalysisCompositionBuilder` convert these fragments into SBOM composition requests and component graphs so the
> diff/emit stages no longer reach back into individual analyzer implementations.
**B) Language ecosystems (installed state only)**
* **Java**: `META-INF/maven/*/pom.properties`, MANIFEST → `pkg:maven/...`
@@ -206,6 +212,9 @@ When `scanner.events.enabled = true`, the WebService serialises the signed repor
* **ELF**: parse `PT_INTERP`, `DT_NEEDED`, RPATH/RUNPATH, **GNU symbol versions**; map **SONAMEs** to file paths; link executables → libs.
* **PE/MachO** (planned M2): import table, delayimports; version resources; code signatures.
* Map libs back to **OS packages** if possible (via file lists); else emit `bin:{sha256}` components.
* The exported metadata (`stellaops.os.*` properties, license list, source package) feeds policy scoring and export pipelines
directly Policy evaluates quiet rules against package provenance while Exporters forward the enriched fields into
downstream JSON/Trivy payloads.
**D) EntryTrace (ENTRYPOINT/CMD → terminal program)**

View File

@@ -23,6 +23,27 @@ Safeguards: freeze boosts when product identity is unknown, clamp outputs ≥0,
| **Phase 2 Deterministic score engine** | Implement a scoring component that executes alongside consensus and persists score envelopes with hashes. | Planned task `EXCITITOR-CORE-02-002` (backlog). |
| **Phase 3 Surfacing & enforcement** | Expose scores via WebService/CLI, integrate with Concelier noise priors, and enforce policy-based suppressions. | To be scheduled after Phase 2. |
## Policy controls (Phase 1)
Operators tune scoring inputs through the Excititor policy document:
```yaml
excititor:
policy:
weights:
vendor: 1.10 # per-tier weight
ceiling: 1.40 # max clamp applied to tiers and overrides (1.05.0)
providerOverrides:
trusted.vendor: 1.35
scoring:
alpha: 0.30 # KEV boost coefficient (defaults to 0.25)
beta: 0.60 # EPSS boost coefficient (defaults to 0.50)
```
* All weights (tiers + overrides) are clamped to `[0, weights.ceiling]` with structured warnings when a value is out of range or not a finite number.
* `weights.ceiling` itself is constrained to `[1.0, 5.0]`, preserving prior behaviour when omitted.
* `scoring.alpha` / `scoring.beta` accept non-negative values up to 5.0; values outside the range fall back to defaults and surface diagnostics to operators.
## Data model (after Phase 1)
```json

View File

@@ -38,7 +38,8 @@ Everything here is opensource and versioned— when you check out a git ta
- **08Module Architecture Dossiers**
- [Scanner](ARCHITECTURE_SCANNER.md)
- [Concelier](ARCHITECTURE_CONCELIER.md)
- [Excititor](ARCHITECTURE_EXCITITOR.md)
- [Excititor](ARCHITECTURE_EXCITITOR.md)
- [Excititor Mirrors](ARCHITECTURE_EXCITITOR_MIRRORS.md)
- [Signer](ARCHITECTURE_SIGNER.md)
- [Attestor](ARCHITECTURE_ATTESTOR.md)
- [Authority](ARCHITECTURE_AUTHORITY.md)
@@ -52,6 +53,7 @@ Everything here is opensource and versioned— when you check out a git ta
- **10[Plugin SDK Guide](10_PLUGIN_SDK_GUIDE.md)**
- **10[Concelier CLI Quickstart](10_CONCELIER_CLI_QUICKSTART.md)**
- **10[BuildX Generator Quickstart](dev/BUILDX_PLUGIN_QUICKSTART.md)**
- **10[Scanner Cache Configuration](dev/SCANNER_CACHE_CONFIGURATION.md)**
- **30[Excititor Connector Packaging Guide](dev/30_EXCITITOR_CONNECTOR_GUIDE.md)**
- **30Developer Templates**
- [Excititor Connector Skeleton](dev/templates/excititor-connector/)

View File

@@ -3,7 +3,7 @@
| ID | Status | Owner(s) | Depends on | Description | Exit Criteria |
|----|--------|----------|------------|-------------|---------------|
| DOC7.README-INDEX | DONE (2025-10-17) | Docs Guild | — | Refresh index docs (docs/README.md + root README) after architecture dossier split and Offline Kit overhaul. | ✅ ToC reflects new component architecture docs; ✅ root README highlights updated doc set; ✅ Offline Kit guide linked correctly. |
| DOC4.AUTH-PDG | REVIEW | Docs Guild, Plugin Team | PLG6.DOC | Copy-edit `docs/dev/31_AUTHORITY_PLUGIN_DEVELOPER_GUIDE.md`, export lifecycle diagram, add LDAP RFC cross-link. | ✅ PR merged with polish; ✅ Diagram committed; ✅ Slack handoff posted. |
| DOC4.AUTH-PDG | DONE (2025-10-19) | Docs Guild, Plugin Team | PLG6.DOC | Copy-edit `docs/dev/31_AUTHORITY_PLUGIN_DEVELOPER_GUIDE.md`, export lifecycle diagram, add LDAP RFC cross-link. | ✅ PR merged with polish; ✅ Diagram committed; ✅ Slack handoff posted. |
| DOC1.AUTH | DONE (2025-10-12) | Docs Guild, Authority Core | CORE5B.DOC | Draft `docs/11_AUTHORITY.md` covering architecture, configuration, bootstrap flows. | ✅ Architecture + config sections approved by Core; ✅ Samples reference latest options; ✅ Offline note added. |
| DOC3.Concelier-Authority | DONE (2025-10-12) | Docs Guild, DevEx | FSR4 | Polish operator/runbook sections (DOC3/DOC5) to document Concelier authority rollout, bypass logging, and enforcement checklist. | ✅ DOC3/DOC5 updated with audit runbook references; ✅ enforcement deadline highlighted; ✅ Docs guild sign-off. |
| DOC5.Concelier-Runbook | DONE (2025-10-12) | Docs Guild | DOC3.Concelier-Authority | Produce dedicated Concelier authority audit runbook covering log fields, monitoring recommendations, and troubleshooting steps. | ✅ Runbook published; ✅ linked from DOC3/DOC5; ✅ alerting guidance included. |
@@ -11,6 +11,10 @@
| FEEDDOCS-DOCS-05-002 | DONE (2025-10-16) | Docs Guild, Concelier Ops | FEEDDOCS-DOCS-05-001 | Ops sign-off captured: conflict runbook circulated, alert thresholds tuned, and rollout decisions documented in change log. | ✅ Ops review recorded; ✅ alert thresholds finalised using `docs/ops/concelier-authority-audit-runbook.md`; ✅ change-log entry linked from runbook once GHSA/NVD/OSV regression fixtures land. |
| DOCS-ADR-09-001 | DONE (2025-10-19) | Docs Guild, DevEx | — | Establish ADR process (`docs/adr/0000-template.md`) and document usage guidelines. | Template published; README snippet linking ADR process; announcement posted (`docs/updates/2025-10-18-docs-guild.md`). |
| DOCS-EVENTS-09-002 | DONE (2025-10-19) | Docs Guild, Platform Events | SCANNER-EVENTS-15-201 | Publish event schema catalog (`docs/events/`) for `scanner.report.ready@1`, `scheduler.rescan.delta@1`, `attestor.logged@1`. | Schemas validated (Ajv CI hooked); docs/events/README summarises usage; Platform Events notified via `docs/updates/2025-10-18-docs-guild.md`. |
| DOCS-EVENTS-09-003 | DONE (2025-10-19) | Docs Guild | DOCS-EVENTS-09-002 | Add human-readable envelope field references and canonical payload samples for published events, including offline validation workflow. | Tables explain common headers/payload segments; versioned sample payloads committed; README links to validation instructions and samples. |
| DOCS-EVENTS-09-004 | DONE (2025-10-19) | Docs Guild, Scanner WebService | SCANNER-EVENTS-15-201 | Refresh scanner event docs to mirror DSSE-backed report fields, document `scanner.scan.completed`, and capture canonical sample validation. | Schemas updated for new payload shape; README references DSSE reuse and validation test; samples align with emitted events. |
| PLATFORM-EVENTS-09-401 | DONE (2025-10-19) | Platform Events Guild | DOCS-EVENTS-09-003 | Embed canonical event samples into contract/integration tests and ensure CI validates payloads against published schemas. | Notify/Scheduler contract suites exercise samples; CI job validates samples with `ajv-cli`; Platform Events changelog notes coverage. |
| RUNTIME-GUILD-09-402 | DONE (2025-10-19) | Runtime Guild | SCANNER-POLICY-09-107 | Confirm Scanner WebService surfaces `quietedFindingCount` and progress hints to runtime consumers; document readiness checklist. | Runtime verification run captures enriched payload; checklist/doc updates merged; stakeholders acknowledge availability. |
| DOCS-RUNTIME-17-004 | TODO | Docs Guild, Runtime Guild | SCANNER-EMIT-17-701, ZASTAVA-OBS-17-005, DEVOPS-REL-17-002 | Document build-id workflows: SBOM exposure, runtime event payloads, debug-store layout, and operator guidance for symbol retrieval. | Architecture + operator docs updated with build-id sections, examples show `readelf` output + debuginfod usage, references linked from Offline Kit/Release guides. |
> Update statuses (TODO/DOING/REVIEW/DONE/BLOCKED) as progress changes. Keep guides in sync with configuration samples under `etc/`.

View File

@@ -0,0 +1,50 @@
# StellaOps BOM Index (`bom-index@1`)
The BOM index is a deterministic, offline-friendly sidecar that accelerates queries for
layer-to-component membership and entrypoint usage. It is emitted alongside CycloneDX
SBOMs and consumed by Scheduler/Notify services.
## File Layout
Binary little-endian encoding, organised as the following sections:
1. **Header**
- `magic` (`byte[7]`): ASCII `"BOMIDX1"` identifier.
- `version` (`uint16`): current value `1`.
- `flags` (`uint16`): bit `0` set when entrypoint usage bitmaps are present.
- `imageDigestLength` (`uint16`) + UTF-8 digest string (e.g. `sha256:...`).
- `generatedAt` (`int64`): microseconds since Unix epoch.
- `layerCount` (`uint32`), `componentCount` (`uint32`), `entrypointCount` (`uint32`).
2. **Layer Table**
- For each layer: `length` (`uint16`) + UTF-8 layer digest (canonical order, base image → top layer).
3. **Component Table**
- For each component: `length` (`uint16`) + UTF-8 identity (CycloneDX purl when available, otherwise canonical key).
4. **Component ↦ Layer Bitmaps**
- For each component (matching table order):
- `bitmapLength` (`uint32`).
- Roaring bitmap payload (`Collections.Special.RoaringBitmap.Serialize`) encoding layer indexes that introduce or retain the component.
5. **Entrypoint Table** *(optional; present when `flags & 0x1 == 1`)*
- For each unique entrypoint/launcher string: `length` (`uint16`) + UTF-8 value (sorted ordinally).
6. **Component ↦ Entrypoint Bitmaps** *(optional)*
- For each component: roaring bitmap whose set bits reference entrypoint indexes used by EntryTrace. Empty bitmap (`length == 0`) indicates the component is not part of any resolved entrypoint closure.
## Determinism Guarantees
* Layer, component, and entrypoint tables are strictly ordered (base → top layer, lexicographically for components and entrypoints).
* Roaring bitmaps are optimised prior to serialisation and always produced from sorted indexes.
* Header timestamp is normalised to microsecond precision using UTC.
## Sample
`sample-index.bin` is generated from the integration fixture used in unit tests. It contains:
* 2 layers: `sha256:layer1`, `sha256:layer2`.
* 3 components: `pkg:npm/a`, `pkg:npm/b`, `pkg:npm/c`.
* Entrypoint bitmaps for `/app/start.sh` and `/app/init.sh`.
The sample can be decoded with the `BomIndexBuilder` unit tests or any RoaringBitmap implementation compatible with `Collections.Special.RoaringBitmap`.

Binary file not shown.

View File

@@ -5,12 +5,12 @@
## 1. Overview
Authority plug-ins extend the **StellaOps Authority** service with custom identity providers, credential stores, and client-management logic. Unlike Concelier plug-ins (which ingest or export advisories), Authority plug-ins participate directly in authentication flows:
- **Use cases:** integrate corporate directories (LDAP/AD), delegate to external IDPs, enforce bespoke password/lockout policies, or add client provisioning automation.
- **Constraints:** plug-ins load only during service start (no hot-reload), must function without outbound internet access, and must emit deterministic results for identical configuration and input data.
- **Ship targets:** target the same .NET 10 preview as the host, honour offline-first requirements, and provide clear diagnostics so operators can triage issues from `/ready`.
- **Use cases:** integrate corporate directories (LDAP/AD)[^ldap-rfc], delegate to external IDPs, enforce bespoke password/lockout policies, or add client provisioning automation.
- **Constraints:** plug-ins load only during service start (no hot-reload), must function without outbound internet access, and must emit deterministic results for identical configuration input.
- **Ship targets:** build against the hosts .NET 10 preview SDK, honour offline-first requirements, and surface actionable diagnostics so operators can triage issues from `/ready`.
## 2. Architecture Snapshot
Authority hosts follow a deterministic plug-in lifecycle. The flow below can be rendered as a sequence diagram in the final authored documentation, but all touchpoints are described here for offline viewers:
Authority hosts follow a deterministic plug-in lifecycle. The exported diagram (`docs/assets/authority/authority-plugin-lifecycle.svg`) mirrors the steps below; regenerate it from the Mermaid source if you update the flow.
1. **Configuration load** `AuthorityPluginConfigurationLoader` resolves YAML manifests under `etc/authority.plugins/`.
2. **Assembly discovery** the shared `PluginHost` scans `PluginBinaries/Authority` for `StellaOps.Authority.Plugin.*.dll` assemblies.
@@ -199,6 +199,8 @@ _Source:_ `docs/assets/authority/authority-rate-limit-flow.mmd`
- Document any external prerequisites (e.g., CA cert bundle) in your plug-in README.
- Update `etc/authority.plugins/<plugin>.yaml` samples and include deterministic SHA256 hashes for optional bootstrap payloads when distributing Offline Kit artefacts.
[^ldap-rfc]: Lightweight Directory Access Protocol (LDAPv3) specification — [RFC 4511](https://datatracker.ietf.org/doc/html/rfc4511).
## 12. Checklist & Handoff
- ✅ Capabilities declared and validated in automated tests.
- ✅ Bootstrap workflows documented (if `bootstrap` capability used) and repeatable.

View File

@@ -20,8 +20,10 @@ dotnet publish src/StellaOps.Scanner.Sbomer.BuildXPlugin/StellaOps.Scanner.Sbome
-o out/buildx
```
- `out/buildx/` now contains `StellaOps.Scanner.Sbomer.BuildXPlugin.dll` and the manifest `stellaops.sbom-indexer.manifest.json`.
- `plugins/scanner/buildx/StellaOps.Scanner.Sbomer.BuildXPlugin/` receives the same artefacts for release packaging.
- `out/buildx/` now contains `StellaOps.Scanner.Sbomer.BuildXPlugin.dll` and the manifest `stellaops.sbom-indexer.manifest.json`.
- `plugins/scanner/buildx/StellaOps.Scanner.Sbomer.BuildXPlugin/` receives the same artefacts for release packaging.
- The CI pipeline also tars and signs (SHA-256 manifest) the OS analyzer plug-ins located under
`plugins/scanner/analyzers/os/` so they ship alongside the BuildX generator artefacts.
## 3. Verify the CAS handshake

View File

@@ -0,0 +1,108 @@
# Scanner Cache Configuration Guide
The scanner cache stores layer-level SBOM fragments and file content that can be reused across scans. This document explains how to configure and operate the cache subsystem introduced in Sprint 10 (Group SP10-G5).
## 1. Overview
- **Layer cache** persists SBOM fragments per layer digest under `<root>/layers/<digest>/` with deterministic metadata (`meta.json`).
- **File CAS** (content-addressable store) keeps deduplicated blobs (e.g., analyzer fixtures, imported SBOM layers) under `<root>/cas/<prefix>/<hash>/`.
- **Maintenance** runs via `ScannerCacheMaintenanceService`, evicting expired entries and compacting the cache to stay within size limits.
- **Metrics** emit on the `StellaOps.Scanner.Cache` meter with counters for hits, misses, evictions, and byte histograms.
- **Offline workflows** use the CAS import/export helpers to package cache warmups inside the Offline Kit.
## 2. Configuration keys (`scanner:cache`)
| Key | Default | Description |
| --- | --- | --- |
| `enabled` | `true` | Globally disable cache if `false`. |
| `rootPath` | `cache/scanner` | Base directory for cache data. Use an SSD-backed path for best warm-scan latency. |
| `layersDirectoryName` | `layers` | Subdirectory for layer cache entries. |
| `fileCasDirectoryName` | `cas` | Subdirectory for file CAS entries. |
| `layerTtl` | `45.00:00:00` | Time-to-live for layer cache entries (`TimeSpan`). `0` disables TTL eviction. |
| `fileTtl` | `30.00:00:00` | Time-to-live for CAS entries. `0` disables TTL eviction. |
| `maxBytes` | `5368709120` (5 GiB) | Hard cap for combined cache footprint. Compaction trims data back to `warmBytesThreshold`. |
| `warmBytesThreshold` | `maxBytes / 5` | Target size after compaction. |
| `coldBytesThreshold` | `maxBytes * 0.8` | Upper bound that triggers compaction. |
| `enableAutoEviction` | `true` | If `false`, callers must invoke `ILayerCacheStore.CompactAsync` / `IFileContentAddressableStore.CompactAsync` manually. |
| `maintenanceInterval` | `00:15:00` | Interval for the maintenance hosted service. |
| `enableFileCas` | `true` | Disable to prevent CAS usage (APIs throw on `PutAsync`). |
| `importDirectory` / `exportDirectory` | `null` | Optional defaults for offline import/export tooling. |
> **Tip:** configure `scanner:cache:rootPath` to a dedicated volume and mount it into worker containers when running in Kubernetes or Nomad.
## 3. Metrics
Instrumentation lives in `ScannerCacheMetrics` on meter `StellaOps.Scanner.Cache`.
| Instrument | Unit | Description |
| --- | --- | --- |
| `scanner.layer_cache_hits_total` | count | Layer cache hit counter. Tag: `layer`. |
| `scanner.layer_cache_misses_total` | count | Layer cache miss counter. Tag: `layer`. |
| `scanner.layer_cache_evictions_total` | count | Layer entries evicted due to TTL or compaction. Tag: `layer`. |
| `scanner.layer_cache_bytes` | bytes | Histogram of per-entry payload size when stored. |
| `scanner.file_cas_hits_total` | count | File CAS hit counter. Tag: `sha256`. |
| `scanner.file_cas_misses_total` | count | File CAS miss counter. Tag: `sha256`. |
| `scanner.file_cas_evictions_total` | count | CAS eviction counter. Tag: `sha256`. |
| `scanner.file_cas_bytes` | bytes | Histogram of CAS payload sizes on insert. |
## 4. Import / Export workflow
1. **Export warm cache**
```bash
dotnet tool run stellaops-cache export --destination ./offline-kit/cache
```
Internally this calls `IFileContentAddressableStore.ExportAsync` which copies each CAS entry (metadata + `content.bin`).
2. **Import on air-gapped hosts**
```bash
dotnet tool run stellaops-cache import --source ./offline-kit/cache
```
The import API merges newer metadata and skips older snapshots automatically.
3. **Layer cache seeding**
Layer cache entries are deterministic and can be packaged the same way (copy `<root>/layers`). For now we keep seeding optional because layers are larger; follow-up tooling can compress directories as needed.
## 5. Hosted maintenance loop
`ScannerCacheMaintenanceService` runs as a background service within Scanner Worker or WebService hosts when `AddScannerCache` is registered. Behaviour:
- At startup it performs an immediate eviction/compaction run.
- Every `maintenanceInterval` it triggers:
- `ILayerCacheStore.EvictExpiredAsync`
- `ILayerCacheStore.CompactAsync`
- `IFileContentAddressableStore.EvictExpiredAsync`
- `IFileContentAddressableStore.CompactAsync`
- Failures are logged at `Error` with preserved stack traces; the next tick continues normally.
Set `enableAutoEviction=false` when hosting the cache inside ephemeral build pipelines that want to drive eviction explicitly.
## 6. API surface summary
```csharp
public interface ILayerCacheStore
{
ValueTask<LayerCacheEntry?> TryGetAsync(string layerDigest, CancellationToken ct = default);
Task<LayerCacheEntry> PutAsync(LayerCachePutRequest request, CancellationToken ct = default);
Task RemoveAsync(string layerDigest, CancellationToken ct = default);
Task<int> EvictExpiredAsync(CancellationToken ct = default);
Task<int> CompactAsync(CancellationToken ct = default);
Task<Stream?> OpenArtifactAsync(string layerDigest, string artifactName, CancellationToken ct = default);
}
public interface IFileContentAddressableStore
{
ValueTask<FileCasEntry?> TryGetAsync(string sha256, CancellationToken ct = default);
Task<FileCasEntry> PutAsync(FileCasPutRequest request, CancellationToken ct = default);
Task<bool> RemoveAsync(string sha256, CancellationToken ct = default);
Task<int> EvictExpiredAsync(CancellationToken ct = default);
Task<int> CompactAsync(CancellationToken ct = default);
Task<int> ExportAsync(string destinationDirectory, CancellationToken ct = default);
Task<int> ImportAsync(string sourceDirectory, CancellationToken ct = default);
}
```
Register both stores via `services.AddScannerCache(configuration);` in WebService or Worker hosts.
---
_Last updated: 2025-10-19_

View File

@@ -0,0 +1,140 @@
# Authority DPoP & mTLS Implementation Plan (2025-10-19)
## Purpose
- Provide the implementation blueprint for AUTH-DPOP-11-001 and AUTH-MTLS-11-002.
- Unify sender-constraint validation across Authority, downstream services, and clients.
- Capture deterministic, testable steps that unblock UI/Signer guilds depending on DPoP/mTLS hardening.
## Scope
- Token endpoint validation, issuance, and storage changes inside `StellaOps.Authority`.
- Shared security primitives consumed by Authority, Scanner, Signer, CLI, and UI.
- Operator-facing configuration, auditing, and observability.
- Out of scope: PoE enforcement (Signer) and CLI/UI client UX; those teams consume the new capabilities.
## Design Summary
- Extract the existing Scanner `DpopProofValidator` stack into a shared `StellaOps.Auth.Security` library used by Authority and resource servers.
- Extend Authority configuration (`authority.yaml`) with strongly-typed `senderConstraints.dpop` and `senderConstraints.mtls` sections (map to sample already shown in architecture doc).
- Require DPoP proofs on `/token` when the registered client policy is `senderConstraint=dpop`; bind issued access tokens via `cnf.jkt`.
- Introduce Authority-managed nonce issuance for “high value” audiences (default: `signer`, `attestor`) with Redis-backed persistence and deterministic auditing.
- Enable OAuth 2.0 mTLS (RFC 8705) by storing certificate bindings per client, requesting client certificates at TLS termination, and stamping `cnf.x5t#S256` into issued tokens plus introspection output.
- Surface structured logs and counters for both DPoP and mTLS flows; provide integration tests that cover success, replay, invalid proof, and certificate mismatch cases.
## AUTH-DPOP-11-001 — Proof Validation & Nonce Handling
**Shared validator**
- Move `DpopProofValidator`, option types, and replay cache interfaces from `StellaOps.Scanner.Core` into a new assembly `StellaOps.Auth.Security`.
- Provide pluggable caches: `InMemoryDpopReplayCache` (existing) and new `RedisDpopReplayCache` (leveraging the Authority Redis connection).
- Ensure the validator exposes the validated `SecurityKey`, `jti`, and `iat` so Authority can construct the `cnf` claim and compute nonce expiry.
**Configuration model**
- Extend `StellaOpsAuthorityOptions.Security` with a `SenderConstraints` property containing:
- `Dpop` (`enabled`, `allowedAlgorithms`, `maxAgeSeconds`, `clockSkewSeconds`, `replayWindowSeconds`, `nonce` settings with `enabled`, `ttlSeconds`, `requiredAudiences`, `maxIssuancePerMinute`).
- `Mtls` (`enabled`, `requireChainValidation`, `clientCaBundle`, `allowedSubjectPatterns`, `allowedSanTypes`).
- Bind from YAML (`authority.security.senderConstraints.*`) while preserving backwards compatibility (defaults keep both disabled).
**Token endpoint pipeline**
- Introduce a scoped OpenIddict handler `ValidateDpopProofHandler` inserted before `ValidateClientCredentialsHandler`.
- Determine the required sender constraint from client metadata:
- Add `AuthorityClientMetadataKeys.SenderConstraint` storing `dpop` or `mtls`.
- Optionally allow per-client overrides for nonce requirement.
- When `dpop` is required:
- Read the `DPoP` header from the ASP.NET request, reject with `invalid_token` + `WWW-Authenticate: DPoP error="invalid_dpop_proof"` if absent.
- Call the shared validator with method/URI. Enforce algorithm allowlist and `iat` window from options.
- Persist the `jkt` thumbprint plus replay cache state in the OpenIddict transaction (`AuthorityOpenIddictConstants.DpopKeyThumbprintProperty`, `DpopIssuedAtProperty`).
- When the requested audience intersects `SenderConstraints.Dpop.Nonce.RequiredAudiences`, require `nonce` in the proof; on first failure respond with HTTP 401, `error="use_dpop_nonce"`, and include `DPoP-Nonce` header (see nonce note below). Cache the rejection reason for audit logging.
**Nonce service**
- Add `IDpopNonceStore` with methods `IssueAsync(audience, clientId, jkt)` and `TryConsumeAsync(nonce, audience, clientId, jkt)`.
- Default implementation `RedisDpopNonceStore` storing SHA-256 hashes of nonces keyed by `audience:clientId:jkt`. TTL comes from `SenderConstraints.Dpop.Nonce.Ttl`.
- Create helper `DpopNonceIssuer` used by `ValidateDpopProofHandler` to issue nonces when missing/expired, enforcing issuance rate limits (per options) and tagging audit/log records.
- On successful validation (nonce supplied and consumed) stamp metadata into the transaction for auditing.
- Update `ClientCredentialsHandlers` to observe nonce enforcement: when a nonce challenge was sent, emit structured audit with `nonce_issued`, `audiences`, and `retry`.
**Token issuance**
- In `HandleClientCredentialsHandler`, if the transaction contains a validated DPoP key:
- Build `cnf.jkt` using thumbprint from validator.
- Include `auth_time`/`dpop_jti` as needed for diagnostics.
- Persist the thumbprint alongside token metadata in Mongo (extend `AuthorityTokenDocument` with `SenderConstraint`, `KeyThumbprint`, `Nonce` fields).
**Auditing & observability**
- Emit new audit events:
- `authority.dpop.proof.validated` (success/failure, clientId, audience, thumbprint, nonce status, jti).
- `authority.dpop.nonce.issued` and `authority.dpop.nonce.consumed`.
- Metrics (Prometheus style):
- `authority_dpop_validations_total{result,reason}`.
- `authority_dpop_nonce_issued_total{audience}` and `authority_dpop_nonce_fails_total{reason}`.
- Structured logs include `authority.sender_constraint=dpop`, `authority.dpop_thumbprint`, `authority.dpop_nonce`.
**Testing**
- Unit tests for the handler pipeline using fake OpenIddict transactions.
- Replay/nonce tests with in-memory and Redis stores.
- Integration tests in `StellaOps.Authority.Tests` covering:
- Valid DPoP proof issuing `cnf.jkt`.
- Missing header → challenge with nonce.
- Replayed `jti` rejected.
- Invalid nonce rejected even after issuance.
- Contract tests to ensure `/.well-known/openid-configuration` advertises `dpop_signing_alg_values_supported` and `dpop_nonce_supported` when enabled.
## AUTH-MTLS-11-002 — Certificate-Bound Tokens
**Configuration model**
- Reuse `SenderConstraints.Mtls` described above; include:
- `enforceForAudiences` list (defaults `signer`, `attestor`, `scheduler`).
- `certificateRotationGraceSeconds` for overlap.
- `allowedClientCertificateAuthorities` absolute paths.
**Kestrel/TLS pipeline**
- Configure Kestrel with `ClientCertificateMode.AllowCertificate` globally and implement middleware that enforces certificate presence only when the resolved client requires mTLS.
- Add `IAuthorityClientCertificateValidator` that validates presented certificate chain, SANs (`dns`, `uri`, optional SPIFFE), and thumbprint matches one of the stored bindings.
- Cache validation results per connection id to avoid rehashing on every request.
**Client registration & storage**
- Extend `AuthorityClientDocument` with `List<AuthorityClientCertificateBinding>` containing:
- `Thumbprint`, `SerialNumber`, `Subject`, `NotBefore`, `NotAfter`, `Sans`, `CreatedAt`, `UpdatedAt`, `Label`.
- Provide admin API mutations (`/admin/clients/{id}/certificates`) for ops tooling (deferred implementation but schema ready).
- Update plugin provisioning store (`StandardClientProvisioningStore`) to map descriptors with certificate bindings and `senderConstraint`.
- Persist binding state in Mongo migrations (index on `{clientId, thumbprint}`).
**Token issuance & introspection**
- Add a transaction property capturing the validated client certificate thumbprint.
- `HandleClientCredentialsHandler`:
- When mTLS required, ensure certificate info present; reject otherwise.
- Stamp `cnf` claim: `principal.SetClaim("cnf", JsonSerializer.Serialize(new { x5t#S256 = thumbprint }))`.
- Store binding metadata in issued token document for audit.
- Update `ValidateAccessTokenHandler` and introspection responses to surface `cnf.x5t#S256`.
- Ensure refresh tokens (if ever enabled) copy the binding data.
**Auditing & observability**
- Audit events:
- `authority.mtls.handshake` (success/failure, clientId, thumbprint, issuer, subject).
- `authority.mtls.binding.missing` when a required client posts without a cert.
- Metrics:
- `authority_mtls_handshakes_total{result}`.
- `authority_mtls_certificate_rotations_total`.
- Logs include `authority.sender_constraint=mtls`, `authority.mtls_thumbprint`, `authority.mtls_subject`.
**Testing**
- Unit tests for certificate validation rules (SAN mismatches, expiry, CA trust).
- Integration tests running Kestrel with test certificates:
- Successful token issuance with bound certificate.
- Request without certificate → `invalid_client`.
- Token introspection reveals `cnf.x5t#S256`.
- Rotation scenario (old + new cert allowed during grace window).
## Implementation Checklist
**DPoP work-stream**
1. Extract shared validator into `StellaOps.Auth.Security`; update Scanner references.
2. Introduce configuration classes and bind from YAML/environment.
3. Implement nonce store (Redis + in-memory), handler integration, and OpenIddict transaction plumbing.
4. Stamp `cnf.jkt`, audit events, and metrics; update Mongo documents and migrations.
5. Extend docs: `docs/ARCHITECTURE_AUTHORITY.md`, `docs/security/audit-events.md`, `docs/security/rate-limits.md`, CLI/UI references.
**mTLS work-stream**
1. Extend client document/schema and provisioning stores with certificate bindings + sender constraint flag.
2. Configure Kestrel/middleware for optional client certificates and validation service.
3. Update token issuance/introspection to honour certificate bindings and emit `cnf.x5t#S256`.
4. Add auditing/metrics and integration tests (happy path + failure).
5. Refresh operator documentation (`docs/ops/authority-backup-restore.md`, `docs/ops/authority-monitoring.md`, sample `authority.yaml`) to cover certificate lifecycle.
Both streams should conclude with `dotnet test src/StellaOps.Authority.sln` and documentation cross-links so dependent guilds can unblock UI/Signer work.

View File

@@ -3,12 +3,47 @@
Platform services publish strongly typed events; the JSON Schemas in this directory define those envelopes. File names follow `<event-name>@<version>.json` so producers and consumers can negotiate contracts explicitly.
## Catalog
- `scanner.report.ready@1.json` — emitted by Scanner.WebService once a signed report is persisted. Consumers: Notify, UI timeline.
- `scanner.report.ready@1.json` — emitted by Scanner.WebService once a signed report is persisted (payload embeds the canonical report plus DSSE envelope). Consumers: Notify, UI timeline.
- `scanner.scan.completed@1.json` — emitted alongside the signed report to capture scan outcomes/summary data for downstream automation. Consumers: Notify, Scheduler backfills, UI timelines.
- `scheduler.rescan.delta@1.json` — emitted by Scheduler when BOM-Index diffs require fresh scans. Consumers: Notify, Policy Engine.
- `attestor.logged@1.json` — emitted by Attestor after storing the Rekor inclusion proof. Consumers: UI attestation panel, Governance exports.
Additive payload changes (new optional fields) can stay within the same version. Any breaking change (removing a field, tightening validation, altering semantics) must increment the `@<version>` suffix and update downstream consumers.
## Envelope structure
All event envelopes share the same deterministic header. Use the following table as the quick reference when emitting or parsing events:
| Field | Type | Notes |
|-------|------|-------|
| `eventId` | `uuid` | Must be globally unique per occurrence; producers log duplicates as fatal. |
| `kind` | `string` | Fixed per schema (e.g., `scanner.report.ready`). Downstream services reject unknown kinds or versions. |
| `tenant` | `string` | Multitenant isolation key; mirror the value recorded in queue/Mongo metadata. |
| `ts` | `date-time` | RFC3339 UTC timestamp. Use monotonic clocks or atomic offsets so ordering survives retries. |
| `scope` | `object` | Optional block used when the event concerns a specific image or repository. See schema for required fields (e.g., `repo`, `digest`). |
| `payload` | `object` | Event-specific body. Schemas allow additional properties so producers can add optional hints (e.g., `reportId`, `quietedFindingCount`) without breaking consumers. For scanner events, payloads embed both the canonical report document and the DSSE envelope so consumers can reuse signatures without recomputing them. See `docs/runtime/SCANNER_RUNTIME_READINESS.md` for the runtime consumer checklist covering these hints. |
When adding new optional fields, document the behaviour in the schemas `description` block and update the consumer checklist in the next sprint sync.
## Canonical samples & validation
Reference payloads live under `docs/events/samples/`, mirroring the schema version (`<event-name>@<version>.sample.json`). They illustrate common field combinations, including the optional attributes that downstream teams rely on for UI affordances and audit trails. Scanner samples reuse the exact DSSE envelope checked into `samples/api/reports/report-sample.dsse.json`, and a unit test (`ReportSamplesTests`) guards that the payload/base64 remain canonical.
Run the following loop offline to validate both schemas and samples:
```bash
# Validate schemas (same check as CI)
for schema in docs/events/*.json; do
npx ajv compile -c ajv-formats -s "$schema"
done
# Validate canonical samples against their schemas
for sample in docs/events/samples/*.sample.json; do
schema="docs/events/$(basename "${sample%.sample.json}").json"
npx ajv validate -c ajv-formats -s "$schema" -d "$sample"
done
```
Consumers can copy the samples into integration tests to guarantee backwards compatibility. When emitting new event versions, include a matching sample and update this README so air-gapped operators stay in sync.
## CI validation
The Docs CI workflow (`.gitea/workflows/docs.yml`) installs `ajv-cli` and compiles every schema on pull requests. Run the same check locally before opening a PR:
@@ -25,6 +60,6 @@ If a schema references additional files, include `-r` flags so CI and local runs
## Working with schemas
- Producers should validate outbound payloads using the matching schema during unit tests.
- Consumers should pin to a specific version and log when encountering unknown versions to catch missing migrations early.
- Store real payload samples under `samples/events/` (mirrors the schema version) to aid contract testing.
- Store real payload samples under `docs/events/samples/` (mirrors the schema version) and mirror them into `samples/events/` when you need fixtures in integration repositories.
Contact the Platform Events group in Docs Guild if you need help shaping a new event or version strategy.

View File

@@ -0,0 +1,21 @@
{
"eventId": "1fdcaa1a-7a27-4154-8bac-cf813d8f4f6f",
"kind": "attestor.logged",
"tenant": "tenant-acme-solar",
"ts": "2025-10-18T15:45:27+00:00",
"payload": {
"artifactSha256": "sha256:8927d9151ad3f44e61a9c647511f9a31af2b4d245e7e031fe5cb4a0e8211c5d9",
"dsseEnvelopeDigest": "sha256:51c1dd189d5f16cfe87e82841d67b4fbc27d6fa9f5a09af0cd7e18945fb4c2a9",
"rekor": {
"index": 563421,
"url": "https://rekor.example/api/v1/log/entries/d6d0f897e7244edc9cb0bb2c68b05c96",
"uuid": "d6d0f897e7244edc9cb0bb2c68b05c96"
},
"signer": "cosign-stellaops",
"subject": {
"name": "scanner/report/sha256-0f0a8de5c1f93d6716b7249f6f4ea3a8",
"type": "report"
}
},
"attributes": {}
}

View File

@@ -0,0 +1,70 @@
{
"eventId": "6d2d1b77-f3c3-4f70-8a9d-6f2d0c8801ab",
"kind": "scanner.report.ready",
"tenant": "tenant-alpha",
"ts": "2025-10-19T12:34:56+00:00",
"scope": {
"namespace": "acme/edge",
"repo": "api",
"digest": "sha256:feedface",
"labels": {},
"attributes": {}
},
"payload": {
"delta": {
"kev": ["CVE-2024-9999"],
"newCritical": 1
},
"dsse": {
"payload": "eyJyZXBvcnRJZCI6InJlcG9ydC1hYmMiLCJpbWFnZURpZ2VzdCI6InNoYTI1NjpmZWVkZmFjZSIsImdlbmVyYXRlZEF0IjoiMjAyNS0xMC0xOVQxMjozNDo1NiswMDowMCIsInZlcmRpY3QiOiJibG9ja2VkIiwicG9saWN5Ijp7InJldmlzaW9uSWQiOiJyZXYtNDIiLCJkaWdlc3QiOiJkaWdlc3QtMTIzIn0sInN1bW1hcnkiOnsidG90YWwiOjEsImJsb2NrZWQiOjEsIndhcm5lZCI6MCwiaWdub3JlZCI6MCwicXVpZXRlZCI6MH0sInZlcmRpY3RzIjpbeyJmaW5kaW5nSWQiOiJmaW5kaW5nLTEiLCJzdGF0dXMiOiJCbG9ja2VkIiwic2NvcmUiOjQ3LjUsInNvdXJjZVRydXN0IjoiTlZEIiwicmVhY2hhYmlsaXR5IjoicnVudGltZSJ9XSwiaXNzdWVzIjpbXX0=",
"payloadType": "application/vnd.stellaops.report\u002Bjson",
"signatures": [{
"algorithm": "hs256",
"keyId": "test-key",
"signature": "signature-value"
}]
},
"generatedAt": "2025-10-19T12:34:56+00:00",
"links": {
"ui": "https://scanner.example/ui/reports/report-abc"
},
"quietedFindingCount": 0,
"report": {
"generatedAt": "2025-10-19T12:34:56+00:00",
"imageDigest": "sha256:feedface",
"issues": [],
"policy": {
"digest": "digest-123",
"revisionId": "rev-42"
},
"reportId": "report-abc",
"summary": {
"blocked": 1,
"ignored": 0,
"quieted": 0,
"total": 1,
"warned": 0
},
"verdict": "blocked",
"verdicts": [
{
"findingId": "finding-1",
"status": "Blocked",
"score": 47.5,
"sourceTrust": "NVD",
"reachability": "runtime"
}
]
},
"reportId": "report-abc",
"summary": {
"blocked": 1,
"ignored": 0,
"quieted": 0,
"total": 1,
"warned": 0
},
"verdict": "fail"
},
"attributes": {}
}

View File

@@ -0,0 +1,78 @@
{
"eventId": "08a6de24-4a94-4d14-8432-9d14f36f6da3",
"kind": "scanner.scan.completed",
"tenant": "tenant-alpha",
"ts": "2025-10-19T12:34:56+00:00",
"scope": {
"namespace": "acme/edge",
"repo": "api",
"digest": "sha256:feedface",
"labels": {},
"attributes": {}
},
"payload": {
"delta": {
"kev": ["CVE-2024-9999"],
"newCritical": 1
},
"digest": "sha256:feedface",
"dsse": {
"payload": "eyJyZXBvcnRJZCI6InJlcG9ydC1hYmMiLCJpbWFnZURpZ2VzdCI6InNoYTI1NjpmZWVkZmFjZSIsImdlbmVyYXRlZEF0IjoiMjAyNS0xMC0xOVQxMjozNDo1NiswMDowMCIsInZlcmRpY3QiOiJibG9ja2VkIiwicG9saWN5Ijp7InJldmlzaW9uSWQiOiJyZXYtNDIiLCJkaWdlc3QiOiJkaWdlc3QtMTIzIn0sInN1bW1hcnkiOnsidG90YWwiOjEsImJsb2NrZWQiOjEsIndhcm5lZCI6MCwiaWdub3JlZCI6MCwicXVpZXRlZCI6MH0sInZlcmRpY3RzIjpbeyJmaW5kaW5nSWQiOiJmaW5kaW5nLTEiLCJzdGF0dXMiOiJCbG9ja2VkIiwic2NvcmUiOjQ3LjUsInNvdXJjZVRydXN0IjoiTlZEIiwicmVhY2hhYmlsaXR5IjoicnVudGltZSJ9XSwiaXNzdWVzIjpbXX0=",
"payloadType": "application/vnd.stellaops.report\u002Bjson",
"signatures": [{
"algorithm": "hs256",
"keyId": "test-key",
"signature": "signature-value"
}]
},
"findings": [
{
"cve": "CVE-2024-9999",
"id": "finding-1",
"reachability": "runtime",
"severity": "Critical"
}
],
"policy": {
"digest": "digest-123",
"revisionId": "rev-42"
},
"report": {
"generatedAt": "2025-10-19T12:34:56+00:00",
"imageDigest": "sha256:feedface",
"issues": [],
"policy": {
"digest": "digest-123",
"revisionId": "rev-42"
},
"reportId": "report-abc",
"summary": {
"blocked": 1,
"ignored": 0,
"quieted": 0,
"total": 1,
"warned": 0
},
"verdict": "blocked",
"verdicts": [
{
"findingId": "finding-1",
"status": "Blocked",
"score": 47.5,
"sourceTrust": "NVD",
"reachability": "runtime"
}
]
},
"reportId": "report-abc",
"summary": {
"blocked": 1,
"ignored": 0,
"quieted": 0,
"total": 1,
"warned": 0
},
"verdict": "fail"
},
"attributes": {}
}

View File

@@ -0,0 +1,20 @@
{
"eventId": "51d0ef8d-3a17-4af3-b2d7-4ad3db3d9d2c",
"kind": "scheduler.rescan.delta",
"tenant": "tenant-acme-solar",
"ts": "2025-10-18T15:40:11+00:00",
"payload": {
"impactedDigests": [
"sha256:0f0a8de5c1f93d6716b7249f6f4ea3a8db451dc3f3c3ff823f53c9cbde5d5e8a",
"sha256:ab921f9679dd8d0832f3710a4df75dbadbd58c2d95f26a4d4efb2fa8c3d9b4ce"
],
"reason": "policy-change:scoring/v2",
"scheduleId": "rescan-weekly-critical",
"summary": {
"newCritical": 0,
"newHigh": 1,
"total": 4
}
},
"attributes": {}
}

View File

@@ -21,14 +21,28 @@
"type": "object",
"required": ["verdict", "delta", "links"],
"properties": {
"reportId": {"type": "string"},
"generatedAt": {"type": "string", "format": "date-time"},
"verdict": {"enum": ["pass", "warn", "fail"]},
"summary": {
"type": "object",
"properties": {
"total": {"type": "integer", "minimum": 0},
"blocked": {"type": "integer", "minimum": 0},
"warned": {"type": "integer", "minimum": 0},
"ignored": {"type": "integer", "minimum": 0},
"quieted": {"type": "integer", "minimum": 0}
},
"additionalProperties": false
},
"delta": {
"type": "object",
"properties": {
"newCritical": {"type": "integer", "minimum": 0},
"newHigh": {"type": "integer", "minimum": 0},
"kev": {"type": "array", "items": {"type": "string"}}
}
},
"additionalProperties": false
},
"links": {
"type": "object",
@@ -37,6 +51,30 @@
"rekor": {"type": "string", "format": "uri"}
},
"additionalProperties": false
},
"quietedFindingCount": {"type": "integer", "minimum": 0},
"report": {"type": "object"},
"dsse": {
"type": "object",
"required": ["payloadType", "payload", "signatures"],
"properties": {
"payloadType": {"type": "string"},
"payload": {"type": "string"},
"signatures": {
"type": "array",
"items": {
"type": "object",
"required": ["keyId", "algorithm", "signature"],
"properties": {
"keyId": {"type": "string"},
"algorithm": {"type": "string"},
"signature": {"type": "string"}
},
"additionalProperties": false
}
}
},
"additionalProperties": false
}
},
"additionalProperties": true

View File

@@ -0,0 +1,97 @@
{
"$id": "https://stella-ops.org/schemas/events/scanner.scan.completed@1.json",
"$schema": "http://json-schema.org/draft-07/schema#",
"type": "object",
"required": ["eventId", "kind", "tenant", "ts", "scope", "payload"],
"properties": {
"eventId": {"type": "string", "format": "uuid"},
"kind": {"const": "scanner.scan.completed"},
"tenant": {"type": "string"},
"ts": {"type": "string", "format": "date-time"},
"scope": {
"type": "object",
"required": ["repo", "digest"],
"properties": {
"namespace": {"type": "string"},
"repo": {"type": "string"},
"digest": {"type": "string"}
}
},
"payload": {
"type": "object",
"required": ["reportId", "digest", "verdict", "summary"],
"properties": {
"reportId": {"type": "string"},
"digest": {"type": "string"},
"verdict": {"enum": ["pass", "warn", "fail"]},
"summary": {
"type": "object",
"properties": {
"total": {"type": "integer", "minimum": 0},
"blocked": {"type": "integer", "minimum": 0},
"warned": {"type": "integer", "minimum": 0},
"ignored": {"type": "integer", "minimum": 0},
"quieted": {"type": "integer", "minimum": 0}
},
"additionalProperties": false
},
"delta": {
"type": "object",
"properties": {
"newCritical": {"type": "integer", "minimum": 0},
"newHigh": {"type": "integer", "minimum": 0},
"kev": {"type": "array", "items": {"type": "string"}}
},
"additionalProperties": false
},
"policy": {
"type": "object",
"properties": {
"revisionId": {"type": "string"},
"digest": {"type": "string"}
},
"additionalProperties": false
},
"findings": {
"type": "array",
"items": {
"type": "object",
"properties": {
"id": {"type": "string"},
"severity": {"type": "string"},
"cve": {"type": "string"},
"purl": {"type": "string"},
"reachability": {"type": "string"}
},
"additionalProperties": true
}
},
"report": {"type": "object"},
"dsse": {
"type": "object",
"required": ["payloadType", "payload", "signatures"],
"properties": {
"payloadType": {"type": "string"},
"payload": {"type": "string"},
"signatures": {
"type": "array",
"items": {
"type": "object",
"required": ["keyId", "algorithm", "signature"],
"properties": {
"keyId": {"type": "string"},
"algorithm": {"type": "string"},
"signature": {"type": "string"}
},
"additionalProperties": false
}
}
},
"additionalProperties": false
}
},
"additionalProperties": true
}
},
"additionalProperties": false
}

View File

@@ -0,0 +1,32 @@
{
"schemaVersion": "notify.channel@1",
"channelId": "channel-slack-sec-ops",
"tenantId": "tenant-01",
"name": "slack:sec-ops",
"type": "slack",
"displayName": "SecOps Slack",
"description": "Primary incident response channel.",
"config": {
"secretRef": "ref://notify/channels/slack/sec-ops",
"target": "#sec-ops",
"properties": {
"workspace": "stellaops-sec"
},
"limits": {
"concurrency": 2,
"requestsPerMinute": 60,
"timeout": "PT10S"
}
},
"enabled": true,
"labels": {
"team": "secops"
},
"metadata": {
"createdByTask": "NOTIFY-MODELS-15-102"
},
"createdBy": "ops:amir",
"createdAt": "2025-10-18T17:02:11+00:00",
"updatedBy": "ops:amir",
"updatedAt": "2025-10-18T17:45:00+00:00"
}

View File

@@ -0,0 +1,34 @@
{
"eventId": "8a8d6a2f-9315-49fe-9d52-8fec79ec7aeb",
"kind": "scanner.report.ready",
"version": "1",
"tenant": "tenant-01",
"ts": "2025-10-19T03:58:42+00:00",
"actor": "scanner-webservice",
"scope": {
"namespace": "prod-payment",
"repo": "ghcr.io/acme/api",
"digest": "sha256:79c1f9e5...",
"labels": {
"environment": "production"
},
"attributes": {}
},
"payload": {
"delta": {
"kev": [
"CVE-2025-40123"
],
"newCritical": 1,
"newHigh": 2
},
"links": {
"rekor": "https://rekor.stella.local/api/v1/log/entries/1",
"ui": "https://ui.stella.local/reports/sha256-79c1f9e5"
},
"verdict": "fail"
},
"attributes": {
"correlationId": "scan-23a6"
}
}

View File

@@ -0,0 +1,63 @@
{
"schemaVersion": "notify.rule@1",
"ruleId": "rule-secops-critical",
"tenantId": "tenant-01",
"name": "Critical digests to SecOps",
"description": "Escalate KEV-tagged findings to on-call feeds.",
"enabled": true,
"match": {
"eventKinds": [
"scanner.report.ready",
"scheduler.rescan.delta"
],
"namespaces": [
"prod-*"
],
"repositories": [],
"digests": [],
"labels": [],
"componentPurls": [],
"minSeverity": "high",
"verdicts": [],
"kevOnly": true,
"vex": {
"includeAcceptedJustifications": false,
"includeRejectedJustifications": false,
"includeUnknownJustifications": false,
"justificationKinds": [
"component-remediated",
"not-affected"
]
}
},
"actions": [
{
"actionId": "email-digest",
"channel": "email:soc",
"digest": "hourly",
"template": "digest",
"enabled": true,
"metadata": {
"locale": "en-us"
}
},
{
"actionId": "slack-oncall",
"channel": "slack:sec-ops",
"template": "concise",
"throttle": "PT5M",
"metadata": {},
"enabled": true
}
],
"labels": {
"team": "secops"
},
"metadata": {
"source": "sprint-15"
},
"createdBy": "ops:zoya",
"createdAt": "2025-10-19T04:12:27+00:00",
"updatedBy": "ops:zoya",
"updatedAt": "2025-10-19T04:45:03+00:00"
}

View File

@@ -0,0 +1,19 @@
{
"schemaVersion": "notify.template@1",
"templateId": "tmpl-slack-concise",
"tenantId": "tenant-01",
"channelType": "slack",
"key": "concise",
"locale": "en-us",
"body": "{{severity_icon payload.delta.newCritical}} {{summary}}",
"description": "Slack concise message for high severity findings.",
"renderMode": "markdown",
"format": "slack",
"metadata": {
"version": "2025-10-19"
},
"createdBy": "ops:zoya",
"createdAt": "2025-10-19T05:00:00+00:00",
"updatedBy": "ops:zoya",
"updatedAt": "2025-10-19T05:45:00+00:00"
}

View File

@@ -0,0 +1,73 @@
{
"$id": "https://stella-ops.org/schemas/notify/notify-channel@1.json",
"$schema": "http://json-schema.org/draft-07/schema#",
"title": "Notify Channel",
"type": "object",
"required": [
"schemaVersion",
"channelId",
"tenantId",
"name",
"type",
"config",
"enabled",
"createdAt",
"updatedAt"
],
"properties": {
"schemaVersion": {"type": "string", "const": "notify.channel@1"},
"channelId": {"type": "string"},
"tenantId": {"type": "string"},
"name": {"type": "string"},
"type": {
"type": "string",
"enum": ["slack", "teams", "email", "webhook", "custom"]
},
"displayName": {"type": "string"},
"description": {"type": "string"},
"config": {"$ref": "#/$defs/channelConfig"},
"enabled": {"type": "boolean"},
"labels": {"$ref": "#/$defs/stringMap"},
"metadata": {"$ref": "#/$defs/stringMap"},
"createdBy": {"type": "string"},
"createdAt": {"type": "string", "format": "date-time"},
"updatedBy": {"type": "string"},
"updatedAt": {"type": "string", "format": "date-time"}
},
"additionalProperties": false,
"$defs": {
"channelConfig": {
"type": "object",
"required": ["secretRef"],
"properties": {
"secretRef": {"type": "string"},
"target": {"type": "string"},
"endpoint": {"type": "string", "format": "uri"},
"properties": {"$ref": "#/$defs/stringMap"},
"limits": {"$ref": "#/$defs/channelLimits"}
},
"additionalProperties": false
},
"channelLimits": {
"type": "object",
"properties": {
"concurrency": {"type": "integer", "minimum": 1},
"requestsPerMinute": {"type": "integer", "minimum": 1},
"timeout": {
"type": "string",
"pattern": "^P(T.*)?$",
"description": "ISO 8601 duration"
},
"maxBatchSize": {"type": "integer", "minimum": 1}
},
"additionalProperties": false
},
"stringMap": {
"type": "object",
"patternProperties": {
".*": {"type": "string"}
},
"additionalProperties": false
}
}
}

View File

@@ -0,0 +1,56 @@
{
"$id": "https://stella-ops.org/schemas/notify/notify-event@1.json",
"$schema": "http://json-schema.org/draft-07/schema#",
"title": "Notify Event Envelope",
"type": "object",
"required": ["eventId", "kind", "tenant", "ts", "payload"],
"properties": {
"eventId": {"type": "string", "format": "uuid"},
"kind": {
"type": "string",
"description": "Event kind identifier (e.g. scanner.report.ready).",
"enum": [
"scanner.report.ready",
"scanner.scan.completed",
"scheduler.rescan.delta",
"attestor.logged",
"zastava.admission",
"feedser.export.completed",
"vexer.export.completed"
]
},
"version": {"type": "string"},
"tenant": {"type": "string"},
"ts": {"type": "string", "format": "date-time"},
"actor": {"type": "string"},
"scope": {
"type": "object",
"properties": {
"namespace": {"type": "string"},
"repo": {"type": "string"},
"digest": {"type": "string"},
"component": {"type": "string"},
"image": {"type": "string"},
"labels": {"$ref": "#/$defs/stringMap"},
"attributes": {"$ref": "#/$defs/stringMap"}
},
"additionalProperties": false
},
"payload": {
"type": "object",
"description": "Event specific body; see individual schemas for shapes.",
"additionalProperties": true
},
"attributes": {"$ref": "#/$defs/stringMap"}
},
"additionalProperties": false,
"$defs": {
"stringMap": {
"type": "object",
"patternProperties": {
".*": {"type": "string"}
},
"additionalProperties": false
}
}
}

View File

@@ -0,0 +1,96 @@
{
"$id": "https://stella-ops.org/schemas/notify/notify-rule@1.json",
"$schema": "http://json-schema.org/draft-07/schema#",
"title": "Notify Rule",
"type": "object",
"required": [
"schemaVersion",
"ruleId",
"tenantId",
"name",
"enabled",
"match",
"actions",
"createdAt",
"updatedAt"
],
"properties": {
"schemaVersion": {"type": "string", "const": "notify.rule@1"},
"ruleId": {"type": "string"},
"tenantId": {"type": "string"},
"name": {"type": "string"},
"description": {"type": "string"},
"enabled": {"type": "boolean"},
"match": {"$ref": "#/$defs/ruleMatch"},
"actions": {
"type": "array",
"minItems": 1,
"items": {"$ref": "#/$defs/ruleAction"}
},
"labels": {"$ref": "#/$defs/stringMap"},
"metadata": {"$ref": "#/$defs/stringMap"},
"createdBy": {"type": "string"},
"createdAt": {"type": "string", "format": "date-time"},
"updatedBy": {"type": "string"},
"updatedAt": {"type": "string", "format": "date-time"}
},
"additionalProperties": false,
"$defs": {
"ruleMatch": {
"type": "object",
"properties": {
"eventKinds": {"$ref": "#/$defs/stringArray"},
"namespaces": {"$ref": "#/$defs/stringArray"},
"repositories": {"$ref": "#/$defs/stringArray"},
"digests": {"$ref": "#/$defs/stringArray"},
"labels": {"$ref": "#/$defs/stringArray"},
"componentPurls": {"$ref": "#/$defs/stringArray"},
"minSeverity": {"type": "string"},
"verdicts": {"$ref": "#/$defs/stringArray"},
"kevOnly": {"type": "boolean"},
"vex": {"$ref": "#/$defs/ruleMatchVex"}
},
"additionalProperties": false
},
"ruleMatchVex": {
"type": "object",
"properties": {
"includeAcceptedJustifications": {"type": "boolean"},
"includeRejectedJustifications": {"type": "boolean"},
"includeUnknownJustifications": {"type": "boolean"},
"justificationKinds": {"$ref": "#/$defs/stringArray"}
},
"additionalProperties": false
},
"ruleAction": {
"type": "object",
"required": ["actionId", "channel", "enabled"],
"properties": {
"actionId": {"type": "string"},
"channel": {"type": "string"},
"template": {"type": "string"},
"digest": {"type": "string"},
"throttle": {
"type": "string",
"pattern": "^P(T.*)?$",
"description": "ISO 8601 duration"
},
"locale": {"type": "string"},
"enabled": {"type": "boolean"},
"metadata": {"$ref": "#/$defs/stringMap"}
},
"additionalProperties": false
},
"stringArray": {
"type": "array",
"items": {"type": "string"}
},
"stringMap": {
"type": "object",
"patternProperties": {
".*": {"type": "string"}
},
"additionalProperties": false
}
}
}

View File

@@ -0,0 +1,55 @@
{
"$id": "https://stella-ops.org/schemas/notify/notify-template@1.json",
"$schema": "http://json-schema.org/draft-07/schema#",
"title": "Notify Template",
"type": "object",
"required": [
"schemaVersion",
"templateId",
"tenantId",
"channelType",
"key",
"locale",
"body",
"renderMode",
"format",
"createdAt",
"updatedAt"
],
"properties": {
"schemaVersion": {"type": "string", "const": "notify.template@1"},
"templateId": {"type": "string"},
"tenantId": {"type": "string"},
"channelType": {
"type": "string",
"enum": ["slack", "teams", "email", "webhook", "custom"]
},
"key": {"type": "string"},
"locale": {"type": "string"},
"body": {"type": "string"},
"description": {"type": "string"},
"renderMode": {
"type": "string",
"enum": ["markdown", "html", "adaptiveCard", "plainText", "json"]
},
"format": {
"type": "string",
"enum": ["slack", "teams", "email", "webhook", "json"]
},
"metadata": {"$ref": "#/$defs/stringMap"},
"createdBy": {"type": "string"},
"createdAt": {"type": "string", "format": "date-time"},
"updatedBy": {"type": "string"},
"updatedAt": {"type": "string", "format": "date-time"}
},
"additionalProperties": false,
"$defs": {
"stringMap": {
"type": "object",
"patternProperties": {
".*": {"type": "string"}
},
"additionalProperties": false
}
}
}

View File

@@ -86,7 +86,7 @@
- **Air-gapped replication:** replicate archives via the Offline Update Kit transport channels; never attach USB devices without scanning.
- **Retention:** maintain 30 daily snapshots + 12 monthly archival copies. Rotate encryption keys annually.
- **Key compromise:** if signing keys are suspected compromised, restore from the latest clean backup, rotate via OPS3 (see `ops/authority/key-rotation.sh` and `docs/11_AUTHORITY.md`), and publish a revocation notice.
- **Mongo version:** keep dump/restore images pinned to the deployment version (compose uses `mongo:7`). Restoring across major versions requires a compatibility review.
- **Mongo version:** keep dump/restore images pinned to the deployment version (compose uses `mongo:7`). Driver 3.5.0 requires MongoDB **4.2+**—clusters still on 4.0 must be upgraded before restore, and future driver releases will drop 4.0 entirely. citeturn1open1
## Verification Checklist
- [ ] `/ready` reports all identity providers ready.

View File

@@ -0,0 +1,81 @@
# Scanner Runtime Readiness Checklist
Last updated: 2025-10-19
This runbook confirms that Scanner.WebService now surfaces the metadata Runtime Guild consumers requested: quieted finding counts in the signed report events and progress hints on the scan event stream. Follow the checklist before relying on these fields in production automation.
---
## 1. Prerequisites
- Scanner.WebService release includes **SCANNER-POLICY-09-107** (adds quieted provenance and score inputs to `/reports`).
- Docs repository at commit containing `docs/events/scanner.report.ready@1.json` with `quietedFindingCount`.
- Access to a Scanner environment (staging or sandbox) with an image capable of producing policy verdicts.
---
## 2. Verify quieted finding hints
1. **Trigger a report** run a scan that produces at least one quieted finding (policy with `quiet: true`). After the scan completes, call:
```http
POST /api/v1/reports
Authorization: Bearer <token>
Content-Type: application/json
```
Ensure the JSON response contains `report.summary.quieted` and that the DSSE payload mirrors the same count.
2. **Check emitted event** pull the latest `scanner.report.ready` event (from the queue or sample capture). Confirm the payload includes:
- `quietedFindingCount` equal to the `summary.quieted` value.
- Updated `summary` block with the quieted counter.
3. **Schema validation** optionally validate the payload against `docs/events/scanner.report.ready@1.json` to guarantee downstream compatibility:
```bash
npx ajv validate -c ajv-formats \
-s docs/events/scanner.report.ready@1.json \
-d <payload.json>
```
(Use `npm install --no-save ajv ajv-cli ajv-formats` once per clone.)
> Snapshot fixtures: see `docs/events/samples/scanner.report.ready@1.sample.json` for a canonical event that already carries `quietedFindingCount`.
---
## 3. Verify progress hints (SSE / JSONL)
Scanner streams structured progress messages for each scan. The `data` map inside every frame carries the hints Runtime systems consume (force flag, client metadata, additional stage-specific attributes).
1. **Submit a scan** with custom metadata (for example `pipeline=github`, `build=1234`).
2. **Stream events**:
```http
GET /api/v1/scans/{scanId}/events?format=jsonl
Authorization: Bearer <token>
Accept: application/x-ndjson
```
3. **Confirm payload** each frame should resemble:
```json
{
"scanId": "2f6c17f9b3f548e2a28b9c412f4d63f8",
"sequence": 1,
"state": "Pending",
"message": "queued",
"timestamp": "2025-10-19T03:12:45.118Z",
"correlationId": "2f6c17f9b3f548e2a28b9c412f4d63f8:0001",
"data": {
"force": false,
"meta.pipeline": "github"
}
}
```
Subsequent frames include additional hints as analyzers progress (e.g., `stage`, `meta.*`, or analyzer-provided keys). Ensure newline-delimited JSON consumers preserve the `data` dictionary when forwarding to runtime dashboards.
> The same frame structure is documented in `docs/09_API_CLI_REFERENCE.md` §2.6. Copy that snippet into integration tests to keep compatibility.
---
## 4. Sign-off matrix
| Stakeholder | Checklist | Status | Notes |
|-------------|-----------|--------|-------|
| Runtime Guild | Sections 2 & 3 completed | ☐ | Capture sample payloads for webhook regression tests. |
| Notify Guild | `quietedFindingCount` consumed in notifications | ☐ | Update templates after Runtime sign-off. |
| Docs Guild | Checklist published & linked from updates | ☑ | 2025-10-19 |
Mark the stakeholder boxes as each team completes its validation. Once all checks are green, update `docs/TASKS.md` to reflect task completion.

View File

@@ -2,18 +2,58 @@
The **Scanner Core** library provides shared contracts, observability helpers, and security utilities consumed by `Scanner.WebService`, `Scanner.Worker`, analyzers, and tooling. These primitives guarantee deterministic identifiers, timestamps, and log context for all scanning flows.
## DTOs
## Canonical DTOs
- `ScanJob` & `ScanJobStatus` canonical job metadata (image reference/digest, tenant, correlation ID, timestamps, failure details). Constructors normalise timestamps to UTC microsecond precision and canonicalise image digests. Round-trips with `JsonSerializerDefaults.Web` using `ScannerJsonOptions`.
- `ScanProgressEvent` & `ScanStage`/`ScanProgressEventKind` stage-level progress surface for queue/stream consumers. Includes deterministic sequence numbers, optional progress percentage, attributes, and attached `ScannerError`.
- `ScannerError` & `ScannerErrorCode` shared error taxonomy spanning queue, analyzers, storage, exporters, and signing. Carries severity, retryability, structured details, and microsecond-precision timestamps.
- `ScanJobId` strongly-typed identifier rendered as `Guid` (lowercase `N` format) with deterministic parsing.
### Canonical JSON samples
The golden fixtures consumed by `ScannerCoreContractsTests` document the wire shape shared with downstream services. They live under `src/StellaOps.Scanner.Core.Tests/Fixtures/` and a representative extract is shown below.
```json
{
"id": "8f4cc9c582454b9d9b4f5ae049631b7d",
"status": "running",
"imageReference": "registry.example.com/stellaops/scanner:1.2.3",
"imageDigest": "sha256:abcdef",
"createdAt": "2025-10-18T14:30:15.123456+00:00",
"updatedAt": "2025-10-18T14:30:20.123456+00:00",
"correlationId": "scan-analyzeoperatingsystem-8f4cc9c582454b9d9b4f5ae049631b7d",
"tenantId": "tenant-a",
"metadata": {
"requestId": "req-1234",
"source": "ci"
},
"failure": {
"code": "analyzerFailure",
"severity": "error",
"message": "Analyzer failed to parse layer",
"timestamp": "2025-10-18T14:30:15.123456+00:00",
"retryable": false,
"stage": "AnalyzeOperatingSystem",
"component": "os-analyzer",
"details": {
"layerDigest": "sha256:deadbeef",
"attempt": "1"
}
}
}
```
Progress events follow the same conventions (`jobId`, `stage`, `kind`, `timestamp`, `attributes`, optional embedded `ScannerError`). The fixtures are verified via deterministic JSON comparison in every CI run.
## Deterministic helpers
- `ScannerIdentifiers` derives `ScanJobId`, correlation IDs, and SHA-256 hashes from normalised inputs (image reference/digest, tenant, salt). Ensures case-insensitive stability and reproducible metric keys.
- `ScannerTimestamps` trims to microsecond precision, provides ISO-8601 (`yyyy-MM-ddTHH:mm:ss.ffffffZ`) rendering, and parsing helpers.
- `ScannerJsonOptions` standard JSON options (web defaults, camel-case enums) shared by services/tests.
- `ScanAnalysisStore` & `ScanAnalysisKeys` shared in-memory analysis cache flowing through Worker stages. OS analyzers populate
`analysis.os.packages` (raw output), `analysis.os.fragments` (per-analyzer component fragments), and merge into
`analysis.layers.fragments` so emit/diff stages can compose SBOMs and diffs without knowledge of individual analyzer
implementations.
## Observability primitives
@@ -22,9 +62,74 @@ The **Scanner Core** library provides shared contracts, observability helpers, a
- `ScannerCorrelationContext` & `ScannerCorrelationContextAccessor` ambient correlation propagation via `AsyncLocal` for log scopes, metrics, and diagnostics.
- `ScannerLogExtensions` `ILogger` scopes for jobs/progress events with automatic correlation context push, minimal allocations, and consistent structured fields.
### Observability overhead validation
A micro-benchmark executed on 2025-10-19 (4vCPU runner, .NET 10.0.100-rc.1) measured the average scope cost across 1000000 iterations:
| Scope | Mean (µs/call) |
|-------|----------------|
| `BeginScanScope` (logger attached) | 0.80 |
| `BeginScanScope` (noop logger) | 0.31 |
| `BeginProgressScope` | 0.57 |
To reproduce, run `dotnet test src/StellaOps.Scanner.Core.Tests -c Release` (see `ScannerLogExtensionsPerformanceTests`) or copy the snippet below into a throwaway `dotnet run` console project and execute it with `dotnet run -c Release`:
```csharp
using System.Collections.Generic;
using System.Diagnostics;
using Microsoft.Extensions.Logging;
using StellaOps.Scanner.Core.Contracts;
using StellaOps.Scanner.Core.Observability;
using StellaOps.Scanner.Core.Utility;
var factory = LoggerFactory.Create(builder => builder.AddFilter(static _ => true));
var logger = factory.CreateLogger("bench");
var jobId = ScannerIdentifiers.CreateJobId("registry.example.com/stellaops/scanner:1.2.3", "sha256:abcdef", "tenant-a", "benchmark");
var correlationId = ScannerIdentifiers.CreateCorrelationId(jobId, nameof(ScanStage.AnalyzeOperatingSystem));
var now = ScannerTimestamps.Normalize(new DateTimeOffset(2025, 10, 19, 12, 0, 0, TimeSpan.Zero));
var job = new ScanJob(jobId, ScanJobStatus.Running, "registry.example.com/stellaops/scanner:1.2.3", "sha256:abcdef", now, now, correlationId, "tenant-a", new Dictionary<string, string>(StringComparer.Ordinal) { ["requestId"] = "req-bench" });
var progress = new ScanProgressEvent(jobId, ScanStage.AnalyzeOperatingSystem, ScanProgressEventKind.Progress, 42, now, 10.5, "benchmark", new Dictionary<string, string>(StringComparer.Ordinal) { ["sample"] = "true" });
Console.WriteLine("Scanner Core Observability micro-bench (1,000,000 iterations)");
Report("BeginScanScope (logger)", Measure(static ctx => ctx.Logger.BeginScanScope(ctx.Job, ctx.Stage, ctx.Component), new ScopeContext(logger, job, nameof(ScanStage.AnalyzeOperatingSystem), "os-analyzer")));
Report("BeginScanScope (no logger)", Measure(static ctx => ScannerLogExtensions.BeginScanScope(null, ctx.Job, ctx.Stage, ctx.Component), new ScopeContext(logger, job, nameof(ScanStage.AnalyzeOperatingSystem), "os-analyzer")));
Report("BeginProgressScope", Measure(static ctx => ctx.Logger.BeginProgressScope(ctx.Progress!, ctx.Component), new ScopeContext(logger, job, nameof(ScanStage.AnalyzeOperatingSystem), "os-analyzer", progress)));
static double Measure(Func<ScopeContext, IDisposable> factory, ScopeContext context)
{
const int iterations = 1_000_000;
for (var i = 0; i < 10_000; i++)
{
using var scope = factory(context);
}
GC.Collect();
GC.WaitForPendingFinalizers();
GC.Collect();
var sw = Stopwatch.StartNew();
for (var i = 0; i < iterations; i++)
{
using var scope = factory(context);
}
sw.Stop();
return sw.Elapsed.TotalSeconds * 1_000_000 / iterations;
}
static void Report(string label, double microseconds)
=> Console.WriteLine($"{label,-28}: {microseconds:F3} µs");
readonly record struct ScopeContext(ILogger Logger, ScanJob Job, string? Stage, string? Component, ScanProgressEvent? Progress = null);
```
Both guardrails enforce the ≤5µs acceptance target for SP9-G1.
## Security utilities
- `AuthorityTokenSource` caches short-lived OpToks per audience+scope using deterministic keys and refresh skew (default 30 s). Integrates with `StellaOps.Auth.Client`.
- `AuthorityTokenSource` caches short-lived OpToks per audience+scope using deterministic keys and refresh skew (default 30s). Integrates with `StellaOps.Auth.Client`.
- `DpopProofValidator` validates DPoP proofs (alg allowlist, `htm`/`htu`, nonce, replay window, signature) backed by pluggable `IDpopReplayCache`. Ships with `InMemoryDpopReplayCache` for restart-only deployments.
- `RestartOnlyPluginGuard` enforces restart-time plug-in registration (deterministic path normalisation; throws if new plug-ins added post-seal).
- `ServiceCollectionExtensions.AddScannerAuthorityCore` DI helper wiring Authority client, OpTok source, DPoP validation, replay cache, and plug-in guard.
@@ -33,10 +138,10 @@ The **Scanner Core** library provides shared contracts, observability helpers, a
Unit tests (`StellaOps.Scanner.Core.Tests`) assert:
- DTO JSON round-trips are stable and deterministic.
- DTO JSON round-trips are stable and deterministic (`ScannerCoreContractsTests` + golden fixtures).
- Identifier/hash helpers ignore case and emit lowercase hex.
- Timestamp normalisation retains UTC semantics.
- Log scopes push/pop correlation context predictably.
- Log scopes push/pop correlation context predictably while staying under the 5µs envelope.
- Authority token caching honours refresh skew and invalidation.
- DPoP validator accepts valid proofs, rejects nonce mismatch/replay, and enforces signature validation.
- Restart-only plug-in guard blocks runtime additions post-seal.

View File

@@ -0,0 +1,12 @@
# Docs Guild Update — 2025-10-19
**Subject:** Event envelope reference & canonical samples
**Audience:** Docs Guild, Platform Events, Runtime Guild
- Extended `docs/events/README.md` with envelope field tables, offline validation commands, and guidance for optional payload fields.
- Added canonical sample payloads under `docs/events/samples/` for `scanner.report.ready@1`, `scheduler.rescan.delta@1`, and `attestor.logged@1`; validated them with `ajv-cli` to match the published schemas.
- Documented the validation loop so air-gapped operators can mirror the CI checks before rolling new event versions.
Next steps:
- Platform Events to embed the canonical samples into their contract tests.
- Runtime Guild checklist for quieted finding counts & progress hints published in `docs/runtime/SCANNER_RUNTIME_READINESS.md`; gather stakeholder sign-off.

View File

@@ -0,0 +1,10 @@
# Platform Events Update — 2025-10-19
**Subject:** Canonical event samples enforced across tests & CI
**Audience:** Platform Events Guild, Notify Guild, Scheduler Guild, Docs Guild
- Scanner WebService contract tests deserialize `scanner.report.ready@1` and `scanner.scan.completed@1` samples, validating DSSE payloads and canonical ordering via `NotifyCanonicalJsonSerializer`.
- Notify and Scheduler model suites now round-trip the published event samples (including `attestor.logged@1` and `scheduler.rescan.delta@1`) to catch drift in consumer expectations.
- Docs CI (`.gitea/workflows/docs.yml`) validates every sample against its schema with `ajv-cli`, keeping offline bundles and repositories aligned.
No additional follow-ups — downstream teams can rely on the committed samples for integration coverage.

View File

@@ -0,0 +1,5 @@
# 2025-10-19 Scanner ↔ Policy Sync
- Scanner WebService now emits `scanner.report.ready` and `scanner.scan.completed` via Redis Streams when `scanner.events.enabled=true`; DSSE envelopes are embedded verbatim to keep Notify/UI consumers in sync.
- Config plumbing introduces `scanner:events:*` settings (driver, DSN, stream, publish timeout) with validation and Redis-backed publisher wiring.
- Policy Guild coordination task `POLICY-RUNTIME-17-201` opened to track Zastava runtime feed contract; `SCANNER-RUNTIME-17-401` now depends on it so reachability tags stay aligned once runtime endpoints ship.

View File

@@ -0,0 +1,8 @@
# Scheduler Storage Update — 2025-10-19
**Subject:** Mongo bootstrap + canonical fixtures
**Audience:** Scheduler Storage Guild, Scheduler WebService/Worker teams
- Added `StellaOps.Scheduler.Storage.Mongo` bootstrap (`AddSchedulerMongoStorage`) with collection/index migrations for schedules, runs (incl. TTL), impact snapshots, audit, and locks.
- Introduced Mongo2Go-backed tests that round-trip the published scheduler samples (`samples/api/scheduler/*.json`) to ensure canonical JSON stays intact.
- `ISchedulerMongoInitializer.EnsureMigrationsAsync` now provides the single entry point for WebService/Worker hosts to apply migrations at startup.