up
Some checks failed
Docs CI / lint-and-preview (push) Has been cancelled
Build Test Deploy / build-test (push) Has been cancelled
Build Test Deploy / authority-container (push) Has been cancelled
Build Test Deploy / docs (push) Has been cancelled
Build Test Deploy / deploy (push) Has been cancelled
Some checks failed
Docs CI / lint-and-preview (push) Has been cancelled
Build Test Deploy / build-test (push) Has been cancelled
Build Test Deploy / authority-container (push) Has been cancelled
Build Test Deploy / docs (push) Has been cancelled
Build Test Deploy / deploy (push) Has been cancelled
This commit is contained in:
@@ -158,6 +158,90 @@ Client then generates SBOM **only** for the `missing` layers and re‑posts `/sc
|
||||
| `POST` | `/policy/validate` | Lint only; returns 400 on error |
|
||||
| `GET` | `/policy/history` | Paginated change log (audit trail) |
|
||||
|
||||
### 2.4 Scanner – Queue a Scan Job *(SP9 milestone)*
|
||||
|
||||
```
|
||||
POST /api/v1/scans
|
||||
Authorization: Bearer <token with scanner.scans.enqueue>
|
||||
Content-Type: application/json
|
||||
```
|
||||
|
||||
```json
|
||||
{
|
||||
"image": {
|
||||
"reference": "registry.example.com/acme/app:1.2.3"
|
||||
},
|
||||
"force": false,
|
||||
"clientRequestId": "ci-build-1845",
|
||||
"metadata": {
|
||||
"pipeline": "github",
|
||||
"trigger": "pull-request"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
| Field | Required | Notes |
|
||||
| ------------------- | -------- | ------------------------------------------------------------------------------------------------ |
|
||||
| `image.reference` | no\* | Full repo/tag (`registry/repo:tag`). Provide **either** `reference` or `digest` (sha256:…). |
|
||||
| `image.digest` | no\* | OCI digest (e.g. `sha256:…`). |
|
||||
| `force` | no | `true` forces a re-run even if an identical scan (`scanId`) already exists. Default **false**. |
|
||||
| `clientRequestId` | no | Free-form string surfaced in audit logs. |
|
||||
| `metadata` | no | Optional string map stored with the job and surfaced in observability feeds. |
|
||||
|
||||
\* At least one of `image.reference` or `image.digest` must be supplied.
|
||||
|
||||
**Response 202** – job accepted (idempotent):
|
||||
|
||||
```http
|
||||
HTTP/1.1 202 Accepted
|
||||
Location: /api/v1/scans/2f6c17f9b3f548e2a28b9c412f4d63f8
|
||||
```
|
||||
|
||||
```json
|
||||
{
|
||||
"scanId": "2f6c17f9b3f548e2a28b9c412f4d63f8",
|
||||
"status": "Pending",
|
||||
"location": "/api/v1/scans/2f6c17f9b3f548e2a28b9c412f4d63f8",
|
||||
"created": true
|
||||
}
|
||||
```
|
||||
|
||||
- `scanId` is deterministic – resubmitting an identical payload returns the same identifier with `"created": false`.
|
||||
- API is cancellation-aware; aborting the HTTP request cancels the submission attempt.
|
||||
- Required scope: **`scanner.scans.enqueue`**.
|
||||
|
||||
**Response 400** – validation problem (`Content-Type: application/problem+json`) when both `image.reference` and `image.digest` are blank.
|
||||
|
||||
### 2.5 Scanner – Fetch Scan Status
|
||||
|
||||
```
|
||||
GET /api/v1/scans/{scanId}
|
||||
Authorization: Bearer <token with scanner.scans.read>
|
||||
Accept: application/json
|
||||
```
|
||||
|
||||
**Response 200**:
|
||||
|
||||
```json
|
||||
{
|
||||
"scanId": "2f6c17f9b3f548e2a28b9c412f4d63f8",
|
||||
"status": "Pending",
|
||||
"image": {
|
||||
"reference": "registry.example.com/acme/app:1.2.3",
|
||||
"digest": null
|
||||
},
|
||||
"createdAt": "2025-10-18T20:15:12.482Z",
|
||||
"updatedAt": "2025-10-18T20:15:12.482Z",
|
||||
"failureReason": null
|
||||
}
|
||||
```
|
||||
|
||||
Statuses: `Pending`, `Running`, `Succeeded`, `Failed`, `Cancelled`.
|
||||
|
||||
**Response 404** – `application/problem+json` payload with type `https://stellaops.org/problems/not-found` when the scan identifier is unknown.
|
||||
|
||||
> **Tip** – poll `Location` from the submission call until `status` transitions away from `Pending`/`Running`.
|
||||
|
||||
```yaml
|
||||
# Example import payload (YAML)
|
||||
version: "1.0"
|
||||
@@ -181,6 +265,23 @@ Validation errors come back as:
|
||||
}
|
||||
```
|
||||
|
||||
```json
|
||||
# Preview response excerpt
|
||||
{
|
||||
"success": true,
|
||||
"policyDigest": "9c5e...",
|
||||
"revisionId": "rev-12",
|
||||
"changed": 1,
|
||||
"diffs": [
|
||||
{
|
||||
"baseline": {"findingId": "finding-1", "status": "pass"},
|
||||
"projected": {"findingId": "finding-1", "status": "blocked", "ruleName": "Block Critical"},
|
||||
"changed": true
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 2.4 Attestation (Planned – Q1‑2026)
|
||||
|
||||
@@ -120,7 +120,18 @@ rules:
|
||||
action: escalate
|
||||
```
|
||||
|
||||
Validation is performed by `policy:mapping.yaml` JSON‑Schema embedded in backend.
|
||||
Validation is performed by `policy:mapping.yaml` JSON‑Schema embedded in backend.
|
||||
|
||||
Canonical schema source: `src/StellaOps.Policy/Schemas/policy-schema@1.json` (embedded into `StellaOps.Policy`).
|
||||
`PolicyValidationCli` (see `src/StellaOps.Policy/PolicyValidationCli.cs`) provides the reusable command handler that the main CLI wires up; in the interim it can be invoked from a short host like:
|
||||
|
||||
```csharp
|
||||
await new PolicyValidationCli().RunAsync(new PolicyValidationCliOptions
|
||||
{
|
||||
Inputs = new[] { "policies/root.yaml" },
|
||||
Strict = true,
|
||||
});
|
||||
```
|
||||
|
||||
### 4.1 Rego Variant (Advanced – TODO)
|
||||
|
||||
|
||||
@@ -76,6 +76,12 @@ UI: [https://\<host\>:8443](https://<host>:8443) (self‑signed cert
|
||||
> `stella-ops:latest` with the immutable digest printed by
|
||||
> `docker images --digests`.
|
||||
|
||||
> **Repo bundles** – Development, staging, and air‑gapped Compose profiles live
|
||||
> under `deploy/compose/`, already tied to the release manifests in
|
||||
> `deploy/releases/`. Helm users can pull the same channel overlays from
|
||||
> `deploy/helm/stellaops/values-*.yaml` and validate everything with
|
||||
> `deploy/tools/validate-profiles.sh`.
|
||||
|
||||
### 1.1 · Concelier authority configuration
|
||||
|
||||
The Concelier container reads configuration from `etc/concelier.yaml` plus
|
||||
|
||||
@@ -234,6 +234,11 @@ release:
|
||||
|
||||
The manifest is **cosign‑signed**; UI/CLI can verify a bundle without talking to registries.
|
||||
|
||||
> Deployment guardrails – The repository keeps channel-aligned Compose bundles
|
||||
> in `deploy/compose/` and Helm overlays in `deploy/helm/stellaops/`. Both sets
|
||||
> pull their digests from `deploy/releases/` and are validated by
|
||||
> `deploy/tools/validate-profiles.sh` to guarantee lint/dry-run cleanliness.
|
||||
|
||||
### 6.2 Image labels (release metadata)
|
||||
|
||||
Each image sets OCI labels:
|
||||
|
||||
@@ -42,6 +42,35 @@ src/
|
||||
|
||||
Analyzer assemblies and buildx generators are packaged as **restart-time plug-ins** under `plugins/scanner/**` with manifests; services must restart to activate new plug-ins.
|
||||
|
||||
### 1.1 Queue backbone (Redis / NATS)
|
||||
|
||||
`StellaOps.Scanner.Queue` exposes a transport-agnostic contract (`IScanQueue`/`IScanQueueLease`) used by the WebService producer and Worker consumers. Sprint 9 introduces two first-party transports:
|
||||
|
||||
- **Redis Streams** (default). Uses consumer groups, deterministic idempotency keys (`scanner:jobs:idemp:*`), and supports lease claim (`XCLAIM`), renewal, exponential-backoff retries, and a `scanner:jobs:dead` stream for exhausted attempts.
|
||||
- **NATS JetStream**. Provisions the `SCANNER_JOBS` work-queue stream + durable consumer `scanner-workers`, publishes with `MsgId` for dedupe, applies backoff via `NAK` delays, and routes dead-lettered jobs to `SCANNER_JOBS_DEAD`.
|
||||
|
||||
Metrics are emitted via `Meter` counters (`scanner_queue_enqueued_total`, `scanner_queue_retry_total`, `scanner_queue_deadletter_total`), and `ScannerQueueHealthCheck` pings the active backend (Redis `PING`, NATS `PING`). Configuration is bound from `scanner.queue`:
|
||||
|
||||
```yaml
|
||||
scanner:
|
||||
queue:
|
||||
kind: redis # or nats
|
||||
redis:
|
||||
connectionString: "redis://queue:6379/0"
|
||||
streamName: "scanner:jobs"
|
||||
nats:
|
||||
url: "nats://queue:4222"
|
||||
stream: "SCANNER_JOBS"
|
||||
subject: "scanner.jobs"
|
||||
durableConsumer: "scanner-workers"
|
||||
deadLetterSubject: "scanner.jobs.dead"
|
||||
maxDeliveryAttempts: 5
|
||||
retryInitialBackoff: 00:00:05
|
||||
retryMaxBackoff: 00:02:00
|
||||
```
|
||||
|
||||
The DI extension (`AddScannerQueue`) wires the selected transport, so future additions (e.g., RabbitMQ) only implement the same contract and register.
|
||||
|
||||
**Runtime form‑factor:** two deployables
|
||||
|
||||
* **Scanner.WebService** (stateless REST)
|
||||
|
||||
@@ -31,12 +31,13 @@ Everything here is open‑source and versioned — when you check out a git ta
|
||||
- **03 – [Vision & Road‑map](03_VISION.md)**
|
||||
- **04 – [Feature Matrix](04_FEATURE_MATRIX.md)**
|
||||
|
||||
### Reference & concepts
|
||||
- **05 – [System Requirements Specification](05_SYSTEM_REQUIREMENTS_SPEC.md)**
|
||||
- **07 – [High‑Level Architecture](07_HIGH_LEVEL_ARCHITECTURE.md)**
|
||||
- **08 – Module Architecture Dossiers**
|
||||
- [Scanner](ARCHITECTURE_SCANNER.md)
|
||||
- [Concelier](ARCHITECTURE_CONCELIER.md)
|
||||
### Reference & concepts
|
||||
- **05 – [System Requirements Specification](05_SYSTEM_REQUIREMENTS_SPEC.md)**
|
||||
- **07 – [High‑Level Architecture](07_HIGH_LEVEL_ARCHITECTURE.md)**
|
||||
- **08 – [Architecture Decision Records](adr/index.md)**
|
||||
- **08 – Module Architecture Dossiers**
|
||||
- [Scanner](ARCHITECTURE_SCANNER.md)
|
||||
- [Concelier](ARCHITECTURE_CONCELIER.md)
|
||||
- [Excititor](ARCHITECTURE_EXCITITOR.md)
|
||||
- [Signer](ARCHITECTURE_SIGNER.md)
|
||||
- [Attestor](ARCHITECTURE_ATTESTOR.md)
|
||||
@@ -48,8 +49,9 @@ Everything here is open‑source and versioned — when you check out a git ta
|
||||
- [Zastava Runtime](ARCHITECTURE_ZASTAVA.md)
|
||||
- [Release & Operations](ARCHITECTURE_DEVOPS.md)
|
||||
- **09 – [API & CLI Reference](09_API_CLI_REFERENCE.md)**
|
||||
- **10 – [Plug‑in SDK Guide](10_PLUGIN_SDK_GUIDE.md)**
|
||||
- **10 – [Concelier CLI Quickstart](10_CONCELIER_CLI_QUICKSTART.md)**
|
||||
- **10 – [Plug‑in SDK Guide](10_PLUGIN_SDK_GUIDE.md)**
|
||||
- **10 – [Concelier CLI Quickstart](10_CONCELIER_CLI_QUICKSTART.md)**
|
||||
- **10 – [BuildX Generator Quickstart](dev/BUILDX_PLUGIN_QUICKSTART.md)**
|
||||
- **30 – [Excititor Connector Packaging Guide](dev/30_EXCITITOR_CONNECTOR_GUIDE.md)**
|
||||
- **30 – Developer Templates**
|
||||
- [Excititor Connector Skeleton](dev/templates/excititor-connector/)
|
||||
|
||||
@@ -9,8 +9,8 @@
|
||||
| DOC5.Concelier-Runbook | DONE (2025-10-12) | Docs Guild | DOC3.Concelier-Authority | Produce dedicated Concelier authority audit runbook covering log fields, monitoring recommendations, and troubleshooting steps. | ✅ Runbook published; ✅ linked from DOC3/DOC5; ✅ alerting guidance included. |
|
||||
| FEEDDOCS-DOCS-05-001 | DONE (2025-10-11) | Docs Guild | FEEDMERGE-ENGINE-04-001, FEEDMERGE-ENGINE-04-002 | Publish Concelier conflict resolution runbook covering precedence workflow, merge-event auditing, and Sprint 3 metrics. | ✅ `docs/ops/concelier-conflict-resolution.md` committed; ✅ metrics/log tables align with latest merge code; ✅ Ops alert guidance handed to Concelier team. |
|
||||
| FEEDDOCS-DOCS-05-002 | DONE (2025-10-16) | Docs Guild, Concelier Ops | FEEDDOCS-DOCS-05-001 | Ops sign-off captured: conflict runbook circulated, alert thresholds tuned, and rollout decisions documented in change log. | ✅ Ops review recorded; ✅ alert thresholds finalised using `docs/ops/concelier-authority-audit-runbook.md`; ✅ change-log entry linked from runbook once GHSA/NVD/OSV regression fixtures land. |
|
||||
| DOCS-ADR-09-001 | TODO | Docs Guild, DevEx | — | Establish ADR process (`docs/adr/0000-template.md`) and document usage guidelines. | Template published; README snippet linking ADR process; announcement posted. |
|
||||
| DOCS-EVENTS-09-002 | TODO | Docs Guild, Platform Events | SCANNER-EVENTS-15-201 | Publish event schema catalog (`docs/events/`) for `scanner.report.ready@1`, `scheduler.rescan.delta@1`, `attestor.logged@1`. | Schemas validated; docs/events/README summarises usage; Notify/Scheduler teams acknowledge. |
|
||||
| DOCS-ADR-09-001 | DONE (2025-10-19) | Docs Guild, DevEx | — | Establish ADR process (`docs/adr/0000-template.md`) and document usage guidelines. | Template published; README snippet linking ADR process; announcement posted (`docs/updates/2025-10-18-docs-guild.md`). |
|
||||
| DOCS-EVENTS-09-002 | DONE (2025-10-19) | Docs Guild, Platform Events | SCANNER-EVENTS-15-201 | Publish event schema catalog (`docs/events/`) for `scanner.report.ready@1`, `scheduler.rescan.delta@1`, `attestor.logged@1`. | Schemas validated (Ajv CI hooked); docs/events/README summarises usage; Platform Events notified via `docs/updates/2025-10-18-docs-guild.md`. |
|
||||
| DOCS-RUNTIME-17-004 | TODO | Docs Guild, Runtime Guild | SCANNER-EMIT-17-701, ZASTAVA-OBS-17-005, DEVOPS-REL-17-002 | Document build-id workflows: SBOM exposure, runtime event payloads, debug-store layout, and operator guidance for symbol retrieval. | Architecture + operator docs updated with build-id sections, examples show `readelf` output + debuginfod usage, references linked from Offline Kit/Release guides. |
|
||||
|
||||
> Update statuses (TODO/DOING/REVIEW/DONE/BLOCKED) as progress changes. Keep guides in sync with configuration samples under `etc/`.
|
||||
|
||||
@@ -3,16 +3,32 @@
|
||||
## Status
|
||||
Proposed
|
||||
|
||||
## Date
|
||||
YYYY-MM-DD
|
||||
|
||||
## Authors
|
||||
- Name (team)
|
||||
|
||||
## Deciders
|
||||
- Names of approvers / reviewers
|
||||
|
||||
## Context
|
||||
- What decision needs to be made?
|
||||
- What are the forces (requirements, constraints, stakeholders)?
|
||||
- Why now? What triggers the ADR?
|
||||
|
||||
## Decision
|
||||
- Summary of the chosen option.
|
||||
- Key rationale points.
|
||||
|
||||
## Consequences
|
||||
- Positive/negative consequences.
|
||||
- Follow-up actions or tasks.
|
||||
- Rollback plan or re-evaluation criteria.
|
||||
|
||||
## Alternatives Considered
|
||||
- Option A — pros/cons.
|
||||
- Option B — pros/cons.
|
||||
|
||||
## References
|
||||
- Links to related ADRs, issues, documents.
|
||||
|
||||
41
docs/adr/index.md
Normal file
41
docs/adr/index.md
Normal file
@@ -0,0 +1,41 @@
|
||||
# Architecture Decision Records (ADRs)
|
||||
|
||||
Architecture Decision Records document long-lived choices that shape StellaOps architecture, security posture, and operator experience. They complement RFCs by capturing the final call and the context that led to it.
|
||||
|
||||
## When to file an ADR
|
||||
- Decisions that affect cross-module contracts, persistence models, or external interfaces.
|
||||
- Security or compliance controls with on-going operational ownership.
|
||||
- Rollout strategies that require coordination across guilds or sprints.
|
||||
- Reversals or deprecations of previously accepted ADRs.
|
||||
|
||||
Small, module-local refactors that do not modify public behaviour can live in commit messages instead.
|
||||
|
||||
## Workflow at a glance
|
||||
1. Copy `docs/adr/0000-template.md` to `docs/adr/NNNN-short-slug.md` with a zero-padded sequence (see **Numbering**).
|
||||
2. Fill in context, decision, consequences, and alternatives. Include links to RFCs, issues, benchmarks, or experiments.
|
||||
3. Request async review from the impacted guilds. Capture sign-offs in the **Deciders** field.
|
||||
4. Merge the ADR with the code/config changes (or in a preparatory PR).
|
||||
5. Announce the accepted ADR in the Docs Guild channel or sprint notes so downstream teams can consume it.
|
||||
|
||||
## Numbering and status
|
||||
- Use zero-padded integers (e.g., `0001`, `0002`) in file names and the document header. Increment from the highest existing number.
|
||||
- Valid statuses: `Proposed`, `Accepted`, `Rejected`, `Deprecated`, `Superseded`. Update the status when follow-up work lands.
|
||||
- When an ADR supersedes another, link them in both documents’ **References** sections.
|
||||
|
||||
## Review expectations
|
||||
- Highlight edge-case handling, trade-offs, and determinism requirements.
|
||||
- Include operational checklists for any new runtime path (quota updates, schema migrations, credential rotation, etc.).
|
||||
- Attach diagrams under `docs/adr/assets/` when visuals improve comprehension.
|
||||
- Add TODO tasks for follow-up work in the relevant module’s `TASKS.md` and link them from the ADR.
|
||||
|
||||
## Verification checklist
|
||||
- [ ] `Status`, `Date`, `Authors`, and `Deciders` populated.
|
||||
- [ ] Links to code/config PRs or experiments recorded under **References**.
|
||||
- [ ] Consequences call out migration or rollback steps.
|
||||
- [ ] Announcement posted to Docs Guild updates (or sprint log).
|
||||
|
||||
## Related resources
|
||||
- [Docs Guild Task Board](../TASKS.md)
|
||||
- [High-Level Architecture Overview](../07_HIGH_LEVEL_ARCHITECTURE.md)
|
||||
- [Coding Standards](../18_CODING_STANDARDS.md)
|
||||
- [Release Engineering Playbook](../13_RELEASE_ENGINEERING_PLAYBOOK.md)
|
||||
@@ -241,7 +241,59 @@ jobs:
|
||||
|
||||
---
|
||||
|
||||
## 4 · Troubleshooting cheat‑sheet
|
||||
## 4 · Docs CI (Gitea Actions & Offline Mirror)
|
||||
|
||||
StellaOps ships a dedicated Docs workflow at `.gitea/workflows/docs.yml`. When mirroring the pipeline offline or running it locally, install the same toolchain so markdown linting, schema validation, and HTML preview stay deterministic.
|
||||
|
||||
### 4.1 Toolchain bootstrap
|
||||
|
||||
```bash
|
||||
# Node.js 20.x is required; install once per runner
|
||||
npm install --no-save \
|
||||
markdown-link-check \
|
||||
remark-cli \
|
||||
remark-preset-lint-recommended \
|
||||
ajv \
|
||||
ajv-cli \
|
||||
ajv-formats
|
||||
|
||||
# Python 3.11+ powers the preview renderer
|
||||
python -m pip install --upgrade pip
|
||||
python -m pip install markdown pygments
|
||||
```
|
||||
|
||||
**Offline tip.** Add the packages above to your artifact mirror (for example `ops/devops/offline-kit.json`) so runners can install them via `npm --offline` / `pip --no-index`.
|
||||
|
||||
### 4.2 Schema validation step
|
||||
|
||||
Ajv compiles every event schema to guard against syntax or format regressions. The workflow uses `ajv-formats` for UUID/date-time support.
|
||||
|
||||
```bash
|
||||
for schema in docs/events/*.json; do
|
||||
npx ajv compile -c ajv-formats -s "$schema"
|
||||
done
|
||||
```
|
||||
|
||||
Run this loop before committing schema changes. For new references, append `-r additional-file.json` so CI and local runs stay aligned.
|
||||
|
||||
### 4.3 Preview build
|
||||
|
||||
```bash
|
||||
python scripts/render_docs.py --source docs --output artifacts/docs-preview --clean
|
||||
```
|
||||
|
||||
Host the resulting bundle via any static file server for review (for example `python -m http.server`).
|
||||
|
||||
### 4.4 Publishing checklist
|
||||
|
||||
- [ ] Toolchain installs succeed without hitting the public internet (mirror or cached tarballs).
|
||||
- [ ] Ajv validation passes for `scanner.report.ready@1`, `scheduler.rescan.delta@1`, `attestor.logged@1`.
|
||||
- [ ] Markdown link check (`npx markdown-link-check`) reports no broken references.
|
||||
- [ ] Preview bundle archived (or attached) for stakeholders.
|
||||
|
||||
---
|
||||
|
||||
## 5 · Troubleshooting cheat‑sheet
|
||||
|
||||
| Symptom | Root cause | First things to try |
|
||||
| ------------------------------------- | --------------------------- | --------------------------------------------------------------- |
|
||||
@@ -253,6 +305,7 @@ jobs:
|
||||
|
||||
---
|
||||
|
||||
### Change log
|
||||
|
||||
* **2025‑08‑04** – Variable clean‑up, removed Docker‑socket & cache mounts, added Jenkins / CircleCI / Gitea examples, clarified Option B comment.
|
||||
### Change log
|
||||
|
||||
* **2025‑10‑18** – Documented Docs CI toolchain (Ajv validation, static preview) and offline checklist.
|
||||
* **2025‑08‑04** – Variable clean‑up, removed Docker‑socket & cache mounts, added Jenkins / CircleCI / Gitea examples, clarified Option B comment.
|
||||
|
||||
117
docs/dev/BUILDX_PLUGIN_QUICKSTART.md
Normal file
117
docs/dev/BUILDX_PLUGIN_QUICKSTART.md
Normal file
@@ -0,0 +1,117 @@
|
||||
# BuildX Generator Quickstart
|
||||
|
||||
This quickstart explains how to run the StellaOps **BuildX SBOM generator** offline, verify the CAS handshake, and emit OCI descriptors that downstream services can attest.
|
||||
|
||||
## 1. Prerequisites
|
||||
|
||||
- Docker 25+ with BuildKit enabled (`docker buildx` available).
|
||||
- .NET 10 (preview) SDK matching the repository `global.json`.
|
||||
- Optional: network access to a StellaOps Attestor endpoint (the quickstart uses a mock service).
|
||||
|
||||
## 2. Publish the plug-in binaries
|
||||
|
||||
The BuildX generator publishes as a .NET self-contained executable with its manifest under `plugins/scanner/buildx/`.
|
||||
|
||||
```bash
|
||||
# From the repository root
|
||||
DOTNET_CLI_HOME="${PWD}/.dotnet" \
|
||||
dotnet publish src/StellaOps.Scanner.Sbomer.BuildXPlugin/StellaOps.Scanner.Sbomer.BuildXPlugin.csproj \
|
||||
-c Release \
|
||||
-o out/buildx
|
||||
```
|
||||
|
||||
- `out/buildx/` now contains `StellaOps.Scanner.Sbomer.BuildXPlugin.dll` and the manifest `stellaops.sbom-indexer.manifest.json`.
|
||||
- `plugins/scanner/buildx/StellaOps.Scanner.Sbomer.BuildXPlugin/` receives the same artefacts for release packaging.
|
||||
|
||||
## 3. Verify the CAS handshake
|
||||
|
||||
```bash
|
||||
dotnet out/buildx/StellaOps.Scanner.Sbomer.BuildXPlugin.dll handshake \
|
||||
--manifest out/buildx \
|
||||
--cas out/cas
|
||||
```
|
||||
|
||||
The command performs a deterministic probe write (`sha256`) into the provided CAS directory and prints the resolved path.
|
||||
|
||||
## 4. Emit a descriptor + provenance placeholder
|
||||
|
||||
1. Build or identify the image you want to describe and capture its digest:
|
||||
|
||||
```bash
|
||||
docker buildx build --load -t stellaops/buildx-demo:ci samples/ci/buildx-demo
|
||||
DIGEST=$(docker image inspect stellaops/buildx-demo:ci --format '{{index .RepoDigests 0}}')
|
||||
```
|
||||
|
||||
2. Generate a CycloneDX SBOM for the built image (any tool works; here we use `docker sbom`):
|
||||
|
||||
```bash
|
||||
docker sbom stellaops/buildx-demo:ci --format cyclonedx-json > out/buildx-sbom.cdx.json
|
||||
```
|
||||
|
||||
3. Invoke the `descriptor` command, pointing at the SBOM file and optional metadata:
|
||||
|
||||
```bash
|
||||
dotnet out/buildx/StellaOps.Scanner.Sbomer.BuildXPlugin.dll descriptor \
|
||||
--manifest out/buildx \
|
||||
--image "$DIGEST" \
|
||||
--sbom out/buildx-sbom.cdx.json \
|
||||
--sbom-name buildx-sbom.cdx.json \
|
||||
--artifact-type application/vnd.stellaops.sbom.layer+json \
|
||||
--sbom-format cyclonedx-json \
|
||||
--sbom-kind inventory \
|
||||
--repository git.stella-ops.org/stellaops/buildx-demo \
|
||||
--build-ref $(git rev-parse HEAD) \
|
||||
> out/buildx-descriptor.json
|
||||
```
|
||||
|
||||
The output JSON captures:
|
||||
|
||||
- OCI artifact descriptor including size, digest, and annotations (`org.stellaops.*`).
|
||||
- Provenance placeholder (`expectedDsseSha256`, `nonce`, `attestorUri` when provided).
|
||||
- Generator metadata and deterministic timestamps.
|
||||
|
||||
## 5. (Optional) Send the placeholder to an Attestor
|
||||
|
||||
The plug-in can POST the descriptor metadata to an Attestor endpoint, returning once it receives an HTTP 202.
|
||||
|
||||
```bash
|
||||
python3 - <<'PY' &
|
||||
from http.server import BaseHTTPRequestHandler, HTTPServer
|
||||
class Handler(BaseHTTPRequestHandler):
|
||||
def do_POST(self):
|
||||
_ = self.rfile.read(int(self.headers.get('Content-Length', 0)))
|
||||
self.send_response(202); self.end_headers(); self.wfile.write(b'accepted')
|
||||
def log_message(self, fmt, *args):
|
||||
return
|
||||
server = HTTPServer(('127.0.0.1', 8085), Handler)
|
||||
try:
|
||||
server.serve_forever()
|
||||
except KeyboardInterrupt:
|
||||
pass
|
||||
finally:
|
||||
server.server_close()
|
||||
PY
|
||||
MOCK_PID=$!
|
||||
|
||||
dotnet out/buildx/StellaOps.Scanner.Sbomer.BuildXPlugin.dll descriptor \
|
||||
--manifest out/buildx \
|
||||
--image "$DIGEST" \
|
||||
--sbom out/buildx-sbom.cdx.json \
|
||||
--attestor http://127.0.0.1:8085/provenance \
|
||||
--attestor-token "$STELLAOPS_ATTESTOR_TOKEN" \
|
||||
> out/buildx-descriptor.json
|
||||
|
||||
kill $MOCK_PID
|
||||
```
|
||||
|
||||
Set `STELLAOPS_ATTESTOR_TOKEN` (or pass `--attestor-token`) when the Attestor requires bearer authentication. Use `--attestor-insecure` for lab environments with self-signed certificates.
|
||||
|
||||
## 6. CI workflow example
|
||||
|
||||
A reusable GitHub Actions workflow is provided under `samples/ci/buildx-demo/github-actions-buildx-demo.yml`. It publishes the plug-in, runs the handshake, builds the demo image, emits a descriptor, and uploads both the descriptor and the mock-Attestor request as artefacts.
|
||||
|
||||
Add the workflow to your repository (or call it via `workflow_call`) and adjust the SBOM path + Attestor URL as needed.
|
||||
|
||||
---
|
||||
|
||||
For deeper integration guidance (custom SBOM builders, exporting DSSE bundles), track ADRs in `docs/ARCHITECTURE_SCANNER.md` §7 and follow upcoming Attestor API releases.
|
||||
@@ -1,9 +1,30 @@
|
||||
# Event Envelope Schemas
|
||||
|
||||
Versioned JSON Schemas for platform events consumed by Scheduler, Notify, and UI.
|
||||
Platform services publish strongly typed events; the JSON Schemas in this directory define those envelopes. File names follow `<event-name>@<version>.json` so producers and consumers can negotiate contracts explicitly.
|
||||
|
||||
- `scanner.report.ready@1.json`
|
||||
- `scheduler.rescan.delta@1.json`
|
||||
- `attestor.logged@1.json`
|
||||
## Catalog
|
||||
- `scanner.report.ready@1.json` — emitted by Scanner.WebService once a signed report is persisted. Consumers: Notify, UI timeline.
|
||||
- `scheduler.rescan.delta@1.json` — emitted by Scheduler when BOM-Index diffs require fresh scans. Consumers: Notify, Policy Engine.
|
||||
- `attestor.logged@1.json` — emitted by Attestor after storing the Rekor inclusion proof. Consumers: UI attestation panel, Governance exports.
|
||||
|
||||
Producers must bump the version suffix when introducing breaking changes; consumers validate incoming payloads against these schemas.
|
||||
Additive payload changes (new optional fields) can stay within the same version. Any breaking change (removing a field, tightening validation, altering semantics) must increment the `@<version>` suffix and update downstream consumers.
|
||||
|
||||
## CI validation
|
||||
The Docs CI workflow (`.gitea/workflows/docs.yml`) installs `ajv-cli` and compiles every schema on pull requests. Run the same check locally before opening a PR:
|
||||
|
||||
```bash
|
||||
for schema in docs/events/*.json; do
|
||||
npx ajv compile -c ajv-formats -s "$schema"
|
||||
done
|
||||
```
|
||||
|
||||
Tip: run `npm install --no-save ajv ajv-cli ajv-formats` once per clone so `npx` can resolve the tooling offline.
|
||||
|
||||
If a schema references additional files, include `-r` flags so CI and local runs stay consistent.
|
||||
|
||||
## Working with schemas
|
||||
- Producers should validate outbound payloads using the matching schema during unit tests.
|
||||
- Consumers should pin to a specific version and log when encountering unknown versions to catch missing migrations early.
|
||||
- Store real payload samples under `samples/events/` (mirrors the schema version) to aid contract testing.
|
||||
|
||||
Contact the Platform Events group in Docs Guild if you need help shaping a new event or version strategy.
|
||||
|
||||
42
docs/scanner-core-contracts.md
Normal file
42
docs/scanner-core-contracts.md
Normal file
@@ -0,0 +1,42 @@
|
||||
# Scanner Core Contracts
|
||||
|
||||
The **Scanner Core** library provides shared contracts, observability helpers, and security utilities consumed by `Scanner.WebService`, `Scanner.Worker`, analyzers, and tooling. These primitives guarantee deterministic identifiers, timestamps, and log context for all scanning flows.
|
||||
|
||||
## DTOs
|
||||
|
||||
- `ScanJob` & `ScanJobStatus` – canonical job metadata (image reference/digest, tenant, correlation ID, timestamps, failure details). Constructors normalise timestamps to UTC microsecond precision and canonicalise image digests. Round-trips with `JsonSerializerDefaults.Web` using `ScannerJsonOptions`.
|
||||
- `ScanProgressEvent` & `ScanStage`/`ScanProgressEventKind` – stage-level progress surface for queue/stream consumers. Includes deterministic sequence numbers, optional progress percentage, attributes, and attached `ScannerError`.
|
||||
- `ScannerError` & `ScannerErrorCode` – shared error taxonomy spanning queue, analyzers, storage, exporters, and signing. Carries severity, retryability, structured details, and microsecond-precision timestamps.
|
||||
- `ScanJobId` – strongly-typed identifier rendered as `Guid` (lowercase `N` format) with deterministic parsing.
|
||||
|
||||
## Deterministic helpers
|
||||
|
||||
- `ScannerIdentifiers` – derives `ScanJobId`, correlation IDs, and SHA-256 hashes from normalised inputs (image reference/digest, tenant, salt). Ensures case-insensitive stability and reproducible metric keys.
|
||||
- `ScannerTimestamps` – trims to microsecond precision, provides ISO-8601 (`yyyy-MM-ddTHH:mm:ss.ffffffZ`) rendering, and parsing helpers.
|
||||
- `ScannerJsonOptions` – standard JSON options (web defaults, camel-case enums) shared by services/tests.
|
||||
|
||||
## Observability primitives
|
||||
|
||||
- `ScannerDiagnostics` – global `ActivitySource`/`Meter` for scanner components. `StartActivity` seeds deterministic tags (`job_id`, `stage`, `component`, `correlation_id`).
|
||||
- `ScannerMetricNames` – centralises metric prefixes (`stellaops.scanner.*`) and deterministic job/event tag builders.
|
||||
- `ScannerCorrelationContext` & `ScannerCorrelationContextAccessor` – ambient correlation propagation via `AsyncLocal` for log scopes, metrics, and diagnostics.
|
||||
- `ScannerLogExtensions` – `ILogger` scopes for jobs/progress events with automatic correlation context push, minimal allocations, and consistent structured fields.
|
||||
|
||||
## Security utilities
|
||||
|
||||
- `AuthorityTokenSource` – caches short-lived OpToks per audience+scope using deterministic keys and refresh skew (default 30 s). Integrates with `StellaOps.Auth.Client`.
|
||||
- `DpopProofValidator` – validates DPoP proofs (alg allowlist, `htm`/`htu`, nonce, replay window, signature) backed by pluggable `IDpopReplayCache`. Ships with `InMemoryDpopReplayCache` for restart-only deployments.
|
||||
- `RestartOnlyPluginGuard` – enforces restart-time plug-in registration (deterministic path normalisation; throws if new plug-ins added post-seal).
|
||||
- `ServiceCollectionExtensions.AddScannerAuthorityCore` – DI helper wiring Authority client, OpTok source, DPoP validation, replay cache, and plug-in guard.
|
||||
|
||||
## Testing guarantees
|
||||
|
||||
Unit tests (`StellaOps.Scanner.Core.Tests`) assert:
|
||||
|
||||
- DTO JSON round-trips are stable and deterministic.
|
||||
- Identifier/hash helpers ignore case and emit lowercase hex.
|
||||
- Timestamp normalisation retains UTC semantics.
|
||||
- Log scopes push/pop correlation context predictably.
|
||||
- Authority token caching honours refresh skew and invalidation.
|
||||
- DPoP validator accepts valid proofs, rejects nonce mismatch/replay, and enforces signature validation.
|
||||
- Restart-only plug-in guard blocks runtime additions post-seal.
|
||||
14
docs/updates/2025-10-18-docs-guild.md
Normal file
14
docs/updates/2025-10-18-docs-guild.md
Normal file
@@ -0,0 +1,14 @@
|
||||
# Docs Guild Update — 2025-10-18
|
||||
|
||||
**Subject:** ADR process + events schema validation shipped
|
||||
**Audience:** Docs Guild, DevEx, Platform Events
|
||||
|
||||
- Published the ADR contribution guide at `docs/adr/index.md` and enriched the template to capture authorship, deciders, and alternatives. All new cross-module decisions should follow this workflow.
|
||||
- Linked the ADR hub from `docs/README.md` so operators and engineers can discover the process without digging through directories.
|
||||
- Extended Docs CI (`.gitea/workflows/docs.yml`) to compile event schemas with Ajv (including `ajv-formats`) and documented the local loop in `docs/events/README.md`.
|
||||
- Captured the mirror/offline workflow in `docs/ci/20_CI_RECIPES.md` so runners know how to install the Ajv toolchain and publish previews without internet access.
|
||||
- Validated `scanner.report.ready@1`, `scheduler.rescan.delta@1`, and `attestor.logged@1` schemas locally to unblock Platform Events acknowledgements.
|
||||
|
||||
Next steps:
|
||||
- Platform Events to confirm Notify/Scheduler consumers have visibility into the schema docs.
|
||||
- DevEx to add ADR announcement blurb to the next sprint recap if broader broadcast is needed.
|
||||
Reference in New Issue
Block a user