Rewrite architecture docs and add Vexer connector template
This commit is contained in:
389
docs/ARCHITECTURE_CLI.md
Normal file
389
docs/ARCHITECTURE_CLI.md
Normal file
@@ -0,0 +1,389 @@
|
||||
# component_architecture_cli.md — **Stella Ops CLI** (2025Q4)
|
||||
|
||||
> **Scope.** Implementation‑ready architecture for **Stella Ops CLI**: command surface, process model, auth (Authority/DPoP), integration with Scanner/Vexer/Feedser/Signer/Attestor, Buildx plug‑in management, offline kit behavior, packaging, observability, security posture, and CI ergonomics.
|
||||
|
||||
---
|
||||
|
||||
## 0) Mission & boundaries
|
||||
|
||||
**Mission.** Provide a **fast, deterministic, CI‑friendly** command‑line interface to drive Stella Ops workflows:
|
||||
|
||||
* Build‑time SBOM generation via **Buildx generator** orchestration.
|
||||
* Post‑build **scan/compose/diff/export** against **Scanner.WebService**.
|
||||
* **Policy** operations and **VEX/Vuln** data pulls (operator tasks).
|
||||
* **Verification** (attestation, referrers, signatures) for audits.
|
||||
* Air‑gapped/offline **kit** administration.
|
||||
|
||||
**Boundaries.**
|
||||
|
||||
* CLI **never** signs; it only calls **Signer**/**Attestor** via backend APIs when needed (e.g., `report --attest`).
|
||||
* CLI **does not** store long‑lived credentials beyond OS keychain; tokens are **short** (Authority OpToks).
|
||||
* Heavy work (scanning, merging, policy) is executed **server‑side** (Scanner/Vexer/Feedser).
|
||||
|
||||
---
|
||||
|
||||
## 1) Solution layout & runtime form
|
||||
|
||||
```
|
||||
src/
|
||||
├─ StellaOps.Cli/ # net10.0 (Native AOT) single binary
|
||||
├─ StellaOps.Cli.Core/ # verb plumbing, config, HTTP, auth
|
||||
├─ StellaOps.Cli.Plugins/ # optional verbs packaged as plugins
|
||||
├─ StellaOps.Cli.Tests/ # unit + golden-output tests
|
||||
└─ packaging/
|
||||
├─ msix / msi / deb / rpm / brew formula
|
||||
└─ scoop manifest / winget manifest
|
||||
```
|
||||
|
||||
**Language/runtime**: .NET 10 **Native AOT** for speed/startup; Linux builds use **musl** static when possible.
|
||||
|
||||
**OS targets**: linux‑x64/arm64, windows‑x64/arm64, macOS‑x64/arm64.
|
||||
|
||||
---
|
||||
|
||||
## 2) Command surface (verbs)
|
||||
|
||||
> All verbs default to **JSON** output when `--json` is set (CI mode). Human output is concise, deterministic.
|
||||
|
||||
### 2.1 Auth & profile
|
||||
|
||||
* `auth login`
|
||||
|
||||
* Modes: **device‑code** (default), **client‑credentials** (service principal).
|
||||
* Produces **Authority** access token (OpTok) + stores **DPoP** keypair in OS keychain.
|
||||
* `auth status` — show current issuer, subject, audiences, expiry.
|
||||
* `auth logout` — wipe cached tokens/keys.
|
||||
|
||||
### 2.2 Build‑time SBOM (Buildx)
|
||||
|
||||
* `buildx install` — install/update the **StellaOps.Scanner.Sbomer.BuildXPlugin** on the host.
|
||||
* `buildx verify` — ensure generator is usable.
|
||||
* `buildx build` — thin wrapper around `docker buildx build --attest=type=sbom,generator=stellaops/sbom-indexer` with convenience flags:
|
||||
|
||||
* `--attest` (request Signer/Attestor via backend post‑push)
|
||||
* `--provenance` pass‑through (optional)
|
||||
|
||||
### 2.3 Scanning & artifacts
|
||||
|
||||
* `scan image <ref|digest>`
|
||||
|
||||
* Options: `--force`, `--wait`, `--view=inventory|usage|both`, `--format=cdx-json|cdx-pb|spdx-json`, `--attest` (ask backend to sign/log).
|
||||
* Streams progress; exits early unless `--wait`.
|
||||
* `diff image --old <digest> --new <digest> [--view ...]` — show layer‑attributed changes.
|
||||
* `export sbom <digest> [--view ... --format ... --out file]` — download artifact.
|
||||
* `report final <digest> [--policy-revision ... --attest]` — request PASS/FAIL report from backend (policy+vex) and optional attestation.
|
||||
|
||||
### 2.4 Policy & data
|
||||
|
||||
* `policy get/set/apply` — fetch active policy, apply staged policy, compute digest.
|
||||
* `feedser export` — trigger/export canonical JSON or Trivy DB (admin).
|
||||
* `vexer export` — trigger/export consensus/raw claims (admin).
|
||||
|
||||
### 2.5 Verification
|
||||
|
||||
* `verify attestation --uuid <rekor-uuid> | --artifact <sha256> | --bundle <path>` — call **Attestor /verify** and print proof summary.
|
||||
* `verify referrers <digest>` — ask **Signer /verify/referrers** (is image Stella‑signed?).
|
||||
* `verify image-signature <ref|digest>` — standalone cosign verification (optional, local).
|
||||
|
||||
### 2.6 Runtime (Zastava helper)
|
||||
|
||||
* `runtime policy test --images <digest,...> [--ns <name> --labels k=v,...]` — ask backend `/policy/runtime` like the webhook would.
|
||||
|
||||
### 2.7 Offline kit
|
||||
|
||||
* `offline kit pull` — fetch latest **Feedser JSON + Trivy DB + Vexer exports** as a tarball from a mirror.
|
||||
* `offline kit import <tar>` — upload the kit to on‑prem services (Feedser/Vexer).
|
||||
* `offline kit status` — list current seed versions.
|
||||
|
||||
### 2.8 Utilities
|
||||
|
||||
* `config set/get` — endpoint & defaults.
|
||||
* `whoami` — short auth display.
|
||||
* `version` — CLI + protocol versions; release channel.
|
||||
|
||||
---
|
||||
|
||||
## 3) AuthN: Authority + DPoP
|
||||
|
||||
### 3.1 Token acquisition
|
||||
|
||||
* **Device‑code**: the CLI opens an OIDC device code flow against **Authority**; the browser login is optional for service principals.
|
||||
* **Client‑credentials**: service principals use **private_key_jwt** or **mTLS** to get tokens.
|
||||
|
||||
### 3.2 DPoP key management
|
||||
|
||||
* On first login, the CLI generates an **ephemeral JWK** (Ed25519) and stores it in the **OS keychain** (Keychain/DPAPI/KWallet/Gnome Keyring).
|
||||
* Every request to backend services includes a **DPoP proof**; CLI refreshes tokens as needed.
|
||||
|
||||
### 3.3 Multi‑audience & scopes
|
||||
|
||||
* CLI requests **audiences** as needed per verb:
|
||||
|
||||
* `scanner` for scan/export/report/diff
|
||||
* `signer` (indirect; usually backend calls Signer)
|
||||
* `attestor` for verify
|
||||
* `feedser`/`vexer` for admin verbs
|
||||
|
||||
CLI rejects verbs if required scopes are missing.
|
||||
|
||||
---
|
||||
|
||||
## 4) Process model & reliability
|
||||
|
||||
### 4.1 HTTP client
|
||||
|
||||
* Single **http2** client with connection pooling, DNS pinning, retry/backoff (idempotent GET/POST marked safe).
|
||||
* **DPoP nonce** handling: on `401` with nonce challenge, CLI replays once.
|
||||
|
||||
### 4.2 Streaming
|
||||
|
||||
* `scan` and `report` support **server‑sent JSON lines** (progress events).
|
||||
* `--json` prints machine events; human mode shows compact spinners and crucial updates only.
|
||||
|
||||
### 4.3 Exit codes (CI‑safe)
|
||||
|
||||
| Code | Meaning |
|
||||
| ---- | ------------------------------------------- |
|
||||
| 0 | Success |
|
||||
| 2 | Policy fail (final report verdict=fail) |
|
||||
| 3 | Verification failed (attestation/signature) |
|
||||
| 4 | Auth error (invalid/missing token/DPoP) |
|
||||
| 5 | Resource not found (image/SBOM) |
|
||||
| 6 | Rate limited / quota exceeded |
|
||||
| 7 | Backend unavailable (retryable) |
|
||||
| 9 | Invalid arguments |
|
||||
|
||||
---
|
||||
|
||||
## 5) Configuration model
|
||||
|
||||
**Precedence:** CLI flags → env vars → config file → defaults.
|
||||
|
||||
**Config file**: `${XDG_CONFIG_HOME}/stellaops/config.yaml` (Windows: `%APPDATA%\StellaOps\config.yaml`)
|
||||
|
||||
```yaml
|
||||
cli:
|
||||
authority: "https://authority.internal"
|
||||
backend:
|
||||
scanner: "https://scanner-web.internal"
|
||||
attestor: "https://attestor.internal"
|
||||
feedser: "https://feedser-web.internal"
|
||||
vexer: "https://vexer-web.internal"
|
||||
auth:
|
||||
audienceDefault: "scanner"
|
||||
deviceCode: true
|
||||
output:
|
||||
json: false
|
||||
color: auto
|
||||
tls:
|
||||
caBundle: "/etc/ssl/certs/ca-bundle.crt"
|
||||
offline:
|
||||
kitMirror: "s3://mirror/stellaops-kit"
|
||||
```
|
||||
|
||||
Environment variables: `STELLAOPS_AUTHORITY`, `STELLAOPS_SCANNER_URL`, etc.
|
||||
|
||||
---
|
||||
|
||||
## 6) Buildx generator orchestration
|
||||
|
||||
* `buildx install` locates the Docker root directory, writes the **generator** plugin manifest, and pulls `stellaops/sbom-indexer` image (pinned digest).
|
||||
* `buildx build` wrapper injects:
|
||||
|
||||
* `--attest=type=sbom,generator=stellaops/sbom-indexer`
|
||||
* `--label org.stellaops.request=sbom`
|
||||
* Post‑build: CLI optionally calls **Scanner.WebService** to **verify referrers**, **compose** image SBOMs, and **attest** via Signer/Attestor.
|
||||
|
||||
**Detection**: If Buildx or generator unavailable, CLI falls back to **post‑build scan** with a warning.
|
||||
|
||||
---
|
||||
|
||||
## 7) Artifact handling
|
||||
|
||||
* **Downloads** (`export sbom`, `report final`): stream to file; compute sha256 on the fly; write sidecar `.sha256` and optional **verification bundle** (if `--bundle`).
|
||||
* **Uploads** (`offline kit import`): chunked upload; retry on transient errors; show progress bar (unless `--json`).
|
||||
|
||||
---
|
||||
|
||||
## 8) Security posture
|
||||
|
||||
* **DPoP private keys** stored in **OS keychain**; metadata cached in config.
|
||||
* **No plaintext tokens** on disk; short‑lived **OpToks** held in memory.
|
||||
* **TLS**: verify backend certificates; allow custom CA bundle for on‑prem.
|
||||
* **Redaction**: CLI logs remove `Authorization`, DPoP headers, PoE tokens.
|
||||
* **Supply chain**: CLI distribution binaries are **cosign‑signed**; `stellaops version --verify` checks its own signature.
|
||||
|
||||
---
|
||||
|
||||
## 9) Observability
|
||||
|
||||
* `--verbose` adds request IDs, timings, and retry traces.
|
||||
* **Metrics** (optional, disabled by default): Prometheus text file exporter for local monitoring in long‑running agents.
|
||||
* **Structured logs** (`--json`): per‑event JSON lines with `ts`, `verb`, `status`, `latencyMs`.
|
||||
|
||||
---
|
||||
|
||||
## 10) Performance targets
|
||||
|
||||
* Startup ≤ **20 ms** (AOT).
|
||||
* `scan image` request/response overhead ≤ **5 ms** (excluding server work).
|
||||
* Buildx wrapper overhead negligible (<1 ms).
|
||||
* Large artifact download (100 MB) sustained ≥ **80 MB/s** on local networks.
|
||||
|
||||
---
|
||||
|
||||
## 11) Tests & golden outputs
|
||||
|
||||
* **Unit tests**: argument parsing, config precedence, URL resolution, DPoP proof creation.
|
||||
* **Integration tests** (Testcontainers): mock Authority/Scanner/Attestor; CI pipeline with fake registry.
|
||||
* **Golden outputs**: verb snapshots for `--json` across OSes; kept in `tests/golden/…`.
|
||||
* **Contract tests**: ensure API shapes match service OpenAPI; fail build if incompatible.
|
||||
|
||||
---
|
||||
|
||||
## 12) Error envelopes (human + JSON)
|
||||
|
||||
**Human:**
|
||||
|
||||
```
|
||||
✖ Policy FAIL: 3 high, 1 critical (VEX suppressed 12)
|
||||
- pkg:rpm/openssl (CVE-2025-12345) — affected (vendor) — fixed in 3.0.14
|
||||
- pkg:npm/lodash (GHSA-xxxx) — affected — no fix
|
||||
See: https://ui.internal/scans/sha256:...
|
||||
Exit code: 2
|
||||
```
|
||||
|
||||
**JSON (`--json`):**
|
||||
|
||||
```json
|
||||
{ "event":"report", "status":"fail", "critical":1, "high":3, "url":"https://ui..." }
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 13) Admin & advanced flags
|
||||
|
||||
* `--authority`, `--scanner`, `--attestor`, `--feedser`, `--vexer` override config URLs.
|
||||
* `--no-color`, `--quiet`, `--json`.
|
||||
* `--timeout`, `--retries`, `--retry-backoff-ms`.
|
||||
* `--ca-bundle`, `--insecure` (dev only; prints warning).
|
||||
* `--trace` (dump HTTP traces to file; scrubbed).
|
||||
|
||||
---
|
||||
|
||||
## 14) Interop with other tools
|
||||
|
||||
* Emits **CycloneDX Protobuf** directly to stdout when `export sbom --format cdx-pb --out -`.
|
||||
* Pipes to `jq`/`yq` cleanly in JSON mode.
|
||||
* Can act as a **credential helper** for scripts: `stellaops auth token --aud scanner` prints a one‑shot token for curl.
|
||||
|
||||
---
|
||||
|
||||
## 15) Packaging & distribution
|
||||
|
||||
* **Installers**: deb/rpm (postinst registers completions), Homebrew, Scoop, Winget, MSI/MSIX.
|
||||
* **Shell completions**: bash/zsh/fish/pwsh.
|
||||
* **Update channel**: `stellaops self-update` (optional) fetches cosign‑signed release manifest; corporate environments can disable.
|
||||
|
||||
---
|
||||
|
||||
## 16) Security hard lines
|
||||
|
||||
* Refuse to print token values; redact Authorization headers in verbose output.
|
||||
* Disallow `--insecure` unless `STELLAOPS_CLI_ALLOW_INSECURE=1` set (double opt‑in).
|
||||
* Enforce **short token TTL**; refresh proactively when <30 s left.
|
||||
* Device‑code cache binding to **machine** and **user** (protect against copy to other machines).
|
||||
|
||||
---
|
||||
|
||||
## 17) Wire sequences
|
||||
|
||||
**A) Scan & wait with attestation**
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
autonumber
|
||||
participant CLI
|
||||
participant Auth as Authority
|
||||
participant SW as Scanner.WebService
|
||||
participant SG as Signer
|
||||
participant AT as Attestor
|
||||
|
||||
CLI->>Auth: device code flow (DPoP)
|
||||
Auth-->>CLI: OpTok (aud=scanner)
|
||||
|
||||
CLI->>SW: POST /scans { imageRef, attest:true }
|
||||
SW-->>CLI: { scanId }
|
||||
CLI->>SW: GET /scans/{id} (poll)
|
||||
SW-->>CLI: { status: completed, artifacts, rekor? } # if attested
|
||||
|
||||
alt attestation pending
|
||||
SW->>SG: POST /sign/dsse (server-side)
|
||||
SG-->>SW: DSSE
|
||||
SW->>AT: POST /rekor/entries
|
||||
AT-->>SW: { uuid, proof }
|
||||
end
|
||||
|
||||
CLI->>SW: GET /sboms/<digest>?format=cdx-pb&view=usage
|
||||
SW-->>CLI: bytes
|
||||
```
|
||||
|
||||
**B) Verify attestation by artifact**
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
autonumber
|
||||
participant CLI
|
||||
participant AT as Attestor
|
||||
|
||||
CLI->>AT: POST /rekor/verify { artifactSha256 }
|
||||
AT-->>CLI: { ok:true, uuid, index, logURL }
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 18) Roadmap (CLI)
|
||||
|
||||
* `scan fs <path>` (local filesystem tree) → upload to backend for analysis.
|
||||
* `policy test --sbom <file>` (simulate policy results offline using local policy bundle).
|
||||
* `runtime capture` (developer mode) — capture small `/proc/<pid>/maps` for troubleshooting.
|
||||
* Pluggable output renderers for SARIF/HTML (admin‑controlled).
|
||||
|
||||
---
|
||||
|
||||
## 19) Example CI snippets
|
||||
|
||||
**GitHub Actions (post‑build)**
|
||||
|
||||
```yaml
|
||||
- name: Login (device code w/ OIDC broker)
|
||||
run: stellaops auth login --json --authority ${{ secrets.AUTHORITY_URL }}
|
||||
|
||||
- name: Scan
|
||||
run: stellaops scan image ${{ steps.build.outputs.digest }} --wait --json
|
||||
|
||||
- name: Export (usage view, protobuf)
|
||||
run: stellaops export sbom ${{ steps.build.outputs.digest }} --view usage --format cdx-pb --out sbom.pb
|
||||
|
||||
- name: Verify attestation
|
||||
run: stellaops verify attestation --artifact $(sha256sum sbom.pb | cut -d' ' -f1) --json
|
||||
```
|
||||
|
||||
**GitLab (buildx generator)**
|
||||
|
||||
```yaml
|
||||
script:
|
||||
- stellaops buildx install
|
||||
- docker buildx build --attest=type=sbom,generator=stellaops/sbom-indexer -t $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA .
|
||||
- stellaops scan image $CI_REGISTRY_IMAGE@$IMAGE_DIGEST --wait --json
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 20) Test matrix (OS/arch)
|
||||
|
||||
* Linux: ubuntu‑20.04/22.04/24.04 (x64, arm64), alpine (musl).
|
||||
* macOS: 13–15 (x64, arm64).
|
||||
* Windows: 10/11, Server 2019/2022 (x64, arm64).
|
||||
* Docker engines: Docker Desktop, containerd‑based runners.
|
||||
|
||||
Reference in New Issue
Block a user