stella-ops.org/git.stella-ops.org
1
0
Fork 0
Code Issues Pull Requests Actions 1 Releases Wiki Activity

feat(docs): Add comprehensive documentation for Vexer, Vulnerability Explorer, and Zastava modules

Browse Source 
- Introduced AGENTS.md, README.md, TASKS.md, and implementation_plan.md for Vexer, detailing mission, responsibilities, key components, and operational notes.
- Established similar documentation structure for Vulnerability Explorer and Zastava modules, including their respective workflows, integrations, and observability notes.
- Created risk scoring profiles documentation outlining the core workflow, factor model, governance, and deliverables.
- Ensured all modules adhere to the Aggregation-Only Contract and maintain determinism and provenance in outputs.
This commit is contained in:
master
2025-10-30 00:09:39 +02:00
parent 3154c67978
commit 7b5bdcf4d3
503 changed files with 16136 additions and 54638 deletions
Download Patch File Download Diff File Expand all files Collapse all files

408
docs/modules/cli/architecture.md Normal file
View File

@@ -0,0 +1,408 @@
# component_architecture_cli.md — **Stella Ops CLI** (2025Q4)
> Consolidates requirements captured in the Policy Engine, Policy Studio, Vulnerability Explorer, Export Center, and Notifications implementation plans and module guides.
> **Scope.** Implementation‑ready architecture for **Stella Ops CLI**: command surface, process model, auth (Authority/DPoP), integration with Scanner/Excititor/Concelier/Signer/Attestor, Buildx plug‑in management, offline kit behavior, packaging, observability, security posture, and CI ergonomics.
---
## 0) Mission & boundaries
**Mission.** Provide a **fast, deterministic, CI‑friendly** command‑line interface to drive Stella Ops workflows:
* Build‑time SBOM generation via **Buildx generator** orchestration.
* Post‑build **scan/compose/diff/export** against **Scanner.WebService**.
* **Policy** operations and **VEX/Vuln** data pulls (operator tasks).
* **Verification** (attestation, referrers, signatures) for audits.
* Air‑gapped/offline **kit** administration.
**Boundaries.**
* CLI **never** signs; it only calls **Signer**/**Attestor** via backend APIs when needed (e.g., `report --attest`).
* CLI **does not** store long‑lived credentials beyond OS keychain; tokens are **short** (Authority OpToks).
* Heavy work (scanning, merging, policy) is executed **server‑side** (Scanner/Excititor/Concelier).
---
## 1) Solution layout & runtime form
```
src/
├─ StellaOps.Cli/ # net10.0 (Native AOT) single binary
├─ StellaOps.Cli.Core/ # verb plumbing, config, HTTP, auth
├─ StellaOps.Cli.Plugins/ # optional verbs packaged as plugins
├─ StellaOps.Cli.Tests/ # unit + golden-output tests
└─ packaging/
├─ msix / msi / deb / rpm / brew formula
└─ scoop manifest / winget manifest
```
**Language/runtime**: .NET 10 **Native AOT** for speed/startup; Linux builds use **musl** static when possible.
**Plug-in verbs.** Non-core verbs (Excititor, runtime helpers, future integrations) ship as restart-time plug-ins under `plugins/cli/**` with manifest descriptors. The launcher loads plug-ins on startup; hot reloading is intentionally unsupported. The inaugural bundle, `StellaOps.Cli.Plugins.NonCore`, packages the Excititor, runtime, and offline-kit command groups and publishes its manifest at `plugins/cli/StellaOps.Cli.Plugins.NonCore/`.
**OS targets**: linux‑x64/arm64, windows‑x64/arm64, macOS‑x64/arm64.
---
## 2) Command surface (verbs)
> All verbs default to **JSON** output when `--json` is set (CI mode). Human output is concise, deterministic.
### 2.1 Auth & profile
* `auth login`
* Modes: **device‑code** (default), **client‑credentials** (service principal).
* Produces **Authority** access token (OpTok) + stores **DPoP** keypair in OS keychain.
* `auth status` — show current issuer, subject, audiences, expiry.
* `auth logout` — wipe cached tokens/keys.
### 2.2 Build‑time SBOM (Buildx)
* `buildx install` — install/update the **StellaOps.Scanner.Sbomer.BuildXPlugin** on the host.
* `buildx verify` — ensure generator is usable.
* `buildx build` — thin wrapper around `docker buildx build --attest=type=sbom,generator=stellaops/sbom-indexer` with convenience flags:
* `--attest` (request Signer/Attestor via backend post‑push)
* `--provenance` pass‑through (optional)
### 2.3 Scanning & artifacts
* `scan image <ref|digest>`
* Options: `--force`, `--wait`, `--view=inventory|usage|both`, `--format=cdx-json|cdx-pb|spdx-json`, `--attest` (ask backend to sign/log).
* Streams progress; exits early unless `--wait`.
* `diff image --old <digest> --new <digest> [--view ...]` — show layer‑attributed changes.
* `export sbom <digest> [--view ... --format ... --out file]` — download artifact.
* `report final <digest> [--policy-revision ... --attest]` — request PASS/FAIL report from backend (policy+vex) and optional attestation.
### 2.4 Policy & data
* `policy get/set/apply` — fetch active policy, apply staged policy, compute digest.
* `concelier export` — trigger/export canonical JSON or Trivy DB (admin).
* `excititor export` — trigger/export consensus/raw claims (admin).
### 2.5 Verification
* `verify attestation --uuid <rekor-uuid> | --artifact <sha256> | --bundle <path>` — call **Attestor /verify** and print proof summary.
* `verify referrers <digest>` — ask **Signer /verify/referrers** (is image Stella‑signed?).
* `verify image-signature <ref|digest>` — standalone cosign verification (optional, local).
### 2.6 Runtime (Zastava helper)
* `runtime policy test --image/-i <digest> [--file <path> --ns <name> --label key=value --json]` — ask backend `/policy/runtime` like the webhook would (accepts multiple `--image`, comma/space lists, or stdin pipelines).
### 2.7 Offline kit
* `offline kit pull` — fetch latest **Concelier JSON + Trivy DB + Excititor exports** as a tarball from a mirror.
* `offline kit import <tar>` — upload the kit to on‑prem services (Concelier/Excititor).
* `offline kit status` — list current seed versions.
### 2.8 Utilities
* `config set/get` — endpoint & defaults.
* `whoami` — short auth display.
* `version` — CLI + protocol versions; release channel.
### 2.9 Aggregation-only guard helpers
* `sources ingest --dry-run --source <id> --input <path|uri> [--tenant ... --format table|json --output file]`
* Normalises documents (handles gzip/base64), posts them to the backend `aoc/ingest/dry-run` route, and exits non-zero when guard violations are detected.
* Defaults to table output with ANSI colour; `--json`/`--output` produce deterministic JSON for CI pipelines.
* `aoc verify [--since <ISO8601|duration>] [--limit <count>] [--sources list] [--codes list] [--format table|json] [--export file] [--tenant id] [--no-color]`
* Replays guard checks against stored raw documents. Maps backend `ERR_AOC_00x` codes onto deterministic exit codes so CI can block regressions.
* Supports pagination hints (`--limit`, `--since`), tenant scoping via `--tenant` or `STELLA_TENANT`, and JSON exports for evidence lockers.
---
## 3) AuthN: Authority + DPoP
### 3.1 Token acquisition
* **Device‑code**: the CLI opens an OIDC device code flow against **Authority**; the browser login is optional for service principals.
* **Client‑credentials**: service principals use **private_key_jwt** or **mTLS** to get tokens.
### 3.2 DPoP key management
* On first login, the CLI generates an **ephemeral JWK** (Ed25519) and stores it in the **OS keychain** (Keychain/DPAPI/KWallet/Gnome Keyring).
* Every request to backend services includes a **DPoP proof**; CLI refreshes tokens as needed.
### 3.3 Multi‑audience & scopes
* CLI requests **audiences** as needed per verb:
* `scanner` for scan/export/report/diff
* `signer` (indirect; usually backend calls Signer)
* `attestor` for verify
* `concelier`/`excititor` for admin verbs
CLI rejects verbs if required scopes are missing.
---
## 4) Process model & reliability
### 4.1 HTTP client
* Single **http2** client with connection pooling, DNS pinning, retry/backoff (idempotent GET/POST marked safe).
* **DPoP nonce** handling: on `401` with nonce challenge, CLI replays once.
### 4.2 Streaming
* `scan` and `report` support **server‑sent JSON lines** (progress events).
* `--json` prints machine events; human mode shows compact spinners and crucial updates only.
### 4.3 Exit codes (CI‑safe)
| Code | Meaning |
| ---- | ------------------------------------------- |
| 0 | Success |
| 2 | Policy fail (final report verdict=fail) |
| 3 | Verification failed (attestation/signature) |
| 4 | Auth error (invalid/missing token/DPoP) |
| 5 | Resource not found (image/SBOM) |
| 6 | Rate limited / quota exceeded |
| 7 | Backend unavailable (retryable) |
| 9 | Invalid arguments |
| 11–17 | Aggregation-only guard violation (`ERR_AOC_00x`) |
| 18 | Verification truncated (increase `--limit`) |
| 70 | Transport/authentication failure |
| 71 | CLI usage error (missing tenant, invalid cursor) |
---
## 5) Configuration model
**Precedence:** CLI flags → env vars → config file → defaults.
**Config file**: `${XDG_CONFIG_HOME}/stellaops/config.yaml` (Windows: `%APPDATA%\StellaOps\config.yaml`)
```yaml
cli:
authority: "https://authority.internal"
backend:
scanner: "https://scanner-web.internal"
attestor: "https://attestor.internal"
concelier: "https://concelier-web.internal"
excititor: "https://excititor-web.internal"
auth:
audienceDefault: "scanner"
deviceCode: true
output:
json: false
color: auto
tls:
caBundle: "/etc/ssl/certs/ca-bundle.crt"
offline:
kitMirror: "s3://mirror/stellaops-kit"
```
Environment variables: `STELLAOPS_AUTHORITY`, `STELLAOPS_SCANNER_URL`, etc.
---
## 6) Buildx generator orchestration
* `buildx install` locates the Docker root directory, writes the **generator** plugin manifest, and pulls `stellaops/sbom-indexer` image (pinned digest).
* `buildx build` wrapper injects:
* `--attest=type=sbom,generator=stellaops/sbom-indexer`
* `--label org.stellaops.request=sbom`
* Post‑build: CLI optionally calls **Scanner.WebService** to **verify referrers**, **compose** image SBOMs, and **attest** via Signer/Attestor.
**Detection**: If Buildx or generator unavailable, CLI falls back to **post‑build scan** with a warning.
---
## 7) Artifact handling
* **Downloads** (`export sbom`, `report final`): stream to file; compute sha256 on the fly; write sidecar `.sha256` and optional **verification bundle** (if `--bundle`).
* **Uploads** (`offline kit import`): chunked upload; retry on transient errors; show progress bar (unless `--json`).
---
## 8) Security posture
* **DPoP private keys** stored in **OS keychain**; metadata cached in config.
* **No plaintext tokens** on disk; short‑lived **OpToks** held in memory.
* **TLS**: verify backend certificates; allow custom CA bundle for on‑prem.
* **Redaction**: CLI logs remove `Authorization`, DPoP headers, PoE tokens.
* **Supply chain**: CLI distribution binaries are **cosign‑signed**; `stellaops version --verify` checks its own signature.
---
## 9) Observability
* `--verbose` adds request IDs, timings, and retry traces.
* **Metrics** (optional, disabled by default): Prometheus text file exporter for local monitoring in long‑running agents.
* **Structured logs** (`--json`): per‑event JSON lines with `ts`, `verb`, `status`, `latencyMs`.
---
## 10) Performance targets
* Startup ≤ **20 ms** (AOT).
* `scan image` request/response overhead ≤ **5 ms** (excluding server work).
* Buildx wrapper overhead negligible (<1 ms).
* Large artifact download (100 MB) sustained ≥ **80 MB/s** on local networks.
---
## 11) Tests & golden outputs
* **Unit tests**: argument parsing, config precedence, URL resolution, DPoP proof creation.
* **Integration tests** (Testcontainers): mock Authority/Scanner/Attestor; CI pipeline with fake registry.
* **Golden outputs**: verb snapshots for `--json` across OSes; kept in `tests/golden/…`.
* **Contract tests**: ensure API shapes match service OpenAPI; fail build if incompatible.
---
## 12) Error envelopes (human + JSON)
**Human:**
```
✖ Policy FAIL: 3 high, 1 critical (VEX suppressed 12)
- pkg:rpm/openssl (CVE-2025-12345) — affected (vendor) — fixed in 3.0.14
- pkg:npm/lodash (GHSA-xxxx) — affected — no fix
See: https://ui.internal/scans/sha256:...
Exit code: 2
```
**JSON (`--json`):**
```json
{ "event":"report", "status":"fail", "critical":1, "high":3, "url":"https://ui..." }
```
---
## 13) Admin & advanced flags
* `--authority`, `--scanner`, `--attestor`, `--concelier`, `--excititor` override config URLs.
* `--no-color`, `--quiet`, `--json`.
* `--timeout`, `--retries`, `--retry-backoff-ms`.
* `--ca-bundle`, `--insecure` (dev only; prints warning).
* `--trace` (dump HTTP traces to file; scrubbed).
---
## 14) Interop with other tools
* Emits **CycloneDX Protobuf** directly to stdout when `export sbom --format cdx-pb --out -`.
* Pipes to `jq`/`yq` cleanly in JSON mode.
* Can act as a **credential helper** for scripts: `stellaops auth token --aud scanner` prints a one‑shot token for curl.
---
## 15) Packaging & distribution
* **Installers**: deb/rpm (postinst registers completions), Homebrew, Scoop, Winget, MSI/MSIX.
* **Shell completions**: bash/zsh/fish/pwsh.
* **Update channel**: `stellaops self-update` (optional) fetches cosign‑signed release manifest; corporate environments can disable.
---
## 16) Security hard lines
* Refuse to print token values; redact Authorization headers in verbose output.
* Disallow `--insecure` unless `STELLAOPS_CLI_ALLOW_INSECURE=1` set (double opt‑in).
* Enforce **short token TTL**; refresh proactively when <30 s left.
* Device‑code cache binding to **machine** and **user** (protect against copy to other machines).
---
## 17) Wire sequences
**A) Scan & wait with attestation**
```mermaid
sequenceDiagram
autonumber
participant CLI
participant Auth as Authority
participant SW as Scanner.WebService
participant SG as Signer
participant AT as Attestor
CLI->>Auth: device code flow (DPoP)
Auth-->>CLI: OpTok (aud=scanner)
CLI->>SW: POST /scans { imageRef, attest:true }
SW-->>CLI: { scanId }
CLI->>SW: GET /scans/{id} (poll)
SW-->>CLI: { status: completed, artifacts, rekor? } # if attested
alt attestation pending
SW->>SG: POST /sign/dsse (server-side)
SG-->>SW: DSSE
SW->>AT: POST /rekor/entries
AT-->>SW: { uuid, proof }
end
CLI->>SW: GET /sboms/<digest>?format=cdx-pb&view=usage
SW-->>CLI: bytes
```
**B) Verify attestation by artifact**
```mermaid
sequenceDiagram
autonumber
participant CLI
participant AT as Attestor
CLI->>AT: POST /rekor/verify { artifactSha256 }
AT-->>CLI: { ok:true, uuid, index, logURL }
```
---
## 18) Roadmap (CLI)
* `scan fs <path>` (local filesystem tree) → upload to backend for analysis.
* `policy test --sbom <file>` (simulate policy results offline using local policy bundle).
* `runtime capture` (developer mode) — capture small `/proc/<pid>/maps` for troubleshooting.
* Pluggable output renderers for SARIF/HTML (admin‑controlled).
---
## 19) Example CI snippets
**GitHub Actions (post‑build)**
```yaml
- name: Login (device code w/ OIDC broker)
run: stellaops auth login --json --authority ${{ secrets.AUTHORITY_URL }}
- name: Scan
run: stellaops scan image ${{ steps.build.outputs.digest }} --wait --json
- name: Export (usage view, protobuf)
run: stellaops export sbom ${{ steps.build.outputs.digest }} --view usage --format cdx-pb --out sbom.pb
- name: Verify attestation
run: stellaops verify attestation --artifact $(sha256sum sbom.pb | cut -d' ' -f1) --json
```
**GitLab (buildx generator)**
```yaml
script:
- stellaops buildx install
- docker buildx build --attest=type=sbom,generator=stellaops/sbom-indexer -t $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA .
- stellaops scan image $CI_REGISTRY_IMAGE@$IMAGE_DIGEST --wait --json
```
---
## 20) Test matrix (OS/arch)
* Linux: ubuntu‑20.04/22.04/24.04 (x64, arm64), alpine (musl).
* macOS: 13–15 (x64, arm64).
* Windows: 10/11, Server 2019/2022 (x64, arm64).
* Docker engines: Docker Desktop, containerd‑based runners.