Files
git.stella-ops.org/docs/09_API_CLI_REFERENCE.md
master 607e72e2a1
Some checks failed
Build Test Deploy / docs (push) Has been cancelled
Build Test Deploy / deploy (push) Has been cancelled
Build Test Deploy / build-test (push) Has been cancelled
Build Test Deploy / authority-container (push) Has been cancelled
Docs CI / lint-and-preview (push) Has been cancelled
up
2025-10-12 20:37:18 +03:00

488 lines
21 KiB
Markdown
Executable File
Raw Blame History

This file contains invisible Unicode characters

This file contains invisible Unicode characters that are indistinguishable to humans but may be processed differently by a computer. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# API & CLI Reference
*Purpose* give operators and integrators a single, authoritative spec for REST/GRPC calls **and** firstparty CLI tools (`stella-cli`, `zastava`, `stella`).
Everything here is *sourceoftruth* for generated Swagger/OpenAPI and the `--help` screens in the CLIs.
---
## 0 Quick Glance
| Area | Call / Flag | Notes |
| ------------------ | ------------------------------------------- | ------------------------------------------------------------------------------ |
| Scan entry | `POST /scan` | Accepts SBOM or image; sub5s target |
| Delta check | `POST /layers/missing` | <20ms reply; powers *delta SBOM* feature |
| Ratelimit / quota | | Headers **`XStellaQuotaRemaining`**, **`XStellaReset`** on every response |
| Policy I/O | `GET /policy/export`, `POST /policy/import` | YAML now; Rego coming |
| Policy lint | `POST /policy/validate` | Returns 200 OK if ruleset passes |
| Auth | `POST /connect/token` (OpenIddict) | Clientcredentials preferred |
| Health | `GET /healthz` | Simple liveness probe |
| Attestation * | `POST /attest` (TODO Q12026) | SLSA provenance + Rekor log |
| CLI flags | `--sbom-type` `--delta` `--policy-file` | Added to `stella` |
\* Marked **TODO** delivered after sixth month (kept on Feature Matrix To Do list).
---
## 1 Authentication
StellaOps uses **OAuth 2.0 / OIDC** (token endpoint mounted via OpenIddict).
```
POST /connect/token
ContentType: application/x-www-form-urlencoded
grant_type=client_credentials&
client_id=cibot&
client_secret=REDACTED&
scope=stella.api
```
Successful response:
```json
{
"access_token": "eyJraWQi...",
"token_type": "Bearer",
"expires_in": 3600
}
```
> **Tip**  pass the token via `Authorization: Bearer <token>` on every call.
---
## 2 REST API
###2.0Obtain / Refresh OfflineToken
```text
POST /token/offline
Authorization: Bearer <admintoken>
```
| Body field | Required | Example | Notes |
|------------|----------|---------|-------|
| `expiresDays` | no | `30` | Max 90 days |
```json
{
"jwt": "eyJhbGciOiJSUzI1NiIsInR5cCI6...",
"expires": "20250817T00:00:00Z"
}
```
Token is signed with the backends private key and already contains
`"maxScansPerDay": {{ quota_token }}`.
### 2.1 Scan  Upload SBOM **or** Image
```
POST /scan
```
| Param / Header | In | Required | Description |
| -------------------- | ------ | -------- | --------------------------------------------------------------------- |
| `XStellaSbomType` | header | no | `trivy-json-v2`, `spdx-json`, `cyclonedx-json`; omitted autodetect |
| `?threshold` | query | no | `low`, `medium`, `high`, `critical`; default **critical** |
| body | body | yes | *Either* SBOM JSON *or* Docker image tarball/upload URL |
Every successful `/scan` response now includes:
| Header | Example |
|--------|---------|
| `XStellaQuotaRemaining` | `129` |
| `XStellaReset` | `20250718T23:59:59Z` |
| `XStellaTokenExpires` | `20250817T00:00:00Z` |
**Response 200** (scan completed):
```json
{
"digest": "sha256:…",
"summary": {
"Critical": 0,
"High": 3,
"Medium": 12,
"Low": 41
},
"policyStatus": "pass",
"quota": {
"remaining": 131,
"reset": "2025-07-18T00:00:00Z"
}
}
```
**Response 202** queued; polling URL in `Location` header.
---
### 2.2 Delta SBOM Layer Cache Check
```
POST /layers/missing
ContentType: application/json
Authorization: Bearer <token>
```
```json
{
"layers": [
"sha256:d38b...",
"sha256:af45..."
]
}
```
**Response 200**<20ms target:
```json
{
"missing": [
"sha256:af45..."
]
}
```
Client then generates SBOM **only** for the `missing` layers and reposts `/scan`.
---
### 2.3 Policy Endpoints
| Method | Path | Purpose |
| ------ | ------------------ | ------------------------------------ |
| `GET` | `/policy/export` | Download live YAML ruleset |
| `POST` | `/policy/import` | Upload YAML or Rego; replaces active |
| `POST` | `/policy/validate` | Lint only; returns 400 on error |
| `GET` | `/policy/history` | Paginated change log (audit trail) |
```yaml
# Example import payload (YAML)
version: "1.0"
rules:
- name: Ignore Low dev
severity: [Low, None]
environments: [dev, staging]
action: ignore
```
Validation errors come back as:
```json
{
"errors": [
{
"path": "$.rules[0].severity",
"msg": "Invalid level 'None'"
}
]
}
```
---
### 2.4 Attestation (Planned  Q12026)
```
POST /attest
```
| Param | Purpose |
| ----------- | ------------------------------------- |
| body (JSON) | SLSA v1.0 provenance doc |
| | Signed + stored in local Rekor mirror |
Returns `202 Accepted` and `Location: /attest/{id}` for async verify.
---
## 3 StellaOps CLI (`stellaops-cli`)
The new CLI is built on **System.CommandLine2.0.0beta5** and mirrors the Feedser backend REST API.
Configuration follows the same precedence chain everywhere:
1. Environment variables (e.g. `API_KEY`, `STELLAOPS_BACKEND_URL`, `StellaOps:ApiKey`)
2. `appsettings.json`  `appsettings.local.json`
3. `appsettings.yaml`  `appsettings.local.yaml`
4. Defaults (`ApiKey = ""`, `BackendUrl = ""`, cache folders under the current working directory)
**Authority auth client resilience settings**
| Setting | Environment variable | Default | Purpose |
|---------|----------------------|---------|---------|
| `StellaOps:Authority:Resilience:EnableRetries` | `STELLAOPS_AUTHORITY_ENABLE_RETRIES` | `true` | Toggle Polly wait-and-retry handlers for discovery/token calls |
| `StellaOps:Authority:Resilience:RetryDelays` | `STELLAOPS_AUTHORITY_RETRY_DELAYS` | `1s,2s,5s` | Comma/space-separated backoff sequence (HH:MM:SS) |
| `StellaOps:Authority:Resilience:AllowOfflineCacheFallback` | `STELLAOPS_AUTHORITY_ALLOW_OFFLINE_CACHE_FALLBACK` | `true` | Reuse cached discovery/JWKS metadata when Authority is temporarily unreachable |
| `StellaOps:Authority:Resilience:OfflineCacheTolerance` | `STELLAOPS_AUTHORITY_OFFLINE_CACHE_TOLERANCE` | `00:10:00` | Additional tolerance window added to the discovery/JWKS cache lifetime |
See `docs/dev/32_AUTH_CLIENT_GUIDE.md` for recommended profiles (online vs. air-gapped) and testing guidance.
| Command | Purpose | Key Flags / Arguments | Notes |
|---------|---------|-----------------------|-------|
| `stellaops-cli scanner download` | Fetch and install scanner container | `--channel <stable\|beta\|nightly>` (default `stable`)<br>`--output <path>`<br>`--overwrite`<br>`--no-install` | Saves artefact under `ScannerCacheDirectory`, verifies digest/signature, and executes `docker load` unless `--no-install` is supplied. |
| `stellaops-cli scan run` | Execute scanner container against a directory (auto-upload) | `--target <directory>` (required)<br>`--runner <docker\|dotnet\|self>` (default from config)<br>`--entry <image-or-entrypoint>`<br>`[scanner-args...]` | Runs the scanner, writes results into `ResultsDirectory`, emits a structured `scan-run-*.json` metadata file, and automatically uploads the artefact when the exit code is `0`. |
| `stellaops-cli scan upload` | Re-upload existing scan artefact | `--file <path>` | Useful for retries when automatic upload fails or when operating offline. |
| `stellaops-cli db fetch` | Trigger connector jobs | `--source <id>` (e.g. `redhat`, `osv`)<br>`--stage <fetch\|parse\|map>` (default `fetch`)<br>`--mode <resume|init|cursor>` | Translates to `POST /jobs/source:{source}:{stage}` with `trigger=cli` |
| `stellaops-cli db merge` | Run canonical merge reconcile | — | Calls `POST /jobs/merge:reconcile`; exit code `0` on acceptance, `1` on failures/conflicts |
| `stellaops-cli db export` | Kick JSON / Trivy exports | `--format <json\|trivy-db>` (default `json`)<br>`--delta`<br>`--publish-full/--publish-delta`<br>`--bundle-full/--bundle-delta` | Sets `{ delta = true }` parameter when requested and can override ORAS/bundle toggles per run |
| `stellaops-cli auth <login\|logout\|status\|whoami>` | Manage cached tokens for StellaOps Authority | `auth login --force` (ignore cache)<br>`auth status`<br>`auth whoami` | Uses `StellaOps.Auth.Client`; honours `StellaOps:Authority:*` configuration, stores tokens under `~/.stellaops/tokens` by default, and `whoami` prints subject/scope/expiry |
| `stellaops-cli auth revoke export` | Export the Authority revocation bundle | `--output <directory>` (defaults to CWD) | Writes `revocation-bundle.json`, `.json.jws`, and `.json.sha256`; verifies the digest locally and includes key metadata in the log summary. |
| `stellaops-cli auth revoke verify` | Validate a revocation bundle offline | `--bundle <path>` `--signature <path>` `--key <path>`<br>`--verbose` | Verifies detached JWS signatures, reports the computed SHA-256, and can fall back to cached JWKS when `--key` is omitted. |
| `stellaops-cli config show` | Display resolved configuration | — | Masks secret values; helpful for airgapped installs |
When running on an interactive terminal without explicit override flags, the CLI uses Spectre.Console prompts to let you choose per-run ORAS/offline bundle behaviour.
**Logging & exit codes**
- Structured logging via `Microsoft.Extensions.Logging` with single-line console output (timestamps in UTC).
- `--verbose / -v` raises log level to `Debug`.
- Command exit codes bubble up: backend conflict → `1`, cancelled via `CTRL+C``130`, scanner exit codes propagate as-is.
**Artifact validation**
- Downloads are verified against the `X-StellaOps-Digest` header (SHA-256). When `StellaOps:ScannerSignaturePublicKeyPath` points to a PEM-encoded RSA key, the optional `X-StellaOps-Signature` header is validated as well.
- Metadata for each bundle is written alongside the artefact (`*.metadata.json`) with digest, signature, source URL, and timestamps.
- Retry behaviour is controlled via `StellaOps:ScannerDownloadAttempts` (default **3** with exponential backoff).
- Successful `scan run` executions create timestamped JSON artefacts inside `ResultsDirectory` plus a `scan-run-*.json` metadata envelope documenting the runner, arguments, timing, and stdout/stderr. The artefact is posted back to Feedser automatically.
#### Trivy DB export metadata (`metadata.json`)
`stellaops-cli db export --format trivy-db` (and the backing `POST /jobs/export:trivy-db`) always emits a `metadata.json` document in the OCI layout root. Operators consuming the bundle or delta updates should inspect the following fields:
| Field | Type | Purpose |
| ----- | ---- | ------- |
| `mode` | `full` \| `delta` | Indicates whether the current run rebuilt the entire database (`full`) or only the changed files (`delta`). |
| `baseExportId` | string? | Export ID of the last full baseline that the delta builds upon. Only present for `mode = delta`. |
| `baseManifestDigest` | string? | SHA-256 digest of the manifest belonging to the baseline OCI layout. |
| `resetBaseline` | boolean | `true` when the exporter rotated the baseline (e.g., repo change, delta chain reset). Treat as a full refresh. |
| `treeDigest` | string | Canonical SHA-256 digest of the JSON tree used to build the database. |
| `treeBytes` | number | Total bytes across exported JSON files. |
| `advisoryCount` | number | Count of advisories included in the export. |
| `exporterVersion` | string | Version stamp of `StellaOps.Feedser.Exporter.TrivyDb`. |
| `builder` | object? | Raw metadata emitted by `trivy-db build` (version, update cadence, etc.). |
| `delta.changedFiles[]` | array | Present when `mode = delta`. Each entry lists `{ "path": "<relative json>", "length": <bytes>, "digest": "sha256:..." }`. |
| `delta.removedPaths[]` | array | Paths that existed in the previous manifest but were removed in the new run. |
When the planner opts for a delta run, the exporter copies unmodified blobs from the baseline layout identified by `baseManifestDigest`. Consumers that cache OCI blobs only need to fetch the `changedFiles` and the new manifest/metadata unless `resetBaseline` is true.
When pushing to ORAS, set `feedser:exporters:trivyDb:oras:publishFull` / `publishDelta` to control whether full or delta runs are copied to the registry. Offline bundles follow the analogous `includeFull` / `includeDelta` switches under `offlineBundle`.
Example configuration (`appsettings.yaml`):
```yaml
feedser:
exporters:
trivyDb:
oras:
enabled: true
publishFull: true
publishDelta: false
offlineBundle:
enabled: true
includeFull: true
includeDelta: false
```
**Authentication**
- API key is sent as `Authorization: Bearer <token>` automatically when configured.
- Anonymous operation is permitted only when Feedser runs with
`authority.allowAnonymousFallback: true`. This flag is temporary—plan to disable
it before **2025-12-31 UTC** so bearer tokens become mandatory.
Authority-backed auth workflow:
1. Configure Authority settings via config or env vars (see sample below). Minimum fields: `Url`, `ClientId`, and either `ClientSecret` (client credentials) or `Username`/`Password` (password grant).
2. Run `stellaops-cli auth login` to acquire and cache a token. Use `--force` if you need to ignore an existing cache entry.
3. Execute CLI commands as normal—the backend client injects the cached bearer token automatically and retries on transient 401/403 responses with operator guidance.
4. Inspect the cache with `stellaops-cli auth status` (shows expiry, scope, mode) or clear it via `stellaops-cli auth logout`.
5. Run `stellaops-cli auth whoami` to dump token subject, audience, issuer, scopes, and remaining lifetime (verbose mode prints additional claims).
6. Expect Feedser to emit audit logs for each `/jobs*` request showing `subject`,
`clientId`, `scopes`, `status`, and whether network bypass rules were applied.
Tokens live in `~/.stellaops/tokens` unless `StellaOps:Authority:TokenCacheDirectory` overrides it. Cached tokens are reused offline until they expire; the CLI surfaces clear errors if refresh fails.
**Configuration file template**
```jsonc
{
"StellaOps": {
"ApiKey": "your-api-token",
"BackendUrl": "https://feedser.example.org",
"ScannerCacheDirectory": "scanners",
"ResultsDirectory": "results",
"DefaultRunner": "docker",
"ScannerSignaturePublicKeyPath": "",
"ScannerDownloadAttempts": 3,
"Authority": {
"Url": "https://authority.example.org",
"ClientId": "feedser-cli",
"ClientSecret": "REDACTED",
"Username": "",
"Password": "",
"Scope": "feedser.jobs.trigger",
"TokenCacheDirectory": ""
}
}
}
```
Drop `appsettings.local.json` or `.yaml` beside the binary to override per environment.
---
### 2.5 Misc Endpoints
| Path | Method | Description |
| ---------- | ------ | ---------------------------- |
| `/healthz` | GET | Liveness; returns `"ok"` |
| `/metrics` | GET | Prometheus exposition (OTel) |
| `/version` | GET | Git SHA + build date |
---
### 2.6 Authority Admin APIs
Administrative endpoints live under `/internal/*` on the Authority host and require the bootstrap API key (`x-stellaops-bootstrap-key`). Responses are deterministic and audited via `AuthEventRecord`.
| Path | Method | Description |
| ---- | ------ | ----------- |
| `/internal/revocations/export` | GET | Returns the revocation bundle (JSON + detached JWS + digest). Mirrors the output of `stellaops-cli auth revoke export`. |
| `/internal/signing/rotate` | POST | Promotes a new signing key and marks the previous key as retired without restarting the service. |
**Rotate request body**
```json
{
"keyId": "authority-signing-2025",
"location": "../certificates/authority-signing-2025.pem",
"source": "file",
"provider": "default"
}
```
The API responds with the active `kid`, previous key (if any), and the set of retired key identifiers. Always export a fresh revocation bundle after rotation so downstream mirrors receive signatures from the new key.
---
## 3 FirstParty CLI Tools
### 3.1 `stella`
> *Package SBOM + Scan + Exit code* designed for CI.
```
Usage: stella [OPTIONS] IMAGE_OR_SBOM
```
| Flag / Option | Default | Description |
| --------------- | ----------------------- | -------------------------------------------------- |
| `--server` | `http://localhost:8080` | API root |
| `--token` | *env `STELLA_TOKEN`* | Bearer token |
| `--sbom-type` | *auto* | Force `trivy-json-v2`/`spdx-json`/`cyclonedx-json` |
| `--delta` | `false` | Enable delta layer optimisation |
| `--policy-file` | *none* | Override server rules with local YAML/Rego |
| `--threshold` | `critical` | Fail build if ≥ level found |
| `--output-json` | *none* | Write raw scan result to file |
| `--wait-quota` | `true` | If 429 received, automatically wait `RetryAfter` and retry once. |
**Exit codes**
| Code | Meaning |
| ---- | ------------------------------------------- |
| 0 | Scan OK, policy passed |
| 1 | Vulnerabilities ≥ threshold OR policy block |
| 2 | Internal error (network etc.) |
---
### 3.2 `stellazastava`
> *Daemon / K8s DaemonSet* watch container runtime, push SBOMs.
Core flags (excerpt):
| Flag | Purpose |
| ---------------- | ---------------------------------- |
| `--mode` | `listen` (default) / `enforce` |
| `--filter-image` | Regex; ignore infra/busybox images |
| `--threads` | Worker pool size |
---
### 3.3 `stellopsctl`
> *Admin utility* policy snapshots, feed status, user CRUD.
Examples:
```
stellopsctl policy export > policies/backup-2025-07-14.yaml
stellopsctl feed refresh # force OSV merge
stellopsctl user add dev-team --role developer
```
---
## 4 Error Model
Uniform problemdetails object (RFC 7807):
```json
{
"type": "https://stella-ops.org/probs/validation",
"title": "Invalid request",
"status": 400,
"detail": "Layer digest malformed",
"traceId": "00-7c39..."
}
```
---
## 5 Rate Limits
Default **40 requests / second / token**.
429 responses include `Retry-After` seconds header.
---
## 6 FAQ & Tips
* **Skip SBOM generation in CI** supply a *prebuilt* SBOM and add `?sbom-only=true` to `/scan` for <1s path.
* **Airgapped?** point `--server` to `http://oukgw:8080` inside the Offline Update Kit.
* **YAML vs Rego** YAML simpler; Rego unlocks timebased logic (see samples).
* **Cosign verify plugins** enable `SCANNER_VERIFY_SIG=true` env to refuse unsigned plugins.
---
## 7 Planned Changes (Beyond 6 Months)
These stay in *Feature Matrix → To Do* until design is frozen.
| Epic / Feature | API Impact Sketch |
| ---------------------------- | ---------------------------------- |
| **SLSA L1L3** attestation | `/attest` (see §2.4) |
| Rekor transparency log | `/rekor/log/{id}` (GET) |
| Plugin Marketplace metadata | `/plugins/market` (catalog) |
| Horizontal scaling controls | `POST /cluster/node` (add/remove) |
| Windows agent support | Update LSAPI to PDE, no API change |
---
## 8 References
* OpenAPI YAML `/openapi/v1.yaml` (served by backend)
* OAuth2 spec: <https://datatracker.ietf.org/doc/html/rfc6749>
* SLSA spec: <https://slsa.dev/spec/v1.0>
---
## 9 Changelog (truncated)
* **20250714** added *delta SBOM*, policy import/export, CLI `--sbom-type`.
* **20250712** initial public reference.
---