Initial commit (history squashed)

2025-10-07 10:14:21 +03:00
commit b97fc7685a
1132 changed files with 117842 additions and 0 deletions
--- a/docs/09_API_CLI_REFERENCE.md
+++ b/docs/09_API_CLI_REFERENCE.md
@@ -0,0 +1,461 @@
+# API & CLI Reference
+
+*Purpose* – give operators and integrators a single, authoritative spec for REST/GRPC calls **and** first‑party CLI tools (`stella-cli`, `zastava`, `stella`).  
+Everything here is *source‑of‑truth* for generated Swagger/OpenAPI and the `--help` screens in the CLIs.
+
+---
+
+## 0 Quick Glance
+
+| Area               | Call / Flag                                 | Notes                                                                          |
+| ------------------ | ------------------------------------------- | ------------------------------------------------------------------------------ |
+| Scan entry         | `POST /scan`                                | Accepts SBOM or image; sub‑5 s target                                          |
+| Delta check        | `POST /layers/missing`                      | <20 ms reply; powers *delta SBOM* feature                                      |
+| Rate‑limit / quota | —                                           | Headers **`X‑Stella‑Quota‑Remaining`**, **`X‑Stella‑Reset`** on every response |
+| Policy I/O         | `GET /policy/export`, `POST /policy/import` | YAML now; Rego coming                                                          |
+| Policy lint        | `POST /policy/validate`                     | Returns 200 OK if ruleset passes                                               |
+| Auth               | `POST /connect/token` (OpenIddict)          | Client‑credentials preferred                                                   |
+| Health             | `GET /healthz`                              | Simple liveness probe                                                          |
+| Attestation *      | `POST /attest` (TODO Q1‑2026)               | SLSA provenance + Rekor log                                                    |
+| CLI flags          | `--sbom-type` `--delta` `--policy-file`     | Added to `stella`                                                             |
+
+\* Marked **TODO** → delivered after sixth month (kept on Feature Matrix “To Do” list).
+
+---
+
+## 1 Authentication
+
+Stella Ops uses **OAuth 2.0 / OIDC** (token endpoint mounted via OpenIddict).
+
+```
+POST /connect/token
+Content‑Type: application/x-www-form-urlencoded
+
+grant_type=client_credentials&
+client_id=ci‑bot&
+client_secret=REDACTED&
+scope=stella.api
+```
+
+Successful response:
+
+```json
+{
+  "access_token": "eyJraWQi...",
+  "token_type": "Bearer",
+  "expires_in": 3600
+}
+```
+
+> **Tip** – pass the token via `Authorization: Bearer <token>` on every call.
+
+---
+
+## 2 REST API
+
+### 2.0 Obtain / Refresh Offline‑Token
+
+```text
+POST /token/offline
+Authorization: Bearer <admin‑token>
+```
+
+| Body field | Required | Example | Notes |
+|------------|----------|---------|-------|
+| `expiresDays` | no | `30` | Max 90 days |
+
+```json
+{
+  "jwt": "eyJhbGciOiJSUzI1NiIsInR5cCI6...",
+  "expires": "2025‑08‑17T00:00:00Z"
+}
+```
+
+Token is signed with the backend’s private key and already contains
+`"maxScansPerDay": {{ quota_token }}`.
+
+
+### 2.1 Scan – Upload SBOM **or** Image
+
+```
+POST /scan
+```
+
+| Param / Header       | In     | Required | Description                                                           |
+| -------------------- | ------ | -------- | --------------------------------------------------------------------- |
+| `X‑Stella‑Sbom‑Type` | header | no       | `trivy-json-v2`, `spdx-json`, `cyclonedx-json`; omitted ➞ auto‑detect |
+| `?threshold`         | query  | no       | `low`, `medium`, `high`, `critical`; default **critical**             |
+| body                 | body   | yes      | *Either* SBOM JSON *or* Docker image tarball/upload URL               |
+
+Every successful `/scan` response now includes:
+
+| Header | Example |
+|--------|---------|
+| `X‑Stella‑Quota‑Remaining` | `129` |
+| `X‑Stella‑Reset` | `2025‑07‑18T23:59:59Z` |
+| `X‑Stella‑Token‑Expires` | `2025‑08‑17T00:00:00Z` |
+
+**Response 200** (scan completed):
+
+```json
+{
+  "digest": "sha256:…",
+  "summary": {
+    "Critical": 0,
+    "High": 3,
+    "Medium": 12,
+    "Low": 41
+  },
+  "policyStatus": "pass",
+  "quota": {
+    "remaining": 131,
+    "reset": "2025-07-18T00:00:00Z"
+  }
+}
+```
+
+**Response 202** – queued; polling URL in `Location` header.
+
+---
+
+### 2.2 Delta SBOM – Layer Cache Check
+
+```
+POST /layers/missing
+Content‑Type: application/json
+Authorization: Bearer <token>
+```
+
+```json
+{
+  "layers": [
+    "sha256:d38b...",
+    "sha256:af45..."
+  ]
+}
+```
+
+**Response 200** — <20 ms target:
+
+```json
+{
+  "missing": [
+    "sha256:af45..."
+  ]
+}
+```
+
+Client then generates SBOM **only** for the `missing` layers and re‑posts `/scan`.
+
+---
+
+### 2.3 Policy Endpoints
+
+| Method | Path               | Purpose                              |
+| ------ | ------------------ | ------------------------------------ |
+| `GET`  | `/policy/export`   | Download live YAML ruleset           |
+| `POST` | `/policy/import`   | Upload YAML or Rego; replaces active |
+| `POST` | `/policy/validate` | Lint only; returns 400 on error      |
+| `GET`  | `/policy/history`  | Paginated change log (audit trail)   |
+
+```yaml
+# Example import payload (YAML)
+version: "1.0"
+rules:
+  - name: Ignore Low dev
+    severity: [Low, None]
+    environments: [dev, staging]
+    action: ignore
+```
+
+Validation errors come back as:
+
+```json
+{
+  "errors": [
+    {
+      "path": "$.rules[0].severity",
+      "msg": "Invalid level 'None'"
+    }
+  ]
+}
+```
+
+---
+
+### 2.4 Attestation (Planned – Q1‑2026)
+
+```
+POST /attest
+```
+
+| Param       | Purpose                               |
+| ----------- | ------------------------------------- |
+| body (JSON) | SLSA v1.0 provenance doc              |
+|             | Signed + stored in local Rekor mirror |
+
+Returns `202 Accepted` and `Location: /attest/{id}` for async verify.
+
+---
+
+## 3 StellaOps CLI (`stellaops-cli`)
+
+The new CLI is built on **System.CommandLine 2.0.0‑beta5** and mirrors the Feedser backend REST API.  
+Configuration follows the same precedence chain everywhere:
+
+1. Environment variables (e.g. `API_KEY`, `STELLAOPS_BACKEND_URL`, `StellaOps:ApiKey`)  
+2. `appsettings.json` → `appsettings.local.json`  
+3. `appsettings.yaml` → `appsettings.local.yaml`  
+4. Defaults (`ApiKey = ""`, `BackendUrl = ""`, cache folders under the current working directory)
+
+**Authority auth client resilience settings**
+
+| Setting | Environment variable | Default | Purpose |
+|---------|----------------------|---------|---------|
+| `StellaOps:Authority:Resilience:EnableRetries` | `STELLAOPS_AUTHORITY_ENABLE_RETRIES` | `true` | Toggle Polly wait-and-retry handlers for discovery/token calls |
+| `StellaOps:Authority:Resilience:RetryDelays` | `STELLAOPS_AUTHORITY_RETRY_DELAYS` | `1s,2s,5s` | Comma/space-separated backoff sequence (HH:MM:SS) |
+| `StellaOps:Authority:Resilience:AllowOfflineCacheFallback` | `STELLAOPS_AUTHORITY_ALLOW_OFFLINE_CACHE_FALLBACK` | `true` | Reuse cached discovery/JWKS metadata when Authority is temporarily unreachable |
+| `StellaOps:Authority:Resilience:OfflineCacheTolerance` | `STELLAOPS_AUTHORITY_OFFLINE_CACHE_TOLERANCE` | `00:10:00` | Additional tolerance window added to the discovery/JWKS cache lifetime |
+
+See `docs/dev/32_AUTH_CLIENT_GUIDE.md` for recommended profiles (online vs. air-gapped) and testing guidance.
+
+| Command | Purpose | Key Flags / Arguments | Notes |
+|---------|---------|-----------------------|-------|
+| `stellaops-cli scanner download` | Fetch and install scanner container | `--channel <stable\|beta\|nightly>` (default `stable`)<br>`--output <path>`<br>`--overwrite`<br>`--no-install` | Saves artefact under `ScannerCacheDirectory`, verifies digest/signature, and executes `docker load` unless `--no-install` is supplied. |
+| `stellaops-cli scan run` | Execute scanner container against a directory (auto-upload) | `--target <directory>` (required)<br>`--runner <docker\|dotnet\|self>` (default from config)<br>`--entry <image-or-entrypoint>`<br>`[scanner-args...]` | Runs the scanner, writes results into `ResultsDirectory`, emits a structured `scan-run-*.json` metadata file, and automatically uploads the artefact when the exit code is `0`. |
+| `stellaops-cli scan upload` | Re-upload existing scan artefact | `--file <path>` | Useful for retries when automatic upload fails or when operating offline. |
+| `stellaops-cli db fetch` | Trigger connector jobs | `--source <id>` (e.g. `redhat`, `osv`)<br>`--stage <fetch\|parse\|map>` (default `fetch`)<br>`--mode <resume|init|cursor>` | Translates to `POST /jobs/source:{source}:{stage}` with `trigger=cli` |
+| `stellaops-cli db merge` | Run canonical merge reconcile | — | Calls `POST /jobs/merge:reconcile`; exit code `0` on acceptance, `1` on failures/conflicts |
+| `stellaops-cli db export` | Kick JSON / Trivy exports | `--format <json\|trivy-db>` (default `json`)<br>`--delta`<br>`--publish-full/--publish-delta`<br>`--bundle-full/--bundle-delta` | Sets `{ delta = true }` parameter when requested and can override ORAS/bundle toggles per run |
+| `stellaops-cli auth <login\|logout\|status\|whoami>` | Manage cached tokens for StellaOps Authority | `auth login --force` (ignore cache)<br>`auth status`<br>`auth whoami` | Uses `StellaOps.Auth.Client`; honours `StellaOps:Authority:*` configuration, stores tokens under `~/.stellaops/tokens` by default, and `whoami` prints subject/scope/expiry |
+
+When running on an interactive terminal without explicit override flags, the CLI uses Spectre.Console prompts to let you choose per-run ORAS/offline bundle behaviour.
+| `stellaops-cli config show` | Display resolved configuration | — | Masks secret values; helpful for air‑gapped installs |
+
+**Logging & exit codes**
+
+- Structured logging via `Microsoft.Extensions.Logging` with single-line console output (timestamps in UTC).  
+- `--verbose / -v` raises log level to `Debug`.  
+- Command exit codes bubble up: backend conflict → `1`, cancelled via `CTRL+C` → `130`, scanner exit codes propagate as-is.
+
+**Artifact validation**
+
+- Downloads are verified against the `X-StellaOps-Digest` header (SHA-256). When `StellaOps:ScannerSignaturePublicKeyPath` points to a PEM-encoded RSA key, the optional `X-StellaOps-Signature` header is validated as well.
+- Metadata for each bundle is written alongside the artefact (`*.metadata.json`) with digest, signature, source URL, and timestamps.
+- Retry behaviour is controlled via `StellaOps:ScannerDownloadAttempts` (default **3** with exponential backoff).
+- Successful `scan run` executions create timestamped JSON artefacts inside `ResultsDirectory` plus a `scan-run-*.json` metadata envelope documenting the runner, arguments, timing, and stdout/stderr. The artefact is posted back to Feedser automatically.
+
+#### Trivy DB export metadata (`metadata.json`)
+
+`stellaops-cli db export --format trivy-db` (and the backing `POST /jobs/export:trivy-db`) always emits a `metadata.json` document in the OCI layout root. Operators consuming the bundle or delta updates should inspect the following fields:
+
+| Field | Type | Purpose |
+| ----- | ---- | ------- |
+| `mode` | `full` \| `delta` | Indicates whether the current run rebuilt the entire database (`full`) or only the changed files (`delta`). |
+| `baseExportId` | string? | Export ID of the last full baseline that the delta builds upon. Only present for `mode = delta`. |
+| `baseManifestDigest` | string? | SHA-256 digest of the manifest belonging to the baseline OCI layout. |
+| `resetBaseline` | boolean | `true` when the exporter rotated the baseline (e.g., repo change, delta chain reset). Treat as a full refresh. |
+| `treeDigest` | string | Canonical SHA-256 digest of the JSON tree used to build the database. |
+| `treeBytes` | number | Total bytes across exported JSON files. |
+| `advisoryCount` | number | Count of advisories included in the export. |
+| `exporterVersion` | string | Version stamp of `StellaOps.Feedser.Exporter.TrivyDb`. |
+| `builder` | object? | Raw metadata emitted by `trivy-db build` (version, update cadence, etc.). |
+| `delta.changedFiles[]` | array | Present when `mode = delta`. Each entry lists `{ "path": "<relative json>", "length": <bytes>, "digest": "sha256:..." }`. |
+| `delta.removedPaths[]` | array | Paths that existed in the previous manifest but were removed in the new run. |
+
+When the planner opts for a delta run, the exporter copies unmodified blobs from the baseline layout identified by `baseManifestDigest`. Consumers that cache OCI blobs only need to fetch the `changedFiles` and the new manifest/metadata unless `resetBaseline` is true.
+When pushing to ORAS, set `feedser:exporters:trivyDb:oras:publishFull` / `publishDelta` to control whether full or delta runs are copied to the registry. Offline bundles follow the analogous `includeFull` / `includeDelta` switches under `offlineBundle`.
+
+Example configuration (`appsettings.yaml`):
+
+```yaml
+feedser:
+  exporters:
+    trivyDb:
+      oras:
+        enabled: true
+        publishFull: true
+        publishDelta: false
+      offlineBundle:
+        enabled: true
+        includeFull: true
+        includeDelta: false
+```
+
+
+**Authentication**
+
+- API key is sent as `Authorization: Bearer <token>` automatically when configured.  
+- Anonymous operation is permitted only when Feedser runs with
+  `authority.allowAnonymousFallback: true`. This flag is temporary—plan to disable
+  it before **2025-12-31 UTC** so bearer tokens become mandatory.
+
+Authority-backed auth workflow:
+1. Configure Authority settings via config or env vars (see sample below). Minimum fields: `Url`, `ClientId`, and either `ClientSecret` (client credentials) or `Username`/`Password` (password grant).  
+2. Run `stellaops-cli auth login` to acquire and cache a token. Use `--force` if you need to ignore an existing cache entry.  
+3. Execute CLI commands as normal—the backend client injects the cached bearer token automatically and retries on transient 401/403 responses with operator guidance.  
+4. Inspect the cache with `stellaops-cli auth status` (shows expiry, scope, mode) or clear it via `stellaops-cli auth logout`.  
+5. Run `stellaops-cli auth whoami` to dump token subject, audience, issuer, scopes, and remaining lifetime (verbose mode prints additional claims).
+6. Expect Feedser to emit audit logs for each `/jobs*` request showing `subject`,
+   `clientId`, `scopes`, `status`, and whether network bypass rules were applied.
+
+Tokens live in `~/.stellaops/tokens` unless `StellaOps:Authority:TokenCacheDirectory` overrides it. Cached tokens are reused offline until they expire; the CLI surfaces clear errors if refresh fails.
+
+**Configuration file template**
+
+```jsonc
+{
+  "StellaOps": {
+    "ApiKey": "your-api-token",
+    "BackendUrl": "https://feedser.example.org",
+    "ScannerCacheDirectory": "scanners",
+    "ResultsDirectory": "results",
+    "DefaultRunner": "docker",
+    "ScannerSignaturePublicKeyPath": "",
+    "ScannerDownloadAttempts": 3,
+    "Authority": {
+      "Url": "https://authority.example.org",
+      "ClientId": "feedser-cli",
+      "ClientSecret": "REDACTED",
+      "Username": "",
+      "Password": "",
+      "Scope": "feedser.jobs.trigger",
+      "TokenCacheDirectory": ""
+    }
+  }
+}
+```
+
+Drop `appsettings.local.json` or `.yaml` beside the binary to override per environment.
+
+---
+
+### 2.5 Misc Endpoints
+
+| Path       | Method | Description                  |
+| ---------- | ------ | ---------------------------- |
+| `/healthz` | GET    | Liveness; returns `"ok"`     |
+| `/metrics` | GET    | Prometheus exposition (OTel) |
+| `/version` | GET    | Git SHA + build date         |
+
+---
+
+## 3 First‑Party CLI Tools
+
+### 3.1 `stella`
+
+> *Package SBOM + Scan + Exit code* – designed for CI.
+
+```
+Usage: stella [OPTIONS] IMAGE_OR_SBOM
+```
+
+| Flag / Option   | Default                 | Description                                        |
+| --------------- | ----------------------- | -------------------------------------------------- |
+| `--server`      | `http://localhost:8080` | API root                                           |
+| `--token`       | *env `STELLA_TOKEN`*    | Bearer token                                       |
+| `--sbom-type`   | *auto*                  | Force `trivy-json-v2`/`spdx-json`/`cyclonedx-json` |
+| `--delta`       | `false`                 | Enable delta layer optimisation                    |
+| `--policy-file` | *none*                  | Override server rules with local YAML/Rego         |
+| `--threshold`   | `critical`              | Fail build if ≥ level found                        |
+| `--output-json` | *none*                  | Write raw scan result to file                      |
+| `--wait-quota`  | `true`                  | If 429 received, automatically wait `Retry‑After` and retry once. |
+
+**Exit codes**
+
+| Code | Meaning                                     |
+| ---- | ------------------------------------------- |
+| 0    | Scan OK, policy passed                      |
+| 1    | Vulnerabilities ≥ threshold OR policy block |
+| 2    | Internal error (network etc.)               |
+
+---
+
+### 3.2 `stella‑zastava`
+
+> *Daemon / K8s DaemonSet* – watch container runtime, push SBOMs.
+
+Core flags (excerpt):
+
+| Flag             | Purpose                            |
+| ---------------- | ---------------------------------- |
+| `--mode`         | `listen` (default) / `enforce`     |
+| `--filter-image` | Regex; ignore infra/busybox images |
+| `--threads`      | Worker pool size                   |
+
+---
+
+### 3.3 `stellopsctl`
+
+> *Admin utility* – policy snapshots, feed status, user CRUD.
+
+Examples:
+
+```
+stellopsctl policy export > policies/backup-2025-07-14.yaml
+stellopsctl feed refresh  # force OSV merge
+stellopsctl user add dev-team --role developer
+```
+
+---
+
+## 4 Error Model
+
+Uniform problem‑details object (RFC 7807):
+
+```json
+{
+  "type": "https://stella-ops.org/probs/validation",
+  "title": "Invalid request",
+  "status": 400,
+  "detail": "Layer digest malformed",
+  "traceId": "00-7c39..."
+}
+```
+
+---
+
+## 5 Rate Limits
+
+Default **40 requests / second / token**.  
+429 responses include `Retry-After` seconds header.
+
+---
+
+## 6 FAQ & Tips
+
+* **Skip SBOM generation in CI** – supply a *pre‑built* SBOM and add `?sbom-only=true` to `/scan` for <1 s path.  
+* **Air‑gapped?** – point `--server` to `http://oukgw:8080` inside the Offline Update Kit.  
+* **YAML vs Rego** – YAML simpler; Rego unlocks time‑based logic (see samples).  
+* **Cosign verify plug‑ins** – enable `SCANNER_VERIFY_SIG=true` env to refuse unsigned plug‑ins.
+
+---
+
+## 7 Planned Changes (Beyond 6 Months)
+
+These stay in *Feature Matrix → To Do* until design is frozen.
+
+| Epic / Feature               | API Impact Sketch                  |
+| ---------------------------- | ---------------------------------- |
+| **SLSA L1‑L3** attestation   | `/attest` (see §2.4)               |
+| Rekor transparency log       | `/rekor/log/{id}` (GET)            |
+| Plug‑in Marketplace metadata | `/plugins/market` (catalog)        |
+| Horizontal scaling controls  | `POST /cluster/node` (add/remove)  |
+| Windows agent support        | Update LSAPI to PDE, no API change |
+
+---
+
+## 8 References
+
+* OpenAPI YAML → `/openapi/v1.yaml` (served by backend)  
+* OAuth2 spec: <https://datatracker.ietf.org/doc/html/rfc6749>  
+* SLSA spec: <https://slsa.dev/spec/v1.0>  
+
+---
+
+## 9 Changelog (truncated)
+
+* **2025‑07‑14** – added *delta SBOM*, policy import/export, CLI `--sbom-type`.  
+* **2025‑07‑12** – initial public reference.
+
+---